INFP

What is INFP?

INFP is an advanced system designed to transform static portrait images into interactive, talking head videos that naturally alternate between speaking and listening states in multi-turn conversations. Unlike traditional methods that require manual role assignment, INFP uses audio input to dynamically guide the agent's facial expressions and head motions, capturing verbal and non-verbal cues with high fidelity. It leverages a novel two-stage process involving motion latent space encoding and conditional diffusion transformers, supported by a large-scale DyConv dataset of real-life dyadic conversations. The framework achieves real-time performance and preserves individual facial details and speaking styles, making it suitable for applications requiring realistic virtual avatars and interactive agents.

Key Features

• Dynamic Role Switching

  Automatically alternates the animated portrait between speaking and listening states based on dyadic audio input without manual intervention.

• Two-Stage Motion Generation

  Combines motion-based head imitation with audio-guided motion mapping to produce natural and synchronized facial and head movements.

• Person-Generic and Real-Time

  Supports any individual’s static image and generates animations in real time, enabling broad applicability.

• High-Fidelity Facial Detail Preservation

  Maintains individual facial features and speaking styles through dual-attention mechanisms and style modulation.

• Large-Scale DyConv Dataset

  Trained on an extensive collection of authentic dyadic conversations, enhancing the system’s realism and interaction quality.

Use Cases

• Virtual Communication Agents : Create responsive avatars for customer service, virtual assistants, and social robots that engage naturally in conversations.
• Content Creation and Entertainment : Generate lip-synced talking head videos for storytelling, dubbing, and interactive media.
• Remote Education and Training : Develop interactive tutors or presenters that visually respond to audio input for enhanced learner engagement.
• Social Media and Marketing : Produce personalized video messages and promotional content with realistic animated portraits.

FAQs