Role Overview
A research intern position focused on advancing multimodal controllable video generation, integrating diverse control signals (keyboard, controller, trajectory, voice) and developing high-fidelity generation capabilities within a game development R&D environment.
Responsibilities
- Design a multimodal controllable generation framework supporting various inputs such as keyboard, controller, trajectory, and voice.
- Research high-fidelity generation under sparse control signals and explore the extension of conditional control paradigms in the video domain.
- Achieve the decoupling and combination of trajectory control, camera control, motion control, and posture control.
- Design dedicated control modules tailored to different game genres.
- Develop video generation capabilities for multi-player collaborative control, ensuring low-latency response and inter-frame state retention.
- Collaborate with the Engine/Agent teams to define standardized interfaces for controllable signals.
- Drive the join optimization of controllable generation and Reinforcement Learning post-training.
Requirements
- Ph.D. in AI-related fields, focusing on Computer Vision/Deep Learning.
- Proficient in conditional generation models and multi-player/multi-channel control technologies; familiar with the principles of diffusion models.
- Project experience in camera control, trajectory control, or posture control.
- Understanding of game engine input/output logic and frame synchronization mechanisms.
- Proficient in Python/PyTorch, with publications in controllable video generation or conditional generation.
- Experience in Game AI or game development is preferred.
- Experience in real-time/interactive video generation projects is preferred.