Research Engineer - World Modeling
Sunnyvale, CA
Engineering /
Full-time /
On-site
About the Institute of Foundation Models
We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.
As part of our team, you’ll have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries. Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub for high-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AI pioneers.
The Role
We are the AllWorld Team under the Institute of Foundation Model (IFM) at MBZUAI. At AllWorld, we are pioneering the development of the PAN (Physical, Agentic, and Networked) world models—the next-generation foundation models to unlock machine intelligence beyond lingual.
Our mission is to tackle the fundamental challenges of world modeling and establish a new paradigm for next-generation machine reasoning. We are looking for passionate individuals who share our vision and are eager to push the boundaries of AI together.
Key Responsibilities: Data Infrastructure & Pipelines
- Design, implement, and maintain scalable video data pipelines to support large-scale training.
- Develop data preprocessing, transformation, and synthesis workflows to support world model training.
- Contribute to building high-quality data annotation pipelines to ensure accurate and consistent labels across large-scale datasets.
Key Responsibilities: Training & Inference Systems
- Support the training of multimodal foundation models (e.g., video diffusion models, world models) by developing and optimizing distributed training systems.
- Improve inference and serving efficiency for real-time interaction through model optimization and system tuning.
- Monitor system health and performance, and contribute to debugging and optimization at scale.
Key Responsibilities: Collaboration & Integration
- Work closely with research teams to understand experimental goals and translate ideas into reliable and maintainable infrastructure and tools.
- Integrate novel research prototypes into production-ready systems and ensure reproducibility at scale.
- Participate in design and code reviews, ensuring code quality, efficiency, and compliance with best practices.
Key Responsibilities: Benchmarking & Evaluation
- Contribute to the development of tools and infrastructure to evaluate model performance using rigorous quantitative benchmarks, including metrics for physical accuracy and controllability.
Key Responsibilities: Codebase & Documentation
- Maintain and extend shared codebases, contribute to internal documentation, and support onboarding of new team members or collaborators.
- Write clean, efficient, and well-tested code for components across the model development lifecycle.
Key Responsibilities
- Support contributions to research papers and demos when engineering work plays a significant role.
- Help represent the team’s engineering excellence in internal and external forums when appropriate.
Academic Qualifications
- MSc or PhD in Machine Learning or Computer Science, or equivalent industry experience.
Professional Experience Required
- Proficient in data collection, cleaning, and transformation at scale, including designing robust pipelines for multimodal datasets (e.g., video, audio, text).
- Practical experience with web scraping and crawling frameworks (e.g., scrapy, selenium, playwright, BeautifulSoup) to collect and curate high-quality web-scale datasets.
- Experience in large-scale model training (LLMs or Diffusion Models) on large clusters.
- Hands-on experience with state-of-the-art video generative models (e.g., Sora, Veo2, MovieGen, CogVideoX, etc.).
- Experiences in building and optimizing large-scale video data pipelines.
- Experience in accelerating diffusion model inference for improved efficiency.
- Exceptional problem-solving and troubleshooting skills to tackle complex technical challenges.
- Strong systems and engineering expertise in deep learning frameworks such as PyTorch.
- Strong communication and collaboration skills for effective cross-functional teamwork.
- Demonstrated ability to solve complex system-level challenges and debug failures across the training/inference stack (e.g., memory issues, deadlocks, I/O bottlenecks).
$100,000 - $650,000 a year
Salary Range & Description
The starting base pay for this position is as shown above. The actual base pay is dependent upon a variety of job-related factors such as professional background, training, work experience, location, business needs and market demand. Therefore, in some circumstances, the actual salary could fall outside of this expected range. This pay range is subject to change and may be modified in the future.
Visa Sponsorship
This position is eligible for visa sponsorship.
Benefits Include
*Comprehensive medical, dental, and vision benefits
*Bonus
*401K Plan
*Generous paid time off, sick leave and holidays
*Paid Parental Leave
*Employee Assistance Program
*Life insurance and disability
