WorldSynth: World Foundation Models for Photorealistic Synthetic Data Generation in Robotic Machine Tending

In the high-tech electronics manufacturing sector, particularly among SMEs in the Brainport region, there is a growing demand for flexible robotic machine tending: the automated loading, unloading and monitoring of production machines such as CNC machining centres, test rigs and assembly cells. Training the perception and control models that drive these robots requires large amounts of labelled visual data, which is costly and difficult to collect in real production environments.
Simulation-based synthetic data offers a partial solution, but traditional physics-based simulators often produce imagery that differs significantly from real factory scenes, creating a “sim-to-real gap.” Recent advances in World Foundation Models (WFMs), such as NVIDIA Cosmos, introduce a new approach. These generative models are trained on large-scale video data and can produce photorealistic, physically plausible scenes, potentially reducing this gap.
This project investigates whether WFMs can be effectively used to generate synthetic training data for robotic machine tending in SME manufacturing contexts, where constraints such as limited 3D assets, modest GPU resources and strict reliability requirements apply. Fontys University of Applied Sciences, Affix Engineering and Datory will: (1) define representative use cases and data requirements, (2) develop a pipeline for generating photorealistic datasets using domain randomisation, (3) train and benchmark perception models against traditional simulation approaches, and (4) validate results on a physical robot arm.
The project will deliver a proof-of-concept pipeline, benchmarked datasets, a lab demonstrator and practical guidelines for SMEs, while contributing to education and enabling future deployment in real production environments.