Multimodal AI for Context-Aware Defect Generation in Quality Inspection

Accurate defect detection is crucial to the food and by-product processing industry's inspection workflow. It ensures product quality, safety, and compliance with regulations, thereby reducing waste. However, industry professionals frequently highlight the challenge of obtaining actual defect samples, as they are often scarce in production or subject to strict food safety standards that prevent the deliberate introduction of defects for research purposes. At the same time, historical inspection records provide valuable data, both in images and text, describing past defects.
This project addresses the sparse data challenge in AI-driven defect detection by developing a proof-of-concept multimodal synthetic defect generation system. A combination of Natural Language Processing (NLP) and Computer Vision (CV) techniques. The system will extract product-specific attributes (eg, texture, colour, and defect characteristics) from golden image samples and textual records. These extracted features will condition a generative model to create realistic synthetic defect images that accurately reflect real-world defect variations. By generating a diverse and representative dataset, this approach eliminates reliance on random data augmentation, leading to more robust defect detection models.
The project is structured into three primary technical phases: (1) development of the NLP &CV pipeline for extracting attributes from textual and image data, (2) synthetic data generation conditioned on desired attributes, and (3) evaluation of the synthetic dataset's quality for the defect detection workflow.
The industrial partners, Ekro and QING, who lead the meat processing industry as part of the consortium, will significantly benefit from this research. This research will contribute to improving food/byproduct quality inspection by reducing reliance on scarce defect samples and facilitating the training of more accurate defect detection models. The expected outcome is a methodology and a prototype software system that demonstrates the viability of synthetic defect generation, applicable to inspection applications.