GeoPointTransformer - Simultaneous segmentation and geometry extraction from Mobile-Mapping Point Clouds

This project introduces GeoPointTransformer, a new AI model that automates the extraction of geometric features—such as poles, curbs, and rails—from 3D scans collected by mobile mapping vehicles. These scans, created using car-mounted laser sensors, are widely used to map Dutch roads and public infrastructure to create accurate centimeter- and millimeter-accurate “digital twins”. While the technology to recognize objects in these 3D scans has advanced rapidly in recent years, extracting precise shapes and dimensions from this data still relies heavily on manual effort and rule-based software.
GeoPointTransformer is designed to perform both tasks—object recognition and shape fitting—within a single, compact neural network. Unlike existing methods that treat these steps separately, this model learns to understand what an object is and how to describe it geometrically in one unified process. For example, it can identify a streetlight and fit a cylinder to its shape or trace the edge of a curb as a smooth curve. The model is trained using both synthetic scenes, where the ground truth is perfectly known, and real-world mobile mapping data
A key innovation of GeoPointTransformer is its built-in feedback loop: the system doesn't just predict an object’s label or shape, it also evaluates how well the shape fits the original point cloud and uses this fit to refine its own predictions. Despite this additional computational demand, the model remains compact and efficient, making it suitable for real-world deployment on standard hardware.
In this project, we will develop a prototype implementation of GeoPointTransformer and test it on both a large synthetic dataset and a real Dutch use case involving mobile mapping data. The resulting system will reach Technology Readiness Level (TRL) 4, representing a working proof-of-concept validated in a relevant, real-world context.