Qualcomm Technologies, Inc.

High-Fidelity Retail Simulation for Mobile Manipulation with Qualcomm

Robotec.ai partnered with Qualcomm to build a photorealistic retail simulation for a mobile manipulator performing shelf tasks like restocking and facing. Synthetic data generation and hardware-in-the-loop validation closed the sim2real gap and confirmed real-time performance on Qualcomm edge hardware.

Goals

Build a photorealistic retail simulation for benchmarking perception, control, and learning on a mobile manipulator
Integrate open-source and proprietary simulation components across a ROS 2 stack
Generate high-quality synthetic datasets to close the sim2real gap
Validate real-time, hardware-in-the-loop execution on Qualcomm edge hardware

Case study: High-Fidelity Retail Simulation for Mobile Manipulation with Qualcomm

Partners: Qualcomm Technologies, Inc.

Challenge

Qualcomm engaged us to develop a high-fidelity simulation of a mobile manipulator operating in a photorealistic retail environment: enabling advanced perception, control, and learning for shelf-oriented tasks. The work spanned the full embodied-AI pipeline, from scene and asset construction through sensor modeling, synthetic data generation, and closed-loop task execution, while ensuring real-time performance on target Qualcomm edge hardware.

Photorealistic simulation: Build a dynamic retail aisle with adjustable shelves, planograms, and a diverse set of SKU objects to benchmark navigation, perception, and grasping under configurable scene variants.
Synthetic data & sim2real: Produce richly labeled datasets with segmentation, 2D/3D boxes, and keypoints to fine-tune perception and reduce the sim2real gap.
Embodied task execution: Integrate perception and planning nodes to autonomously execute shelf tasks – restock, facing, cleanup – validated under real-time constraints.

Solution

High-fidelity retail environment: A photorealistic aisle scene with adjustable shelves, planograms, simulated SKUs, configurable variants, and a ROS 2 node for scene reset and domain randomization.
Mobile manipulator stack: An imported AMR with a 7-DOF arm and gripper (URDF), integrated with ROS 2 control and manipulation, achieving stable, reliable grasp physics.
Sensor simulation & synthetic data: Simulated RGB-D camera, LiDAR, and additional sensors with realistic noise models, exposed over ROS topics, generating large labeled datasets across diverse scene variants, built on Robotec platform, synthetic-data Gems, Advanced Camera Gem, and Robotec GPU Sensors (RGS).
Software-in-the-loop & HiL: ROS 2 interfaces coupled to external perception and planning nodes, SLAM integration and semantic mapping, and a hardware-in-the-loop preview running on Qualcomm hardware.

Results

Real-time, closed-loop control validated on target Qualcomm edge hardware.
Reliable manipulation with stable grasp physics and autonomous execution of restock, facing, and cleanup tasks.
Production-grade synthetic datasets across diverse scene variants, with validated sensor realism and measurable sim2real gap reduction.
A deployable, containerized release with full evaluation support for Qualcomm's integration and sim-to-real transferability.