A Modular Robotic System Purpose-Built for Sandwich Assembly
How It Works
01
Perception
Our perception stack combines YOLO and the Segment Anything Model to identify ingredients in real time, converting detections into robot coordinates through eye-in-hand and eye-to-hand calibration — validated to sub-centimeter accuracy in controlled conditions.
02
Manipulation
Because food is soft, slippery, and irregular, we built custom end-effectors for specific ingredients: a vacuum-based tool for cheese, a glove-style gripper for lettuce, and a motorized needle-poking mechanism for tomatoes — refined through rapid 3D-printed prototyping.
03
Execution
The system detects ingredients, plans a path, selects the right tool, and assembles the sandwich step by step with force-controlled precision.
Phase 1 vs. Phase 2
Phase 1
Full technical foundation validated: motion planning, camera integration, perception, calibration, and basic pick-and-place behavior across multiple Subway-style sandwich configurations.
Phase 2
Robust real-world deployment: precision calibration refinement, expanded AI training datasets, refined gripper mechanisms, full end-to-end assembly automation, a production-grade prototype, and a live QSR pilot deployment.
Achievements & Open Challenges
ACHIEVEMENTS
- Robotics & motion planning architecture complete
- RGB-D perception & camera calibration validated
- Ingredient perception pipeline operational
- Custom end-effector prototypes built & tested
- Proof-of-concept validation completed
ACTIVE CHALLENGES
- Deformable food handling — edge cases remain
- Hardware iteration cost management
- Perception reliability in varied lighting
- Multi-subsystem synchronization complexity
- Sim-to-real transfer for learning-based policies