Author: Zach Zweig-Vinegar, Chief Software Architect, Miso
Introduction
In our continuous pursuit of innovation, Miso Robotics has embraced NVIDIA’s state-of-the-art tools to enrich Flippy’s capabilities in the challenging kitchen environment. Building on the success of integration with NVIDIA cuMotion, a CUDA-accelerated motion planning library available through NVIDIA Isaac Manipulator, detailed in our previous post on NVIDIA Isaac. This blog post delves deeper into the technical enhancements and efficiencies achieved through direct integrations with NVIDIA Isaac Sim, a reference application built on NVIDIA Omniverse that enables developers to develop, simulate and test AI-driven robots in physically-based virtual environments.
Seamless CAD to Simulation: Transforming Our Workflow
Historically, the transition from CAD designs in Onshape to our simulation environments involved numerous steps, including manual adjustments, file conversions, and troubleshooting. This cumbersome process not only consumed significant engineering resources but also posed challenges in agility and scalability. However, with the integration of Isaac Sim, we have streamlined this process:
- Direct Export and Import: Today, engineers can export CAD designs from Onshape directly and import them into Isaac Sim. This transition is now reduced to just a few streamlined steps, cutting down the required engineer time and simplifying our workflow.
- Onboarding Efficiency Gains: Our ability to efficiently onboard new customer configurations is bolstered by this capability. The reduction in steps has decreased onboarding time, enabling us to test and deploy multiple configurations more swiftly, enhancing our capacity to meet client demands.
- Problem Detection and Testing in Simulation: Using Isaac Sim, we can identify issues with new hardware designs before manufacturing begins. By thoroughly testing these designs in diverse simulated cooking scenarios, we ensure robust performance and functionality, reducing potential issues post-production.
Optimizing Motion: The Role of Isaac Sim and cuMotion
Once configurations are successfully imported into our simulator, we leverage the platform alongside NVIDIA cuMotion to fine-tune robotic motions.
- Simulation-Driven Planning: Isaac Sim provides a realistic simulation environment, allowing us to simulate kitchen scenarios with high fidelity models imported directly from CAD. Coupled with cuMotion, it optimizes motion trajectories, ensuring Flippy navigates through complex layouts smoothly and efficiently.
- Performance Metrics: After using cuMotion, we have observed significant reductions in motion planning times and improvements in task completion speeds, thereby elevating operational efficiency and reliability significantly. After optimization with cuMotion and eliminating other inefficiencies in the robot’s movements, we saw Flippy’s maximum throughput increase by 15-20%.
By creating digital twins of kitchen environments in Isaac Sim and optimizing our motion planning workflow with cuMotion, we can ensure the reliability and performance of Flippy in simulation before deploying in real-world settings. This was affirmed by Steve Foreman, White Castle’s director of operations, who stated, “[Flippy] takes some of that load off the team, allowing members to step away from the fryer and focus on other tasks.” This operational efficiency is only achievable because Flippy has been put through rigorous simulation testing and is optimized to cook at a high rate to keep up with peak rush times in the restaurant.
Future Optimization and Expansions
While our current process marks significant advancement, the journey doesn’t end here. We are laying the groundwork for further optimization:
- Training with Synthetic Data: With Isaac Sim’s physically accurate and photo-realistic rendering, future iterations will generate synthetic datasets for training and testing perception models. Additionally, we will try FoundationPose from NVIDIA Isaac Manipulator that will help estimate and track the pose of 3D objects. This will enable precise localization of kitchen equipment without needing fiducial markers.
- Scaling with Simulated Environments: Leveraging simulated environments for testing at scale will allow us to validate Flippy’s performance across numerous configurations in parallel, accelerating our development and deployment timelines.
Conclusion
Through the seamless integration of cuMotion and NVIDIA Isaac Sim platform into our development process, Miso Robotics is redefining the paradigm of kitchen automation. These advancements not only enhance Flippy’s adaptability and efficiency but also significantly shorten onboarding times and optimize robotic operations for diverse customer requirements. As we push the boundaries of robotics, we are excited about the possibilities NVIDIA’s cutting-edge technologies enable us to realize in our quest to revolutionize food preparation.
Stay tuned for more technical insights and updates on our journey to harnessing the full potential of AI and robotics in kitchen automation.
