Startup Demos Chatbot That Controls Robot Arm Via Natural Language
• Peter Chen, CEO of Covariant, demos a chatbot that can discuss and manipulate objects using a robot arm. This shows progress towards more flexible, general-purpose robots.
• The chatbot uses Covariant's RFM-1 model, trained on large amounts of text plus robot motion and video data. This connects language to physical action.
• RFM-1 can control robot hardware it wasn't trained on, meaning it could potentially control various types of robots. This could enable quicker adaptation to new tasks.
• Researchers are exploring using simulation and human demonstration data to train more broadly capable robot models, like Google's RT-X dataset.
• An advantage of Covariant's approach is the model better understands real world physics compared to text-only models like Sora.