AI Voice-Controlled Robot

The gap between a sentence and a motor

Voice control of a robot is easy to demo and hard to make behave. A spoken instruction is ambiguous, arrives late, and says nothing about the obstacle two feet ahead. Meanwhile the microcontroller driving the motors has none of the context needed to interpret it.

This robot splits the problem cleanly: Gemini turns natural language into a command, and the ESP8266 stays responsible for everything that has to happen in milliseconds.

Two layers, deliberately separated

• A Flutter app captures the command and provides the interface — a recording state, a speaking state, and a visible command queue so you can see what the robot has been asked to do and what it has left.
• Gemini interprets natural-language instructions rather than matching a fixed keyword list.
• The robot answers back in speech, not just motion — it introduces itself as Jarvis, a robot assistant built for RoboCell, the robotics club at NIT Durgapur.
• Commands travel over WebSockets — a persistent bidirectional channel, so control is continuous rather than request-response.
• An ESP8266 drives the motors and reads the sensors.

It does not need to be told everything

Underneath the language layer the robot keeps its own reflexes: line following and obstacle avoidance run on the microcontroller regardless of what is being said to it. High-level intent comes from above; the safety and tracking behaviour stays local, where it can react at sensor speed.

Talking to it

Source

Code on GitHub.