A video is worth a million words. Here’s Desi Jarvis in action before we get into the details:

What if you could talk to your robots and they’d just listen? Not a scripted voice command system, but an actual AI assistant that understands context, speaks back to you, and controls a fleet of robots anywhere in the world. And yes, it works in Hindi and Hinglish too.

That’s what I built with yakrover-8004-mcp and yakrover-talk-mcp - a Desi Jarvis of sorts - and it’s been one of the most fun things I’ve worked on.

From Cryptobot to MCP

Our earlier Tumbller Cryptobot experiment explored the intersection of crypto and robotics - enabling payments and robot control via Farcaster frames. That was fun, but the control interface was tightly coupled to Farcaster and needed a centralized frame server sitting between the user and the robot.

yakrover-8004-mcp takes a completely different approach. Instead of a centralized server mediating control, each robot registers itself on the ERC-8004 on-chain registry and exposes its capabilities as an MCP server. Any MCP client - Claude Code, Claude Desktop, or anything that speaks the protocol - can discover the robot on-chain and connect to it directly. No central server, no custom UI, no app to install.

It’s also a modular framework with a plugin architecture. Each robot is a plugin with three files (~100 lines of code), and the framework handles everything else - MCP server setup, ngrok tunneling, on-chain registration, fleet discovery.

How It Works

The architecture uses FastAPI with ASGI sub-mounting. One port, one ngrok tunnel, multiple isolated robot servers:

LLM / Voice Agent
    |
Single ngrok tunnel
    |
FastAPI Gateway (port 8000)
    |--- /fleet/mcp       -> Fleet Orchestrator (discovery + lifecycle)
    |--- /tumbller/mcp    -> Tumbller self-balancing robot
    |--- /tello/mcp       -> Tello drone
    |--- /fakerover/mcp   -> Software simulator (no hardware needed)

Each robot gets its own FastMCP server instance, completely isolated. The Tumbller talks HTTP to an ESP32-S3, the Tello speaks UDP via djitellopy, and the FakeRover is a pure software simulator I use for development when I don’t have the physical robots around.

Adding a new robot? Just drop three files in src/robots/myrobot/:

# __init__.py - Metadata + plugin registration
class MyRobotPlugin(RobotPlugin):
    def metadata(self) -> RobotMetadata:
        return RobotMetadata(
            name="My Robot",
            robot_type="differential_drive",
            fleet_provider="yakrover",
            ...
        )

# client.py - Communication protocol (HTTP, UDP, serial, whatever)
# tools.py - MCP tool definitions

No framework changes needed. The plugin auto-discovery picks it up.

I explain the full architecture in these two videos from our YakRover weekly meetings:

On-Chain Discovery with ERC-8004

Here’s where it gets interesting. Every robot in the fleet is registered on Ethereum Sepolia as an ERC-8004 agent. The metadata - robot name, type, MCP endpoint, available tools - gets stored on-chain and on IPFS via Pinata.

This means anyone, anywhere, can discover what robots are available:

# Find all robots in the yakrover fleet
uv run python scripts/discover.py --provider yakrover

# Auto-add discovered robots to Claude Code config
uv run python scripts/discover.py --add-mcp --scope global

The discovery itself is exposed as an MCP tool. So an LLM can call discover_robot_agents() at runtime, find robots on-chain, connect to their MCP endpoints, and start controlling them. No hardcoded URLs, no manual configuration.

The full chain: LLM -> Ethereum Sepolia -> IPFS -> ngrok tunnel -> FastAPI gateway -> Robot hardware

Enter the Desi Jarvis

The robot control side was working well. But I wanted something more natural - I wanted to talk to my robots. Not type commands, but actually speak to them. And I wanted my AI assistant to talk back.

That’s yakrover-talk-mcp - a voice-output MCP server that turns Claude into a speaking assistant. It uses ElevenLabs for text-to-speech and PipeWire for audio routing. PipeWire is the key here - it lets you route audio to any sink on your system. Local speakers, Bluetooth headphones, or a HomePod in the other room via AirPlay/RAOP. I have a helper script that manages audio sinks, so I can switch where Jarvis speaks from without touching any code. Walk into the living room, switch the sink to the HomePod, and now your robot status updates follow you around the house.

The setup uses MCP’s composability beautifully. Claude Code connects to both servers simultaneously:

• yakrover-8004-mcp for robot control (move, takeoff, land, get sensor data)
• yakrover-talk-mcp for voice output (speak, play templates, switch voices)

So when I say “Tello ko udao aur Tumbller ko aage bhejo” (take off the Tello and move the Tumbller forward), Claude understands the intent, calls the right MCP tools on the right robots, and speaks back confirming what it did.

The Language Routing

One design decision I’m happy with is how language routing works. The server has clear rules baked into the tool docstrings (which Claude reads as instructions):

• English responses -> speak() via ElevenLabs TTS
• Hinglish/Hindi responses -> Text only, no TTS (ElevenLabs doesn’t handle Hindi well yet)
• Dramatic moments -> Pre-recorded Bollywood templates via play_template()

Yes, I have pre-recorded Gabbar Singh and Sholay audio clips as response templates. When something dramatic happens - like a drone flip or a robot going offline - Claude can play the appropriate filmy response instead of synthesizing it. Because what’s the point of a Desi Jarvis without a little Bollywood drama?

Controlling Robots from Anywhere

The combination of ngrok tunneling, on-chain discovery, and MCP gives you true remote control from anywhere in the world. Here’s what a typical session looks like:

1. I start the gateway on my machine in Finland where the robots are physically connected
2. The ngrok tunnel makes the MCP endpoints publicly accessible
3. From my iPad at a coffee shop in Helsinki, I connect to Claude Code via Termius + Tailscale
4. Claude discovers the robots on-chain and connects to their MCP servers
5. I talk, Claude listens, robots move, Claude talks back

The FakeRover simulator is useful here too. When I’m away from the physical robots, I can still develop and test the full voice -> LLM -> MCP -> robot pipeline with simulated hardware. The simulator mimics the Tumbller’s HTTP endpoints exactly, including auto-stop behavior and drifting sensor readings.

What’s Next

The plugin system doesn’t care what’s on the other end of the wire. A “robot” in the framework is really just anything with an API - any cyber-physical or IoT system works. A smart thermostat, a CNC machine, a greenhouse controller, a factory floor sensor array. If it exposes an interface, you can wrap it in a plugin and bring it into the fleet. Register it on-chain, and now anyone with an MCP client can discover and interact with it.

I’m currently working on adding a 3D printer as the next plugin. The idea is the same - expose the printer’s capabilities (start print, monitor progress, adjust temperatures) as MCP tools so Claude can manage print jobs conversationally. “Print the gear file I uploaded yesterday at 0.2mm layer height” is a lot nicer than navigating OctoPrint.

Beyond that, there’s more I want to do:

• Speech-to-text MCP server - Right now I use Wispr Flow or Voquill for voice input. I want a proper STT MCP server so the entire voice pipeline lives within MCP
• Better Hindi/Hinglish TTS - Waiting for TTS models that handle code-mixed Indian languages well
• Video streaming - Adding camera feeds as MCP resources so the LLM can see what the robots see
• Multi-user sessions - Collaborative robot control sounds fun

The whole point of this project is to make interacting with physical systems as natural as talking to a friend. No special apps, no custom UIs, just conversation. MCP makes this possible by giving LLMs a standardized way to discover and interact with the physical world.

Try It Yourself

Both repos are on GitHub: yakrover-8004-mcp and yakrover-talk-mcp.

If you want to try it, you can start with just the FakeRover simulator - no hardware needed. Start the simulator and the MCP gateway:

# Terminal 1 — Start the simulator
PYTHONPATH=src uv run python -m robots.fakerover.simulator

# Terminal 2 — Start the MCP gateway
PYTHONPATH=src uv run python scripts/serve.py --robots fakerover

Then add the MCP server to Claude Code with a single command:

claude mcp add --transport http fakerover http://localhost:8000/fakerover/mcp

That’s it. Now you can ask Claude to move the rover, read its temperature, or check if it’s online. When you’re ready for real hardware, swap fakerover for tumbller or tello and point it at your robot.

For robots running remotely (say, in another country), the discovery script can find them on-chain and add them to your Claude config automatically:

# Discover all yakrover fleet robots and add them to Claude Code
uv run python scripts/discover.py --add-mcp --scope global

This reads the MCP endpoints from Ethereum Sepolia, and writes them into your ~/.claude.json as HTTP MCP servers - ready to use from anywhere.

Want to run Claude Code from your iPhone or iPad? Here’s how I set it up - and yes, it actually works great for quick fixes on the go.

What You Need

• An Ubuntu machine (local, cloud VM, whatever)
• Termius app on your iPhone/iPad (free version works)
• A Tailscale account (also free)

Step 1: Set Up Your Dev Machine

SSH into your Ubuntu machine and run my setup script:

Friendly reminder: Always read scripts before running them.

TLDR: ./setup-machine.sh && sudo tailscale up → tailscale login → npm install -g @anthropic-ai/claude-code

git clone https://github.com/anuraj-rp/dotfiles.git ~/dotfiles
cd ~/dotfiles
./setup-machine.sh

Don’t want everything? Run individual scripts instead:

./install-tailscale.sh  # Required - for remote access
./install-nodejs.sh     # Required - for Claude Code
./install-tmux.sh       # Recommended - keeps sessions alive

Skip what you don’t need: install-docker.sh, install-vim.sh, install-starship.sh, install-nerdfonts.sh

All scripts support cleanup - e.g. ./install-vim.sh --clean (remove configs) or --clean-all (full uninstall).

The full setup installs:

• tmux - keeps your sessions alive when you disconnect
• vim with plugins (NERDTree, themes, git integration)
• Docker
• Tailscale - the magic that makes this work from anywhere
• Starship prompt - because life’s too short for ugly terminals

Step 2: Connect to Tailscale

sudo tailscale up

Follow the URL it gives you to authenticate. Then grab your Tailscale IP:

tailscale ip

Keep this IP handy - you’ll need it for Termius.

Step 3: Set Up Termius

1. Open Termius → tap “+” → “New Host”
2. Enter your Tailscale IP as hostname
3. Add your username and password (or set up SSH keys)

Connect and you’re in!

Step 4: Using tmux (Essential for Mobile)

Start a named session:

tmux new -s dev

Now here’s the beauty - if your connection drops (happens on mobile), just reconnect and run:

tmux attach -t dev

Everything’s exactly where you left it.

Quick tmux commands (prefix is Ctrl+a):

Step 5: Install Claude Code

npm install -g @anthropic-ai/claude-code

Now just run claude and start coding from your phone!

Step 6: Add Claude HUD (Optional)

Claude HUD adds a statusline showing context usage, active tools, running agents, and todo progress.

Claude HUD demo

— Jarrod Watts (@jarrodwatts) January 2026

Install it from within Claude Code:

/plugin marketplace add jarrodwatts/claude-hud
/plugin install claude-hud
/claude-hud:setup

Step 7: Add Speech-to-Text (Optional)

Typing on a phone keyboard gets old fast. Speech-to-text makes Claude Code on mobile genuinely productive - dictate your prompts instead of pecking at tiny keys.

For iPhone/iPad: Wispr Flow

Wispr Flow is an AI voice keyboard that works inside any app. Install it, enable the keyboard in iOS Settings, then switch to it when you want to dictate in Termius.

• Automatically adds punctuation and removes filler words
• Works in 100+ languages
• Free tier gives 2,000 words/week

To use: tap the globe icon on your keyboard → select Wispr Flow → start talking.

For Desktop (Mac/Windows/Linux): Voquill

If you’re also using Claude Code from your desktop, Voquill is an open-source alternative worth checking out.

• Runs Whisper locally (no cloud required) or via Groq API
• AI cleanup removes filler words automatically
• Personal glossary for technical terms
• AGPLv3 licensed

Both tools turn voice into polished text, which is exactly what you want when dictating code prompts.

Pro Tips

tmux plugins - Press Ctrl+a Shift+i after first launch to install session persistence and vim integration.

Multiple machines? Run tailscale status to see all your devices. Add each to Termius and switch between them easily.

Why This Works So Well

Tailscale creates a secure mesh network between your devices. Termius gives you a proper terminal with a mobile-friendly keyboard row (Ctrl, Esc, arrows). And tmux means you never lose your work when switching apps or losing signal.

I’ve used this setup on trains, buses and coffee shops. It’s surprisingly usable for quick bug fixes and code reviews. Give it a shot!

• https://www.youtube.com/watch?v=_1u7IOnivnM
• PlatformIO blog - https://piolabs.com/blog/insights/debugging-introduction.html
• GDB colourful dashboard with python - https://github.com/cyrus-and/gdb-dashboard
• https://stackoverflow.com/questions/3775445/general-jtag-working-flow-in-an-embedded-development-system
• https://www.actuatedrobots.com/debugging-with-jtag/
• https://www.youtube.com/watch?v=-E5py-0JSXk
• Memfault Blog on GDB Python API - https://interrupt.memfault.com/blog/automate-debugging-with-gdb-python-api

Hardware

Tumbller is a self-balancing robotics educational kit from Elegoo. It runs on an Arduino Nano and does not have WiFi. To add WiFi to the Tumbller we decided to use an Arduino Nano ESP32 because it has the same pinout as Arduino Nano. But we had a small problem - the Tumbller PCB runs on 5V and Nano ESP32 on 3.3V. We designed a custom plugin board to fit on the Tumbller PCB with voltage level translator in between the MCU and PCB board. We got the plugin board manufactured from JLCPCB in China. The plugin board had some issues in bringing back the encoder and sensor signals back to Arduino Nano ESP32. So we added the caster wheel to the robot to allow testing robot without self-balancing functionality.

Here is a 3D render of the plugin board we designed.

The video stream from our YakRover weekly meetings where I explain the entire process of PCB design and manufacturing for Tumbller Plugin.

We also put a camera ESP32 on the robot to make it FPV bot. The camera is basically ESP32-CAM devkit. Each of our tumbller had two ESP32 microcontrollers. Both the ESP32 MCUs connected to the same wifi access point. Let’s call the ESP32 MCU controlling the tumbller robot motion tumbller-esp32-s3 and the camera ESP32 as tumbller-esp-cam. Each of the ESP32 MCU exposed functionalities or affordances of the robot on the local network as a http REST endpoint.

Software

A bird’s eye view of the software stack to control the robots with farcaster frame-v1 and paybot payment gate is shown below.

The tumbller-esp32-s3 exposes the motion and motor control affordance of the cryptobot via five endpoints by running a webserver. The endpoints /move/forward, /move/backward, /move/left, /move/right and /move/stop control the movements for forward, backward, left, right and stop respectively.

The tumbller-esp-cam exposes the camera image via the /getImage endpoint so that when a GET request is sent to the endpoint, the server on tumbller-esp-cam would return a image taken by the camera.

The server for frames-v1 is written in python. The frames-v1 server, tumbller-esp32-s3 and the tumbller-esp-cam need to be on the same network or VPN. We used tailscale to put all three on the same VPN. The ESP32s can be exposed on the tailscale VPN with tailscale subnets. If the frames server is running locally on a computer without DNS name then ngrok can be used to expose to the outside world. The payment gate for the rovers is generated using the awesome Paybot farcaster bot by Rob Recht and Richard Sun. They helped us a lot in setting up the payment gate quickly with their farcaster bot.

The frame-v1 server sits in between the Tumbllers and farcaster translating the frame UI actions to appropriate tumbller-esp32-s3 and tumbller-esp-cam REST calls. I explain and help the setup of the frame server for Venkat’s Tumbller in Seattle here in this video.

The current version of the farcaster frame-v1 server allows selection between two rovers and manages the session and payment gates for each rover independently. Each session runs for 2 minutes after a payment of 1USDC.

Devcon and future work

We did have small hiccups during setup at Devcon but we were able to solve all the problems and have the demo running for the two rovers at Devcon.

First we had some problems with the WiFi setup because of 2.4GHz congestion and Tailscale VPN. Since I had the ESP32s on a subnet via the RaspberryPi, they would become invisible without the RaspberryPi to my frame-v1 server without the Raspberry turned on even though all the devices were on the same network. It took me almost a day to figure this out. As a temporary solution at Devcon I turned off the VPN after which the system worked nicely.

I also had problems figuring out how to trigger a particular user’s wallet with fid information from user interaction with frame and paybot. I finally just used the farcaster-py library to get the fname from fid and then use the fname and paybot together to get the payment gate working properly.

But after these two problems were solved the system mostly worked smoothly.

There is a lot of improvement still needed and a quick list suggested by Venkat in our discord group.

• Ruggedize basics (new board, wheel mechanics fixing, permanent mount for cam, cleaner comms architecture)
• UI bugs on frames
• Fix wallet flow limitations
• Better UX (eg photo per step, display)
• Expand payload (sensors etc)
• Onchain memory, recording images, game aspect etc
• Extend to sunfounder and non crypto UX
• Crypto-economic UX

We had a discussion about the future steps for our cryptobot network on our Yak Robotics Garage Weekly Meeting.

For the next steps, we should be able to get some parts of the system design shown below in the coming months with frames-v2. For a speculative riff on Crypto-economics of the rovers here is Venkat’s post on his substack Contraptions

I have been showing around our social cryptobots to people at Devcon and later here in Finland. It is really satisfying to watch people’s reactions to the social cryptobots. There seems to be something rather unique about them because it resonates with so many people with a wide variety of backgrounds, engineers and non-engineers alike.

We are in the process of making an update to the plugin board so that other people could just buy the Tumbller kit and join the cryptobot party.

Crypto + Robotics = Cryptobotics

I think Cryptobotics broadly captures the vibe and what the field is all about - permissionless-ness, decentralization, distributed systems, blockchains, smart-contracts, zk-proofs, zk-SNARKS, cryptocurrencies and robotics.

The time for Cryptobotics has arrived and as Marc Andreessen would say “It’s time to build” - Cryptobots.

Infact we at Yak Robotics Garage built a small FPV cryptobot, controlled with Farcaster frames. We are bringing it to Devcon in Bangkok.

So if you want to play with our cryptobots, come meet us at Devcon. Follow us on Farcaster/Twitter to know where we have setup our cryptobots. We are vgr and anurajenp on Farcaster/Twitter.

The audio still needs some work. But overall I am happy with the scene layout. The CSS added to the discord streamkit chat overlay below

body { background-color: rgba(0, 0, 0, 0); margin: 0px auto; overflow: hidden;} 
div[class^='Chat_chatContainer__'] { height: 950px !important; width: auto !important; } 
div[class^='Chat_channelName__'] { float: left; width 100%; } 
ul[class^='Chat_messages__'] { height: calc(950px - 52px) !important; width: calc(100% - 16px); } 
li[class^='Chat_message__'] { max-height: none !important; }

The CSS extends the chat window to the entire scene instead of a small rectangular box in previous distributed systems study group videos. Next step is to update the voice channel overlay CSS and enhance the audio somehow.

But I have taken tiny steps, and moving forward really slowly. I could go faster if I was doing this full time. I tried a lot of things in the last one year and spent quite some money on different gears and trying out different software and video platforms. I made a lot of cringe content too, lol.

Some of the things I tried:

Gear

• DJI Pocket 2
  It was a really good buy I think. It shoots in 4K and has a gimbal. The best thing about it is that it fits into my pocket. :D
• Selfie stick with tripod and iPhone
  I have filming a lot with my iPhone too, so a simple tripod and selfie stick was useful
• Gaming Laptop
  Another good buy. I thought I could edit with my old computer but it was just not powerful enough for full 4K video editing. So now I have a ASUS Zephyrus G14 Gaming Laptop
• Tripod
  A good camera tripod. Haven’t used it much except the monopod stand from one of its legs which I used for the DJI Pocket 2

Software

• Descript
  I used it in the beginning when I started. Really good transcription based editing feature. I wasn’t using it much so I canceled my subscription. It doesn’t have all the features of a full video editing software it seems.
• Davinci Resolve
  Free full fledged video editing software and steeper learning curve. I use it right now for all my editing.
• Canva
  Good software for all graphics needs. I use it mostly to make YouTube thumbnails.
• OBS Studio
  Live Streaming and Screen Recording. Lots of feature and takes some time to get used to it.
• Twitch Studio
  Tried it a few times to stream to Twitch my coding sessions. Easy to use.
• Filmic Pro for iPhone
  Good software for iPhone for filming. Another phone can be used as a remote if they are on the same wifi. Good buy.
• Luma Fusion for iOS
  Bought a few weeks ago. Trying it out now.

Video Platoform

• Youtube
  I am mostly editing The Yak Collective channel. The content is created during our weekly meetings which I edit before uploading.
• Instagram
  Tried out some travel videos on my personal instagram. Video editing tools inbuilt into instagram are good for stories. But auto suggested things like reels on instagram still not good as TikTok.
• TikTok
  The suggested videos after browsing it only for some time is quite good.
• Twitch
  Learning to stream my coding session. Feels live streaming is going to be much bigger in the coming decade.

A lot of my weekends during the last one year has been going into trying out all the different software, gear and platforms. I haven’t much progress on the content creation and don’t have much to show for the amount of time I have spent on this. But I hope to keep continuing to learn as much as I can about video editing and content creation. I enjoy video editing, so let’s see where I will be next year.

find_package(Qt${QT_VERSION_MAJOR} COMPONENTS Widgets REQUIRED)

Found package configuration file: /home/xxx/Qt/6.0.4/gcc_64/lib/cmake/Qt6/Qt6Config.make but it set Qt6_FOUND to FALSE so package "Qt6" is considered to be NOT FOUND. Reason given by package: Failed to find Qt component "Widgets" config file at""

But the solution was to install some OpenGL libraries. Seems on Ubuntu installation of OpenGL is a requirement for running Qt. The problem went away after I installed libgl1-mesa-dev with the command

apt install libgl1-mesa-dev

This got me excited as I could start building this rover and have fun playing with this toy rover. The software stack uses ROS which I last played with in 2014.

The global pandemic has restricted travel for more than an year and it seems traveling would limited for atleast one more year. I think this is a good time to start a new hobby of building small robots and rovers and end my hiatus from robotics.

I hope to document the progress on a blog and on a YouTube channel. Let’s see how far I can get building this rover.

Steps to get the my current setup are just to clone my dotfiles repo on GitHub and copy-paste the content of wintermprofiles.json into the profiles.json file of windows terminal.

Opening the windows terminal settings should open the profiles.json file in the editor associated with json files.

Depending on the version of the Powershell Core used, it maybe required to change the path of the executable for Powershell Core to either 6 or 7 in the profiles.json file. By default it points to version 7.

Finally create a hard link between profiles.json file and json file in the GitHub repo.

$cd Path\To\Microsoft.WindowsTerminal_8wekyb3d8bbwe\LocalState
$mklink /h "profiles.json" "Path\To\dotfiles.git repo\wintermprofiles.json"