Have you ever wondered if it was possible for a fish to drive it’s own tank around a room? I did.
This year, my team and I managed to get a few hundred dollars of research budget allocated to us. We of course, immediately decided to do no actual research, and instead make something cool.
Overview
The overall project consists of three main parts:
- The fish tank itself.
- Something to move the fish tank around (wheels perhaps).
- Something like a camera to keep track of the fish’s motions so that they could drive the tank.
The biggest problem was with the fish tank base, the thing that moves it all around. We didn’t want to use tank treads or anything similar because having to rotate the tank above would defeat the whole point of letting the fish choose the direction and speed that the fish tank moved at. We ended up going with four omnidirectional wheels placed at forty-five degree angles to the corners of the tank. This let me drive the stepper motors in different directions to have the wheels effectively pull the tank in any direction without rotating. Seeing it move is somewhat funny, it looks like the wheels are rotating and the tank is just sliding across the ground.
For the fish tracking system, I decided to just use a Raspberry Pi with a cheap Picamera module mounted to the top of the fish tank looking down. Here’s a photo of the initial solid model done by one of my teammates:
And here’s how the final result turned out:
There were a whole slew of engineering issues, screws not quite fitting, tolerances being too high or low, not enough clearance, etc. Ultimately, nothing that a roll of Duct Tape and some hastily 3D-printed parts couldn’t fix.
The fish tracker
This was my part of the project, the little bit of magic that I did my best to add. The challenge of accurately tracking the fish, and converting that into movement vectors was vastly harder and vastly easier than I thought it would be. First, I want to go over how it all works.
Fish detection
I implemented the fish detection system using OpenCV. I didn’t feel the need for anything more advanced than what the toolkit offered. In a world of AI, image segmentation for a goldfish felt like overkill.
Instead, I just taped a sheet of black paper to the base of the tank and searched for “high intensity colors” i.e. bright colors that are far from black.
This proved to be almost perfectly accurate already, a little bit of number tweaking, and mask refining and I could figure out where the fish was with almost perfect accuracy. Nice.
Fish prediction
Next, I created a ring buffer of fish locations, basically a list of the last n locations that the fish had been in since this image frame was taken. Applying a polynomial fit to these x and y coordinates was enough for me to accurately predict where the fish would be in the next hundred milliseconds or so. This proved to be the secret sauce of the whole project, making the tank move where the fish was about to be, not where it just was. This made the tank a lot more responsive to the fish, instead of always driving to where it had just been swimming.
Driving the motors
The whole platform was driven by four stepper motors wired to a single ESP-32 micro controller. I found that the simplest possible way to direct it from the Raspberry Pi was to literally plug it into the Raspberry Pi and send it data over a serial connection. The Raspberry Pi would do all the calculation and then just send a polar coordinate of the form (magnitude, angle) to the ESP-32, which would then determine which directions to set the stepper motors to in order to move at roughly that angle (one of the drawbacks of the omnidirectional platform was how it didn’t have a lot of degrees of motion).
Overall
At the end of the day, it’s a silly project that I had a lot of fun making. There are some very interesting components of the system I worked on, but ultimately they’re buried in the heaps of jank required to make the final tank work in time to present it at the research conference.
I hope to continue refining this project over the summer, perhaps I’ll write more about it then.