Magic Wand Remote

How It Started

The summer before my second year of high school, the head of my department asked if I wanted to come up with a topic for a project competition. I’d been rewatching the Harry Potter movies that summer and was kind of obsessed with the wizarding world, so I casually threw out the idea of a magic wand remote. To my surprise, this totally random idea actually got approved! On top of that, a classmate of mine wanted to try building an LED cube, and I figured the cube could work as a display for casting spells. My classmate was in right away, so we signed up together for the creative category of the project competition!

How It Works

1. An ESP32 reads raw angular velocity from an MPU6050
2. A Kalman filter plus integration turns those values into relative angles
3. The angle data plus the Pythagorean theorem gives the wand’s movement path, which gets recorded
4. The path gets normalized (scaled so it touches all four edges of a 32×32 box)
5. Feature recognition:
   1. If the X-axis scale factor during normalization is above a threshold, it counts as a vertical line
   2. If the Y-axis scale factor during normalization is above a threshold, it counts as a horizontal line
   3. If the number of points on a single horizontal row along the top and bottom edges is above a threshold, it counts as a square
   4. If that number is above a threshold but below the square threshold, it counts as a circle
   5. Anything else counts as a triangle
   6. Each shape triggers a matching action (in the demo version, it controls music playback on a phone over Bluetooth, like pause and play)

What I Learned

We originally planned to enter this in the creative category of a project competition. But right before the preliminary round’s submission deadline, it turned out we’d printed the rules from the wrong year, and by the time we went to submit, the deadline had already passed. So we had to drop out.

Building it was quite the saga, though. For example, the original plan for gesture recognition was to train a TensorFlow Lite model and run it on the ESP32 to recognize the gesture path. But when we trained the model, the results were way worse than expected. My best guess was that it had to do with the model choice and some parameter tuning. Since I had zero machine learning experience, I brought the problem to our project advisor first. He said the deadline was close, so we should just try brute-forcing it with good old if-else for now. So I drew on my past experience brute-forcing image recognition with if-else and came up with an algorithm that actually works well enough.

At that point, the plan was to do the rest, like controlling appliances with infrared, after submitting the preliminary materials. As long as we made it to the next round, we’d be ready in time for the exhibition. But then the whole missed-registration thing happened, and once winter break started, the project kind of just fizzled out. That is, until the vocational high school expo, when our advisor suddenly said our project would be on display. So I pulled an all-nighter and finally got Bluetooth music control for phones working in time for the exhibition!

Through all this, I learned just how important it is to double-check the dates on documents. I also got a lot of practice with array logic, and got my first taste of TensorFlow, which has helped me a ton since!