Flight Glove
A glove that reads hand poses with on-device machine learning, used to fly a Gulfstream G650ER in a flight simulator I built in Unity.
- Role
- Solo project. Hardware, firmware, ML, and the simulator.
- Year
- 2026
- Tools
- Arduino Nano 33 BLE Sense, MPU-6050 IMUs, TensorFlow Lite Micro, C++, Python, Unity, C#
- Links
The problem
I’ve always wanted to know what it feels like to control a plane. Nobody’s handing a 17-year-old the keys to a Boeing 737, so I asked a smaller question: what would it feel like to fly with just my hand?
What I built
Two pieces that talk to each other over USB.
The glove. An Arduino Nano 33 BLE Sense reads three MPU-6050 motion sensors through an I²C multiplexer: one on the back of the hand and one each on the index and middle fingers. Instead of mapping raw angles straight to the controls, the glove runs two small neural networks on the microcontroller itself:
- A pose model reads the hand sensor and classifies what you’re doing: neutral, climb, dive, bank left, bank right, or a climb or dive combined with a bank.
- A throttle model reads the finger sensors. Extending your fingers brings the power up, and curling them brings it back.
The two models run independently, so you can hold a bank and adjust the throttle at the same time. Their outputs stream to the computer about 65 times a second.
The simulator. In Unity, each pose maps to a pitch and roll target, which feed an aerodynamic flight model with lift, drag, and stalls. Yaw is automatic: the airplane’s weathervane stability turns it from the bank alone, so there’s no rudder pose to hold. On top of that are an engine spool-up before takeoff, a takeoff clearance callout, a chase camera, an airspeed readout, and crash effects for when things go wrong. The airplane is a Gulfstream G650ER model.
Training the models
I wrote a capture tool that records labeled motion data from the glove and a script that trains both models and exports them as C headers for the firmware. The dataset is about 51,000 labeled samples. The models look at 32-sample windows of motion.
One thing I had to get right was validation. Overlapping windows cut from the same recording are nearly identical, so splitting them randomly into training and validation sets makes the accuracy look better than it is. I hold out whole recordings instead.
My role
Everything: the wiring, the firmware, the data collection and training pipeline, and the Unity simulator.
Outcome
The glove flies the G650ER well in the simulator, with hand poses alone. Because the readings are uncalibrated, wearing the glove differently means capturing new data and retraining.