Portfolio project · Experiment · 3D
I gave a fruit fly a keyboard. Nothing is trained. It plays badly.
Scientists have mapped every connection in a fly's brain. So what would it do with a keyboard?
It started on Instagram. I kept seeing people making things with a fly's brain, and I wanted to know how. Looking into it led me to the study behind it: researchers have mapped the complete wiring of a fruit fly's nervous system, every neuron and every synapse, and released it for anyone to use.
I found that fascinating, and I wanted to see if I could do something with it myself. The concept I came up with was the most ridiculous honest test I could think of: take the real wiring, plug the fly's eyes into a game of Tetris, connect its own command neurons to the arrow keys, and just watch.
It's a quick, silly, free project, but it's built on real science, and it's honest about what it is: an art piece and a toy, not a neuroscience result.
The fly's brain is a 318-neuron slice cut from a complete map of a real fly's nervous system. Every connection weight is a real synapse count. I chose the slice to run from the cells in its eyes that detect looming objects to the cells that send commands down to its body, so signals have a real path from what it sees to what it does.
Falling blocks loom bigger as they drop, and tall stacks loom too. The left eye watches the left half of the board, the right eye the right.
60 looming detectors, LC4 and LPLC2: the cells that fire when something is about to hit the fly.
240 neurons on the strongest real paths, simulated every 0.1 ms as spiking neurons.
18 descending neurons. Whichever group is most unusually active presses its key.
So a block dropping towards the fly can set off its escape reflex, and that slams the piece down. Which is roughly what a fly would do.
Live preview
The real thing, running live in your browser. Drag to move the camera, and hover the brain map in the corner to see each neuron.
I first pictured a pixel fly at a desk, but it was stronger as something that feels real. So it became a real 3D fly at an old beige box computer, alone in a stark white, high-contrast room. My reference was Batman's garage in The Dark Knight: clean, empty and lit like a lab.
A slow cinematic camera drifts between six framed shots, and you can drag to take over at any time.
The best bit
The panel in the corner is the fly's brain while it plays. Every dot is a neuron at its real position in the fly's head, flashing red when it fires, with lines tracing signals along its strongest connections. Hover a neuron to see its cell type and neurotransmitter, or click it to poke it and see what happens.
A log shows why each key was pressed: which neuron group fired, and how far above its usual activity it was. The meters show which key's neurons are most excited at that moment.
When it's idle it rubs its front legs together, the way flies do. Its antennae twitch, its abdomen breathes, and its wings flick when it slams the drop key.
The body is NeuroMechFly, rebuilt from microscope scans of a real fly: 39 parts on their measured skeleton. I added shaders for the facets of the compound eyes, the veins in the see-through wings and the cuticle texture, plus fine bristles scattered over the body.
An inverse kinematics solver, kept within the real joint limits of a fly's leg, places the front feet on the keys while the middle and back feet stay planted on the desk.
Faint blue columns on the CRT show what each eye is watching. And the screen itself glows like an old monitor, with scanlines and phosphor bloom.
The spiking simulation of 318 neurons and 13,620 synapses runs live on your computer's processor, stepped every tenth of a millisecond, using the same neuron model as a 2024 study in Nature.
Projects like this are easy to oversell, so I wanted to be clear about which parts are science and which are my choices. A real fly would just fly away.
Built on open science: the MaleCNS connectome (FlyEM / HHMI Janelia, Cambridge, MRC LMB, Google Research, CC BY 4.0), the NeuroMechFly body (Apache-2.0), the neuron model from Shiu et al., Nature 2024, and preprocessing adapted from Xenova's fruit-fly simulation (MIT). Tetris is a trademark of The Tetris Company; this is an unaffiliated, non-commercial fan project.
No framework and no build step: plain JavaScript and three.js, so you can open the page and it runs. Python tools cut the brain slice out of the full connectome and pack the fly's body for the web, and a script can fetch the full 1.1 GB dataset for anyone who wants to go further.