CymaticBridge
The future of robot hearing
Model scores
Experiment progress
transductionCARFAC turns sound into the firing pattern of the inner ear
Architecture overview
Noise test
Fold-by-fold validation
Noise test
One dog bark, both ears
Latest log
Ear pass
Machine
Training
Run 1 · the test
Runs 1 to 5: what we learned, what runs next
Run 1 verdict
Mel won: 82.7% against 69.9% on ESC-50, and it stayed ahead in noise. Both models learned (untrained copies score 26 to 33%), so the training worked; the cochlea input was the weak link.
Why the cochlea lost
Quick screens scored eleven ways of reading the ear in about 15 minutes. Run 1's cochlea input squashed the nerve signal with a log curve that buries quiet detail. With the ear's own cube-root compression the same signal already ties mel, and beats it in noise.
Closer to the real nerve
The real auditory nerve has three kinds of fibre: very sensitive, medium and hard to excite. CARFAC can model all three. Together with a first "brainstem" step (how fast each band's loudness flickers), this was the best input of all.
| Quick screen (simple scorer, not a trained model) | ESC-50 | Pink 20 dB | Pink 10 dB | Pink 0 dB |
|---|---|---|---|---|
| Mel, as in Run 1 | 53.7% | 37.8% | 25.5% | 6.2% |
| Cochlea, as in Run 1 | 50.1% | 36.8% | 24.0% | 9.8% |
| Nerve, cube-root compression | 54.5% | 44.8% | 32.0% | 7.2% |
| Three fibre types | 56.9% | 46.5% | 35.0% | 9.0% |
| Mel + cube-root + brainstem (fair control) | 58.9% | 52.2% | 42.0% | 12.2% |
| Three fibres + brainstem | 62.4% | 56.2% | 39.5% | 11.5% |
Teaching machines to hear the way people do
Machine hearing today starts from a picture of sound. We start from a model of the human ear.
About CymaticBridge
The goal
The goal is robots that hear the way people do: that catch happiness, sadness or anger in a voice, and hear a violinist playing Mozart as music rather than as a stream of numbers. In the long run, we want that machine to listen to music and then create it, using our virtual studio, the Hearing Machine (under How hearing works). Because it can make unlimited audio where we know exactly how every sound was made, we can teach the AI to listen to music: first to judge mixes, later to write its own patches.
Why we start with the ear
Today's listening AI starts from a mel spectrogram, a picture of sound that keeps only rough loudness per frequency band. It then learns patterns from huge amounts of data. It can label a sound without perceiving it the way we do.
CymaticBridge starts from the other end: give the machine a human-shaped ear first (cochlea, auditory nerve, later brainstem and cortex-like processing), then let it learn. If the ear is right, the machine should come to share our perception, including our quirks and illusions.
How we test it
We trained two of our own AI listening models from scratch: one hears through a cochlea model, the other through a standard spectrogram, and everything else is identical. Neither started from a pre-trained AI. Both followed the same published, open design and learned from about 110 hours of everyday sound and speech, without ever being told what any sound was. Then we test both on how well they recognize 50 kinds of sound, in quiet and in noise.
The first milestone is practical: a cochlea front end must beat mel on a standard listening test, fairly. Until it does, a human-shaped ear is only an idea.
Live from the lab
Lab notebook
Every run is a fair race: our ear AI against the standard mel AI, same network, same training, same test. Here is every result so far, and what each one taught us.
| Loading the results… |
The Living Ear Map
Every stage of hearing, ear to brain, and how far our robot's hearing has come. It grows as we build: each stage turns green when our model has it.
built: in our model now · stand-in: a trained network does this job for now · next: the next stage we build · not yet: still to come. Dashed lines are feedback running back down.
Explore
Contact
Questions, ideas or want to work together? Get in touch with Ryan Hatch.