The same note goes into two engines. Each engine saves a WAV file. The computer compares the two files.
Send the same note to both engines.
note = A4 // 440 Hz
A is the reference. B is my engine. Both get the same note.
A = reference.render(note) B = myEngine.render(note)
Each engine writes a WAV file. A WAV file is a list of numbers.
A.wav = [0.00, 0.31, 0.59, 0.81, ...] B.wav = [ ... ] // no speaker, no microphone
A flat line means the two files are the same. Here the line is not flat, so B is different.
diff = A - B max(|diff|) = 0.27 // 0 = same
FFT shows every overtone. At 1320 Hz, B is 12 dB lower than A. That is the bug.
fft(B) - fft(A) 440 Hz 0 dB PASS 880 Hz 0 dB PASS 1320 Hz -12 dB FAIL
The computer checks every number. FAIL: fix the code and run again. It takes seconds and needs no person.
Hz pitch PASS dB level PASS ms timing PASS NOISE PASS FFT overtones FAIL while (FAIL) { fix(); test(); }
When every check is PASS, listen one time. Ears answer one question: does it sound beautiful?
5 / 5 PASS → listen 1×
| My engine is | How to get the WAV |
|---|---|
| a plug-in (VST3, AU) | REAPER or Pedalboard loads it and saves the WAV |
| a web app | OfflineAudioContext (built into the browser) |
| my own program | add one command: engine --render note.mid out.wav |
| Job | Program | Cost |
|---|---|---|
| Play the same note through A and B, save WAV | REAPER reaper -renderproject test.rpp | paid, 60-day full trial |
| The same job from Python, no window | Pedalboard | free |
| Read the WAV, do A - B and FFT | Python + NumPy + SciPy | free |
| Say PASS or FAIL | pytest | free |
| Run after every code change | Git + a CI runner (GitHub Actions) | free to start |
| Look at a WAV when a test fails | Sonic Visualiser or Audacity | free |
| Check that a plug-in does not crash | pluginval | free |
| Blind listening at the end (step 7) | foobar2000 + ABX Comparator | free |
| Kind of sound | Program | Cost |
|---|---|---|
| Instruments from recordings | Kontakt Player | free |
| Piano built by math | Pianoteq | paid; trial has 8 silent notes and stops after 20 minutes |
| Wind instruments built by math | SWAM | paid |
| Guitar amp | Neural DSP | paid, free trial |
| Clean effects: EQ, compressor | FabFilter | paid, 30-day full trial |
| Reverb | Valhalla Supermassive | free |
| Synthesizer | Vital or Serum | Vital Basic is free; Serum is paid |
| The same recordings in both engines | Salamander Grand Piano | free |
Save the WAV files of A one time, during the free trial. After that, compare with the saved files. Measure their sound. Do not copy their recordings into your product.
Free and paid were checked on 2026-11-17. Check each site for the terms today.
| Ears | Numbers | |
|---|---|---|
| Time for one test | minutes | seconds |
| Same result every time | no | yes |
| Shows where the bug is | no | yes: 1320 Hz, -12 dB |
| Runs after every code change | no | yes |
| Knows if it sounds beautiful | yes | no |