Axiom-Drift

Currently we have:

• 190 Hey Beast samples
• 300 Unknown speech samples
• 100 Noise samples
• 590 recordings total, from 4 speakers.

What we built till now:

• Built the audio preprocessing pipeline:

  • 16 kHz mono audio
  • 1.2-second KWS windows
  • 40 × 121 log-Mel spectrogram features
  • Proper train/validation/test split.
• Developed our own Tiny CNN KWS model — no pre-trained global wake-word model.

• Improved the model through V1 → V2 → V3:

  • V1 accuracy: 28.6%
  • V2 accuracy: 71.4%
  • V3 accuracy: 78.82%
• V3 achieved 100% Hey Beast recall on the test set before thresholding.

• Tuned the wake threshold to 0.80:

  • 94.74% Hey Beast recall
  • 0% noise false activations
  • 5.36% unknown-speech false activation
• Converted the model to INT8 TensorFlow Lite for edge deployment.
• INT8 model size: ~11.7 KB
• Verified INT8 model: 78.82% test accuracy, exactly matching the Keras model.

🔜 Next Steps

Once the ESP32-S3 + INMP441 arrive:

1. Connect INMP441 → ESP32-S3.
2. Capture 16 kHz audio directly on the ESP32.
3. Run the INT8 KWS model locally.
4. Detect "Hey Beast" on-device.
5. Implement audio buffering after detection.
6. Send subsequent audio over Wi-Fi to an ASR server.
7. Measure keyword-ending → ASR-receiving latency.
8. Measure ESP32 RAM and CPU usage to verify the hackathon constraints.

Later, improve V3 using real hardware recordings/hard negatives if needed.

(Also need to collect more data samples )

Current status: 🟢 PC-side KWS model is working and INT8-ready. Hardware integration is the next phase.