Birdnet-go is a real-time sound analyzer that detects bird species. It can take in audio from microphones. I set mine up to stream audio from my security cameras that I already had installed. There’s even a Home Assistant App, if you’re using it.
I live in a more suburban/semi-urban environment, but was really surprised to so many different bird species.
LLM and VLM are generally the type of model that needs a ton of resources to run. Birdnet uses an audio classification model, but there are dozens of types of models.
I’m building specialized task-focused pipelines which keep the memory footprint small for self-hosted users, and I’m working on a unified framework too make building other pipelines and applications much easier: https://docs.circuitforge.tech/
If you give me some details on what you’re after I might be able to find you some projects or at least clarify keywords
Edit: an example of how efficient these can get, I have a little ESP32-driven waveshare display running an ONNX wake-word detection model which is absolutely nuts. 4mb psram and a dual-core 240mhz cpu and its only job is handling “hey Mycroft” and establishing a proper speech-to-text connection with my server downstairs