Hugging Face Releases Swift Packages for On-Device Speech Recognition and Decision Making

Hugging Face has released AnySpeechRecognizer and AnyDecisionModel, two new Swift packages that allow developers to run open-source speech recognition and decision-making models directly on Apple devices. The most significant shift here is the decoupling of the inference engine from the API: developers can now swap Apple’s proprietary models for open alternatives like NVIDIA’s Parakeet or Alibaba’s Qwen3-ASR without rewriting their application logic.

What happened

According to the Hugging Face Blog, AnySpeechRecognizer mirrors Apple’s SpeechAnalyzer API introduced in macOS 26 and iOS 26. By keeping the interface identical, developers can change the underlying engine by simply altering how the transcriber is instantiated. The package supports three primary engines: Parakeet v3 (NVIDIA’s 0.6B polyglot model), Parakeet Ultra (Moondream’s post-trained variant), and Qwen3-ASR (Alibaba’s 0.6B and 1.7B models). All three run on a custom Core ML runtime. Parakeet models support live input, word timing, and confidence scores, while Qwen3-ASR is better suited for file transcription due to its slower GPU-based processing.

The package also introduces SpeakerDiarizer, a module that runs NVIDIA’s Streaming Sortformer on Core ML to attribute speech to up to four speakers simultaneously—a capability Apple’s framework currently lacks. Hugging Face reports that on an M4 Max Mac Studio, Parakeet can transcribe an hour of audio in 20 to 40 seconds from a file, using approximately 100 MB of memory. However, the blog notes that word error rates exceeded 23% on AMI meeting recordings, highlighting that overlapping speech remains a challenge for all tested engines.

AnyDecisionModel addresses a different need: zero-shot classification. Instead of prompting a large language model and parsing text output, this package uses decision models to return structured probabilities. For example, it can determine if a message is urgent or which team should handle a ticket, returning a Choice type with probabilities for each option rather than a single text string. The MLX backend builds prompts with one-token labels and reads probabilities in a single forward pass.

Why it matters for small company builders

For small teams, the cost and latency of sending audio to cloud APIs are significant bottlenecks. AnySpeechRecognizer solves this by enabling high-quality, private, on-device transcription. If you are building a customer support tool or a field service app, you can now process voice notes locally, eliminating per-minute API fees and data privacy concerns. The ability to swap engines is crucial: if Parakeet struggles with a specific accent, you can switch to Qwen3-ASR, which supports 30 languages and 22 Chinese dialects, without refactoring your codebase. This modularity mirrors the best practices of open-source agent frameworks, giving you control over the inference layer while maintaining type-safe Swift interfaces.

What to do this week

1. Clone the AnySpeechRecognizer repository and run the hear command-line tool on your own audio files to benchmark word error rates against your specific use case.
2. If you currently use Apple’s SpeechAnalyzer, create a test branch where you swap the import to AnySpeechRecognizer and instantiate a Parakeet transcriber to verify compatibility with your existing async sequence handling.
3. Experiment with AnyDecisionModel by implementing a simple classifier for a repetitive task in your company, such as categorizing incoming support tickets, to evaluate the utility of probability-based outputs over LLM text parsing.

Sources

• Introducing AnySpeechRecognizer and AnyDecisionModel: Open Models for Speech and Decisions on Apple Platforms — Hugging Face Blog