mlx-local-inference
von bendusy
DevOps & CloudKurzbeschreibung
Full local AI inference stack on Apple Silicon Macs via MLX.
Beschreibung
Was dieser Skill macht
Runs a full AI inference stack locally on Apple Silicon Macs using the MLX framework. Covers LLM chat (Qwen3-14B, Gemma3-12B), speech-to-text via Whisper and Qwen3-ASR, text embeddings, OCR with PaddleOCR, and text-to-speech. All services expose OpenAI-compatible APIs on localhost.
Eliminates cloud API costs and keeps all data local while exposing the same OpenAI-compatible API format that most AI tooling already supports.
Wann du ihn einsetzt
- Transcribing Cantonese audio without uploading to cloud services
- Running privacy-sensitive LLM queries entirely on-device
- Extracting text from scanned documents or images via OCR
- Building local semantic search with on-device text embeddings
- Auto-transcribing a folder of meeting recordings in the background
Beispiel-Workflow
User asks: Transcribe and clean up the Cantonese meeting recording at ~/Downloads/meeting.m4a
- 1
Copy meeting.m4a into ~/transcribe/ to trigger the daemon
- 2
Daemon detects the file within 15 seconds and sends it to Qwen3-ASR on port 8788
- 3
ASR output is saved as meeting_raw.md with the raw transcript
- 4
Qwen3-14B reads the raw transcript, corrects homophones, adds punctuation, and preserves Cantonese characters
- 5
Corrected file is moved to ~/transcribe/done/meeting_corrected.md
meeting_corrected.md in ~/transcribe/done/ with a clean, punctuated Cantonese transcript
Voraussetzungen
- Apple Silicon Mac (M1, M2, M3, or M4)
- Python packages: mlx, mlx-lm, mlx-audio, mlx-vlm
Installation
clawhub install bendusy/mlx-local-inferenceFühre den CLI-Befehl aus oder füge den Prompt in Claude Code, Cursor, Codex oder deinen Coding-Agenten ein.