Bỏ qua tới nội dung
TakoAPI
Quay lại kỹ năng

mlx-local-inference

bởi bendusy

DevOps & Cloud
329Lượt tải do clawskills.sh công bố
1Sao
0Lượt xem
0 lượt thíchĐăng nhập để thích kỹ năng này
—Chưa có đánh giá
Đánh giá của bạn

Tóm tắt

Full local AI inference stack on Apple Silicon Macs via MLX.

Mô tả

Kỹ năng này làm gì

Runs a full AI inference stack locally on Apple Silicon Macs using the MLX framework. Covers LLM chat (Qwen3-14B, Gemma3-12B), speech-to-text via Whisper and Qwen3-ASR, text embeddings, OCR with PaddleOCR, and text-to-speech. All services expose OpenAI-compatible APIs on localhost.

Eliminates cloud API costs and keeps all data local while exposing the same OpenAI-compatible API format that most AI tooling already supports.

Khi nào nên dùng

  • Transcribing Cantonese audio without uploading to cloud services
  • Running privacy-sensitive LLM queries entirely on-device
  • Extracting text from scanned documents or images via OCR
  • Building local semantic search with on-device text embeddings
  • Auto-transcribing a folder of meeting recordings in the background

Quy trình ví dụ

ĐẦU VÀO

User asks: Transcribe and clean up the Cantonese meeting recording at ~/Downloads/meeting.m4a

AGENT
  1. 1

    Copy meeting.m4a into ~/transcribe/ to trigger the daemon

  2. 2

    Daemon detects the file within 15 seconds and sends it to Qwen3-ASR on port 8788

  3. 3

    ASR output is saved as meeting_raw.md with the raw transcript

  4. 4

    Qwen3-14B reads the raw transcript, corrects homophones, adds punctuation, and preserves Cantonese characters

  5. 5

    Corrected file is moved to ~/transcribe/done/meeting_corrected.md

ĐẦU RA

meeting_corrected.md in ~/transcribe/done/ with a clean, punctuated Cantonese transcript

Yêu cầu

  • Apple Silicon Mac (M1, M2, M3, or M4)
  • Python packages: mlx, mlx-lm, mlx-audio, mlx-vlm

Cài đặt

clawhub install bendusy/mlx-local-inference

Chạy lệnh CLI, hoặc dán prompt vào Claude Code, Cursor, Codex hay coding agent bạn đang dùng.

Liên kết