Bỏ qua tới nội dung
TakoAPI
Quay lại kỹ năng

local-first-llm

bởi joelnishanth

Coding Agents & IDEs
318Lượt tải do clawskills.sh công bố
1Sao
0Lượt xem
0 lượt thíchĐăng nhập để thích kỹ năng này
—Chưa có đánh giá
Đánh giá của bạn

Tóm tắt

Routes LLM requests to a local model (Ollama, LM Studio, llamafile) before falling back to cloud APIs.

Mô tả

Kỹ năng này làm gì

Routes LLM requests to a local model (Ollama, LM Studio, or llamafile) before falling back to cloud APIs. Routing decisions factor in data sensitivity, prompt complexity, and local provider availability. Token usage and cost savings are logged to a persistent dashboard.

Combines automatic routing logic with cost tracking, so users get privacy and savings benefits without manually deciding which model to use each time.

Khi nào nên dùng

  • Keeping sensitive prompts off cloud APIs
  • Reducing OpenAI API costs on repetitive tasks
  • Summarizing meeting transcripts with a local model
  • Checking how much money local routing has saved this month
  • Automatically routing complex queries to cloud while simple ones stay local

Quy trình ví dụ

ĐẦU VÀO

User asks: summarize this meeting transcript and keep it private

AGENT
  1. 1

    Run check_local.py to detect available local providers

  2. 2

    Run route_request.py with the prompt and token estimate; sensitivity check routes to local

  3. 3

    Send the prompt to Ollama via its local API endpoint

  4. 4

    Run track_savings.py log to record the token count and routing decision

  5. 5

    Run dashboard.py to display updated cumulative savings

ĐẦU RA

Meeting transcript summarized locally; dashboard shows 800 tokens saved and cost avoidance logged

Yêu cầu

  • Ollama, LM Studio, or llamafile installed and running locally

Cài đặt

clawhub install joelnishanth/local-first-llm

Chạy lệnh CLI, hoặc dán prompt vào Claude Code, Cursor, Codex hay coding agent bạn đang dùng.

Liên kết