कंटेंट पर जाएँ
TakoAPI
स्किल्स पर वापस

homelab-cluster

mlesnews द्वारा

DevOps & Cloud
609clawskills.sh द्वारा बताए गए डाउनलोड
2स्टार
0व्यूज़
0 लाइकइस स्किल को लाइक करने के लिए साइन इन करें
—अभी तक कोई रेटिंग नहीं
आपकी रेटिंग

संक्षिप्त

Manage multi-tier AI inference clusters for homelabs.

विवरण

यह स्किल क्या करती है

Manages multi-tier AI inference clusters for homelab environments. Covers health monitoring across GPU and CPU nodes, MoE-based request routing, automatic recovery for downed nodes, and model deployment via Ollama and llama.cpp. Includes GPU memory planning, Docker volume strategies, and a unified LiteLLM API gateway.

Consolidates health checks, failover routing, and node recovery into one workflow instead of managing each Ollama or llama.cpp instance separately.

इसे कब इस्तेमाल करें

  • Routing coding tasks to a dedicated coder model on the local GPU node
  • Recovering a remote Docker Desktop crash without manual intervention
  • Planning which quantized models fit within 24GB VRAM across a cluster
  • Restarting GPU containers sequentially to avoid CUDA initialization deadlocks
  • Exposing all cluster models through a single OpenAI-compatible endpoint

उदाहरण वर्कफ़्लो

इनपुट

User asks: route this code review request to the best available model

एजेंट
  1. 1

    Classify the prompt as a 'code' task

  2. 2

    Query the health endpoint for the designated coder model node

  3. 3

    If healthy, return the coder model endpoint; if degraded, check fallback tier

  4. 4

    Select next-best available model from health results

  5. 5

    Return the chosen model endpoint and tier metadata to the caller

आउटपुट

Request routed to remote/mark-i running Qwen2.5-Coder-7B at full GPU capacity

आवश्यकताएँ

  • SSH access credentials stored in a vault (e.g., Azure Key Vault or HashiCorp Vault) for remote node recovery

इंस्टॉलेशन

clawhub install mlesnews/homelab-cluster

CLI कमांड चलाएँ, या प्रॉम्प्ट को Claude Code, Cursor, Codex या जो भी कोडिंग एजेंट आप इस्तेमाल करते हैं उसमें पेस्ट करें।

लिंक