Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.blindference.xyz/llms.txt

Use this file to discover all available pages before exploring further.

​
Model Backends

Blindference Node uses a pluggable backend registry to support multiple inference providers. You can add custom backends without modifying core code.

​
Built-in Backends

BackendDescriptionAvailabilityModel IDs
vllmLocal GPU inference via vLLMRequires NVIDIA GPU + vllm packagefacebook/opt-125m (tier 0)
groqGroq cloud API (OpenAI-compatible)Requires GROQ_API_KEY env vargroq:llama-3.3-70b-versatile
geminiGoogle Gemini REST APIRequires GOOGLE_API_KEY env vargemini:gemini-2.5-flash
mockDeterministic SHA-256 fallbackAlways available* (universal fallback)

​
Backend Architecture

All backends implement the ModelBackend abstract base class: 
from blindference_node.models.base import ModelBackend

class MyBackend(ModelBackend):
    def name(self) -> str:
        return "my-backend"

    def is_available(self) -> bool:
        return True  # Check GPU, API key, etc.

    def supported_models(self) -> list[str]:
        return ["my-model-v1"]

    def run(self, model_id: str, prompt: str) -> str:
        # Must produce identical output for identical (model_id, prompt)
        return "deterministic result"

​
Adding Custom Backends

​
1. Config Dotted Path (Fastest)

Write a Python module anywhere on your PYTHONPATH: 
# my_backend.py
from blindference_node.models.base import ModelBackend

class OllamaBackend(ModelBackend):
    def name(self): return "ollama"
    def is_available(self): return True
    def supported_models(self): return ["ollama:llama3.1"]
    def run(self, model_id, prompt):
        # Your inference logic
        return "result"
Register via CLI: 
blindference-node models add my_backend:OllamaBackend
The path is validated (imported + instantiated), then persisted in ~/.blindference/config.json: 
{
  "custom_backends": ["my_backend:OllamaBackend"]
}

​
2. Entry-Point Plugins (For Published Packages)

Declare in your pyproject.toml: 
[project.entry-points."blindference.backends"]
ollama = "my_package.backends:OllamaBackend"
Users install your package: 
pip install blindference-node-ollama
The node auto-discovers it on startup via importlib.metadata.

​
3. Fork & Extend

Add your backend class under blindference_node/models/ and register it in backend_loader.py::_register_builtin().

​
CLI Backend Management

# List all registered backends and availability
blindference-node models list

# Test a specific backend
blindference-node models test --backend groq --model groq:llama-3.3-70b-versatile --prompt "Hello"

# Test all available backends
blindference-node models test --model qwen2.5-7b --prompt "2+2="

# Add a custom backend (validates import before saving)
blindference-node models add my_package.backends:CustomBackend

​
Registration Order & Precedence

  1. Built-in (vLLM, Groq, Gemini, Mock)
  2. Entry-point plugins (third-party pip packages)
  3. Config dotted paths (custom_backends in config.json)
Later registrations override earlier ones for the same model_id.

​
Determinism Requirement

All backends must produce byte-identical output for the same (model_id, prompt) pair. Required for quorum consensus. Tips:
  • Set temperature=0, top_p=1, top_k=-1
  • Use fixed seed=42 (Groq supports this)
  • Document any backend-specific non-determinism