> ## Documentation Index
> Fetch the complete documentation index at: https://docs.blindference.xyz/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Backends

> Pluggable inference backend system

# Model Backends

Blindference Node uses a **pluggable backend registry** to support multiple inference providers. You can add custom backends without modifying core code.

## Built-in Backends

| Backend  | Description                        | Availability                         | Model IDs                      |
| -------- | ---------------------------------- | ------------------------------------ | ------------------------------ |
| `vllm`   | Local GPU inference via vLLM       | Requires NVIDIA GPU + `vllm` package | `facebook/opt-125m` (tier 0)   |
| `groq`   | Groq cloud API (OpenAI-compatible) | Requires `GROQ_API_KEY` env var      | `groq:llama-3.3-70b-versatile` |
| `gemini` | Google Gemini REST API             | Requires `GOOGLE_API_KEY` env var    | `gemini:gemini-2.5-flash`      |
| `mock`   | Deterministic SHA-256 fallback     | Always available                     | `*` (universal fallback)       |

## Backend Architecture

All backends implement the `ModelBackend` abstract base class:

```python theme={null}
from blindference_node.models.base import ModelBackend

class MyBackend(ModelBackend):
    def name(self) -> str:
        return "my-backend"

    def is_available(self) -> bool:
        return True  # Check GPU, API key, etc.

    def supported_models(self) -> list[str]:
        return ["my-model-v1"]

    def run(self, model_id: str, prompt: str) -> str:
        # Must produce identical output for identical (model_id, prompt)
        return "deterministic result"
```

## Adding Custom Backends

### 1. Config Dotted Path (Fastest)

Write a Python module anywhere on your `PYTHONPATH`:

```python theme={null}
# my_backend.py
from blindference_node.models.base import ModelBackend

class OllamaBackend(ModelBackend):
    def name(self): return "ollama"
    def is_available(self): return True
    def supported_models(self): return ["ollama:llama3.1"]
    def run(self, model_id, prompt):
        # Your inference logic
        return "result"
```

Register via CLI:

```bash theme={null}
blindference-node models add my_backend:OllamaBackend
```

The path is validated (imported + instantiated), then persisted in `~/.blindference/config.json`:

```json theme={null}
{
  "custom_backends": ["my_backend:OllamaBackend"]
}
```

### 2. Entry-Point Plugins (For Published Packages)

Declare in your `pyproject.toml`:

```toml theme={null}
[project.entry-points."blindference.backends"]
ollama = "my_package.backends:OllamaBackend"
```

Users install your package:

```bash theme={null}
pip install blindference-node-ollama
```

The node auto-discovers it on startup via `importlib.metadata`.

### 3. Fork & Extend

Add your backend class under `blindference_node/models/` and register it in `backend_loader.py::_register_builtin()`.

## CLI Backend Management

```bash theme={null}
# List all registered backends and availability
blindference-node models list

# Test a specific backend
blindference-node models test --backend groq --model groq:llama-3.3-70b-versatile --prompt "Hello"

# Test all available backends
blindference-node models test --model qwen2.5-7b --prompt "2+2="

# Add a custom backend (validates import before saving)
blindference-node models add my_package.backends:CustomBackend
```

## Registration Order & Precedence

1. **Built-in** (vLLM, Groq, Gemini, Mock)
2. **Entry-point plugins** (third-party pip packages)
3. **Config dotted paths** (`custom_backends` in `config.json`)

Later registrations override earlier ones for the same `model_id`.

## Determinism Requirement

All backends **must produce byte-identical output** for the same `(model_id, prompt)` pair. Required for quorum consensus.

**Tips**:

* Set `temperature=0`, `top_p=1`, `top_k=-1`
* Use fixed `seed=42` (Groq supports this)
* Document any backend-specific non-determinism