MCP Server

Model Context Protocol (MCP) server providing a unified tool interface for LLMs to interact with Haiven infrastructure.

Overview

Architecture

                                    +------------------+
                                    |   Echo/LibreChat |
                                    |  (MCP Client)    |
                                    +--------+---------+
                                             |
                                             | MCP Protocol (HTTP)
                                             v
+---------------------------+       +--------+---------+       +------------------+
|   Traefik                 |       |                  |       |   LiteLLM        |
|   mcp.haiven.site:443    +------>+   MCP Server     +------>+   (Port 4000)    |
|                           |       |   (Port 8765)    |       |                  |
+---------------------------+       +--------+---------+       +--------+---------+
                                             |                          |
                            +----------------+----------------+         |
                            |                |                |         v
                    +-------v------+ +-------v------+ +-------v------+ +--------+
                    |  Prometheus  | |    Loki      | |   ComfyUI    | |Langfuse|
                    |  (Metrics)   | |   (Logs)     | | (Image Gen)  | |(Traces)|
                    +--------------+ +--------------+ +--------------+ +--------+

LiteLLM Integration

As of version 1.2.0, all LLM calls from MCP Server are routed through LiteLLM for observability.

Benefits

• Langfuse Observability: All LLM calls are traced and logged to Langfuse (ai-ops.haiven.site)
• Cost Tracking: Unified cost tracking across all clients using LiteLLM
• Model Access: Access to all 33 LiteLLM models including TTS/STT capabilities
• API Key Management: Centralized API key management through LiteLLM

Configuration

The following environment variables control the LiteLLM integration:

Viewing Traces

1. Navigate to https://ai-ops.haiven.site
2. Log in to Langfuse
3. View traces under the MCP Server project
4. Analyze costs, latencies, and token usage

Changelog

2026-05-01 — v2 Search/Scrape Remediations

Seven remediations landed across search.py, scrape.py, and research.py. Container restart required to activate (bind-mounted source).

search/web new params:
- source_type (enum: general|academic|news|code|social|primary) — expands to a curated SearXNG engine CSV; prefer over hand-typed engines. general uses SearXNG's default mix (no engines override).
- snippet_min_chars (int, default 0, range 0–1000) — when >0, flags results with short snippets as snippet_too_short: true. Does not auto-refetch; use as a signal to call scrape/url.

search/web new response fields:
- engine_status — per-engine dict: {name: {status, result_count, reason}}. Distinguishes real zero hits from engine timeouts.
- engines_failed — flat list of unresponsive engine names.
- snippet_length — character count of each result's snippet.

search/and_fetch new params:
- source_type — same enum as search/web; forwarded to the underlying search call.
- body_max_chars (int, default 2000, max 50000) — per-body markdown truncation limit. 0 = no truncation.

search/and_fetch new response field:
- suppressed_global — URLs dropped by the cross-query dedup/per-domain-cap pass; exposed for diagnostics.

scrape/* envelope promotion:
- scrape/url, scrape/batch, scrape/site, scrape/sitemap now use the unified _envelope response shape (build_success/build_error) matching the search tools.
- error_category on all scrape errors: transient (timeout/connection — worth retrying) | client (bad input — don't retry) | upstream (Crawl4AI or remote returned empty/error).

scrape/batch new param:
- body_max_chars (int, default 2000, max 50000, 0 = no truncation) — replaces hardcoded 2000-char truncation.

Langfuse child spans added:
- scrape.url.crawl4ai_post on scrape/url
- scrape.batch.crawl4ai_post on scrape/batch

Tools

MCP Server provides 23 tools across 12 namespaces:

System Status

Docker Management

File Operations

Code Execution

Search

Web Scraping (Crawl4AI)

Monitoring

Audio

Uptime

Image Generation

API Endpoints

Environment Variables

Service Endpoints

Embedded Whisper

Server Configuration

Security

Security Features

• Path Validation: Files validated with resolve() + relative_to()
• File Blocklist: Blocks .env, .ssh/, *.key, *.pem, secrets
• Container Allowlist: Only specified containers can be restarted
• Sandbox Isolation: Code execution in isolated containers with no network
• Command Sanitization: Dangerous command patterns are blocked

Health Check

curl -f http://localhost:8765/health

Response:

{
  "status": "healthy",
  "service": "mcp-server",
  "version": "1.2.0",
  "uptime_seconds": 12345
}

Usage Examples

List Available Tools

curl https://mcp.haiven.site/tools

Get Docker Status

curl -X POST https://mcp.haiven.site/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "method": "tools/call",
    "params": {
      "name": "status/docker",
      "arguments": {}
    }
  }'

Get GPU Status

curl -X POST https://mcp.haiven.site/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "method": "tools/call",
    "params": {
      "name": "status/gpu",
      "arguments": {"gpu_id": "all"}
    }
  }'

List Available Models (via LiteLLM)

curl -X POST https://mcp.haiven.site/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "method": "tools/call",
    "params": {
      "name": "status/models",
      "arguments": {}
    }
  }'

Transcribe Audio

curl -X POST https://mcp.haiven.site/v1/audio/transcriptions \
  -F "file=@audio.mp3" \
  -F "model=whisper-1"

Docker Commands

Start Service

cd /mnt/apps/docker/ai/mcp-server
docker compose up -d

View Logs

docker logs -f mcp-server

Restart Service

docker compose restart mcp-server

Rebuild After Code Changes

docker compose build --no-cache
docker compose up -d

Directory Structure

/mnt/apps/docker/ai/mcp-server/
├── docker-compose.yml      # Service configuration
├── README.md               # This file
└── USER_GUIDE.md           # End-user documentation

/mnt/apps/src/mcp-server/   # Source code (hot-reloaded)
├── Dockerfile
├── server.py               # Main server
├── config.py               # Configuration
├── whisper_manager.py      # Embedded Whisper
├── tools/                  # Tool implementations
│   ├── status.py           # System status tools
│   ├── docker_tools.py     # Docker management
│   ├── files.py            # File operations
│   ├── sandbox.py          # Code execution
│   ├── monitoring.py       # Loki/Prometheus
│   ├── audio.py            # TTS/STT
│   ├── scraping.py         # Crawl4AI integration
│   └── ...
└── utils/                  # Utilities
    ├── docker_client.py
    └── errors.py

Dependencies

Troubleshooting

LLM calls failing

1. Check LiteLLM is running: docker ps | grep litellm
2. Verify API key: Check LITELLM_API_KEY environment variable
3. Check Langfuse for error traces: https://ai-ops.haiven.site

Models not listing

1. Verify LiteLLM connection: curl http://litellm:4000/v1/models
2. Check MCP server logs: docker logs mcp-server

Whisper transcription failing

1. Verify RTX PRO 6000 pro6000-alpha is available: nvidia-smi
2. Check model is loaded: Look for Whisper init in logs
3. Ensure audio format is supported (mp3, wav, m4a, flac, ogg, webm)

Connection refused errors

1. Check service is running: docker ps | grep mcp-server
2. Verify port binding: docker port mcp-server
3. Check Traefik routing: docker logs traefik 2>&1 | grep mcp

Sandbox execution timeouts

1. Default timeout is 30 seconds
2. Increase via SANDBOX_TIMEOUT environment variable
3. Check Docker socket permissions

Related Documentation

• LiteLLM Configuration
• Langfuse Dashboard
• API Documentation - Tier 3 OpenAPI spec
• Services Index