Local LLM Server Manager

Appearance

Local LLM Server ManagerUnified Local AI Orchestrator

Control Ollama, Stable Diffusion Forge, ComfyUI, and Kokoro TTS with automated VRAM orchestration.

Get Started
View on GitHub
⚡

VRAM Orchestrator

Monitor GPU memory in real time. The system unloads inactive text models before heavy image, video, or 3D generation tasks.

🧩

Multi-Engine Support

Manage Ollama, Stable Diffusion Forge, ComfyUI, and Kokoro TTS from one unified desktop window or background service.

🎨

Multimodal Studio

Create images, 3D meshes, videos, and synthesized speech with interactive WebGL, audio waveform, and video player controls.

🤖

MCP Integration

Connect external AI coding assistants directly to your local hardware with fourteen official Model Context Protocol tools.

Overview ​

Local LLM Server Manager is a cross-platform orchestrator for local artificial intelligence engines. The application operates on Windows and Linux. The system coordinates Large Language Models, image generation, 3D mesh reconstruction, video synthesis, and speech generation.

Desktop Dashboard Overview

User Interface Layout ​

Layout SectionPrimary FunctionActive Elements
Telemetry Header & RibbonLive Hardware & Engine MonitoringGPU name, real-time VRAM allocation bar, interactive engine status cards (Ollama, ComfyUI, SD Forge, Kokoro TTS) with live status dots, tooltips, and click-to-manage toggles.
Activity RailWorkspace Domain NavigationPrimary navigation across Models (Local, Hugging Face, CivitAI), Multimodal Studio, Can I Run It (Hardware Fit), and Settings, plus Copilot and Documentation sidebar launcher buttons.
Fluid Studio Canvas & DockCreative Generation CenterModality selector bar (🎨 Image, 💬 Text, 🎬 Video, 🧊 3D Mesh, 🎙️ Audio), generous center stage preview canvas, style preset bar, and bottom creative prompt dock with parameters flyout (⚙️ Settings).
Docked Copilot & KnowledgeUnified Assistant & Guides SidebarRight-side docked sidebar hosting interactive Copilot AI chat (🤖 Copilot) and procedural documentation guides (📖 Knowledge) with seamless tab switching, magnetic companion pop-out, and contextual prompt transfer.
Companion WindowsFloating Multi-Window SystemDetachable Documentation and AI Assist windows with magnetic flank docking and lockstep tracking.

Core Capabilities ​

  • Automated VRAM Management: Prevent out-of-memory errors during heavy diffusion or 3D tasks.
  • Real Engine Test Flight: Verify engine network connectivity and inference pipelines with one click.
  • Model Discovery: Search and download models from Hugging Face Hub and CivitAI directly.
  • Unified Reverse Proxy: Route all AI engine traffic through a single port (5246).
  • Flexible Deployment: Run as a native Avalonia desktop application, system tray app, or headless background service.
  • Magnetic Companion Windows: Detach helper windows and dock them magnetically to the main application window.
  • Model Context Protocol: Give AI coding assistants secure control over your local AI infrastructure.

Documentation Sections ​

Explore the documentation guides to set up and operate your local AI stack:

Released under the MIT License.

Copyright © Spelech