Local LLM Server Manager

Appearance

Local LLM Server ManagerUnified Local AI Orchestrator

Control Ollama, Stable Diffusion Forge, ComfyUI, and Kokoro TTS with automated VRAM orchestration.

Get Started
View on GitHub
⚡

VRAM Orchestrator

Monitor GPU memory in real time. The system unloads inactive text models before heavy image, video, or 3D generation tasks.

🧩

Multi-Engine Support

Manage Ollama, Stable Diffusion Forge, ComfyUI, and Kokoro TTS from one unified desktop window or background service.

🎨

Multimodal Studio

Create images, 3D meshes, videos, and synthesized speech with interactive WebGL, audio waveform, and video player controls.

🤖

MCP Integration

Connect external AI coding assistants directly to your local hardware with eleven official Model Context Protocol tools.

Overview ​

Local LLM Server Manager is a cross-platform orchestrator for local artificial intelligence engines. The application operates on Windows and Linux. The system coordinates Large Language Models, image generation, 3D mesh reconstruction, video synthesis, and speech generation.

Desktop Dashboard Overview

User Interface Layout ​

Layout SectionPrimary FunctionActive Elements
Telemetry HeaderLive Hardware MonitoringGPU name, total/used VRAM bar, service health indicator, and refresh button.
Primary NavigationWorkspace SwitcherTabs for My Models, Hugging Face Hub, CivitAI, Multimodal Studio, AI Assistant, and Settings.
Main Content CanvasEngine InteractionModel cards, KV cache context calculator, download progress, WebGL canvas, and video player.
Companion WindowsFloating Multi-WindowDetachable Documentation and AI Assist windows with magnetic flank docking.

Core Capabilities ​

  • Automated VRAM Management: Prevent out-of-memory errors during heavy diffusion or 3D tasks.
  • Real Engine Test Flight: Verify engine network connectivity and inference pipelines with one click.
  • Model Discovery: Search and download models from Hugging Face Hub and CivitAI directly.
  • Unified Reverse Proxy: Route all AI engine traffic through a single port (5246).
  • Flexible Deployment: Run as a native Avalonia desktop application, system tray app, or headless background service.
  • Magnetic Companion Windows: Detach helper windows and dock them magnetically to the main application window.
  • Model Context Protocol: Give AI coding assistants secure control over your local AI infrastructure.

Documentation Sections ​

Explore the documentation guides to set up and operate your local AI stack:

Released under the MIT License.

Copyright © Spelech