Skip to content

ML Cloud LLM

Pilot

The ML Cloud offers API access to self-hosted open weight LLM(s). As this service is in the pilot phase, the following conditions apply:

  • Access: A separate password is required to access the webUI.
  • Cost: The service is currently free to use without limits.
  • Model: Available model(s) are generally quantized versions in order to handle concurrency within allocated hardware.

Info

If you did not receive an email invite, or forgot the password, please raise a ticket at support@mlcloud.uni-tuebingen.de with [Local LLM] in the subject. There is no UI to reset the password yourself at the moment

To integrate these models into your workflow, you can generate unique API keys. These keys allow you to easily configure AI agents harnesses (eg. Claude Code) or agentic frameworks (eg. LangChain) to use the local LLM as the primary reasoning engine, ensuring your data stays local.

🚀 Head on over to https://llm.mlcloud.uni-tuebingen.de to create your first virtual key.

Chat Interface

The ML Cloud does not offer a full fledged web interface to interact with the LLM(s). However, for quick, short chats that do not require tool use, the LiteLLM webUI does offer Model Playground. No API keys are required for this. Just make sure to select a model first.

On the top, there is also a Compare tab to compare responses against models.

LiteLLM Chat interface

🚀 Check it out now

For a local and more complete chat experience, check out Jan.ai or OpenWebUI. After installing it, open settings and set up a custom provider.

Base URL: "https://llm.mlcloud.uni-tuebingen.de/v1"
API KEY: "<VIRTUAL KEY>" # this typically starts with sk-

Agent Harness

For most agent harnesses that support OpenAI compatible endpoints, it is as easy as just setting a Base URL and and API key.

export OPENAI_BASE_URL="https://llm.mlcloud.uni-tuebingen.de/v1"
export OPENAI_API_KEY="<VIRTUAL KEY>" # this typically starts with sk-

Info

LiteLLM generally maintains docs on configuring popular AI tools and harnesses here

For enabling web search, please refer here.

Claude Code

For Claude Code, there are currently 2 ways to go about this:

Model Configuration

There are currently 2 ways to go about this:

In your project root, create a .claude/settings.local.json (if it doesn't exist already) and type in following

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://llm.mlcloud.uni-tuebingen.de",
    "ANTHROPIC_AUTH_TOKEN": "<YOUR API KEY>",
    "CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY":1, # OPTIONAL
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "Qwen/Qwen3.6-35B-A3B" # OPTIONAL
  }
}

Then launch claude specifying the model 

claude --model Qwen/Qwen3.6-35B-A3B # Replace with available model of choice

Setting environment variables
export ANTHROPIC_BASE_URL="https://llm.mlcloud.uni-tuebingen.de"
export ANTHROPIC_AUTH_TOKEN="< YOUR API KEY>"

# Then launch claude specifying the model
claude --model Qwen/Qwen3.6-35B-A3B # Replace with available model of choice

Info

For more detailed or uptodate configs, please refer the Claude Code docs

When using third party models or gateways with Claude Code, the default web search tool is not exposed to the model. This can be fixed by adding a suitable web search MCP (Model Context Protocol), such as Brave or Tavily - both offer free web searches per month (upto a limit) but require an API key.

For the examples below, Tavily is used, but the procedures are same for any MCP.

MCP: Claude Code

Let Claude install the MCP of your choice:

claude mcp add tavily-remote-mcp --transport http https://mcp.tavily.com/mcp/
MCP: Manual

In your project root, create a .mcp.json file

{
    "mcpServers": {
        "tavily": {
            "command": "docker",
            "args": [
                "run",
                "-i",
                "--rm",
                "-e",
                "TAVILY_API_KEY",
                "mcp/tavily"
            ],
            "env": {
                "TAVILY_API_KEY": "<YOUR TAVILY API KEY>"
            }
        }
    }
}

OpenCode

COMING SOON

VSCode

Complete Copilot functionality is not available to third party models at the moment (Tracked here: #308515, 318545).

There are open source alternatives such as ZooCode or KiloCode.

Last update: July 6, 2026
Created: July 6, 2026