ML Cloud LLM
The ML Cloud offers API access to self-hosted open weight LLM(s). As this service is in the pilot phase, the following conditions apply:
- Access: A separate password is required to access the webUI.
- Cost: The service is currently free to use without limits.
- Model: Available model(s) are generally quantized versions in order to handle concurrency within allocated hardware.
Info
If you did not receive an email invite, or forgot the password, please raise a ticket at support@mlcloud.uni-tuebingen.de with [Local LLM] in the subject. There is no UI to reset the password yourself at the moment.
To integrate these models into your workflow, you can generate unique API keys. These keys allow you to easily configure AI agents harnesses (eg. Claude Code) or agentic frameworks (eg. LangChain) to use the local LLM as the primary reasoning engine, ensuring your data stays local.
Head on over to https://llm.mlcloud.uni-tuebingen.de to create your first virtual key.
Chat Interface
The ML Cloud does not offer a full fledged web interface to interact with the LLM(s). However, for quick, short chats that do not require tool use, the LiteLLM webUI does offer Model Playground. No API keys are required for this. Just make sure to select a model first.
On the top, there is also a Compare tab to compare responses against models.
For a local and more complete chat experience, check out Jan.ai or OpenWebUI. After installing it, open settings and set up a custom provider.
Base URL: "https://llm.mlcloud.uni-tuebingen.de/v1"
API KEY: "<VIRTUAL KEY>" # this typically starts with sk-
Agent Harness
For most agent harnesses that support OpenAI compatible endpoints, it is as easy as just setting a Base URL and and API key.
export OPENAI_BASE_URL="https://llm.mlcloud.uni-tuebingen.de/v1"
export OPENAI_API_KEY="<VIRTUAL KEY>" # this typically starts with sk-
Info
LiteLLM generally maintains docs on configuring popular AI tools and harnesses here.
Note
Most, if not all, agent harnesses support global and project/workspace level configs, the sections below only cover project level configuration.
Claude Code
For Claude Code, there are currently 2 ways to go about this:
Model Configuration
There are currently 2 ways to go about this:
Using settings.json [RECOMMENDED]
In your project root, create a .claude/settings.local.json (if it doesn't exist already) and type in following
{
"env": {
"ANTHROPIC_BASE_URL": "https://llm.mlcloud.uni-tuebingen.de",
"ANTHROPIC_AUTH_TOKEN": "<YOUR API KEY>",
"CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY":1, # OPTIONAL
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "Qwen/Qwen3.6-35B-A3B" # OPTIONAL
}
}
Then launch claude specifying the model
claude --model Qwen/Qwen3.6-35B-A3B # Replace with available model of choice
Setting environment variables
export ANTHROPIC_BASE_URL="https://llm.mlcloud.uni-tuebingen.de"
export ANTHROPIC_AUTH_TOKEN="< YOUR API KEY>"
# Then launch claude specifying the model
claude --model Qwen/Qwen3.6-35B-A3B # Replace with available model of choice
Info
For more detailed or uptodate configs, please refer the Claude Code docs
Web search
When using third party models or gateways with Claude Code, the default web search tool is not exposed to the model. This can be fixed by adding a suitable web search MCP (Model Context Protocol), such as Brave or Tavily - both offer free web searches per month (upto a limit) but require an API key.
For the examples below, Tavily is used, but the procedures are same for any MCP.
MCP: Claude install
Let Claude install the MCP of your choice:
claude mcp add tavily-remote-mcp --transport http https://mcp.tavily.com/mcp/
MCP: Manual
In your project root, create a .mcp.json file
{
"mcpServers": {
"tavily": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"TAVILY_API_KEY",
"mcp/tavily"
],
"env": {
"TAVILY_API_KEY": "<YOUR TAVILY API KEY>"
}
}
}
}
OpenCode (CLI)
Model Configuration
Manual
In your project root, create a opencode.jsonc file
{
"$schema": "https://opencode.ai/config.json",
// [OPTIONAL] Set a default model in the form of <PROVIDER ID>/<LITELLM MODEL NAME>
"model": "mlcloud/Qwen/Qwen3.6-35B-A3B",
"provider": {
// `mlcloud` is the provider ID used internally by Opencode, can be anything without spaces, special chars or uppercase
"mlcloud": {
"npm": "@ai-sdk/openai-compatible",
// Display name for the provider in the `/connect` menu. Rename as desired
"name": "ML Cloud",
"options": {
"baseURL": "https://llm.mlcloud.uni-tuebingen.de/v1",
"apiKey": "<YOUR API KEY>",
},
// Every new model needs to be manually added here
"models": {
"Qwen/Qwen3.6-35B-A3B": {
"name": "Qwen3.6 35B A3B"
},
// "<LITELLM MODEL NAME>" : {
// "name" : "<MODEL DISPLAY NAME>"
// }
},
},
},
"mcp": {}
}
Plugin
The opencode-models-discovery plugin can automatically fetch all available models for any OpenAI compatible endpoint.
In your project root, create a opencode.jsonc file
{
"$schema": "https://opencode.ai/config.json",
"plugin": ["opencode-models-discovery@latest"],
"provider": {
"mlcloud": {
"npm": "@ai-sdk/openai-compatible",
"name": "ML Cloud",
"options": {
"baseURL": "https://llm.mlcloud.uni-tuebingen.de/v1",
"apiKey": "<YOUR API KEY>",
"modelsDiscovery": {
"enabled": true
}
}
}
},
"mcp": {}
}
Websearch
While Opencode does have websearch builtin (via Exa), this is only available when using their subscription plan(s) or model(s). So, similar to the Claude Code section, pick a suitable MCP and add it the config.
Below is an example for Tavily MCP via docker
"mcp": {
"tavily": {
"enabled": true,
"type": "local",
// Do note that opencode has a slightly different structure to the MCP config format
"command": [
"docker",
"run",
"-i",
"--rm",
"-e",
"TAVILY_API_KEY",
"mcp/tavily"
],
"environment": {
"TAVILY_API_KEY": "<YOUR TAVILY API KEY>"
}
VSCode
Copilot chat can be configured without any additional extensions, follow the instruction in the VS Code docs here.
Complete Copilot functionality is not available to third party models at the moment (Tracked here: #308515, 318545).
There are open source alternatives such as ZooCode or KiloCode.
Created: July 6, 2026