Supported Models and Setup

Use this guide to choose a supported model, complete the initial setup, and verify model connectivity.

The model configuration shown on this page consists of environment variables. Provide it according to how you use Midscene:

In Playground, paste the configuration text from this page directly into the settings page.
With the SDK or CLI, load the configuration as described in Set environment variables.

To understand the model roles, see Model strategy. For complete parameter definitions, see Model configuration reference.

Supported models

Midscene supports the following multimodal models for operating user interfaces. Each configuration requires a Base URL, API key, model name, and MIDSCENE_MODEL_FAMILY. The model family (MIDSCENE_MODEL_FAMILY) determines how Midscene adapts to the selected model.

Doubao Seed Series

Common model provider: Volcano Engine

Model version	Commonly used model names	`MIDSCENE_MODEL_FAMILY`	Notes
2.x series	`Doubao-Seed-2.1-turbo`, `Doubao-Seed-2.0-Lite`	`doubao-seed`	`Doubao-Seed-2.1-turbo` has the fastest localization speed and strong localization quality in our current private evaluation set. Recommended.
1.x series	`Doubao-Seed-1.6-Vision`, `Doubao-Seed-1.8`	`doubao-seed`	The 1.x series is an older generation of Doubao models and is no longer competitive overall. We recommend using the 2.x series instead. For compatibility with existing configurations, `MIDSCENE_MODEL_FAMILY="doubao-vision"` remains supported. New configurations should use `doubao-seed`.

Environment variable configuration example, using doubao-seed-2.1-turbo:

🎯 Use as default model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_MODEL_API_KEY="...."
MIDSCENE_MODEL_NAME="doubao-seed-2.1-turbo"
MIDSCENE_MODEL_FAMILY="doubao-seed"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="...."
MIDSCENE_PLANNING_MODEL_NAME="doubao-seed-2.1-turbo"
MIDSCENE_PLANNING_MODEL_FAMILY="doubao-seed"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="...."
MIDSCENE_INSIGHT_MODEL_NAME="doubao-seed-2.1-turbo"
MIDSCENE_INSIGHT_MODEL_FAMILY="doubao-seed"

If your Volcano Engine account has Fast Tier quota enabled, add the extra request body below to use it. This usually improves model response speed by about 30%-50%.

MIDSCENE_MODEL_EXTRA_BODY_JSON={"service_tier":"fast"}

Qwen Series

Common model provider: Alibaba Cloud

Model version	Commonly used model names	`MIDSCENE_MODEL_FAMILY`	Notes
Qwen3.x series	`qwen3.7-plus`, `qwen3.5-plus`, `qwen3.6-plus`	`qwen3`	Based on localization evaluation results, the recommended order is Qwen3.7 > Qwen3.5 > Qwen3.6. The previous `qwen3.5` and `qwen3.6` families remain compatible.
Qwen3-VL series	`qwen3-vl-plus`	`qwen3-vl`	As an older model generation, it is not recommended. Use the Qwen3.x series instead.
Qwen2.5-VL series	`qwen-vl-max-latest`	`qwen2.5-vl`	As an older model generation, it is not recommended. Use the Qwen3.x series instead.

Environment variable configuration example, using qwen3.7-plus:

🎯 Use as default model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="qwen3.7-plus"
MIDSCENE_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

MIDSCENE_PLANNING_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="qwen3.7-plus"
MIDSCENE_PLANNING_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="qwen3.7-plus"
MIDSCENE_INSIGHT_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

Google Gemini Series

Common model provider: Google Gemini

Model version	Commonly used model names	`MIDSCENE_MODEL_FAMILY`	Notes
Gemini 3.x series	`gemini-3.5-flash`, `gemini-3-flash-preview`	`gemini`	`gemini-3.5-flash` currently performs best for localization in our private evaluation set.

Environment variable configuration example, using gemini-3.5-flash:

🎯 Use as default model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_MODEL_FAMILY="gemini"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_PLANNING_MODEL_FAMILY="gemini"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_INSIGHT_MODEL_FAMILY="gemini"

OpenAI GPT Series

Common model provider: OpenAI

Model version	Commonly used model names	`MIDSCENE_MODEL_FAMILY`	Notes
GPT-5 series	`gpt-5.4`, `gpt-5.5`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`	`gpt-5`	Models before GPT-5.4 do not support visual localization and can only be used as Planning or Insight models. In practical localization tests, GPT-5.5 and GPT-5.6 perform noticeably better than GPT-5.4; we recommend using GPT-5.5 or GPT-5.6 first.

Environment variable configuration example, using gpt-5.5:

🎯 Use as default model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_MODEL_API_KEY="sk-..."
MIDSCENE_MODEL_NAME="gpt-5.5"
MIDSCENE_MODEL_FAMILY="gpt-5"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="sk-..."
MIDSCENE_PLANNING_MODEL_NAME="gpt-5.5"
MIDSCENE_PLANNING_MODEL_FAMILY="gpt-5"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="sk-..."
MIDSCENE_INSIGHT_MODEL_NAME="gpt-5.5"
MIDSCENE_INSIGHT_MODEL_FAMILY="gpt-5"

Use Codex App Server (OAuth, no API Key)

If you already signed in with Codex CLI (codex login) and want Midscene to use that OAuth session directly, set:

export MIDSCENE_MODEL_BASE_URL="codex://app-server"
export MIDSCENE_MODEL_NAME="gpt-5.4" # or another model shown by Codex model/list
export MIDSCENE_MODEL_FAMILY="gpt-5"

Notes:

MIDSCENE_MODEL_API_KEY is not required in this mode.
Midscene will call codex app-server through stdio.
Make sure codex is available in your PATH and verify auth with codex login status.

Known limitation

Compared with calling an OpenAI-compatible API directly, we have observed that this route may take longer and consume more tokens. We are still investigating the cause.

When using GPT-5, note the following:

For UI localization with GPT, Midscene currently supports gpt-5.4 and later models. To get the best localization quality, image requests need "detail": "original". According to OpenAI, this option is available on gpt-5.4 and future models, while smaller GPT-5 variants such as gpt-5.4-mini and gpt-5.4-nano, as well as older models, do not support original and will fail if you send it. See the Images and Vision guide and the Computer use guide.
According to OpenAI, GPT-5 may still struggle with non-Latin text and with text that is too small in the image. See the Images and Vision guide.
In the computer use guide, OpenAI says they observe good performance around 1440x900 and 1600x900. We recommend adjusting screenshot sizes accordingly. In Midscene, you can control screenshot compression with screenshotShrinkFactor in the agent options. For browser automation, you can also control the page size and scale through the browser viewport.
With Azure OpenAI, Azure may not handle "detail": "original" correctly, causing click-coordinate offsets. See Clicks are offset when using Azure OpenAI.
If you use an older GPT-5 model, we recommend using it only as the planning model and pairing it with another multimodal model for localization. See the multi-model combination example.

Moonshot Kimi Series

Common model provider: Moonshot AI platform

Model version	Commonly used model names	`MIDSCENE_MODEL_FAMILY`	Notes
K3 series	`kimi-k3`	`kimi3`	According to the Kimi documentation, K3 always has reasoning enabled and cannot be disabled. Its reasoning effort defaults to `max`.
K2.x series	`kimi-k2.5`, `kimi-k2.6`	`kimi`	—

Environment variable configuration example, using kimi-k3:

🎯 Use as default model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="kimi-k3"
MIDSCENE_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="kimi-k3"
MIDSCENE_PLANNING_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="kimi-k3"
MIDSCENE_INSIGHT_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

Xiaomi MiMo Series

Common model provider: Xiaomi MiMo API Open Platform

Model version	Commonly used model names	`MIDSCENE_MODEL_FAMILY`	Notes
V2.x series	`mimo-v2.5`	`xiaomi-mimo`	Only the Omni series supports multimodal input; the Pro series is text-only and cannot be used for Midscene visual tasks.

Environment variable configuration example, using mimo-v2.5:

🎯 Use as default model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="mimo-v2.5"
MIDSCENE_MODEL_FAMILY="xiaomi-mimo"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="mimo-v2.5"
MIDSCENE_PLANNING_MODEL_FAMILY="xiaomi-mimo"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="mimo-v2.5"
MIDSCENE_INSIGHT_MODEL_FAMILY="xiaomi-mimo"

Zhipu GLM-V Series

Common model providers: Z.AI (Global), BigModel (CN)

Model version	Commonly used model names	`MIDSCENE_MODEL_FAMILY`	Notes
GLM-5V series	`glm-5v-turbo`	`glm-v`	—
GLM-4.6 series	`glm-4.6v`	`glm-v`	`glm-4.6v` is open-source.

Environment variable configuration example, using glm-5v-turbo:

🎯 Use as default model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="glm-5v-turbo"
MIDSCENE_MODEL_FAMILY="glm-v"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="glm-5v-turbo"
MIDSCENE_PLANNING_MODEL_FAMILY="glm-v"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="glm-5v-turbo"
MIDSCENE_INSIGHT_MODEL_FAMILY="glm-v"

Learn more about the open-source GLM-4.6V model

Github: https://github.com/zai-org/GLM-V
Hugging Face: https://huggingface.co/zai-org/GLM-4.6V

Set environment variables

Midscene reads model configuration from environment variables. Choose the method that matches how you use Midscene, or keep using your project's existing environment-management approach.

Set environment variables in the current shell

The following example works in Bash and Zsh. These variables are available only to the current shell session and its child processes.

# Replace every value with the configuration for your selected model provider
export MIDSCENE_MODEL_BASE_URL="https://replace-with-your-model-service-url/v1"
export MIDSCENE_MODEL_API_KEY="replace-with-your-api-key"
export MIDSCENE_MODEL_NAME="replace-with-your-model-name"
export MIDSCENE_MODEL_FAMILY="replace-with-the-family-for-your-model"

Load `.env` with the CLI

Create a .env file in the directory where you run the midscene command. @midscene/cli loads this file automatically.

# Replace every value with the configuration for your selected model provider
MIDSCENE_MODEL_BASE_URL="https://replace-with-your-model-service-url/v1"
MIDSCENE_MODEL_API_KEY="replace-with-your-api-key"
MIDSCENE_MODEL_NAME="replace-with-your-model-name"
MIDSCENE_MODEL_FAMILY="replace-with-the-family-for-your-model"

Do not add export at the beginning of each line. When running YAML tasks, existing variables in the current shell take precedence over values with the same name in .env. To let .env override them, use --dotenv-override.

midscene model verify is an exception. This command uses values from .env to override variables with the same name in the current shell.

Load `.env` with dotenv for the JavaScript SDK

The Midscene JavaScript SDK reads model configuration directly from the Node.js process environment (process.env). If the shell, container, or deployment platform already provides these variables, you do not need dotenv. Use dotenv only when the configuration is stored in a .env file and needs to be loaded into process.env.

npm

yarn

pnpm

bun

deno

npm install dotenv

Create a .env file in the directory where you run the script:

# Replace every value with the configuration for your selected model provider
MIDSCENE_MODEL_BASE_URL="https://replace-with-your-model-service-url/v1"
MIDSCENE_MODEL_API_KEY="replace-with-your-api-key"
MIDSCENE_MODEL_NAME="replace-with-your-model-name"
MIDSCENE_MODEL_FAMILY="replace-with-the-family-for-your-model"

Import dotenv before creating a Midscene Agent:

import 'dotenv/config';

If a variable with the same name already exists in the current shell, dotenv keeps the existing value by default. The Midscene demo project uses this loading method as well.

Verify the setup

After setting the environment variables, run one of the following commands:

# Use the CLI installed in the current project
npx midscene model verify

# Or use the latest CLI version
npx @midscene/cli@latest model verify

For failed checks and connectivity problems, see Model debugging and observability.

Optional: configure multiple models

Multi-model setup is optional. For most use cases, a Default model is enough for UI localization and actions. Add a Planning or Insight model only when complex planning or page understanding requires a separate model. You can configure either one or both.

For guidance on when to combine models, see Model strategy.

The following example uses Qwen 3.5 as the Default model for visual grounding. GPT-5.4 serves as the Planning and Insight models for complex reasoning.

# Default multimodal model: Qwen 3.5
export MIDSCENE_MODEL_BASE_URL="https://..."       # Qwen 3.5 endpoint
export MIDSCENE_MODEL_API_KEY="..."                # Your Qwen 3.5 API key
export MIDSCENE_MODEL_NAME="qwen3.5-plus"
export MIDSCENE_MODEL_FAMILY="qwen3.5"

# Planning model: GPT-5.4
export MIDSCENE_PLANNING_MODEL_API_KEY="sk-..."    # Your GPT-5.4 API key
export MIDSCENE_PLANNING_MODEL_BASE_URL="https://..."
export MIDSCENE_PLANNING_MODEL_NAME="gpt-5.4"
export MIDSCENE_PLANNING_MODEL_FAMILY="gpt-5"

# Insight model: GPT-5.4
export MIDSCENE_INSIGHT_MODEL_API_KEY="sk-..."     # Your GPT-5.4 API key
export MIDSCENE_INSIGHT_MODEL_BASE_URL="https://..."
export MIDSCENE_INSIGHT_MODEL_NAME="gpt-5.4"
export MIDSCENE_INSIGHT_MODEL_FAMILY="gpt-5"

Other compatible models

The following smaller models are also compatible with Midscene and designed for automation. They require less deployment hardware but may struggle with complex tasks or large screenshots. Evaluate them against your tasks and deployment constraints before choosing one.

Zhipu AutoGLM Series

Zhipu AutoGLM is an open-source mobile UI automation model (9B parameters) from Zhipu AI.

After obtaining an API key from Z.AI (Global) or BigModel (CN), configure:

MIDSCENE_MODEL_BASE_URL="https://api.z.ai/api/paas/v4" # Or https://open.bigmodel.cn/api/paas/v4
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="autoglm-phone"
MIDSCENE_MODEL_FAMILY="auto-glm" # Or "auto-glm-multilingual"

About MIDSCENE_MODEL_FAMILY Configuration

AutoGLM provides two model versions, distinguished by MIDSCENE_MODEL_FAMILY:

auto-glm - Corresponds to AutoGLM-Phone-9B, optimized for Chinese mobile applications
auto-glm-multilingual - Corresponds to AutoGLM-Phone-9B-Multilingual, supports English and other languages

Choose the appropriate version based on your application language.

Info

AutoGLM is best suited for mobile interaction. APIs such as aiAssert and aiQuery require page understanding. When using these APIs, configure a separate Insight model with the MIDSCENE_INSIGHT_MODEL_... environment variables. See Model strategy for details.

Learn more about Zhipu AutoGLM

Github: https://github.com/zai-org/Open-AutoGLM
Hugging Face: https://huggingface.co/zai-org/AutoGLM-Phone-9B

UI-TARS Series

Use the deployed doubao-1.5-ui-tars on Volcano Engine:

MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
MIDSCENE_MODEL_API_KEY="...."
MIDSCENE_MODEL_NAME="ep-2025..." # Inference endpoint ID or model name from Volcano Engine
MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao-1.5"

About MIDSCENE_MODEL_FAMILY

This variable selects the UI-TARS version. Supported values:

vlm-ui-tars – for the 1.0 release
vlm-ui-tars-doubao – for the 1.5 release deployed on Volcano Engine (equivalent to vlm-ui-tars-doubao-1.5)
vlm-ui-tars-doubao-1.5 – for the 1.5 release deployed on Volcano Engine

Info

The legacy configurations MIDSCENE_USE_VLM_UI_TARS=DOUBAO or MIDSCENE_USE_VLM_UI_TARS=1.5 are still supported but deprecated. Please migrate to MIDSCENE_MODEL_FAMILY.

Migration mapping:

MIDSCENE_USE_VLM_UI_TARS=1.0 → MIDSCENE_MODEL_FAMILY="vlm-ui-tars"
MIDSCENE_USE_VLM_UI_TARS=1.5 → MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao-1.5"
MIDSCENE_USE_VLM_UI_TARS=DOUBAO → MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao"

Next steps

Learn when to use Default, Planning, and Insight models in Model strategy.
Look up every environment variable in Model configuration reference.
Diagnose connectivity and compatibility issues in Model debugging and observability.

#Supported Models and Setup

#Supported models

#Doubao Seed Series

#Qwen Series

#Google Gemini Series

#OpenAI GPT Series

#Moonshot Kimi Series

#Xiaomi MiMo Series

#Zhipu GLM-V Series

#Set environment variables

#Set environment variables in the current shell

#Load .env with the CLI

#Load .env with dotenv for the JavaScript SDK

#Verify the setup

#Optional: configure multiple models

#Other compatible models

#Zhipu AutoGLM Series

#UI-TARS Series

#Next steps

Supported Models and Setup

Supported models

Doubao Seed Series

Qwen Series

Google Gemini Series

OpenAI GPT Series

Moonshot Kimi Series

Xiaomi MiMo Series

Zhipu GLM-V Series

Set environment variables

Set environment variables in the current shell

Load `.env` with the CLI

Load `.env` with dotenv for the JavaScript SDK

Verify the setup

Optional: configure multiple models

Other compatible models

Zhipu AutoGLM Series

UI-TARS Series

Next steps