Use a model API key

Connect Midscene using an API key from a model provider. This guide covers model configuration and connection verification. If you already have a ChatGPT subscription, you can also follow Use a ChatGPT subscription.

The model configuration shown on this page consists of environment variables. Provide it according to how you use Midscene:

In Playground, paste the configuration text from this page directly into the settings page.
With the SDK or CLI, load the configuration as described in Set environment variables.

For advanced parameter definitions, see Model advanced configuration and debugging.

Supported models

For an overview of supported models and connection methods, see the Model Selection Guide. Configuration examples for each model follow.

Each configuration requires a Base URL, API key, model name, and MIDSCENE_MODEL_FAMILY. The model family (MIDSCENE_MODEL_FAMILY) selects the model adapter, which determines coordinate handling, supported API protocols, and how reasoning and output parameters are sent. Use the family listed for your model below.

Doubao Seed Series

Doubao-Seed-2.1 Turbo

View configurationHide configuration

Common model provider: Volcano Engine

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
2.x series `doubao-seed-2-1-pro-260915` `doubao-seed-2-1-turbo-260628`	`doubao-seed`	—
1.x series `Doubao-Seed-1.6-Vision` `Doubao-Seed-1.8`	`doubao-seed`	For compatibility with existing configurations, `MIDSCENE_MODEL_FAMILY="doubao-vision"` remains supported. New configurations should use `doubao-seed`.

Environment variable configuration example, using doubao-seed-2-1-turbo-260628:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="doubao-seed-2-1-turbo-260628"
MIDSCENE_MODEL_FAMILY="doubao-seed"

MIDSCENE_LOCATE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="doubao-seed-2-1-turbo-260628"
MIDSCENE_LOCATE_MODEL_FAMILY="doubao-seed"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="doubao-seed-2-1-turbo-260628"
MIDSCENE_PLANNING_MODEL_FAMILY="doubao-seed"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3" # Volcano Engine endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="doubao-seed-2-1-turbo-260628"
MIDSCENE_INSIGHT_MODEL_FAMILY="doubao-seed"

If your Volcano Engine account has Fast Tier quota enabled, add the extra request body below to use it. This usually improves model response speed by about 30%-50%.

MIDSCENE_MODEL_EXTRA_BODY_JSON={"service_tier":"fast"}

Qwen Series

Qwen3.7 Plus

View configurationHide configuration

Common model provider: Alibaba Cloud

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
Qwen3.x series `qwen3.7-plus` `qwen3.5-plus` `qwen3.6-plus`	`qwen3`	The previous `qwen3.5` and `qwen3.6` families remain compatible.
Qwen3-VL series `qwen3-vl-plus`	`qwen3-vl`	Older generation.
Qwen2.5-VL series `qwen-vl-max-latest`	`qwen2.5-vl`	Older generation.

Environment variable configuration example, using qwen3.7-plus:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="qwen3.7-plus"
MIDSCENE_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

MIDSCENE_LOCATE_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="qwen3.7-plus"
MIDSCENE_LOCATE_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

MIDSCENE_PLANNING_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="qwen3.7-plus"
MIDSCENE_PLANNING_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1" # Alibaba Cloud endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="qwen3.7-plus"
MIDSCENE_INSIGHT_MODEL_FAMILY="qwen3" # If you use another Qwen version, replace this with the corresponding model family

DeepSeek Series

DeepSeek V4.1 Flash

View configurationHide configuration

Midscene supports DeepSeek starting from v1.12.0.

Common model provider: DeepSeek - Vision

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
DeepSeek V4 series `deepseek-flash` (V4.1 Flash) `deepseek-v4-flash-vision-exp` (V4 Flash Vision)	`deepseek`	DeepSeek-V4-Pro does not support visual input and cannot be used with Midscene.

Environment variable configuration example, using deepseek-flash:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://api.deepseek.com" # DeepSeek API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="deepseek-flash"
MIDSCENE_MODEL_FAMILY="deepseek"

MIDSCENE_LOCATE_MODEL_BASE_URL="https://api.deepseek.com" # DeepSeek API endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="deepseek-flash"
MIDSCENE_LOCATE_MODEL_FAMILY="deepseek"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.deepseek.com" # DeepSeek API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="deepseek-flash"
MIDSCENE_PLANNING_MODEL_FAMILY="deepseek"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.deepseek.com" # DeepSeek API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="deepseek-flash"
MIDSCENE_INSIGHT_MODEL_FAMILY="deepseek"

Google Gemini Series

Gemini 3.5 Flash

View configurationHide configuration

Common model provider: Google Gemini

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
Gemini 3.x series `gemini-3.5-flash` `gemini-3-flash-preview`	`gemini`	—

Environment variable configuration example, using gemini-3.5-flash:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_MODEL_FAMILY="gemini"

MIDSCENE_LOCATE_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_LOCATE_MODEL_FAMILY="gemini"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_PLANNING_MODEL_FAMILY="gemini"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/" # Google Gemini API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="gemini-3.5-flash"
MIDSCENE_INSIGHT_MODEL_FAMILY="gemini"

OpenAI GPT Series

GPT-6 Sol

View configurationHide configuration

Info

To connect using your ChatGPT subscription, see Use a ChatGPT subscription.

Common model provider: OpenAI

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
GPT-6 series `gpt-6-astra` `gpt-6-sol` `gpt-6-luna`	`gpt-6`	—
GPT-5 series `gpt-5.4` `gpt-5.5` `gpt-5.6-sol` `gpt-5.6-terra` `gpt-5.6-luna`	`gpt-5`	Models before GPT-5.4 do not support visual localization and can only be used as Planning or Insight models.

Use an API Key

Environment variable configuration example, using gpt-6-sol:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="gpt-6-sol"
MIDSCENE_MODEL_FAMILY="gpt-6"

MIDSCENE_LOCATE_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="gpt-6-sol"
MIDSCENE_LOCATE_MODEL_FAMILY="gpt-6"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="gpt-6-sol"
MIDSCENE_PLANNING_MODEL_FAMILY="gpt-6"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.openai.com/v1" # OpenAI API endpoint; or your compatible endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="gpt-6-sol"
MIDSCENE_INSIGHT_MODEL_FAMILY="gpt-6"

Enable Fast mode (service_tier="fast")

Both GPT-5.6 Sol and GPT-6 Astra support Fast mode. Add the service_tier request body parameter to enable it. For example, with GPT-5.6 Sol:

MIDSCENE_MODEL_NAME="gpt-5.6-sol"
MIDSCENE_MODEL_FAMILY="gpt-5"
MIDSCENE_MODEL_EXTRA_BODY_JSON='{"service_tier":"fast"}'

According to OpenAI, Fast mode can deliver up to 2–2.5× the speed at twice the Standard rate. Refer to the official documentation for current speed, pricing, and usage restrictions.

To reuse a Codex CLI session, see Use Codex App Server.

When using GPT, note the following:

GPT-5.4 or later is required for UI localization. Midscene does not support earlier versions.
According to OpenAI, GPT-5 may still struggle with non-Latin text and with text that is too small in the image. See the Images and Vision guide.
In the computer use guide, OpenAI says they observe good performance around 1440x900 and 1600x900. We recommend adjusting screenshot sizes accordingly. In Midscene, you can control screenshot compression with screenshotShrinkFactor in the agent options. For browser automation, you can also control the page size and scale through the browser viewport.
Azure OpenAI's Chat Completions API may not handle "detail": "original" correctly, causing click-coordinate offsets. If you encounter this issue, you can switch to the Responses API. See Clicks are offset when using Azure OpenAI.

Moonshot Kimi Series

Kimi K3

View configurationHide configuration

Common model provider: Moonshot AI platform

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
K3 series `kimi-k3`	`kimi3`	According to the Kimi documentation, K3 always has reasoning enabled and cannot be disabled. Its reasoning effort defaults to `max`.
K2.x series `kimi-k2.5` `kimi-k2.6`	`kimi`	—

Environment variable configuration example, using kimi-k3:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="kimi-k3"
MIDSCENE_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

MIDSCENE_LOCATE_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="kimi-k3"
MIDSCENE_LOCATE_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="kimi-k3"
MIDSCENE_PLANNING_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.moonshot.cn/v1" # Moonshot AI API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="kimi-k3"
MIDSCENE_INSIGHT_MODEL_FAMILY="kimi3" # For kimi-k2.6, use "kimi"

Xiaomi MiMo Series

MiMo V2.6 Pro

View configurationHide configuration

Common model provider: Xiaomi MiMo API Open Platform

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
V2.x series `mimo-v2.6-pro` `mimo-v2.6-flash` `mimo-v2.6-pro-ultraspeed` `mimo-v2.5`	`xiaomi-mimo`	All 2.6 models support multimodal input; for 2.5, only the Omni series supports multimodal input.

Environment variable configuration example, using mimo-v2.6-pro:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="mimo-v2.6-pro"
MIDSCENE_MODEL_FAMILY="xiaomi-mimo"

MIDSCENE_LOCATE_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="mimo-v2.6-pro"
MIDSCENE_LOCATE_MODEL_FAMILY="xiaomi-mimo"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="mimo-v2.6-pro"
MIDSCENE_PLANNING_MODEL_FAMILY="xiaomi-mimo"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.xiaomimimo.com/v1" # Xiaomi MiMo API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="mimo-v2.6-pro"
MIDSCENE_INSIGHT_MODEL_FAMILY="xiaomi-mimo"

Zhipu GLM-V Series

GLM-5V-Turbo

View configurationHide configuration

Common model providers: Z.AI (Global), BigModel (CN)

Model `MIDSCENE_MODEL_NAME`	`MIDSCENE_MODEL_FAMILY`	Notes
GLM-5V series `glm-5v-turbo`	`glm-v`	—
GLM-4.6 series `glm-4.6v`	`glm-v`	`glm-4.6v` is open-source.

Environment variable configuration example, using glm-5v-turbo:

🤖 Use as default model

🎯 Use as Locate model

🧠 Use as Planning model

🔎 Use as Insight model

MIDSCENE_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="glm-5v-turbo"
MIDSCENE_MODEL_FAMILY="glm-v"

MIDSCENE_LOCATE_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="glm-5v-turbo"
MIDSCENE_LOCATE_MODEL_FAMILY="glm-v"

MIDSCENE_PLANNING_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="glm-5v-turbo"
MIDSCENE_PLANNING_MODEL_FAMILY="glm-v"

MIDSCENE_INSIGHT_MODEL_BASE_URL="https://open.bigmodel.cn/api/paas/v4" # BigModel API endpoint; use https://api.z.ai/api/paas/v4 for Z.AI
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="glm-5v-turbo"
MIDSCENE_INSIGHT_MODEL_FAMILY="glm-v"

Learn more about the open-source GLM-4.6V model

Github: https://github.com/zai-org/GLM-V
Hugging Face: https://huggingface.co/zai-org/GLM-4.6V

Jev (experimental)

Jev 1.14.0

View configurationHide configuration

Jev is a model for making choices. You can use it to select the next action, locate UI elements, and evaluate whether a condition holds. It is fast and inexpensive, but is not suitable for tasks that require complex reasoning.

Midscene support for Jev is experimental.

Model provider: TypeSafe

First configure a general-purpose default model (MIDSCENE_MODEL_*) and keep that configuration, then add the Jev configuration below. Jev handles Planning, Locate, and Insight; the general-purpose default model still handles tasks Jev cannot perform. See Multiple models for default model configuration.

Copy the following configuration to use Jev for Planning, Locate, and Insight:

# Planning
MIDSCENE_PLANNING_MODEL_BASE_URL="https://api.typesafe.ai/v1" # TypeSafe official API endpoint
MIDSCENE_PLANNING_MODEL_API_KEY="......"
MIDSCENE_PLANNING_MODEL_NAME="jev-1.14.0"
MIDSCENE_PLANNING_MODEL_FAMILY="jev"

# Locate
MIDSCENE_LOCATE_MODEL_BASE_URL="https://api.typesafe.ai/v1" # TypeSafe official API endpoint
MIDSCENE_LOCATE_MODEL_API_KEY="......"
MIDSCENE_LOCATE_MODEL_NAME="jev-1.14.0"
MIDSCENE_LOCATE_MODEL_FAMILY="jev"

# Insight
MIDSCENE_INSIGHT_MODEL_BASE_URL="https://api.typesafe.ai/v1" # TypeSafe official API endpoint
MIDSCENE_INSIGHT_MODEL_API_KEY="......"
MIDSCENE_INSIGHT_MODEL_NAME="jev-1.14.0"
MIDSCENE_INSIGHT_MODEL_FAMILY="jev"

When the corresponding model role is configured to use Jev, tasks are handled as follows:

Locate: Jev performs element localization.
Insight: Jev evaluates aiBoolean, aiAssert, and aiWaitFor. The general-purpose default model handles aiQuery, aiString, aiNumber, and aiAsk.
Planning: In aiAct, Jev plans simple actions such as clicking, double-clicking, and going back. The default model plans complex actions such as input and scrolling.

Model selection is based on the capabilities required by each task. Failed Jev requests do not automatically switch to the default model.

Jev uses TypeSafe's System One protocol, not an OpenAI protocol. Some OpenAI model settings do not apply, such as temperature, native reasoning, response format, and streaming output.

Notes

Base URL: The Base URL must include /v1, for example https://api.typesafe.ai/v1.
Image input limits: The latest version of Jev currently limits the longest edge of an image to 1568 pixels. Use the Agent's screenshotShrinkFactor option to control screenshot dimensions. To avoid image quality degradation from repeated resizing and accommodate image dimension requirements in future Jev versions, Midscene does not automatically resize images to match the limits of a specific Jev version.

Set environment variables

Midscene reads model configuration from environment variables. Choose the method that matches how you use Midscene, or keep using your project's existing environment-management approach.

Set environment variables in the current shell

The following example works in Bash and Zsh. These variables are available only to the current shell session and its child processes.

# Replace every value with the configuration for your selected model provider
export MIDSCENE_MODEL_BASE_URL="https://replace-with-your-model-service-url/v1"
export MIDSCENE_MODEL_API_KEY="replace-with-your-api-key"
export MIDSCENE_MODEL_NAME="replace-with-your-model-name"
export MIDSCENE_MODEL_FAMILY="replace-with-the-family-for-your-model"

Configure through a `.env` file

Midscene Test (@midscene/test) and the legacy YAML script runner (@midscene/cli) automatically load configuration from .env.

For Midscene Test, copy .env.example to .env in the generated test project directory, fill in the model configuration, then run pnpm test. See Create and use test projects for details. For the legacy YAML script runner, create .env in the directory where you run midscene.

Example .env contents:

# Replace every value with the configuration for your selected model provider
MIDSCENE_MODEL_BASE_URL="https://replace-with-your-model-service-url/v1"
MIDSCENE_MODEL_API_KEY="replace-with-your-api-key"
MIDSCENE_MODEL_NAME="replace-with-your-model-name"
MIDSCENE_MODEL_FAMILY="replace-with-the-family-for-your-model"

Do not add export at the beginning of each line. With the legacy YAML script runner, existing shell variables take precedence over matching values in .env; use --dotenv-override to prefer values from the file.

JavaScript SDK

The Midscene SDK reads model configuration from process.env. When writing test scripts, you can install dotenv in your own test project, store model configuration in a .env file at the project root, and add import 'dotenv/config' to your script entry point. Run the script from the project root and load the configuration before creating an Agent.

Other compatible models

The following smaller models are also compatible with Midscene and designed for automation. They require less deployment hardware but may struggle with complex tasks or large screenshots. Evaluate them against your tasks and deployment constraints before choosing one.

Zhipu AutoGLM Series

AutoGLM-Phone-9B

View configurationHide configuration

Zhipu AutoGLM is an open-source mobile UI automation model (9B parameters) from Zhipu AI.

After obtaining an API key from Z.AI (Global) or BigModel (CN), configure:

MIDSCENE_MODEL_BASE_URL="https://api.z.ai/api/paas/v4" # Or https://open.bigmodel.cn/api/paas/v4
MIDSCENE_MODEL_API_KEY="......"
MIDSCENE_MODEL_NAME="autoglm-phone"
MIDSCENE_MODEL_FAMILY="auto-glm" # Or "auto-glm-multilingual"

About MIDSCENE_MODEL_FAMILY Configuration

AutoGLM provides two model versions, distinguished by MIDSCENE_MODEL_FAMILY:

auto-glm - Corresponds to AutoGLM-Phone-9B, optimized for Chinese mobile applications
auto-glm-multilingual - Corresponds to AutoGLM-Phone-9B-Multilingual, supports English and other languages

Choose the appropriate version based on your application language.

Info

AutoGLM is best suited for mobile interaction. APIs such as aiAssert and aiQuery require page understanding. When using these APIs, configure a separate Insight model with the MIDSCENE_INSIGHT_MODEL_... environment variables. See Model Selection Guide for details.

Learn more about Zhipu AutoGLM

Github: https://github.com/zai-org/Open-AutoGLM
Hugging Face: https://huggingface.co/zai-org/AutoGLM-Phone-9B

UI-TARS Series

UI-TARS 1.5

View configurationHide configuration

Use the deployed doubao-1.5-ui-tars on Volcano Engine:

MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
MIDSCENE_MODEL_API_KEY="...."
MIDSCENE_MODEL_NAME="ep-2025..." # Inference endpoint ID or model name from Volcano Engine
MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao-1.5"

About MIDSCENE_MODEL_FAMILY

This variable selects the UI-TARS version. Supported values:

vlm-ui-tars – for the 1.0 release
vlm-ui-tars-doubao – for the 1.5 release deployed on Volcano Engine (equivalent to vlm-ui-tars-doubao-1.5)
vlm-ui-tars-doubao-1.5 – for the 1.5 release deployed on Volcano Engine

Info

The legacy configurations MIDSCENE_USE_VLM_UI_TARS=DOUBAO or MIDSCENE_USE_VLM_UI_TARS=1.5 are still supported but deprecated. Please migrate to MIDSCENE_MODEL_FAMILY.

Migration mapping:

MIDSCENE_USE_VLM_UI_TARS=1.0 → MIDSCENE_MODEL_FAMILY="vlm-ui-tars"
MIDSCENE_USE_VLM_UI_TARS=1.5 → MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao-1.5"
MIDSCENE_USE_VLM_UI_TARS=DOUBAO → MIDSCENE_MODEL_FAMILY="vlm-ui-tars-doubao"

Next steps

Verify model connectivity — Check that your configured models work with Midscene.
Configure models by intent — Use dedicated models for planning, localization, or page understanding, with an expandable configuration example.
Common configuration errors — Resolve model family, endpoint, or model name errors.
When model results are unreliable — Investigate incorrect actions, missed targets, or misinterpreted pages.

#Use a model API key

#Supported models

#Doubao Seed Series

#Qwen Series

#DeepSeek Series

#Google Gemini Series

#OpenAI GPT Series

#Moonshot Kimi Series

#Xiaomi MiMo Series

#Zhipu GLM-V Series

#Jev (experimental)

#Set environment variables

#Set environment variables in the current shell

#Configure through a .env file

#Other compatible models

#Zhipu AutoGLM Series

#UI-TARS Series

#Next steps

Use a model API key

Supported models

Doubao Seed Series

Qwen Series

DeepSeek Series

Google Gemini Series

OpenAI GPT Series

Moonshot Kimi Series

Xiaomi MiMo Series

Zhipu GLM-V Series

Jev (experimental)

Set environment variables

Set environment variables in the current shell

Configure through a `.env` file

Other compatible models

Zhipu AutoGLM Series

UI-TARS Series

Next steps