#Model advanced configuration and debugging

This page covers advanced model settings, connection troubleshooting, logs, and tracing.

#Verify model connectivity

Check whether your configured models can connect and work with Midscene:

pnpm dlx @midscene/test@latest model verify

The command uses your environment variables and optionally loads .env from the current directory. Values in .env override matching environment variables.

Example success output (excerpt)

Model role  Check                Result  Requests  Time
planning    Text response        PASS    1         820 ms
insight     Image understanding  PASS    1         1250 ms
locate      AI locate            PASS    1         1460 ms
default     Text response        PASS    1         760 ms

Requests sent: 4. Checks: 4 passed, 0 failed.
✅ Model verify passed.

Example failure output (excerpt)

Requests sent: 4. Checks: 3 passed, 1 failed.
❌ Model verify failed with messages:
────────────────────────────────────────

[Vision check - ep-vision (insight)]: 404 Not Found

────────────────────────────────────────

MIDSCENE_MODEL_BASE_URL='replace with your baseUrl'
MIDSCENE_MODEL_API_KEY='replace with your API key'
MIDSCENE_MODEL_NAME='replace with your model name'
# MIDSCENE_MODEL_PROTOCOL="openai-responses" # Uncomment this line to use Responses API; defaults to Chat API.

if [ "$MIDSCENE_MODEL_PROTOCOL" = "openai-responses" ]; then
  model_endpoint="responses"
  model_input_field="input"
else
  model_endpoint="chat/completions"
  model_input_field="messages"
fi

curl -X POST "${MIDSCENE_MODEL_BASE_URL%/}/${model_endpoint}" \
  -H "Authorization: Bearer ${MIDSCENE_MODEL_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "'"${MIDSCENE_MODEL_NAME}"'",
  "'"${model_input_field}"'": [
    {
      "role": "user",
      "content": "What is 1+1?"
    }
  ]
}'

#Model configuration by intent

One Default model is sufficient for most tasks. When a task needs stronger planning, more precise localization, or better page understanding, configure a dedicated model for that intent to use a model better suited to that capability.

You can configure separate models for Planning, Locate, and Insight intents. These settings use the same parameter suffixes and meanings as the Default model settings below. Replace the prefix for each role:

Provide a complete configuration for each dedicated model, including its endpoint, API key (if required by the connection method), model name, and family, along with any optional settings you need.

A role without a dedicated model uses the Default model.

# Default model
export MIDSCENE_MODEL_BASE_URL="https://..."
export MIDSCENE_MODEL_API_KEY="..."
export MIDSCENE_MODEL_NAME="qwen3.8-flash"
export MIDSCENE_MODEL_FAMILY="qwen3"

# Planning model
export MIDSCENE_PLANNING_MODEL_BASE_URL="https://..."
export MIDSCENE_PLANNING_MODEL_API_KEY="..."
export MIDSCENE_PLANNING_MODEL_NAME="gpt-6-astra"
export MIDSCENE_PLANNING_MODEL_FAMILY="gpt-6"

# Locate model
export MIDSCENE_LOCATE_MODEL_BASE_URL="https://..."
export MIDSCENE_LOCATE_MODEL_API_KEY="..."
export MIDSCENE_LOCATE_MODEL_NAME="gemini-3.5-flash"
export MIDSCENE_LOCATE_MODEL_FAMILY="gemini"

# Insight model
export MIDSCENE_INSIGHT_MODEL_BASE_URL="https://..."
export MIDSCENE_INSIGHT_MODEL_API_KEY="..."
export MIDSCENE_INSIGHT_MODEL_NAME="gpt-6.1-sol"
export MIDSCENE_INSIGHT_MODEL_FAMILY="gpt-6"

#Advanced settings (optional)

#Configuration reference

To configure a separate model for Planning, Locate, or Insight, use the corresponding environment variable prefix. See Model configuration by intent.

#Use the Responses API

Midscene uses Chat Completions (openai-chat) by default. Among the built-in model adapters, only gpt-5 and gpt-6 currently support the Responses API. Setting openai-responses for another model family causes an unsupported-protocol error.

For the official OpenAI endpoint, add the following to your GPT model configuration:

MIDSCENE_MODEL_BASE_URL="https://api.openai.com/v1"
MIDSCENE_MODEL_PROTOCOL="openai-responses"

Set the model name, API key, and family as shown in the GPT configuration example. Do not append /responses to the Base URL; the SDK adds it automatically.

The Responses API is not restricted to the official OpenAI endpoint. Azure OpenAI and other compatible endpoints can also be used, but both the endpoint and the selected GPT model must support this protocol. Chat Completions compatibility alone does not imply Responses compatibility. See the Azure OpenAI FAQ for configuration and the verified coordinate-offset scenario.

For a dedicated GPT model, use MIDSCENE_PLANNING_MODEL_PROTOCOL, MIDSCENE_LOCATE_MODEL_PROTOCOL, or MIDSCENE_INSIGHT_MODEL_PROTOCOL for the corresponding role. Each model's protocol is configured independently and does not affect the others.

#Model-native thinking

Midscene disables model-native thinking by default for better execution speed and stability. If a model cannot disable native thinking, Midscene reduces it as much as possible by controlling the thinking granularity or budget.

The following environment variables provide a unified Midscene abstraction over provider-specific parameters. The parameters sent to the model depend on MIDSCENE_MODEL_FAMILY.

MIDSCENE_MODEL_REASONING_ENABLED explicitly controls whether model-native thinking is enabled:

• false: Force-disable model-native thinking. This is the Midscene default.
• true: Force-enable model-native thinking.
• default: Follow the model's default behavior. Midscene does not send a parameter that enables or disables thinking, and it ignores explicit MIDSCENE_MODEL_REASONING_BUDGET and MIDSCENE_MODEL_REASONING_EFFORT settings.

Provider support and accepted values vary. See each provider's official documentation for supported model versions and values. If the current model does not support an explicit setting, Midscene ignores it instead of guessing a provider-specific private parameter.

#Troubleshooting

#Common configuration errors

MIDSCENE_MODEL_FAMILY is not set to a multimodal model

If you see MIDSCENE_MODEL_FAMILY is not set to a multimodal model with UI localization, make sure the MIDSCENE_MODEL_FAMILY environment variable for the multimodal model is set correctly.

Starting with version 1.0, Midscene recommends using MIDSCENE_MODEL_FAMILY to specify the multimodal model type. Legacy MIDSCENE_USE_... settings remain compatible but are deprecated.

See Use a model API key for the correct model family and a complete configuration example.

Base URL or model name is incorrect

Confirm that MIDSCENE_MODEL_BASE_URL points to the provider's API endpoint. It commonly ends with a version such as /v1. Do not append /chat/completions or /responses; the underlying SDK adds the request path based on the selected API type.

Also confirm that MIDSCENE_MODEL_NAME matches a model available from that endpoint.

#When model results are unreliable

If the model connects successfully but produces incorrect actions, misses targets, or misinterprets the page:

• Check the replay report to identify where the task went wrong.
• Try a newer supported model version or compare models on representative tasks.
• If failures concentrate in planning, localization, or page understanding, consider a dedicated model for that intent.

See the Model Selection Guide for model choices and role descriptions.

#Debugging capabilities

#Debug logs

Set DEBUG when you need additional diagnostic output. Common selectors include:

• DEBUG=midscene:ai:profile:stats prints model latency and Token usage.
• DEBUG=midscene:ai:call prints AI response details.
• DEBUG=midscene:* prints all Midscene Debug logs.

For the complete selector list, log location, and handling notes, see Runtime configuration: Debug logs.

Usage statistics are also available in generated report files.

#Record model calls

Set MIDSCENE_RECORD_MODEL_CALL=true to write model requests, responses, and streaming chunks to a JSONL file:

midscene_run/model-requests/<start-time>-<pid>.jsonl

Each process creates one file, with one JSON event per line. Local recording is available only in Node.js and Electron. Browsers and Workers do not write local files. Codex App Server records also include available protocol metadata.

Every event has a type of request, chunk, response, or error. It also includes an executionId that groups calls and retries under the same execution ID. For HTTP model requests, this value is also sent in the x-midscene-execution-id header. Calls outside a report execution, such as connectivity checks, use a generated ID prefixed with unscoped-.

#Request tracing headers

Midscene automatically adds the following headers to OpenAI-compatible HTTP model requests:

Calls outside a report execution, such as connectivity checks, use a generated execution ID prefixed with unscoped-. These two headers are sent by default; if headers with the same names are customized in MIDSCENE_*_INIT_CONFIG_JSON, Midscene overrides them.

#Observability platforms

The automatic integrations below wrap the OpenAI SDK client used for OpenAI-compatible HTTP requests. They do not apply to Codex App Server or Jev's System One protocol.

Notes:

• LangSmith and Langfuse can be enabled simultaneously.
• This integration supports Node.js only. Browser environments throw an error.
• createOpenAIClient receives the original OpenAI client. If the callback returns a client, that client replaces the automatically wrapped client; apply any tracing wrappers you need inside the callback. If it returns no client, the automatic integrations remain in effect.

#LangSmith

LangSmith is a platform for debugging large language models. Midscene provides automatic integration support through a dependency and environment variables.

Install the dependency

npm install langsmith

Set environment variables

# Enable Midscene's LangSmith auto-integration
export MIDSCENE_LANGSMITH_DEBUG=1

# LangSmith configuration
export LANGCHAIN_API_KEY="your-langchain-api-key-here"
export LANGCHAIN_TRACING=true
export LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"
# export LANGCHAIN_ENDPOINT="https://eu.api.smith.langchain.com" # If signed up in the EU region

When Midscene creates an OpenAI client for a model request, it logs the enabled wrapper:

DEBUGGING MODE: langsmith wrapper enabled

For finer-grained control, such as enabling LangSmith only for specific tasks, use createOpenAIClient to wrap the client manually.

#Langfuse

Langfuse is an LLM observability platform. Midscene integrates Langfuse's observeOpenAI wrapper to trace OpenAI API calls automatically.

Because Langfuse tracing uses OpenTelemetry, initialize the OpenTelemetry SDK when the application starts.

Install the dependencies

npm install @langfuse/openai @langfuse/otel @opentelemetry/sdk-node

Initialize OpenTelemetry

Add this code at the very top of the application entry file:

import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";

const sdk = new NodeSDK({
  spanProcessors: [new LangfuseSpanProcessor()],
});
sdk.start();

Set environment variables

# Enable Midscene's Langfuse auto-integration
export MIDSCENE_LANGFUSE_DEBUG=1

# Langfuse configuration
export LANGFUSE_PUBLIC_KEY="your-langfuse-public-key-here"
export LANGFUSE_SECRET_KEY="your-langfuse-secret-key-here"
export LANGFUSE_BASE_URL="https://cloud.langfuse.com" # 🇪🇺 EU region
# export LANGFUSE_BASE_URL="https://us.cloud.langfuse.com" # 🇺🇸 US region

When Midscene creates an OpenAI client for a model request, it logs the enabled wrapper:

DEBUGGING MODE: langfuse wrapper enabled

This log identifies the wrapper being enabled; it does not confirm that traces reached Langfuse. Run a model request and check your Langfuse project to verify trace delivery.

See the Langfuse OpenAI integration documentation for more configuration options and best practices.

#Legacy configuration

These settings remain supported; use the replacements below for new configurations.