• English
  • Migrate to Midscene Test

    Legacy YAML and native Midscene Test share the same underlying execution kernel and report page, but they have separate user-facing entries. The midscene command owns legacy tasks/flow files, YAML batch configuration, and legacy CLI options. The midscene-test command owns native cases/steps files, TypeScript or JavaScript project configuration, lifecycle hooks, and Nodes.

    What “legacy YAML” means

    In this guide, “legacy YAML” specifically means the YAML automation protocol from @midscene/cli, not every YAML file. It typically has these characteristics: the workflow has top-level tasks and sequences actions through tasks[].flow; platform, Agent, and output configuration is written directly in workflow YAML; batch execution uses YAML configuration with fields such as files, setup, concurrent, and retry; and the entry command is midscene. A file with top-level cases/steps and lifecycle hooks that runs through midscene-test is native Test YAML, not legacy YAML.

    The legacy path receives compatibility maintenance and necessary fixes only. It will not receive new syntax, capabilities, configuration fields, or CLI options. Use native Midscene Test for new projects, new cases, and future capability development; existing legacy projects can continue to run and migrate gradually.

    There are two migration paths:

    • Upgrade in place: keep legacy cases, batch configuration, directory layout, and the midscene command unchanged. This adopts the shared execution kernel and new report without changing the input contract.
    • Native migration: rewrite complete files as Test Documents, Cases, and Steps; replace batch YAML with project configuration; then switch those files to midscene-test.

    The complete legacy field reference remains available in YAML script runner and Workflow in YAML format for maintaining and migrating existing projects.

    Path 1: upgrade legacy execution in place

    If the project already uses @midscene/cli, upgrade it and keep the existing command. midscene preserves the legacy input contract while executing through the shared kernel and generating the new report page:

    pnpm exec midscene ./flow.yaml
    pnpm exec midscene --config ./batch.yaml

    Platform configuration, agent, environment interpolation, task continuation, whole-file retries, output files, and summary retain their legacy semantics. Reports use the new Test page; existing HTML files need no conversion, so rerun to regenerate them. Legacy agent.generateReport: false remains respected.

    Legacy flags such as --files, --setup, --concurrent, --retry, --continue-on-error, --summary, --headed, --keep-window, --share-browser-context, --dotenv-override, --dotenv-debug, --<target>.<field>, and --no-<target>.<field> remain accepted by midscene. They are not midscene-test options and do not appear in its help.

    Command and file boundaries
    • midscene accepts only legacy tasks/flow workflow files and YAML batch configuration.
    • midscene-test accepts only native cases/steps workflow files and TypeScript or JavaScript project configuration.
    • A YAML file cannot mix legacy tasks with native cases or lifecycle hooks.
    • One command invocation cannot select both formats. During gradual migration, keep them in separate directories or file selections and run two commands.
    • --project, --result-dir, and midscene-test nodes apply only to native Test projects. Legacy scheduling and platform flags remain on midscene.

    Both commands validate this boundary before creating browser, device, or Agent resources. They report the wrong-format files and point to the matching command instead of silently rerouting them.

    Path 2: migrate to a native Test project

    Use the following order so each step can be run and compared independently:

    1. Run the existing project with midscene and save its exit status, summary, and new report as a baseline.
    2. Use pnpm dlx @midscene/test create to create a Test project for the target platform. Start with the generated midscene.config.ts and setup.
    3. Map the legacy batch configuration to an Execution Project: put shared resources in setup, per-file pages, devices, and Agents in fileSetup, and business prerequisites in YAML beforeAll or beforeEach.
    4. Convert one complete tasks/flow file to cases/steps, move it into the native file selection, and run it with midscene-test. Keep unconverted files on midscene.
    5. Run pnpm exec midscene-test nodes --project <name> to generate the current Project's Node Spec, then validate action names, inputs, and string shorthand against it.
    6. After all files are converted, remove the legacy batch configuration, legacy command, and legacy-only command-line options.

    Case structure mapping

    Legacy YAML:

    web:
      url: https://example.com
    
    tasks:
      - name: Search for a product
        continueOnError: true
        flow:
          - ai: Search for a mug
          - aiAssert: The results contain a mug
            errorMessage: The expected product was not found

    Native Test:

    beforeEach:
      - gotoUrl: https://example.com
    
    cases:
      - name: Search for a product
        steps:
          - aiAct: Search for a mug
          - aiAssert:
              prompt: The results contain a mug
              message: The expected product was not found
    Legacy fieldNative Test field or mechanismNotes
    taskscasesOne legacy task becomes one Case.
    tasks[].namecases[].namePreserve the name.
    tasks[].flowcases[].stepsOne flow item becomes one Step.
    tasks[].continueOnError: trueNative Case continuation defaultmidscene preserves legacy task continuation. Native Test continues to the next Case by default.
    Omitted continueOnErrorLegacy failure policymidscene stops the legacy file. There is no native onFailure option: put shared prerequisites in beforeAll or dependent steps in one Case.
    Initial page or app in a top-level platform fieldsetup / fileSetup plus lifecycle NodesCreate resources according to their sharing scope. Use gotoUrl, launch, or another Node in beforeEach when every Case must return to an initial state.
    No legacy equivalentbeforeAll, beforeEach, afterEach, afterAllNative file and Case lifecycle hooks.
    No legacy equivalentcases[].tagsSelect Cases per Execution Project.

    Keeping old files unchanged preserves their failure policy. Rewriting them as native Cases adopts native semantics; it is not an exact scheduling equivalence. Step $: { continue-on-error: true } continues later Steps in that phase, not later legacy tasks, and does not make a failed Case pass.

    Flow action mapping

    The legacy runtime keeps accepting the fields in the left column and adapts them internally to the shared kernel. That internal adaptation is not a native Test input contract. When migrating a file, write the native form in the right column and use the Nodes registered in the project's midscene-node-spec.md as the source of truth.

    Legacy flow fieldNative Test Node / inputMigration notes
    ai, aiAction, aiAct, instructionaiAct.promptUse aiAct; move remaining settings into options.
    aiAssertaiAssert.promptRename errorMessage to message; move extraction settings into options.
    aiQuery, aiNumber, aiString, aiBoolean, aiAsk, aiLocateSame-name Node with promptPlace images and extraction settings under prompt or options as described by the Node Spec.
    aiWaitForaiWaitFor.promptLegacy timeout maps to options.timeoutMs in the Node input.
    aiTapaiTap.promptNormalize legacy locate, images, and locate settings into prompt and options.
    aiScrollaiScrollUse the locate description as prompt and scrolling settings as options.
    aiInputaiInput.value and aiInput.promptMove the entered value to value and the locate description to prompt.
    aiKeyboardPressaiKeyboardPress.keyName and optional promptUse the inputs described by the project Node Spec.
    sleepsleep.msFor example, change sleep: 1000 to sleep: { ms: 1000 }. New cases can also use the common wait Node.
    javascriptjavascript.scriptIts return value is recorded in the Step result.
    recordToReport, logScreenshotrecordToReportMap the title to title and content to options.content.
    runGherkinScenariorunGherkinScenario.scenarioLegacy execution disables caching for this action.
    runAdbShellrunAdbShellMap the command to command; preserve legacy timeout as Node input.
    FinalizeLegacy-only planning markerAccepted by the legacy runtime; omit it when writing native Cases. It is not a public Node.
    Other platform and custom actionsA registered same-name Node, or the generic action NodeRun midscene-test nodes first and rewrite inputs according to the target platform's Node Spec.
    Flow item nameStructured Node resultmidscene still writes named legacy outputs. Native Test has no named cross-Step result expression; share data through custom Nodes and the project context.

    Concrete example: migrate one order-submission Case

    The following legacy case fills in a recipient, waits for the submit button, submits the order, and records the result. It uses task failure policy, an action alias, locate input, action timeout, an assertion message, a fixed wait, and report recording:

    web:
      url: https://shop.example.com/checkout
    
    tasks:
      - name: Submit an order
        continueOnError: true
        flow:
          - aiInput: Alice
            locate: Recipient name input
          - aiWaitFor: The submit order button is enabled
            timeout: 10000
          - ai: Click the submit order button
          - aiAssert: The page shows that the order was submitted
            errorMessage: No success message appeared after submission
          - sleep: 500
          - recordToReport: Order submission result
            content: The checkout flow completed

    The same Case in native Test YAML is:

    beforeEach:
      - gotoUrl: https://shop.example.com/checkout
    
    cases:
      - name: Submit an order
        steps:
          - aiInput:
              prompt: Recipient name input
              value: Alice
          - aiWaitFor:
              prompt: The submit order button is enabled
              options:
                timeoutMs: 10000
          - aiAct: Click the submit order button
          - aiAssert:
              prompt: The page shows that the order was submitted
              message: No success message appeared after submission
          - wait:
              duration: 500
              unit: ms
          - recordToReport:
              title: Order submission result
              options:
                content: The checkout flow completed

    The conversion has several parts:

    1. web.url no longer sits beside case content. Preparation functions in the project configuration create the browser and page. Because every Case must start on the checkout page, gotoUrl in beforeEach owns the URL.
    2. The task becomes a Case. Native Test already continues after a failed Case, so this example needs no continuation option.
    3. The aiInput value and locate description move from legacy sibling fields to the Node's value and prompt.
    4. aiWaitFor.timeout limits that wait operation, so it becomes options.timeoutMs. To limit the complete Step instead, use $: { timeout: 10000 }.
    5. ai becomes aiAct, and aiAssert.errorMessage becomes aiAssert.message.
    6. The fixed delay uses the native common wait Node. The report title and body move to recordToReport.title and options.content.

    This example changes the input structure without changing business execution order. During a real migration, run pnpm exec midscene-test nodes --project <name> to generate the current Project's midscene-node-spec.<project-name>.md. Use this Node Spec—not examples from another platform—to confirm whether gotoUrl, aiInput, aiWaitFor, and the other referenced Nodes are available and how to provide their inputs.

    Script and Agent configuration mapping

    midscene continues to read top-level legacy YAML configuration. In a native migration, move these fields out of Case YAML and into midscene.config.ts, setup / fileSetup, or Nodes. Use setup for shared resources, fileSetup for each file's resources, and YAML hooks for business prerequisites. See setup, fileSetup, and beforeAll.

    Legacy script fieldNative Test configuration or mechanismNotes
    targetWeb setup / fileSetupThis field is deprecated. Do not carry it into new configuration; configure the actual Playwright, Puppeteer, or bridge runtime.
    page, browser, webWeb setup / fileSetup plus Web NodesPut the shared Browser in setup and independent pages in fileSetup. Put repeatable navigation and Cookie operations in lifecycle Nodes.
    androidAndroid setup / fileSetup plus Android NodesUse device ID, screenshot, and input settings to create the Device and Agent. Put launch and other case actions in beforeEach or steps.
    iosiOS setup / fileSetup plus iOS NodesPut WDA connection and Device/Agent creation in setup or fileSetup. Use registered Nodes for application launch and interaction.
    harmonyHarmonyOS setup / fileSetup plus HarmonyOS NodesPut HDC connection, device settings, and application mappings in setup or fileSetup. Use registered Nodes for application and shell actions.
    computerComputer setup / fileSetup plus Computer NodesPut display and device settings in setup or fileSetup. Use registered Nodes for UI actions.
    interfaceCustom setup / fileSetup plus createMidsceneNodes()Import and create the custom Interface, Device, and Agent in setup or fileSetup, then expose the Agent to Nodes.
    agent.generateReportLegacy report settingOld YAML can disable its report. Native Test always generates a combined report.
    agent.reportFileName, agent.testIdLegacy Agent artifact settingsOpen the report using the path printed by the CLI or the report link in the summary. testId remains deprecated.
    agent.groupName, agent.groupDescriptionProject and Case names plus report structureThere is no same-name native field. Express grouping through Project names, Case names, or tags.
    agent.replanningCycleLimit, agent.aiContexts, agent.aiActContext, agent.aiActionContext, agent.cache, agent.screenshotShrinkFactorAgent construction optionsContinue passing the relevant options when creating the Agent instead of placing them in Case YAML.
    agent.outputFormat, agent.persistExecutionDump, agent.autoPrintReportMsgTest reporting or Agent optionsPrefer the combined Test report. Preserve an option when creating the Agent only when its Agent-specific behavior is still required.
    config.output, or output under a platform fieldTest structured resultsmidscene still writes legacy output files. Native Test uses --result-dir, summary.json, and the report. Return business data from custom Nodes or write it to external storage.
    config.unstableLogContentTest report and structured resultsThere is no same-name native field. Migrate consumers away from the legacy experimental log-content file.
    tasksNative Workflow DocumentMap it to cases and lifecycle hooks as described above.

    See Configure the runtime environment for complete preparation and platform Node registration examples.

    Batch configuration mapping

    Legacy batch.yaml:

    files:
      - flows/cases/**/*.yaml
    setup: flows/setup.yaml
    concurrent: 2
    retry: 1
    continueOnError: false
    summary: result.json
    headed: true

    Native Test moves execution settings into midscene.config.ts:

    import { defineTestProject } from '@midscene/test/config';
    import { nodes, setup } from './test-runtime';
    
    export default defineTestProject({
      nodes,
      projects: [
        {
          name: 'default',
          setup,
          files: {
            include: ['flows/cases/**/*.yaml'],
          },
          retry: 1,
        },
      ],
      test: {
        bail: 1,
      },
      output: {
        reportDir: './midscene_run/report',
      },
    });
    Legacy batch settingNative Test configuration or mechanismNotes
    filesprojects[].files.includemidscene preserves old order and repeated invocations. Native selection is sorted and deduplicated, with no files.order switch.
    setupsetup / fileSetup / YAML beforeAllLegacy execution runs a prerequisite YAML file first. Native preparation functions create resources according to sharing scope, and beforeAll runs file prerequisites; there is no setupFile option.
    concurrenttest.maxConcurrentFiles + fileSetupExplicitly enable file concurrency and prepare an independent environment for each file.
    retryLegacy whole-file retry; native projects[].retryNative retries only a failed Case. Keep legacy files on midscene when whole-file replay is required; there is no retryScope switch.
    continueOnError: falsetest.bail: 1Stop scheduling new files or Cases when the failure threshold is reached. Concurrent work that has already started still completes cleanup.
    continueOnError: truetest.bail: 0Disable global failure-threshold stopping. midscene separately preserves old task continuation; native Cases continue by default.
    summaryTest summary.json, report, and programmatic APIsmidscene still writes the legacy summary. Native Test has no same-name setting; migrate consumers and use --result-dir to select the structured result directory.
    shareBrowserContextProject setup and fileSetupNative Test can share a Browser in setup and create a separate BrowserContext per file in fileSetup; this changes the legacy shared-session behavior.
    headedBrowser or device setup launch settingsFor example, Playwright uses chromium.launch({ headless: false }).
    keepWindowsetup cleanup policyUse only for local debugging; the project must decide whether to delay or skip browser teardown.
    dotenvOverride, dotenvDebugProject environment-loading policyNative Test reads process.env. If you rely on an .env file, load it explicitly in the config or launch script and configure precedence and debugging there.
    web, android, ios, harmony, computer, interfacesetup / fileSetup, variables, and registered platform NodesPrepare resources according to sharing scope; keep serializable environment differences in variables.

    The native example above migrates file selection, case retries, and the failure threshold; it does not preserve legacy setup-file behavior or file concurrency. For concurrency, add test.maxConcurrentFiles and fileSetup as shown in File concurrency. To keep existing cases and batch configuration, continue using midscene; legacy concurrent / --concurrent and setup-file behavior remain unchanged.

    The midscene entry preserves legacy output naming: explicit flow name values, numeric keys for unnamed extracted results, and the existing overwrite behavior for duplicate names.

    Validate the migration

    Run the legacy version with midscene and the converted version with midscene-test, then compare at least:

    • Process exit status and continuation after a failure.
    • Whether retries apply to one Case or the complete file.
    • Environment variables, platform connection settings, and initial page state.
    • AI action inputs, timeouts, and assertion failure messages.
    • Business output consumers and Step results in the new Test report.
    • Page, Agent, device, and test-data isolation during concurrent execution.
    • Lifecycle hooks and resource cleanup after failures, timeouts, and Ctrl+C interruption.

    After migration, write new cases with cases/steps. Keep tasks/flow only in files that have not yet been migrated so the compatibility format does not expand into new code.