Testing

Testing flow

Local (ro) → Generation rules (RunWhen Local) → Platform (upload SLXs)

1. Run .robot files locally to verify they pass against test infrastructure
2. Test generation rules with RunWhen Local to verify they produce the right SLX configurations
3. Upload generated SLXs to a RunWhen Platform workspace for final end-to-end verification

Local testing with ro

The ro runner (from the devtools container) wraps robot with isolated working directories, structured log output, and selective test execution:

# Run all .robot files in the current directory
ro

# Run a specific file
ro runbook.robot
ro sli.robot

# Run a single test case by name
ro --test "Check Health" runbook.robot

# Run tests in a different codebundle
ro ../other-codebundle/

HTML reports are written to /robot_logs and served at localhost:3000. Open the Ports tab in Codespaces to access the log server.

Environment variables and secrets

All required environment variables and authentication prerequisites should be documented in the codebundle’s README.md. In dev mode (RW_MODE=dev), secrets are read from env vars or local files via RW_FROM_FILE and RW_SECRET_REMAP — see the RW.Core Reference for details.

Taskfiles for test infrastructure

Most codebundles include a .test/ directory with a Taskfile for managing test infrastructure:

cd codebundles/your-bundle/.test
task -l
# Available tasks:
# build-terraform-infra     Build test infrastructure
# generate-rwl-config       Generate RunWhen Local configuration
# run-rwl-discovery         Run RunWhen Local Discovery on test infrastructure
# upload-slxs               Upload SLX files to the RunWhen Platform
# delete-slxs               Delete SLX objects from the platform
# validate-generation-rules Validate YAML files in .runwhen/generation-rules
# clean                     Run cleanup tasks

These tasks handle Terraform setup/teardown, RunWhen Local discovery, and platform uploads — they’re portable across codebundles with minor adjustments.

Generation rule testing

RunWhen Local can test your generation rules against real infrastructure:

cd codebundles/your-bundle/.test
task generate-rwl-config
task run-rwl-discovery

This generates a workspaceInfo.yaml, runs the workspace-builder, and produces SLX configuration files under output/workspaces/:

output/
├── resource-dump.yaml
└── workspaces/
    └── [workspace-name]/
        └── slxs/
            └── [slx-name]/
                ├── runbook.yaml
                ├── sli.yaml
                └── slx.yaml

Review the generated files to verify the generation rules match the right resources.

Platform testing

Once local testing passes, upload the generated SLXs to a RunWhen Platform workspace:

export RW_API_URL="https://papi.beta.runwhen.com"
export RW_WORKSPACE="your-workspace-name"
export RW_PAT="your-personal-access-token"

task upload-slxs

Within a few minutes, the SLX appears in the RunWhen Workspace Map. You may need to update the SLX’s secrets configuration (Edit → secretsProvided) to point at workspace-managed secrets. Then verify:

1. Health — the SLI starts running and producing metrics
2. Tasks — create a RunSession to execute all runbook tasks
3. Report — review the RunSession report for issues and next steps

CI integration

The devtools container is also used in CI for PR review. Setting PR_NUMBER checks out the PR branch automatically, allowing automated or human review of Skill Template changes before merge.