Author Custom Tasks with MCP

Two ways to read this guide. Skim the short story below for the 60-second summary, or jump into the long story for the full walkthrough with copy-pasteable YAML, real bundle examples, and screenshots.

Short story (TL;DR)

• You describe a check to an MCP agent in Cursor or Claude Desktop (“alert me when Crossplane Buckets fall out of Ready state”). The agent first searches the Skills Registry — a matching Skill Template is always faster than a custom script.
• If nothing matches, the agent writes a Python or Bash main() that returns RunWhen’s issue-contract shape, statically validates it, and executes it against your live workspace on the runner to confirm it actually finds signal.
• Once the script works, the agent renders a complete bundle into a subdirectory of your Custom Discovery CodeCollection: a generation rule, an SLX template, a TaskSet template with the script embedded as a base64-encoded GEN_CMD for the shared tool-builder runtime, plus a human-readable SKILL_TEMPLATE.md and the decoded raw_script.{py,sh} for review.
• You (or a reviewer) open a pull request on the private repo. A round-trip check decodes the TaskSet’s GEN_CMD and asserts it matches the reviewed script byte-for-byte — the safeguard against a stale base64 blob diverging from what humans approved. Merge the PR, and runwhen-local’s next reconcile renders one SLX per matched workspace.
• Promote the same reviewed bundle dev → stage → prod via git branches. Each environment has its own runwhen-local instance and its own workspace secrets; the LLM authors in dev and dispatches in prod, but it never merges PRs and never touches production infrastructure directly.

Ready for the details? Continue to the full walkthrough →

Long story (full walkthrough)

This guide picks up where the Custom Discovery CodeCollection walkthrough left off. That guide shows how to hand-author a rules-only bundle for a Kubernetes CRD; this guide shows how to have an MCP agent author the same shape of bundle for a custom check, with a live test loop against your workspace and a git-based review gate before the check reaches production.

The delivery vehicle is identical: a private CodeCollection carrying only rules and templates, delegating execution to a generic Skill Template in the RunWhen Generic CodeCollection. What’s new is the runtime and the authoring loop:

• Runtime: tool-builder — a shared Bash/Python runner in the RunWhen Generic CodeCollection that accepts your script as a base64-encoded GEN_CMD value and decodes it at execution time. In this flow the MCP agent handles the encoding and templating for you.
• Authoring loop: the MCP server exposes tools (search_registry, validate_script, run_script_and_wait, render_codecollection_skill, commit_slx) that let the agent iterate against a real workspace before anything hits git.

What you’ll build

By the end of this guide you’ll have:

1. An MCP-authored codebundle inside your existing Custom Discovery CodeCollection — same repo, same layout conventions.
2. A PR-reviewed script whose decoded content matches the base64 blob that will run in production, verified by an automatic round-trip check at commit time.
3. A dev → stage → prod promotion path that uses git branches (or separate repos) with a distinct runwhen-local instance per environment. Same reviewed bundle promoted through each.
4. Concrete guarantees about where the LLM does and doesn’t touch infrastructure — it authors in dev, it dispatches already-reviewed tasks in prod, it never merges PRs or connects to prod infrastructure directly.

When to use this flow

Choose the MCP-authored flow when:

• The check you need isn’t in the registry (search_registry came up empty) and you don’t want to hand-write Robot Framework or maintain a public codebundle.
• You want the script to be reviewed and versioned — audit trail, git revert, diff over time, workspace-scoped access control on the private repo.
• You want the same script to cover many workspaces in a fleet. One bundle, one match rule, one PR review; SLXs render into every matched workspace automatically.
• You’re comfortable with a base64-encoded GEN_CMD blob in the committed YAML. The round-trip check keeps it honest, but the reviewable source of truth lives in raw_script.{py,sh} next to the bundle.

Prefer the alternatives when:

• A registry Skill Template already exists. Use deploy_registry_codebundle — production-tested, keeps up with platform-team fixes, no git operations.
• You need one-workspace-only iteration and can accept inline storage. Use commit_slx inline — the script lives directly in the workspace repo as YAML, no cross-workspace reuse but zero infrastructure to set up.
• The check needs custom Robot keywords, cross-language libraries, or a runtime that isn’t tool-builder. You’re in traditional-CodeCollection territory — see Tasks & CodeBundles.

Prerequisites

• An MCP client connected to a workspace-scoped runwhen-platform-mcp server. See the MCP end-to-end flow for the connection model — the same server you use for workspace_chat and run_slx exposes the task-authoring tools this guide relies on.
• A Custom Discovery CodeCollection already wired into your runner. If you’ve never set one up, read that guide first — the runwhen-local workspaceInfo.yaml wiring, RBAC, and Gitea/GitHub credentials it covers are prerequisites here.
• A RunWhen Local runner on a recent enough build that its indexer emits platform: runwhen workspace resources. Earlier builds silently skip generation rules that select on platform: runwhen; if you’re unsure, upgrade before wiring the bundle in.
• A dev workspace the agent can safely test against — one where empty-signal issues, throwaway secrets, and half-broken scripts won’t page anyone.

Vocabulary primer

Terminology from prior guides is assumed. Concepts introduced here:

Term	Meaning
MCP agent	The LLM in your IDE (Cursor, Claude Desktop) plus the runwhen-platform-mcp server it talks to. Authors, validates, and renders bundles on your behalf.
Skills Registry	registry.runwhen.com — the searchable catalog of published Skill Templates. Always the first stop before authoring anything custom.
GEN_CMD	The base64-encoded script value in the rendered TaskSet YAML. Decoded by the tool-builder runtime at execution time.
SKILL_TEMPLATE.md	The human-readable review companion to a rendered bundle. Explains match rules in plain English, lists secrets/env vars/runtime vars, and points reviewers at the decoded script.
raw_script.{py,sh}	The decoded copy of the script, checked into .runwhen/ alongside the base64-encoded YAML. This is what humans review; the round-trip check asserts it matches GEN_CMD.
Round-trip check	A commit-time assertion that base64 -d $GEN_CMD == cat raw_script.{py,sh}. Blocks hand-edited YAML from silently diverging from the reviewed script.
platform: runwhen	A generation-rule platform selector. Matches workspace resources emitted by the runwhen-local indexer instead of Kubernetes / cloud resources.

The full authoring flow

Phase 1 — Intent and registry check

The agent’s first move is always search_registry. Custom scripts are the last resort:

“Alert me when any Crossplane Bucket falls out of Ready=True on this cluster.”

If a matching Skill Template exists, the agent will offer deploy_registry_codebundle instead of writing new code — faster, safer, benefits from platform-team fixes for free. If nothing matches, it moves to Phase 2.

Phase 2 — Author, validate, test against a live workspace

1. Load workspace conventions. get_workspace_context reads your workspace’s RUNWHEN.md rules (naming, severity thresholds, replica targeting, etc.) so the generated script respects them.
2. Write the script. A Python or Bash main() that returns the RunWhen issue-contract shape:

   def main():
       return [{
           "issue title": "Bucket X is not Ready",
           "issue description": "Ready=False, Synced=True",
           "issue severity": 2,
           "issue next steps": "Check Crossplane provider logs",
       }]

3. Statically validate. validate_script checks that main() exists, there’s no __main__ guard, severity is 1–4, and — critically — the issue keys are spelled exactly right. Typos like issue desription used to cause opaque KeyErrors at runtime; they’re now blocked at author time.
4. Test against live infrastructure. run_script_and_wait executes the script through the dev runner, against real workspace secrets and real infrastructure. The parsed output comes back to the IDE. This catches the ~20% of contract issues that static analysis can’t — missing secrets, RBAC gaps, unexpected data shapes, and the classic “check ran, but no signal was generated because thresholds were never crossed.”

The agent loops through 2–4 until the script both validates and returns the issues you expected on the dev workspace.

Phase 3 — Collect template inputs and render the bundle

Before rendering, the agent explicitly gathers three input categories:

• Secrets. discover-secrets walks the workspace configuration and returns the real workspaceKey values (not bare Vault names) that map to the secret env vars the script needs.
• Environment variables. Static config that matched what worked during the dev-runner test.
• Runtime variables. Per-run inputs (like NAMESPACE, TARGET_URL) with defaults and validation.

Then render_codecollection_skill writes a complete codebundle directory to your local CodeCollection checkout:

codebundles/<bundle-name>/
├── README.md                                # bundle overview
└── .runwhen/
    ├── SKILL_TEMPLATE.md                    # the human review artifact
    ├── raw_script.py                        # (or raw_script.sh) — decoded script
    ├── generation-rules/
    │   └── <bundle-name>.yaml               # platform: runwhen, resourceTypes: [workspace]
    └── templates/
        ├── <bundle-name>-slx.yaml           # SLX metadata
        └── <bundle-name>-taskset.yaml       # base64 GEN_CMD lives here

The base64 vs plaintext relationship is deliberate. The TaskSet YAML — what the runner actually executes — carries the script as GEN_CMD (base64). The review companions in .runwhen/ carry the decoded script. Both must match, byte-for-byte; the renderer is authoritative, and hand-editing the base64 is what breaks in every prototype where someone has tried it.

Phase 4 — Review, round-trip check, commit, wire the runner

1. Open the diff. In the PR, reviewers read .runwhen/SKILL_TEMPLATE.md and .runwhen/raw_script.{py,sh}. The SKILL_TEMPLATE spells out the match rules in plain English, the secrets by name, the env/runtime vars, and the operational metadata (timeout, access class, data class, owners).
2. Automatic round-trip check. Before the commit is finalized, the commit-to-codecollection skill decodes the TaskSet GEN_CMD and asserts it matches raw_script.{py,sh} byte-for-byte. If they diverge — because someone hand-edited the YAML or forgot to re-render after a script change — the commit is blocked and the agent is instructed to re-render (never edit base64 by hand).
3. Approve, merge, push. Standard PR flow to the private repo (Gitea for airgap, GitHub otherwise).
4. Wire the runner (first time only). If this is a new repo, add it to runwhen-local’s workspaceInfo.yaml under codeCollections and reconcile. Subsequent bundles land in the same repo and go live on the next reconcile.

Phase 5 — Discovery, render, execute

1. Reconcile. runwhen-local pulls the private repo (plus, for airgap deployments, the in-cluster rw-generic-codecollection catalog mirror).
2. Indexer emits workspace resources. The platform: runwhen indexer surfaces every workspace this runner is bound to as a matchable resource — that’s what the generation rule selects on.
3. Generation rule matches. The workspace-builder evaluates matchRules and renders one SLX + Runbook per match, substituting {{slx_name}}, {{default_location}}, {{workspace.owner_email}}, and baking GEN_CMD into the Runbook.
4. Workspace-builder commits rendered YAML. The rendered SLX and Runbook are pushed to the workspace repo. The RunWhen Platform picks up the change automatically, so the SLX becomes visible in the UI and available to workspace_chat.
5. User triggers the SLX. When a person, scheduler, or alert runs the task, the platform reads the pre-rendered Runbook from the workspace repo, assembles the run configuration (TASK_TITLE, GEN_CMD, INTERPRETER, CONFIG_ENV_MAP, SECRET_ENV_MAP, TIMEOUT_SECONDS, plus the secret bindings), and dispatches it to the runner in your cluster.
6. The task-worker executes it. Your runner dispatches to a task-worker pod whose image already contains rw-generic-codecollection/codebundles/tool-builder — no runtime git clone. The worker decodes GEN_CMD, mounts secrets from its local secret store, and runs the script inside the tool-builder Robot task.

Anatomy of a rendered bundle

Here’s what the agent actually writes to disk. This example is system-info-check from the simple-private-codecollection reference repo — a minimal Python task that runs uname -a on the runner and returns an informational issue with the kernel version.

.runwhen/raw_script.py — the reviewable source

import subprocess

def main():
    issues = []
    total = 0
    failed = 0

    try:
        result = subprocess.run(
            ["uname", "-a"], capture_output=True, text=True, timeout=10
        )
        total += 1
        kernel_info = result.stdout.strip()
        if result.returncode == 0:
            issues.append({
                "issue title": "Kernel info collected successfully",
                "issue description": f"Kernel info retrieved: {kernel_info}",
                "issue severity": 4,
                "issue next steps": "No action needed. Informational check.",
            })
        else:
            failed += 1
            issues.append({
                "issue title": "Failed to collect kernel info",
                "issue description": f"uname failed with rc {result.returncode}: {result.stderr.strip()}",
                "issue severity": 2,
                "issue next steps": "Check runner image and available commands.",
            })
    except Exception as e:
        failed += 1
        issues.append({
            "issue title": "Error collecting system info",
            "issue description": f"Exception: {str(e)}",
            "issue severity": 2,
            "issue next steps": "Investigate runner environment.",
        })

    issues.append({
        "issue title": "System Info Check — Summary",
        "issue description": f"Checked {total} system commands; {failed} failed.",
        "issue severity": 4,
        "issue next steps": "Informational. Review details in description.",
    })
    return issues

This is what the PR reviewer reads. It’s ordinary Python — not templated, not base64’d, not YAML-embedded. If it doesn’t match GEN_CMD in the TaskSet, the commit is blocked.

.runwhen/generation-rules/system-info-check.yaml — the discovery rule

apiVersion: runwhen.com/v1
kind: GenerationRules
spec:
  platform: runwhen
  generationRules:
  - resourceTypes:
    - workspace
    matchRules:
    - type: pattern
      pattern: .+
      properties:
      - name
      mode: substring
    slxs:
    - baseName: system-info-che
      qualifiers:
      - workspace
      baseTemplateName: system-info-check
      levelOfDetail: detailed
      outputItems:
      - type: slx
      - type: runbook
        templateName: system-info-check-taskset.yaml

Note the platform: runwhen and resourceTypes: [workspace] — this is what makes the workspace itself the discovered resource. pattern: .+ matches every workspace this runner is bound to; tighten it to pattern: prod- (or similar) if you want a subset.

.runwhen/templates/system-info-check-taskset.yaml — the execution target

apiVersion: runwhen.com/v1
kind: Runbook
metadata:
  name: {{slx_name}}
  labels:
    {% include "common-labels.yaml" %}
  annotations:
    {% include "common-annotations.yaml" %}
spec:
  location: {{default_location}}
  codeBundle:
    repoUrl: https://github.com/runwhen-contrib/rw-generic-codecollection.git
    ref: main
    pathToRobot: codebundles/tool-builder/runbook.robot
  configProvided:
    - name: TASK_TITLE
      value: Run system info check
    - name: GEN_CMD
      value: aW1wb3J0IHN1YnByb2Nlc3MKCmRlZiBtYWluKCk6...   # base64 of raw_script.py
    - name: INTERPRETER
      value: python
    - name: CONFIG_ENV_MAP
      value: '{}'
    - name: SECRET_ENV_MAP
      value: '[]'
    - name: TIMEOUT_SECONDS
      value: '300'

codeBundle.repoUrl points at rw-generic-codecollection — but this is an identifier, not a fetch URL. The task-worker image already carries the codebundle; the URL just tells the worker which pre-baked bundle to execute. For airgap deployments the runbook YAML doesn’t have to reach the public internet at runtime.

.runwhen/SKILL_TEMPLATE.md — the plain-English review card

The SKILL_TEMPLATE.md is what reviewers read first. It restates the match rules in prose, tabulates the env vars / secrets / runtime vars, records the operational metadata (timeout, access class, data class, owners), and points at the correct review artifact for the decoded script. It also carries provenance — which MCP tool generated it, from which workspace, at what time — so an auditor can trace the bundle back to the authoring session.

Runtime view — a single execution

Two things make this cheap to operate:

• The RunWhen Platform pre-assembles the run configuration from the workspace repo and hands your runner a fully-resolved TaskSet — including secret bindings — the moment a user triggers the SLX. Your runner does not need to look anything up on its own once it has the request.
• Task-worker pods carry the runtime. rw-generic-codecollection/codebundles/tool-builder is baked into the task-worker image at build time — no runtime git clone. The codeBundle.repoUrl in the Runbook is used by runwhen-local at render time and by the worker to identify which pre-baked codebundle to load; the URL is never fetched over the network at execution time. That’s what makes this pattern safe for airgap deployments.

Debugging map. When an execution fails, the RunSession detail page in the RunWhen UI is the first stop — it shows exactly where in the sequence the run stopped and (usually) why. The categories below cover the observable failure modes:

Symptom	Likely cause	Where to look
SLX visible in UI, task never starts	Runbook YAML is missing required config (empty configProvided, missing secret bindings), or the workspace-builder hasn’t rendered the SLX yet	Run session detail page; workspace-builder logs
Task starts but immediately errors	GEN_CMD decoded to something that isn’t valid Bash/Python — usually because someone hand-edited the base64 blob	Re-render the bundle with render_codecollection_skill; never edit GEN_CMD by hand
Task starts, main() throws	Missing env var expected by the script, or a secret file path that isn’t readable on this runner location	RunSession logs; re-check CONFIG_ENV_MAP and SECRET_ENV_MAP values
Task runs but issues never surface in the UI	The runner couldn’t reach back to the platform to report results	Runner logs; network / firewall between the runner and the RunWhen Platform
Task worker rejects the request	Worker image is older than the codeBundle.ref pinned in the Runbook YAML	Ask your platform team which rw-generic-codecollection version is baked into your worker image; keep the bundle’s ref in sync
Robot task fails after main() returns	Issue-key typo in the returned list (e.g. issue desription)	Caught earlier now by validate_script, but older bundles authored before that check may still fail here — re-run validate_script on the source and re-render

Lifecycle — dev → stage → prod promotion

A bundle isn’t a one-shot artifact. It moves through environments as authors, reviewers, and operators promote it. The standard shape mirrors any software-delivery pipeline: build against a dev workspace, promote to stage, then to prod — each hop gated by a pull request.

DEV — author and test against live infra

The MCP-enabled IDE does its most active work here.

1. render_codecollection_skill writes the bundle to local files (still on the author’s laptop).
2. run_script_and_wait executes the same script that will be baked into the TaskSet against the dev workspace’s real infrastructure — via the dev runner and dev secrets, so this is a genuine end-to-end validation, not a mock.
3. The author opens a PR from a feature branch into the private CodeCollection repo.

STAGE — review, merge, watch it render

Once the PR is reviewed and merged into the stage branch:

4. The stage runwhen-local instance tracks the stage branch through its workspaceInfo.yaml. Reconciliation pulls the new bundle.
5. The generation rule matches, workspace-builder renders SLXs into the stage workspace, and the runner executes tasks against stage infrastructure — exactly the same code path prod will use.
6. Humans (and, optionally, an AI reviewer in read-only mode) verify behavior in the stage workspace UI: SLX renders, issues are actionable, secrets resolve, run time is reasonable.
7. When stage looks good, a second PR promotes stage → prod.

PROD — promote, run against production

8. The prod runwhen-local instance tracks the prod branch. Reconcile pulls the same bundle whose behavior was verified in stage.
9. Generation rule matches, SLXs render into the prod workspace, and the runner executes against prod infrastructure. Because the base64 GEN_CMD is byte-identical to what stage ran, there is nothing about the script itself that stage did not exercise.
10. Ops teams trigger the SLX from the UI, workspace_chat, a schedule, or an alert response.

Why the git-branch model works for this

• Reversibility. git revert on the prod branch removes the SLX from the prod workspace on the next reconcile. No database rollback, no support ticket.
• Auditability. “When did this check land in prod?” is a git log stage..prod on the private codecollection repo. Everything that runs in prod exists in git history first.
• Environment isolation. Dev / stage / prod each have their own runwhen-local instance, their own workspaceInfo.yaml, and their own runner + secrets. A bundle that only exists on the stage branch cannot render into the prod workspace even if a match rule would allow it.
• Multi-workspace fan-out per environment. Inside a single environment, one bundle can render into many workspaces (one per matched resource). Retiring by removing the rule cleans them up everywhere at once, within that environment.

Silent no-op to watch for. A bundle can be merged and reconciled but never render an SLX because its match rule doesn’t select anything. Nothing errors, no SLX appears. Catch this two ways: the SKILL_TEMPLATE.md explains match rules in plain English at PR time, and runwhen-local’s Workspace Explorer (port 8000) shows candidate resources at runtime.

Where the LLM does and doesn’t touch

This is the recurring security question — “does the AI touch prod?” The honest answer is no:

Phase	LLM role	What that means concretely
DEV — author	Active	Writes and edits the script, calls validate_script, calls run_script_and_wait, calls render_codecollection_skill. Sees dev-infra output because it’s driving the dev test.
PR → stage	Optional, read-only	Can summarize a PR diff or answer questions about the bundle. Does not merge.
STAGE — verify	Optional, read-only	Can read stage RunWhen issues and RunSessions via workspace_chat to answer “did the new check work?”. Does not modify code.
PR → prod	Optional, read-only	Same as PR → stage.
PROD — execute	Dispatch only	Users chatting with workspace_chat in the prod workspace can ask the LLM to run this SLX. The LLM calls run_slx. The runner executes the already-reviewed, already-merged task.
Any phase	Never	Direct connection to infrastructure. Bypassing PR review. Running unreviewed code in stage/prod. Editing task scripts in stage/prod without a new dev-authored PR.

The LLM is an orchestrator, not an operator. Every action that touches infrastructure goes through the same runner → task-worker path a human clicking “Run” in the UI would use, and every task it can run in stage or prod was reviewed and merged by a human. This mirrors the general principle in the MCP end-to-end flow guide: LLMs dispatch existing tasks, they do not connect to infrastructure directly.

Common failure modes and their guardrails

Failure	Root cause	What now catches it
Runtime KeyError: 'issue description'	Script contained an issue-key typo (e.g. issue desription)	validate_script blocks non-canonical issue keys at author time
TaskSet GEN_CMD decoded to a different script than raw_script.{py,sh}	Someone hand-edited the YAML and left a stale base64 blob	Commit-time round-trip decode assertion in commit-to-codecollection; renderer is authoritative
Worker image doesn’t include the referenced codebundle version	Runbook pinned a ref newer than what’s baked into the worker image	Ask the platform team which generic_runtime_ref matches the customer’s runner image; keep the bundle in sync
Missing secret at runtime	Wrong workspaceKey or secret not provisioned on the target runner location	discover-secrets is a required checklist item before rendering
Silent discovery no-op	runwhen-local without platform: runwhen indexer support	Diagram calls it out; upgrade the runner before wiring bundles that use platform: runwhen

Where to go from here

• Read the sibling guide. The Custom Discovery CodeCollection guide covers the CodeCollection layout, RBAC, and runner wiring end-to-end — the same pieces this guide’s bundle plugs into, just with a jq-based check in place of an MCP-authored script.
• Reference the runtime directly. The tool-builder Skill Template page on the RunWhen Skills Registry documents the config surface (INTERPRETER, GEN_CMD, CONFIG_ENV_MAP, SECRET_ENV_MAP, TIMEOUT_SECONDS) and links to the Robot source in rw-generic-codecollection/codebundles/tool-builder/.
• Understand the wider MCP model. The MCP end-to-end flow guide explains why LLMs dispatch existing tasks instead of connecting to infrastructure directly, and how the RunWhen MCP server enforces that boundary.
• Browse the registry first, always. The RunWhen Skills Registry is the source of production-tested Skill Templates. If your check matches something in the RunWhen Generic CodeCollection, deploy that instead of authoring a new bundle.
• Manage the CodeCollection as code. The Managing Context as Code (GitOps) guide covers keeping your rules, tasks, and knowledge under version control end-to-end.