Bug-Fix MT Generation Pipeline

Generates "fix bug" type main tasks by introducing mutations into environment codebases, then using integration test failures as task descriptions.

Quick Start

# Generate 5 bug-fix tasks for web_scraping
ct run mtgen-bugfix -e web_scraping --count 5

# With auto-promote to main_tasks/
ct run mtgen-bugfix -e web_scraping --count 5 --promote

The pipeline: ideate (LLM invents a realistic bug) → generate (LLM writes a patch) → validate (Docker: apply patch, run tests, check they fail) → optionally promote (copy to main_tasks/).

Expect ~8% overall validation success rate. Run --count 10 per environment to get 1-2 valid bugs.

How It Works

LLM mode (default): the fast model ideates realistic bugs for the environment, the smart model generates patches, then the pipeline validates them in Docker by checking that integration tests break.

CLI Reference

ct run mtgen-bugfix [OPTIONS]

Options:
  -e, --env TEXT       Environment name (required)
  --count INTEGER      Number of tasks to generate (default: 5)
  --promote            Auto-promote valid tasks to main_tasks/
  --model TEXT         Model for patch generation (default: the smart model alias)

Environment Support

The pipeline discovers installed environments with mtgen/bugfix.yml configs.

To add a new environment, you need:

• mutable_paths — where the source code lives
• integration_test_path — test file for the LLM prompt
• docker_compose_path — compose file for validation
• Path transformation in llm_pipeline.py for host-to-container mapping

Validation Output