Fix a bug with a regression test

This tutorial creates a disposable Python repository with one deliberately failing test. You will give the agent a reproducible failure rather than asking it to guess what is wrong. The same sequence applies to a real issue in your project.

Prerequisites

Install Nix with flakes enabled and Haskell Agent. Complete authentication before starting. The fixture uses Python's standard library; it requires no Python package installation. Run the following commands in a shell, not in the agent composer.

1. Create an isolated fixture

TUTORIAL_DIRECTORY=$(mktemp -d -t agent-documentation-tutorial.XXXXXX)
cd "$TUTORIAL_DIRECTORY"
cat > flake.nix <<'EOF'
{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05";
  outputs = { self, nixpkgs }: let
    systems = [ "aarch64-darwin" "x86_64-darwin" "aarch64-linux" "x86_64-linux" ];
  in {
    devShells = nixpkgs.lib.genAttrs systems (system: let
      pkgs = import nixpkgs { inherit system; };
    in { default = pkgs.mkShell { packages = [ pkgs.python3 pkgs.git ]; }; });
  };
}
EOF
cat > statistics.py <<'EOF'
def average(values):
    return sum(values) / len(values)
EOF
cat > test_statistics.py <<'EOF'
import unittest
from statistics import average

class AverageTests(unittest.TestCase):
    def test_two_values(self):
        self.assertEqual(average([2, 4]), 3)

    def test_empty_input(self):
        with self.assertRaises(ValueError):
            average([])
EOF
printf '__pycache__/\n' > .gitignore
nix develop -c sh -c 'git init && git add flake.nix flake.lock .gitignore statistics.py test_statistics.py'
nix develop -c git -c user.name='Documentation Example' -c user.email='example@example.invalid' commit -m 'Add reproducible average fixture'

Keep this shell open: $TUTORIAL_DIRECTORY identifies the temporary checkout. The flake supplies both Python and Git, and flake.lock records the resolved dependency revision. The commit is local; nothing is published.

2. Reproduce the failure

nix develop -c python -m unittest -v

The command must exit unsuccessfully. The ordinary two-item test passes, but the empty-input test raises ZeroDivisionError instead of the required ValueError. If Python cannot start, fix the environment before asking the agent to change application code.

3. Request the smallest correction

agent-cli --cwd "$TUTORIAL_DIRECTORY"

Do not add --worktree here: you already created a disposable checkout with the fixture you want the agent to inspect. Submit:

Run `nix develop -c python -m unittest -v` and reproduce the failure.
The contract is: average([]) raises ValueError; nonempty input returns its arithmetic mean.
Make the smallest correction in statistics.py. Do not weaken the tests, add dependencies,
or commit changes. Rerun the tests and report the exact command and result.

Review any tool approval before accepting it. The agent should inspect the files, run the test, modify statistics.py, and rerun the test. Its exact wording and tool sequence depend on the selected model.

4. Verify the patch independently

In the agent, run /diff. Then leave with /quit and execute:

nix develop -c python -m unittest -v
nix develop -c git diff --check
nix develop -c git diff -- statistics.py test_statistics.py
nix develop -c git status --short

Both tests should pass. Check that the correction explicitly rejects an empty list, preserves the average for nonempty input, and does not weaken or delete the tests. A plausible implementation is:

def average(values):
    if not values:
        raise ValueError("average requires at least one value")
    return sum(values) / len(values)

The exact exception message is not specified by this fixture. If your real application promises an error message, include that contract in a test too.

5. Diagnose an incomplete result

For an actual project, finish with its broader test suite and normal review process. This two-test exercise demonstrates the workflow; it does not prove correctness for every numeric input or production requirement.