Refactron 0.2.0. A measured look at deterministic refactoring at scale.

00 · Abstract

Refactron 0.2.0 analyzes a 100k-LOC tree in a median 11.13 seconds — 45% faster than 0.1.0-beta.2, with run-to-run variance compressed by 65%. On a real Python project the full analyze → plan → apply loop, including the 3-gate verifier running pytest on a shadow tree, completes in roughly five seconds.

This report measures the cost of safety-first deterministic refactoring[3]. Every figure is wall-clock; every harness is public.

01 · Headline result

vs 0.1.0-beta.2. Median dropped from 20.58s to 11.13s, and the long-tail variance compressed by 65% — predictable enough to drop into a pre-commit hook.

02 · Methodology

Reproducible by design.

Each measurement is the wall-clock real time reported by /usr/bin/time -p, captured over five runs after a single warm-up. We report median, min, and max — never a single best case. For the apply step, the fixture is freshly copied per iteration because the command mutates the tree.

03 · Results — synthetic

How fast can we walk a cold tree?

Synthetic fixtures generated fresh per run from bench/gen-fixture.ts[1] — mixed Python and TypeScript with every legacy pattern Refactron's ten transforms target. This isolates the analyze step.

Table 1. Median over five runs. Bars show the spread of all five measured runs; min and max in the same row. Δ% is the v0.1 → v0.2 median improvement.

04 · The full pipeline

Real fixture, real test suite.

Synthetic numbers isolate the analyze step. Real users run the whole loop. We measure against python-legacy-mini[2] — 9 files, 189 LOC, with a pytest suite that exercises every function the transforms touch.

Table 2. Median wall-clock per step, five runs each, fresh fixture copy per apply. End-to-end: ~5.2s from scan to atomically-written refactor.

05 · Inside the apply step

Three gates. All or nothing.

Of the 3.38-second apply budget on this fixture, roughly 3s is the test gate — pytest cold-start dominates at 9 files. On larger projects the ratio inverts: the test gate becomes bound by your suite, while plan and verification overhead stay roughly constant.

06 · Discussion

What this report does and doesn't claim.

07 · Reproducibility

Run it yourself.

Both bench scripts ship in the public repo. No special hardware, no proprietary fixtures, no telemetry. If your numbers come out meaningfully different on Apple Silicon, please open an issue.

git clone https://github.com/Refactron-ai/Refactron_Lib_TS
cd Refactron_Lib_TS && npm ci
bash bench/run-bench.sh

pip install pytest libcst requests
bash bench/run-pipeline-bench.sh