mirror of
https://github.com/addyosmani/agent-skills.git
synced 2026-08-12 18:07:26 +02:00
docs(performance): add Step 4 (verify with keep-or-revert)
The optimization workflow lists five steps but only documents three; VERIFY has no section of its own. Add it: re-measure under baseline conditions, change one thing at a time, beat run-to-run variance, and keep only what pays for itself. Neutral and test-breaking results revert. Also log every attempt, including the reverted ones, so a discarded idea does not get re-run later. Add the corresponding Common Rationalizations, Red Flags, and Verification entries.
This commit is contained in:
@@ -33,7 +33,7 @@ Measure before optimizing. Performance work without measurement is guessing —
|
||||
1. MEASURE → Establish baseline with real data
|
||||
2. IDENTIFY → Find the actual bottleneck (not assumed)
|
||||
3. FIX → Address the specific bottleneck
|
||||
4. VERIFY → Measure again, confirm improvement
|
||||
4. VERIFY → Measure again; keep or revert
|
||||
5. GUARD → Add monitoring or tests to prevent regression
|
||||
```
|
||||
|
||||
@@ -289,6 +289,41 @@ app.use('/static', express.static('public', {
|
||||
res.set('Cache-Control', 'public, max-age=300'); // 5 minutes
|
||||
```
|
||||
|
||||
### Step 4: Verify (Keep or Revert)
|
||||
|
||||
A fix is a hypothesis until you re-measure. This step decides whether it survives.
|
||||
|
||||
**Re-measure the way you measured the baseline:** same command, same conditions, same fixed budget (wall-clock, sample count, or request count). A baseline taken on a cold cache against a result taken on a warm one measures the cache, not your change.
|
||||
|
||||
**Change one thing at a time.** Three optimizations landed together produce one number, and you cannot attribute it. If they must ship together, measure each in isolation first.
|
||||
|
||||
**Beat the noise, not just the mean.** Repeat the measurement and compare the delta against run-to-run variance. A 3% gain inside ±5% variance is not a gain; it is a different sample.
|
||||
|
||||
Then decide, strictly:
|
||||
|
||||
| Result vs. baseline | Action |
|
||||
|---|---|
|
||||
| Past the threshold, tests green | **Keep.** Commit with the before/after numbers in the message. |
|
||||
| Within noise (no measurable change) | **Revert.** |
|
||||
| Worse | **Revert.** |
|
||||
| Improved, but a test went red | **Revert.** A regression wearing a win's clothing. |
|
||||
|
||||
**"Neutral" is a revert, not a keep.** This is the step teams skip: the change is already written, throwing it away feels wasteful, so it lands unmeasured, and the codebase accretes complexity that never bought anything. Code you keep, you maintain forever. Make it pay for itself.
|
||||
|
||||
**Correctness gates the metric.** The suite stays green *and* the number moves. An "optimization" that wins by dropping work the product needed (skipping a validation, caching something that must be fresh, removing an `await` that was load-bearing) is a regression, not a win.
|
||||
|
||||
#### Log every attempt, including the reverted ones
|
||||
|
||||
Reverted work leaves no trace in git history, which is exactly why the same dead idea gets tried again next quarter. Keep a short ledger so a discarded idea stays discarded:
|
||||
|
||||
| Idea | Baseline → Result | Verdict | Why |
|
||||
|---|---|---|---|
|
||||
| Memoize the row component | INP 240ms → 235ms | reverted | Inside noise (±15ms). Rows weren't the bottleneck. |
|
||||
| Virtualize the list | INP 240ms → 90ms | kept | Long tasks gone from the trace. |
|
||||
| Preconnect to the API origin | LCP 2.8s → 2.8s | reverted | Already same-origin. |
|
||||
|
||||
A section in the PR description or a `PERF.md` in the repo both work. What matters is that the next person (or the next agent) reads it before proposing an experiment, and doesn't re-run one that already failed.
|
||||
|
||||
## Performance Budget
|
||||
|
||||
Set budgets and enforce them:
|
||||
@@ -326,6 +361,9 @@ For detailed performance checklists, optimization commands, and anti-pattern ref
|
||||
| "This optimization is obvious" | If you didn't measure, you don't know. Profile first. |
|
||||
| "Users won't notice 100ms" | Research shows 100ms delays impact conversion rates. Users notice more than you think. |
|
||||
| "The framework handles performance" | Frameworks prevent some issues but can't fix N+1 queries or oversized bundles. |
|
||||
| "It didn't help much, but it doesn't hurt" | Neutral changes are a revert. You pay maintenance on them forever and got nothing back. |
|
||||
| "We already wrote it, may as well keep it" | Sunk cost. The measurement doesn't care how long the change took to write. |
|
||||
| "The improvement is obvious, no need to re-measure" | Then re-measuring is cheap and proves it. Unmeasured wins are how neutral complexity lands. |
|
||||
|
||||
## Red Flags
|
||||
|
||||
@@ -336,12 +374,20 @@ For detailed performance checklists, optimization commands, and anti-pattern ref
|
||||
- Bundle size growing without review
|
||||
- No performance monitoring in production
|
||||
- `React.memo` and `useMemo` everywhere (overusing is as bad as underusing)
|
||||
- Optimizations kept without a re-measurement that justifies them
|
||||
- Several optimizations bundled into one measurement, so no single change can be attributed
|
||||
- A "win" that required a test to be changed, skipped, or deleted
|
||||
- The same failed optimization attempted more than once because nobody recorded the first attempt
|
||||
|
||||
## Verification
|
||||
|
||||
After any performance-related change:
|
||||
|
||||
- [ ] Before and after measurements exist (specific numbers)
|
||||
- [ ] The result was re-measured the same way as the baseline (same command, same conditions)
|
||||
- [ ] The improvement exceeds run-to-run variance, not just the mean
|
||||
- [ ] Changes that didn't beat the baseline were reverted, not kept as neutral
|
||||
- [ ] Attempts are logged, kept and reverted alike, so a dead idea isn't re-run
|
||||
- [ ] The specific bottleneck is identified and addressed
|
||||
- [ ] Core Web Vitals are within "Good" thresholds
|
||||
- [ ] Bundle size hasn't increased significantly
|
||||
|
||||
Reference in New Issue
Block a user