Regression bugs are the worst. They don't show up in your new feature branch — they wait until after deploy, when users start reporting that something that worked fine yesterday is now broken. The code didn't change in that area, you say. But it did. Somewhere.
I've spent hours chasing regressions that turned out to be a single line change in a completely unrelated module. The key is to stop guessing and start bisecting.
Step 1: Confirm It's a Regression
Before you start blaming the latest deploy, verify that the bug is actually new. Check your monitoring dashboards, look at error logs, and ask: 'When did this last work?' If you have a CI pipeline that runs nightly, find the last passing run and the first failing run. That gives you a commit range.
If you don't have CI, ask users for a timestamp. A support ticket that says 'I noticed this yesterday afternoon' is gold — it narrows the window.
Always capture the exact failing input and the exact expected output. A fuzzy description like 'the page doesn't load' is useless. Get a screenshot, a HAR file, or a curl command that reproduces the issue.
Step 2: Create a Minimal Reproducer
A reproducer is the single most important tool. Without it, you're debugging blind. Strip away everything unnecessary: authentication, extraneous data, complex UI interactions. Aim for a script that crashes in one line.
Here's an example from a real regression I fixed in a Node.js API. The bug was that a specific GET endpoint returned 500 when a query parameter was missing. The reproducer was a one-liner:
curl -v 'http://localhost:3000/api/users?sort=name&page=1' # returns 200
curl -v 'http://localhost:3000/api/users?page=1' # returns 500Once you have a reproducer, commit it to a test file. Don't delete it after fixing — it becomes your regression test.
Step 3: Binary Search with git bisect
Git bisect is the fastest way to find the offending commit. You give it a known good commit (e.g., from last week) and a known bad commit (the current one), and it does a binary search, checking out commits for you to test.
The workflow looks like this:
# Start bisect with good and bad commits
git bisect start
git bisect bad # current commit is bad
git bisect good v1.2.0 # last release was good
# Git checks out a commit halfway. You run your reproducer:
curl -v 'http://localhost:3000/api/users?page=1'
# If it fails: git bisect bad
# If it passes: git bisect good
# Repeat until git shows the first bad commit:
# e4f7a3c is the first bad commit
# End bisect
git bisect resetIf your reproducer is a script, you can automate the entire bisect with `git bisect run`.
git bisect start
git bisect bad
git bisect good v1.2.0
git bisect run ./test_reproducer.shA Real-World War Story: The Phantom 500
I once debugged a regression where a production API endpoint would intermittently return 500 errors. The errors only appeared under load, and only for users with a specific account type. My reproducer was a Python script that sent 50 concurrent requests with the right headers.
I bisected over 200 commits, but the first bad commit pointed to a change in a completely different microservice — a logging library upgrade. That upgrade changed the order of asynchronous callbacks, causing a race condition in our request middleware. The fix was to pin the logging version and add a mutex. The regression test I wrote runs daily in CI and catches any similar issues.
Step 4: Understand the Root Cause
Once bisect identifies the commit, examine the diff. Look for changes that could affect the behavior you're seeing. Common culprits:
- Changes to input validation or parsing
- Refactored logic that accidentally removed a needed condition
- Dependency updates that alter behavior
- Configuration changes that disable a feature
Don't assume the bug is in the diff itself — sometimes the diff exposes a pre-existing latent bug.
- arrow_rightCheck if the commit introduced a new dependency or updated an existing one.
- arrow_rightLook for removed lines — often a regression is caused by deleting a safety check.
- arrow_rightVerify that the commit's author intended the change (maybe it was a merge gone wrong).
Step 5: Write the Regression Test First
Before you fix the bug, write a test that reproduces it. This ensures your fix actually works and prevents the bug from coming back. The test should be specific — use the exact input that triggered the bug.
Here's a Jest test for the earlier Node.js example:
test('GET /api/users returns 200 when sort param is missing', async () => {
const response = await request(app)
.get('/api/users')
.query({ page: 1 });
expect(response.status).toBe(200);
});Run the test against the bad commit — it should fail. Then apply your fix and run the test — it should pass. Then run the full test suite to ensure nothing else broke.
Step 6: Fix and Verify
Now fix the code. In the Node.js example, the fix was to add a default value for the sort parameter:
const sort = req.query.sort || 'created_at';After the fix, run the regression test and the full test suite. Then deploy and monitor.
Preventing Regressions
The best fix is one that prevents future regressions. Beyond writing a regression test, consider:
- Adding a CI step that runs `git bisect run` on a failing test to automatically identify the culprit in a pull request.
- Using feature flags to isolate risky changes.
- Implementing contract tests for API endpoints.
- Reviewing code changes with a focus on 'what could break?'
A culture of writing regression tests for every bug fix pays off exponentially.
of regressions are caught by automated tests when a regression test is written at the time of fix
When the Regression Is Not in Your Code
Sometimes the regression is caused by an external dependency — a database driver, a third-party API, or infrastructure change. In that case, bisect won't help directly. You need to check changelogs, compare response payloads, and use network inspection tools like Wireshark or mitmproxy.
For example, I once spent a day chasing a regression that turned out to be a new version of the PostgreSQL driver that changed the default timezone handling. The fix was to pin the driver version.
Summary
- 1Confirm it's a regression (find the last good version).
- 2Create a minimal, automated reproducer.
- 3Use git bisect (or equivalent) to find the exact commit.
- 4Understand the root cause from the diff.
- 5Write a regression test that fails without the fix.
- 6Apply the fix and verify with the test.
- 7Prevent future regressions with CI and testing culture.
A regression without a regression test is just a bug waiting to happen again.
Frequently asked questions
What is a regression bug?
A regression bug is a defect that appears in a previously working feature after a code change. It means something that used to work no longer works, often introduced by an unrelated commit.
How does git bisect work?
Git bisect performs a binary search over a range of commits. You mark a known good commit and a known bad commit. Git then checks out a commit halfway between them, you test it and mark it good or bad, and the process repeats until the first bad commit is identified.
What if git bisect is not practical (e.g., manual tests)?
For manual or slow tests, you can still bisect by writing a script that automates the test (e.g., using a headless browser or API calls). If automation is impossible, you can manually skip commits or use a custom bisect script that narrows the search.
How do I prevent regressions from recurring?
Write a regression test that reproduces the exact bug scenario, run it in CI for every push, and consider adding integration or end-to-end tests for critical paths. Also, use feature flags to gradually roll out changes.