Skip to main content

Two PRs Pass Their Tests. Together, They Fail.

Published:
TTomohiro Iida
AI agentsNode.jsGitHubTestingEngineering
Two separately passing modules, one producing totalCents and one expecting total, fail at the combined API contract.

Imagine receiving two AI-assisted pull requests, each with green local tests. Before accepting them together, test the actual combined code. A passing test tells you something about a specific input and revision; it does not automatically establish compatibility with another change.

This is a fictional educational example using Node.js's built-in node:test. It is not a Netsujo or customer incident, nor does it reproduce a textual Git merge conflict. It demonstrates an application-level API contract mismatch.

How both isolated tests pass

In this example, PR A changes the order API amount from floating-point total: 49.95 to integer minor units totalCents: 4995.

agent-a.mjs:
export function fetchOrder() {
  return { id: 'demo-order-1', totalCents: 4995 };
}

Its isolated test, agent-a.test.mjs:

import test from 'node:test';
import assert from 'node:assert/strict';
import { fetchOrder } from './agent-a.mjs';

test('producer returns totalCents', () => {
  assert.deepEqual(fetchOrder(), {
    id: 'demo-order-1', totalCents: 4995
  });
});

PR B exports the amount for a CSV file. However, its isolated test uses a stale fixture matching the old API.

agent-b.mjs:
export function exportAmount(order) {
  return order.total.toFixed(2);
}
agent-b.test.mjs:
import test from 'node:test';
import assert from 'node:assert/strict';
import { exportAmount } from './agent-b.mjs';

test('old fixture passes', () => {
  assert.equal(
    exportAmount({ id: 'demo-order-1', total: 49.95 }),
    '49.95'
  );
});

The two tests both pass:

node --test agent-a.test.mjs agent-b.test.mjs
# tests 2 / pass 2 / fail 0

That proves the consumer works against its old fixture. It has not been tested against the new producer response.

Use the real producer response: one expected failure

combined.test.mjs:
import test from 'node:test';
import assert from 'node:assert/strict';
import { fetchOrder } from './agent-a.mjs';
import { exportAmount } from './agent-b.mjs';

test('producer and consumer contract', () => {
  assert.equal(exportAmount(fetchOrder()), '49.95');
});
node --test combined.test.mjs
# tests 1 / pass 0 / fail 1 / exit 1 (intentional)

The response from fetchOrder() no longer contains total. The expression order.total.toFixed(2) throws a TypeError. Git does not need to detect any overlapping text edits for this mismatch to arise. It only becomes visible when the two components run together.

Explore upcoming events: dates, venues and participation requirementsΒ β†’

Align the contract and rerun the same combination

agent-b-fixed.mjs:
export function exportAmount(order) {
  return (order.totalCents / 100).toFixed(2);
}
repaired.test.mjs:
import test from 'node:test';
import assert from 'node:assert/strict';
import { fetchOrder } from './agent-a.mjs';
import { exportAmount } from './agent-b-fixed.mjs';

test('aligned contract passes', () => {
  assert.equal(exportAmount(fetchOrder()), '49.95');
});
node --test repaired.test.mjs
# tests 1 / pass 1 / fail 0

This simple conversion covers only the sample amount. It is not a general guarantee about currency rounding, multi-currency representation, compatibility or error handling. Real systems need their own contract and edge-case checks.

Four checks before you accept the PRs

1. Which commit was green? Record the tested or reviewed SHA and whether it changed.

2. What did the test mock? Check whether a stale fixture hides an API contract change.

3. What code was combined? Use the actual producer and consumer from the target candidate.

4. Was it retested after integration? A new main commit or revised PR changes the evidence subject.

An independent reviewer should also inspect the raw diff, exact revision and actual test execution instead of relying solely on the author's summary. That independence does not replace testing the combined candidate.

The originating discussion of independent AI review asks whether two agents examining the same PR amount to independent verification. This tutorial extends the question to two separately green changes that must operate together. For worktrees, integration ownership and sequencing, see Netsujo's existing guide.

The seven-file synthetic exercise was executed using Node.js v24.22.1 in an isolated preparation directory: two isolated PASS, one intentional combined FAIL, and one repaired PASS. This is not source-branch required CI, independent QC, or evidence that actual AI-generated PRs were safe to merge.

Bring a small reproducible question like this to a MiyakodeIT study session. Find forthcoming meetups in the events listing.

Source data

The figures cited in this article are based on primary data in the Miyako de IT annual statistics report. It publishes yearly event counts, venue distribution, and event-format data.

Written by

T

Tomohiro Iida

Event artwork published by the organizer (Nakagyo Ward, Kyoto)Registration open

Dec 5 (Sat)

Original event title

Time15:00–18:00 JST
AreaNakagyo Ward, Kyoto