Guide

Test Isolation Strategies: From Test Code to CI Pipeline

Your tests pass alone but fail together. Here's how to fix the shared state causing a third of all test failures.

Dhaval Shreyas

Co-founder & CEO at Pie

9 min read

What Test Isolation Actually Means

A test is isolated when it can run by itself, before any other test, after any other test, or alongside any other test, and produce the same result every time. No test should affect or be affected by any other test.

Simple in concept. In practice, isolation requires discipline across four dimensions:

• State isolation — Tests don’t share mutable data. One test creating a user record shouldn’t affect another test expecting an empty database.
• Environment isolation — Tests don’t assume specific machine configurations, network conditions, or file system states beyond what they explicitly set up.
• Time isolation — Tests don’t depend on real clock time. A test that passes at 11:59 PM but fails at 12:01 AM isn’t isolated.
• Execution isolation — Tests don’t depend on running in a specific order or with specific parallelism settings.

When all four are addressed, you have truly isolated tests. Most teams achieve one or two and wonder why their suite is still flaky.

Why Isolation Prevents Flaky Tests

Flaky tests fail intermittently without code changes. The cause is always some form of hidden dependency that varies between runs.

Isolation eliminates hidden dependencies by making all dependencies explicit.

Test A creates a user with email “[email protected]”. Test B, running later, tries to create the same user. If they run in one order, both pass. In the other order, Test B fails with a duplicate key error.

This isn’t a bug in your code. It’s a bug in your test architecture. Both tests assume they own the database state, but neither actually does.

With proper isolation, each test either:

• Runs against its own database instance
• Wraps operations in transactions that roll back
• Uses unique identifiers that can’t collide
• Cleans up its own data in teardown

The specific technique matters less than the principle: after a test runs, the world should look exactly as it did before.

Test-Level Isolation

Isolation starts in the test code itself. Three patterns form the foundation.

1. Fresh Context Per Test

Start each test from a clean slate. In browser testing, this means a new browser context or incognito session. In API testing, this means fresh authentication tokens. In unit tests, this means reconstructing objects rather than reusing them.

// Good: Fresh context
beforeEach(async () => {
  context = await browser.newContext();
  page = await context.newPage();
});

afterEach(async () => {
  await context.close();
});

// Bad: Shared context
const page = await browser.newPage(); // Created once, reused everywhere

The overhead of creating fresh contexts is negligible compared to the debugging time saved.

2. Unique Test Data

Never hardcode test data that could collide. Use unique identifiers, timestamps, or UUIDs to ensure each test operates on its own records.

// Good: Unique data
const email = `test-${Date.now()}@example.com`;
const user = await createUser({ email });

// Bad: Hardcoded data
const user = await createUser({ email: '[email protected]' });

This simple change eliminates an entire category of order-dependent failures.

3. Setup and Teardown Symmetry

Whatever your test creates, it should destroy. Whatever state it modifies, it should restore. This isn’t just cleanup, it’s a contract with every other test in your suite.

let createdUserId;

beforeEach(async () => {
  const user = await createTestUser();
  createdUserId = user.id;
});

afterEach(async () => {
  if (createdUserId) {
    await deleteUser(createdUserId);
  }
});

Some teams rely on “cleanup runs” before test execution. This is fragile. If a previous run crashed, the cleanup didn’t happen. Symmetric teardown in each test is more reliable.

Data Layer Isolation

Databases are the most common source of test pollution. Moving up from test code to the data layer, three approaches work well.

1. Transaction Rollback

Wrap each test in a database transaction that gets rolled back after assertions complete. Fast and simple, but doesn’t test commit behavior.

beforeEach(async () => {
  await db.query('BEGIN');
});

afterEach(async () => {
  await db.query('ROLLBACK');
});

2. Schema Isolation

Create a fresh schema or database for each test run. More overhead, but tests real commit behavior.

const schemaName = `test_${process.env.CI_JOB_ID || Date.now()}`;
await db.query(`CREATE SCHEMA ${schemaName}`);
await db.query(`SET search_path TO ${schemaName}`);

3. Containerized Databases

For integration tests that need real database behavior, spin up fresh containers. Tools like Testcontainers make this straightforward.

const { PostgreSqlContainer } = require('testcontainers');

let container;

beforeAll(async () => {
  container = await new PostgreSqlContainer().start();
  process.env.DATABASE_URL = container.getConnectionUri();
});

afterAll(async () => {
  await container.stop();
});

Each test run gets its own database. No state leaks between runs. No conflicts with other developers or CI jobs.

Choose based on your testing needs. Transaction rollback handles most cases. Containers handle the rest.

Service Layer Isolation

Service dependencies complicate isolation. When your application calls external APIs or microservices, each becomes a potential source of non-determinism.

1. Mock External Dependencies

External services introduce variability your tests can’t control. Third-party APIs have rate limits, network latency, and occasional outages. Mock them.

// Mock payment processor
await page.route('**/api.stripe.com/**', route => {
  route.fulfill({
    status: 200,
    body: JSON.stringify({ success: true, charge_id: 'ch_test' })
  });
});

Mocking isn’t cheating. It’s isolating your code from external variability. Just ensure your mocks reflect realistic responses.

2. Service Virtualization

Record real service responses once, then replay them during tests. Tools like WireMock, Mountebank, or Playwright’s route interception make this manageable.

// Intercept and stub auth service
await page.route('**/auth-service/verify', route => {
  route.fulfill({
    status: 200,
    body: JSON.stringify({ valid: true, user_id: 'test-user' })
  });
});

3. Contract Testing

Verifies your mocks stay synchronized with real services. Pact or Spring Cloud Contract ensure your stubs reflect reality.

The goal: tests should fail only when your code is wrong, not when a dependent service is slow or unavailable.

Pipeline-Level Isolation

Isolation that works locally but breaks in CI isn’t isolation. Your pipeline needs to enforce it at the infrastructure level.

1. Randomize Test Order

Run tests in random order by default. If tests pass alphabetically but fail when randomized, you have isolation problems. Jest’s --randomize flag or pytest’s pytest-random-order plugin expose these issues.

# GitHub Actions example
- name: Run tests with random order
  run: npm test -- --randomize

2. Parallelize Aggressively

Run tests across multiple workers. If they fail under parallelization, they’re sharing state they shouldn’t.

- name: Run tests in parallel
  run: npm test -- --workers=4

3. Fail on Flaky Detection

Playwright’s --fail-on-flaky-tests flag catches tests that pass on retry. A test that needs retrying is a test with isolation problems.

When Isolation Isn’t Enough

Some tests legitimately need integration. End-to-end tests verifying multi-service workflows can’t be fully isolated without mocking away the behavior you’re actually testing.

Three approaches help manage these tests:

• Accept longer feedback cycles. True E2E tests against real services run slower. Execute them less frequently, but keep them in the suite for release gates.
• Use dedicated test environments. Shared staging environments cause collisions. Per-branch or per-PR environments eliminate the problem.
• Use a platform that handles isolation for you. Manual isolation strategies create maintenance overhead. Autonomous QA platforms run each test in a fresh browser context with no shared state, removing the burden of managing test infrastructure.

The teams that scale their test suites successfully don’t fight isolation problems manually. They use tooling that makes isolation automatic.

Build Isolation In From Day One

Test isolation isn’t a nice-to-have you add after the suite gets painful. Build it in from the start, and your tests scale. Bolt it on later, and you’re refactoring under pressure while the suite continues to rot.

The patterns here work. Fresh contexts per test. Unique data. Symmetric setup and teardown. Mocked externals. Containerized dependencies. None of them are complicated individually. Together, they eliminate the shared state that causes most test failures.

Invest in isolation now. Your future self will stop cursing past you.

Dhaval Shreyas

Co-founder & CEO at Pie

13 years building mobile infrastructure at Square, Facebook, and Instacart. Payment systems, video platforms, the works. Now building the QA platform he wished existed the whole time. LinkedIn →