API automation

The layer most suites under-use

A UI test for "creating a case with a blank title is rejected" takes eight seconds, drives a browser, and can fail because a button moved. The same rule tested against the endpoint takes 200 milliseconds and fails only when the rule itself breaks.

That is the trade the first lesson of this track was about, made concrete: push each test as low as it can go while it still says something true about what the user gets. Validation rules, permissions, error codes and business logic are almost always lower than the browser.

You have already been using this layer. The previous lesson arranged data through the API because doing it through the UI was slow and brittle. This lesson is the same tool pointed at the thing under test rather than at the setup.

Playwright tests APIs without a browser

No new dependency, no second framework:

import { test, expect } from "@playwright/test";

const PROJECT = process.env.TF_PROJECT!;   // your sandbox project's slug

test("rejects a case with a blank title", async ({ request }) => {
  const res = await request.post(`/api/v1/projects/${PROJECT}/cases`, {
    data: { title: "" },
  });

  expect(res.status()).toBe(422);
  const { error } = await res.json();
  expect(error.code).toBe("validation_error");
  expect(error.details.map((d) => d.field)).toContain("title");
});

Every write route on this API is project-scoped — the slug is part of the path, and there is no /api/v1/cases collection above it. Reading /api/v1/openapi once, before writing any of these tests, is cheaper than discovering the shape one 404 at a time.

The request fixture is an HTTP client with the config's baseURL and its own cookie jar. Two things follow from that: it does not open a browser, so these tests run in milliseconds; and it can share authentication with your UI tests rather than needing a separate login mechanism.

Note the assertion style. expect(res.status()) is a plain value comparison, not a web-first assertion — there is nothing to poll, because an HTTP response either arrived or did not. The retry rules from the assertions lesson apply to locators; here the ordinary form is correct.

Authenticating once

// playwright.config.ts
use: {
  baseURL: process.env.TF_BASE_URL,
  extraHTTPHeaders: {
    Authorization: `Bearer ${process.env.TF_API_KEY}`,
  },
},

An API key from the environment is the simplest correct answer, and it is what TestForge itself expects. When a test needs a different identity — checking that a viewer cannot delete a suite — build a client for it rather than mutating the shared one:

test("a viewer cannot delete a suite", async ({ playwright }) => {
  const viewer = await playwright.request.newContext({
    baseURL: process.env.TF_BASE_URL,
    extraHTTPHeaders: { Authorization: `Bearer ${process.env.TF_VIEWER_KEY}` },
  });

  const res = await viewer.delete(`/api/v1/projects/${PROJECT}/suites/${suiteId}`);
  expect(res.status()).toBe(403);

  await viewer.dispose();
});

Authorization tests are the highest-value thing on this layer, and they are close to impossible through a UI that simply hides the button. A hidden button is not a permission check — the endpoint is — and this is how you find out which one your application actually has. The manual track made the same argument about checking authorization by URL first; this is its automated form.

What to assert on a response

More than the status code, and less than everything:

const path = `/api/v1/projects/${PROJECT}/cases`;
const res = await request.post(path, { data: { title: "TC-12", suiteId } });

expect(res.status()).toBe(201);                       // 1. status
expect(res.headers()["content-type"]).toContain("application/json");

const { id, displayId } = await res.json();           // create answers with ids only
expect(displayId).toMatch(/^TC-[A-Z0-9-]+-\d{3}$/);   // 2. shape, not exact value

const created = await (await request.get(`${path}/${id}`)).json();
expect(created).toMatchObject({ title: "TC-12", suiteId, priority: "MEDIUM" });
expect(new Date(created.createdAt).getTime()).toBeGreaterThan(0);

Two habits in that snippet. toMatchObject is the workhorse: it checks the fields you name and ignores the rest, so a new field added to the response does not break forty tests. Asserting deep equality against a whole payload is the API equivalent of a CSS selector chain — it fails on changes that are not defects. Note that priority is not something the test sent: asserting the server's default is how you find out when someone changes it.

And assert shape for anything the server generates. displayId matching TC-<SLUG>-<nnn> is a real contract, one the JUnit capstone depends on; displayId === "TC-DEMO-012" is today's counter. The opaque id is worth asserting nothing about beyond being a non-empty string.

It is also worth noticing what this endpoint does not return. A create that answers with identifiers rather than the whole record is common, and it means verification is a second request — which is no bad thing, because reading the resource back is a stronger check than trusting the response to the write.

Status codes worth being precise about

A test that accepts "any error" is barely a test. The difference between these is usually a real defect:

Code	Means	Common bug it catches
400	Malformed request	Validation returning 500 instead
401	Not authenticated	An endpoint that forgot to require auth
403	Authenticated, not allowed	The big one — permissions not enforced server-side
404	Not found	Leaking existence: returning 403 vs 404 for other users' records
409	Conflict	Duplicate handling that silently overwrites
422	Understood, semantically invalid	Business rules bypassed

401 versus 403 and 403 versus 404 are the two pairs worth testing explicitly. The second is subtler than it looks: returning 403 for a record that exists but belongs to someone else tells an attacker it exists. Whichever your application chooses, it should choose consistently, and a test is how that stays true.

TestForge is a worked example of that choice, and you can prove it in one request. Post a case to a project you are not a member of and you get 404, not 403 — the API declines to confirm that the project exists at all:

test("a project you are not in is indistinguishable from one that does not exist", async ({ request }) => {
  const res = await request.post("/api/v1/projects/someone-elses-project/cases", {
    data: { title: "probe" },
  });
  expect(res.status()).toBe(404);
});

Testing the error paths is the point

The happy path is usually already covered by a UI test. The value of this layer is everything the UI cannot easily reach:

const cases = [
  { data: {}, field: "title", why: "no fields at all" },
  { data: { title: "" }, field: "title", why: "blank title" },
  { data: { title: "TC-1", priority: "URGENT" }, field: "priority", why: "a priority off the list" },
  { data: { title: "TC-1", suiteId: "does-not-exist" }, field: "suiteId", why: "a suite from another project" },
];

for (const c of cases) {
  test(`rejects ${c.why}`, async ({ request }) => {
    const res = await request.post(`/api/v1/projects/${PROJECT}/cases`, { data: c.data });
    expect(res.status()).toBe(422);
    const { error } = await res.json();
    expect(error.details.map((d) => d.field)).toContain(c.field);
  });
}

Note what the table asserts. All four are 422, so a test that only checked the status would pass on a server that rejected every one of them for the wrong reason — naming the offending field is what makes the row a real test. The fourth row is the interesting one: a suite id that belongs to a different project is a validation failure, not a 404, because admitting "that suite exists, just not here" would be the same leak the previous section closed.

Generating tests from a table is legitimate here in a way it is not in the UI: each case is one fast request, the failure message names which row failed, and adding the fifteenth boundary costs a line. Keep them as separate test() calls rather than a loop inside one test, so a failure reports the specific case and one failing row does not hide the four after it.

The hybrid test is where this pays off most

test("TC-SHOP-31 a case created by API appears in the case list", async ({ page, request }) => {
  const title = `login ${Date.now()}`;
  const res = await request.post(`/api/v1/projects/${PROJECT}/cases`, {
    data: { title, suiteId },
  });
  const created = await res.json();

  await page.goto(`/projects/${PROJECT}/cases`);
  await expect(page.getByRole("row", { name: title })).toBeVisible();
  await expect(page.getByText(created.displayId)).toBeVisible();
});

Arrange below, act and assert above. This is the shape most of a mature suite ends up in, and it is why the previous lesson and this one belong together: the API is both a thing to test and the tool that makes UI tests fast and independent.

The reverse direction is worth knowing too — perform an action in the UI, then verify through the API that the stored state is right. A form that appears to save but writes the wrong field is a bug the screen will happily hide from you.

What this layer will not tell you

Being honest about the limits keeps the pyramid argument honest:

• That the feature works for a person. Every endpoint can be correct while the button that calls them is disabled.
• Anything about rendering, layout, or accessibility.
• That the client sends what you think it sends. Your test constructs the request; the real application constructs a different one. This is the gap contract testing exists to close, and it is on the Beyond Functional track.

So the split is not "API tests instead of UI tests". It is: the rules, the permissions and the error paths below; a small number of journeys a user actually takes above.

Where TestForge fits

The capstone uses this lesson's tooling for real: /api/v1/junit is an endpoint, your upload is a POST with a multipart body, and the run it creates is something you can then read back and assert on. Practising against /api/v1/projects/<slug>/cases in your sandbox project now is exactly the muscle the capstone needs — the same project-scoped shape T2's API testing lesson had you sending by hand.

GET    /api/v1/openapi                                 every route, machine-readable
GET    /api/v1/projects/<slug>/cases                   list
POST   /api/v1/projects/<slug>/cases                   create → { id, displayId }
DELETE /api/v1/projects/<slug>/cases/<caseId>          soft delete
POST   /api/v1/projects/<slug>/suites                  create → { id, name, parentId }
Header: Authorization: Bearer <API_KEY>

Worth trying once for the shape of it: create a case through the API, upload a JUnit result whose test name carries that case's displayId, and read the run back to confirm the match landed. That is the whole product loop in three requests, and it is the thing the last two lessons of this track assemble properly.

Next: running the suite in CI with GitHub Actions — workflows, artifacts, and keeping the pipeline under ten minutes so people actually wait for it.