Parity testing

When a plugin is ported from WinterCMS, its existing clients define the contract: the Go port must answer every request the way the PHP backend did. tide turns that rule into tests. It records the reference backend's real responses as YAML fixtures, replays the same requests against the port, and diffs the responses after masking the values that legitimately differ. WinterCMS has no counterpart.

Flows#

A fixture is a tide.Flow: a versioned, ordered list of steps, each a request with its recorded response. You write a spec, a flow with requests only:

version: 1
name: blog-posts
description: List the posts of one blog
steps:
  - id: list-posts
    route_id: GET /api/blog/posts
    request:
      method: GET
      path: /api/blog/posts?page=1

Recording sends each request to the reference backend and fills in the responses. Replaying sends them to the port and compares status, a fixed set of contract headers and the body. JSON bodies are compared structurally after masking id, *_id and *_ids values and *_at timestamps; other bodies byte for byte:

ctx := context.Background()
// The reference (PHP) backend and two Go ports. IDs and *_at timestamps
// legitimately differ; the second port changed a title.
reference := backend(`{"data":[{"id":12,"title":"Hello","created_at":"2026-09-30T10:00:00+00:00"}]}`)
defer reference.Close()
port := backend(`{"data":[{"id":3,"title":"Hello","created_at":"2026-10-01T08:30:00+00:00"}]}`)
defer port.Close()
broken := backend(`{"data":[{"id":3,"title":"hello","created_at":"2026-10-01T08:30:00+00:00"}]}`)
defer broken.Close()

raw, err := os.ReadFile("testdata/docs/posts-spec.yaml")
if err != nil {
	fmt.Println(err)
	return
}
spec, err := tide.ParseFlow(raw)
if err != nil {
	fmt.Println(err)
	return
}
// Record the reference once; the flow is what testdata/parity keeps.
flow, err := tide.RecordFlow(ctx, spec, tide.RecordConfig{Target: reference.URL})
if err != nil {
	fmt.Println(err)
	return
}
for _, target := range []string{port.URL, broken.URL} {
	res, err := tide.ReplayFlow(ctx, flow, tide.ReplayConfig{Target: target})
	var mismatch *tide.MismatchError
	if errors.As(err, &mismatch) {
		res = mismatch.Result // a difference is an error carrying the result
	} else if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println("ok:", res.OK)
	for _, step := range res.Steps {
		for _, d := range step.Diffs {
			fmt.Printf("  %s %s: want %s, got %s\n", step.ID, d.Path, d.Expected, d.Actual)
		}
	}
}
// Output:
// ok: true
// ok: false
//   list-posts $.data[0].title: want "Hello", got "hello"

The first port returns other IDs and timestamps and passes; the second changed a title and fails with the JSON path of the difference. A difference makes tide.ReplayFlow return a tide.MismatchError that carries the full tide.Result.

The parity commands#

The summer CLI wraps tide. Run the reference backend and the port on loopback addresses:

summer parity:record --spec testdata/parity/posts.spec.yaml --target http://127.0.0.1:8000 --output testdata/parity/posts.yaml --vars /tmp/parity/vars.yaml
summer parity:replay --fixtures testdata/parity --target http://127.0.0.1:8080 --vars /tmp/parity/vars.yaml

parity:proxy records a real client instead: it runs a reverse proxy in front of the reference backend, and each named session of traffic becomes one fixture. Point the existing frontend at the proxy and click through a feature. The proxy binds to and forwards to loopback addresses only.

Values captured during a flow, such as tokens and created IDs, live in a variables file (--vars) readable only by its owner, and fixtures refer to them as {{name}} placeholders. Recording refuses to write a fixture that still holds a token- or password-shaped value, so credentials do not end up in committed fixtures. Keep the variables file outside the repository.

A route manifest (tide.Manifest) lists a plugin's routes with their auth groups, status and cases; parity:record --manifest records the missing cases in batches, and parity:replay --manifest reports coverage. The Console utilities page lists every flag.

Broadcast goldens#

Realtime side effects are part of the contract too. summer parity:broadcasts runs a flow against the reference backend while a fake Centrifugo server (tide.NewCentrifugoRecorder) records the publications the backend sends, and writes them to a golden file:

summer parity:broadcasts --flow testdata/broadcasts/flows/post-lifecycle.yaml --step delete --name deleted --target http://127.0.0.1:8000 --vars /tmp/parity/vars.yaml --out testdata/broadcasts/deleted.yaml

Point the reference backend's Centrifugo API URL at the recorder (127.0.0.1:8424 by default). --step keeps only the publications of one step, running the earlier steps as setup. Timestamps, the actor and captured IDs are masked (tide.NormalizePublications), so the Go port's publications, recorded the same way, compare with tide.DiffPublications. A golden with --pending set is recorded but not yet asserted.

On the Go side, the memory realtime driver records publications the same way in tests; see Realtime.