Counting on Nothing: How Broken Pagination Contracts Are Quietly Undermining API Trust

Pagination is one of those API features that earns almost no celebration when it works and catastrophic blame when it does not. It occupies a quiet corner of every developer portal—usually a short section beneath authentication and rate limits—described in confident, unambiguous prose. "Pass a cursor. Receive the next page. Repeat until done." That description is accurate, right up until it is not.

The failure modes embedded in most pagination implementations are not edge cases. They are structural consequences of design decisions that prioritize simplicity of documentation over correctness of behavior. Understanding them requires examining not just how pagination is implemented, but what contract it implicitly promises—and how regularly that contract is broken in ways that neither the API nor the client ever formally acknowledge.

The False Promise of Total Count

The total_count field is perhaps the most widely trusted and most routinely misleading value in API responses. Developers use it for progress indicators, completion checks, and upstream business logic. It appears authoritative. It is, in practice, a snapshot.

In any system where records are being written concurrently with a client's pagination traversal, total_count reflects the state of the dataset at the moment the first request was processed. By page three, that number may already be inaccurate. New records may have been inserted. Existing records may have been soft-deleted. Filtered results may have shifted because an underlying attribute changed.

The client, however, has no mechanism to detect this. It continues iterating, trusting a count that has quietly become a historical artifact. Applications that halt traversal when items_received >= total_count will terminate early. Applications that allocate memory or pre-size data structures based on total_count will misbehave under concurrent write load.

The correction is not simply to remove the field—it is to define its semantics explicitly in your schema and surface a timestamp indicating when that count was evaluated. Clients can then make informed decisions about whether the value remains meaningful for their use case.

Offset Drift Under Concurrent Modification

Offset-based pagination carries a well-documented vulnerability that nonetheless continues to appear in production APIs across the industry. When a client requests records 20 through 40 from a sorted result set, it assumes the underlying dataset has remained stable since the previous page was fetched. In most production systems, that assumption fails continuously.

If a record is deleted between page one and page two, offset-based traversal will skip a record silently. If a record is inserted and sorts into an earlier position, the client will receive a duplicate. Neither condition raises an error. Neither triggers a retry. The client receives a structurally valid response that contains corrupt data relative to the intended traversal.

This is not a theoretical concern. It is the default behavior of offset pagination applied to any dataset that accepts concurrent writes—which describes the overwhelming majority of production APIs.

Keyset pagination, also referred to as cursor-based pagination, addresses this by anchoring traversal to a specific value in the data rather than a numerical position. The cursor encodes the last observed value of the sort key, and each subsequent request begins from that anchor rather than from an integer offset. This approach is not immune to all consistency challenges, but it eliminates the class of failures caused by positional drift.

Cursor Corruption and the Opacity Problem

Cursor-based pagination introduces its own category of failure, particularly when cursors are treated as opaque tokens that clients are expected to pass through without modification. The opacity is intentional—it protects internal implementation details and discourages cursor manipulation. But it also creates a debugging environment that is nearly impossible to navigate when something goes wrong.

When a cursor is invalidated—because the underlying query changed, because a schema migration altered the sort key, or because a deployment rotated an encryption key used to sign cursor values—the client receives an error that provides no actionable information. The cursor it was given is no longer valid. It cannot reconstruct where it was in the traversal. It must restart from the beginning.

For long-running batch processes, this is a significant operational problem. For clients building incremental sync pipelines, it can mean hours of reprocessing. The solution is not to make cursors transparent, but to make cursor invalidation explicit. When a cursor expires or becomes invalid, the error response should indicate whether the client can safely restart from the beginning or whether data may have changed in ways that require a full re-evaluation of the traversal strategy.

Additionally, cursor lifetimes should be documented precisely. A cursor that expires after fifteen minutes is a fundamentally different tool than one that remains valid for thirty days. Treating this as a minor implementation detail rather than a first-class API contract guarantees that integrators will build pipelines that fail at the worst possible moments.

Schema-Driven Pagination as a Design Discipline

Most pagination failures share a common root cause: the pagination contract was not formally specified before implementation began. The response envelope evolved organically, the cursor format was chosen for convenience, and the semantics of edge cases were left to be addressed when they surfaced in production.

Schema-first API design applies particular leverage here. When pagination behavior is defined in an OpenAPI or AsyncAPI specification before a single line of implementation code is written, the team is forced to make explicit decisions about cursor lifetime, total count semantics, empty page behavior, and error conditions. Those decisions become testable. They become documentable. They become part of the contract that downstream integrators can rely on.

This is not a bureaucratic exercise. It is the mechanism by which API teams prevent the quiet, compounding erosion of integrator trust that results from pagination behavior that is technically functional but contractually undefined.

Consider the difference between a specification that says "returns a cursor for the next page" and one that specifies the cursor's encoding format, its maximum valid duration, the conditions under which it may be invalidated, and the error code returned when an expired cursor is submitted. The first invites assumption. The second establishes a contract.

What Integrators Are Actually Experiencing

Developers integrating against poorly specified pagination systems tend to discover failures late—often in production, often during high-volume operations, and almost always without adequate error signal to diagnose the root cause. A batch job that successfully pages through ten thousand records on a staging environment may silently skip records in production because write volume is higher and offset drift becomes significant.

This is the category of failure that generates support tickets written in frustration rather than precision. The integrator knows the data is wrong. They cannot explain why. The API team's dashboards show no errors. Every response returned HTTP 200. The contract was technically honored. The data was quietly incorrect.

Pagination deserves the same rigorous specification discipline applied to authentication, rate limiting, and error handling. It is not a secondary concern. It is the mechanism by which clients traverse the most fundamental resource of any data API, and its failure modes are invisible by design.

Building pagination that actually works means accepting that "next page" is not a simple instruction. It is a promise—about consistency, about completeness, about the stability of the data between requests. That promise should be written down, versioned, tested, and honored. Anything less is a contract waiting to be broken.