Pagination, quotas & data states

Retrieve bounded top-row samples while handling fresh, final, and hourly data correctly.

Automatable now Advanced

Search Analytics pagination retrieves successive pages from a bounded top-row dataset. It does not turn the API into a complete export. Each response can contain at most 25,000 rows, and the method exposes at most 50,000 top rows per day per Search type, sorted by clicks.

A reliable collector must separately handle response pagination, the daily data cap, API quota, and preliminary data. Confusing any two of those concepts creates silent gaps or unstable dashboards.

Four limits that mean different things

Limit	What it controls	What to do
`rowLimit`	Rows in one response; 1–25,000, default 1,000	Request up to 25,000 and paginate
`startRow`	Zero-based offset into that query’s result	Increment by the response page size
50,000 rows/day/Search type	Maximum top rows exposed by Search Analytics for a day	Detect saturation; use bulk export if completeness matters
API quota	Request rate and computational load	Partition, cache, throttle, and retry safely

The UI’s 1,000-row table limit is another separate limit. API pagination can retrieve more than the interface, while still not retrieving every underlying combination.

Request 25,000-row pages correctly

Keep every part of the request identical except startRow:

{
  "startDate": "2026-06-15",
  "endDate": "2026-06-15",
  "type": "web",
  "dimensions": ["page", "query"],
  "aggregationType": "auto",
  "dataState": "final",
  "rowLimit": 25000,
  "startRow": 0
}

Then request offsets 25000, 50000, and so on until the service returns fewer rows than rowLimit. If the first page contains exactly 25,000 and the second also contains 25,000, request the next page: an empty response is the unambiguous terminator.

Conceptual loop:

offset = 0
repeat
  response = query(same_request, startRow: offset, rowLimit: 25000)
  upsert(response.rows)
  offset += response.rows.length
until response.rows.length < 25000

Use the actual number of returned rows for the next offset. Do not increment after a failed request, and do not append the same page twice after a timeout whose outcome is unknown.

Keep the paginated result stable

Pagination assumes the underlying ordering does not move between requests. Search Analytics sorts results by clicks descending; ties can have arbitrary order. Fresh data can change while pages are being requested, which can shift rows across offsets.

For reproducible multi-page extraction:

prefer dataState: final;
query one day at a time;
keep dimensions, filters, type, and aggregation byte-for-byte equivalent;
collect the pages in one bounded job;
upsert by the full dimensional key;
rerun the whole date partition after an interrupted or inconsistent job.

Do not run page 1 with final and page 2 with all, or change a filter between offsets.

What the 50,000-row limit actually means

Google documents a maximum of 50,000 rows per day per Search type for Search Analytics. These are top rows sorted by clicks. If the property has more combinations, pagination finishes without exposing the remaining long tail.

Consequences:

50,000 rows is a ceiling, not evidence that exactly 50,000 underlying combinations exist.
A page/query extract can saturate while a page-only extract does not.
Totals from detailed visible rows may be lower than a low-dimensional property total.
Splitting a request into countries, devices, or query filters creates different top-row samples; it is not a guaranteed reconstruction of hidden rows.
Privacy-protected queries remain unavailable as identifiable text.

Treat a daily extract that reaches 50,000 rows as possibly truncated. Record a saturation flag and avoid claims such as “all queries.” If complete long-tail analysis affects decisions, enable BigQuery bulk export before the needed historical period.

Why one-day requests are safer

Google’s extraction guidance recommends querying one day at a time. This approach:

aligns the documented 50,000-row limit with a clear partition;
reduces Search Analytics load quota;
makes retries and replacements idempotent;
isolates late or anomalous days;
simplifies Pacific Time boundaries;
makes row saturation visible instead of blending it into a long range.

For a 30-day report, collect 30 daily partitions and aggregate them in your storage layer. Do not assume a single 30-day detailed query returns the union of each day’s available top rows.

Choose the correct data state

`final`

final returns finalized data and is the default when dataState is omitted. Use it for scheduled reporting, baselines, period comparisons, and stable pagination.

Final does not mean complete query text or unlimited rows. It describes processing state, not row coverage.

`all`

all includes fresh daily data. When the request groups by date and includes incomplete dates, the response metadata can include first_incomplete_date. Every date on or after that marker can still change noticeably.

Use this for operational visibility only. Store the marker, label the data preliminary, and replace those partitions on later runs.

`hourly_all`

hourly_all enables hourly data and should be used with the hour dimension. When the range contains incomplete hours, response metadata can include first_incomplete_hour.

Hourly timestamps and incomplete metadata use the America/Los_Angeles time zone. Hourly data is useful during an incident or time-sensitive launch, not as a finalized daily scorecard.

Do not infer completeness from missing metadata

The incomplete markers are returned only under documented combinations: all grouped by date or hourly_all grouped by hour, when the requested range actually includes incomplete points.

An absent marker can mean the range is finalized, but it does not mean:

every query was exposed;
every underlying row was returned;
privacy suppression did not apply;
the request avoided the daily row cap.

Store processing state and row-coverage warnings as separate fields.

Distinguish data limits from request quota

Requesting fewer rows does not increase the 50,000-row daily data ceiling, and having quota remaining does not make more data rows available.

Current documented Search Analytics rate limits include:

1,200 queries per minute per site;
1,200 queries per minute per user;
40,000 queries per minute per project;
30,000,000 queries per day per project.

Search Analytics also enforces short-term and daily load quotas. Long date ranges and expensive groupings consume more load. The quota-exceeded response does not identify every underlying quota type.

Reduce load by querying short date ranges, avoiding unnecessary dimensions, caching finalized days, and not polling unchanged history.

URL Inspection has different quotas—currently 600 requests per minute and 2,000 per day per site—and should not be placed in the same quota budget.

Retry without creating duplicates

For retryable rate or service failures:

honor any server-provided retry guidance;
apply exponential backoff with random jitter;
cap attempts and total elapsed time;
preserve the same request and offset;
upsert rather than blindly append;
mark the partition incomplete if the job ultimately fails.

Do not retry authentication, authorization, invalid-argument, or persistent quota design errors indefinitely. Surface them to an owner with the property, request signature, response code, and last successful partition.

A durable daily collector

Use two passes:

Finalized pass

Each day, find the most recent finalized date and fill any missing daily partitions through it. Finalized partitions should be immutable unless Google documents a correction or the extraction definition changes.

Fresh pass

Optionally query recent dates with all or recent hours with hourly_all. Store them in a preliminary layer and upsert the whole affected range. Promote a partition only after it is returned as finalized.

Each partition should record:

request hash and extractor version;
property, Search type, aggregation, dimensions, and filters;
row count and saturation status;
data state and first incomplete marker;
extraction timestamp;
retry count and completion status.

Reconciliation checks

For each Search type and date:

compare low-dimensional API totals with the same UI property and filters;
verify SUM(clicks) / SUM(impressions) rather than average CTR;
explain differences between property and page aggregation;
flag detailed-row sums below the property total as a coverage difference, not automatically an extraction failure;
alert on zero rows only when the property normally has data;
alert when pagination stops because of an error, not when it ends with a short page;
track how frequently dates reach 50,000 rows.

Common pagination mistakes

Leaving rowLimit at its 1,000-row default and assuming the response is complete.
Stopping after a full 25,000-row page without requesting the next offset.
Calling two 25,000-row pages “all data.”
Paginating fresh data while its ordering is changing.
Changing dimensions or filters between pages.
Querying long ranges repeatedly and exhausting load quota.
Treating first_incomplete_date as a row-coverage marker.
Retrying by appending, creating duplicate rows.
Trying to bypass the daily cap by slicing filters and then summing overlapping samples.