The 10,000-Record Ceiling: Architecting HubSpot Syncs Around the Search API Limit

Published June 14, 2026

TL;DR: HubSpot’s CRM Search API caps every query at 10,000 returned results, and a request that tries to page past that boundary is rejected with a 400 error. The silent data loss is not the API failing quietly; it is jobs that swallow or mishandle that 400, or that page with an unstable sort so records shift between pages. Either way, “working” sync and export jobs end up missing rows nobody notices until a downstream count disagrees. The reliable pattern is not deeper pagination; it is sharding the query into windows that are each smaller than 10,000 records, keyed on a monotonically increasing property such as the record ID or a last-modified timestamp, then paging within each window. This is a data-engineering constraint with a data-engineering fix, and it is the kind of thing a foundation audit catches before it corrupts a report.

The limit, stated precisely

The CRM Search endpoint paginates with an after cursor, but it will not page beyond the 10,000th record for a given set of filters and sort. When a request asks for results past that point, HubSpot rejects it with a 400 error rather than returning more rows (HubSpot CRM Search API). The dangerous failure mode is what jobs do with that 400: a loop wrapped in a broad try/catch swallows the error and exits as if it had simply reached the end, so a nightly export of 38,000 contacts paging by 100 returns its first 10,000, hits the 400, and reports success while two-thirds of the table never moves. A second, quieter loss comes from an unstable sort: if the sort key is not unique and monotonic, records shift position between pages, so some are returned twice and others skipped even below the ceiling. Either way the symptom is the same, a job that looks healthy while the data is incomplete.

This is different from the batch read and write endpoints, which cap at 100 records per call (HubSpot batch operations). The batch cap is a throughput knob; the Search cap is a correctness boundary. Confusing the two is how teams convince themselves that adding more pages or a longer timeout will fix a problem that more paging cannot fix.

Why the obvious fix does not work

The instinct, when an export comes up short, is to page harder: smaller page sizes, more retries, a longer run window. None of it helps, because the boundary is on the total result set, not on the page. The query can only ever reach its first 10,000 records; pushing the cursor past that point returns a 400, not more data. You can prove the gap to yourself with a count: ask the same filter group for its total using a limit of 1, and compare it to how many records your paging loop actually retrieved. When total is above 10,000 and your loop stopped at 10,000, the loop is not broken, the ceiling is.

The fix: shard on a monotonic key

The durable pattern is to split one too-large query into a sequence of windows, each guaranteed to hold fewer than 10,000 records, and to page normally inside each window. The window boundary has to be a property that only increases, so that consecutive windows never overlap and never skip.

1. Shard by record ID. Every HubSpot object has a numeric hs_object_id that increases over time. Query with a filter of hs_object_id greater than the last ID you saw, sorted ascending, page until that window empties or approaches 10,000, then start the next window from the highest ID returned. Because the key is strictly increasing and you carry the high-water mark forward, no record is read twice and none is skipped.

2. Shard by last-modified time for incremental syncs. When you only want what changed, filter on hs_lastmodifieddate greater than the previous run’s checkpoint, sorted ascending, and advance the checkpoint to the newest timestamp you processed. Keep each time window narrow enough that it cannot itself exceed 10,000 records; on high-volume portals that can mean hours, not days.

3. Hold the sort stable. The shard key and the sort key must be the same property and direction. If you filter on ID but sort on something else, the 10,000-record window and your high-water mark stop agreeing, and you reintroduce the gap you were trying to close.

The result is a job whose correctness does not depend on the table staying under 10,000 rows. It scales because each query is bounded by construction, not by hope.

Respect the rate limit while you shard

Sharding multiplies the number of calls, so it has to coexist with HubSpot’s rate limits rather than fight them. HubSpot publishes per-account request limits and returns a 429 with a Retry-After header when you exceed them (HubSpot API usage and rate limits). A correct job reads Retry-After and waits exactly that long rather than guessing, backs off exponentially on 5xx, and keeps a small fixed delay between calls so a backfill does not starve the same portal’s production integrations. The point is not to go as fast as possible; it is to finish every window without tripping a limit that would leave the run half-complete.

How to know your sync is actually whole

The reason this defect survives so long is that a job which silently stops still looks like it succeeded. Three checks turn that blind spot into a signal:

• Count reconciliation. After each run, compare the source total (the count-only Search call) against the number of records the job wrote downstream. A standing gap is the ceiling, not noise.
• High-water-mark logging. Record the last ID or timestamp each window reached. If the mark stops advancing while the source keeps growing, the shard logic has stalled.
• A deliberate over-10,000 test. On a sandbox or a known-large object, run the job against a set you know exceeds 10,000 and confirm it returns all of them. A job that has never been tested past the boundary has never been tested at all.

Where this sits in the foundation

The 10,000-record ceiling is one instance of a wider truth: integrations fail quietly far more often than they fail loudly, and the quiet failures are the ones that corrupt the numbers a business steers by. Identity resolution, property architecture, and sync correctness are the same discipline viewed from different angles, and they are the layers a HubSpot Data Foundation Audit inspects before anything is built on top of them. A sync that loses a third of its records is indistinguishable, on the dashboard, from a business that simply has fewer customers.

The takeaway

If a HubSpot export or sync touches an object with more than 10,000 records, assume the Search API ceiling applies until you have proven otherwise. Do not page harder; shard the query on a monotonically increasing key, hold the sort stable, advance a high-water mark, and respect the rate limit on the way through. Then reconcile counts every run so a silent stop becomes a caught one. Done that way, a sync stops being a nightly act of faith and becomes a verifiable, bounded job.

If you suspect a sync is quietly dropping records, that is the symptom this method exists to catch. Start with a Data Foundation Audit, where sync correctness is one of the layers we verify before anything downstream relies on the data.

Sources

1. HubSpot Developers, CRM Search API (pagination and the 10,000-result limit): https://developers.hubspot.com/docs/api/crm/search
2. HubSpot Developers, CRM objects and batch operations (100-record batch cap): https://developers.hubspot.com/docs/api/crm/contacts
3. HubSpot Developers, API usage details and rate limits (429 and Retry-After): https://developers.hubspot.com/docs/api/usage-details

← Back to all field notes