Combine Paginated API Responses into One JSON Array

Result: Combined JSON (example output)

After: the merged JSON array contains every customer record across the three pages, with the page wrappers removed.

[
  { "id": "cus_001", "email": "maya@example.com" },
  { "id": "cus_002", "email": "liam@example.com" },
  { "id": "cus_003", "email": "ava@example.com" },
  { "id": "cus_004", "email": "noah@example.com" },
  { "id": "cus_005", "email": "ella@example.com" },
  { "id": "cus_006", "email": "owen@example.com" }
]

After merging, downstream tools no longer need to know which page a record came from. They can work with one clean array.

What is API pagination?

API pagination is the practice of returning data in pages instead of one large response, using page numbers, cursors, or next URLs.

Combining paginated API responses is the process of extracting records from those separate pages and merging them into one reusable JSON array.

Quick 3-step summary

1. Collect each API page into one JSON array
2. Extract the result field from every page
3. Flatten the extracted arrays into one list

This ensures one complete dataset for export to CSV, analytics, deduplication, or downstream APIs.

Why paginated API responses are hard to use

• Data is split across multiple pages
• Each page wraps results differently
• You cannot directly export or analyze paginated data
• Duplicate records can appear across pages

To make the data usable, you must combine the pages into one dataset.

Try it with your API data

Paste your paginated API JSON and merge it instantly:

Merge all API pages into one JSON instantly ->

No code. No loops. No pagination logic. Reusable workflow.

How to combine paginated API responses

To combine paginated API responses:

1. Collect all pages into one JSON array
2. Extract the result field (data, items, or results)
3. Flatten the arrays into one list
4. Deduplicate records if needed

The goal is not just concatenation. The goal is a complete dataset: every page's records appear once, wrappers are removed, and downstream tools no longer need pagination logic.

Combining paginated API responses usually turns multiple pages into an array of arrays, which must be flattened into a single list.

The pagination merge pattern

This reusable pagination merge pattern applies to:

• page-based APIs
• cursor APIs
• webhook batches
• ETL pipelines

Pattern:

1. Collect pages
2. Extract the result path
3. Flatten arrays
4. Deduplicate when pages can overlap

Paginated API → pages → extract → flatten → dedupe → final JSON

Reuse this pattern across APIs when you need to merge paginated JSON without rebuilding pagination logic for every source.

Merge paginated JSON arrays

Merging paginated JSON arrays means extracting each page's record array and flattening those arrays into a single list.

For API pagination data, this usually means identifying the repeated result path (data, items, results, or a nested path), extracting it from each page, and flattening the array of arrays. A pagination merge pipeline is the most reliable way to standardize API data across systems because the same result-path and flattening rules can run every time new pages arrive.

Shape + Aggregate workflow

This workflow uses 2 steps:

1. Shape extracts the result array from each page
2. Aggregate flattens the extracted arrays into one list

The combination handles both wrapper removal and list merging. Each step has a single job, which keeps the workflow easy to audit and rerun.

Open Aggregate ->

Pagination formats

For more background on REST pagination patterns, see GitHub's guide to using pagination in the REST API.

Paginated API response examples

Real-world pagination merging usually starts with where the combined data is going next:

• Customer exports: merge each page's data records before CSV export.
• Webhook batches: combine paginated webhook payloads before automation.
• Analytics pipelines: merge pages into one array before flattening nested data.
• BI dashboards: dedupe overlapping page results before loading reports.
• Third-party integrations: standardize data, items, and results wrappers across vendors.

These cases all use the same pattern: collect pages, extract the record array, flatten page arrays, then clean API responses, dedupe, export, or validate the combined dataset.

Example (Before / After)

Before: a paginated customers API returns three pages, each wrapping the records under a data field with a next URL pointing to the following page.

[
  {
    "page": 1,
    "next": "/customers?page=2",
    "data": [
      { "id": "cus_001", "email": "maya@example.com" },
      { "id": "cus_002", "email": "liam@example.com" }
    ]
  },
  {
    "page": 2,
    "next": "/customers?page=3",
    "data": [
      { "id": "cus_003", "email": "ava@example.com" },
      { "id": "cus_004", "email": "noah@example.com" }
    ]
  },
  {
    "page": 3,
    "next": null,
    "data": [
      { "id": "cus_005", "email": "ella@example.com" },
      { "id": "cus_006", "email": "owen@example.com" }
    ]
  }
]

The video below shows the same workflow visually; use the live tool above when you want to run the transformation yourself.

Workflow: Shape + Aggregate

To merge paginated API data, decide which path holds each page's records, extract that path from every page, then concatenate the resulting arrays into one flat list.

The Shape and Aggregate utilities can apply the same pipeline repeatedly once the method is clear.

You can copy this setup for the Shape step (extract mode, scoped per page):

{
  "mode": "extract",
  "targetPaths": ["$[*]"],
  "path": "data"
}

Then add the Aggregate step. You can copy this setup for Aggregate with flatten (concatenate arrays):

{
  "targetPaths": [],
  "operation": "flatten"
}

For this example, the workflow does:

1. Use Shape with extract mode (targetPaths: ["$[*]"], path: "data") to pull the data array out of each page object, leaving an array of arrays such as [[cus_001, cus_002], [cus_003, cus_004], [cus_005, cus_006]].
2. Run Aggregate with flatten (concatenate arrays) to join those page arrays end-to-end.
3. Return one flat array containing every customer record across the three pages.

Workflow result

You end up with:

• id
• email
• a single flat array that can be exported, deduped, or sent to a downstream API
• a stable workflow that handles any number of pages

The pipeline or support material below shows the example input, aggregate config, and final flat output.

How to choose pagination merge rules

Choose merge rules based on how the API returns records:

• If every page uses data, extract data from each page.
• If wrappers differ across APIs, define one result path per source before merging.
• If pages can overlap, dedupe records by a stable identifier such as id.
• If records are nested, flatten or normalize them after the merge.
• If page fetches can fail, confirm page completeness before merging.

The most important rule is completeness. Do not treat a successful merge as a complete dataset unless all expected pages were collected first.

Common mistakes when combining paginated API responses

The most common mistakes happen when fetching, extracting, and flattening are blurred together:

• Merging page wrappers instead of the records inside them.
• Assuming every API uses the same result key.
• Forgetting to dedupe overlapping pages after retries.
• Flattening nested record data before page arrays are merged.

Avoid these by collecting pages first, extracting result arrays second, and flattening the page arrays third.

What happens if you don't combine paginated API data?

Split paginated data creates hidden reporting problems:

• CSV exports include only the first page
• Analytics dashboards undercount records
• Automation rules run on partial datasets
• Downstream systems need pagination logic in every workflow

Combining prevents these problems by turning page-based output into one explicit dataset before cleanup, export, or analysis.

Combine before CSV export

Combining paginated API responses is the bridge between raw API pagination and export-ready data. Once pages become one array, CSV exporters, dashboards, and downstream APIs can process the records without pagination logic.

If your pipeline combines, cleans, and then exports in that order, each stage has a single job and failures are easier to diagnose. Start by combining pages here, clean API responses, then export combined JSON to CSV.

Combine paginated API responses: code vs workflow

Most developers start with loops and pagination logic — but this quickly becomes hard to maintain.

JavaScript approach

Code is useful when the fetch loop and response shape are stable:

const pages = [page1, page2, page3];
const combined = pages.flatMap((page) => page.data);

Problems

• Pagination logic spreads across scripts, dashboards, and imports
• Result paths are harder to debug when wrappers change
• One-off loops are not reusable across multiple APIs

Workflow approach

A workflow is better when merge rules need to be reused, reviewed, or adjusted:

• 2 steps: extract + flatten
• Reusable across repeated API pulls
• Visual enough to inspect before export
• Non-developers can inspect the result path and flattening rules
• Each merge step can be previewed before export
• The same workflow can run against future page sets
• Fetching, merging, cleanup, and export can stay separate

Use this when:

• multiple APIs return different wrappers
• page shapes are inconsistent
• teams need auditability before export or analytics

For repeated pagination cleanup, a reusable workflow is easier to maintain than one-off scripts scattered across reports, dashboards, and imports.

Why not just merge paginated data manually?

Manual merging works for one export, but it breaks down when page counts, wrappers, or retries change.

Manual merging also hides transformation rules in spreadsheets, scripts, or ad hoc edits. A pipeline approach is repeatable, auditable, and scalable because result paths stay visible.

JSON workflow pipeline

Use pagination merging as the collection step in an end-to-end JSON workflow:

1. Combine paginated API responses with this guide so page records become one dataset.
2. Clean API responses to remove noise, whitespace, null-heavy fields, and values you do not need.
3. Normalize inconsistent API fields so types, empty values, booleans, and enums follow one rule.
4. Flatten nested JSON or export JSON to CSV, depending on where the data goes next.
5. Validate JSON against a schema before loading the combined dataset into stricter systems.
6. Compare JSON changes across future API pulls when you need to catch response drift over time.
7. Merge JSON configs separately when deployment settings need base-and-override behavior instead of page aggregation.

Limitations

Combining paginated API data can break down when the page structure changes or when the transform layer is asked to do work that belongs in the fetch layer.

Common limitations include:

• cursor-based pagination still needs fetching outside the transform — the workflow cannot follow next_cursor tokens on its own
• very large APIs may produce thousands of pages that are easier to stream than to paste at once
• incomplete or partially failed page fetches can leave gaps the merge step cannot recover
• very large JSON files may exceed the editor's preview limit and need chunked processing

For very large JSON files, consider streaming or chunked processing before running the full merge workflow.

For these cases, fetch and assemble the pages first, then use Shape and Aggregate to perform the final merge in one auditable step.

If paginated API data changes often, a reusable merge workflow is not optional.

Without it, exports miss records, analytics undercount, and workflows fail silently.

Start with a reusable pagination merge workflow and apply it to every repeated API pull.

Use the support material below to open this sample input and run the Shape and Aggregate workflow directly in the editor.

Related workflows

Clean and normalize data

• Clean API responses
• Normalize inconsistent API fields

Transform structure

• Flatten nested JSON
• Merge JSON configs

Export and analysis

• Export JSON to CSV
• Validate JSON against a schema

FAQ

What is API pagination?

API pagination is the practice of returning data in pages instead of one large response, using page numbers, cursors, or next URLs. Each page contains a slice of the result set, and the consumer is expected to combine the pages before using the data.

How do I combine paginated API responses?

Paste the pages into the editor as one JSON array, run Shape with extract mode on the result path (such as data), then run Aggregate with flatten (concatenate arrays) to join the page arrays into a single flat list. The output is one clean JSON array that can be exported, deduped, or sent to a downstream API.

Can I merge cursor-based API data?

Yes — once the cursor pages have been fetched and collected into a single JSON array, the same Shape plus Aggregate workflow merges them. The cursor logic itself still needs to run outside the transform, since the workflow does not call the API to follow next_cursor tokens.

Should I dedupe paginated results?

Often yes, especially when pages overlap or retries can fire twice. After the merge step, follow up with a dedupe pass on a stable identifier such as id, or clean API responses before storage for broader normalization.

How do I merge paginated API responses in JavaScript?

Fetch or collect each page, then flatten the record arrays with a stable result path:

const pages = [page1, page2, page3];
const records = pages.flatMap((page) => page.data);

This works when every page has the same shape. Use a workflow when result paths, dedupe rules, or export steps need to be reviewed and reused.

How do I handle cursor pagination?

Follow the API's cursor tokens first, collect every returned page, then merge the collected responses. The merge step combines fetched pages; it does not call the API or follow next_cursor automatically.

How do I deduplicate paginated results?

Deduplicate after combining pages, using a stable key such as id, email, or another unique record identifier. This prevents overlapping pages or retry responses from creating duplicate rows in exports and analytics.