Loop Through Pagination
An example request that uses Gaffa to loop through different type of pages and perform actions.
The following examples are prebuilt requests we've put together to show you Gaffa's capabilities against our demo site. You can run any of them right now in the Gaffa API Playground.
These examples demonstrate the loop action, which repeats a sequence of nested actions on every page of a paginated site, capturing the DOM (and optionally a screenshot) of each page until the pagination control disappears or a fixed number of iterations is reached, or a timeout is hit. This is useful for scraping listing pages, catalogues, or search results that span multiple pages, without having to send a separate request per page.
Pagination UIs vary from site to site, so below are three common patterns and how to loop through each of them:
Numbered Pagination
This example opens an e-commerce demo site that pages through its product listing using numbered page controls (1, 2, 3…), and loops through all 5 pages, capturing the DOM and a full-page screenshot of each one.
API Request
The request below uses the POST endpoint to open the demo site, wait for the first product to render, then loop up to 5 times: on each iteration, it waits briefly for the page to settle, captures the DOM, takes a full-screen screenshot, and clicks the next numbered page button. You can run this request in the Gaffa API Playground.
{
"url": "https://demo.gaffa.dev/simulate/ecommerce?loadTime=0&showModal=false&modalDelay=0&loadingMode=paged&pagingStyle=numbered&pageCount=5&pageSize=10&itemCount=50&itemLoadTime=0&isVirtualScroll=false&page=1",
"proxy_location": null,
"async": false,
"max_cache_age": 0,
"settings": {
"record_request": true,
"max_media_bandwidth": null,
"time_limit": 60000,
"actions": [
{
"type": "wait",
"selector": "[data-testid=\"product-1\"]",
"timeout": 10000
},
{
"type": "loop",
"custom_id": "pagination-loop",
"max_iterations": 5,
"timeout": 45000,
"stop_on_fail": true,
"continue_on_fail": true,
"actions": [
{
"type": "wait",
"time": 500,
"custom_id": "settle"
},
{
"type": "capture_dom",
"custom_id": "page-dom"
},
{
"type": "capture_screenshot",
"size": "fullscreen"
},
{
"type": "click",
"selector": "nav[aria-label=\"Pagination\"] button[aria-current=\"page\"] + button",
"timeout": 5000,
"custom_id": "next-page"
}
]
}
]
}
}Response
The loop is returned as a single action. Its nested actions from every iteration are listed in execution order in the loop entry's own actions array, and the iterations count on the loop entry shows how many times the loop ran. On the last page, the click for the next page can't find its button, so it fails with action_timed_out and the loop ends. Because the loop has continue_on_fail: true, the request still completes:
Next Button Pagination
This example opens a demo site that shows a cookie-consent modal on load and pages through a virtualised product list using a single "Next page" button, looping through all 3 pages.
API Request
The request below uses the POST endpoint to open the demo site, dismiss the cookie modal, wait for the first product to render, then loop up to 3 times: on each iteration, it waits briefly, captures the DOM, and clicks "Next page". You can run this request in the Gaffa API Playground.
Response
As with the numbered pagination example, the loop's nested actions are listed in the loop entry's own actions array, in execution order, once per page. The actions before the loop stay at the top level of actions. On the last page, the "Next page" click fails with action_timed_out and the loop ends, and because the loop has continue_on_fail: true, the request still completes:
Show More Pagination
This example opens a demo site that shows a cookie-consent modal on load and loads more products via a "Show more" button, looping through all 4 pages of results.
API Request
The request below uses the POST endpoint to open the demo site, dismiss the cookie modal, wait for the first product to render, then loop up to 4 times: on each iteration, it waits briefly, captures the DOM, takes a full-screen screenshot, and clicks "Show more". You can run this request in the Gaffa API Playground.
Response
As with the other pagination examples, the loop's nested actions are listed in the loop entry's own actions array, in execution order, once per page. On the last page the "Show more" button is gone, so the click fails with action_timed_out and the loop ends. Because the loop has continue_on_fail: true, the request still completes:
Last updated