# Introduction

What is Gaffa?

Gaffa is a powerful API for browser automation that lets you control real web browsers at scale through a simple interface with no configuration required. We'll handle the complexities of managing infrastructure, such as virtual machines, proxies, and caching, so you can focus on building powerful, reliable web automation and AI applications!

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>API Playground</strong></td><td>Start experimenting with the Gaffa API right now.</td><td><a href="https://gaffa.dev/dashboard/playground">https://gaffa.dev/dashboard/playground</a></td></tr><tr><td><strong>Get Started</strong></td><td>The simple steps to get you started using Gaffa in your apps.</td><td><a href="/docs/get-started">Get Started</a></td></tr><tr><td><strong>API Reference</strong></td><td>Explore the API and docs for the finer details</td><td><a href="/docs/api-reference/api-reference">API Reference</a></td></tr></tbody></table>

## Key features

Gaffa is ready to power your web automations:

* **Simplicity** - there's no need to learn another new framework; Gaffa is accessible through a simple REST API - just tell it what site you want to visit and what actions you want to perform, and it will be carried out as soon as you send the request.
* **Real browsers** - headless browsers are popular but we make it simple to control real cloud-hosted browsers at scale which render JavaScript sites exactly as they would on a local machine, are harder to detect when doing scraping, and allow full observability. We're also planning to let you go beyond just controlling web browsers!
* **Proxies** - you can easily choose to route your traffic through a network of residential proxy IP addresses to help avoid bot-detection on sites you are trying to automate.
* **Scalable** - whether you want to control a single cloud browser or 100s in parallel with Gaffa, you can do that easily without one thought about infrastructure management.
* **Powerful data processing** - once you've accessed your desired site, you can export your data in a constantly growing number of formats. If you want the [page content in Markdown](/docs/features/browser-requests/actions/generate-markdown) to feed into a large language model, or [an image](/docs/features/browser-requests/actions/capture-screenshot) to feed into a vision modal we can help.

## Ready to work with Gaffa?

{% content-ref url="/pages/kzTlst3tKo255yz4YpDi" %}
[Get Started](/docs/get-started)
{% endcontent-ref %}

## Stay up to date

We'll be announcing updates and new features in our newsletter - [sign up here](https://gaffa.dev/#newsletter).


# Get Started

An introduction to the Gaffa Browser API. Learn how you can get started building fast, powerful web automations!

Welcome to the Gaffa documentation site! You'll find everything you need here to get started using the API, including [interactive API definitions](/docs/api-reference/post-v1-browser-requests), [a comprehensive list of actions](/docs/features/browser-requests/actions) you can use to interact with our cloud browsers, and [breakdowns of our example requests](/docs/features/browser-requests/api-playground-examples) you can run right away in our API Playground.

{% hint style="info" %}
Gaffa is currently in its very early stages, so we'd love to hear how we can improve our docs and API to make life easier for our users. If you have any questions or comments, please [email us](mailto:support@gaffa.dev) or use [the support tool on our site](https://go.crisp.chat/chat/embed/?website_id=87a5807c-14f5-4ed3-9fbe-3d161610357b).\
\
To stay up to date with the latest developments, features, and news on the mission to support the development of revolutionary AI Agents, sign up for sporadic [newsletter](https://gaffa.dev/#newsletter) updates.
{% endhint %}

{% stepper %}
{% step %}

## Create an account

You can sign up to create a Gaffa account [here](https://gaffa.dev/auth/sign-in). After signing up, you can use the API to access our [API Playground](https://gaffa.dev/dashboard/playground), which includes several prebuilt automations for [our demo site](https://demo.gaffa.dev/) that simulate a range of scenarios.

#### Accessing the open web

When you're ready to use Gaffa on the open web, you'll need to choose a plan that suits your needs and pay for it. After that, the full internet will be available for you to automate.

{% hint style="warning" %}
To avoid scaling issues for our existing customers, we are currently using a queuing system for new accounts. Simply join the queue when prompted on your [account dashboard](https://gaffa.dev/dashboard), and we'll let you know when you have access.\
\
If you want to jump the queue, you can fill out a short survey to help us better understand our users, and we'll approve your account sooner!
{% endhint %}
{% endstep %}

{% step %}

## Making your first browser request

The easiest way to make your first Gaffa [browser request](/docs/features/browser-requests) is to use our [API Playground](https://gaffa.dev/dashboard/playground), where you can see several pre-made interactive browser request examples of automations we've built against our test site, which simulates some common scraping and web automation scenarios. You can run these examples without a paid account and edit them easily to experiment. Once you have a paid account, you can also use the playground to build your automations for other sites.

### Gaffa API Playground examples

Here are all the sample requests we've created for use in the API Playground.

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Print to PDF</strong></td><td>Export a web page to PDF and wait for elements to load with the Gaffa API.</td><td><a href="/docs/features/browser-requests/api-playground-examples/export-web-page-to-pdf">Export Web Page to PDF</a></td></tr><tr><td><strong>Convert to Markdown</strong></td><td>Export a web page to markdown format - useful feeding into LLM apps.</td><td><a href="/docs/features/browser-requests/api-playground-examples/convert-web-page-to-markdown">Convert Web Page to Markdown</a></td></tr><tr><td><strong>Infinitely Scroll</strong></td><td>Scroll the bottom of a page that infinitely loads items and record the interaction.</td><td><a href="/docs/features/browser-requests/api-playground-examples/infinitely-scroll-an-e-commerce-site">Infinitely Scroll an E-commerce Site</a></td></tr><tr><td><strong>Capture Screenshot</strong></td><td>Interact with a page and capture the a screenshot of the whole page.</td><td><a href="/docs/features/browser-requests/api-playground-examples/capture-a-full-height-screenshot">Capture a Full-Height Screenshot</a></td></tr><tr><td><strong>Form Completion</strong></td><td>Fill out a form in a human-like way and record the interaction</td><td><a href="/docs/features/browser-requests/api-playground-examples/automated-form-filling">Automated Form Filling</a></td></tr></tbody></table>
{% endstep %}

{% step %}

## Building your own browser requests

Once you have a paid account and are ready to start building your own browser requests, you'll want to read about all the other [actions ](/docs/features/browser-requests/actions)you can use for your solution, as well as how you can easily use [proxy servers](/docs/features/browser-requests#proxy-servers), [our cache](/docs/features/browser-requests#caching), and the [other endpoints that are part of the API](/docs/api-reference/api-reference)
{% endstep %}
{% endstepper %}

## <sup>**Want to build faster with AI assistance?**</sup>

You can use Gaffa's [`llms.txt`](https://gaffa.dev/docs/llms-full.txt) file to give AI assistants like ChatGPT or Claude instant, accurate context about the Gaffa API, so they can generate working code for you straight away, without you having to explain the API yourself. [Learn how to use the Gaffa LLMs.txt file →](https://gaffa.dev/blog/let-your-ai-assistant-write-your-gaffa-code)


# Credits and Pricing

{% hint style="info" %}
View our current pricing plans on the Gaffa [homepage](https://gaffa.dev/#pricing)
{% endhint %}

## Browser Requests

Browser requests are charged in terms of credits based on the following factors:

* **Request length:** Billed at 1 credit per 30 seconds, the request takes to run on the browser.
  * If screen recording is enabled, this is doubled to 2 credits per 30 seconds.
* **Proxy bandwidth usage:** All requests that use a `proxy_location` parameter use our network of residential proxies and are billed at 1500 credits per 1GB of bandwidth used.
* **Paid Actions:** Some actions will incur additional costs for their usage in a browser request. These are:
  * [JSON Parsing](/docs/features/browser-requests/actions/parse-json)

Each successful request will deduct the corresponding number of credits from your monthly allowance. Be sure to use as many of your monthly credits as you want, as they don't roll over month to month.

## Mapping Requests

Mapping requests are also charged in credits at a rate of **1 credit per mapping request.**


# Browser Requests

Making web automation requests has never been so simple.

Browser Requests allow you to send the Gaffa API a URL and a list of actions you want to be carried out, including any outputs you want from the page. We'll carry out the request in our cloud browsers and return the response, so you don't have to worry about proxies, IP rotation, web automation frameworks, or scaling.

There's absolutely zero configuration needed, and you can interact with Gaffa from any program that can send web requests. We think it's by far the simplest way to automate basic web tasks, and the good news is that we're just getting started and have much more planned.

### How It Works

A browser request consists of three main components:

1. **Parameters** — Control the basics like URL, proxy location, and caching
2. **Settings** — Configure recording, media limits, and timing
3. **Actions** — Define the tasks you want performed on the page

### Example Request

Running a new browser request is as simple as sending the following [POST body to our endpoint](/docs/api-reference/post-v1-browser-requests). Below, you can see the URL ([our demo site](https://demo.gaffa.dev/)) and a list of actions that instruct Gaffa to wait for the table to load, then print the page to PDF.

You can read more about this particular example and how you can run it right now in our API Playground [here](/docs/features/browser-requests/api-playground-examples/export-web-page-to-pdf).

```json
{
  "url": "https://demo.gaffa.dev/simulate/table?loadTime=3&rowCount=20",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "max_media_bandwidth": null,
    "actions": [
      {
        "type": "wait",
        "selector": "table"
      },
      {
        "type": "print",
        "size": "A4",
        "margin": 20,
        "orientation": "portrait"
      }
    ]
  }
}
```

### Stealth

We believe your AI Agents should be able to use the internet exactly how humans would. Gaffa can help you access sites with some of the most challenging anti-bot restrictions by combining proxies, human-like behaviour, captcha solving, and a custom browser implementation. We handle and maintain all of that so you can focus on building your solution!

### Learn More

[**Parameters**](/docs/features/browser-requests/parameters) — Learn about URL, proxy settings, async mode, and caching

[**Settings**](/docs/features/browser-requests/settings) — Explore recording, media bandwidth controls, and time limits

[**Actions**](/docs/features/browser-requests/actions) — Discover all available actions like screenshots, markdown generation, and more

[**Examples**](/docs/features/browser-requests/api-playground-examples) — View pre-built requests and start using them in the API Playground

[**API Reference**](/docs/api-reference/api-reference)— Complete endpoint documentation and technical details

### Examples

We've created a number of sample browser requests you can read about [here](/docs/features/browser-requests/api-playground-examples), or you can jump straight into the [API Playground](https://gaffa.dev/dashboard/playground) to run them right now.

### API Endpoints

Check out our API reference for more details on the available endpoints, particularly [those you can use to query for past requests by ID or status](/docs/api-reference/get-v1-browser-requests).


# Parameters

Parameters are the top-level settings that control the fundamental behaviour of your automation. These parameters define where your request goes, how it's routed, whether it runs synchronously or asynchronously, and how caching is handled.

Below you'll find detailed documentation for each available parameter.

## Proxy servers

{% hint style="info" %}
In order to access public sites and use proxy servers, you'll need to sign up for a [paid account](https://gaffa.dev/#pricing), but after that, you'll be able to build automations for any site you wish.
{% endhint %}

Gaffa makes it super simple to proxy your traffic through a global network of residential proxies. Setting `proxy_location` in your request will allow you to utilize one of our partner third-party proxy services to gain local access to a site.

Not setting a `proxy_location` will mean the request does not use a proxy server and will use a generic datacenter IP.

### Available Locations

| Proxy Server Location | Country Code |
| --------------------- | ------------ |
| United States         | `us`         |
| Ireland               | `ie`         |
| Singapore             | `sg`         |
| France                | `fr`         |

{% hint style="info" %}
At the moment, all our servers are in one location, but we aim to deploy local machines at our proxy locations to improve realistic end-user load times. If this interests you, please contact support.
{% endhint %}

### IP Types

Currently, all our IP addresses are residential IP addresses, which are procured through reputable third parties.

### IP Rotation

IP rotation is an essential part of any web data scraping or automation task. In Gaffa, each browser request is treated as unique. We regularly rotate the IP addresses used, so you should assume each request is made from a different IP address than the last.

{% hint style="info" %}
We are working to support a wider range of IP address scenarios, including static IPs in the future, and to enable more trusted proxies for requests that require enhanced security (logins, etc.).
{% endhint %}

### Restrictions

Whilst we'll do our best to provide access to as wide a range of sites as possible, we may have to restrict access to certain sites to prevent abuse of our service or of other services. Our proxy partners may also enforce restrictions on certain sites and categories of sites that we don't have any control over.

***

## Caching

`max_cache_age`: integer

When we were building Gaffa, we noticed that many existing scraping tools don't let users easily share their scraped web data, even though many users request the same pages on the same sites. Not only is this a waste of a user's allowance, but it also puts a burden on the site owners who are serving the same data to different users for the same purpose. Because of this, we have created a service-wide cache in Gaffa.

### How it works

When making a browser request, you can provide a `max_cache_age` parameter that is **a number in seconds equal to or greater than 0**. This value denotes the maximum age of data you would accept from the API.\
\
If another user of our service has requested the same URL with exactly the same parameters and actions as you in this timeframe, the response will be returned to you immediately and will not be processed by one of our browsers. If there are multiple identical requests in the given timeframe, then the most recent will be returned.\
\
This will save you time waiting for a response and credits, because requests returned from the cache don't use any bandwidth.

***

## **Settings**

The `settings` object allows you to configure how your browser requests behave. It currently supports four parameters that control recording, media downloads, and execution time limits.

You can read more about all available settings parameters [here](/docs/features/browser-requests/settings).


# Settings

The `settings` object in your browser request allows you to configure various aspects of how your automation behaves. Below are all the available settings parameters you can use.

***

## Screen Recording

**Parameter:** `record_request` (boolean)

By specifying `record_request`, you can ask Gaffa to screen record your automation and return a video in the response, allowing you to view the magic happening or to debug your automation.

Recording requests come at an [additional cost](/docs/credits-and-pricing).

**Example:**

```json
{
  "url": "https://example.com",
  "settings": {
    "record_request": true,
    "actions": [...]
  }
}
```

***

## Max Media Bandwidth

**Parameter:** `max_media_bandwidth` (integer or null)

If you're using Gaffa on a site with lots of images and videos but are more interested in the text data on the page, you can cap how much media content a page loads using the `max_media_bandwidth` setting. This makes your automation faster and prevents spending credits on data you aren't interested in.

### Setting Options

You can set `max_media_bandwidth` in three ways:

* `"max_media_bandwidth": 0` — Block all images and videos completely
* `"max_media_bandwidth": 5` — Cap media downloads at 5MB (or any number you specify)
* `"max_media_bandwidth": null` — No limit (default)

### How It Works

When the `max_media_bandwidth` value is set, Gaffa monitors the data being downloaded by the page. When the downloaded media exceeds the specified MB limit, any further downloads of images or videos will be cancelled.

{% hint style="info" %}
**Important:** When enabled, only image and video downloads are blocked. HTML, CSS, JavaScript, and other essential page resources load normally, preserving functionality.
{% endhint %}

### Common Use Cases

This setting is particularly useful for:

* **Scraping news articles for text only** — Extract headlines and article content without downloading thumbnails
* **E-commerce price monitoring** — Track product prices and descriptions without loading product images
* **Extracting reviews and text content** — Capture customer reviews without profile pictures
* **SEO and content analysis** — Analyze page structure, headings, and text without media files

{% hint style="success" %}
**Performance Benefits:** Testing on image-heavy news sites showed up to **43% token savings** with no loss of text data. Sites with more media content see even greater savings in both cost and request speed.
{% endhint %}

{% hint style="warning" %}
**When NOT to Use: Not recommended for capturing screenshots, verifying images, or analysing visual content.**
{% endhint %}

### Getting Started

Start with `max_media_bandwidth: 0` for maximum savings, then adjust upward only if you encounter issues with specific sites. Setting a value of `0` will cause no images to load, which works well on most sites, but on some could lead to the site thinking you are using an ad blocker.

**Example:**

```json
{
  "url": "https://www.bbc.com/",
  "settings": {
    "max_media_bandwidth": 0,
    "actions": [
      {
        "type": "generate_markdown"
      }
    ]
  }
}
```

**Learn more:** See our detailed [guide](https://gaffa.dev/blog/how-to-slash-your-gaffa-credit-costs-by-40-percent) on optimizing browser requests with max\_media\_bandwidth, including real-world testing, use cases, and best practices.

***

## Time Limit

**Parameter:** `time_limit` (integer)

Using the `time_limit` setting caps the maximum running time of the request in milliseconds. If this time expires, all incomplete actions will be cancelled, and the request will return an error.

This value cannot exceed your plan's maximum request runtime. If omitted, it defaults to that maximum.

### Plan limits

| Plan          | Maximum `time_limit`   |
| ------------- | ---------------------- |
| Pay As You Go | 60,000 ms (60 seconds) |
| Starter       | 60,000 ms (60 seconds) |
| Startup       | 120,000 ms (2 minutes) |
| Growth        | 300,000 ms (5 minutes) |

**Example:**

```json
{
  "url": "https://example.com",
  "settings": {
    "time_limit": 30000,
    "actions": [...]
  }
}
```

***

## Ad Blocking

**Parameter:** `block_ads` (boolean)

{% hint style="danger" %}
Beta feature: Ad blocking is available to all users but is currently in beta. If you encounter ad networks that aren't being blocked, [get in touch](https://gaffa.dev/contact), and we'll add them.
{% endhint %}

If you are automating or scraping content on ad-heavy websites, third-party ad network requests can slow down your page load significantly, even though you don't need them. By enabling `block_ads` , Gaffa intercepts and immediately aborts requests to known ad-serving domains before they load, reducing page load times without affecting the core page content.

### Setting options

You can set `block_ads` in two ways:

* `"block_ads": false` — Ad blocking disabled (default)
* `"block_ads": true` — Ad blocking enabled

**Example:**

```json
{
  "url": "https://www.allrecipes.com",
  "settings": {
    "block_ads": true,
    "actions": [
      {
        "type": "capture_dom"
      }
    ]
  }
}
```

***

## Redirect Logging

**Parameter:** `log_redirects` (boolean)

If you're automating a flow that passes through one or more redirects before landing on a final page such as an affiliate link, a shortened URL, a tracking pixel, or a marketing campaign link, `log_redirects` captures every URL the browser was sent through along the way, not just the final destination.

By enabling `log_redirects`, Gaffa records each redirect hop encountered during the request, whether triggered by the initial page load or by an action such as a [`click`](/docs/features/browser-requests/actions/click), and returns them in the `redirects` field of the response.

### Setting options

* `"log_redirects": false` — Redirect logging disabled (default). The `redirects` field is omitted from the response entirely.
* `"log_redirects": true` — Redirect logging enabled. `redirects` is populated with every hop captured during the request.

**Example**

Tracking the redirect chain behind the marketing email link, from the shortened click-tracking URL through to the final landing page:

```json
{
  "url": "https://link.example-brand.com/e/click?upn=abc123-XYZ",
  "max_cache_age": 0,
  "settings": {
    "log_redirects": true,
    "actions": [
      { "type": "wait", "time": 3000 }
    ]
  }
}
```

The relevant part of the **response**:

```json
{
  "actual_url": "https://www.example-brand.com/products/new-arrivals?utm_source=email&utm_medium=campaign&utm_campaign=spring_launch&subscriber_id=987654",
  "redirects": [
    "https://click.example-esp.com/track/click?upn=abc123-XYZ&sub_id=987654",
    "https://link.example-brand.com/e/click?upn=abc123-XYZ",
    "https://www.example-brand.com/products/new-arrivals?utm_source=email&utm_medium=campaign&utm_campaign=spring_launch&subscriber_id=987654"
  ]
}
```

### Common use cases

* **Marketing campaign QA** — check that UTM parameters and click IDs on an email, ad, or social link survive the full redirect chain instead of getting dropped or overwritten partway through.
* **Affiliate and voucher link auditing** — confirm that a "get code" or "shop now" link actually routes through the correct affiliate network and arrives at the intended retailer page, with tracking parameters (like `utm_source`, click IDs) intact at each hop. Useful for verifying a partner integration is wired up correctly, or diagnosing why commission tracking isn't crediting properly.
* **Redirect chain / SEO auditing** — after a site migration or URL restructure, confirm old URLs resolve to the correct new destination in as few hops as possible. Long or looping redirect chains hurt both page speed and SEO.
* **Link safety verification** — see every intermediate domain a link passes through before landing on its final destination, useful for checking that shortened or obfuscated links (in emails, ads, QR codes) aren't routing through anything unexpected.
* **Debugging unexpected destinations** — when a request ends up somewhere you didn't expect (a geo-redirect, an A/B test split, a broken campaign link), the full chain shows exactly which hop diverged, rather than just the final URL.

***

## Actions

**Parameter:** `actions` (array)

The `actions` parameter defines the specific tasks you want Gaffa to perform on the page once it loads. Actions are executed in the order they appear in your array and can include tasks such as waiting for elements, capturing screenshots, generating Markdown, printing to PDF, and more.

We support different types of actions, each designed for specific automation needs. [Learn more about all available actions here](/docs/features/browser-requests/actions).

**Example:**

```json
{
  "url": "https://example.com",
  "settings": {
    "actions": [
      {
        "type": "wait",
        "selector": "table"
      },
      {
        "type": "print",
        "size": "A4",
        "margin": 20,
        "orientation": "portrait"
      }
    ]
  }
}
```

***

## Complete Example

Here's a browser request using multiple settings parameters:

```json
{
  "url": "https://www.bbc.com/",
  "proxy_location": "us",
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "max_media_bandwidth": 0,
    "time_limit": 60000,
    "block_ads": true,
    "actions": [
      {
        "type": "wait",
        "selector": "table"
      },
      {
        "type": "print",
        "size": "A4",
        "margin": 20,
        "orientation": "portrait"
      }
    ]
  }
}
```


# Selectors

Many Gaffa actions take a `selector` — a short piece of text that tells Gaffa which element on the page to work with, like a button to click or a table to read.

Most people write CSS selectors, and all of those work. But Gaffa uses Playwright's selector engine, so you can also match elements by their text or their visibility. XPath works too.

### Writing selectors in JSON

Your selector goes inside a JSON string, so any double quotes inside it will break the request. Use single quotes instead.

This is valid:

```json
"selector": "input[name='email']"
```

This is not:

```json
"selector": "input[name="email"]"
```

You can also escape the double quotes with backslashes, but single quotes are easier to get right. This matters more than it looks, because most of the extensions below take a quoted argument.

***

### Standard CSS

Every standard CSS selector works, including newer ones like `:has()`, `:is()` and `:not()`. Attribute selectors, combinators and structural selectors all behave exactly as they do in a browser.

Pseudo-elements such as `::before`, `::after` and `::placeholder` can't be targeted, because there's no real element to return.

See the [MDN CSS selectors reference](https://developer.mozilla.org/en-US/docs/Web/CSS/Reference/Selectors) for the full list.

***

### Playwright extensions

Alongside standard CSS selectors, Gaffa supports Playwright's selector extensions.

#### Finding elements by text

| Selector                          | What it matches                                                                    |
| --------------------------------- | ---------------------------------------------------------------------------------- |
| `:has-text('foo')`                | Any element whose contents include that text anywhere inside it. Case-insensitive. |
| `:text('foo')`                    | The smallest element directly containing that text. Case-insensitive, trimmed.     |
| `:text-is('foo')`                 | Exact match only. Case-sensitive, trimmed.                                         |
| `:text-matches('regex', 'flags')` | Matches the element's text against a regular expression.                           |

Use these when a button or link has no useful class or ID, which is common on sites built with utility CSS frameworks.

```json
{ "type": "click", "selector": "button:text('Accept all')" }
```

#### Other extensions

<table><thead><tr><th width="236.577392578125">Selector</th><th>What it does</th></tr></thead><tbody><tr><td><code>:visible</code></td><td>Matches only elements that are actually rendered and visible.</td></tr><tr><td><code>:nth-match(selector, n)</code></td><td>The nth match across the whole page, unlike <code>:nth-child</code> which counts within one parent.</td></tr><tr><td><code>:light()</code></td><td>Matches only the normal page DOM, ignoring content inside web components.</td></tr><tr><td><code>>></code></td><td>Chains selector types together, for example <code>css=... >> xpath=...</code>.</td></tr></tbody></table>

### XPath

You can use XPath anywhere a selector is accepted. Start the selector with `//` or `xpath=`.

```json
{ "type": "click", "selector": "//button[contains(text(), 'Submit')]" }
```

XPath is useful when you need to select an element by its relationship to another one, a parent, or a preceding sibling, which CSS can't express.

***

### Elements inside iframes

An iframe is a separate web page embedded inside the one you requested. Embedded videos, payment forms, maps and comment widgets are usually iframes.

* **Same-origin iframes**, meaning frames served from the same domain as the page, can be reached. Write your selector as normal, and Gaffa will find the element inside the frame; you don't need to target the frame first.
* **Cross-origin iframes**, meaning frames served from a different domain, cannot currently be reached. Selectors won't match elements inside them. Embedded YouTube players, payment widgets and third-party tools are usually cross-origin.

To tell which you're dealing with, compare the iframe's `src` with the page's own URL. A different domain means cross-origin.

### **Elements inside web components**

Some sites build parts of the page as web components, which keep their internal markup in a shadow DOM. Gaffa looks inside open shadow roots automatically, so you can write a selector for an element inside a component without doing anything special.

{% hint style="info" %}
**An element missing from a DOM capture may still be reachable.**

[`capture_dom`](/docs/features/browser-requests/actions/capture-dom) and [`generate_simplified_dom`](/docs/features/browser-requests/actions/generate-simplified-dom) return the main page only. Iframes and web components both appear as empty tags, with no content. So if you can see something on screen but can't find it in a capture, don't assume it can't be selected. If the frame is same-origin, or the content is inside a web component, try the selector.
{% endhint %}

***

### Examples

**Click a button that has no ID or class**

```json
{ "type": "click", "selector": "button:text('Accept all')" }
```

**Fill a field identified by its label**

```json
{ "type": "type", "selector": "input[aria-label='Email']", "text": "test@test.com" }
```

**Click the retry button in the row whose status says "Failed"**

```json
{ "type": "click", "selector": "tr:has(td:text-is('Failed')) button[aria-label='Retry']" }
```

**Wait for a success message inside a dialog**

```json
{ "type": "wait", "selector": "[role='dialog'] h2:has-text('Success!')", "timeout": 10000 }
```

***

### Traps worth knowing

* **`:has-text()` matches parents too.** `div:has-text('Submit')` matches every `div` containing that text, including the outermost one. Use `:text()` when you want the tightest element, or scope it to a container first.
* **Generated class names change.** Class names produced by build tools are rebuilt on every deploy. Prefer `id`, `data-testid`, `aria-label` or a text selector.
* **Avoid positional selectors.** Playwright has deprecated `:right-of()` and the others, and matching on layout is unreliable — a small change moves things around.
* **Long selectors are hard to debug.** If a selector needs more than two or three conditions, it's usually easier to `click` your way to a narrower part of the page first.

***

### FAQs

#### What kind of selectors does Gaffa support?

Gaffa uses Playwright's selector engine, so you can use any standard CSS selector, Playwright's extensions for matching text and visibility, and XPath expressions.

#### How do I write a selector with quotes inside a JSON request?

Use single quotes inside the selector so the JSON string stays valid, like `"input[name='email']"`. You can also escape double quotes with backslashes. Unescaped double quotes will break the request.

#### How do I select an element that has no ID or class?

Use a text selector like `button:text('Submit')`, or match on a stable attribute such as `aria-label` or `data-testid`. Both are more reliable than generated class names.

#### Can Gaffa select elements inside an iframe?

Same-origin iframes can be reached, so write your selector as normal, without targeting the frame first. Cross-origin iframes served from a different domain can't currently be accessed.

#### Why can't I click something inside an embedded video or payment widget?

Those are usually cross-origin iframes, served from a different domain to the page. Selectors can't currently reach inside them. Same-origin frames work normally.

#### Why can't I find an element in my DOM capture even though I can see it on the page?

It's probably inside an iframe. DOM captures return the main page only, showing iframes as empty tags. If the frame is same-origin, the element may still be selectable — try your selector directly.

#### What's the difference between :has-text() and :text()?

`:has-text()` matches any element containing that text, including parent elements. `:text()` matches only the smallest element directly containing it, which is usually what you want.


# Actions

When [making a Browser Request](/docs/api-reference/post-v1-browser-requests), you can specify a list of actions you want us to perform on the requested web page. These actions conform to the following format:

{% code overflow="wrap" fullWidth="false" %}

```json
{
    "type": "", //the type of the action
    //other params follow as key-value pairs
    "key": value //string, number, etc. 
}
```

{% endcode %}

### Universal Parameters

All actions have the following parameters:

<table data-full-width="false"><thead><tr><th width="226">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>type</code></td><td><code>string</code></td><td>true</td><td>The type name of the action.</td></tr><tr><td><code>continue_on_fail</code></td><td><code>boolean</code></td><td>false</td><td>Should execution of further actions continue or throw an error if this action fails.<br><strong>Default:</strong> <code>false</code></td></tr><tr><td><code>custom_id</code></td><td><code>string</code></td><td>false</td><td>A custom_id to help you find the action in the response.<br><strong>Default:</strong> <code>null</code></td></tr></tbody></table>

#### Action Execution

Actions are carried out in the order they are submitted. Every action type has a `continue_on_fail` parameter, which defaults to `false`.This means that if any action fails, the execution of the browser request ends, and an error will be returned. Setting `continue_on_fail` to `true` ensures that all actions are carried out, regardless of the previous action's results, and an error will not be returned.

#### Custom Id

As shown above, you can submit a `custom_id` with each action you submit to the API. We'll include this ID in the outputs from the browser request so you can find a certain action's output and/or status easily in the response.

## Response Format

When a browser request has completed, information on an action's execution

{% code fullWidth="false" %}

```json
{
    "id": "", //a unique id given to the action by Gaffa
    "type": "capture_screenshot", //the type of the action
    "query": "", //a representation of the action in querystring format
    "timestamp": "", //the UTC timestamp the action was executed
    "output": "" //if the action has an output, you will find a URL for this here,
    "error": "" //if the request fails, the error message will be returned here
}
```

{% endcode %}

## Choosing an action

<table data-header-hidden><thead><tr><th width="271.1328125"></th><th width="242.2528076171875"></th><th></th></tr></thead><tbody><tr><td>I want to…</td><td>Use</td><td>Not</td></tr><tr><td>Send page content to an LLM</td><td><a href="/docs/features/browser-requests/actions/generate-markdown"><code>generate_markdown</code></a></td><td><a href="/docs/features/browser-requests/actions/capture-dom"><code>capture_dom</code></a> — too big and too noisy</td></tr><tr><td>Find selectors on a page I don't know</td><td><a href="/docs/features/browser-requests/actions/generate-simplified-dom"><code>generate_simplified_dom</code></a></td><td><a href="/docs/features/browser-requests/actions/generate-markdown"><code>generate_markdown</code></a> — it drops the structure you need</td></tr><tr><td>Work out why a selector isn't matching</td><td><a href="/docs/features/browser-requests/actions/capture-dom"><code>capture_dom</code></a></td><td><a href="/docs/features/browser-requests/actions/generate-simplified-dom"><code>generate_simplified_dom</code></a> — it may have removed what you're looking for</td></tr><tr><td>Turn one HTML table into rows</td><td><a href="/docs/features/browser-requests/actions/parse-table"><code>parse_table</code></a></td><td><a href="/docs/features/browser-requests/actions/parse-json"><code>parse_json</code></a> — you'd pay credits for something free, and it wouldn’t be deterministic</td></tr><tr><td>Pull structured data out of prose, a PDF, or a messy layout</td><td><a href="/docs/features/browser-requests/actions/parse-json"><code>parse_json</code></a></td><td><a href="/docs/features/browser-requests/actions/parse-table"><code>parse_table</code></a> — it only reads &#x3C;table> markup</td></tr><tr><td>Get the contents of one element I know about</td><td><a href="/docs/features/browser-requests/actions/capture-element"><code>capture_element</code></a></td><td><a href="/docs/features/browser-requests/actions/capture-dom"><code>capture_dom</code></a> — it returns the whole page</td></tr><tr><td>Show what a page looked like</td><td><a href="/docs/features/browser-requests/actions/capture-screenshot"><code>capture_screenshot</code></a></td><td><a href="/docs/features/browser-requests/actions/generate-markdown"><code>generate_markdown</code></a> — no visual record</td></tr><tr><td>Keep a copy of a page I can look at again later</td><td><a href="/docs/features/browser-requests/actions/capture-snapshot"><code>capture_snapshot</code></a></td><td><a href="/docs/features/browser-requests/actions/capture-screenshot"><code>capture_screenshot</code></a> — you can't search an image</td></tr><tr><td>Turn a page into a PDF</td><td><a href="/docs/features/browser-requests/actions/print"><code>print</code></a></td><td><a href="/docs/features/browser-requests/actions/capture-snapshot"><code>capture_snapshot</code></a> — that gives you HTML</td></tr><tr><td>Get a file the browser has open</td><td><a href="/docs/features/browser-requests/actions/download-file"><code>download_file</code></a></td><td><a href="/docs/features/browser-requests/actions/print"><code>print</code></a> — it re-renders the page instead of taking the file</td></tr><tr><td>Capture several pages of results in one request</td><td><a href="/docs/features/browser-requests/actions/loop"><code>loop</code></a></td><td>Sending one request per page — you'd pay the page load each time |</td></tr></tbody></table>

## Supported Actions

The Gaffa API supports the following actions, detailed below. Click the "read more" buttons to read more information about each type.

### Actions without outputs

<table data-view="cards" data-full-width="true"><thead><tr><th>Type</th><th>Description</th><th>Read More</th></tr></thead><tbody><tr><td><code>block_dom_removals</code></td><td>Prevents the page from removing elements from the DOM.</td><td><a href="/docs/features/browser-requests/actions/block-dom-removals">Block DOM Removals</a></td></tr><tr><td><code>click</code></td><td>Click on a given element</td><td><a href="/docs/features/browser-requests/actions/click">Click</a></td></tr><tr><td><code>scroll</code></td><td>Scroll to a particular point on the page or, in the case of pages with infinite scrolling, scroll until a given time has elapsed.</td><td><a href="/docs/features/browser-requests/actions/scroll">Scroll</a></td></tr><tr><td><code>type</code></td><td>Type the provided text into a given element</td><td><a href="/docs/features/browser-requests/actions/type">Type</a></td></tr><tr><td><code>wait</code></td><td>Wait for a given time to elapse or an element to appear on page before proceeding to the next action.</td><td><a href="/docs/features/browser-requests/actions/wait">Wait</a></td></tr></tbody></table>

### Actions with outputs

<table data-view="cards" data-full-width="true"><thead><tr><th>Type</th><th>Description</th><th>Read More</th></tr></thead><tbody><tr><td><code>capture_cookies</code></td><td>Save a JSON object of cookies for the current page</td><td><a href="/docs/features/browser-requests/actions/capture-cookies">Capture Cookies</a></td></tr><tr><td><code>capture_dom</code></td><td>Export the raw DOM page data</td><td><a href="/docs/features/browser-requests/actions/capture-dom">DOM</a></td></tr><tr><td><code>capture_screenshot</code></td><td>Capture a screenshot of the web page</td><td><a href="/docs/features/browser-requests/actions/capture-screenshot">Screenshot</a></td></tr><tr><td><code>capture_element</code></td><td>Export the contents of a single element on the page</td><td><a href="/docs/features/browser-requests/actions/capture-element">Capture Element</a></td></tr><tr><td><code>capture_snapshot</code></td><td>Create a completely static version of the web page which can be accessed offline</td><td><a href="/docs/features/browser-requests/actions/capture-snapshot">Snapshot</a></td></tr><tr><td><code>download_file</code></td><td>Download an online file using Gaffa</td><td><a href="/docs/features/browser-requests/actions/download-file">Download File</a></td></tr><tr><td><code>generate_markdown</code></td><td>Convert the page into markdown</td><td><a href="/docs/features/browser-requests/actions/generate-markdown">Markdown</a></td></tr><tr><td><code>generate_simplified_dom</code></td><td>Generate a simplified version of the DOM</td><td><a href="/docs/features/browser-requests/actions/generate-simplified-dom">Simplified DOM</a></td></tr><tr><td><code>parse_json</code></td><td>Parse online data to a defined JSON schema</td><td><a href="/docs/features/browser-requests/actions/parse-json">JSON Parsing</a></td></tr><tr><td><code>parse_table</code></td><td>Convert an HTML table into JSON rows</td><td><a href="/docs/features/browser-requests/actions/parse-table">Parse Table</a></td></tr><tr><td><code>print</code></td><td>Print the web page to a PDF</td><td><a href="/docs/features/browser-requests/actions/print">Print</a></td></tr></tbody></table>

### Flow Actions

<table data-view="cards"><thead><tr><th>Type</th><th>Description</th><th>Read more</th></tr></thead><tbody><tr><td><code>loop</code></td><td>Runs nested actions repeatedly until it reaches a limit, times out, or an action fails.</td><td><a href="/docs/features/browser-requests/actions/loop">Loop</a></td></tr></tbody></table>

<details>

<summary><strong>Tips</strong></summary>

* Actions run in the order you list them, so put your capture actions last.
* Set `continue_on_fail: true` on anything optional, like closing a cookie banner. Otherwise, a single failure halts the entire request.
* Set a `custom_id` on each action so you can find its output in the response.
* You don't need a [`wait`](/docs/features/browser-requests/actions/wait) before [`click`](/docs/features/browser-requests/actions/click), [`capture_element`](/docs/features/browser-requests/actions/capture-element) or [`parse_table`](/docs/features/browser-requests/actions/parse-table). They already wait for their own [`selector`](/docs/features/browser-requests/selectors).
* Add a `selector` to [`generate_markdown`](/docs/features/browser-requests/actions/generate-markdown) or [`parse_json`](/docs/features/browser-requests/actions/parse-json) to reduce what you capture and what you pay for.
* If you're not sure an action worked, add a [`capture_screenshot`](/docs/features/browser-requests/actions/capture-screenshot) after it.
* Use [`loop`](/docs/features/browser-requests/actions/loop) when you need more than one page from a single request, rather than sending a request per page.
* Inside a [`loop`](/docs/features/browser-requests/actions/loop), put capture actions before the action that navigates. Anything after a failing action is skipped for that pass.

</details>

***

<details>

<summary><strong>FAQs</strong></summary>

#### In what order do Gaffa actions run?

Actions run one after another, in the order you list them in the actions array.

#### How do I capture multiple pages in a single request?

Use the [`loop`](/docs/features/browser-requests/actions/loop) action. It repeats a nested list of actions, so one request can page through search results or a numbered list. Loops can't be nested inside each other.

#### What happens when a Gaffa action fails?

The browser request stops and returns an error, and the remaining actions are skipped. Set `continue_on_fail: true` on that action if you want the rest to run anyway. Inside a [`loop`](/docs/features/browser-requests/actions/loop), a failure skips the rest of that iteration rather than ending the request, and the loop's `stop_on_fail` decides whether it exits or starts the next pass.

#### How do I find the output of a specific action in the response?

Give the action a `custom_id` when you send it. Gaffa returns that ID in the response, so you can look up the action's output without counting through the array.

#### Can I use the same action twice in one request?

Yes. Add it at each point where you need it — for example, a before-and-after screenshot of a click. Give each one a different `custom_id` so you can tell them apart.

#### Which Gaffa actions cost extra credits?

[`parse_json`](/docs/features/browser-requests/actions/parse-json) is the only one. It's billed on the tokens sent to and returned by the AI model. Keep costs down with `selector`, `input_token_cap` and `max_pages`.

</details>


# Block DOM Removals

**Type:** `block_dom_removals`

This action will prevent the page from removing items from the page. This is useful if you are trying to scrape data from a JavaScript-based web application that removes items from the page when they are out of view, which can make grabbing data difficult.

Using this action will block DOM removals for the rest of the browser request.

<details>

<summary><strong>Tips for using <code>block_dom_removal</code> action</strong></summary>

* Put this action first, before any scrolling. It only protects what happens after it runs.
* Use it if you scroll a long list and only the last few rows end up in your capture.
* The usual pattern is `block_dom_removals`, then [`scroll`](/docs/features/browser-requests/actions/scroll), then a capture action.
* Expect a larger DOM since nothing gets cleared.

</details>

### Parameters

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

Block DOM removals for the current browser request

```
"actions": [
    {
      "type": "block_dom_removals"
    }
]
```

### FAQs

#### When do I use the block\_dom\_removals action?

Use it when a site removes elements from the page as you scroll past them. This is common in long feeds and data grids. Blocking removals keeps every loaded row on the page.

#### How do I know a site is removing elements?

Scroll the page yourself with DevTools open and watch the Elements panel. If earlier rows disappear as new ones load, the site is recycling elements, and you need this action.

#### Does the block\_dom\_removals action affect the whole request?

Yes. Once it runs, removals are blocked for all subsequent actions. Put it early because it can't restore elements the page has already removed.

#### Does the block\_dom\_removals action take any parameters?

No. It only takes the [universal parameters](/docs/features/browser-requests/actions#universal-parameters): `type`, `continue_on_fail`, and `custom_id`. Add `"type": "block_dom_removals"` to your actions array and nothing else.


# Capture Cookies

**Type:** `capture_cookies`

This action will capture the browser cookies currently saved for the web page you are on and return them as a JSON object with key/values.

<details>

<summary>Tips for using <code>capture_cookies</code> action</summary>

* Run this after the steps that set the cookies you want, such as logging in or accepting a cookie pop-up.
* Treat the output like a password. Session cookies are credentials.
* Use it to work out why a page looks different to what you expected; the cookies often explain it.
* Pair it with a screenshot when you're debugging a login, so you can see the page and its cookies together.

</details>

### Parameters

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

Capture the cookies of the current page

```
"actions": [
    {
      "type": "capture_cookies"
    }
]
```

### FAQs

#### When do I use capture\_cookies?

Use it when you need to see or save the session a page is running on after a login, after accepting cookies, or when a page behaves oddly, and you want to investigate why.

#### What does capture\_cookies return?

It returns a JSON object containing the cookie names and values the browser holds for the page, captured at the moment the action runs in your list.

#### When should I run capture\_cookies in my action list?

Run it after anything that sets the cookies you care about, like logging in or dismissing a banner. If you capture first, those cookies won't exist yet.

#### How do I capture cookies after logging into a site?

Use [`type`](/docs/features/browser-requests/actions/type) for the username and password, [`click`](/docs/features/browser-requests/actions/click) for the submit button, a [`wait`](/docs/features/browser-requests/actions/wait) for something on the logged-in page, then `capture_cookies`.

#### Can I reuse captured cookies in another request?

Not at the moment. Each browser request starts a fresh session, so cookies captured in one request can't be sent with the next. We're working on it. If this would be useful for what you're building, email [support](mailto:support@gaffa.dev).

#### What information does capture\_cookies return?

Cookie names and values only. Domain, expiry and flags like `httpOnly` aren't included in the output at the moment.


# Capture DOM

**Type:** `capture_dom`

This action will capture and return the site's raw DOM, which you can then extract data from on your end.

For common AI scenarios, you may find that this returns too much data, so we have provided a [`generate_simplified_dom`](/docs/features/browser-requests/actions/generate-simplified-dom) , an action that distils the DOM to only the important elements.

<details>

<summary><strong>Tips for using <code>cature_dom</code> action</strong></summary>

* Use it when you want to parse the page yourself with a traditional HTML library such as BeautifulSoup, lxml, or Cheerio.
* It can also be used for debugging. When a selector isn't matching, the raw DOM shows you what's actually on the page.
* Use it when you need something the other actions strip out, like inline scripts, `data-` attributes or full link URLs.
* Don't send the output to an LLM. Use [`generate_simplified_dom`](/docs/features/browser-requests/actions/generate-simplified-dom) or [`generate_markdown`](/docs/features/browser-requests/actions/generate-markdown) instead, which carry the same information for far fewer tokens.
* It captures the page as it is at that point in the run, so you can use it multiple times to compare the state before and after an interaction.
* If content is missing from the capture, the page probably hadn't finished loading. Add a [`wait`](/docs/features/browser-requests/actions/wait) before it.

</details>

### Parameters

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

Capture the raw DOM of the current page

```
"actions": [
    {
      "type": "capture_dom"
    }
]
```

### Example Output

{% file src="/files/l8xETXQjit3lKZXjIN2q" %}

### FAQs

#### When do I use the capture\_dom action?

Use it when you want to parse the page yourself with a library like BeautifulSoup, or when you need scripts, attributes and link query strings that the simplified and Markdown outputs remove. See our[ guide to scraping tables with Python](https://gaffa.dev/blog/how-to-scrape-a-table-with-python-\(the-easy-way\)).

#### What does capture\_dom action return?

It returns the page's raw DOM as the browser rendered it, including scripts, styles and every attribute. This is the page after JavaScript has run, not the original server HTML.

#### Why is content missing from my capture\_dom output?

The action probably ran before the page finished loading. Add a [`wait`](/docs/features/browser-requests/actions/wait) action with a selector for the content you're expecting, so Gaffa captures the page once it's there.

#### When should I use generate\_simplified\_dom instead of capture\_dom?

Use [`generate_simplified_dom`](/docs/features/browser-requests/actions/generate-simplified-dom) whenever the output goes to an LLM, or you're looking for `selectors`. It keeps the page structure but removes the scripts and styling that make raw DOM so large. See our[ guide to simplifying a webpage DOM](https://gaffa.dev/blog/how-to-extract-and-simplify-a-webpage-dom-with-gaffa).

#### Why is my capture\_dom output so big?

Raw DOM includes all scripts and styles on the page. Use [`generate_simplified_dom`](/docs/features/browser-requests/actions/generate-simplified-dom) to strip those out, or [`capture_element`](/docs/features/browser-requests/actions/capture-element) with a selector to capture only the part you need.

#### How do I capture only part of the page with capture\_dom action?

Use [`capture_element`](/docs/features/browser-requests/actions/capture-element) with a selector instead. It gives you the same raw HTML, but only for the element you point it at.


# Capture Screenshot

**Type:** `capture_screenshot`

Takes a screenshot of the current page. You can take a full-screen screenshot of the entire page or just the current view.

<details>

<summary><strong>Tips for using <code>capture_screenshot</code> action</strong></summary>

* Use `fullscreen` to capture the whole page, and `view` to capture only what's visible on screen. `view` is the default.
* Scroll to the bottom first if the page loads images as you scroll (lazy-loading), or they'll appear blank.
* Add a screenshot after each step when a sequence isn't working. It's the quickest way to see where it went wrong.
* When debugging a [`parse_json`](/docs/features/browser-requests/actions/parse-json) action, take one first to check the content is actually on the page.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="212">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>size</code></td><td><code>string</code></td><td>false</td><td>The area of the page to capture in the screenshot.<br><strong>Default:</strong> <code>view</code><br><strong>Accepted</strong>: <code>["view", "fullscreen"]</code></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

The following captures the current section of the page currently visible in the browser.

```json
"actions": [
    {
        "type": "capture_screenshot",
        "size": "view"
    }
]
```

### Example Output

An example screenshot in `fullscreen` mode.

<figure><img src="https://2341007864-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyUba6osOT5MkKiV0wmgr%2Fuploads%2F1jESouuLiHMOwvVpbIha%2FGaffaFullHeightScreenshotExample.png?alt=media&amp;token=d9540785-4cac-42c8-90a3-8d757c918ec0" alt=""><figcaption></figcaption></figure>

### FAQs

#### When do I use capture\_screenshot?

Use it when you need to see the page rather than read it — to check that an action worked, monitor how a page looks over time, or feed an image to a vision model.

#### How do I take a full-page screenshot with Gaffa?

Set "`size": "fullscreen"` on the `capture_screenshot` action. The default is `view`, which captures only what's visible on screen and excludes content below the fold.

#### Why are images missing from my full-page screenshot?

The page is probably lazy-loading them as you scroll. Add a [`scroll`](/docs/features/browser-requests/actions/scroll) action to 100% with a `wait_time` before the screenshot, so every image comes into view and loads first.

#### Can I screenshot a single element?

Not directly. Screenshots cover the viewport or the full page. To isolate one element, use [`capture_element`](/docs/features/browser-requests/actions/capture-element) for its HTML, or scroll it into view and capture with `size: "view"`.


# Capture Element

**Type**: `capture_element`

Returns the [outerHTML](https://developer.mozilla.org/en-US/docs/Web/API/Element/outerHTML) of the element matching your selector, the element itself, and everything inside it. Use it when you need an element's contents, rather than the entire page.

<details>

<summary><strong>Tips for using <code>capture_element</code> action</strong></summary>

* Use this instead of [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) when you know where the content is. Pointing at main or article cuts out navigation and footers for you.
* You don't need a [`wait`](/docs/features/browser-requests/actions/wait) first. This action already waits for its own selector, up to 5 seconds by default, and you can change that with `timeout`.
* The action captures as soon as the element appears, so a longer `timeout` doesn't slow anything down. It's a ceiling, not a delay.
* Increase the `timeout` for pages that load content after the initial render, so the action doesn't give up before the element exists.
* It returns the element and everything inside it, including its opening and closing tags.
* If your selector matches more than one element, only the first is captured.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="212">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>selector</code></td><td><code>string</code></td><td>true</td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> that defines the element whose contents you want to capture.</td></tr><tr><td><code>timeout</code></td><td><code>integer</code></td><td>false</td><td>The maximum amount of time the browser should wait for the element defined by the selector to appear. <strong>Default: 5000 (5s)</strong></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

The following code will wait 1 second for the `.page_contents` element to appear and return an HTML file containing the div's innerHTML.

```json
"actions": [
    {
      "type": "capture_element",
      "selector": ".page_contents",
      "timeout": 1000
    }
]
```

### FAQs

#### When do I use the capture\_element action?

Use it when you want the HTML for a specific part of a page and already know its [`selector`](/docs/features/browser-requests/selectors): a results list, an article body, or a product panel. It's smaller and cleaner than capturing the whole DOM.

#### What does the capture\_element action return?

It returns the matched element as an HTML file, including the element's own opening and closing tags. Capturing a `table` gives you the complete `<table>` element, not just its contents.

#### How long does capture\_element wait for the element?

Up to five seconds by default, configurable with `timeout` in milliseconds. It captures as soon as the element appears, so increasing the timeout incurs no cost on pages where the element is already present.

#### Do I need a wait action before capture\_element?

No. It already waits for its selector using `timeout`. Adding a separate wait on the same element just slows the request.

#### What happens if my selector doesn't match anything?

The action fails once the timeout is reached, and by default that stops the request. Set `continue_on_fail: true` if the element is optional, and use [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) to check what the [`selector`](/docs/features/browser-requests/selectors) should be.

#### When should I use generate\_markdown instead of capture\_element?

Use [`generate_markdown`](/docs/features/browser-requests/actions/generate-markdown) with a [`selector`](/docs/features/browser-requests/selectors) when you want that part of the page as readable text. Use `capture_element` when you need the HTML exactly as it is, with the tags and attributes intact.

#### What happens if my selector matches more than one element?

Only the first match is captured. To get several elements, select a shared container instead, or add a separate `capture_element` action for each one.


# Capture Snapshot

**Type:** `capture_snapshot`

This output type will return an HTML file that captures a static version of the page state. The page will load offline and can be saved to your local machine.

This will:

* Load and embed all images on the page.
* Embed all CSS files

Currently, JavaScript is disabled, and interactivity might not work as expected, but this feature should help preserve the page state as it was and allow you to view it offline.

<details>

<summary><strong>Tips for using <code>capture_snapshot</code> action</strong></summary>

* Use this when you want to revisit the page later. Unlike a screenshot, you can still read and search the HTML.
* JavaScript is switched off in the saved file, so anything interactive is frozen as it was.
* Open any tabs or expand any sections before you capture, because they won't open in the saved file.
* Snapshots are large, since every image and stylesheet is embedded in the file.

</details>

### Parameters

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters)

### Usage

The following captures the current section of the page currently visible in the browser.

```json
"actions": [
    {
        "type": "capture_snapshot"
    }
]
```

### Example Output

Here's an example that shows an offline snapshot of a site

{% file src="/files/e5MQqPjILYNixHCus4iU" %}

### FAQs

#### When do I use the capture\_snapshot action?

Use it when you need a copy of a page to open and read later for record-keeping, evidence, or to check back on a page after it's changed or gone offline.

#### What does the capture\_snapshot action produce?

It produces a single HTML file with all the images and CSS built in, so the page opens offline on your machine without needing the original site or a network connection.

#### Does JavaScript work in a Gaffa snapshot?

No. JavaScript is disabled in the saved file, so tabs, carousels, and dropdowns won't respond. The snapshot keeps the page exactly as it was when you captured it.

#### How do I take a snapshot of a page with content behind a tab?

Add a [`click`](/docs/features/browser-requests/actions/click) for the tab, then a [`wait`](/docs/features/browser-requests/actions/wait) for its content to appear, then `capture_snapshot`. Scripts won't run in the saved file, so open everything you need beforehand.

#### When should I use capture\_screenshot instead of capture\_snapshot?

Use a screenshot when you just need to see the page. Use a snapshot when you'll want to read or pull data out of it later, since a snapshot is still HTML.

#### When should I use print instead of capture\_snapshot?

Use [`print`](/docs/features/browser-requests/actions/print) when you need a PDF to share or file. Use `capture_snapshot` when you need a working copy of how the page looked in the browser.


# Click

**Type**: `click`

Request that the browser click a particular element on the page.

<details>

<summary><strong>Tips for using <code>click</code> action</strong></summary>

* Pick [`selectors`](/docs/features/browser-requests/selectors) that won't break, like id, data-testid or aria-label. Avoid generated class names and nth-child.
* You don't need a [`wait`](/docs/features/browser-requests/actions/wait) first. `click` already waits up to 5 seconds for its element.
* Set `continue_on_fail: true` when closing cookie banners and pop-ups, since they aren't always there.
* After a click that loads new content, add a [`wait`](/docs/features/browser-requests/actions/wait) for something in that new content before you capture.
* If a click isn't working, add a screenshot after it to see what the page did.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="212">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>selector</code></td><td><code>string</code></td><td>true</td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> that defines the page element that the browser should click on.</td></tr><tr><td><code>timeout</code></td><td><code>integer</code></td><td>false</td><td>The maximum amount of time the browser should wait for the element defined by the selector to appear. <strong>Default: 5000 (5s)</strong></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

#### Click an element on the page

The following clicks the page logo.

```json
"actions": [
    {
      "type": "click",
      "selector": "a.header__logo"
    }
]
```

#### Click an element when it appears

The following waits up to 5 seconds for the logo to appear, then clicks it. It continues if the click fails.

```json
"actions": [
      {
        "type": "click",
        "selector": "a.header__logo",
        "timeout": 5000,
        "continue_on_fail": true
      }
]
```

### FAQs

#### When do I use the click action?

Use it whenever you need the browser to interact with something — a button, a link, a tab, a checkbox, or a "load more" control. Anything a user would click.

#### How do I click a button with Gaffa?

Add a `click` action with a CSS selector for the button. Gaffa waits up to five seconds for it to appear before clicking. Change that with the `timeout` parameter.

#### What selector should I use for a click?

Use something stable like `id`, `data-testid` or `aria-label`. Generated class names and positional selectors like `nth-child` break as soon as the site changes.

#### How do I close a cookie banner before scraping a page?

Add a `click` for the accept button with `continue_on_fail: true`, before your other actions. That flag stops the request failing on pages where the banner doesn't show.

#### How do I wait for content to load after a click?

Follow the click with a [`wait`](/docs/features/browser-requests/actions/wait) action using a selector that only exists in the new content, plus a `timeout`. That's more reliable than waiting a fixed number of seconds.

#### Why did my click fail?

Usually the [`selector`](/docs/features/browser-requests/selectors) didn't match in time, or the element was there but covered by a pop-up. [Capture the DOM](/docs/features/browser-requests/actions/capture-dom) to check the selector, and close any overlay first.


# Download File

**Type**: `download_file`

Request a copy of the most recently viewed file in the browser.

<details>

<summary><strong>Tips for using <code>download_file</code> action</strong></summary>

* Gaffa keeps track of files the browser downloads, so you can run this action at any point afterwards to collect one.
* It works whether the file downloads automatically when the page loads or is triggered by an earlier action, such as a [`click`](/docs/features/browser-requests/actions/click).
* It returns the most recent download, so add a `download_file` action for each file if you're expecting several.
* Set a `timeout` long enough for the file to finish downloading. Larger files need longer, and the action waits until that limit is reached.
* Check your file type is supported: **.pdf, .jpg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif** and **.img.**
* Each `download_file` action collects a file, which is then consumed. Add one action per file you're expecting.
* Don't add more `download_file` actions than there are files. Any extras will wait out their timeout and fail.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="214">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>timeout</code></td><td><code>integer</code></td><td>false</td><td>The maximum amount of time the browser should wait for a file to download. <strong>Default: 5,000 (5s)</strong></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Files Supported

Currently, this only works with the following file formats: **.pdf, .jpg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .img**

### Usage

#### Download a copy of a PDF and open it in the Browser

The following waits 20s for a file to download and then returns it.

```
"actions": [
    {
        "type": "download_file",
        "timeout": 20000
    }
]
```

And the service responds with the file being in the action output:

```
"actions": [
      {
        "id": "act_VHhrUbXjZSaYCPTqbBYD4acCzzeFGH",
        "type": "download_file",
        "query": "download_file?continue_on_fail=false&timeout=20000",
        "timestamp": "2025-05-30T15:02:06.6615306Z",
        "output": "https://storage.gaffa.dev/brq/downloads/5845df07-3749-424e-9c64-9602be19a857.pdf"
      }
    ]
```

### FAQs

#### When do I use the download\_file action?

Use it when the thing you want is already a file — a PDF, an image, a report. It returns the original file rather than a version re-rendered from the page.

#### How do I download a PDF with Gaffa?

Point the browser request to the PDF's URL and add a `download_file` action with a `timeout` long enough for the file to finish downloading. The response gives you a storage URL.

#### Which file types does the download\_file action support?

It supports **.pdf, .jpg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif** and **.img**. Other file types can't be downloaded with this action.

#### What timeout should I set for the download\_file action?

Set it to cover the time it takes to download the file. The 5,000ms default suits small images, while larger documents need more — 20,000ms is a good starting point.

#### How do I download a file that starts downloading on its own?

Just add a `download_file` action. Gaffa tracks downloads as they occur, so the file is collected whether it starts on page load or is triggered by an earlier action.

#### How do I download a file that's behind a link?

Add a [`click`](/docs/features/browser-requests/actions/click) for the link, then a `download_file` action with a long enough `timeout`. The click starts the download and `download_file` collects it.

#### How do I download multiple files in a request?

Add a `download_file` action for each file. Each action collects one download and removes it from the queue, so the next action picks up the following file. Give each a `custom_id` to tell them apart.

#### Why did my second download\_file action time out?

It was waiting for a download that hadn't happened. Each action consumes one file, so a second action only works once a second download has been triggered.

#### Does download\_file work on a PDF that opens in the browser?

Yes. It doesn't matter whether the PDF opens in the browser's viewer or downloads when you click a link. Either way, the action returns a storage URL for the file.

#### Where do I find the downloaded file?

The action's output field in the response contains a storage URL for the file, alongside the action's `id`, `type`, `query` and `timestamp`. The file keeps its original name.


# Generate Markdown

Type: `generate_markdown`

The Markdown output format exports page data (articles, tables, etc.) in a human- and LLM-readable format, removing unnecessary styling and other "junk" that is only relevant to the site's proper functioning.

Gaffa exports [GitHub-flavoured markdown](https://github.github.com/gfm/) with comments removed and unknown tags ignored.

<details>

<summary><strong>Tips for using <code>generate_markdown</code></strong></summary>

* Use this as your default when sending page content to an LLM. It keeps headings, lists, tables and links, and drops the rest.
* Add a `selector` to focus on a particular area of the page and cut out the noise. Pointing at `article` or `main` skips navigation, sidebars and footers.
* Use `output_type: "inline"` to get the Markdown straight back in the response. Use `file` for long pages.
* Use [`generate_simplified_dom`](/docs/features/browser-requests/actions/generate-simplified-dom) instead if you need selectors, because Markdown drops classes and attributes.

</details>

### Parameters

<table><thead><tr><th width="184.21875">Name</th><th width="130.66796875">Type</th><th width="106.7734375" data-type="checkbox">Required</th><th width="206.73046875">Description</th></tr></thead><tbody><tr><td><code>selector</code></td><td>string</td><td>false</td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> that defines an element you want to generate markdown from. This is useful if you are only interested in the contents of a certain element.</td></tr><tr><td><code>output_type</code></td><td>string</td><td>false</td><td>Should the action output be saved to a file where a URL will be returned or should the output be included directly in the request.<br><br><strong>Default:</strong> <code>file</code><br><strong>Accepted</strong>: <code>["file", "inline"]</code></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

The following converts the current page to Markdown:

```json
"actions": [
  {
    "type": "generate_markdown"
  }
]
```

The following converts only a specific element to Markdown and returns it inline:

```json
"actions": [
  {
    "type": "generate_markdown",
    "selector": "article",
    "output_type": "inline"
  }
]
```

### Example Output

{% file src="/files/mTvssb40IQyvq8lyNhhG" %}

### FAQs

#### When do I use generate\_markdown?

Use it when you want the readable content of a page, article, listing, or documentation page, especially for sending to an LLM. It's the cleanest and smallest output Gaffa produces.

#### What does generate\_markdown output?

It outputs GitHub-flavoured Markdown of the page, with comments removed and unknown tags ignored. Headings, lists, links and tables are kept. Styling and scripts are stripped out.

#### How do I convert only part of a page to Markdown?

Set the `selector` parameter to the element you want, like `"article"` or `"main"`. Gaffa converts only that element's contents, so you skip navigation, sidebars and footers.

#### Should I use file or inline output?

Use `inline` to get the Markdown directly in the response, which suits agents and shorter pages. Use `file`, the default, when the page is long.

#### **When should I use capture\_dom instead of generate\_markdown?**

Use [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) when you need the actual markup attributes, scripts, and full link URLs. Use `generate_markdown` when you only need the content, which is most of the time.

#### When should I use parse\_json instead of generate\_markdown?

Use [`parse_json`](/docs/features/browser-requests/actions/parse-json) when you need specific fields in a fixed structure. Use `generate_markdown` when you want the whole page as text and will handle the rest yourself.

#### Does generate\_markdown keep tables?

Yes, as Markdown tables. If you plan to process the table in code, use [`parse_table`](/docs/features/browser-requests/actions/parse-table) instead; it returns proper JSON rows rather than text.

#### Should I use file or inline output for generate\_markdown?

Use `inline` to get the Markdown directly in the action's `output` field, which suits agents and shorter pages. Use `file` (the default) when the page is long — that returns a storage URL instead.


# Generate Simplified DOM

**Type:** `generate_simplified_dom`

When you're looking at the DOM of a web page, there's a lot of unnecessary data that can be discarded if you are only interested in the page's elements or looking to export the data into an LLM.\
\
The `generate_simplified_dom` output format processes the HTML in the following way:

* Removes all links in the `head`
* Removes all `script` nodes and links to scripts
* Removes all `style` nodes
* Remove `style` attributes from all elements
* Remove all links to stylesheets
* Remove all `noscript` elements outside of the body
* Finds all `hrefs` with query strings and removes the query strings
* Important `meta` tags are kept, all others are removed
* Remove all `alternate` links
* Remove all SVG paths
* Remove empty text nodes and excessive spacing

<details>

<summary><strong>Tips for using <code>generate_simplified_dom</code></strong></summary>

* Use this when an agent needs to look at a page and work out what to click or type next. It keeps the classes, IDs and structure that selectors are built from.
* Use it instead of [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) for anything going to an LLM. Same structure, far fewer tokens.
* Don't use it to debug styling — styles and stylesheets are removed.
* Note that query strings are stripped from links, so use [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) if you're collecting URLs with parameters.

</details>

### Parameters

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

The following JSON captures the page's DOM and simplifies it.

```json
"actions": [
    {
        "type": "generate_simplified_dom"
    }
]
```

{% hint style="info" %}
We are actively working to improve this and to make this process more configurable - let us know if there's something you think we can improve.
{% endhint %}

### Example Output

{% file src="/files/ywz6DNXI6USCGkeInqGv" %}

### **FAQs**

#### When do I use generate\_simplified\_dom?

Use it when you need to understand a page's structure — usually to find selectors on a site you haven't scraped before, or to give an AI agent something to plan its next action from.

#### What does generate\_simplified\_dom remove?

It removes scripts, styles and style attributes, stylesheet and alternate links, noscript elements outside the body, SVG paths, most meta tags, query strings on links, and empty text.

#### When should I use capture\_dom instead of generate\_simplified\_dom?

Use [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) when you need something simplification removes — inline scripts, style attributes or full link URLs. Use the simplified version for everything else.

#### When should I use generate\_markdown instead of generate\_simplified\_dom?

Use [`generate_markdown`](/docs/features/browser-requests/actions/generate-markdown) when you only need the content. Use `generate_simplified_dom` when you need the structure too, like classes and IDs to build selectors from.

#### How can an AI agent find selectors on a page it doesn't know?

Run `generate_simplified_dom` first, look through the structure for stable IDs, classes or ARIA labels, then build [`click`](/docs/features/browser-requests/actions/click), [`type`](/docs/features/browser-requests/actions/type) or [`capture_element`](/docs/features/browser-requests/actions/capture-element) actions from what you find.

#### Does generate\_simplified\_dom keep links?

Yes, but without their query strings. If you need full URLs with parameters, use [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) or [`capture_element`](/docs/features/browser-requests/actions/capture-element) instead.


# Loop

{% hint style="danger" %}
Beta feature: This action is available on all plans but is still in beta, so the behaviour and parameters are subject to change.If you have any feedback, please email [support](https://gaffa.dev/contact).
{% endhint %}

**Type**: `loop`

Repeat a sequence of nested actions, either for a fixed number of iterations or until one of them fails.

This is how you capture multiple pages in a single browser request. Rather than sending a request per page, you nest your actions inside a `loop` and Gaffa runs them over and over, so a single request can paginate through a set of results, click through a numbered list, or keep interacting with dynamic content until it runs out.

<details>

<summary><strong>Tips for using the <code>loop</code> action</strong></summary>

* Set `timeout` explicitly. The 20-second default covers the whole loop, not each iteration, and most pagination runs need considerably more than that.
* Put capture actions before the action that navigates. A failure skips everything after it in that pass, so a capture placed after the click never runs on the final page.
* Give each nested action a `custom_id`. Everything comes back in one flat list, and an id is the most reliable way to tell which step produced which output.
* Handle cookie banners and consent popups outside the loop, with `continue_on_fail: true`. They only appear once, so there's no reason to retry them on every pass.
* Set `continue_on_fail: true` on the loop if you have actions queued after it; otherwise, a normal pagination-ending stop the request.
* Wait for the pagination control to exist before the loop starts. If the first iteration runs before the page has rendered, the click fails, and the loop exits on pass one.
* Keep `max_iterations` close to the real page count. A high value combined with a bad selector means the browser keeps clicking until the timeout catches it.

</details>

### Parameters

<table><thead><tr><th width="173.61932373046875">Name</th><th width="139.83172607421875">Type</th><th width="110.1640625" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>actions</code></td><td><code>action[]</code></td><td>true</td><td>The ordered list of actions to run on every iteration. Nested actions are written exactly like the actions in your request and support the full set of action types, with the exception of <code>loop</code> itself.</td></tr><tr><td><code>max_iterations</code></td><td><code>integer</code></td><td>false</td><td>The upper bound on how many times the loop can run. Also acts as the iteration count when <code>iterations</code> is not set.<br><strong>Default: 10</strong><br><strong>Min:</strong> 1<br><strong>Max:</strong> 1,000</td></tr><tr><td><code>iterations</code></td><td><code>integer</code></td><td>false</td><td>A fixed number of times to run the nested actions. If you send this alongside <code>max_iterations</code>, the loop runs whichever of the two is lower. <strong>Min:</strong> 1<br><strong>Max:</strong> 100</td></tr><tr><td><code>timeout</code></td><td><code>integer</code></td><td>false</td><td><p>The maximum amount of time the whole loop can run before it is cancelled, in milliseconds. This covers every iteration combined, not each one. </p><p><strong>Default: 20,000 (20s)</strong></p></td></tr><tr><td><code>stop_on_fail</code></td><td><code>boolean</code></td><td>false</td><td>Whether the loop should exit when a nested action fails. A nested action with its own <code>continue_on_fail</code> set to <code>true</code> does not trigger this.<br><strong>Default:</strong> <code>true</code></td></tr><tr><td><code>continue_on_fail</code></td><td><code>boolean</code></td><td>false</td><td>Whether the loop itself is reported as a success when it exits early. Set it to <code>false</code> to pass the failure up to the parent action list, or <code>true</code> to treat an early exit as a clean finish.<br><strong>Default:</strong> <code>false</code></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

{% hint style="warning" %}
The default `timeout` is 20 seconds for the entire loop, not per iteration. That is short for most pagination jobs, so set it explicitly to something higher than the combined time of all your iterations. Bear in mind the whole request is still capped by your plan's [`time_limit`](/docs/features/browser-requests/settings#time-limit), so a loop `timeout` above that won't buy you extra time.
{% endhint %}

### How the loop ends

A loop stops for one of three reasons:

* It reaches its iteration count.
* A nested action fails while `stop_on_fail` is `true`.
* It hits its `timeout`.

The second one is what makes pagination work. You don't need to know how many pages a site has. You point `click` at the "next" control, set a safe `max_iterations`, and when there is no next page left, the selector matches nothing, the click fails, and the loop exits.

### What happens when a nested action fails

When a nested action fails without `continue_on_fail: true` of its own, the rest of that iteration is abandoned. Every action positioned after it is skipped for that pass, and `stop_on_fail` then decides what the loop does next: exit, or start the following iteration.

Setting `continue_on_fail: true` on a nested action changes this. The failure is treated as expected, so the remaining actions in that pass still run and the loop carries on regardless of `stop_on_fail`.

This is why capture actions belong before the action that navigates, never after it. A capture placed after the click that ends the loop never runs on the final page.

<table data-header-hidden><thead><tr><th width="166.59515380859375"></th><th width="162.758544921875"></th><th width="172.38421630859375"></th><th></th></tr></thead><tbody><tr><td>Nested action <code>continue_on_fail</code></td><td>Loop <code>stop_on_fail</code></td><td>Loop <code>continue_on_fail</code></td><td>Result</td></tr><tr><td><code>true</code></td><td>either</td><td>either</td><td>The failure is ignored. The rest of the iteration runs and the loop continues.</td></tr><tr><td><code>false</code></td><td><code>true</code></td><td><code>false</code></td><td>The rest of the iteration is skipped. The loop exits as failed. The parent action list stops unless it has <code>continue_on_fail: true</code>.</td></tr><tr><td><code>false</code></td><td><code>true</code></td><td><code>true</code></td><td>The rest of the iteration is skipped. The loop exits successfully. The parent action list continues.</td></tr><tr><td><code>false</code></td><td><code>false</code></td><td>either</td><td>The rest of the iteration is skipped. The loop moves to the next iteration and eventually succeeds.</td></tr></tbody></table>

### When a loop times out

A loop that hits its `timeout` is reported with `action_timed_out` and treated the same as any other failure, so its `continue_on_fail` decides whether the request carries on. Iterations already completed are kept, along with their outputs, and any actions finished during the final incomplete pass are recorded too. The `iterations` count only includes passes that ran to completion.

### Actions that come after the loop

A pagination loop normally ends on a failed click, which is a successful run rather than a broken one. If you have further actions queued after the loop, leaving `continue_on_fail` as `false` means those actions never execute and the request returns an `action_failed` error. Set it to `true` when you want the request to carry on, for example to screenshot the final page once the loop is done:

```json
"actions": [
    {
      "type": "loop",
      "max_iterations": 20,
      "timeout": 300000,
      "stop_on_fail": true,
      "continue_on_fail": true,
      "actions": [
        { "type": "capture_dom" },
        { "type": "click", "selector": "a.next-page", "timeout": 15000 }
      ]
    },
    { "type": "capture_screenshot" }
]
```

Here the loop runs until the next-page link is gone, exits cleanly, and the screenshot is still taken. But with `continue_on_fail` set to `false` the screenshot would appear in the response marked `action_cancelled` instead.

### Choosing max\_iterations

<table><thead><tr><th width="330.438232421875">Scenario</th><th>Suggested value</th></tr></thead><tbody><tr><td>Known, fixed page count</td><td>The exact count</td></tr><tr><td>Unknown but bounded site</td><td>A conservative upper bound, for example 50 to 100</td></tr><tr><td>Open-ended, as a safety net</td><td>1,000, combined with a tight <code>timeout</code></td></tr></tbody></table>

Set it high enough to cover the real page count, but not so high that a bad selector leaves the browser clicking for minutes. The `timeout` is your primary safety valve for open-ended loops, and requests are also bound by the [maximum running time](/docs/features/browser-requests/settings#time-limit) on your account, which varies by plan.

### Usage

#### Paginate until the next button disappears

The pattern below is the one you'll reach for most often. Capture the page, click through to the next one, and let the failing click end the loop.

```json
"actions": [
    {
      "type": "loop",
      "max_iterations": 20,
      "timeout": 300000,
      "stop_on_fail": true,
      "continue_on_fail": false,
      "actions": [
        { "type": "wait", "time": 1000 },
        { "type": "scroll", "percentage": 100 },
        { "type": "capture_dom" },
        {
          "type": "click",
          "selector": "a.next-page",
          "timeout": 15000
        }
      ]
    }
]
```

Note the order. Each iteration takes a snapshot of the page it is currently on and then navigates away. A capture placed after the click would never run on the final pass, because the click that ends the loop takes the rest of the iteration with it.

#### Dismiss banners, then paginate

Consent banners and cookie notices only appear once, so handle them outside the loop rather than on every pass. Give each one `continue_on_fail: true` so the request keeps going when the banner isn't there.

```json
{
  "url": "https://www.airfleets.net/listing/a350-1.htm",
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": true,
    "actions": [
      {
        "type": "click",
        "selector": "input.buttonform[value=\"I'am not a robot\"]",
        "timeout": 10000,
        "continue_on_fail": true
      },
      {
        "type": "click",
        "selector": "button.qc-usp-close-icon",
        "timeout": 10000,
        "continue_on_fail": true
      },
      {
        "type": "click",
        "selector": "button[aria-label='Close success modal']",
        "timeout": 10000,
        "continue_on_fail": true
      },
      {
        "type": "loop",
        "stop_on_fail": true,
        "continue_on_fail": false,
        "max_iterations": 2,
        "timeout": 300000,
        "actions": [
          { "type": "wait", "time": 1000 },
          { "type": "capture_dom" },
          {
            "type": "click",
            "selector": "a.page:has-text('Next page')",
            "timeout": 15000
          }
        ]
      }
    ],
    "time_limit": 300000,
    "block_ads": true
  }
}
```

The three clicks before the loop each time out on this site because the banners don't appear, but `continue_on_fail: true` means the request carries on regardless. Inside the loop, `a.page:has-text('Next page')` matches the next-page link by its visible text, and the request finishes on page three after two iterations.

#### Click through numbered pagination

Not every site labels its next-page control. Where pagination is a row of numbers, a CSS sibling [`selector`](/docs/features/browser-requests/selectors) takes you from the active page to the next page.

```json
{
  "url": "https://repertoire.hebergement.tourisme.gouv.qc.ca",
  "proxy_location": "us",
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": true,
    "actions": [
      {
        "type": "wait",
        "selector": "[alt=\"Suivant\"]",
        "timeout": 10000,
        "continue_on_fail": true
      },
      {
        "type": "loop",
        "stop_on_fail": true,
        "continue_on_fail": false,
        "max_iterations": 5,
        "timeout": 300000,
        "actions": [
          { "type": "wait", "time": 1000 },
          { "type": "scroll", "percentage": 100 },
          {
            "type": "click",
            "selector": "#pagination span.active + span",
            "timeout": 15000
          }
        ]
      }
    ],
    "time_limit": 300000,
    "block_ads": true
  }
}
```

`#pagination span.active + span` selects the span immediately after the currently active page indicator. On the last page, that sibling doesn't exist, the click fails, and the loop exits. The `wait` before the loop makes sure the pagination has rendered before the first iteration runs.

This site paginates without changing the URL, so `actual_url` in the response stays on the original address no matter how many pages you move through.

### Response format

The loop is returned as a single action containing an `iterations` count and a nested `actions` array. Adding a `custom_id` to each nested action makes the result far easier to read, since it comes back on every entry unchanged:

```json
{
  "id": "act_VuCHkaVpgkmKZsZ7YwomELdtbEkZCz",
  "type": "loop",
  "custom_id": "pagination-loop",
  "timestamp": "2026-08-11T09:51:27.7874109Z",
  "iterations": 2,
  "actions": [
    {
      "id": "act_VuCHkmUpTy9mZodXsL3hTZvVPUpsnZ",
      "type": "wait",
      "custom_id": "settle",
      "timestamp": "2026-08-11T09:51:17.2365316Z"
    },
    {
      "id": "act_VuCHmS83cSrkua3Wo6wWqCUocE3yZA",
      "type": "capture_dom",
      "custom_id": "page-html",
      "timestamp": "2026-08-11T09:51:20.4515024Z",
      "output": "https://storage.gaffa.dev/brq/dom/brq_VuCHkdpZFrk6SaFnb3Zxw8iskoGaKt/act_VuCHmS83cSrkua3Wo6wWqCUocE3yZA_raw.txt"
    },
    {
      "id": "act_VuCHmSdCjKQ3DSrooqFUp5EHNk6fTh",
      "type": "click",
      "custom_id": "next-page",
      "timestamp": "2026-08-11T09:51:20.9377327Z"
    },
    {
      "id": "act_VuCHmXEa7CUpkb19itmkYDYEWk3R5X",
      "type": "wait",
      "custom_id": "settle",
      "timestamp": "2026-08-11T09:51:24.1053288Z"
    },
    {
      "id": "act_VuCHnE6Z1TkdTE8UBGyyxgn26nauAB",
      "type": "capture_dom",
      "custom_id": "page-html",
      "timestamp": "2026-08-11T09:51:24.1206194Z",
      "output": "https://storage.gaffa.dev/brq/dom/brq_VuCHkdpZFrk6SaFnb3Zxw8iskoGaKt/act_VuCHnE6Z1TkdTE8UBGyyxgn26nauAB_raw.txt"
    },
    {
      "id": "act_VuCHnJVQ3etENxshyaUeHTNqZzem4N",
      "type": "click",
      "custom_id": "next-page",
      "timestamp": "2026-08-11T09:51:24.6279232Z"
    }
  ]
}
```

Every action from every pass is recorded in execution order as a single flat list rather than grouped by iteration. Outputs sit on the individual nested actions, so reading the [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) entries, top to bottom, gives you page one, then page two, and so on.

Don't rely on counting entries to work out where one iteration ends and the next begins. A pass that failed partway through, or one interrupted by the loop's `timeout` records only the actions it completed, which leaves an uneven final group.

### FAQs

#### When do I use the loop action?

Use it when you need more than one page from a single browser request. Common cases include paginated search results, numbered lists, and feeds where you repeat the same few actions until the content runs out.

#### How do I paginate a site when I don't know how many pages there are?

Point a [`click`](/docs/features/browser-requests/actions/click) at the next-page control and leave `stop_on_fail` as `true`. When you reach the last page, the selector matches nothing, the click fails, and the loop exits on its own. Use `max_iterations` as a safety ceiling rather than an exact count.

#### How do I capture each page inside a loop?

Add a capture action such as [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) to the nested `actions` array, positioned before the action that navigates. It runs once per iteration, and each capture appears as a separate entry with its own output URL in the response.

#### Why does my request return action\_failed when the loop did what I wanted?

Because the loop ended on a failed click, which is the normal way pagination finishes, and the loop's `continue_on_fail` defaults to `false`. Set it to `true` to treat a clean early exit as a success. Your captured pages are in the response either way.

#### What is the difference between iterations and max\_iterations?

`max_iterations` is the ceiling and defaults to 10. `iterations` is an optional fixed count. Send both, and the loop runs whichever is lower. Send neither, and the loop runs 10 times.

#### Why did an action inside my loop not run?

An action is skipped when something before it in the same iteration failed. That happens regardless of `stop_on_fail`, which only controls whether the loop then exits or starts the next pass. To keep the remaining actions running, set `continue_on_fail: true` on the action that is failing.

#### How do I stop a loop from running too long?

Set the loop's `timeout`, which caps the total time across all iterations. Combine it with a realistic `max_iterations` and a `timeout` on each nested action so no single step can stall the whole run.

#### How do I tell which captured page came from which iteration?

Give each nested action a `custom_id`. It's returned unchanged on every entry, so you can read the flat list as repeating groups. Counting entries alone isn't reliable, because an iteration cut short by a failure or by the loop's `timeout` records only the actions it completed.

#### Can I put a loop inside another loop?

No. Nesting a `loop` inside another `loop` is rejected before the request runs, with an `invalid_action_param` error. For two-level pagination, such as a list of categories, each with its own paged results, send a separate request per category, with a single loop inside each.


# Print

**Type**: `print`

Request that the browser print the page to a PDF.

<details>

<summary><strong>Tips for using <code>print</code> action</strong></summary>

* The PDF uses the site's print styles, so it may not look like the page on screen.
* Use `"orientation": "landscape"` for wide content, such as tables and dashboards.
* Set `"margin": 0` if you want the content to reach the edge of the page. The default is 20px.
* Don't use `print` on a page that's already a PDF. Use [`download_file`](/docs/features/browser-requests/actions/download-file) to get the original instead.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="226">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>size</code></td><td><code>string</code></td><td>false</td><td>The size of paper the page should be printed to.<br><strong>Default:</strong> <code>A4</code><br><strong>Accepted</strong>: <code>["A4"]</code></td></tr><tr><td><code>margin</code></td><td><code>integer</code></td><td>false</td><td>The margin of the page in pixels when the page is printed to PDF.<br><strong>Default: 20</strong></td></tr><tr><td><code>orientation</code></td><td><code>string</code></td><td>false</td><td>The orientation of the printed PDF page.<br><strong>Default: portrait</strong><br><strong>Accepted:</strong> <code>["portrait", "landscape"]</code></td></tr><tr><td></td><td></td><td>false</td><td></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

#### Print a page in landscape to PDF

The following JSON prints the page to a PDF in landscape orientation with a 20px margin.

```json
"actions": [
    {
        "type": "print",
        "size": "A4",
        "orientation": "landscape",
        "margin": 20
    }
]
```

### Example Output

{% file src="/files/977bvo93zl5BF0hIBxk9" %}

### FAQs

#### When do I use the print action?

Use it when you need a PDF of a web page — for a report, a record, or something to share. It gives you a paginated A4 document rather than an image or HTML.

#### How do I convert a web page to PDF with Gaffa?

Add a `print` action to your request. It renders the page to an A4 PDF using the site's print styles. Use `orientation` and `margin` to control the layout.

#### How do I print a page in landscape?

Set `"orientation": "landscape"`. Use it for wide tables, dashboards and anything that loses columns when squeezed into portrait A4.

#### Why does my PDF look different to the web page?

Most sites have a separate print stylesheet that hides navigation or changes colours. If you need the page as it looks on screen, use [`capture_screenshot`](/docs/features/browser-requests/actions/capture-screenshot) with `size: "fullscreen"` instead.

#### What paper sizes can I use?

A4 only at the moment, and it's the default. Use `orientation` and `margin` to control everything else about the layout.

#### How do I remove the margins?

Set `"margin": 0`. The default is 20 pixels, which suits documents but crops designs meant to reach the edge of the page.

#### When should I use download\_file action instead of print action?

Use [`download_file`](/docs/features/browser-requests/actions/download-file) when the browser already has a PDF open, and you want the original file. Use `print` to create a PDF from a page that isn't one yet.


# Parse JSON

{% hint style="info" %}
**Paid Action:** This action consumes credits based on the amount of content parsed. See more [below](#pricing).
{% endhint %}

**Type:** `parse_json`

The `parse_json` action extracts data from web pages and online PDFs. It uses AI to parse web content from text into a pre-defined data schema and return it as a JSON object.

The action lets you convert unstructured content, such as academic papers, forms, and webpages, into JSON objects that you can use in automations, analysis, or further processing.

*This feature currently works for online PDFs and web page text.*

<details>

<summary><strong>Tips for using <code>parse_json</code></strong></summary>

* Prefer a deterministic action where one exists. AI parsing is powerful, but an action that reads the page directly gives the same answer every time — [`parse_table`](/docs/features/browser-requests/actions/parse-table) is the obvious example when the data is in an HTML table.
* Add a `selector` so the model only sees the part of the page you care about. It's cheaper, faster and more accurate.
* Write field descriptions as instructions, not labels. Say what format you want and what to do when a value is missing.
* The more specific your descriptions and instructions, the better the result.
* Use `input_token_cap` and `max_pages` to keep costs predictable on jobs you run often.
* Save schemas you reuse with `POST /v1/schemas` and call them with `data_schema_id`.
* Use `output_type: "inline"` to get the JSON back in the response instead of a file URL.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="212">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>data_schema_id</code></td><td><code>string</code></td><td>true</td><td>The id of the data schema you have defined that you want to transform the content into.<br><br><strong>You must provide a <code>data_schema</code> or <code>data_schema_id</code> with your request.</strong></td></tr><tr><td><code>data_schema</code></td><td><code>json</code></td><td>true</td><td><p>A JSON object describing the data_schema you want to transform the content into.<br></p><p><strong>You must provide a <code>data_schema</code> or <code>data_schema_id</code> with your request.</strong></p></td></tr><tr><td><code>instruction</code></td><td><code>string</code></td><td>false</td><td>A custom instruction, in addition to any detail you have added to the data schema, that you want to include with this particular parse.</td></tr><tr><td><code>model</code></td><td><code>string</code></td><td>false</td><td>The AI model you wish to use to parse the content into JSON. <br><strong>Default:</strong> <code>gpt-4o-mini</code><br><strong>Accepted</strong>: <code>["gpt-4o-mini"]</code></td></tr><tr><td><code>input_token_cap</code></td><td><code>int</code></td><td>false</td><td>The max number of source input tokens that will be passed to the AI model to parse. This can be used to prevent unnecessary credit usage. If your source input is longer than the token cap, it will be abbreviated.<br><strong>Default:</strong> 1,000,000</td></tr><tr><td><code>selector</code></td><td><code>string</code></td><td>false</td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> that defines an element you want to parse the content of - this is useful if you are only interested in the contents of a certain element.</td></tr><tr><td><code>output_type</code></td><td><code>string</code></td><td>false</td><td>Should the action output be saved to a file where a URL will be returned or should the parsed  JSON object be included directly in the request.<br><br><strong>Default:</strong> <code>file</code><br><strong>Accepted</strong>: <code>["file", "inline"]</code></td></tr><tr><td><code>max_pages</code></td><td><code>int</code></td><td>false</td><td>If you are parsing a PDF you can specify this parameter to limit the number of pages that are passed to the LLM.<br><br><strong>Default:</strong> no limit</td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

#### Defining Data Schemas

A data schema tells the model exactly what JSON structure to produce.

You can define schemas in two ways:

* **Inline schemas** (defined directly inside the action)
* Reusable schemas (created via the Schema API and referenced by ID in your requests)

### Schema Structure

A schema has:

<table><thead><tr><th width="156">Property</th><th width="138">Type</th><th>Description</th></tr></thead><tbody><tr><td><code>description</code></td><td>string</td><td>Explains what data the schema extracts and provides context to help the AI model understand the extraction goal.<br><strong>Example</strong>: <code>"Extract product details from this e-commerce product page"</code></td></tr><tr><td><code>fields</code></td><td>array</td><td>Each field defines a piece of data to extract from the content. See field properties below.</td></tr><tr><td><code>name</code></td><td>string</td><td>This identifies the schema and should clearly indicate what data it extracts.<br><strong>Example</strong>: <code>"ProductInfo"</code>, <code>"ArticleMetadata"</code>, <code>"ContactForm"</code></td></tr></tbody></table>

Each field in the `fields` array has:

<table><thead><tr><th width="154"></th><th width="143"></th><th></th></tr></thead><tbody><tr><td><code>description</code></td><td>string</td><td><p>Include details about format, handling of missing values, or special cases.</p><p><strong>Example</strong>: <code>"Maximum salary in GBP. If only one value is provided, use the same value for both min and max. Return null if not provided."</code></p></td></tr><tr><td><code>fields</code></td><td>array</td><td>Required only for <code>object</code> and <code>array</code> types.</td></tr><tr><td><code>name</code></td><td>string</td><td>Use clear, descriptive names that follow your preferred naming convention (e.g., <code>snake_case</code> or <code>camelCase</code>). <strong>Example</strong>: <code>"product_name"</code>, <code>"published_date"</code>, <code>"author_email"</code></td></tr><tr><td><code>type</code></td><td>string</td><td>Determines how the AI interprets and structures the extracted data. Must be one of the supported types below.</td></tr></tbody></table>

#### Supported Field Types

| Type     | Description              |
| -------- | ------------------------ |
| array    | List of items            |
| boolean  | True/False               |
| datetime | timestamp                |
| decimal  | Precise decimal          |
| double   | Floating-point number    |
| integer  | Whole number             |
| object   | Nested structured object |
| string   | Text value               |

### Inline Schema Example

```json
"actions": [
  {
    "type": "parse_json",
    "model": "gpt-4o-mini",
    "output_type": "inline",
    "instruction": "Extract metadata from an article",
    "data_schema": {
      "name": "ArticleMetadata",
      "description": "Schema for parsing article Metadata",
      "fields": [
        {
          "type": "string",
          "name": "title",
          "description": "Article title"
        },
        {
          "type": "string",
          "name": "author",
          "description": "Author name"
        },
        {
          "type": "datetime",
          "name": "published",
          "description": "Publication date"
        }
      ]
    },
  }
]
```

This example shows:

* **Simple fields** (`string`, `datetime`) for basic data
* **Object fields** for grouped related data with nested `fields`
* **Array fields** for lists of items with nested `fields` defining each item's structure

### Schema Operations

Instead of defining schemas inline each time, you can save them to your Gaffa account and reuse them across multiple requests. This makes your actions more readable, easier to maintain, and ensures consistency when parsing similar content.

#### Creating a Saved Schema

Use the [POST /v1/schemas](https://gaffa.dev/docs/api-reference/post-v1-schemas) endpoint to create a reusable schema:

```bash
curl -L \
  --request POST \
  --url 'https://api.gaffa.dev/v1/schemas' \
  --header 'X-API-Key: YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "ProductInfo",
    "description": "Extract product details from e-commerce pages",
    "fields": [
      {
        "type": "string",
        "name": "product_name",
        "description": "The product title"
      },
      {
        "type": "decimal",
        "name": "price",
        "description": "Current price"
      },
      {
        "type": "boolean",
        "name": "in_stock",
        "description": "Product availability"
      },
      {
        "type": "object",
        "name": "ratings",
        "description": "Product rating information",
        "fields": [
          {
            "type": "double",
            "name": "average",
            "description": "Average rating score"
          },
          {
            "type": "integer",
            "name": "total_reviews",
            "description": "Number of reviews"
          }
        ]
      },
      {
        "type": "array",
        "name": "tags",
        "description": "Product tags",
        "fields": [
          {
            "type": "string",
            "name": "tag",
            "description": "Individual tag name"
          }
        ]
      }
    ]
  }'
```

**Response:**

```json
{
  "id": "schema_abc123xyz",
  "name": "ProductInfo",
  "description": "Extract product details from e-commerce pages",
  "fields": [...]
}
```

Save the `id` returned in the response, you'll use this to reference the schema in your requests

### Managing Schemas

#### **List all schemas**:

Allows you to view all schemas saved to your account:

Endpoint: [GET /v1/schemas](https://gaffa.dev/docs/api-reference/get-v1-schemas)

```bash
curl -L \
  --url 'https://api.gaffa.dev/v1/schemas' \
  --header 'X-API-Key: YOUR_API_KEY' \
  --header 'Accept: */*'
```

#### **Update a schema**:

Allows you to modify an existing schema by its ID:

Endpoint: [PUT /v1/schemas](https://gaffa.dev/docs/api-reference/put-v1-schemas)

```bash
curl -L \
  --request PUT \
  --url 'https://api.gaffa.dev/v1/schemas/{id}' \
  --header 'X-API-Key: YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "id": "schema_abc123xyz",
    "name": "ProductInfo",
    "description": "Extract detailed product information from e-commerce pages",
    "fields": [
      {
        "type": "string",
        "name": "product_name",
        "description": "The product title"
      },
      {
        "type": "decimal",
        "name": "price",
        "description": "Current price"
      },
      {
        "type": "string",
        "name": "brand",
        "description": "Product brand name"
      }
    ]
  }'
```

#### **Delete a schema**:

Removes a schema from your account:

Endpoint: [DELETE /v1/schemas/:id](https://gaffa.dev/docs/api-reference/delete-v1-schemas-id)

```bash
curl -L \
  --request DELETE \
  --url 'https://api.gaffa.dev/v1/schemas/{id}' \
  --header 'X-API-Key: YOUR_API_KEY' \
  --header 'Accept: */*'
```

### Common Schema Patterns

Simple List Extraction

```json
{
  "name": "TagList",
  "description": "Extract article tags",
  "fields": [
    {
      "type": "array",
      "name": "tags",
      "description": "List of article tags",
      "fields": [
        {
          "type": "string",
          "name": "tag",
          "description": "Individual tag name"
        }
      ]
    }
  ]
}
```

**Nested Objects**

```json
{
  "name": "ProductWithReviews",
  "description": "Product details with nested review data",
  "fields": [
    {
      "type": "string",
      "name": "product_name",
      "description": "Product name"
    },
    {
      "type": "object",
      "name": "pricing",
      "description": "Pricing information",
      "fields": [
        {
          "type": "decimal",
          "name": "current_price",
          "description": "Current price"
        },
        {
          "type": "decimal",
          "name": "original_price",
          "description": "Original price before discount"
        },
        {
          "type": "integer",
          "name": "discount_percentage",
          "description": "Discount percentage"
        }
      ]
    }
  ]
}
```

### Pricing

The credits this action uses depend on the model used. Here are the current supported models and their pricing:

| Model         | Input Token Cost                 | Output Token Cost                 |
| ------------- | -------------------------------- | --------------------------------- |
| `gpt-4o-mini` | 1 credit per 20,000 input tokens | 1 credit per 10,000 output tokens |

### FAQs

#### When do I use the parse\_json action?

Use it when you need specific fields from content that isn't already structured, such as an article, a PDF, a product page, or a form. It uses AI to fit the content into a schema you define.

#### Does parse\_json return the same result every time?

Not necessarily. Because it uses an AI model, running the same request twice can produce slightly different output, particularly for summaries or free-text fields.

#### How do I extract structured data from a web page?

Add a `parse_json` action with a `data_schema` listing the fields you want, or point at a saved schema with `data_schema_id`. Gaffa returns the content in that structure.

#### What's the difference between data\_schema and data\_schema\_id?

`data_schema` defines the structure inside the action itself. `data_schema_id` points to a schema you've saved to your account. Send one or the other with every request.

#### How do I keep parse\_json costs down?

Narrow the input with `selector`, cap it with `input_token_cap`, and limit PDFs with `max_pages`. Use [`parse_table`](/docs/features/browser-requests/actions/parse-table) instead whenever the data is already in an HTML table.

#### How do I get more accurate results from parse\_json?

The more specific your instructions, the better the result. Describe each field's format, units and what to do when a value is missing; add an instruction for context, and use selectors so the model sees less noise.

#### What field types can I use in a schema?

**string, integer, decimal, double, boolean, datetime, object** and **array**. Object and array fields need a nested **fields** list describing what's inside them.

#### Can parse\_json read a PDF?

Yes, it works on online PDFs as well as web pages. Use `max_pages` to limit how many pages get sent to the model, which keeps costs down on long documents.

#### How do I extract a list of items?

Use an **array** field with a nested `fields` list describing each item. For a simple list, nest one **string** field. For records, nest one field per property.

#### When should I use parse\_table instead of parse\_json?

Use [`parse_table`](/docs/features/browser-requests/actions/parse-table) when the data is in a real HTML table. It's exact, gives the same result every time, and costs no credits.


# Parse Table

**Type**: `parse_table`

The `parse_table` action finds a table on a page using a CSS selector and converts it into a structured JSON array with no HTML parsing or post-processing required on your end.

The action reads the table's header row and converts each header into a property name (lowercased, with non-alphanumeric characters replaced by underscores). It then maps each cell value to its corresponding header for every row, returning a clean, ready-to-use JSON array. At the moment, all values are returned as `string` types.

For cases where you need more control, such as handling merged cells, skipping rows, or applying custom transformations, consider using [`capture_dom`](https://gaffa.dev/docs/features/browser-requests/actions/capture-dom) with a parsing library like BeautifulSoup instead.

<details>

<summary><strong>Tips for using <code>parse_table</code></strong></summary>

* Use this rather than [`parse_json`](/docs/features/browser-requests/actions/parse-json) whenever the data is in a real `<table>`. It's exact and costs nothing.
* Headers become your property names, lowercased with non-letters turned into underscores. `Ticket Price (£)` becomes `ticket_price_`.
* Every value comes back as a string, so convert numbers and dates on your side.
* Point the selector at the specific table. Pages often have several, including ones used for layout.
* Scroll and wait first if the table pages or loads as you scroll, so the rows are there when the action runs.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="212">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>selector</code></td><td><code>string</code></td><td>true</td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> that identifies the table you want to parse.</td></tr><tr><td><code>timeout</code></td><td><code>integer</code></td><td>false</td><td>The maximum time in milliseconds to wait for the table to appear. <strong>Default: 5000 (5s)</strong></td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

#### Basic examples

The following request waits up to 1 second for a `.large_table` element to appear, then parses it into JSON:

```json
{
  "url": "https://example.com",
  "settings": {
    "actions": [
      {
        "type": "parse_table",
        "selector": ".large_table",
        "timeout": 1000
      }
    ]
  }
}
```

#### Real-world example

Here is an example using Wikipedia's [List of Countries by GDP (Nominal)](https://en.wikipedia.org/wiki/List_of_countries_by_GDP_\(nominal\)). Wikipedia applies a consistent CSS class to its data tables, making it straightforward to target with a selector:

```json
{
  "url": "https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)",
  "settings": {
    "actions": [
      {
        "type": "parse_table",
        "selector": ".wikitable",
        "timeout": 5000
      }
    ]
  }
}
```

#### Sample output

Notice how column headers like "Country/Territory" and "IMF 2026" are automatically normalized into `country_territory` and `imf_2026.` Spaces and special characters are replaced with underscores, and everything is lowercased, so the output is immediately usable without any cleanup:

```json
[
  {
    "country_territory": "United States",
    "imf_2026": "30,337",
    "imf_year": "2026",
    "world_bank_2023": "27,361",
    "world_bank_year": "2023"
  },
  {
    "country_territory": "China",
    "imf_2026": "19,534",
    "imf_year": "2026",
    "world_bank_2023": "17,795",
    "world_bank_year": "2023"
  }
]
```

For a full walkthrough, including a comparison with the `capture_dom` + BeautifulSoup approach, see our blog post on [how to scrape a table with Python (the easy way)](https://gaffa.dev/blog/how-to-scrape-a-table-with-python-\(the-easy-way\)).

### FAQs

#### When do I use parse\_table?

Use it when the data you want is in an HTML table, and you want it as JSON rows. It's faster, cheaper and more predictable than AI parsing for this one job.

#### How do I extract a table as JSON?

Add a `parse_table` action with a selector for the table. Gaffa turns the headers into property names and each row into an object, and returns the result as JSON.

#### How does parse\_table name the properties?

It takes each header, makes it lowercase, and replaces anything that isn't a letter or number with an underscore. `Ticket Price (£)` becomes `ticket_price_`, so check your keys before mapping them.

#### What data types does parse\_table return?

Everything comes back as a string at the moment, including numbers and dates. Convert them to the types you need after you get the response.

#### When should I use parse\_json instead of parse\_table?

Use [`parse_json`](/docs/features/browser-requests/actions/parse-json) when the data isn't in a real table — a grid built from divs, a list, a PDF, or plain prose — or when you need typed values rather than strings.

#### What if the data looks like a table but isn't one?

`parse_table` only works on `<table>` markup. For grids made of divs or lists, use [`parse_json`](/docs/features/browser-requests/actions/parse-json) with a `schema`, or [`capture_element`](/docs/features/browser-requests/actions/capture-element) and parse the HTML yourself.


# Scroll

**Type**: `scroll`

Request that the browser scrolls to a certain point on the page or, in the case of pages with infinite scrolling, scrolls for a particular amount of time.

<details>

<summary><strong>Tips for using <code>scroll</code> action</strong></summary>

* Use `wait_time` for pages that load more content when you reach the bottom. Gaffa will keep scrolling as the page grows.
* Use `max_scroll_time` to stop an infinite scroll after a set time. It stops the action without failing it.
* Match the speed to the site: `instant` when there's no rate limiting, `slow` when the site throttles fast scrolling, and `instant` with a large `interval` when limits are strict.
* Use [`selector`](/docs/features/browser-requests/selectors) to scroll inside a modal or side panel, since scrolling the page body won't move them.
* Add [`block_dom_removals`](/docs/features/browser-requests/actions/block-dom-removals) first if the site drops rows as you scroll past them.
* `percentage` is a position on the page, not a distance. 50 takes you halfway down, 100 to the bottom.
* Plan your `percentage` values in ascending order across a request. Once you've scrolled to `100%`, asking for `50%` won't take you back — there's no way to scroll up within a request.
* 0 and values lower than the current position are accepted but do nothing, because scroll only moves forward.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="215">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>percentage</code></td><td><code>integer</code></td><td>true</td><td><p>How far down the page to scroll, as a target position rather than a distance. Scrolling only moves forward — see the tip above.</p><p><strong>Range: [0 - 100]</strong><br><strong>Default: 100 (% - scroll to bottom)</strong></p></td></tr><tr><td><code>wait_time</code></td><td>integer</td><td>false</td><td>After arriving at the desired scroll location, the time Gaffa should monitor for changes in the page height before marking the action as successful. Read more <a href="#wait-time">below</a>.<br><strong>Default: 0</strong></td></tr><tr><td><code>max_scroll_time</code></td><td><code>integer</code></td><td>false</td><td>The maximum amount of time the page should be scrolled for, in milliseconds. After this time passes, the action will be cancelled. This doesn't cause the action to fail.<br><strong>Default: 20,000 (20s)</strong><br></td></tr><tr><td><code>scroll_speed</code></td><td><code>string</code></td><td>false</td><td>The speed which the page should scroll to the desired point. You can read more about this <a href="#scroll-speed">below</a>.<br><strong>Default:</strong> <code>medium</code><br><strong>Accepted</strong>: [<code>slow</code>, <code>medium</code>, <code>instant</code>]</td></tr><tr><td><code>interval</code></td><td><code>integer</code></td><td>false</td><td>The amount of time, in milliseconds, that scrolling should pause between scroll events. Read more about this <a href="#scroll-speed-and-interval">below</a>.<br><strong>Default</strong>: 0</td></tr><tr><td><code>timeout</code></td><td><code>integer</code></td><td>false</td><td>The maximum amount of time Gaffa will wait for the page to become scrollable <br><strong>Default: 0</strong></td></tr><tr><td><code>selector</code></td><td><code>string</code></td><td>false</td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> that identifies the element to scroll. If not provided, the page body will be scrolled.</td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Scroll Speed & Interval

Gaffa gives you flexibility over how fast you scroll down the page, which can be really useful to get around restrictions enforced by some sites that detect and limit fast scrolling. By experimenting with `scroll_speed` and `interval`, you will be able to create the perfect scrolling action for your scenario. The speed settings are as follows:

* `instant`- The page will smoothly scroll to the desired position immediately, useful for sites with no rate limits or loading events caused by scroll actions.
* `medium` - Human-like scrolling at a normal speed to the desired position. Gaffa will scroll in much the same way as you would using a mouse.
* `slow`- Human-like scrolling at a very slow speed to the desired position. The speed is comparable to scrolling while reading a page.

`interval`allows you to adjust the scroll speed further by inserting pauses between scroll events.

{% hint style="info" %}
We've found some sites with infinite scrolling and strict rate limits respond better to `instant` speed scroll events to the bottom of the page with large `intervals` between these scrolls to keep within rate limits.
{% endhint %}

### Wait Time

If `wait_time` is set to 0, and Gaffa arrives at the desired location, then Gaffa will immediately mark the action as succeeded. However, if another value is set, the page will be monitored for the specified duration to check for further expansions. If, during this period, the page expands again, then Gaffa will continue scrolling to the desired location, and the wait will reset.

{% hint style="info" %}
This can be really useful if you find that the site takes some time to load additional items when you reach the bottom of the page, and more items load after the action has succeeded.
{% endhint %}

### Usage

#### Scroll a particular percentage down the page

The following code will scroll halfway down the page.

```json
"actions": [
      {
        "type": "scroll",
        "percentage": 50
      }
]
```

#### Scroll an infinitely scrolling webpage

The following code will scroll to the bottom of the page and then keep scrolling when new content loads for a maximum of 25 seconds, waiting 1 second for new content and scrolling at a slow pace with 1 second between scroll actions.

```json
"actions": [
      {
        "type": "scroll",
        "percentage": 100,
        "scroll_speed": "slow",
        "max_scroll_time": 25000,
        "interval": 1000,
        "wait_time": 1000
      }
]
```

### Read more

<table data-view="cards"><thead><tr><th></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="image">Cover image</th></tr></thead><tbody><tr><td>How to Handle Infinite Scrolling and Dynamic Loading with Gaffa’s Scroll Action</td><td><a href="https://gaffa.dev/blog/how-to-handle-infinite-scrolling-and-dynamic-loading-with-gaffas-scroll-action">https://gaffa.dev/blog/how-to-handle-infinite-scrolling-and-dynamic-loading-with-gaffas-scroll-action</a></td><td><a href="https://2341007864-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyUba6osOT5MkKiV0wmgr%2Fuploads%2F8CL05BsikQX2AP0xfEDT%2Fscroll_illustration_1920x1080_V4.jpg?alt=media&amp;token=893fae8e-abc6-45a7-863d-34b8a344f1ba">scroll_illustration_1920x1080_V4.jpg</a></td></tr></tbody></table>

### FAQs

#### When do I use scroll action?

Use it when content loads as you move down the page, when you need to reach something further down before clicking it, or when images only load once they're on screen.

#### How do I scrape an infinite scroll page?

Use a `scroll` action with `"percentage": 100`, a `wait_time` so Gaffa keeps going as new content loads, and a `max_scroll_time` to stop it after a set period.

#### What does wait\_time do?

After reaching the position you asked for, Gaffa watches the page for that duration to see if it grows. If it does, scrolling continues, and the wait starts again, until the page stops growing.

#### What does max\_scroll\_time do?

It sets the maximum duration the scroll can run, defaulting to 20 seconds. When that time is up, the action stops. It doesn't fail, so the rest of your actions still run.

#### Which scroll speed should I use?

Use `instant` when the site has no rate limits, `medium` for normal human-like scrolling, and `slow` for sites that throttle fast scrolling. For strict limits, use `instant` with a large `interval`.

#### Can I scroll back up a page with Gaffa?

No. `percentage` sets a target position, and scrolling only moves forward, so once you've reached a point, requesting a lower value does nothing. Order your `scroll` actions from smallest to largest across a request.

#### What does the percentage parameter actually scroll to?

It's a target position on the page, not a fixed point at the bottom. `"percentage": 50` scrolls to the halfway mark from wherever the page currently is, as long as that's further down than the current position.

#### How do I scroll inside a modal or side panel?

Set [`selector`](/docs/features/browser-requests/selectors) to the element that scrolls. Without it, Gaffa scrolls the page body, which does nothing when the scrollable part is a panel inside the page.

#### Why does my capture only show the last few rows?

The site is removing rows as they scroll out of view. Add a [`block_dom_removals`](/docs/features/browser-requests/actions/block-dom-removals) action before the scroll so every loaded row stays on the page.


# Type

**Type**: `type`

Request that the browser enter a specific piece of text into a field.

<details>

<summary><strong>Tips for using <code>type</code> action</strong></summary>

* Typing doesn't submit the form. Add a `click` for the submit button afterwards.
* Add a [`wait`](/docs/features/browser-requests/actions/wait) for the input before typing if the form loads after the page does.
* Point the [`selector`](/docs/features/browser-requests/selectors) at the input itself, not the div around it.
* Use single quotes inside attribute [`selectors`](/docs/features/browser-requests/selectors) so your JSON stays valid: `"input[name='email']"`. You can also escape double quotes with backslashes.
* Close any cookie banner or modal first. An overlay can block the field even when your selector is correct.
* Gaffa types character by character rather than pasting the whole string, which suits sites that watch for unusual input.

</details>

### Parameters

<table data-full-width="false"><thead><tr><th width="212">Name</th><th width="130">Type</th><th width="108" data-type="checkbox">Required</th><th>Description</th></tr></thead><tbody><tr><td><code>selector</code></td><td><code>string</code></td><td>true</td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> that defines the text field where the browser should type.</td></tr><tr><td><code>text</code></td><td><code>string</code></td><td>true</td><td>The text the browser should enter into the text field.</td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

{% hint style="info" %}
Sites that use more advanced bot detection often use keyboard events to detect unusual activity on their site, rather than immediately dropping all characters of the text into a field, our platform types the text in a human-like manner.
{% endhint %}

### Usage

#### Type into a text box

The following action will type into a particular text field.

```json
"actions": [
      {
            "type": "type",
            "selector": "#postform-text",
            "text": "Hello world!"
      }
]
```

### FAQs

#### When do I use the type action?

Use it to enter text into a field — a search box, a login form, a filter. For checkboxes, radio buttons and dropdowns, use [`click`](/docs/features/browser-requests/actions/click) instead.

#### How do I fill in a form field with Gaffa?

Add a `type` action with a [`selector`](/docs/features/browser-requests/selectors) for the input and the text you want entered. Gaffa types the characters one at a time, as a person would.

#### Does the type action submit the form?

No. It only enters the text. Add a [`click`](/docs/features/browser-requests/actions/click) for the submit button afterwards, then a [`wait`](/docs/features/browser-requests/actions/wait) for something on the next page before you capture anything.

#### How do I fill in more than one field?

Add one [`type`](/docs/features/browser-requests/actions/type) action per field, in the order someone would fill them in. Finish with a [`click`](/docs/features/browser-requests/actions/click) on the submit button and a [`wait`](/docs/features/browser-requests/actions/wait) for the result.

#### Why isn't my type action working?

Usually the [`selector`](/docs/features/browser-requests/selectors) points at the wrapper rather than the input, the field hasn't loaded yet, or a cookie banner or modal is sitting over it. Target the `input` directly, add a [`wait`](/docs/features/browser-requests/actions/wait) for slow forms, and close any overlay first.

#### How do I write an attribute selector inside JSON?

Use single quotes inside the [`selector`](/docs/features/browser-requests/selectors), like `"input[name='email']"`, or escape the double quotes with backslashes: `"input[name=\"email\"]"`. Unescaped double quotes will break the request.

#### Does Gaffa's typing avoid bot detection?

Gaffa types character by character with real keyboard events instead of dropping the whole string in at once, which handles sites that watch for that. It won't defeat every detection method.


# Wait

**Type**: `wait`

The wait action tells the browser to pause for a specified time or until a particular element appears on the page.

<details>

<summary><strong>Tips for using <code>wait</code> action</strong></summary>

* Wait for a [`selector`](/docs/features/browser-requests/selectors) rather than a fixed `time` wherever you can. It moves on as soon as the element appears.
* Never set both `time` and [`selector`](/docs/features/browser-requests/selectors). If you do, time wins and the timeout is ignored completely.
* Set `continue_on_fail: true` when the element you're waiting for might not appear.
* You don't need a `wait` before [`click`](/docs/features/browser-requests/actions/click), [`capture_element`](/docs/features/browser-requests/actions/capture-element) or [`parse_table`](/docs/features/browser-requests/actions/parse-table), since they wait for their own selectors.
* A fixed `time` is still the right choice for animations settling or for slowing a sequence down on purpose.

</details>

### Parameters

{% hint style="info" %}
**Parameter Requirements:** You must provide either `time` or `selector`. When using `selector`, `timeout` is required.
{% endhint %}

<table data-full-width="false"><thead><tr><th width="214">Name</th><th width="130">Type</th><th>Description</th></tr></thead><tbody><tr><td><code>time</code></td><td><code>integer</code></td><td>The time in milliseconds the browser should wait.</td></tr><tr><td><code>selector</code></td><td><code>string</code></td><td>The <a href="/docs/features/browser-requests/selectors">selector</a> for the element to wait for. You must provide a <code>timeout</code> when using <code>selector</code>.</td></tr><tr><td><code>timeout</code></td><td><code>integer</code></td><td>The maximum time in milliseconds to wait for <code>selector</code> to appear. Default: 5,000 (5s). Required when using <code>selector</code>.</td></tr></tbody></table>

See [universal parameters](/docs/features/browser-requests/actions#universal-parameters).

### Usage

#### Wait for a particular amount of time

The following code will wait 1 second, then continue to the next action.

**Wait 1 second:**

```json
"actions": [
      {
        "type": "wait",
        "time": 1000
      }
]
```

#### Wait for a particular element to appear

The following code will wait for a table to appear on the page for up to 5 seconds. If the table has not appeared after 5 seconds, the next action will be executed.

**Wait for table to appear:**

```json
"actions": [
      {
        "type": "wait",
        "selector": "table",
        "timeout": 5000,
        "continue_on_fail": true
      }
]
```

{% hint style="warning" %}
Using both `time` and `selector` + `timeout` together does not result in an error. But note that when both are provided, `time` takes priority and the browser will proceed after the `time` duration expires, regardless of whether `selector` has appeared, and the `timeout` is ignored.
{% endhint %}

### FAQs

#### When do I use the wait action?

Use it when you need to pause between actions — waiting for content to load after a click, for an animation to finish, or for a slow page to finish rendering before you capture it.

#### How do I wait for an element to appear?

Add a wait action with a [`selector`](/docs/features/browser-requests/selectors) for the element and a `timeout` in milliseconds. Gaffa continues as soon as the element appears or when the timeout expires.

#### How do I pause for a fixed amount of time?

Add a `wait` action with `time` set in milliseconds. Use this for animations or deliberate pacing, rather than for content that might take a variable amount of time to load.

#### What happens if I set both time and selector?

`time` takes over and `timeout` is ignored. The browser moves on once the time is up, whether or not the element appeared. Use one or the other, never both.

#### Should I wait for a selector or a fixed time?

Wait for a [`selector`](/docs/features/browser-requests/selectors) where you can. It moves on as soon as the element exists, instead of always using the full time, which makes sequences quicker and more reliable.

#### What if the element never appears?

The action fails once the timeout is up, and by default the request stops. Set `continue_on_fail: true` if the element is optional and the rest should still run.

#### Do I need a wait before every capture?

No. Add one only when the content loads after the previous action finishes. [`click`](/docs/features/browser-requests/actions/click), [`capture_element`](/docs/features/browser-requests/actions/capture-element) and [`parse_table`](/docs/features/browser-requests/actions/parse-table) already wait for their own [`selectors`](/docs/features/browser-requests/selectors).


# Tips & Best Practices

### Tips

#### Cost and speed

* Set `max_media_bandwidth: 0` when you only want text. It blocks images and videos, keeps the page working, and reduces token usage by up to 43% on media-heavy sites.
* Don't block media when you're screenshotting, printing or snapshotting. Those need the images.
* Turn on `block_ads` on ad-heavy sites, and add a [`selector`](/docs/features/browser-requests/selectors) to [`generate_markdown`](/docs/features/browser-requests/actions/generate-markdown) and [`parse_json`](/docs/features/browser-requests/actions/parse-json) so you only pay for the part you want.
* Use `max_cache_age` while you're developing, so re-running the same request doesn't cost you each time.

#### Proxies

* Leave `proxy_location` unset, and the request goes out from a data centre IP. Set it when a site blocks those, or serves different content by country.
* Match the location to the content you want: us for `US` pricing, `fr` for French listings.
* Assume a different IP on every request. They rotate, and proxies need a paid account.

#### Reliability and debugging

* Add up your action timeouts before setting `time_limit`. A single [`scroll`](/docs/features/browser-requests/actions/scroll) can run for 20 seconds.
* Set `continue_on_fail: true` on optional actions, so one missing element doesn't cost you the whole request.
* Add a [`capture_screenshot`](/docs/features/browser-requests/actions/capture-screenshot) between actions to find the step that broke.
* Turn on `record_request` when something fails, and you can't tell why, then turn it back off. It costs extra.

### FAQs

#### How do I reduce my Gaffa credit usage?

Set `max_media_bandwidth: 0` to skip images and video, which saves up to 43% of tokens on media-heavy sites. Add a [`selector`](/docs/features/browser-requests/selectors) to narrow captures, and use [`parse_table`](/docs/features/browser-requests/actions/parse-table) instead of [`parse_json`](/docs/features/browser-requests/actions/parse-json) for tables.

#### When should I not block media?

Don't block it when you're capturing screenshots, printing to PDF, taking a snapshot, or checking anything visual. All of those need the images to actually load.

#### Why does a site behave differently when I block media?

Setting `max_media_bandwidth: 0` stops images from loading, and some sites read that as an ad blocker. Raise the limit to a small figure like 5 rather than blocking outright.

#### How does caching work in Gaffa?

Set `max_cache_age` in milliseconds to say how old a result you'll accept. If anyone has made the same request within that window, you get their response immediately, at no additional bandwidth cost.

#### Why didn't my request come back from the cache?

A cached result is returned only when the URL, parameters, and actions all match a previous request exactly. Any difference, however small, means the request runs fresh.

#### When do I need to use a proxy?

Use one when a site blocks datacenter traffic, or when content changes by country — regional pricing, local search results, geo-restricted pages. Set `proxy_location` to the country you want to appear from.

#### Which proxy locations can I use?

United States (us), Ireland (ie), Singapore (sg) and France (fr). All proxy IPs are residential addresses. Proxies require a paid account.

#### **Can I keep the same IP across several requests?**

No. Each request is treated separately, and IPs rotate, so assume a different address every time. If a task requires continuity, do it all in a single browser request.

#### What does the time\_limit setting do?

It caps the maximum duration the whole request can run, in milliseconds. When the time runs out, unfinished actions are cancelled and the request errors. It can't exceed your plan's maximum runtime.

#### Why does my request keep timing out?

Usually one slow action is using the whole budget, often a [`scroll`](/docs/features/browser-requests/actions/scroll) with a long `max_scroll_time` or a [`wait`](/docs/features/browser-requests/actions/wait) for an element that never appears. Check those before raising the limit.

#### How do I debug a browser request that isn't working?

Turn on `record_request` to watch it back as video, add [`screenshots`](/docs/features/browser-requests/actions/capture-screenshot) between actions to find the failing step, and run a [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) to confirm your selectors exist on the page.

#### Why did my request return an error partway through?

By default, one failed action stops everything after it. Set `continue_on_fail: true` on actions that are allowed to fail, so the rest still run and you keep the results.


# API Playground Examples

On the following pages, you can view all prebuilt requests we've created to show what is possible with the Gaffa web automation API.

**You can start using these in the** [**API Playground**](https://gaffa.dev/dashboard/playground) **once you've created an account.**


# Export Web Page to PDF

An example request that uses Gaffa to convert an HTML page to a PDF. There are lots of HMTL to PDF API's but Gaffa handles it easily, as well as doing much more.

*The following example is a request we've prebuilt to show you Gaffa's capabilities on our* [*demo site.*](https://demo.gaffa.dev) ***You can run this request right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=html_to_pdf)***.***

Gaffa's print-to-PDF feature allows you to easily export web pages as PDF files. Unlike the standard "Print to PDF" in your local browser, Gaffa's feature waits for specific items to load, uses proxies, and scales with your product's growth. Enhance your customer experience and streamline your PDF export process

## API Request

The request below uses the [POST endpoint](/docs/api-reference/post-v1-browser-requests) to open the demo site on the table page, wait for the table to load, and then print the webpage to a PDF in A4 size with a 20-point margin and in portrait orientation.

```json
{
  "url": "https://demo.gaffa.dev/simulate/table?loadTime=3&rowCount=20",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "actions": [
      {
        "type": "wait",
        "selector": "table"
      },
      {
        "type": "print",
        "size": "A4",
        "margin": 20,
        "orientation": "portrait"
      }
    ]
  }
}
```

## Actions

Read the full documentation for these actions here.

{% content-ref url="/pages/Py3syTPEzIuvQYXyaDso" %}
[Wait](/docs/features/browser-requests/actions/wait)
{% endcontent-ref %}

{% content-ref url="/pages/SdEl6iIwtsv5C7XRPjvX" %}
[Print](/docs/features/browser-requests/actions/print)
{% endcontent-ref %}

## Response

Here's an example of the PDF returned by the request after the table has loaded.

{% file src="/files/977bvo93zl5BF0hIBxk9" %}


# Convert Web Page to Markdown

An example request that uses Gaffa to convert a web page page to markdown. This could be used to export web page reports or to print the content of a page in a readable format.

*The following example is a request we've prebuilt to demonstrate Gaffa's capabilities on our* [*demo site.*](https://demo.gaffa.dev) ***You can run this request right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=article_to_markdown)***.***

Gaffa converts web pages to clean markdown, stripping away styling, scripts, and images. This optimises content for LLM applications by reducing credit usage while preserving essential information.

## API Request

The request below uses the POST endpoint to open the demo site on the article simulator, wait for the article to load, and then generate a markdown from the page's content, which you can download for use in your program.

```json
{
  "url": "https://demo.gaffa.dev/simulate/article?loadTime=3&paragraphs=10&images=3",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "actions": [
      {
        "type": "wait",
        "selector": "article"
      },
      {
        "type": "generate_markdown"
      }
    ]
  }
}
```

## Actions

{% content-ref url="/pages/Py3syTPEzIuvQYXyaDso" %}
[Wait](/docs/features/browser-requests/actions/wait)
{% endcontent-ref %}

{% content-ref url="/pages/QtDLsZyUE94zYAaCimWo" %}
[Generate Markdown](/docs/features/browser-requests/actions/generate-markdown)
{% endcontent-ref %}

## Response

Here's an example of the PDF returned by the request after the article has loaded.

{% file src="/files/mTvssb40IQyvq8lyNhhG" %}


# Infinitely Scroll an E-commerce Site

An example request that uses Gaffa to infinitely scroll down a simulated ecommerce site whilst recording the interaction.

*The following example is a request we've prebuilt to show you Gaffa's capabilities on our* [*demo site.*](https://demo.gaffa.dev) ***You can run this request right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=infinite_scroll)***.***

Gaffa automates infinite scrolling on dynamic pages, such as e-commerce storefronts. Set a duration, and Gaffa will capture all content as it scrolls. Each session can be recorded as a video for playback, letting you debug or review the interaction.

## API Request

The request below uses the [POST endpoint](/docs/api-reference/post-v1-browser-requests) to open the demo site in the e-commerce site simulator, featuring an infinitely scrolling storefront. It will wait for and dismiss a dialog box, wait for a product to load, and then scroll down the page for a maximum of 20 seconds - if new items load, it will keep scrolling.

```json
{
  "url": "https://demo.gaffa.dev/simulate/ecommerce?loadTime=3&showModal=true&modalDelay=0&itemCount=infinite",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": true,
    "actions": [
      {
        "type": "wait",
        "selector": "div[role=\"dialog\"]",
        "timeout": 10000
      },
      {
        "type": "click",
        "selector": "[data-testid=\"accept-all-button\"]"
      },
      {
        "type": "wait",
        "selector": "[data-testid^=\"product-1\"]",
        "timeout": 5000
      },
      {
        "type": "scroll",
        "percentage": 100,
        "max_scroll_time": 20000
      }
    ]
  }
}
```

## Actions

{% content-ref url="/pages/Py3syTPEzIuvQYXyaDso" %}
[Wait](/docs/features/browser-requests/actions/wait)
{% endcontent-ref %}

{% content-ref url="/pages/1Cx0fCd84ZhpvRD9FVxt" %}
[Click](/docs/features/browser-requests/actions/click)
{% endcontent-ref %}

{% content-ref url="/pages/6wXXyX2KmvSFDvKqwGOQ" %}
[Scroll](/docs/features/browser-requests/actions/scroll)
{% endcontent-ref %}

## Response

Here's a video showing Gaffa scrolling the page for 20 seconds as more items load.

{% embed url="<https://youtu.be/s4WsBYxGWOo>" %}
Gaffa scrolling to the bottom of a simulated ecommerce page!
{% endembed %}

## Read More

Read more about screen recording here. (TODO)

{% content-ref url="/pages/kzTlst3tKo255yz4YpDi" %}
[Get Started](/docs/get-started)
{% endcontent-ref %}


# Capture a Full-Height Screenshot

An example request that uses Gaffa to dismiss a modal, scroll to the bottom of a page and then capture a full height screenshot.

*The following example is a request we've prebuilt to show you Gaffa's capabilities on our* [*demo site.*](https://demo.gaffa.dev) ***You can run this request right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=screenshot_ecommerce)***.***

Gaffa can also capture screenshots at any point during your interaction for use in your app or to work out exactly what was shown at a given time. You can capture just what is shown, as if you were looking at the screen or the full height of the page.

## API Request

The request below uses the [POST endpoint](/docs/api-reference/post-v1-browser-requests) to open the demo site on the ecommerce page with 20 items, wait for and dismiss the dialog, scroll to the bottom of the page, and capture a full height screenshot.

```json
{
  "url": "https://demo.gaffa.dev/simulate/ecommerce?loadTime=3&showModal=true&modalDelay=0&itemCount=20",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "actions": [
      {
        "type": "wait",
        "selector": "div[role=\"dialog\"]",
        "timeout": 10000
      },
      {
        "type": "click",
        "selector": "[data-testid=\"accept-all-button\"]"
      },
      {
        "type": "wait",
        "selector": "[data-testid^=\"product-1\"]",
        "timeout": 5000
      },
      {
        "type": "scroll",
        "percentage": 100
      },
      {
        "type": "capture_screenshot",
        "size": "fullscreen"
      }
    ]
  }
}
```

## Actions

{% content-ref url="/pages/Py3syTPEzIuvQYXyaDso" %}
[Wait](/docs/features/browser-requests/actions/wait)
{% endcontent-ref %}

{% content-ref url="/pages/1Cx0fCd84ZhpvRD9FVxt" %}
[Click](/docs/features/browser-requests/actions/click)
{% endcontent-ref %}

{% content-ref url="/pages/6wXXyX2KmvSFDvKqwGOQ" %}
[Scroll](/docs/features/browser-requests/actions/scroll)
{% endcontent-ref %}

{% content-ref url="/pages/vuNr1wFsHSlW2rBFRoTL" %}
[Capture Screenshot](/docs/features/browser-requests/actions/capture-screenshot)
{% endcontent-ref %}

## Response

The full-height export screenshot of the page showing all items.

<figure><img src="https://2341007864-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyUba6osOT5MkKiV0wmgr%2Fuploads%2F1jESouuLiHMOwvVpbIha%2FGaffaFullHeightScreenshotExample.png?alt=media&amp;token=d9540785-4cac-42c8-90a3-8d757c918ec0" alt=""><figcaption><p>Gaffa's full height screenshot</p></figcaption></figure>


# Automated Form Filling

An example request that uses Gaffa to automate the completion of a form and waits for a success modal to appear.

*The following example is a request we've prebuilt to show you Gaffa's capabilities on our* [*demo site.*](https://demo.gaffa.dev) ***You can run this request right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=form_fill)***.***

## API Request

```json
{
  "url": "https://demo.gaffa.dev/simulate/form?loadTime=3&showModal=false&modalDelay=0&formType=address&firstName=John&lastName=Doe&address1=123%20Main%20Street&city=London&country=UK",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": true,
    "actions": [
      {
        "type": "type",
        "selector": "#email",
        "text": "johndoe@example.com"
      },
      {
        "type": "type",
        "selector": "#state",
        "text": "CA"
      },
      {
        "type": "type",
        "selector": "#zipCode",
        "text": "12345"
      },
      {
        "type": "click",
        "selector": "button[type='submit']"
      },
      {
        "type": "wait",
        "selector": "[role=\"dialog\"] h2:has-text(\"Success!\")",
        "timeout": 10000
      }
    ]
  }
}
```

## Actions

{% content-ref url="/pages/TjjKKIilt0eFTzDDyZdD" %}
[Type](/docs/features/browser-requests/actions/type)
{% endcontent-ref %}

{% content-ref url="/pages/1Cx0fCd84ZhpvRD9FVxt" %}
[Click](/docs/features/browser-requests/actions/click)
{% endcontent-ref %}

{% content-ref url="/pages/Py3syTPEzIuvQYXyaDso" %}
[Wait](/docs/features/browser-requests/actions/wait)
{% endcontent-ref %}

## Response

Here's a video showing Gaffa filling out the page and waiting for the success modal.

{% embed url="<https://youtu.be/TGPnuc-71Bs>" %}
Gaffa can help automatically fill out your forms!
{% endembed %}

## Read More

Read more about screen recording here (TODO).


# Parse PDF to Structured JSON

An example request that uses Gaffa to extract structured data from an online PDF.

*The following example is a request we've pre-built to show you Gaffa's capabilities against our* [*demo site*](https://demo.gaffa.dev)*. **You can run this request right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=parse_academic_paper_pdf)***.***

This example demonstrates how to extract data from PDF documents. Gaffa downloads the PDF and uses AI to intelligently parse the content according to your schema, making it perfect for building research databases, citation managers, or literature review tools.

**This feature currently works for online PDFs.**

## API Request

The request below uses the [POST endpoint](https://gaffa.dev/docs/api-reference/post-v1-browser-requests) to download a demo research paper from the hosted PDFs, wait for it to load, and then parse the first page to extract author information and paper metadata.

```json
{
  "url": "https://demo.gaffa.dev/simulate/pdf/ReasoningAboutActionAndChange.pdf",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "max_media_bandwidth": null,
  "time_limit": null,
  "settings": {
    "record_request": false,
    "actions": [
      {
        "type": "download_file"
      },
      {
        "type": "parse_json",
        "output_type": "inline",
        "model": "gpt-4o-mini",
        "instruction": "Parse this academic paper focusing on the title, abstract, and author information typically found on the first page. Extract all author names, their institutional affiliations with department and location details, and their contact information.",
        "data_schema": {
          "name": "AcademicPaper",
          "description": "Schema for parsing academic paper summary and author information",
          "fields": [
            {
              "type": "string",
              "name": "title",
              "description": "The full title of the academic paper"
            },
            {
              "type": "string",
              "name": "abstract",
              "description": "The paper's abstract or summary"
            },
            {
              "type": "array",
              "name": "authors",
              "description": "List of authors who contributed to the paper",
              "fields": [
                {
                  "type": "string",
                  "name": "name",
                  "description": "Author's full name as it appears in the paper"
                },
                {
                  "type": "array",
                  "name": "affiliations",
                  "description": "Institutional affiliations for this author",
                  "fields": [
                    {
                      "type": "string",
                      "name": "institution",
                      "description": "Name of the university or research institution"
                    },
                    {
                      "type": "string",
                      "name": "department",
                      "description": "Department or division name"
                    },
                    {
                      "type": "string",
                      "name": "city",
                      "description": "City where the institution is located"
                    },
                    {
                      "type": "string",
                      "name": "country",
                      "description": "Country of the institution"
                    }
                  ]
                },
                {
                  "type": "string",
                  "name": "email",
                  "description": "Author's contact email address if provided"
                }
              ]
            }
          ]
        },
        "max_pages": 1
      }
    ]
  }
}
```

## Actions

{% content-ref url="/pages/FvBSaG7VbCnEutHxcCj2" %}
[Download File](/docs/features/browser-requests/actions/download-file)
{% endcontent-ref %}

{% content-ref url="/pages/7bb96jtp13gAqQoJ3aqV" %}
[Parse JSON](/docs/features/browser-requests/actions/parse-json)
{% endcontent-ref %}

## Response

The parsed data is returned as a structured JSON object matching your schema:

```json
{
  "data": {
    "id": "brq_VugUcjuFPM7ZyrMiRK9L8PzYMuPwhX",
    "url": "https://demo.gaffa.dev/simulate/pdf/ReasoningAboutActionAndChange.pdf",
    "state": "completed",
    "credit_usage": 0,
    "http_status_code": 200,
    "from_cache": false,
    "started_at": "2026-08-17T09:43:48.9008466Z",
    "completed_at": "2026-08-17T09:44:06.9238666Z",
    "running_time": "00:00:18.0230200",
    "page_load_time": "00:00:00.4931304",
    "actions": [
      {
        "id": "act_VugUckntLDEu8uJGcEg2nNPt8W3UM3",
        "type": "download_file",
        "timestamp": "2026-08-17T09:43:51.1202524Z",
        "output": "https://storage.gaffa.dev/brq/downloads/brq_VugUcjuFPM7ZyrMiRK9L8PzYMuPwhX/ReasoningAboutActionAndChange.pdf"
      },
      {
        "id": "act_VugUco5GmpoqLF1Khu7J9tUjiy3gkW",
        "type": "parse_json",
        "timestamp": "2026-08-17T09:44:06.9237903Z",
        "output": {
          "title": "Reasoning about Action and Change",
          "abstract": "This chapter presents the state of research concerning the formalisation of an agent reasoning about a dynamic system which can be partially observed and acted upon. We first define the basic concepts of the area: system states, ontic and epistemic actions, observations; then the basic reasoning processes: prediction, progression, regression, postdiction, filtering, abduction, and extrapolation. We then recall the classical action representation problems and show how these problems are solved in some standard frameworks. For space reasons, we focus on these major settings: the situation calculus, STRIPS and some propositional action languages, dynamic logic, and dynamic Bayesian networks. We finally address a special case of progression, namely belief update.",
          "authors": [
            {
              "name": "Florence Dupin de Saint-Cyr",
              "affiliations": [
                {
                  "institution": "Universit´ e Paul Sabatier",
                  "department": "IRIT-CNRS",
                  "city": "Toulouse",
                  "country": "France"
                }
              ],
              "email": ""
            },
            {
              "name": "Andreas Herzig",
              "affiliations": [
                {
                  "institution": "Universit´ e Paul Sabatier",
                  "department": "IRIT-CNRS",
                  "city": "Toulouse",
                  "country": "France"
                }
              ],
              "email": ""
            },
            {
              "name": "J´ erˆ ome Lang",
              "affiliations": [
                {
                  "institution": "CNRS, Universit´ e Paris-Dauphine, PSL Research University",
                  "department": "LAMSADE",
                  "city": "Paris",
                  "country": "France"
                }
              ],
              "email": ""
            },
            {
              "name": "Pierre Marquis",
              "affiliations": [
                {
                  "institution": "CRIL-CNRS, Universit´ e d’Artois & Institut Universitaire de France",
                  "department": "",
                  "city": "Lens",
                  "country": "France"
                }
              ],
              "email": ""
            }
          ]
        },
        "reference": "https://storage.gaffa.dev/brq/downloads/brq_VugUcjuFPM7ZyrMiRK9L8PzYMuPwhX/ReasoningAboutActionAndChange.pdf"
      }
    ]
  }
}
```


# Parse HTML Form to Structured JSON

An example request that uses Gaffa to analyze a web form and extract all input fields, their labels, types, and properties into structured JSON.

*The following example is a request we've pre-built to show you Gaffa's capabilities against our* [*demo site*](https://demo.gaffa.dev)*. **You can run this request right now in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=parse_html_form_to_structured_json)***.***

This example demonstrates how to extract structured information from HTML forms on web pages. Gaffa uses AI to identify form elements and their properties, making it perfect for form automation, testing, accessibility audits, or building form-filling assistants.

## API Request

The request below uses the [POST endpoint](https://gaffa.dev/docs/api-reference/post-v1-browser-requests) to open the demo form page, wait for the modal to appear, and then parse the visible form to extract all field information, including labels, input names, placeholders, and dropdown options.

```json
{
  "url": "https://demo.gaffa.dev/simulate/form?loadTime=3&showModal=true&modalDelay=5&formType=address",
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "actions": [
      {
        "type": "wait",
        "selector": "form",
        "timeout": 10000
      },
      {
        "type": "parse_json",
        "data_schema": {
          "name": "FormFields",
          "description": "Extract all form input fields",
          "fields": [
            {
              "type": "string",
              "name": "form_title",
              "description": "Form title"
            },
            {
              "type": "array",
              "name": "fields",
              "description": "List of all input fields",
              "fields": [
                {
                  "type": "string",
                  "name": "label",
                  "description": "Field label"
                },
                {
                  "type": "string",
                  "name": "field_name",
                  "description": "Field name attribute"
                },
                {
                  "type": "string",
                  "name": "field_type",
                  "description": "Input type"
                },
                {
                  "type": "boolean",
                  "name": "required",
                  "description": "Is required?"
                },
                {
                  "type": "string",
                  "name": "placeholder",
                  "description": "Placeholder text"
                }
              ]
            }
          ]
        },
        "instruction": "Extract all form fields with their properties",
        "model": "gpt-4o-mini",
        "output_type": "inline"
      }
    ]
  }
}
```

## Actions

{% content-ref url="/pages/Py3syTPEzIuvQYXyaDso" %}
[Wait](/docs/features/browser-requests/actions/wait)
{% endcontent-ref %}

{% content-ref url="/pages/7bb96jtp13gAqQoJ3aqV" %}
[Parse JSON](/docs/features/browser-requests/actions/parse-json)
{% endcontent-ref %}

## Response

The parsed form data is returned as a structured JSON object:

```json
{
  "data": {
    "id": "brq_VrwuWctcUAj75jbs5XUFkMbGGeBeLP",
    "url": "https://demo.gaffa.dev/simulate/form?loadTime=3&showModal=true&modalDelay=5&formType=address",
    "state": "completed",
    "credit_usage": 0,
    "http_status_code": 200,
    "from_cache": false,
    "started_at": "2026-07-14T16:20:29.8409557Z",
    "completed_at": "2026-07-14T16:20:43.1357433Z",
    "running_time": "00:00:13.2947876",
    "page_load_time": "00:00:00.5453910",
    "actions": [
      {
        "id": "act_VrwuWbYiipbS2e91UHf3vokLudWjez",
        "type": "wait",
        "timestamp": "2026-07-14T16:20:37.1333682Z"
      },
      {
        "id": "act_VrwuWkyuTBi8w3pBaQD98naDi9MvaK",
        "type": "parse_json",
        "timestamp": "2026-07-14T16:20:43.1357329Z",
        "output": {
          "form_title": "Form Submission Test",
          "fields": [
            {
              "label": "First Name *",
              "field_name": "first_name",
              "field_type": "text",
              "required": true,
              "placeholder": ""
            },
            {
              "label": "Last Name *",
              "field_name": "last_name",
              "field_type": "text",
              "required": true,
              "placeholder": ""
            },
            {
              "label": "Email *",
              "field_name": "email",
              "field_type": "email",
              "required": true,
              "placeholder": ""
            },
            {
              "label": "Address Line 1 *",
              "field_name": "address_line_1",
              "field_type": "text",
              "required": true,
              "placeholder": ""
            },
            {
              "label": "Address Line 2",
              "field_name": "address_line_2",
              "field_type": "text",
              "required": false,
              "placeholder": ""
            },
            {
              "label": "City *",
              "field_name": "city",
              "field_type": "text",
              "required": true,
              "placeholder": ""
            },
            {
              "label": "State/Province *",
              "field_name": "state_province",
              "field_type": "text",
              "required": true,
              "placeholder": ""
            },
            {
              "label": "ZIP/Postal Code *",
              "field_name": "zip_postal_code",
              "field_type": "text",
              "required": true,
              "placeholder": ""
            },
            {
              "label": "Country *",
              "field_name": "country",
              "field_type": "text",
              "required": true,
              "placeholder": ""
            }
          ]
        },
        "reference": "https://storage.gaffa.dev/brq/dom/brq_VrwuWctcUAj75jbs5XUFkMbGGeBeLP/act_VrwuWkyuTBi8w3pBaQD98naDi9MvaK_raw.txt"
      }
    ]
  }
}
```


# Parse an HTML Table to JSON

An example request that uses Gaffa to extract structured data (JSON) from a table on a webpage

*The following example is a prebuilt request that demonstrates Gaffa's capabilities on our* [*demo site*](https://demo.gaffa.dev/)*. **You can run this request right here in the*** [***Gaffa API Playground***](https://gaffa.dev/dashboard/playground?templateId=parse_table)***.***

This example demonstrates how to extract tabular data from any webpage without writing a scraper. Gaffa renders the page using a real browser, waits for the table to load, and returns the rows as a clean JSON array, making it perfect for building data pipelines, monitoring dashboards, or feeding structured data into LLM workflows.

## API Request

The request below uses the [POST endpoint](https://gaffa.dev/docs/api-reference/post-v1-browser-requests) to load a demo table page, waits for the table element to appear, and parses each row into a structured JSON array, using the table's header row as property names.

```json
{
  "url": "https://demo.gaffa.dev/simulate/table?loadTime=1&rowCount=3",
  "proxy_location": null,
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "actions": [
      {
        "type": "wait",
        "selector": "table",
        "timeout": 5000
      },
      {
        "type": "parse_table",
        "selector": "table"
      }
    ]
  }
}
```

## Actions

{% content-ref url="/pages/Py3syTPEzIuvQYXyaDso" %}
[Wait](/docs/features/browser-requests/actions/wait)
{% endcontent-ref %}

{% content-ref url="/pages/k6OW4oynrx9l5KvPURSV" %}
[Parse Table](/docs/features/browser-requests/actions/parse-table)
{% endcontent-ref %}

## Response

The `parse_table` action returns an `output` URL pointing to the extracted JSON:

```json
{
  "data": {
    "id": "brq_abc123ExampleRequestId",
    "url": "https://demo.gaffa.dev/simulate/table?loadTime=1&rowCount=10",
    "state": "completed",
    "credit_usage": 1,
    "http_status_code": 200,
    "from_cache": false,
    "started_at": "2025-06-09T12:00:00.000Z",
    "completed_at": "2025-06-09T12:00:04.321Z",
    "running_time": "00:00:04.3210000",
    "page_load_time": "00:00:01.1230000",
    "actions": [
      {
        "id": "act_wait001",
        "type": "wait",
        "query": "wait?selector=table&timeout=5000&continue_on_fail=false",
        "timestamp": "2025-06-09T12:00:01.500Z"
      },
      {
        "id": "act_parse001",
        "type": "parse_table",
        "query": "parse_table?selector=table",
        "timestamp": "2025-06-09T12:00:01.600Z",
        "output": "https://storage.gaffa.dev/brq/results/brq_abc123ExampleRequestId/act_parse001_table.json"
      }
    ]
  }
}
```

Fetching that URL gives you the table rows as a ready-to-use array:

```json
[
  {
    "id": "1",
    "name": "Item 1",
    "quantity": "30",
    "price": "$56.05"
  },
  {
    "id": "2",
    "name": "Item 2",
    "quantity": "68",
    "price": "$76.89"
  },
  {
    "id": "3",
    "name": "Item 3",
    "quantity": "67",
    "price": "$20.44"
  }
]
```


# Mapping Requests

Mapping requests allow you to extract all URLs from a website's sitemap. Gaffa mapping requests have the following useful features:

* **Sitemap Discovery:** No need to manually find a site's sitemap URL; we'll find it automatically.
* **Caching:** If you or another Gaffa user has retrieved a sitemap within a defined timeframe, we'll quickly return the cached data instead of fetching it again.
* **Index Traversal:** If the sitemap references other sitemap files, we'll automatically process each one and add its URLs to the list, ensuring the entire hierarchy is captured.
* **Aggregation and Duplicate Prevention:** In rare cases where the sitemap contains duplicate entries, we'll automatically remove them for you and return all URLs sorted alphabetically.
* **Proxies:** Gaffa uses its residential proxies behind the scenes to ensure your sitemap retrieval requests aren't blocked.

## Example Request

The [POST v1/site/map](/docs/api-reference/post-v1-site-map) endpoint allows you to create a new request and await the result. It's a request with a simple payload containing the URL of the site you want to extract the sitemap of, and a `max_cache_age` in seconds, you would accept a response returned from the cache; the default is 0, and Gaffa will never return a cached response when used.

```json
{
  "url": "https://gaffa.dev",
  "max_cache_age": 100
}
```

{% hint style="info" %}
The request currently has a maximum running time of 60 seconds, after which an error will be returned.
{% endhint %}

For the Gaffa site, this will return the following response:

<pre class="language-json"><code class="lang-json">{
  "data": {
    "id": "smr_VQW4E66TdcQFZfCs6qavgdowPj3Bzk",
    "url": "https://gaffa.dev",
    "state": "completed",
    "credit_usage": 1,
    "from_cache": true,
    "started_at": "2025-08-22T11:05:43.328175Z",
    "completed_at": "2025-08-22T11:05:47.857941Z",
    "running_time": "00:00:04.5297660",
    "links": [
      "https://gaffa.dev",
      "https://gaffa.dev/about",
      "https://gaffa.dev/blog",
      "https://gaffa.dev/blog/convert-any-web-page-to-llm-ready-markdown-using-gaffa",
      "https://gaffa.dev/blog/how-to-extract-and-simplify-a-webpage-dom-with-gaffa",
      "https://gaffa.dev/blog/printing-webpages-to-pdf-html-to-pdf-using-gaffa",
      "https://gaffa.dev/docs",
      "https://gaffa.dev/docs/api-reference/api-authentication",
      ....and so on
    ],
    "link_count": 52
  }
<strong>}
</strong></code></pre>

As you'll see from the [API Reference section](/docs/api-reference/api-reference) of the site, there are also requests to retrieve site mapping requests for your account.

## Pricing

See the [Credits and Pricing page](/docs/credits-and-pricing) for the current cost of mapping requests.


# API Reference

Complete HTTP API documentation for Gaffa. Each endpoint page includes interactive OpenAPI definitions you can try from the docs.

Start with [API Authentication](/docs/api-reference/api-authentication) to create and use API keys, then explore the endpoints below.

## Browser requests

* [POST v1/browser/requests](/docs/api-reference/post-v1-browser-requests) — Create and run a browser request
* [GET v1/browser/requests/{id}](/docs/api-reference/get-v1-browser-requests-id) — Get a browser request by ID
* [GET v1/browser/requests](/docs/api-reference/get-v1-browser-requests) — List browser requests

## Schemas

* [POST v1/schemas](/docs/api-reference/post-v1-schemas) — Create a schema
* [PUT v1/schemas](/docs/api-reference/put-v1-schemas) — Update a schema
* [GET v1/schemas](/docs/api-reference/get-v1-schemas) — List schemas
* [DELETE v1/schemas/{id}](/docs/api-reference/delete-v1-schemas-id) — Delete a schema

## Site mapping

* [POST v1/site/map](/docs/api-reference/post-v1-site-map) — Create a site mapping request
* [GET v1/site/map](/docs/api-reference/get-v1-site-map) — List site mapping requests
* [GET v1/site/map/{id}](/docs/api-reference/get-v1-site-map-id) — Get a site mapping request by ID


# API Authentication

We use API Keys for authenticating requests to our API. In this document we'll explain how you can manage and use the keys for your account.

## Creating Keys

Once your account is approved, you will need to create an API key to send your requests to our API.\
\
Go to your account [**Dashboard > API Keys**](https://gaffa.dev/dashboard/api-tokens) and create a new key with a name. Once the key is created, copy the value, and you can immediately start using it to make requests.

{% hint style="info" %}
You can create as many keys as you wish, but always remember to treat the key as a secret and do not reveal it in public blog posts or GitHub repositories. If someone uses your leaked key to make requests, we won't be responsible!
{% endhint %}

## Deleting Keys

If you are worried you have exposed your Gaffa API key, or just want to periodically rotate your keys, you can create a new key and then delete your old keys. Deleted keys will immediately stop working for new API requests, but past browser requests made using old keys will still be available.

## Authenticating Requests

Our API is secured with a customer header `X-API-Key` whose value should be any current API key in your account. That's all you need to add to your request!


# POST v1/browser/requests

{% hint style="info" %}
For more information on browser requests, [see here](/docs/features/browser-requests).
{% endhint %}

The following endpoint creates a browser request and either runs it synchronously or returns immediately with an ID so you can check its status later.

## Create a new browser request

> This endpoint loads the required URL in our browser and then performs the selected actions.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"browserRequestInput_object":{"type":"object","properties":{"proxy_location":{"type":"string","description":"The location of the proxy server that your request will be routed through, null means no proxy is used","default":"null","nullable":true},"url":{"type":"string","description":"The url you want our browsers to visit on your behalf"},"async":{"type":"boolean","description":"Whether the request should be processed asynchronously, synchronous requests can be maximum 60 seconds long.","default":true},"max_cache_age":{"type":"integer","description":"The maximum age of a cached result in seconds. 0 means the cache will never be used","format":"int32","default":0,"nullable":true},"settings":{"$ref":"#/components/schemas/browserRequestSettings_object"}}},"browserRequestSettings_object":{"type":"object","properties":{"record_request":{"type":"boolean","description":"Record a video of this request","default":false,"nullable":true},"actions":{"type":"array","items":{"$ref":"#/components/schemas/dictionary_object"},"description":"A list of the functions you want to perform on the web page"},"time_limit":{"type":"integer","description":"Cap the maximum time the request should take to complete, in milliseconds (default: 60000)","format":"int32","default":60000,"nullable":true},"max_media_bandwidth":{"type":"integer","description":"Cap the maximum bandwidth to use for media downloads, in MB","format":"int32","nullable":true},"block_ads":{"type":"boolean","description":"Enable ad blocking for this request","default":false,"nullable":true}},"description":"The actions and outputs you want to be executed"},"dictionary_object":{"type":"object","additionalProperties":{"type":"object"}},"browserRequestResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the browser request","nullable":true},"url":{"type":"string","description":"URL of the request","nullable":true},"proxy_location":{"type":"string","description":"The proxy location of the request.","nullable":true},"state":{"type":"string","description":"The status of the request","nullable":true},"credit_usage":{"type":"integer","description":"The number of credits used by the request","format":"int32","nullable":true},"error":{"type":"string","description":"The name of the error type","nullable":true},"error_reason":{"type":"string","description":"More detail about the error","nullable":true},"actual_url":{"type":"string","description":"The actual URL captured, after any redirects.","nullable":true},"http_status_code":{"type":"integer","description":"The http status code for the request.","format":"int32"},"from_cache":{"type":"boolean","description":"If this request was served from the cached","nullable":true},"started_at":{"type":"string","description":"The time in UTC when the request started.","format":"date-time"},"completed_at":{"type":"string","description":"The time in UTC when the request finished.","format":"date-time"},"running_time":{"type":"string","description":"The running time of the request","format":"timespan"},"page_load_time":{"type":"string","description":"How long did the page take to fully render.","format":"timespan"},"actions":{"type":"array","items":{"$ref":"#/components/schemas/browerRequestActionResponse"},"description":"Actions carried out and their results","nullable":true},"video":{"type":"string","description":"Video url","nullable":true}}},"browerRequestActionResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the action","nullable":true},"type":{"type":"string","description":"Name of the action","nullable":true},"custom_id":{"type":"string","description":"Custom ID of the action","nullable":true},"timestamp":{"type":"string","description":"Time the action was initiated","format":"date-time"},"output":{"type":"object","description":"Ouput of the action, if any","nullable":true},"reference":{"type":"string","description":"Reference file for the action, if any","nullable":true},"iterations":{"type":"integer","description":"Number of iterations completed for loop actions","format":"int32"},"actions":{"type":"array","items":{"$ref":"#/components/schemas/browerRequestActionResponse"},"description":"Nested actions executed within loop actions","nullable":true},"error":{"type":"string","description":"Error message, if any","nullable":true}}},"apiErrorResponse":{"type":"object","properties":{"type":{"type":"string","description":"The type of object this is concerning","nullable":true},"id":{"type":"string","description":"The id of the item concerned.","nullable":true},"code":{"type":"string","description":"Error code.","nullable":true},"message":{"type":"string","description":"Error description.","nullable":true}}}}},"paths":{"/v1/browser/requests":{"post":{"tags":["Browser Requests"],"summary":"Create a new browser request","description":"This endpoint loads the required URL in our browser and then performs the selected actions.","operationId":"createBrowserRequest","requestBody":{"description":"Browser request input data","content":{"application/json":{"schema":{"$ref":"#/components/schemas/browserRequestInput_object"}}},"required":true},"responses":{"200":{"description":"The browser request response detailing the state and output of the request","content":{"application/json":{"schema":{"$ref":"#/components/schemas/browserRequestResponse"}}}},"408":{"description":"The browser request timed out - an example error","content":{"application/json":{"schema":{"$ref":"#/components/schemas/apiErrorResponse"}}}}}}}}}
```


# GET v1/browser/requests/{id}

{% hint style="info" %}
For more information on browser requests, [see here](/docs/features/browser-requests).
{% endhint %}

The following endpoint allows you to query the browser request for your account by ID.

## Get a browser request by ID

> This endpoint retrieves a browser request by its ID.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"browserRequestResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the browser request","nullable":true},"url":{"type":"string","description":"URL of the request","nullable":true},"proxy_location":{"type":"string","description":"The proxy location of the request.","nullable":true},"state":{"type":"string","description":"The status of the request","nullable":true},"credit_usage":{"type":"integer","description":"The number of credits used by the request","format":"int32","nullable":true},"error":{"type":"string","description":"The name of the error type","nullable":true},"error_reason":{"type":"string","description":"More detail about the error","nullable":true},"actual_url":{"type":"string","description":"The actual URL captured, after any redirects.","nullable":true},"http_status_code":{"type":"integer","description":"The http status code for the request.","format":"int32"},"from_cache":{"type":"boolean","description":"If this request was served from the cached","nullable":true},"started_at":{"type":"string","description":"The time in UTC when the request started.","format":"date-time"},"completed_at":{"type":"string","description":"The time in UTC when the request finished.","format":"date-time"},"running_time":{"type":"string","description":"The running time of the request","format":"timespan"},"page_load_time":{"type":"string","description":"How long did the page take to fully render.","format":"timespan"},"actions":{"type":"array","items":{"$ref":"#/components/schemas/browerRequestActionResponse"},"description":"Actions carried out and their results","nullable":true},"video":{"type":"string","description":"Video url","nullable":true}}},"browerRequestActionResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the action","nullable":true},"type":{"type":"string","description":"Name of the action","nullable":true},"custom_id":{"type":"string","description":"Custom ID of the action","nullable":true},"timestamp":{"type":"string","description":"Time the action was initiated","format":"date-time"},"output":{"type":"object","description":"Ouput of the action, if any","nullable":true},"reference":{"type":"string","description":"Reference file for the action, if any","nullable":true},"iterations":{"type":"integer","description":"Number of iterations completed for loop actions","format":"int32"},"actions":{"type":"array","items":{"$ref":"#/components/schemas/browerRequestActionResponse"},"description":"Nested actions executed within loop actions","nullable":true},"error":{"type":"string","description":"Error message, if any","nullable":true}}},"apiErrorResponse":{"type":"object","properties":{"type":{"type":"string","description":"The type of object this is concerning","nullable":true},"id":{"type":"string","description":"The id of the item concerned.","nullable":true},"code":{"type":"string","description":"Error code.","nullable":true},"message":{"type":"string","description":"Error description.","nullable":true}}}}},"paths":{"/v1/browser/requests/{id}":{"get":{"tags":["Browser Requests"],"summary":"Get a browser request by ID","description":"This endpoint retrieves a browser request by its ID.","operationId":"getBrowserRequestById","parameters":[{"name":"id","in":"path","description":"The unique identifier of the browser request to retrieve.","required":true,"schema":{"type":"string"}},{"name":"id","in":"query","description":"The unique identifiers of the browser request to retrieve.","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"The browser request","content":{"application/json":{"schema":{"$ref":"#/components/schemas/browserRequestResponse"}}}},"404":{"description":"Browser request not found","content":{"application/json":{"schema":{"$ref":"#/components/schemas/apiErrorResponse"}}}}}}}}}
```


# GET v1/browser/requests

{% hint style="info" %}
For more information on browser requests, [see here](/docs/features/browser-requests).
{% endhint %}

The following endpoint allows you to query for multiple browser requests, either by status or a list of particular ids, submitting a request with neither of these will return all requests for your account.

## Get multiple browser requests

> This endpoint retrieves browser requests in bulk by id or status.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"pagedResult_browserRequestResponse":{"type":"object","properties":{"total_pages":{"type":"integer","description":"The total number of pages available","format":"int32","nullable":true},"total_records":{"type":"integer","description":"The total number of records across all pages","format":"int32","nullable":true},"results":{"type":"array","items":{"$ref":"#/components/schemas/browserRequestResponse"},"description":"The records for the current page","nullable":true},"page":{"type":"integer","description":"The page number to return (1-based)","format":"int32","default":1,"nullable":true},"page_size":{"type":"integer","description":"The number of records to return per page","format":"int32","default":30,"nullable":true}}},"browserRequestResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the browser request","nullable":true},"url":{"type":"string","description":"URL of the request","nullable":true},"proxy_location":{"type":"string","description":"The proxy location of the request.","nullable":true},"state":{"type":"string","description":"The status of the request","nullable":true},"credit_usage":{"type":"integer","description":"The number of credits used by the request","format":"int32","nullable":true},"error":{"type":"string","description":"The name of the error type","nullable":true},"error_reason":{"type":"string","description":"More detail about the error","nullable":true},"actual_url":{"type":"string","description":"The actual URL captured, after any redirects.","nullable":true},"http_status_code":{"type":"integer","description":"The http status code for the request.","format":"int32"},"from_cache":{"type":"boolean","description":"If this request was served from the cached","nullable":true},"started_at":{"type":"string","description":"The time in UTC when the request started.","format":"date-time"},"completed_at":{"type":"string","description":"The time in UTC when the request finished.","format":"date-time"},"running_time":{"type":"string","description":"The running time of the request","format":"timespan"},"page_load_time":{"type":"string","description":"How long did the page take to fully render.","format":"timespan"},"actions":{"type":"array","items":{"$ref":"#/components/schemas/browerRequestActionResponse"},"description":"Actions carried out and their results","nullable":true},"video":{"type":"string","description":"Video url","nullable":true}}},"browerRequestActionResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the action","nullable":true},"type":{"type":"string","description":"Name of the action","nullable":true},"custom_id":{"type":"string","description":"Custom ID of the action","nullable":true},"timestamp":{"type":"string","description":"Time the action was initiated","format":"date-time"},"output":{"type":"object","description":"Ouput of the action, if any","nullable":true},"reference":{"type":"string","description":"Reference file for the action, if any","nullable":true},"iterations":{"type":"integer","description":"Number of iterations completed for loop actions","format":"int32"},"actions":{"type":"array","items":{"$ref":"#/components/schemas/browerRequestActionResponse"},"description":"Nested actions executed within loop actions","nullable":true},"error":{"type":"string","description":"Error message, if any","nullable":true}}},"apiErrorResponse":{"type":"object","properties":{"type":{"type":"string","description":"The type of object this is concerning","nullable":true},"id":{"type":"string","description":"The id of the item concerned.","nullable":true},"code":{"type":"string","description":"Error code.","nullable":true},"message":{"type":"string","description":"Error description.","nullable":true}}}}},"paths":{"/v1/browser/requests":{"get":{"tags":["Browser Requests"],"summary":"Get multiple browser requests","description":"This endpoint retrieves browser requests in bulk by id or status.","operationId":"getBrowserRequest","parameters":[{"name":"ids","in":"query","description":"The unique identifiers of the browser requests to retrieve.","schema":{"type":"string"}},{"name":"status","in":"query","description":"The statuses of the browser requests to filter by. Valid values: pending, running, completed, failed","schema":{"type":"string"}},{"name":"pageSize","in":"query","description":"Items to return per page (default: 30).","schema":{"type":"integer","format":"int32"}},{"name":"page","in":"query","description":"Page number of the pagination (default: 1).","schema":{"type":"integer","format":"int32"}},{"name":"ids","in":"query","description":"The unique identifiers of the browser requests to retrieve.","schema":{"type":"string"}},{"name":"status","in":"query","description":"The statuses of the browser requests to filter by.","schema":{"type":"string"}},{"name":"pageSize","in":"query","description":"Items to return per page (default: 30).","schema":{"type":"integer","format":"int32"}},{"name":"page","in":"query","description":"Page number of the pagination.","schema":{"type":"integer","format":"int32"}}],"responses":{"200":{"description":"A collection of browser requests that match the criteria","content":{"application/json":{"schema":{"$ref":"#/components/schemas/pagedResult_browserRequestResponse"}}}},"400":{"description":"Invalid query parameters","content":{"application/json":{"schema":{"$ref":"#/components/schemas/apiErrorResponse"}}}}}}}}}
```


# POST v1/schemas

jjjj

The following endpoint allows you to describe a data schema for parsing an online PDF to JSON.

## Create a new data schema

> Creates a new data schema definition and returns the created schema.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"dataSchema":{"type":"object","properties":{"id":{"type":"string","description":"The unique identifier for the data schema.","nullable":true},"name":{"type":"string","description":"The name of the schema or field.","nullable":true},"description":{"type":"string","description":"A description of the schema or field.","nullable":true},"fields":{"type":"array","items":{"$ref":"#/components/schemas/schemaField"},"description":"The list of fields that make up this object.","nullable":true}}},"schemaField":{"type":"object","properties":{"type":{"enum":[0,1,2,3,4,5,6,7],"type":"integer","description":"The type of the field.","format":"int32"},"name":{"type":"string","description":"The name of the schema or field.","nullable":true},"description":{"type":"string","description":"A description of the schema or field.","nullable":true},"fields":{"type":"array","items":{"$ref":"#/components/schemas/schemaField"},"description":"The list of fields that make up this object.","nullable":true}}}}},"paths":{"/v1/schemas":{"post":{"tags":["Data Schemas"],"summary":"Create a new data schema","description":"Creates a new data schema definition and returns the created schema.","operationId":"createDataSchema","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/dataSchema"}}},"required":true},"responses":{"200":{"description":"Payload of DataSchema","content":{"application/json":{"schema":{"$ref":"#/components/schemas/dataSchema"}}}}}}}}}
```


# PUT v1/schemas

jjjj

The following endpoint allows you to update a data schema by ID.

## Update an existing data schema

> Updates an existing data schema by its ID and returns the updated schema.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"dataSchema":{"type":"object","properties":{"id":{"type":"string","description":"The unique identifier for the data schema.","nullable":true},"name":{"type":"string","description":"The name of the schema or field.","nullable":true},"description":{"type":"string","description":"A description of the schema or field.","nullable":true},"fields":{"type":"array","items":{"$ref":"#/components/schemas/schemaField"},"description":"The list of fields that make up this object.","nullable":true}}},"schemaField":{"type":"object","properties":{"type":{"enum":[0,1,2,3,4,5,6,7],"type":"integer","description":"The type of the field.","format":"int32"},"name":{"type":"string","description":"The name of the schema or field.","nullable":true},"description":{"type":"string","description":"A description of the schema or field.","nullable":true},"fields":{"type":"array","items":{"$ref":"#/components/schemas/schemaField"},"description":"The list of fields that make up this object.","nullable":true}}}}},"paths":{"/v1/schemas/{id}":{"put":{"tags":["Data Schemas"],"summary":"Update an existing data schema","description":"Updates an existing data schema by its ID and returns the updated schema.","operationId":"updateDataSchema","parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"}}],"requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/dataSchema"}}},"required":true},"responses":{"200":{"description":"Payload of DataSchema","content":{"application/json":{"schema":{"$ref":"#/components/schemas/dataSchema"}}}}}}}}}
```


# GET v1/schemas

jjjj

The following endpoint allows you to list data schemas for your account in a paginated list.

## List data schemas

> Retrieves a paginated list of data schemas.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"pagedResult_dataSchema":{"type":"object","properties":{"total_pages":{"type":"integer","description":"The total number of pages available","format":"int32","nullable":true},"total_records":{"type":"integer","description":"The total number of records across all pages","format":"int32","nullable":true},"results":{"type":"array","items":{"$ref":"#/components/schemas/dataSchema"},"description":"The records for the current page","nullable":true},"page":{"type":"integer","description":"The page number to return (1-based)","format":"int32","default":1,"nullable":true},"page_size":{"type":"integer","description":"The number of records to return per page","format":"int32","default":30,"nullable":true}}},"dataSchema":{"type":"object","properties":{"id":{"type":"string","description":"The unique identifier for the data schema.","nullable":true},"name":{"type":"string","description":"The name of the schema or field.","nullable":true},"description":{"type":"string","description":"A description of the schema or field.","nullable":true},"fields":{"type":"array","items":{"$ref":"#/components/schemas/schemaField"},"description":"The list of fields that make up this object.","nullable":true}}},"schemaField":{"type":"object","properties":{"type":{"enum":[0,1,2,3,4,5,6,7],"type":"integer","description":"The type of the field.","format":"int32"},"name":{"type":"string","description":"The name of the schema or field.","nullable":true},"description":{"type":"string","description":"A description of the schema or field.","nullable":true},"fields":{"type":"array","items":{"$ref":"#/components/schemas/schemaField"},"description":"The list of fields that make up this object.","nullable":true}}}}},"paths":{"/v1/schemas":{"get":{"tags":["Data Schemas"],"summary":"List data schemas","description":"Retrieves a paginated list of data schemas.","operationId":"listDataSchemas","parameters":[{"name":"pageSize","in":"query","schema":{"type":"integer","format":"int32"}},{"name":"page","in":"query","schema":{"type":"integer","format":"int32"}}],"responses":{"200":{"description":"Payload of PagedResult containing DataSchema","content":{"application/json":{"schema":{"$ref":"#/components/schemas/pagedResult_dataSchema"}}}}}}}}}
```


# DELETE v1/schemas/{id}

jjjj

The following endpoint allows you to delete a schema from your account.

## Delete a data schema

> Deletes a data schema by its ID.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}}},"paths":{"/v1/schemas/{id}":{"delete":{"tags":["Data Schemas"],"summary":"Delete a data schema","description":"Deletes a data schema by its ID.","operationId":"deleteDataSchema","parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"}}],"responses":{"204":{"description":"No description"}}}}}}
```


# POST v1/site/map

This endpoint creates a new site mapping request and returns the result.

## Create a new sitemap request

> This endpoint processes a website's sitemap and returns all URLs found within it.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"sitemapRequestInput":{"type":"object","properties":{"url":{"type":"string","description":"The url you want our sitemap reader to process on your behalf"},"max_cache_age":{"type":"integer","description":"Maximum cache age in seconds for this request. If a cached result exists within this timeframe, it will be returned. Default is 0 (no cache).","format":"int32","nullable":true}}},"sitemapRequestResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the sitemap request","nullable":true},"url":{"type":"string","description":"URL of the request","nullable":true},"state":{"type":"string","description":"The status of the request","nullable":true},"credit_usage":{"type":"integer","description":"The number of credits used by the request","format":"int32","nullable":true},"error":{"type":"string","description":"The name of the error type","nullable":true},"error_reason":{"type":"string","description":"More detail about the error","nullable":true},"from_cache":{"type":"boolean","description":"If this request was served from the cache","nullable":true},"started_at":{"type":"string","description":"The time in UTC when the request started.","format":"date-time"},"completed_at":{"type":"string","description":"The time in UTC when the request finished.","format":"date-time"},"running_time":{"type":"string","description":"The running time of the request","format":"timespan"},"links":{"type":"array","items":{"type":"string"},"description":"List of URLs found in the sitemap","nullable":true},"link_count":{"type":"integer","description":"Number of links found","format":"int32","nullable":true}}},"apiErrorResponse":{"type":"object","properties":{"type":{"type":"string","description":"The type of object this is concerning","nullable":true},"id":{"type":"string","description":"The id of the item concerned.","nullable":true},"code":{"type":"string","description":"Error code.","nullable":true},"message":{"type":"string","description":"Error description.","nullable":true}}}}},"paths":{"/v1/site/map":{"post":{"tags":["Sitemap Requests"],"summary":"Create a new sitemap request","description":"This endpoint processes a website's sitemap and returns all URLs found within it.","operationId":"createSitemapRequest","requestBody":{"description":"Sitemap request input data","content":{"application/json":{"schema":{"$ref":"#/components/schemas/sitemapRequestInput"}}},"required":true},"responses":{"200":{"description":"The sitemap request response detailing the URLs found","content":{"application/json":{"schema":{"$ref":"#/components/schemas/sitemapRequestResponse"}}}},"408":{"description":"The sitemap request timed out after 60 seconds","content":{"application/json":{"schema":{"$ref":"#/components/schemas/apiErrorResponse"}}}},"503":{"description":"The requested site is unavailable","content":{"application/json":{"schema":{"$ref":"#/components/schemas/apiErrorResponse"}}}}}}}}}
```


# GET v1/site/map

This endpoint retrieves information about previous site mapping requests, filterable by id or status

## Get Sitemap

> This endpoint retrieves sitemap requests in bulk by id or status.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"pagedResult_sitemapRequestResponse":{"type":"object","properties":{"total_pages":{"type":"integer","description":"The total number of pages available","format":"int32","nullable":true},"total_records":{"type":"integer","description":"The total number of records across all pages","format":"int32","nullable":true},"results":{"type":"array","items":{"$ref":"#/components/schemas/sitemapRequestResponse"},"description":"The records for the current page","nullable":true},"page":{"type":"integer","description":"The page number to return (1-based)","format":"int32","default":1,"nullable":true},"page_size":{"type":"integer","description":"The number of records to return per page","format":"int32","default":30,"nullable":true}}},"sitemapRequestResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the sitemap request","nullable":true},"url":{"type":"string","description":"URL of the request","nullable":true},"state":{"type":"string","description":"The status of the request","nullable":true},"credit_usage":{"type":"integer","description":"The number of credits used by the request","format":"int32","nullable":true},"error":{"type":"string","description":"The name of the error type","nullable":true},"error_reason":{"type":"string","description":"More detail about the error","nullable":true},"from_cache":{"type":"boolean","description":"If this request was served from the cache","nullable":true},"started_at":{"type":"string","description":"The time in UTC when the request started.","format":"date-time"},"completed_at":{"type":"string","description":"The time in UTC when the request finished.","format":"date-time"},"running_time":{"type":"string","description":"The running time of the request","format":"timespan"},"links":{"type":"array","items":{"type":"string"},"description":"List of URLs found in the sitemap","nullable":true},"link_count":{"type":"integer","description":"Number of links found","format":"int32","nullable":true}}},"apiErrorResponse":{"type":"object","properties":{"type":{"type":"string","description":"The type of object this is concerning","nullable":true},"id":{"type":"string","description":"The id of the item concerned.","nullable":true},"code":{"type":"string","description":"Error code.","nullable":true},"message":{"type":"string","description":"Error description.","nullable":true}}}}},"paths":{"/v1/site/map":{"get":{"tags":["Sitemap Requests"],"summary":"Get Sitemap","description":"This endpoint retrieves sitemap requests in bulk by id or status.","operationId":"getSitemapRequests","parameters":[{"name":"ids","in":"query","description":"The unique identifiers of the sitemap requests to retrieve.","schema":{"type":"string"}},{"name":"status","in":"query","description":"The statuses of the sitemap requests to filter by. Valid values: pending, completed, failed","schema":{"type":"string"}},{"name":"pageSize","in":"query","description":"Items to return per page (default: 30).","schema":{"type":"integer","format":"int32"}},{"name":"page","in":"query","description":"Page number of the pagination (default: 1).","schema":{"type":"integer","format":"int32"}}],"responses":{"200":{"description":"A collection of sitemap requests that match the criteria","content":{"application/json":{"schema":{"$ref":"#/components/schemas/pagedResult_sitemapRequestResponse"}}}},"400":{"description":"Invalid query parameters","content":{"application/json":{"schema":{"$ref":"#/components/schemas/apiErrorResponse"}}}}}}}}}
```


# GET v1/site/map/{id}

This endpoint retrieves information about a site mapping request.

## Get a sitemap request by ID

> This endpoint retrieves a sitemap request by its ID.

```json
{"openapi":"3.0.1","info":{"title":"Gaffa API Open API Definition","version":"1.0.0"},"servers":[{"url":"https://api.gaffa.dev"}],"security":[{"API Key":[]}],"components":{"securitySchemes":{"API Key":{"type":"apiKey","name":"X-API-Key","in":"header"}},"schemas":{"sitemapRequestResponse":{"type":"object","properties":{"id":{"type":"string","description":"ID of the sitemap request","nullable":true},"url":{"type":"string","description":"URL of the request","nullable":true},"state":{"type":"string","description":"The status of the request","nullable":true},"credit_usage":{"type":"integer","description":"The number of credits used by the request","format":"int32","nullable":true},"error":{"type":"string","description":"The name of the error type","nullable":true},"error_reason":{"type":"string","description":"More detail about the error","nullable":true},"from_cache":{"type":"boolean","description":"If this request was served from the cache","nullable":true},"started_at":{"type":"string","description":"The time in UTC when the request started.","format":"date-time"},"completed_at":{"type":"string","description":"The time in UTC when the request finished.","format":"date-time"},"running_time":{"type":"string","description":"The running time of the request","format":"timespan"},"links":{"type":"array","items":{"type":"string"},"description":"List of URLs found in the sitemap","nullable":true},"link_count":{"type":"integer","description":"Number of links found","format":"int32","nullable":true}}},"apiErrorResponse":{"type":"object","properties":{"type":{"type":"string","description":"The type of object this is concerning","nullable":true},"id":{"type":"string","description":"The id of the item concerned.","nullable":true},"code":{"type":"string","description":"Error code.","nullable":true},"message":{"type":"string","description":"Error description.","nullable":true}}}}},"paths":{"/v1/site/map/{id}":{"get":{"tags":["Sitemap Requests"],"summary":"Get a sitemap request by ID","description":"This endpoint retrieves a sitemap request by its ID.","operationId":"getSitemapRequestById","parameters":[{"name":"id","in":"path","description":"The unique identifier of the sitemap request to retrieve.","required":true,"schema":{"type":"string"}}],"responses":{"200":{"description":"The sitemap request","content":{"application/json":{"schema":{"$ref":"#/components/schemas/sitemapRequestResponse"}}}},"404":{"description":"Sitemap request not found","content":{"application/json":{"schema":{"$ref":"#/components/schemas/apiErrorResponse"}}}}}}}}}
```


# Convert any webpage into LLM-ready Markdown using Gaffa

The ability to convert websites into LLM-friendly markdown is powerful when building applications for summarization, Q\&A, or knowledge extraction. In this guide, you'll learn how to use the [Gaffa API](https://gaffa.dev/) to extract the main content of any web page using browser rendering and convert it into structured markdown.

By the end of this guide, you’ll be able to:

* Render web pages using Gaffa’s API.
* Extract clean page content.
* Generate structured markdown suitable for LLM-based Q\&A or summarization.

### **Prerequistes**

1. Install Python 3.10 or newer.
2. Create a virtual environment

```sh
python -m venv venv && source venv/bin/activate
```

3. Install the required libraries

```sh
pip install requests openai
```

4. Get your [Gaffa API](https://gaffa.dev/dashboard/api-keys) key and [OpenAI API](https://platform.openai.com/signup) key, and store them as environment variables:

```sh
GAFFA_API_KEY=your_gaffa_api_key
OPENAI_API_KEY=your_openai_api_key
```

### Convert a webpage to Markdown

In the code below, we define a function that takes a URL as input, makes a POST request to the Gaffa API, invoking the [generate\_markdown](/docs/features/browser-requests/actions/generate-markdown) action, which uses the browser rendering engine to extract the page's main content and convert it to markdown.

{% code overflow="wrap" lineNumbers="true" %}

```python
import requests
import openai

GAFFA_API_KEY = os.getenv("GAFFA_API_KEY")
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

# Fetch the markdown content from Gaffa
def fetch_markdown_with_gaffa(url):
    payload = {
        "url": url,
        "proxy_location": None,
        "async": False,
        "max_cache_age": 0,
        "settings": {
            "record_request": False,
            "actions": [
                {
                    "type": "wait",
                    "selector": "article"
                },
                {
                    "type": "generate_markdown"
                }
            ]
        }
    }
   
    # Set the headers for the request
    headers = {
        "x-api-key": GAFFA_API_KEY,
        "Content-Type": "application/json"
    }
    # Make the POST request to the Gaffa API
    print("Calling Gaffa API to generate markdown...")
    response = requests.post("https://api.gaffa.dev/v1/browser/requests", json=payload, headers=headers)
    response.raise_for_status()
   
    # Extract the markdown URL from the response
    markdown_url = response.json()["data"]["actions"][1]["output"]
   
    # Fetch the markdown content from the generated URL
    print(f"📥 Fetching markdown from: {markdown_url}")
    markdown_response = requests.get(markdown_url)
    markdown_response.raise_for_status()
   
    return markdown_response.text
```

{% endcode %}

### Ask questions using OpenAI

Now that we have the markdown content, we can ask questions about it using the OpenAI API. The function below takes markdown content and a question as input, then uses the OpenAI API to generate a summary based on the provided content. In this case, we are using the [gpt-3.5-turbo](https://platform.openai.com/docs/models) model, but you can choose any other model.

{% code overflow="wrap" lineNumbers="true" %}

```python
def ask_question(markdown, question):
    openai.api_key = OPENAI_API_KEY
    prompt = (
        f"You are an assistant helping analyze different webpages.\n\n"
        f"Markdown content:\n{markdown[:3000]}\n\n"
        f"Question: {question}\nAnswer as clearly as possible."
    )

    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "user", "content": prompt}
        ]
    )
    return response.choices[0].message["content"]
```

{% endcode %}

The markdown becomes the model’s context, enabling accurate answers about the original web content.

### User Interaction and Execution

Having defined the functions, we can now create a simple command-line interface that lets users enter a URL and ask questions about its content.

{% code overflow="wrap" lineNumbers="true" %}

```python
def main():
    url = input("Enter the URL of the article: ")
    try:
        markdown = fetch_markdown_with_gaffa(url)
        print("\n✅ Markdown successfully retrieved from Gaffa.\n")

        while True:
            question = input("Ask a question about the content (or type 'exit'): ")
            if question.lower() == "exit":
                break
            answer = ask_question(markdown, question)
            print(f"\n💬 Answer: {answer}\n")

    except Exception as e:
        print(f"⚠️ Error: {e}")

 if __name__ == "__main__":
    main()
```

{% endcode %}

### Full Script

The full script is available to download from the [Gaffa Python Examples GitHub repo](https://github.com/GaffaAI/GaffaPythonExamples/blob/main/scripts/WebpageToMarkdown/markdown_generator.py).

### Running the Script

To run the script, simply execute it in your terminal:

```sh
python your_script_name.py
```

With your script running, you can enter any web page URL, and it will fetch the markdown content and let you ask questions about it.


# Capture a full-height screenshot of a webpage

In just a few lines of JSON inlined in a single cURL command, you can automate:

* Dismissing Wikipedia’s EU cookie consent banner (if present)
* Waiting for the main heading on the Artificial Intelligence article
* Scrolling through every section (lazy-loaded images and all)
* Capturing a full-page PNG for archiving, visual regression, or documentation

All without installing Playwright or managing headless browsers, Gaffa handles it for you server-side via the[ Browser Requests API](https://gaffa-1.gitbook.io/gaffa/features/browser-requests).

### Prerequisites

* A valid Gaffa API key
* A simple HTTP client (cURL, Postman, axios, etc.).
* Familiarity with the[ API Playground](https://gaffa.dev/dashboard/playground) for testing browser requests.
* Target URL for this tutorial, for this we'll use Wikipedia: <https://en.wikipedia.org/wiki/Artificial_intelligence>

{% stepper %}
{% step %}

### Execute the Request

Use cURL with the full JSON payload inlined to ensure Gaffa receives exactly what you intend:

```sh
curl https://api.gaffa.dev/v1/browser/requests \
  --request POST \
  --header 'Content-Type: application/json' \
  --header 'X-API-Key: YOUR_API_KEY' \
  --data '{
    "url": "https://en.wikipedia.org/wiki/Artificial_intelligence",
    "async": false,
    "max_cache_age": 0,
    "settings": {
      "actions": [
        {
          "type": "wait",
          "selector": "#cookie-policy-notice",
          "timeout": 10000,
          "continue_on_fail": true
        },
        {
          "type": "click",
          "selector": "#cookie-policy-notice",
          "continue_on_fail": true
        },
        {
          "type": "wait",
          "selector": "#firstHeading",
          "timeout": 10000
        },
        {
          "type": "scroll",
          "percentage": 100
        },
        {
          "type": "capture_screenshot",
          "size": "fullscreen"
        }
      ]
    }
  }'
```

Replace YOUR\_API\_KEY with your actual token from your [Dashboard.](https://gaffa.dev/dashboard/api-keys) This command has the following actions:

1. **Wait** (optional): Detect and accept Wikipedia’s cookie banner if it appears. If it fails, that simply means no banner was present, or it did not load in time. Since continue\_on\_fail defaults to true, Gaffa will continue without halting the workflow, ensuring the remaining steps still execute.
2. **Wait**: Ensure the main heading (#firstHeading) is loaded.
3. **Scroll**: Scroll through the entire page to trigger any lazy-loaded content.
4. **Capture** Screenshot: Produce a full-page PNG.
   {% endstep %}

{% step %}

### Retrieve Your Screenshot

A successful response returns JSON like:

{% code lineNumbers="true" %}

```json
{
  "data": {
    "id": "brq_VJX3mbESLiyCFYvZQEUih9RdDYovog",
    "url": "https://en.wikipedia.org/wiki/Artificial_intelligence",
    "proxy_location": null,
    "state": "completed",
    "credit_usage": 2,
    "http_status_code": 200,
    "from_cache": false,
    "started_at": "2025-06-09T15:55:46.4235903Z",
    "completed_at": "2025-06-09T15:56:27.9381332Z",
    "running_time": "00:00:40.7348244",
    "page_load_time": "00:00:02.2087117",
    "actions": [
      {
        "id": "act_VJX3memaue6YUgFcn44uNscZbVUpYg",
        "type": "wait",
        "query": "wait?selector=%23cookie-policy-notice%2C%20.mw-cookie-consent-container&timeout=10000&continue_on_fail=true",
        "timestamp": "2025-06-09T15:55:48.6323091Z",
        "error": "action_timed_out"
      },
      {
        "id": "act_VJX3mkwfwNPdGiMUpqKr34Tm5xzyUU",
        "type": "click",
        "query": "click?selector=%23cookie-policy-notice%20button%2C%20.mw-cookie-consent-container%20button&continue_on_fail=true&timeout=5000",
        "timestamp": "2025-06-09T15:55:58.7949275Z",
        "error": "action_timed_out"
      },
      {
        "id": "act_VJX3mkSJ3sevWRXUCjFy6zwfD172fV",
        "type": "wait",
        "query": "wait?selector=%23firstHeading&timeout=10000&continue_on_fail=false",
        "timestamp": "2025-06-09T15:56:03.9581113Z"
      },
      {
        "id": "act_VJX3mbq9Jgj8EwADszW2AqdeJJXJiY",
        "type": "scroll",
        "query": "scroll?percentage=100&max_scroll_time=20000&scroll_speed=medium&continue_on_fail=false",
        "timestamp": "2025-06-09T15:56:03.9691994Z"
      },
      {
        "id": "act_VJX3mjBQYv8zTsXv1SkgUnBkzNFmJU",
        "type": "capture_screenshot",
        "query": "capture_screenshot?size=fullscreen&continue_on_fail=false",
        "timestamp": "2025-06-09T15:56:20.0727905Z",
        "output": "https://storage.gaffa.dev/brq/image/brq_VJX3mbESLiyCFYvZQEUih9RdDYovog/act_VJX3mjBQYv8zTsXv1SkgUnBkzNFmJU_full.png"
      }
    ]
  },
  "error": null
}
```

{% endcode %}

The response contains the following information:

* **data.id**: Unique request identifier.
* **data.state**: "completed" means the workflow finished (even if some steps timed out).
* **data.credit\_usage**: Credits consumed for this run.
* **data.started\_at** / **data.completed\_at**: Workflow timing.
* **data.running\_time** and **data.page\_load\_time**: Performance metrics.
* **data.actions**: Each action’s details, including successes, timeouts, and final screenshot URL.

Within the list of actions, you'll be able to see the capture\_screenshot action, which contains an output parameter containing the full-size screenshot that was captured.
{% endstep %}
{% endstepper %}

If you don't want to use cURL, you can also run this query in the [Gaffa API Playground](https://gaffa.dev/dashboard/playground), which is an easy way to get started.

### Use Cases

Gaffa's screenshot action could be used for a huge number of use cases, but here are a few ideas:

* **Visual Regression**: Integrate into your CI pipeline to compare changes over time.
* **Archival**: Schedule daily captures for audit or compliance purposes.
* **Monitoring**: Automate periodic checks to detect visual bugs or layout shifts.

#### All this is powered by Gaffa’s hosted headless browsers with no local setup required. Experiment with more actions and easily build complex browser workflows. Refer to the full[ Browser Requests API documentation](https://gaffa-1.gitbook.io/gaffa/features/browser-requests) for additional capabilities.

<br>


# How to scrape all images from a website using Gaffa

This tutorial will show you how you can use Gaffa to retrieve all images from a site and then download all images across those pages.

Automating the collection of images from a website can save hours of manual work. Whether you're a marketer building a competitor analysis, a developer creating a dataset, or an archiver preserving digital content, doing this manually is tedious and error-prone.

In this tutorial, you'll learn how to use Gaffa's powerful [Mapping](/docs/features/mapping-requests) and [Browser Requests](/docs/features/browser-requests) endpoints to automatically find, extract, and download every image from a website in a short Python script. We'll leverage features like the [`capture_dom`](/docs/features/browser-requests/actions/capture-dom) action, [intelligent sitemap parsing](/docs/features/mapping-requests), and the [`download_file`](/docs/features/browser-requests/actions/download-file) action to handle this efficiently and responsibly.

By the end of this guide, you'll be able to:

* Use Gaffa's [`site/map`](/docs/features/mapping-requests) endpoint to discover every page on a site.
* Render each page with a headless browser to capture its full DOM.
* Parse and download all images using Gaffa's [`download_file`](/docs/features/browser-requests/actions/download-file) action with residential proxies
* Run the process at scale with built-in proxy rotation and caching.

### Prerequisites

* **Python 3.10+** is installed on your machine.
* A **Gaffa API key.** [Sign up for a free account](https://gaffa.dev/sign-up) and get your API key from the dashboard.
* Basic familiarity with the command line.

{% stepper %}
{% step %}

### Set Up Your Environment

First, create a new project directory and install the required Python libraries.

```
# Create a new directory and navigate into it
mkdir gaffa-image-scraper && cd gaffa-image-scraper

# Create a virtual environment (optional but recommended)
python -m venv venv
source venv/bin/activate
```

Next, set your Gaffa API key as an environment variable to keep it secure.

```
# On macOS/Linux
export GAFFA_API_KEY='your_gaffa_api_key_here'
```

{% endstep %}

{% step %}

### The Core Script Explained

Let's build the script step-by-step. The core logic consists of three main parts: mapping the site, capturing the DOM for each page, and extracting images using Gaffa's download system.

**Fetch All URLs from the Sitemap**

The `site/map` endpoint is our starting point. It does the heavy lifting of discovery by reading the sitemap, traversing potential link-outs, and retrieving every page on the website you want to scrape.

```
def get_sitemap_urls(site_url, max_cache_age=86400):
    payload = {
        "url": site_url,
        "max_cache_age": max_cache_age
    }
    print("Retrieving sitemap URLs.")
    response = requests.post("https://api.gaffa.dev/v1/site/map", 
        json=payload, headers=HEADERS)
    return response.json()["data"]["links"]
```

**Capture the Rendered DOM of a Page**

For each URL, we use Gaffa to fully render the page (including JavaScript execution) and capture the final DOM. This is an important step since many websites are actually not fully rendered when we receive them. They contain links to JavaScript files that need to be executed first. These scripts will load further content from the backend, load images and other data. It’s necessary to first generate a fully rendered page before diving deeper into scraping it; otherwise, we would only scrape the content already provided in the initial HTML.

```
def get_dom(url):
    payload = {
        "url": url,
        "async": False,
        "settings": {
            "actions": [
                {"type": "wait", "selector": "img", "timeout": 20000},
                {"type": "capture_dom"}
            ],
            "time_limit": 40000
        }
    }
    print("Capturing DOM URL.")
    response = requests.post("https://api.gaffa.dev/v1/browser/requests", 
        json=payload, headers=HEADERS)
    dom_url = response.json()["data"]["actions"][1]["output"]
    print("Retrieving DOM.")
    dom_response = requests.get(dom_url)
    return dom_response.text
```

**Extract Images and Download with Gaffa**

With the real HTML in hand, we extract image URLs using a simple regex pattern and use Gaffa's [`download_file`](/docs/features/browser-requests/actions/download-file) action for secure, reliable downloads. This also allows us to use caching, which avoids downloading the same image over and over again and putting a load on the target server.

```
def extract_image_urls(dom_content, base_url):
    image_urls = []
    src_pattern = r'<img[^>]+(?:src|data-src)=["\']([^"\']+)["\']'
    matches = re.findall(src_pattern, dom_content)
    
    for src in matches:
        if not src.startswith(('http:', 'https:')):
            src = urljoin(base_url, src)
        image_urls.append(src)
    
    return image_urls

def download_image(image_url, filename):
    payload = {
        "url": image_url,
        "async": False,
        "settings": {
            "actions": [{"type": "download_file"}]
        }
    }
    print("Retrieving download URL.")
    response = requests.post("https://api.gaffa.dev/v1/browser/requests", json=payload, headers=HEADERS)
    actions = response.json()["data"]["actions"]
    download_url = actions[0]["output"]
    download_ext = os.path.splitext(download_url)[1]
    
    print("Downloading image.")
    img_response = requests.get(download_url)
    filepath = f"{filename}{download_ext}"
    with open(filepath, 'wb') as f:
        f.write(img_response.content)

```

{% endstep %}

{% step %}

### Bringing It All Together

The main() function orchestrates the entire workflow: mapping the site, processing each page, and downloading the images using Gaffa's infrastructure.

```
def main():
    site_url = "https://gaffa.dev"
    sitemap_urls = get_sitemap_urls(site_url)[:3]
    
    for i, url in enumerate(sitemap_urls, 1):
        dom_content = get_dom(url)
        image_urls = extract_image_urls(dom_content, url)
        
        if image_urls:
            download_image(image_urls[0], f"image_{i}")

if __name__ == "__main__":
    main()
```

{% endstep %}

{% step %}

### Run the Script

Save the complete code to a file like `gaffa_scrape_images.py` and run it from your terminal:

```
python3 gaffa_scrape_images.py
```

Sit back and watch as Gaffa automatically discovers, renders, and scrapes every image from the site using proxies and real browsers. The script will create timestamped folders and save all the images there.
{% endstep %}
{% endstepper %}

### Why This Gaffa-Powered Approach is Superior

* **Handles JavaScript-Rendered Content:** Unlike simple HTTP scrapers, Gaffa uses a real browser, so it captures anything that is lazy-loaded by JavaScript.
* **Stealth Downloading with Residential Proxies:** The download\_file action uses real browsers and proxies, making your requests appear as legitimate user traffic.
* **Intelligent Caching:** With \`max\_cache\_age\` set to 24 hours, repeated requests for the same image are served from cache, reducing load on target servers and improving efficiency.
* **Built-in Reliability:** Gaffa's infrastructure handles proxy rotation, request pacing, retries automatically and provides the correct file format directly.
* **Respectful Scraping:** Gaffa's infrastructure is designed for responsible automation. Always check a website's robots.txt and terms of service before scraping, and respect reasonable rate limits.

### Use Cases and Ideas

This technique is useful for far more than just downloading pictures. Here are a few ideas:

* **Competitive Analysis**: Analyze competitors' product photography styles using real browsers.
* **AI/ML Datasets**: Build large, curated image datasets for training computer vision models using ethically sourced images.
* **Website Migration & Audits**: Download all assets from an old site before a migration while minimizing server impact through caching.
* **Archival & Documentation**: Preserve visual evidence for journalism or create backups of a site's visual content using proxies for access.

#### Next Steps

The full script is available on our [GitHub repository](https://github.com/GaffaAI/GaffaPythonExamples/tree/main/scripts/ScrapeAllImages).

Ready to automate your image collection with enterprise-grade infrastructure? [Sign up for Gaffa](https://gaffa.dev/sign-up) and start building today.


# Extract and Fill Web Forms Automatically Using Gaffa

Web forms are some of the most common and repetitive elements that users often interact with as developers. Whether you are collecting data, testing user flows, or even building other automation systems.

In this guide, you'll learn how to use `pase_json` action to extract the structure of a web form and then automatically fill and submit it using Gaffa's browser automation features.

By the end of this guide, you will be able to:

* Extract structured form data (labels, input names, required fields, and placeholders) using `parse_json`
* Define and use schemas to reliably understand page structure
* Build a simple interactive CLI that collects user input
* Automatically fill and submit a web form using Gaffa browser actions

### **Prerequistes**

1. Install Python 3.10 or newer.
2. Create a virtual environment

```sh
python -m venv venv && source venv/bin/activate
```

3. Install the required libraries

```sh
pip install requests openai
```

4. Get your [Gaffa API](https://gaffa.dev/dashboard/api-keys) key and store it as an environment variable:

```sh
GAFFA_API_KEY=your_gaffa_api_key
```

5. Install the required library

```sh
pip install requests
```

### What You'll Build

In this tutorial, you'll create a Python script that:

* **Extracts form fields** - Uses Parse JSON to analyze any web form and identify all input fields.
* **Collects user input** - Prompts the user in the terminal to provide values for each field.
* **Submits the form** - Automatically fills and submits the form using Gaffa's browser automation.

By the end, you'll have a working form automation tool that can be adapted for countless use cases.

### Set Up Your Environment

Create a new directory and Python file.

```sh
mkdir gaffa-form-filler
cd gaffa-form-filler
```

Create a file called `form_filler.py` (*or any name that works for you*) and add your configuration.

```python
import requests
import json

# Configuration
GAFFA_API_KEY = "your_api_key_here"  # Replace with your actual API key
GAFFA_API_URL = "https://api.gaffa.dev/v1/browser/requests"

# The demo form we'll work with
FORM_URL = "https://demo.gaffa.dev/simulate/form?loadTime=3&showModal=true&modalDelay=5&formType=address"
```

Replace `your_api_key_here` with your actual Gaffa API key from the [Dashboard](https://gaffa.dev/dashboard).

### Extract Form Fields Using `parse_json`

In the code below, you define a function that takes a form URL as input and makes a POST request to the Gaffa API.

The request uses two actions: first, a `wait` action ensures the form element is fully loaded on the page, then the `parse_json` action that uses AI to intelligently analyze the form structure and extract all input fields along with their properties (labels, names, types, placeholders, and required status). The AI understands the context of the form and returns structured JSON data that we can easily work with.

```python
def extract_form_fields(form_url):
    payload = {
        "url": form_url,
        "async": False,
        "settings": {
            "record_request": False,
            "actions": [
                {
                    "type": "wait", 
                    "selector": "form", 
                    "timeout": 10000
                },
                {
                    "type": "parse_json",
                    "data_schema": {
                        "name": "FormFields",
                        "description": "Extract all form input fields",
                        "fields": [
                            {"type": "string", "name": "form_title", "description": "Form title"},
                            {
                                "type": "array",
                                "name": "fields",
                                "description": "List of all input fields",
                                "fields": [
                                    {"type": "string", "name": "label", "description": "Field label"},
                                    {"type": "string", "name": "field_name", "description": "Field name attribute"},
                                    {"type": "string", "name": "field_type", "description": "Input type"},
                                    {"type": "boolean", "name": "required", "description": "Is required?"},
                                    {"type": "string", "name": "placeholder", "description": "Placeholder text"}
                                ]
                            }
                        ]
                    },
                    "instruction": "Extract all form fields with their properties",
                    "model": "gpt-4o-mini",
                    "output_type": "inline"
                }
            ]
        }
    }
    
    headers = {"X-API-Key": GAFFA_API_KEY, "Content-Type": "application/json"}
    response = requests.post(GAFFA_API_URL, json=payload, headers=headers)
    response.raise_for_status()
    result = response.json()
    
    for action in result["data"]["actions"]:
        if action.get("type") == "parse_json":
            return action["output"]
    
    return None
```

### Collect User Input

Next, you need to define a function that takes the extracted form data and interacts with the user in the terminal. The function will display the form title and then loop through each field, prompting the user to fill in the value.

For each field in the form, a label and a required marker, if applicable, are shown. The function ensures that the required fields are not left empty and allows users to skip optional fields by pressing enter. All the user's input is collected into a dictionary where the keys are the field names and the values are what the user entered.

```python
def collect_user_input(form_data):
    print(f"\n{'='*60}")
    print(f"📋 Form: {form_data.get('form_title', 'Unknown Form')}")
    print(f"{'='*60}\n")
    
    user_values = {}
    fields = form_data.get("fields", [])
    
    if not fields:
        print("⚠️  No fields found in the form")
        return user_values
    
    print(f"Please provide values for {len(fields)} field(s):\n")
    
    for i, field in enumerate(fields, 1):
        label = field.get("label", "Unknown Field")
        field_name = field.get("field_name", "")
        required = field.get("required", False)
        placeholder = field.get("placeholder", "")
        
        required_marker = " *" if required else ""
        placeholder_hint = f" (e.g., {placeholder})" if placeholder else ""
        prompt = f"[{i}/{len(fields)}] {label}{required_marker}{placeholder_hint}: "
        
        while True:
            value = input(prompt).strip()
            
            if required and not value:
                print("  ⚠️  This field is required. Please provide a value.")
                continue
            
            if not value and not required:
                print("  ℹ️  Skipping optional field")
                break
            
            user_values[field_name] = value
            break
    
    return user_values
```

### Fill and Submit the Form

You need a function that will take the form URL and the user's input values, then submit the form to Gaffa's browser automation. The function will build a list of actions.

First, it waits for the form to be ready, then creates a `type` action for each field to enter the user's value into the corresponding input element using CSS selectors. Lastly, it adds a `click` action to submit the form and a `capture_screenshot` action to take a full-screen image of the results.

The function makes a POST request with all these actions and returns the response, which includes the screenshot URL if successful.

```python
def fill_form(form_url, field_values):
    if not field_values:
        return None
    
    actions = [
        {
            "type": "wait", 
            "selector": "form", 
            "timeout": 10000
        }
    ]
    
    for field_name, value in field_values.items():
        if value:
            actions.append({
                "type": "type",
                "selector": f"[name='{field_name}']",
                "text": value
            })
    
    actions.extend([
        {"type": "click", "selector": "button[type='submit']"},
        {"type": "capture_screenshot", "size": "fullscreen"}
    ])
    
    payload = {
        "url": form_url,
        "async": False,
        "settings": {
            "record_request": False,
            "actions": actions
        }
    }
    
    headers = {"X-API-Key": GAFFA_API_KEY, "Content-Type": "application/json"}
    response = requests.post(GAFFA_API_URL, json=payload, headers=headers)
    response.raise_for_status()
    
    return response.json()
```

### User Interaction and Execution

Having defined the functions, we can now create a simple command-line interface that allows users to interact with the form.

```python
def main():
    print("\n" + "="*60)
    print("🤖 Gaffa Form Filler")
    print("="*60)
    print("This tool extracts form fields and helps you fill them out.\n")
    
    print("📋 Step 1: Analyzing form...")
    form_data = extract_form_fields(FORM_URL)
    
    if not form_data:
        print("\n❌ Could not extract form fields")
        return
    
    print(f"✅ Found {len(form_data.get('fields', []))} field(s)\n")
    
    print("📝 Step 2: Collecting your input...")
    user_values = collect_user_input(form_data)
    
    if not user_values:
        print("\n⚠️  No values provided. Exiting.")
        return
    
    print(f"\n{'='*60}")
    print("📊 Summary of values to submit:")
    print(f"{'='*60}")
    for field_name, value in user_values.items():
        print(f"  {field_name}: {value}")
    print(f"{'='*60}\n")
    
    confirm = input("Submit this form? (y/n): ").strip().lower()
    if confirm != 'y':
        print("\n❌ Submission cancelled")
        return
    
    print("\n🚀 Step 3: Submitting form...")
    result = fill_form(FORM_URL, user_values)
    
    if not result:
        print("❌ Form submission failed")
        return
    
    print("\n✅ Form submitted successfully!")
    
    if "data" in result and "actions" in result["data"]:
        for action in result["data"]["actions"]:
            if action.get("type") == "capture_screenshot" and "output" in action:
                print(f"📸 Screenshot: {action['output']}")
    
    print("\n🎉 All done!\n")

if __name__ == "__main__":
    main()
```

### Full Script

The full script is available to download from the [Gaffa Python Examples GitHub repo](https://github.com/GaffaAI/GaffaPythonExamples/blob/main/scripts/AutomatedFormFilling/automated_form_filling.py).

### Running the Script

To run the script, simply execute it in your terminal:

```sh
python your_script_name.py
```

### Example output:

```sh
============================================================
🤖 Gaffa Form Filler
============================================================
This tool extracts form fields and helps you fill them out.

📋 Step 1: Analyzing form...
✅ Found 9 field(s)

📝 Step 2: Collecting your input...

============================================================
📋 Form: Form Submission Test
============================================================

Please provide values for 9 field(s):

[1/9] First Name *: John
[2/9] Last Name *: Smith
[3/9] Email *: john@example.com
...

============================================================
📊 Summary of values to submit:
============================================================
  first_name: John
  last_name: Smith
  email: john@example.com
...

Submit this form? (y/n): y

🚀 Step 3: Submitting form...

✅ Form submitted successfully!

🎉 All done!
```


# Using the Gaffa LLMs.txt File with Your AI Assistant

AI assistants like ChatGPT or Claude can generate working code far more effectively when they have accurate, up-to-date context about an API. That's exactly what Gaffa's `llms.txt` file provides. It provides a concise reference covering Gaffa's endpoints, actions, and code samples that you can drop directly into any AI assistant to get useful, accurate code from the very first prompt.

In this tutorial, we'll walk you through how to use the `llms.txt` file to build a complete Python script that interacts with the Gaffa API.

#### Step 1: Get the LLMs.txt File

Download or open the file at <https://gaffa.dev/docs/llms.txt>. It contains a concise overview of the Gaffa API, including available endpoints, actions, and example payloads.

#### Step 2: Load It Into Your AI Assistant

Start a new chat with ChatGPT, Claude, or your preferred AI assistant, then paste the full contents of the file into the conversation. This gives the assistant accurate, up-to-date context about the Gaffa API before you ask it anything.

#### Step 3: Ask the Assistant to Write Your Script

Once the assistant has the context loaded, you can ask it to build scripts for you. For example:

> *"Write me a Python script that uses Gaffa's browser API to convert a page into Markdown and save the output file locally."*

Because the assistant already has the full API context, it can produce accurate code without you needing to explain endpoint structures or payload formats.

#### Step 4: Example Script

Here's an example of the kind of script your AI assistant might generate, based directly on the Gaffa API. It submits a browser request to convert a page to Markdown, polls until the request completes, and downloads the output file.

```python
import os, time, requests, pathlib, urllib.parse

API_KEY = os.environ.get("GAFFA_API_KEY", "YOUR_API_KEY")
BASE = "https://api.gaffa.dev"

def submit_request(url, actions, async_mode=True):
    payload = {
        "url": url,
        "async": async_mode,
        "settings": {"actions": actions}
    }
    r = requests.post(
        f"{BASE}/v1/browser/requests",
        headers={"X-API-Key": API_KEY, "Content-Type": "application/json"},
        json=payload
    )
    r.raise_for_status()
    return r.json()["data"]

def wait_for_completion(request_id, poll_every=2, max_wait=180):
    start = time.time()
    while True:
        r = requests.get(
            f"{BASE}/v1/browser/requests/{request_id}",
            headers={"X-API-Key": API_KEY}
        )
        data = r.json()["data"]
        if data["state"] in ("completed", "failed"):
            return data
        if time.time() - start > max_wait:
            raise TimeoutError("Request timed out")
        time.sleep(poll_every)

def download_outputs(brq, dest="outputs"):
    dest = pathlib.Path(dest)
    dest.mkdir(parents=True, exist_ok=True)
    files = []
    for act in brq.get("actions") or []:
        out = act.get("output")
        if isinstance(out, str) and out.startswith("http"):
            name = pathlib.Path(urllib.parse.urlparse(out).path).name
            p = dest / name
            with requests.get(out, stream=True) as r:
                with open(p, "wb") as f:
                    for chunk in r.iter_content(8192):
                        if chunk: f.write(chunk)
            files.append(str(p))
    return files

if __name__ == "__main__":
    target_url = "https://demo.gaffa.dev/simulate/article?paragraphs=5"
    actions = [
        {"type": "wait", "selector": "article"},
        {"type": "generate_markdown"}
    ]
    job = submit_request(target_url, actions)
    brq = wait_for_completion(job["id"])
    print("Final state:", brq["state"])
    if brq["state"] == "completed":
        saved = download_outputs(brq)
        print("Downloaded:", saved)
```

Run it with:

```bash
python gaffa_script.py
```

You'll see the job state printed in your terminal and a downloaded Markdown file saved to an `outputs` folder.

#### Step 5: Extend and Customise

From here, you can modify the `actions` list to use other supported operations, such as `generate_pdf`, `screenshot`, or `extract_text`. You can make these changes manually, or simply ask your AI assistant to adapt the script for you. Since it still has the `llms.txt` context loaded, it can adjust the code to your specific requirements without needing any further explanation.


# July

### API Updates

#### `parse_json` now available in the API Playground

You can now test `parse_json` directly in the [API Playground](https://gaffa.dev/dashboard/playground) using either the visual editor or the raw JSON request view. Two ready-to-run templates are available: [extract structured data from an HTML form](https://gaffa.dev/dashboard/playground?templateId=parse_html_form_to_structured_json) and [parse an academic paper PDF](https://gaffa.dev/dashboard/playground?templateId=parse_academic_paper_pdf).

### Samples, Blog & Tutorials

#### How Gaffa gives Chayn's Survivor AI access to platform policies other AI tools can't reach

[Chayn](https://www.chayn.co/) built [Survivor AI](https://tools.chayn.co/) to help people affected by image-based abuse generate takedown letters that cite the exact platform policies being violated. We published a case study on how Gaffa provides Chayn with reliable access to terms of service and community guidelines on platforms that block AI agents from reading those pages directly — all through a single API call. [Read the blog](https://gaffa.dev/blog/how-gaffa-gives-chayns-survivor-ai-access-to-platform-policies-other-ai-tools-cant-reach).


# June

### Pricing & Billing

#### AI credit pricing cut for `gpt-4o-mini`

Input token costs are down 50% (1 credit per 20,000 tokens, previously 10,000), and output token costs are down 75% (1 credit per 10,000 tokens, previously 4 credits per 10,000). AI-powered extraction just got significantly cheaper.

#### Browser bandwidth pricing cut by 50%

Residential proxy bandwidth is now billed at 750 credits per GB, down from 1,500. This applies automatically to all requests using a `proxy_location` parameter — no changes needed on your end.

#### 500 free credits for all accounts

Every account now includes 500 free credits, including accounts created before this change. If you signed up previously and didn't receive free credits, they've been automatically added to your account.

### Samples, Blog & Tutorials

#### How Gaffa powers a universal news plugin for TRMNL with a single API call

We published a case study on how Gaffa powers TRMNL's official news plugin, which lets users turn any news site into a personalised feed on their e-ink display using a single *parse\_json* call. The same integration has been used to pull train departures, cafeteria menus, and niche newsletters — all from the same schema and action. [Read the blog](https://gaffa.dev/blog/how-gaffa-powers-a-universal-news-plugin-for-trmnl-with-a-single-api-call-).


# May

### API Updates

#### **`Scroll` action now supports a selector parameter**&#x20;

[`scroll`](/docs/features/browser-requests/actions/scroll) action now accepts an optional `selector` parameter, allowing you to scroll a specific element on the page rather than the page body.

#### **`block_ads` parameter added**&#x20;

A new [`block_ads`](https://gaffa.dev/docs/features/browser-requests/settings#a-d-blocking) parameter is now available in request settings. When set to `true`, Gaffa intercepts and aborts requests to known ad domains before they load, reducing page load times on ad-heavy sites.&#x20;

#### **`generate_markdown` output improved**

We've made behind-the-scenes improvements to the [`generate_markdown`](https://gaffa.dev/docs/features/browser-requests/actions/generate-markdown) action, refining the output response format and expanding support for a greater variety of HTML elements. The result is cleaner, more consistent markdown across a broader range of pages.

### Samples, Blog & Tutorials

#### Speed up your scraping with ad blocking

We published a breakdown of how enabling [`block_ads`](https://gaffa.dev/docs/features/browser-requests/settings#a-d-blocking) cuts unnecessary ad network requests before they load, reducing page load times by up to 50% on ad-heavy sites. We tested it across The Guardian, Allrecipes, and Daily Mail, and also showed how combining it with [`max_media_bandwidth`](https://gaffa.dev/docs/features/browser-requests/settings#max-media-bandwidth) pushes performance even further. [Read the blog.](https://gaffa.dev/blog/speed-up-your-scraping-with-ad-blocking)


# April

### Samples, Blog & Tutorials

#### How to Scrape a Table with Python (The Easy Way)

We published a full walkthrough of both table scraping approaches using [`capture_dom`](https://gaffa.dev/docs/features/browser-requests/actions/capture-dom) with BeautifulSoup and using Gaffa's [`parse_table`](https://gaffa.dev/docs/features/browser-requests/actions/parse-table) action, directly covering when to use each and how to get clean, structured output either way. Read the [blog](https://gaffa.dev/blog/how-to-scrape-a-table-with-python-\(the-easy-way\)).

#### Web Scraping with JavaScript Using Gaffa

We published a full walkthrough on scraping the web in JavaScript using Gaffa's REST API. It covers two core actions: [`generate_markdown`](https://gaffa.dev/docs/features/browser-requests/actions/generate-markdown) for extracting clean, readable content and [`capture_dom`](https://gaffa.dev/docs/features/browser-requests/actions/capture-dom) for pulling raw HTML when you need more control. Also touches on dynamic content handling, geo-routing, async mode, and actions like [`parse_table`](https://gaffa.dev/docs/features/browser-requests/actions/parse-table) and [`parse_json`](https://gaffa.dev/docs/features/browser-requests/actions/parse-json). Read the [blog](https://gaffa.dev/blog/web-scraping-with-javascript-using-gaffa).

#### Gaffa at Major League Hacking's Global Hack Week

We presented a live session at MLH's Global Hack Week Cloud, walking through web scraping fundamentals, an intro to Gaffa, and two hands-on demos, one scraping a Wikipedia article and feeding it into an OpenAI Q\&A session using [`generate_markdown`](https://gaffa.dev/docs/features/browser-requests/actions/generate-markdown), and another using [`parse_json`](https://gaffa.dev/docs/features/browser-requests/actions/parse-json) to extract structured data from both web pages and hosted PDFs. The session also covered scraping legality, dynamic content, and how `parse_json` field descriptions can enforce specific output formats. We've also added both demo notebooks to our Python examples repository:

* [`webpage_to_markdown_and_qa.ipynb`](https://github.com/GaffaAI/GaffaPythonExamples/blob/main/talks/MLH_Global_Hack_Week/webpage_to_markdown_and_qa.ipynb) — Scraping a Wikipedia page and using it as context for an OpenAI Q\&A session.
* [`structured_data_extraction_with_parse_json.ipynb`](https://github.com/GaffaAI/GaffaPythonExamples/blob/main/talks/MLH_Global_Hack_Week/structured_data_extraction_with_parse_json.ipynb) — Extracting structured data from web pages and hosted PDFs using `parse_json`.

Read the [recap](https://gaffa.dev/blog/gaffa-at-major-league-hackings-global-hack-week).


# Q1

Here's a summary of everything we shipped and published in Q1 2026.

### API Changes

#### `parse_json` Now Publicly Available

[`parse_json`](https://gaffa.dev/docs/features/browser-requests/actions/parse-json) has graduated from beta and is now publicly available. It uses AI to extract structured data from any webpage according to a schema you define, without HTML parsing or brittle CSS selectors. You describe the fields you want, and Gaffa returns a clean JSON object.&#x20;

#### Mapping Requests Now Publicly Available

[Mapping Requests](https://gaffa.dev/docs/features/mapping-requests), which lets you extract all URLs from a site's sitemap, has moved out of beta and is now publicly available to all users. It's useful for building crawlers, auditing site structure, or feeding a list of URLs into a batch scraping workflow.&#x20;

***

### Tools

#### New Tool: HTML to Markdown Converter

We launched [HTML2Markdown](https://html2markdown.gaffa.dev/), a free tool powered by Gaffa that converts any webpage into clean, readable markdown in one click. It's built on the same [`generate_markdown` ](/docs/features/browser-requests/actions/generate-markdown)action available in the API, so it's also a good way to see what the action produces before integrating it into your own project.

***

### Samples, Blog & Tutorials

#### Table Scraping: Python Examples and Full Walkthrough

We added a new set of Python examples to our GitHub samples repository focused on scraping tables, along with a full blog post walkthrough covering both approaches, when to use each, and how to get clean, structured output either way. There are three scripts covering different approaches:

* `capture_dom.py` — Fetches the raw HTML via Gaffa's `capture_dom` action and parses the table locally using BeautifulSoup. Good for when you need full control over how the data is processed.
* `parse_table_demo.py` — Uses Gaffa's `parse_table` action on our demo site to return structured JSON directly, with no HTML parsing required.
* `parse_table_wikipedia.py` — A real-world example using `parse_table` on Wikipedia's GDP by Country table. Shows how headers are automatically normalised into clean JSON keys.

View the [examples](https://github.com/GaffaAI/GaffaPythonExamples/tree/main/scripts/ScrapingTables). Read the [post](https://gaffa.dev/blog/how-to-scrape-a-table-with-python).

#### Automated Form Filling: Python Examples and Tutorial

We added a set of Python examples to our samples repository along with a full tutorial covering how to automate web form interactions end-to-end. It walks through using `parse_json` to extract all fields from a form into a structured schema, prompting for values in the terminal, filling and submitting the form using `type` and `click`, and capturing a screenshot after submission. It's designed for automation workflows that require schema-driven extraction with a human-in-the-loop data-entry step.

View the [examples](https://github.com/GaffaAI/GaffaPythonExamples/tree/main/scripts/AutomatedFormFilling). Read the [tutorial](https://gaffa.dev/docs/tutorials/forms).

#### How to Scrape Every Image from a Website

We published a guide walking through how to use Gaffa to automatically extract every image from a webpage, covering how to combine browser actions to navigate, wait for content to load, and pull out image URLs at scale. [Read the post.](https://gaffa.dev/blog/how-to-automatically-scrape-every-image-from-a-website)

#### How to Slash Your Gaffa Credit Costs by 40+%

We published a breakdown of how blocking unnecessary media downloads using `max_media_bandwidth` can reduce your credit usage by over 40% on image-heavy sites, with no impact on the text content you're trying to extract. [Read the blog.](https://gaffa.dev/blog/how-to-slash-your-gaffa-credit-costs-by-40-percent)

#### Let Your AI Assistant Write Your Gaffa Code

We published a guide showing how to use Gaffa's `llms.txt` file to give AI assistants like ChatGPT or Claude accurate, up-to-date context about the API, so they can generate working code straight away without you needing to explain endpoint structures or payload formats. [Read the blog.](https://gaffa.dev/blog/let-your-ai-assistant-write-your-gaffa-code)

#### Case Study: ivee

We published a case study on how ivee used Gaffa to scrape 50 job boards, tripling their curated job listings and saving 10 hours of manual work per week. It's a good real-world example of what's possible when you remove the infrastructure overhead from a scraping workflow. [Read the case study.](https://gaffa.dev/blog/ivee-case-study)


# 2025 Changelog

Here are some of the things we launched in 2025.

#### Q4

* [`download_file` ](/docs/features/browser-requests/actions/download-file)now preserves original filenames

#### Q3

* [`scroll` ](/docs/features/browser-requests/actions/scroll)— added `timeout` parameter
* [`download_file` ](/docs/features/browser-requests/actions/download-file)— expanded file type support (.pdf, .jpg, .jpeg, .png, .gif, .bmp, .webp, .svg, .tiff, .tif, .img)
* **Beta:** [Mapping Requests](/docs/features/mapping-requests) — extract URLs from sitemaps
* **Beta:** [`parse_json` ](/docs/features/browser-requests/actions/parse-json)— added `data_schema_id`, `data_schema`, `selector`, `output_type`, `max_pages`
* **Beta:** [`block_dom_removals` ](/docs/features/browser-requests/actions/block-dom-removals)— prevent item removal during infinite scroll
* Added [llms.txt](https://gaffa.dev/docs/llms-full.txt) for docs

#### Q2

* Pay-as-you-go credits now available
* [`click`](/docs/features/browser-requests/actions/click), [`type`](/docs/features/browser-requests/actions/type), [`wait` ](/docs/features/browser-requests/actions/wait)— improved iframe support
* [`download_file` ](/docs/features/browser-requests/actions/download-file)— download PDFs via Gaffa
* **Beta:** [`capture_element`](/docs/features/browser-requests/actions/capture-element), [`parse_json`](/docs/features/browser-requests/actions/parse-json), [`parse_table`](/docs/features/browser-requests/actions/parse-table), [`capture_cookies`](/docs/features/browser-requests/actions/capture-cookies)
* **Beta:** [`parse_json` ](/docs/features/browser-requests/actions/parse-json)now supports all web pages

#### Q1

* Added France proxy location
* [`click`](/docs/features/browser-requests/actions/click), [`type`](/docs/features/browser-requests/actions/type), [`wait`](/docs/features/browser-requests/actions/wait)— default timeout now 5 seconds
* [`scroll` ](/docs/features/browser-requests/actions/scroll)— new params: `wait_time`, `max_scroll_time`, `scroll_speed`, `interval`
* Added `max_media_bandwidth` and `time_limit` [settings](/docs/features/browser-requests)
* New stealth browser technology


