Fetch parameters
Everything tunable about a fetch rides in the request’s options block.
The block itself, and the one-line summary of each option, are on
Parameters & Configuration; this page is
what each one does and where every field in the response came from.
Fetch tiers are cumulative in the same way. The page body’s wire key is content
on every result.
| Tier | Adds |
|---|---|
minimalist |
content, content_format, canonical_url, title, http_status, warning, provider, cache_state, request_id, latency_ms |
default |
js_rendered, pdf_mode, content_length, content_sha256, description |
extended |
raw_html, html, chunks, screenshot, page_count, content_type, summary, language, author, publish_date, favicon, thumbnail, links, images, metadata, provider_chain, usage |
max |
media, transcript, excerpt, structured, subpages, enrichments |
The Python SDK’s fetch() does not expose tier yet — send it in options on a
direct POST /v1/fetch call until it does.
Only the page body and the failure signal come from every provider, and title
from every one but linkup; everything else is best-effort, and a
capability_gap warning says which nulls were structural.
A 404, a paywall or an empty JS-only page is the target’s answer
and comes back as a per-URL diagnostic, not a failed request.
Where each field comes from
Section titled “Where each field comes from”One vocabulary, whoever answered. The chain fails over, so the same fetch can be answered by a different engine tomorrow — these are the raw fields behind each normalized one, per provider.
Five fields never come from a provider at all and are omitted below: provider
and provider_chain record who answered and who was tried, and content_length,
content_sha256 and latency_ms are measured by Telem over the body it
received. Two more work the same way everywhere: enrichments carries whatever
provider keys normalization did not consume, verbatim, and title has a
fallback the rest do not — when the answering provider supplies none, the title
already known for that URL is used.
POST api.fetch.tinyfish.ai · document at results[0]
| Normalized | Read from |
|---|---|
content |
text |
content_format |
format — markdown when you asked for text, which tinyfish does not offer |
canonical_url / title |
final_url / title |
description |
description |
language / author |
language / author |
publish_date |
published_date |
links / images |
links / image_links (requested only when the tier includes them) |
A page tinyfish cannot fetch still answers 200: the result list is empty and the reason sits in errors[] as a closed-set code (bot_blocked, timeout, page_not_found, …). That code comes back in a provider_error warning, with the upstream status when tinyfish sent one, and the chain moves on. etag, last_modified and not_modified ride in enrichments.
POST /extract · document at results[0]
| Normalized | Read from |
|---|---|
content |
raw_content, else content / markdown / text |
content_format |
echoes the format you asked for |
canonical_url / title |
url / title |
favicon / images |
favicon / images |
usage / request_id |
usage / request_id (top level) |
A URL that fails lands in failed_results instead of results, and its reason comes back as a warning. The markdown carries site-relative links, flagged by a relative_links warning rather than rewritten — rewriting the body would make content_sha256 disagree with what tavily sent.
/contents · document at results[0]
| Normalized | Read from |
|---|---|
content |
text |
content_format |
always text |
canonical_url / title |
url / title |
summary |
summary (billed per page — only requested at max) |
excerpt |
highlights |
author / publish_date |
author / publishedDate |
favicon / thumbnail |
favicon / image |
links / images |
extras.links / extras.imageLinks (max only) |
metadata / subpages |
entities / subpages |
content_format is text whatever you asked for: exa declares no output format, and the same call was measured returning plain text for one URL and markdown for another. Reporting the request back would be a lie you could not detect.
POST /v2/scrape · document at data, page metadata at data.metadata
| Normalized | Read from |
|---|---|
content |
markdown |
content_format |
always markdown |
html / raw_html |
html / rawHtml |
screenshot / summary |
screenshot / summary |
links / images |
links / images |
structured |
json (surcharged — max only) |
canonical_url |
metadata.sourceURL, else metadata.url |
title / description |
metadata.title / metadata.description |
language / author |
metadata.language / metadata.author |
favicon / thumbnail |
metadata.favicon / metadata.og:image |
publish_date |
whichever date key the site itself used |
page_count / content_type |
metadata.numPages / metadata.contentType |
http_status |
metadata.statusCode |
cache_state / request_id |
metadata.cacheState / metadata.scrapeId |
usage |
metadata.creditsUsed |
pdf_mode |
parsed, when the page had a page count |
js_rendered |
always true |
metadata |
every metadata key no row above consumed |
Those seven formats — markdown, html, rawHtml, links, images, summary and screenshot — bill as one unit, which is why extended can fill most of its row from firecrawl in a single call.
GET r.jina.ai/<url> · document at data, page metadata at data.metadata
| Normalized | Read from |
|---|---|
content |
content, with the reader’s Title: / URL Source: preamble stripped |
content_format |
echoes the format you asked for |
canonical_url / title |
url / title, else the preamble’s |
description |
description |
language / author |
metadata.lang / metadata.author |
publish_date |
the preamble’s Published Time: |
favicon / thumbnail |
favicon (an object, flattened to a string) / metadata.og:image |
links / images / chunks |
links / images / chunks |
http_status |
httpStatus — the upstream status, not the transport’s |
usage |
usage.tokens |
metadata |
every metadata key no row above consumed |
The preamble is stripped before hashing, so content_sha256 covers the document rather than jina’s framing of it. Watch http_status: jina answers 200 with a body even when the upstream fetch failed, so a caller checking the transport status alone would read an error page as content. That case also raises an upstream_error warning.
POST /v1/extract · document at results[0]
| Normalized | Read from |
|---|---|
content |
full_content |
content_format |
always markdown |
canonical_url / title |
url / title |
publish_date |
publish_date |
warning |
warnings[0] (top level) |
request_id |
extract_id (top level) |
usage |
usage — the SKU list (top level) |
Full content is requested on every call, at no extra cost — without it there is no document. A fetch has no search objective to pass, and parallel says so in a warning on every call, which is why warning is always filled on a parallel document. A URL it cannot extract lands in errors[] and comes back as a provider_error naming the error type and upstream status.
POST /v1/contents · document is the first element of a top-level array
| Normalized | Read from |
|---|---|
content |
markdown |
content_format |
always markdown |
raw_html |
html |
canonical_url / title |
url / title (a title that is only the URL is dropped) |
favicon |
metadata.favicon_url |
metadata |
every other metadata key |
Every call asks for a fresh crawl (max_age: 0). you.com can answer from its own cache instead, but that copy measured degraded — a URL for a title and a quarter of the page — so a fresh crawl’s extra seconds buy the whole document.
POST /v1/fetch · the whole body is the document
| Normalized | Read from |
|---|---|
content |
markdown |
content_format |
always markdown |
js_rendered |
always true |
raw_html / content_type |
rawContent / contentType (requested only when the tier includes them) |
canonical_url |
the URL you sent — linkup echoes none |
favicon |
favicon |
images |
images[].url (requested only when the tier includes them) |
linkup returns no title, so title is the one already known for the URL, when there is one. It reports failure only through the HTTP status.
POST /datasets/v3/scrape · one LinkedIn record, the first element of a top-level array
| Normalized | Read from |
|---|---|
content |
the record rendered as markdown: a heading, the key facts, then about, job_summary or post_text |
content_format |
always markdown |
canonical_url |
url |
title |
company name · job job_title · post “user_id on LinkedIn” |
description |
company about (first 300 characters) · job company and location · post text (first 300 characters) |
author / publish_date |
a post’s user_id / date_posted |
structured |
the whole record |
metadata |
record_type (company, job or post), dataset_id, scraped_at, and Bright Data’s own warning when it sent one |
Pin-only, and only for LinkedIn: company and showcase pages (/company/<slug>, /showcase/<slug>), job postings (/jobs/view/<id>) and posts (/posts/<slug>, /feed/update/urn:li:activity:<id>). Every other URL is declined before any call is made. Only a company’s own posts are served — a post by an individual account fails with a provider_error, as do a dead page and a record too thin to be useful. The record is collected from a fixed field allowlist, so the lists that name other people (employees, commenters, tagged people) never reach Telem.
Providers
Section titled “Providers”Eight providers back the default chain; the deployment walks them in order and
the first that answers wins, so a URL one engine cannot render is retried on the
next rather than failing. options.providers pins the chain to a subset — and
a pinned provider never serves from cache. A ninth, brightdata, is never
walked by default: pin it to read LinkedIn company, job and post records. A
pinned chain in which no provider accepts one of your URLs is a 400.
| Provider | Notes |
|---|---|
brightdata |
LinkedIn records, pinned only. Takes a LinkedIn company, job or company-post URL and declines every other; the record comes back rendered as markdown, and whole in structured. |
exa |
Page text, with links and images at max. Declares no output format, so content_format always comes back text whichever format you asked for. |
firecrawl |
The richest document here: a real browser render, carrying HTML, links, images, a summary and a screenshot alongside the markdown. PDFs are billed by the page. |
jina |
A reader that returns the page as clean markdown. The only one that answers with no key at all — a key raises its rate limit rather than unlocking the API. |
linkup |
Page markdown, JavaScript-rendered, with the raw HTML and content type from extended up. Echoes no URL and no title. |
parallel |
Full page markdown with a publish date, a request id and a per-call SKU usage list. |
tavily |
One URL in, one document out: page text with the title, favicon and images. Reports no upstream status — a URL either extracts or comes back failed. |
tinyfish |
Page markdown with the title, description, language, author and publish date, plus links and images from extended up. A page it cannot fetch comes back with tinyfish’s own reason code (bot_blocked, timeout, …). |
you |
Always a fresh crawl: page markdown, the raw HTML and the page’s metadata. |
GET /v1/fetch-providers (no auth) is this chain’s live roster — each
provider’s name, whether it is in the chain, whether it needs a key, and the
fields it structurally cannot supply.
Page content
Section titled “Page content”Three options shape the body itself. content_format picks markdown or plain
text — those two only. inline_content: false returns every field except the
page text: the metadata read, for when you want to know what a page is without
paying to carry it. inline_max_chars caps each inline body at 20 000
characters by default; a longer page comes back truncated and flagged rather
than silently cut.
Whole pages are the point here, not an opt-in as they are on search — every fetch pays for one, so fetch the URLs you have already decided to keep.
Batch query
Section titled “Batch query”Pass a list of URLs and they fetch concurrently inside a single interaction —
faster and cheaper than calling fetch repeatedly. Results come back one per URL,
in the order you sent them, each carrying its own status: one URL failing
never fails the call.
The deployment sets the batch cap; one URL over it is a 400
(over_url_cap) that names the limit.
The Python SDK spells this client.fetch(["https://a", "https://b"]).