Skip to content

Parameters

Fetch parameters

Every option on a Telem page fetch: tiers, the provider chain, the page content settings, batching, and the raw field behind each normalized field.

This page tells you what each fetch option does, and where each field in the response comes from. You set all fetch options in the request’s options block. The block itself and a one-line summary of each option are on Parameters.

Fetch tiers are cumulative, as search tiers are. On every result, the page body is in the key content. To choose one, see Fetch best practices.

Tier Adds
minimalist content, content_format, canonical_url, title, http_status, warning, provider, cache_state, request_id, latency_ms
default js_rendered, pdf_mode, content_length, content_sha256, description
extended raw_html, html, chunks, screenshot, page_count, content_type, summary, language, author, publish_date, favicon, thumbnail, links, images, metadata, provider_chain, usage
max media, transcript, excerpt, structured, subpages, enrichments

In the Python SDK, pass tier to fetch() (client.fetch(urls, tier="extended")), or set TELEM_FETCH_TIER.

Every provider gives the page body and the failure signal. Every provider also gives title, except these four:

  • linkup
  • socialdatax_douyin
  • socialdatax_kuaishou
  • socialdatax_weibo

Everything else is best-effort. A capability_gap warning names the null fields that the provider cannot supply. A paywall, a login wall or a page that needs JavaScript first makes Telem try the next provider. When no provider gets the page, the URL usually comes back failed, with the reason, and the rest of the request still succeeds. A page that is only partly readable can come back as succeeded, with a partial_content warning. On a pinned chain, when every URL fails at the provider, the call returns a 502. A site’s own error page can also come back as succeeded, so check http_status and warnings.

One vocabulary, whoever answered. The chain fails over, so a different provider can answer the same fetch tomorrow. The tabs below show the raw field behind each normalized field, for each provider.

Five fields never come from a provider, so the tabs leave them out:

  • provider records who answered.
  • provider_chain records who was tried.
  • content_length, content_sha256 and latency_ms are measured by Telem over the body it received.

Two more fields work the same way for every provider:

  • enrichments keeps, verbatim, every provider key that normalization did not use.
  • title is the one field with a fallback. When the answering provider supplies no title, Telem uses the title it already knows for that URL.

POST api.fetch.tinyfish.ai · document at results[0]

Normalized Read from
content text
content_format format. It is markdown when you asked for text, which tinyfish does not offer
canonical_url / title final_url / title
description description
language / author language / author
publish_date published_date
links / images links / image_links (requested only when the tier includes them)

A page that tinyfish cannot fetch still answers 200. The result list is empty, and the reason is in errors[] as a closed-set code (bot_blocked, timeout, page_not_found, …). That code comes back in a provider_error warning, with the upstream status when tinyfish sent one. Then the chain moves on. etag, last_modified and not_modified go in enrichments.

Eight providers make up the default chain. Telem tries them one after another. It moves on when a provider fails, or returns a blocked or empty page. The order depends on the URL: for some kinds of page, such as PDFs or Reddit posts, Telem first tries the providers that do best on them, then the rest of the chain.

options.providers pins the chain to a subset, in your order. A pinned chain is never reordered, and no other provider is tried. A pinned provider can serve a saved copy of the page, but a page that it fetches is not saved. If no provider in a pinned chain accepts one of your URLs, you get a 400.

Social, video, review and shopping platforms have their own fetch providers, and a ninth provider, brightdata, reads LinkedIn company, job and post records. None of them is in the default chain, but video and post links on many of these platforms go to their provider first, with no pin. They are listed on Supported platforms.

The response names your pin in pinned_providers: the options.providers list, in lower case and without duplicates. It is null when you did not pin a chain.

Provider Notes
brightdata LinkedIn records. Not in the default chain, but LinkedIn company, job and post links go to it first. Takes a LinkedIn company, job or company-post URL and declines every other URL. The record comes back rendered as markdown, and whole in structured.
exa Page text, with links and images at max. Declares no output format, so content_format always comes back text whichever format you asked for.
firecrawl The richest document here: a real browser render, with HTML, links, images, a summary and a screenshot next to the markdown. PDFs are billed by the page.
jina A reader that returns the page as clean markdown. The only one that answers with no key at all. A key only raises its rate limit.
linkup Page markdown, JavaScript-rendered, with the raw HTML and content type from extended up. Echoes no URL and no title.
parallel Full page markdown with a publish date, a request id and a per-call SKU usage list.
tavily One URL in, one document out: page text with the title, favicon and images. Reports no upstream status. A URL either extracts or comes back failed.
tinyfish Page markdown with the title, description, language, author and publish date, plus links and images from extended up. A page it cannot fetch comes back with tinyfish’s own reason code (bot_blocked, timeout, …).
you Always a fresh crawl: page markdown, the raw HTML and the page’s metadata.

GET /v1/fetch-providers (no auth) is the live list of fetch providers. For each provider, it shows:

  • its name
  • whether it is in the chain
  • whether it needs a key
  • the fields that it can never supply

Three options shape the body itself:

  • content_format picks markdown or plain text. Those are the only two formats.
  • inline_content: false returns every field except the page text. Use this metadata read when you want to know what a page is, without its text in the response.
  • inline_max_chars caps each inline body at 20 000 characters by default. A longer page comes back truncated, with a flag.

A fetch is for whole pages. On search, whole pages are an opt-in. Every fetch pays for a whole page, so fetch the URLs that you already decided to keep.

Pass a list of URLs, and Telem fetches them concurrently in a single interaction. This is faster and cheaper than calling fetch many times. You get one result per URL, in the order you sent them. Each result has its own status. If one URL fails, the call does not fail.

A call takes up to 5 URLs. One more returns a 400 (over_url_cap) that names the limit.

In the Python SDK, this is client.fetch(["https://a", "https://b"]).

Was this page helpful?