Social Data API high-volume integration

The default rate limits are 60 requests a minute per route group, and 60 URLs a minute in batches, for each client IP. Read this before you plan to go past them.

Ramp up in steps

Build against staging, then switch the base URL and the token together; see Environments and authentication.
Start production with a small share of your traffic, inside the default limits.
Before each step up, ask us to raise the limits. Book a call (opens in a new tab) or write on the channel you got your credentials from, with the requests per minute you expect at peak on each route and the IP addresses you call from. Wait for us to confirm before you send more.
After each step, check for 429s in your logs and for errors in GET /auth/usage.

To work out what to ask for, divide the posts you track by the minutes between reads: 3,000 posts read every 30 minutes is 100 reads a minute.

Retry the right errors

Retry: 429, 500, 502, 503 and 504, connection errors, client timeouts, and BATCH_TIMEOUT on a batch result.
Do not retry: 401, 403, 404 and 422. They fail the same way again.

Errors has every code. A request answered with an error is not billed.

read() below retries by HTTP status, waiting for Retry-After on a 429 or any other response that carries it, and otherwise backing off exponentially with jitter. A BATCH_TIMEOUT arrives inside a batch's 200, so read() never sees it; the batch snippet under Use batches picks those URLs out:

const RETRYABLE = new Set([429, 500, 502, 503, 504]);

async function read(url: string, init: RequestInit = {}, attempts = 6): Promise<Response> {
  for (let attempt = 1; ; attempt++) {
    let res: Response | undefined;
    try {
      res = await fetch(url, {
        ...init,
        headers: { Authorization: `Bearer ${API_TOKEN}`, ...init.headers },
        signal: AbortSignal.timeout(120_000),
      });
      if (!RETRYABLE.has(res.status)) return res;
    } catch (err) {
      // A connection error or a client timeout.
      if (attempt === attempts) throw err;
    }
    if (res && attempt === attempts) return res;

    const backoff = Math.min(1000 * 2 ** attempt, 300_000);
    const retryAfter = Number(res?.headers.get("Retry-After")) * 1000;
    const wait = retryAfter > 0 ? retryAfter : backoff + Math.random() * backoff;
    await new Promise((resolve) => setTimeout(resolve, wait));
  }
}

Use batches

POST /v1/stats/batch takes up to 25 URLs in one call, which saves round trips. It does not save rate limit: every URL counts against the 60 batch URLs a minute, and every batch is also one request in the stats group. Ask for both when you ask for more.

A batch reads up to 8 URLs at a time and answers when the slowest is done, within 100 seconds. Read a post on its own when you need it quickly.
A failed URL does not fail the batch, so check ok on every result.
Send only the URLs that failed with a retryable code, not the whole batch: URLs that succeed are billed each time. Wait for Retry-After first if the 200 carries it.

const RETRYABLE_CODES = new Set([
  "SCRAPING_RATE_LIMITED",
  "SCRAPING_ERROR",
  "SCRAPING_UNAVAILABLE",
  "UPSTREAM_TIMEOUT",
  "INTERNAL_ERROR",
  "BATCH_TIMEOUT",
]);

const urls = ["https://www.instagram.com/reel/ABC123/", "https://www.tiktok.com/@creator/video/7300000000000000000"];
const res = await read(`${API_BASE_URL}/v1/stats/batch`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ urls }),
});

type BatchResult = { url: string; ok: boolean; error?: { code: string } };
const { results }: { results: BatchResult[] } = await res.json();
const retry = results.filter((r) => !r.ok && RETRYABLE_CODES.has(r.error?.code ?? "")).map((r) => r.url);

Cache on your side

Counts may be up to 15 minutes old, so reading a post more often returns the same numbers and is still billed.

Store each result with the time you read it, keyed on the platform plus the post's shortcode or id, the last segment of the path. Send one URL form per post: postUrl keeps /p/ or /reel/ as sent, so the two forms of one Instagram post are read and billed separately.
Skip any post you read less than 15 minutes ago.
Spread reads evenly across your interval rather than starting them all at once. Limits are counted per minute.
Stop re-reading a post on every cycle once it returns a permanent error. Check it rarely instead.

Allow for slow reads

Most reads are quick, but some take many seconds; see Response times. Use the 120-second client timeout, and run reads concurrently so that a slow one does not hold up the rest.

The API is hosted in Europe, so calls from other regions add network time to every request. Reuse connections so you do not pay for a new one on every call.

Quote the request id

Log the X-Request-Id of every response next to its result, or send your own. Quote it when you contact us about a request; see Get help with a request.

High-volume integration