Blog · General

API errors and limits: 429, backoff and what to do

A practical guide to a data API's errors: what each HTTP status means, how to handle 429 with Retry-After and backoff, and the code that keeps your bot alive.

Published on October 30, 2026 · 7 min read

The difference between a bot that runs for months on its own and one that dies every week is almost never elegant code: it is error handling. A live data API will return 429 at some point, your connection will wobble, and the server on the other side will restart. This guide covers what each error means, what to do with it, and ends with a Python client that handles them all at once.

The examples use the LynxMetrics API, but the HTTP codes and the patterns apply to any REST API. If "HTTP status" and "header" are still new to you, start with the guide on what an API is and how to connect to one.

The two kinds of error

Separate them from the start, because the handling is the opposite:

  • Your error, which will not fix itself (400, 401, 403, 404): repeating the same request returns the same error. The program should stop and tell you, not keep trying.
  • A temporary error (429, 500, 502, 504, network failures): the same request may work in a few seconds. Here it is worth waiting and retrying.

A loop that repeats a 401 a thousand times a minute is the fastest way to burn your limit without receiving a single piece of data.

The codes, one by one

StatusCodeWhat happenedWhat to do
400bad_requestA parameter in an invalid format, such as a before or after that is neither an ISO 8601 date nor a round idFix the call
401unauthorizedThe key was not sent, is wrong or was revokedCheck the x-api-key header and copy the key again from the dashboard
403key_wrong_apiThe key belongs to another API in the catalogueUse the right API's key; each key opens only one
403trial_expiredThe free trial is overSubscribe to the project, or open a new trial after 24 hours
403subscription_requiredThe project has no active subscriptionReactivate it in the dashboard
402payment_pendingThe payment did not go through (declined card, expired Pix)Pay; the keys come back by themselves within a minute
403plan_limitThe request went beyond what the plan covers, such as a before older than the historyAdjust the request or move up a plan
404not_foundThe resource does not exist, such as an API outside the catalogueCheck the slug
404route_not_foundThe address does not existCheck the endpoint in the documentation
429rate_limitedToo many requests in the same minuteWait the Retry-After and slow down
429too_many_streamsEvery /stream connection is busy at the momentTry again shortly
500internal_errorAn error on our sideRetry with backoff
502upstream_errorThe source did not answerRetry with backoff
504timeoutThe request took too longRetry with backoff

Every error comes with the same body, which keeps the handling short:

{ "error": { "code": "key_wrong_api", "message": "This key belongs to another API." } }

Handle it by the code, never by the message: the text changes language and can be rewritten, the code cannot.

429 and Retry-After

Each plan has a requests-per-minute limit, shared across all of the project's keys. When you go over, the response is 429 and it comes with the headers that tell you exactly where you stand:

HeaderWhat it says
X-RateLimit-LimitHow many requests your plan allows per minute
X-RateLimit-RemainingHow many are left in the current window
X-RateLimit-ResetWhen the window resets
Retry-AfterHow many seconds to wait before the next request

The rule is simple: if Retry-After arrived, obey it. It is not a suggestion, it is the right answer, and sleeping the seconds it asks for is faster than insisting and taking another 429.

if r.status_code == 429:
    time.sleep(int(r.headers.get("Retry-After", "5")))
    continue

And the best way to handle 429 is not to reach it. Three habits cover it:

  • Poll at the data's pace, not at your anxiety's. On a table that runs a round every 34 seconds, polling /latest every 3 seconds is already plenty: that is 20 requests a minute against a limit of 60.
  • Send the ETag. Store the response's ETag and return it in If-None-Match: when nothing has changed, the API answers 304 with no body. The request still counts against the limit, but the traffic and your processing drop a lot.
  • Use /stream when you can. It spends one request on opening and nothing afterwards, as we explain in real-time results.

Exponential backoff, with jitter

For temporary errors without a Retry-After, wait longer each time, with a little randomness. The exponential part avoids hammering a service that is trying to come back; the jitter avoids all your processes returning in the same millisecond.

from random import random

def wait(attempt):
    """1s, 2s, 4s, 8s... with up to 30% variation, capped at 60s."""
    return min(60, 2 ** attempt) * (0.85 + 0.3 * random())

Two rules that go with backoff:

  • Have a ceiling. Waiting 2 hours helps nobody; stop at 30 or 60 seconds and keep trying at that pace.
  • Do not retry what is not retryable. 400, 401, 403 and 404 do not improve with waiting.

Idempotency: the error nobody sees

There is one error that shows up in no status code: handling the same round twice. That happens when a request times out and you repeat it, when your process restarts, or when the stream's event: ready repeats the last round on purpose.

The fix is the same in every case: each round has a fixed id, and that is your latch.

if round_["id"] == last_id:
    return            # already handled this one
last_id = round_["id"]

In the database, the same id as a unique key turns a repeated insert into a silent conflict instead of a duplicate row — the subject of the article on storing real-time data.

The client that handles everything

Putting the pieces together: a definitive error raises, a temporary one waits and retries, and 429 obeys Retry-After.

client.py

import time
from random import random

import requests

BASE = "https://api.lynxmetrics.com/v1"
DEFINITIVE = {400, 401, 402, 403, 404}


class ApiError(Exception):
    pass


def request(path, key, params=None, attempts=6):
    for attempt in range(attempts):
        try:
            r = requests.get(BASE + path, headers={"x-api-key": key}, params=params, timeout=15)
        except requests.RequestException as error:
            print("network:", error, flush=True)
        else:
            if r.status_code == 200:
                return r.json()["data"]
            if r.status_code in DEFINITIVE:
                code = r.json().get("error", {}).get("code", "unknown")
                raise ApiError(code)                          # report and stop: retrying will not help
            if r.status_code == 429:
                time.sleep(int(r.headers.get("Retry-After", "5")))
                continue
            print("temporary:", r.status_code, flush=True)     # 500, 502, 504
        time.sleep(min(60, 2 ** attempt) * (0.85 + 0.3 * random()))
    raise ApiError("no answer after " + str(attempts) + " attempts")

Note that 429 does not go through the backoff: it already arrived with the right wait.

What to log, and what to alert on

A silent bot is a bot you cannot tell is alive. The useful minimum:

  • Log every error with the code, the status and the time. Without that, "the bot stopped" is an investigation; with it, it is one line of log.
  • Alert on the definitive ones (401, 402, 403), because they need action from you and will not pass on their own.
  • Count the temporary ones instead of alerting on each. One isolated 502 is normal life; 502 for ten minutes is a problem.
  • Watch the silence. If no round has arrived in far longer than the table usually takes, something broke, and that alert is worth more than all the others.

Frequently asked questions

What should I do when the API returns 429?

Wait the seconds given in the Retry-After header before trying again, and reduce how often you poll. A 429 means too many requests in the same one-minute window, and the X-RateLimit-Remaining and X-RateLimit-Reset headers show what is left and when the window resets.

What is the difference between a 401 and a 403 from an API?

A 401 is about identity: the key was not sent, is wrong or was revoked. A 403 is about permission: the key exists but cannot access that — for example it belongs to another API, the trial expired, or the request went beyond what the plan covers. Neither improves by repeating the request.

What is exponential backoff with jitter?

It is waiting longer between attempts (1s, 2s, 4s, 8s...) with random variation on top. The exponential part avoids hammering a recovering service, and the jitter avoids several processes returning at the same instant. Use it only for temporary errors, and always with a ceiling.

How do I avoid handling the same round twice?

By comparing the round's id with the last one handled. The id is fixed and never changes, so it works as a latch in the code and as a unique key in the database, where a repeated insert becomes a conflict instead of a duplicate row.

Read also