API errors and limits: 429, backoff and what to do
A practical guide to a data API's errors: what each HTTP status means, how to handle 429 with Retry-After and backoff, and the code that keeps your bot alive.
Published on October 30, 2026 · 7 min read
The difference between a bot that runs for months on its own and one that dies every week is almost never elegant code: it is error handling. A live data API will return 429 at some point, your connection will wobble, and the server on the other side will restart. This guide covers what each error means, what to do with it, and ends with a Python client that handles them all at once.
The examples use the LynxMetrics API, but the HTTP codes and the patterns apply to any REST API. If "HTTP status" and "header" are still new to you, start with the guide on what an API is and how to connect to one.
The two kinds of error
Separate them from the start, because the handling is the opposite:
- Your error, which will not fix itself (400, 401, 403, 404): repeating the same request returns the same error. The program should stop and tell you, not keep trying.
- A temporary error (429, 500, 502, 504, network failures): the same request may work in a few seconds. Here it is worth waiting and retrying.
A loop that repeats a 401 a thousand times a minute is the fastest way to burn your limit without receiving a single piece of data.
The codes, one by one
| Status | Code | What happened | What to do |
|---|---|---|---|
| 400 | bad_request | A parameter in an invalid format, such as a before or after that is neither an ISO 8601 date nor a round id | Fix the call |
| 401 | unauthorized | The key was not sent, is wrong or was revoked | Check the x-api-key header and copy the key again from the dashboard |
| 403 | key_wrong_api | The key belongs to another API in the catalogue | Use the right API's key; each key opens only one |
| 403 | trial_expired | The free trial is over | Subscribe to the project, or open a new trial after 24 hours |
| 403 | subscription_required | The project has no active subscription | Reactivate it in the dashboard |
| 402 | payment_pending | The payment did not go through (declined card, expired Pix) | Pay; the keys come back by themselves within a minute |
| 403 | plan_limit | The request went beyond what the plan covers, such as a before older than the history | Adjust the request or move up a plan |
| 404 | not_found | The resource does not exist, such as an API outside the catalogue | Check the slug |
| 404 | route_not_found | The address does not exist | Check the endpoint in the documentation |
| 429 | rate_limited | Too many requests in the same minute | Wait the Retry-After and slow down |
| 429 | too_many_streams | Every /stream connection is busy at the moment | Try again shortly |
| 500 | internal_error | An error on our side | Retry with backoff |
| 502 | upstream_error | The source did not answer | Retry with backoff |
| 504 | timeout | The request took too long | Retry with backoff |
Every error comes with the same body, which keeps the handling short:
{ "error": { "code": "key_wrong_api", "message": "This key belongs to another API." } }Handle it by the code, never by the message: the text changes language and can be rewritten, the code cannot.
429 and Retry-After
Each plan has a requests-per-minute limit, shared across all of the project's keys. When you go over, the response is 429 and it comes with the headers that tell you exactly where you stand:
| Header | What it says |
|---|---|
X-RateLimit-Limit | How many requests your plan allows per minute |
X-RateLimit-Remaining | How many are left in the current window |
X-RateLimit-Reset | When the window resets |
Retry-After | How many seconds to wait before the next request |
The rule is simple: if Retry-After arrived, obey it. It is not a suggestion, it is the right answer, and sleeping the seconds it asks for is faster than insisting and taking another 429.
if r.status_code == 429:
time.sleep(int(r.headers.get("Retry-After", "5")))
continueAnd the best way to handle 429 is not to reach it. Three habits cover it:
- Poll at the data's pace, not at your anxiety's. On a table that runs a round every 34 seconds, polling
/latestevery 3 seconds is already plenty: that is 20 requests a minute against a limit of 60. - Send the
ETag. Store the response'sETagand return it inIf-None-Match: when nothing has changed, the API answers 304 with no body. The request still counts against the limit, but the traffic and your processing drop a lot. - Use
/streamwhen you can. It spends one request on opening and nothing afterwards, as we explain in real-time results.
Exponential backoff, with jitter
For temporary errors without a Retry-After, wait longer each time, with a little randomness. The exponential part avoids hammering a service that is trying to come back; the jitter avoids all your processes returning in the same millisecond.
from random import random
def wait(attempt):
"""1s, 2s, 4s, 8s... with up to 30% variation, capped at 60s."""
return min(60, 2 ** attempt) * (0.85 + 0.3 * random())Two rules that go with backoff:
- Have a ceiling. Waiting 2 hours helps nobody; stop at 30 or 60 seconds and keep trying at that pace.
- Do not retry what is not retryable. 400, 401, 403 and 404 do not improve with waiting.
Idempotency: the error nobody sees
There is one error that shows up in no status code: handling the same round twice. That happens when a request times out and you repeat it, when your process restarts, or when the stream's event: ready repeats the last round on purpose.
The fix is the same in every case: each round has a fixed id, and that is your latch.
if round_["id"] == last_id:
return # already handled this one
last_id = round_["id"]In the database, the same id as a unique key turns a repeated insert into a silent conflict instead of a duplicate row — the subject of the article on storing real-time data.
The client that handles everything
Putting the pieces together: a definitive error raises, a temporary one waits and retries, and 429 obeys Retry-After.
client.py
import time
from random import random
import requests
BASE = "https://api.lynxmetrics.com/v1"
DEFINITIVE = {400, 401, 402, 403, 404}
class ApiError(Exception):
pass
def request(path, key, params=None, attempts=6):
for attempt in range(attempts):
try:
r = requests.get(BASE + path, headers={"x-api-key": key}, params=params, timeout=15)
except requests.RequestException as error:
print("network:", error, flush=True)
else:
if r.status_code == 200:
return r.json()["data"]
if r.status_code in DEFINITIVE:
code = r.json().get("error", {}).get("code", "unknown")
raise ApiError(code) # report and stop: retrying will not help
if r.status_code == 429:
time.sleep(int(r.headers.get("Retry-After", "5")))
continue
print("temporary:", r.status_code, flush=True) # 500, 502, 504
time.sleep(min(60, 2 ** attempt) * (0.85 + 0.3 * random()))
raise ApiError("no answer after " + str(attempts) + " attempts")Note that 429 does not go through the backoff: it already arrived with the right wait.
What to log, and what to alert on
A silent bot is a bot you cannot tell is alive. The useful minimum:
- Log every error with the
code, the status and the time. Without that, "the bot stopped" is an investigation; with it, it is one line of log. - Alert on the definitive ones (401, 402, 403), because they need action from you and will not pass on their own.
- Count the temporary ones instead of alerting on each. One isolated 502 is normal life; 502 for ten minutes is a problem.
- Watch the silence. If no round has arrived in far longer than the table usually takes, something broke, and that alert is worth more than all the others.
Frequently asked questions
- What should I do when the API returns 429?
Wait the seconds given in the Retry-After header before trying again, and reduce how often you poll. A 429 means too many requests in the same one-minute window, and the X-RateLimit-Remaining and X-RateLimit-Reset headers show what is left and when the window resets.
- What is the difference between a 401 and a 403 from an API?
A 401 is about identity: the key was not sent, is wrong or was revoked. A 403 is about permission: the key exists but cannot access that — for example it belongs to another API, the trial expired, or the request went beyond what the plan covers. Neither improves by repeating the request.
- What is exponential backoff with jitter?
It is waiting longer between attempts (1s, 2s, 4s, 8s...) with random variation on top. The exponential part avoids hammering a recovering service, and the jitter avoids several processes returning at the same instant. Use it only for temporary errors, and always with a ceiling.
- How do I avoid handling the same round twice?
By comparing the round's id with the last one handled. The id is fixed and never changes, so it works as a latch in the code and as a unique key in the database, where a repeated insert becomes a conflict instead of a duplicate row.
Read also
- What an API is and how to connect to one for the first time
APIs explained without the jargon: what they are, how REST, JSON and keys work, and how to make your first call in the browser, the terminal, Python and JavaScript.
- Storing real-time data: from JSON to database, no gaps
How to store what arrives from a live API: a unique key, upsert, time zones, gap detection, and the backfill that closes the window while your program was down.
- How we read a live table: from video to JSON
The path a result takes from the table to the API: video segments, a neural network, the check against the game's rules, and why we discard rather than guess.