Production checklist
What to have in place before you point real traffic at this.
Before you go live
Set your client timeout to at least 120 seconds. The default in most HTTP libraries is far shorter than a slow call, and a timeout on your side still costs you the call.
Store request_id for every call, successful or not. It is the only identifier
that lets us find a specific request afterwards.
Cap your retries at three attempts with exponential backoff, and only retry when
status.retryable is true. A retry loop with no ceiling turns one malformed
request into thousands of billed failures.
Make columns for metrics nullable. See nulls and zeros for why this matters more than it looks.
Key on post_id when collecting from a feed, so a post appearing on two pages
is stored once.
Once you are running
Watch the ratio of success: false responses rather than raw error counts. A
rising ratio usually means a platform changed something, and it shows up in that
ratio before it shows up anywhere else.
Alert on p95 latency, not on averages. These endpoints have a long tail by nature, and an average hides the tail entirely.
Record meta.billable alongside your own request counts. It is the honest
number for reconciling usage, and it will not match your total request count
because failed calls are free.
When something looks wrong
Check the endpoint's page first. Several endpoints have documented platform limits that look like bugs from the outside, such as a surface that a platform only shows to signed-in visitors.
Send one call by hand with the example from the page. If the example works and your call does not, the difference is in your request. If neither works, the problem is on our side and worth reporting.
Include the request_id when you report it, plus the endpoint name and roughly
when it happened. That is enough for us to find the request.
Things that will bite you
Retrying REQUEST_FAILED forever. It is marked retryable, but it is also what a
malformed request returns. See errors and retries.
Treating an empty page as the end of a feed. Use has_more.
Persisting an X search cursor. Those expire in about thirty minutes.
Requesting a large count and assuming you got it. Page sizes are capped and
oversized values are quietly reduced.
Turning on optional enrichment flags across a whole feed. They are per-item costs and they add up quickly.