Writing HTTP API Client Code With LLMs in November 2026: Paste the Real Docs, Handle Pagination and Rate Limits on Purpose, and Test Against Recorded Responses

Almost every project eventually talks to someone else's API: a payment provider, a CRM, a shipping service, an internal team's endpoint. The client code that does this looks simple, and that is the trap. A language model will happily write a fetch call or a requests.get that works on the first try against the happy path. What it will not do on its own is find out how this API paginates, which errors are safe to retry, how it signals rate limits, or what the response looks like when a field is missing.

This guide walks through a practical way to use an LLM to write HTTP API client code that survives contact with production. The core idea: the model is good at typing the boilerplate, but you have to supply the facts, and you have to check the result against real traffic.

Why API client code goes wrong

When you ask a model for "a function that lists all invoices from the Acme API," it has to guess at things it cannot know:

• The exact URL and parameter names. Is it /v1/invoices or /invoices? Is the page size limit, per_page, or page[size]?
• The pagination style. Page numbers, offsets, opaque cursors, or a Link header each need different loop logic.
• Authentication. Bearer token, API key header, basic auth, or a signed request.
• Error shapes. Some APIs return 200 with an error field. Some return 429 with a Retry-After header. Some return 500 for things that are really your fault.
• Field types. Is the amount an integer in cents or a decimal string? Is the timestamp ISO 8601 or Unix seconds?

If the model guesses, the guess often looks plausible and compiles. That is worse than an obvious failure, because the bug shows up later, usually as silently missing data on page two.

Step 1: Paste the real documentation, not the API name

Before you ask for any code, collect the parts of the provider's docs that matter and paste them into the conversation. You do not need the whole reference. You need:

1. The endpoint reference for every call you will make (method, path, parameters, request body).
2. The authentication section.
3. The pagination section.
4. The errors and rate limits section.
5. One or two real example responses. If the docs have none, make a real call with curl and paste the output, with tokens and personal data removed.

If the provider publishes an OpenAPI spec, paste the relevant paths and schemas from it. Then tell the model plainly: "Use only the endpoints, parameters, and fields shown here. If something you need is not in this text, stop and ask instead of guessing."

Also ask yourself whether you should write a client at all. If the provider ships an official SDK that is maintained and fits your language, using it is often less work than owning your own pagination and retry logic. A hand-written client makes sense when there is no SDK, the SDK is abandoned, or you only need two or three calls.

Every connection follows the exchange's rules. Paste the API's real rules before the model writes a single call. Image: Wisconsin Bell Telephone Company Switchboard by Gary Langebartels / FortepanIowa, CC BY-SA 4.0, via Wikimedia Commons.

Step 2: Describe the shape of the client before the code

Ask the model for a short plan first: which functions exist, what each one returns, and where the shared logic lives. A good small client usually has:

• One place that builds requests: base URL, auth header, timeouts, a User-Agent that identifies your app.
• One place that turns HTTP responses into results or typed errors.
• A pagination helper that yields items, so callers never write their own page loops.
• Thin functions per endpoint, such as list_invoices() or get_customer(id).

Reviewing a ten-line plan is much faster than reviewing two hundred lines of code, and it is where you catch "wait, this API uses cursors, not pages."

Step 3: Always set a timeout

Many HTTP libraries have no timeout by default. Python's requests library, for example, will wait indefinitely unless you pass timeout. A model-written client that forgets this works fine until the day the provider hangs, and then a worker thread sits stuck forever. Make it a hard requirement in your prompt: every request has a connect timeout and a read timeout, and the values come from configuration.

Step 4: Handle pagination on purpose

Pagination is the most common place for silent data loss. Tell the model exactly which style the API uses, and ask for a generator or iterator so the loop exists in one place. Here is a sketch for a cursor-based API in Python:

def iter_invoices(session, base_url, page_size=100):
    params = {"limit": page_size}
    while True:
        resp = session.get(f"{base_url}/v1/invoices",
                           params=params, timeout=(5, 30))
        data = check(resp)          # raises on errors, returns parsed JSON
        for item in data["data"]:
            yield item
        if not data.get("has_more"):
            break
        params["starting_after"] = data["data"][-1]["id"]

The field names in that sketch (has_more, starting_after) are examples; your API will have its own, which is why you pasted the docs. Whatever the style, check these cases explicitly:

• An empty first page.
• Exactly one full page, where the "is there more?" signal must say no.
• The last page being shorter than the page size.
• Offset pagination over data that changes while you read it. Items can be skipped or repeated; if that matters, prefer cursors or sort by a stable key.

Step 5: Decide which errors are retryable

Ask the model to sort errors into categories instead of wrapping everything in a generic retry. A reasonable default:

• Retry with backoff: connection errors, timeouts, 429, and 502/503/504.
• Do not retry: 400, 401, 403, 404, 422. Retrying a bad request just sends the same bad request again.
• Retry only if the request is safe to repeat: 500 on a POST that creates something. If the API supports an idempotency key header, send one; otherwise a retry can create a duplicate.

For 429 and 503, honor a Retry-After header when the API sends one, cap the total number of attempts, and add random jitter so many workers do not retry at the same instant. Make sure the final error the caller sees includes the status code, the endpoint, and the provider's request ID if one is returned. That ID is what the provider's support team will ask for.

Sending a request is the easy part. Deciding what to do when nothing comes back is the real work. Image: Pneumatic tube system 160 by Winfried Oberstadt, CC0, via Wikimedia Commons.

Step 6: Validate the response, do not just index into it

Model-written code tends to assume the response always matches the example: data["customer"]["email"]. Real responses have null fields, missing optional fields, and new fields added later. Ask for:

• Parsing into a typed structure (a dataclass, Pydantic model, TypeScript type with a runtime validator, or similar) at the edge of the client.
• Clear handling for optional fields, instead of a KeyError deep in business logic.
• Unknown extra fields ignored rather than rejected, so a provider adding a field does not break you.
• Money and dates converted once, in the client, using the units the docs specify.

Step 7: Keep secrets out of code and logs

Read the API key from the environment or your secret store, never from a string literal the model helpfully filled in. Then check two less obvious places: debug logging that prints full request headers, and exception messages that include the full URL when the key is passed as a query parameter. Ask the model to redact the Authorization header and any key-like query parameters anywhere it logs a request.

Step 8: Test against recorded responses

Do not let the tests the model writes mock its own assumptions. A test that mocks the response the model imagined proves nothing. Instead:

1. Make real calls against a sandbox or test account once, and save the responses to fixture files (scrub secrets and personal data).
2. Use a recording tool or a simple HTTP mocking library to replay those fixtures in tests. In Python, responses or vcrpy are common choices; in JavaScript, nock or MSW.
3. Write tests for the edge cases from steps 4 and 5: an empty list, the last page, a 429 with Retry-After, a 404, a timeout, a response with a null optional field.
4. Assert on behavior: how many requests were made, which ones were retried, and what the caller received.

Then run the client once more against the real sandbox before shipping. Recorded fixtures go stale when the provider changes something, so re-record them occasionally, especially when the provider announces a new API version.

A prompt template you can reuse

I need a small HTTP client in [language, version] using [library, version].
Below are the relevant docs: endpoints, auth, pagination, errors, rate limits,
and real example responses. Use only what is shown. If something is missing, ask.

Requirements:
- Every request has connect and read timeouts from config.
- Auth key comes from the environment; never log it.
- Pagination lives in one iterator; callers never loop pages themselves.
- Retry only timeouts, connection errors, 429, 502, 503, 504, with capped
  exponential backoff plus jitter, honoring Retry-After.
- Never retry POST unless an idempotency key is sent.
- Parse responses into typed models; ignore unknown fields.
- Errors raised to callers include status, endpoint, and provider request ID.

First give me a short plan of functions and types. Then the code.
Then a list of edge cases I should test with recorded responses.

[paste docs and sample responses here]

A short review checklist

• Does every endpoint, parameter, and field name appear in the docs you pasted?
• Does every request have a timeout?
• Can pagination stop correctly on an empty page and on an exactly full last page?
• Are non-idempotent writes protected from duplicate retries?
• Is there a cap on retries and total wait time?
• Are secrets absent from code, logs, and exception messages?
• Do the tests replay real recorded responses, including failures?

The model can save you a lot of typing on API client code. The parts that keep it correct, the real docs, the deliberate choices about pagination and retries, and tests built from real traffic, still come from you.