How to Use an LLM API: A Beginner's Walkthrough

Chat apps are great for trying AI. But the moment you want AI inside your own product, spreadsheet, help desk or internal tool, you need the API. That’s where many beginners stall: the docs assume you already know what a token is, why your request failed with a 429, and why your bill doubled overnight.

This walkthrough covers the path from your first API call to a setup you can trust in production. The examples use the patterns shared by the major providers, so the concepts carry over whether you pick OpenAI, Anthropic, Google or an open-model host.

What an LLM API is and how it works

An LLM API lets your code send text (and often images or files) to a large language model over the internet and get a response back. If APIs in general are new to you, start with our explainer on what an API is.

Every request has the same basic shape:

• An endpoint: the URL you send the request to.
• Authentication: your secret API key, sent in a request header.
• A model name: which model should answer.
• Messages: the conversation so far, usually with roles such as system or developer (your instructions), user (the input) and assistant (earlier model replies).
• Parameters: settings such as the maximum length of the reply.

The response contains the generated text plus metadata, including how many input and output tokens you used. Tokens are what you pay for, so it’s worth reading our guide to AI tokens and context windows before you build anything large.

One more thing that surprises beginners: the API is stateless. The model does not remember your last request. If you’re building a chat, your code has to send the earlier messages again each time.

Steps 1 to 3: pick a model, get a key and make your first call

Step 1: Choose a provider and model

Your choice depends on quality, price, speed, data handling terms and where you already run your infrastructure. Our comparison of ChatGPT vs Claude vs Gemini covers the consumer side; for open-weight options, weigh the trade-offs between open and closed models.

Most providers offer a family of models: a large, more capable and more expensive model, and smaller, faster and cheaper ones. Start testing with a mid-tier model, then try the smaller one once your prompts work.

Step 2: Get an API key and keep it secret

Sign up in the provider’s developer console, add billing, and create an API key. Then treat it like a password.

• Store it in an environment variable, not in your code. The official SDKs read standard variables such as OPENAI_API_KEY or ANTHROPIC_API_KEY automatically.
• Never put the key in browser or mobile app code. OpenAI’s key safety guidance warns that a key exposed client-side lets anyone make requests on your account. Route requests through your own backend instead.
• Never commit it to a code repository, even a private one. Use a secrets manager in production.
• Use separate keys for development and production, and rotate them on a schedule.
• Set a spend limit in the console so a bug can’t run up an unlimited bill.

Step 3: Make your first call

Install the provider’s SDK (for example, pip install openai or pip install anthropic for Python) and send a short request. Here’s the shape of a minimal call with Anthropic’s Python SDK:

```python import anthropic

client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY

message = client.messages.create( model=”MODEL_NAME_FROM_DOCS”, max_tokens=500, system=”You are a concise assistant for a plumbing company.”, messages=[{“role”: “user”, “content”: “Write a 2-sentence reminder for a Tuesday appointment.”}], ) print(message.content[0].text) ```

OpenAI’s quickstart uses the same idea with its Responses API: create a client, call client.responses.create() with a model and an input, and print the output text. Model names change often, so copy the current one from your provider’s model list rather than from a tutorial.

If the call works, you’ll see the reply printed. If not, the error message will usually tell you what’s wrong: a missing key, a typo in the model name or a malformed request.

Step 4: Learn the parameters that matter

You don’t need every setting. These are the ones beginners use most:

• Max tokens / max output tokens. The cap on reply length. Anthropic’s Messages API requires max_tokens. OpenAI notes that your rate limit usage is based on the larger of this value and your estimated input, so set it close to what you actually need.
• System or developer instructions. The rules, tone and role for the model. Keep them stable and put changing input in the user message.
• Temperature. On models that support it, lower values make output more predictable and higher values more varied. Reasoning models may ignore it or use other controls instead.
• Stop sequences. Text that tells the model to stop generating.
• Structured output. Many APIs can return JSON that matches a schema you define. Use this whenever your code needs to parse the response.
• Tools. You can describe functions the model may ask your code to run, such as looking up an order. This is the foundation of agents and of standards like Model Context Protocol.

Step 5: Add streaming for a better user experience

Without streaming, your user stares at a spinner until the full reply is ready. With streaming, text appears as it’s generated. Providers deliver streams using server-sent events (SSE), and the SDKs wrap this in simple loops.

Use streaming for chat interfaces and any long response. Anthropic also recommends streaming or its batch API for long-running requests, because some networks drop idle connections. One gotcha from Anthropic’s docs: with streaming, an error can arrive mid-stream after the request has already returned a 200 status, so your code needs to handle error events inside the stream too.

Key takeaway: A first API call takes minutes. What makes it production-ready is the boring part: secure keys, retries, streaming, logging and spend limits. Build those in before real users arrive.

Step 6: Handle errors and rate limits

Every provider limits how much you can send. OpenAI measures limits in requests per minute and per day, and tokens per minute and per day, and you hit whichever comes first. Limits usually rise as your account moves up usage tiers.

Common errors and what to do:

Best practices from the providers’ docs:

1. Retry with exponential backoff and jitter. Wait a little, then longer, with a random delay so all your workers don’t retry at once. If a retry-after header is present, respect it.
2. Cap retries by number and total time.
3. Don’t retry errors that need action, such as billing or quota problems.
4. Log request IDs. Anthropic returns a request-id header on every response; include it when you contact support.

The official SDKs already retry some transient errors for you. Anthropic’s SDK, for example, retries twice by default with exponential backoff.

Step 7: Control costs from day one

LLM APIs charge per token, with input and output priced separately and output usually costing more. On Anthropic’s published price list, for example, output tokens cost 5 times the input price for its current models. A few habits keep bills predictable:

• Log tokens per request from the usage data in each response, and track cost per feature or customer.
• Set hard spend limits and budget alerts in the console.
• Trim prompts. Don’t resend huge documents or long chat histories when a summary will do.
• Use smaller models for simple tasks like classification and extraction.
• Use batch processing for work that isn’t urgent. OpenAI and Anthropic both offer a 50% discount through their batch APIs.
• Use prompt caching when many requests share the same long instructions or documents.

Our guide on how to reduce LLM API costs goes deeper on each technique.

Where to start: a production checklist

Before you put an LLM feature in front of customers, check off these items:

1. API key stored in a secrets manager and only used server-side
2. Spend limit and budget alerts set in the provider console
3. Streaming enabled for user-facing responses
4. Retries with exponential backoff and a sensible timeout
5. Logging of request IDs, token counts, latency and errors
6. Input limits so users can’t send huge prompts
7. A small test set to check quality whenever you change the prompt or model (see how to evaluate LLM outputs)
8. A fallback message for when the API is down
9. A review of the provider’s data use and retention terms
10. A plan for adding your own data, for example with retrieval-augmented generation

Conclusion

Using an LLM API is easier than it looks: a key, a model name, some messages and a few parameters. The real work is making it reliable, secure and affordable. Start with a small script, get one useful result, then add streaming, retries, logging and cost controls before you scale. If this is the start of a career shift, our roadmap on how to become an AI engineer shows what to learn next.

Sources

• OpenAI API: Developer quickstart — SDK install, OPENAI_API_KEY environment variable and first Responses API call
• OpenAI API: Rate limits — RPM/TPM limits, usage tiers, exponential backoff and max_tokens counting toward limits
• OpenAI Help Center: Best practices for API key safety — keeping keys out of client-side code and repositories
• Claude API: Errors — HTTP error codes, SDK retries, request IDs and mid-stream errors
• Claude API: Pricing — per-token pricing, input vs output prices, 50% batch discount and prompt caching
• OpenAI Help Center: Is API usage included in ChatGPT subscriptions? — API billing is separate from ChatGPT plans