Environment variables and secrets management for AI apps

GuideLast reviewed Oct 16, 2026By Himanshu Ramchandani

Short answer

Never write API keys in your code. Keep them in a .env file on your machine that Git ignores, load them as environment variables, and in production set them in your platform's secret settings or a secret manager. Use separate keys for development and production, give each key only the permissions it needs, set spending limits with your model provider, and if a key leaks, revoke it first and clean up after.

Why this matters more for AI apps

An LLM API key is a credit card with no PIN. Anyone who has it can make calls on your account, and every call costs money. If a key leaks to a public repository, automated scanners can find it within minutes, and someone can use it to run thousands of requests, so the first sign is often a bill or a suspended account.

AI applications also tend to have more keys than ordinary ones: a model provider, an embedding provider, a vector database, a tracing tool, a search API, a cloud account, sometimes several of each. Every one of them is a way in.

What counts as a secret

API keys for model providers, search tools and other services.
Database passwords and connection strings, including the URL and key of your vector database.
Cloud access keys, like an AWS access key ID and secret access key.
Tokens for GitHub, Hugging Face and similar platforms.
Signing keys for user sessions and webhooks.
Private SSH keys.

Configuration that isn't secret, like the model name, the temperature or the log level, can also live in environment variables, which makes it easy to change between environments without changing the code.

Environment variables

An environment variable is a value that lives outside your program, in the environment it runs in, and your code reads it at runtime. The same code can then run on your laptop with your development key and on the server with the production key, without changing a single line.

import os

api_key = os.environ.get("GEMINI_API_KEY")
if not api_key:
    raise ValueError("GEMINI_API_KEY is not set")

Fail at startup when a required variable is missing, with a clear message. Otherwise the app starts, and the first user who triggers a model call gets a confusing error instead.

.env file for local development

Typing export commands every time you open a terminal gets old quickly. A .env file keeps your local variables in one place, and the python-dotenv package loads them into the environment.

GEMINI_API_KEY=your-key-here
MODEL_NAME=gemini-2.5-flash-lite
QDRANT_URL=http://localhost:6333

import os

from dotenv import load_dotenv

load_dotenv()

model = os.environ.get("MODEL_NAME", "gemini-2.5-flash-lite")

Three files work together:

.env holds your real values and is listed in .gitignore, so it never reaches GitHub.
.env.example lists every variable the project needs, with empty or fake values, and is committed, so anyone who clones the repo knows what to set.
.gitignore and .dockerignore both list .env, so it stays out of the repository and out of the Docker image.

GEMINI_API_KEY=
MODEL_NAME=gemini-2.5-flash-lite
QDRANT_URL=http://localhost:6333

Typed settings with pydantic-settings

As the number of variables grows, reading them one by one with os.environ gets messy. pydantic-settings reads environment variables and the .env file into one typed object, validates them at startup, and keeps secrets from being printed by accident.

from pydantic import SecretStr
from pydantic_settings import BaseSettings, SettingsConfigDict


class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_file=".env")

    gemini_api_key: SecretStr
    model_name: str = "gemini-2.5-flash-lite"
    request_timeout_s: float = 30.0


settings = Settings()
print(settings.gemini_api_key)
print(settings.gemini_api_key.get_secret_value()[:4])

SecretStr prints as **********, so the key doesn't end up in your logs or error messages, and you only get the real value when you ask for it explicitly.

Secrets in production

On a server, there is no .env file from your laptop. You set the secrets where the application runs:

On a managed platform like Render, Railway, Cloud Run or Hugging Face Spaces, use the environment variables or secrets section of the service settings.
On AWS, Google Cloud and Azure, use the provider's secret manager (AWS Secrets Manager, Google Secret Manager, Azure Key Vault), which stores secrets encrypted, controls who can read them and keeps a record of access.
In GitHub Actions, use repository or environment secrets, which are masked in the logs.
With Docker, pass them at runtime with --env-file or -e, never in the Dockerfile.

In a team, the production keys are usually managed by the team lead or a senior engineer, who sets them on the server directly. Developers work with their own development keys and never see the production ones, so a leak from one laptop can't touch production.

Separate keys and least privilege

Use different keys for development, testing and production. If your development key leaks, production keeps working, and when you look at usage, you can tell which environment spent the money.

Give each key only what it needs. A key that only reads from the vector database shouldn't be able to delete collections, and a cloud key used to deploy one service shouldn't be able to create new servers. Many providers let you create keys per project, restrict them to certain models or IP addresses, and set limits per key.

Spending limits and alerts

Even with perfect handling, assume a key can leak or a bug can loop. Set a monthly budget or hard spending limit in your model provider's dashboard, add usage alerts at a fraction of it, and set billing alerts in your cloud account. A bug that calls the model in an infinite loop is much less painful when the provider stops it at your limit.

Never put keys in the frontend

Anything that runs in the browser or in a mobile app can be read by the user. If your JavaScript calls the model provider directly with your key, anyone can open the developer tools and copy it.

The frontend calls your backend, and your backend, which holds the key, calls the model. This also lets you add authentication, rate limits and logging in one place.

Keep secrets out of logs, prompts and traces

Secrets leak through more places than Git.

Don't log full request headers, because they contain authorization tokens.
Don't print settings objects or environment variables when debugging a deployed app.
Don't put keys or passwords in prompts. Anything in a prompt is sent to the model provider, can appear in traces, and can be repeated by the model.
Check what your tracing tool records before you send production traffic to it.

Rotation

Rotating a key means replacing it with a new one and revoking the old one. Do it on a schedule for important keys, whenever someone with access leaves the team, and immediately when you suspect a leak. If your code reads keys from the environment, rotation is only a settings change and a restart.

What to do if a key leaks

Revoke the key in the provider's dashboard, right away, before anything else.
Create a new key and update it wherever the app runs.
Check the provider's usage dashboard for calls you didn't make, and contact their support if there are any.
Remove the key from the code and, if it was committed, from the Git history, knowing that anyone who cloned or forked the repository still has it.
Find out how it leaked and fix that, for example by adding .env to .gitignore or adding a secret scanning tool.

Deleting the file and pushing again is not enough, because the key is still in the earlier commit. GitHub's push protection, which is on by default for public repositories, blocks many known key formats before they are pushed, and tools like gitleaks can scan your repository locally, but treat them as safety nets, not as the plan.

From the floor

In the Build30 day 1 session, while setting up the first LLM call, I explained why the .env file never goes to GitHub: in a team, I wouldn't share the env file with anyone, I would upload it to the server myself when deploying, so sensitive information is only available to the person responsible for it.

In one session, while setting up the keys for a LangChain project, I reminded builders never to share an API key anywhere, even a free one, because if someone uses it in production and thousands of users hit it, it is your account that takes the hit.

Next step

Read Pydantic and structured data for LLM apps.

Sources

← PreviousCloud basics for AI engineers: your first deploy