AWS outlines architecture for ChatGPT Codex via LiteLLM gateway

AWS has outlined a reference architecture for placing ChatGPT Codex behind a controlled LiteLLM gateway. This setup utilizes LiteLLM running on AWS Fargate to authenticate callers and apply specific model and consumption policies.

The architecture employs an Application Load Balancer and AWS WAF for security, while Amazon RDS for PostgreSQL manages gateway state and budget data. To ensure security, administrators are advised to keep ECS tasks and databases in private subnets and restrict load balancer access to corporate or VPN networks.

By using LiteLLM, developers can expose stable aliases for models, such as mapping a specific endpoint to an Amazon Bedrock model. This allows for an OpenAI-compatible interface, ensuring that developer configurations remain independent of provider-specific model identifiers.