The LLM Gateway: Build and Operate One in 1 hr
Join this event
About this event
A hands-on session. We build one gateway properly, break it on purpose, put controls on it, and then work through what it takes to operate it as a shared production service.
Most organisations arrive at the same problem. One application talking to one model provider is fine. Four applications talking to four providers is not: every integration carries its own authentication, retry logic, timeout handling, failover behaviour, and log format, implemented separately by teams who have never discussed it. There is no single place to enforce a policy, no shared view of what is failing, and no way to revoke access for one application without affecting the rest.
An LLM gateway turns that sprawl into one controlled, observable access point. This session builds one live, then covers the four things that decide whether it survives contact with production.
Security
Applications stop holding provider credentials entirely.
Reliability
Retries, cooldowns and fallback solve three genuinely different failures, and conflating them is how a provider blip becomes a self-inflicted outage.
Observability
The metrics that actually earn a dashboard: request and failure rates, provider latency, time to first token, rate-limit responses, retry counts, and fallback counts.
DevOps
The gateway configuration contains no secrets, only references, which makes it a deployable artifact that belongs in Git with a pull request, a test and a rollback path.
If you want to follow along
Docker and Docker Compose v2 running, curl and jq installed, and one provider API key. A second provider makes the failover demonstration more convincing but is not required.
Not set up? Join and watch. You will get more out of the session than you would debugging your environment in the background, and the repository is yours either way.
What you take away
The complete repository used on screen: the Docker Compose stack, the gateway configuration, the demo scripts, the CI policy validator, Prometheus alert rules, and on-call runbooks.
Schedule
August 28, 2026 18:30 to 19:30
Recurring on the last Friday of every month
Who Should Join
Cloud engineers, DevOps engineers, platform engineers, SREs, security engineers, architects, developers, engineering managers, and anyone interested in modern infrastructure and software delivery.
Want to find AI jobs?
Join thousands of AI professionals finding their next opportunity