Skip to content
Linear Horizon Training

Ready-made course · 2 days · Advanced

Production AI Architecture

A two-day architecture course for senior engineers and technical leaders: reference architectures, model gateways, RAG, agents, security, evaluation and the operational concerns that decide whether a system can leave the lab.

Production AI systems fail less often because the model is weak, and more often because the surrounding architecture was never designed: no gateway, no evaluation, no identity story, no cost control, and no owner for failure. This course is for people who have to draw that architecture and defend it.

The course works through reference architectures for hosted and self-hosted models, model abstraction, RAG, agents, event-driven and asynchronous workloads. Data boundaries, identity, permissions, secrets and governance are treated as structural, not as a later security review. Observability, prompt versioning, evaluation, rate limits, fallbacks and resilience are part of the same design.

The work is architectural. You complete design exercises and design reviews, including multi-model systems and the latency, scaling and deployment choices those systems force.

You leave with a production-shaped reference you can take back to a real programme, and a review method for designs that still look like a demo.

Who it is for
Software architects, senior engineers, platform engineers and technical leaders responsible for production shape, not only for a prototype.
Before you start
Experience designing or delivering production software. Familiarity with LLMs is expected. You do not need to implement models, but you should be able to read an architecture and argue about interfaces, failure and operations.

You will be able to

  • Design a reference architecture with a model gateway, clear data boundaries and an application-facing interface.
  • Compare hosted, self-hosted and multi-model inference against latency, cost, control and failure domains.
  • Place RAG, agents and event-driven workloads in an architecture without collapsing them into a single chat service.
  • Specify identity, permissions, secrets and governance for model access and retrieved data.
  • Define observability, evaluation and prompt-version management as platform concerns.
  • Design for resilience: rate limits, fallbacks, timeouts and degraded operation.
  • Review a proposed AI system and identify the production gaps that would block release.

Modules

8 quarter-day modules. Each one can go into a custom course on its own.

  1. Reference architectures for AI systems

    • Assistant, RAG, agent, batch and embedded-feature patterns.
    • Components and boundaries in each.
    • Choosing between them for a given problem.
    • Exercise: map an existing system onto a reference architecture.
  2. RAG as an architecture

    • The stages: ingest, index, retrieve, assemble, generate and cite.
    • Where RAG fits, and where fine-tuning or plain search fits better.
    • The failure modes of each stage.
    • Choosing a vector store and a framework, or neither.
  3. Workflows versus agents

    • The spectrum from a single call to an autonomous agent.
    • Chaining, routing, parallel calls and evaluator loops.
    • Choosing the least autonomy that does the job.
    • Exercise: redesign an over-agentic system as a workflow.
  4. Event-driven and batch AI workloads

    • Queues, events and background agents.
    • Batch processing of documents and records.
    • Idempotency, retries and dead letters with a non-deterministic step.
    • Exercise: design a document-processing pipeline.
  5. Identity, security and governance for AI

    • Acting on behalf of a user: identity through to tools and data.
    • Data classification and what may reach which model.
    • Audit, retention and regulatory obligations.
    • Governance that teams can follow without a committee.
  6. The AI gateway and control plane

    • One point of access to every model provider.
    • Keys, quotas, routing and fallbacks.
    • Logging, redaction and policy at the gateway.
    • Build or buy: open-source and managed gateways.
  7. Cost, latency and caching

    • Where the tokens go, and how to measure cost per task.
    • Model routing: small models first, large models when needed.
    • Prompt caching, response caching and batch APIs.
    • Streaming and perceived latency.
  8. Releasing and operating AI systems

    • Versioning prompts, models and indexes together.
    • Canary releases, shadow traffic and rollback.
    • SLOs for quality as well as uptime.
    • Model deprecations and provider incidents.

Practical work

Architecture exercises and design reviews, not coding labs. You produce architecture options, review a weak design, and complete a production architecture for a realistic system including gateway, RAG or agents, security and operations. Reviews are run as they would be in an engineering organisation.

You take away

  • Reference architecture diagrams and component checklists
  • A production-readiness review method
  • Decision notes for hosted versus self-hosted and multi-model routing
  • An observability, evaluation and cost-control baseline

Combine it with

Contact