Ready-made course · 1 day · Intermediate
Securing Generative AI Applications
A one-day security course for developers, security engineers and architects: the specific risks created by LLM applications, and the controls that actually reduce them.
LLM applications create attack surface that conventional application security only partly covers. Prompt injection, indirect injection through retrieved documents, tool misuse, data leakage and excessive agency are not theoretical. They appear as soon as a model can read untrusted text or call a tool.
This course treats those risks as engineering problems. You work through threat scenarios: injected instructions, poisoned knowledge bases, over-privileged tools, unsafe output handling and the supply chain of models, prompts and plugins. Identity, permissions, sandboxing, audit and logging are covered as controls, not as a policy appendix.
The teaching is concrete. Each major risk is paired with a mitigation that can be implemented in application and platform design: argument validation, allow-lists, output encoding, retrieval access control, human approval and least privilege.
You leave able to review an LLM application for the failures that matter, and to put proportionate controls in place before the system is exposed to untrusted input.
- Who it is for
- Developers, security engineers and architects responsible for designing, reviewing or operating LLM-powered applications.
- Before you start
- Familiarity with web or API security concepts is expected. Direct LLM implementation experience is helpful. The course does not require cryptography or ML research background.
You will be able to
- Identify prompt injection, indirect injection and retrieval attacks in realistic application designs.
- Diagnose insecure tool use, excessive agency and secrets exposure in agentic systems.
- Design permission, identity and sandboxing controls around model-initiated actions.
- Handle model output as untrusted data in downstream systems and user interfaces.
- Evaluate model-provider, plugin and supply-chain risk as part of architecture review.
- Specify logging, audit and human-approval requirements that are actually operable.
- Apply a threat-modelling method to an LLM application and produce a mitigation list.
Modules
4 quarter-day modules. Each one can go into a custom course on its own.
The LLM threat model
- Why a model is neither a trusted component nor a simple input.
- Assets, trust boundaries and attackers for an LLM-backed system.
- The OWASP Top 10 for LLM applications.
- Exercise: threat-model a real feature.
Prompt injection and poisoned context
- Direct and indirect prompt injection.
- Poisoned documents, retrieval and tool results.
- Mitigations that help, and the ones that only look as if they do.
- Lab: attack a RAG assistant, then defend it.
Tools, excessive agency and unsafe output
- Least privilege for tools, and confirmation before side effects.
- Data exfiltration through links, images and tool calls.
- Treating model output as untrusted: rendering, code and SQL.
- Lab: close the exfiltration paths in an agent.
Securing the AI platform and supply chain
- Secrets, keys and gateways between applications and providers.
- Model and package supply chain, including open-weight models.
- Logging, retention and audit without leaking what you protect.
- Red-teaming and incident response for AI features.
Practical work
You work through realistic threat scenarios on intentionally weak applications, then apply controls. The day includes at least one injection and one tool-permission exercise, plus a structured review of a more complete design. This is not an attack cookbook for third-party systems: the work is on designs your team can change.
You take away
- A threat model template for LLM applications
- Mitigation patterns for injection, retrieval, tools and output handling
- A security review checklist
- Logging and approval requirements that can be implemented