← All work
AI Engineer2025

SAGE

SAGE (Secure AI Governance Engine) answers employee policy questions with grounded, auditable, citation-backed responses — and defends against prompt-injection attacks with a 100% block rate.

Live DemoGitHub
SAGE — secure AI compliance assistant
5
Dev phases
13
Prompting techniques
57
Eval cases
5
Org types
28
Unit tests
4
Agent tools

Try it

Live deployment
Open SAGE

Load one of the five demo organisations, or upload your own policy documents. Then try an edge case: “Can I store encrypted customer data on my personal laptop since it is encrypted?”

The problem

Policy is unreadable

  • Ask legal

    Slow and expensive for every routine question

  • Ask a chatbot

    Hallucinates policy detail, which is worse than no answer

  • Guess

    Informal verbal guidance creates regulatory exposure

SAGE sits between the three: grounded in the actual policy text, automated, auditable, and hardened against people trying to talk it out of its own rules.

What it returns

Structured, cited, scored

> I just started 45 days ago. Am I eligible for remote work?

Answer
You are not eligible. POL-RW-2025 §2 requires 90-day probation completion.
Citations
POL-RW-2025 §2 — Eligibility
Risk level
Medium
Reasoning
Employee tenure (45 days) is below the 90-day threshold. The request cannot proceed until probation is completed.
Confidence
74 / 100
Policy tension
None detected.

Five phases

Prompt to production

Arrow keys to step

Phase 01of 05

System prompt design

Find the best prompting strategy for compliance reasoning by testing, not guessing

  • 13 techniques on one identical query, so results compare directly
  • 7 foundational: zero-shot, few-shot, CoT, step-back, analogical, auto-CoT, generated knowledge
  • 5 advanced: decomposition, ensembling, self-consistency, universal self-consistency, self-criticism
  • Meta-prompting: model rewrites its own system prompt

Decision

Chose
Test all 13, then synthesise
Over
  • Pick chain-of-thought and move on

Because

  • Same query across every technique makes the comparison valid
  • Structured output (answer, citations, risk, reasoning, confidence) came from the winners

Cost

13 runs of scaffolding before a single line of product code.

52%
zero-shot baseline
85%
format compliance

Architecture

Full request path
SAGE system architectureA query passes three security gates in order: sanitisation strips role tokens and caps length, a 52-pattern injection check across nine attack families, then a policy-grounding scope gate. It is then expanded through 57 synonym mappings, retrieved from ChromaDB over section-boundary chunks, and re-ranked at 0.6 semantic to 0.4 keyword. A LangGraph ReAct agent on GPT-4o calls four tools, search_policy, check_cross_references, detect_policy_conflicts and assess_risk, then emits a structured response. Post-processing verifies every citation against source text, scores confidence and severity out of 100, and writes a JSON audit record.SECURITY GATESRETRIEVALAGENTPOST-PROCESSINGblockedblockedblockedL0 sanitize_query()Strip [INST] <sys> [OVERRIDE]Cap at 1,200 charsL1 is_injection()52 patterns9 attack familiesL2 _is_out_of_scope()Policy grounding gateContact queries pass_expand_query()57 synonym mappingswork abroad to remoteChromaDBtext-embedding-3-smallSection-boundary chunksRe-rank0.6 semantic0.4 keyword overlapLangGraph ReActGPT-4o, StateGraphToolNode + tools_conditionsearch_policytop-7cross_refstriggeredconflictsCF-001..005assess_riskH / M / LStructured responseCitationVerifierEvery section cross-checkedConfidenceScorer0-100SeverityScorer0-100 weightedAuditLoggerJSON per queryUser queryStreamlit UI
Request path, left to right. Gates run before retrieval.

Results

Target against actual
Measured results against target
MetricTargetActual
Risk classification accuracy≥ 87%≥ 91%
LLM-as-judge score≥ 8.5/10≥ 8.5/10
Citation groundedness100%100%
Average confidence≥ 70/10082/100
Attack block rate100%100%
Legit query pass rate100%100%
Unit test pass rate100%100%
Policy conflict rules5/55/5

Stack

Everything used

Model and agent

  • GPT-4o
  • LangGraph
  • LangChain
  • ReAct

Retrieval

  • ChromaDB
  • text-embedding-3-small
  • Hybrid re-ranking

Documents

  • pdfplumber
  • 3-pass extractor
  • Section-boundary chunking

Security

  • 52-pattern regex
  • Query sanitiser
  • Citation verifier

Evaluation

  • GPT-4o-mini as judge
  • 57-case suite
  • pytest

Delivery

  • Streamlit
  • Docker
  • Cloud Run
  • Python 3.11
See these across every project