SAGE

Employees misread dense policy documents and lean on informal guidance, creating real regulatory exposure. Generic chatbots hallucinate policy details, and routing every question through legal doesn't scale. SAGE gives grounded, auditable compliance answers — and, critically, stays secure when users try to manipulate it.

An 8-layer security-and-reasoning pipeline that turns policy PDFs into a trustworthy assistant:

• Layered request flow: query sanitization → injection detection → grounding gate → query expansion → hybrid RAG → ReAct agent → post-processing → audit logging
• Hybrid RAG retrieval (0.6 × semantic + 0.4 × keyword) over section-level chunks, cutting prompt tokens ~80% vs. injecting the full corpus
• A LangGraph ReAct agent (GPT-4o) with tools for policy search, cross-references, conflict detection, and risk assessment
• Structured answers with citations, a High/Medium/Low risk level, a 0–100 confidence score, and policy-conflict flags
• Bring-your-own-documents: upload your own policy PDFs to query a custom corpus, or explore 5 built-in demo organizations
• Production layer: 6-turn conversation memory, JSON audit trails, and a Streamlit chat UI

The trade-offs — and why:

• Chose a ReAct agent over direct prompting so every answer traces back to tool-grounded retrieval instead of free-form generation
• Hardcoded 5 conflict rules (CF-001–CF-005) rather than trusting the model to surface policy tensions consistently
• Used hybrid semantic + keyword scoring to catch compliance jargon that pure vector search misses
• Fine-tuned gpt-4o-mini for ~10× lower cost, with an LLM-as-Judge scoring answers across 5 dimensions as a quality gate

The hard parts:

• Prompt-injection defense: built a 52-pattern pipeline across 9 attack families plus 6 defensive measures — reaching a 100% block rate (37/37 vectors) while still passing 100% of legitimate queries (25/25)
• Output instability: 6 phrasing variations and a temperature sweep exposed 8 instabilities; resolving them lifted format compliance from 52% to 85%
• Retrieval gaps: 57 curated synonym mappings bridged the gap between how employees phrase questions and how policies are written

Across a 57-case evaluation set, SAGE reaches 91%+ risk-classification accuracy, ≥8.5/10 on LLM-as-Judge, 100% citation groundedness, and an 82/100 average confidence — while blocking 100% of adversarial attacks. Backed by 28 unit tests and deployable via Docker on Google Cloud Run.

Multi-document conflict resolution across larger policy corpora, richer analytics on flagged compliance risks, and continuous adversarial red-teaming built into the evaluation harness.