NetSecBot · Team 18

AI Assistant

Stage 4 · Network Security Hardening Guidance

Developer 4
Service port :8004
LLM Llama 3 8B
Guides 32
Tests 233
Slide A — Service Overview

AI Assistant Service

I built the AI Assistant service, which turns raw security findings from the scanner into actionable, device-specific remediation guidance using a local LLM.

Key Tools

Ollama + Llama 3 8B
Runs the LLM locally — zero data egress, air-gap capable
LangChain
Orchestrates the RAG pipeline: retrieval → prompt → generation
ChromaDB
Persistent vector store for the 32 hardening guides
sentence-transformers
Converts guide text to embeddings for semantic search (all-MiniLM-L6-v2)
FastAPI
Serves async REST endpoints consumed by the gateway at Stage 4

Design Trade-offs

Local LLM vs. Cloud API
Chose Ollama / Llama 3 8B
Network scan data never leaves the machine. Trade-off: ~11s per device vs. <2s with a cloud API — acceptable for a post-scan batch job.
RAG vs. Fine-tuning
Chose RAG with markdown guides
New device coverage = add a .md file + reindex. No model retraining, no GPU, no labeled dataset required.

Pipeline Position

Stage 1 · Scanner :8001
Nmap · ARP · mDNS · UPnP
↓
Stage 2 · Classifier :8002
Fingerbank · confidence scoring
↓
Stage 3 · Security :8003
11 check types · CVE lookup · risk score A–F
↓
⭐ Stage 4 · AI Assistant :8004
RAG · 32 guides · per-device hardening guidance
↓
Stage 5 · Reports :8005
PDF · Word · SQLite
Slide B — What I Built + Learned

AI Assistant — What I Built + Learned

What I Owned

endpoint /findings — Security-to-AI bridge
app/main.py · app/models.py · app/rag/prompt_templates.py
Receives classified findings from Stage 3, prioritizes critical/high severity, injects device context, and returns per-finding remediation text. Integrated via shared Pydantic contract model.
knowledge base 32-guide RAG corpus
knowledge_base/guides/*.md
Grew from 5 → 32 hardening guides. Coverage: routers, IP cameras, NVR, smart plugs, VoIP, VPN, IoT generic, smart home hubs, and more.
tests 233 tests · 24 test files
tests/
Full coverage of all endpoints, RAG pipeline, 3 prompt template types, session management, response caching, middleware, and Week 11 Scenario 3 (Remediation).

Performance Improvement

16.6s
→
11.0s
Avg response time per device
Tuned chunk size 1000→600, retrieval k 4→3  ·  −34%
5
→
32
Hardening guides in knowledge base
Covering 12+ device categories

What I Learned — Beyond Curriculum

RAG Architecture
How vector embeddings + semantic search replace keyword lookup for grounding LLM output in real documents. Learned the ingestion pipeline: chunk → embed → store → retrieve → inject into prompt.
→ Shipped a context-aware /findings endpoint that cites actual guide sections, not hallucinated advice.
Local LLM Deployment
Ollama model packaging, offline Docker image baking (embedding model pre-downloaded into image layer), and air-gap operation via env flags HF_HUB_OFFLINE=1.
→ Shipped a fully offline capable service — zero egress after first boot, deployable on isolated enterprise networks.
Prompt Engineering
How prompt structure (context-only instruction, explicit citation requirement, role-aware tone) directly affects output quality and prevents hallucination. Built 3 specialized prompt builders: explain vulnerability, hardening steps, generate checklist.
→ Tests confirm zero hallucination guard — prompts tested for correct structure, not mocked LLM output.
Microservice Contract Design
Learned how to design API contracts using shared Pydantic models so upstream (Security) and downstream (Reports, Gateway) services don't break on schema changes.
→ Fixed missing compliance_refs field in the Stage 3→4 handoff that would have silently dropped compliance data.
Slide C — Internal Architecture

RAG Pipeline — Inside the Assistant

How a security finding becomes actionable guidance

Request Flow

Gateway POST /findings
device_type · findings list · severity
↓
Priority Filter
CRITICAL / HIGH first
+
Device Context
injects device_type into query
↓
Embed Query
all-MiniLM-L6-v2
→
ChromaDB Search
k=3 nearest chunks · 32 guides
↓
Prompt Builder
explain · hardening steps · checklist · retrieved context injected
↓
Ollama · Llama 3 8B
local inference · ~11s avg · no egress
↓
Response
summary · recommendations[] · source_docs[]

Key Files

app/main.py
All API endpoints — /chat, /findings, /explain, /guide, /workflow
app/rag/ingest.py
Loads guides → chunks → embeds → stores in ChromaDB
app/rag/query.py
Semantic search against the vector store
app/rag/prompt_templates.py
3 specialized builders: explain, hardening, checklist
app/cache.py
TTL cache — deduplicates LLM calls for same device type
app/workflows.py
Step-by-step guided procedures (new install · audit)

Knowledge Base — Coverage

Routers
IP Cameras
NVR / DVR
Smart Plugs
VoIP / SIP
VPN Gateway
Smart Home Hub
IoT Generic
Firewall
Network Switch
Printers
+more
32 guides · ChromaDB (chunk=600, k=3)
1 / 4