CORTEX

Why This is Portfolio-Worthy

Most AI portfolio projects are single-file notebooks that call an API. Cortex is an architecture. Here's exactly how it differentiates you.

The Interview Narrative

What Hiring Managers See

System Architecture

Cortex has four layers: the Orchestration layer (Week 1/2), the Intelligence layer (Week 3), the Tools layer (Week 4), and the Infrastructure layer (Weeks 2/3/4).

Full System — User Query to Response

Data Flow — One Query

Skills Mapping: Weeks 1-4 -> Cortex

Every major concept from every week of Phase 1 is represented in the system. This is what makes Cortex a genuine integration project, not a standalone exercise.

Core Components

The 5 Tools

Each tool demonstrates a different Week 4 production pattern. Together they form the "tool suite" — analogous to the Week 4 research assistant assignment, but purpose-built for Cortex.

Why These 5 Specifically?

• knowledge_base_search — connects Week 4 tool design directly to the Week 3 RAG pipeline. The integration between layers is what makes this Cortex, not just a research assistant.
• web_search — the most failure-prone real-world tool. Rate limits, costs, flaky APIs. Perfect vehicle for teaching rate limiting and fallbacks.
• create_support_ticket — represents write operations. Different failure contract than read operations. Tests idempotency thinking.
• get_team_calendar — authentication and token-based auth pattern. Shows secrets management in action.
• generate_report — output tool. Tests budget enforcement — some queries should produce full reports, others a summary if tokens are expensive.

Folder Structure

Clean, modular, production-aligned. Every folder has a single responsibility. The structure itself communicates that you understand software architecture.

cortex/
├── README.md                    # Portfolio write-up with metrics + architecture diagram
├── docker-compose.yml           # PGVector + Redis — starts with one command
├── requirements.txt
├── .env.example                 # Never commit real keys
├── main.py                      # Entry point: start Cortex
│
├── agents/
│   ├── supervisor.py            # LangGraph state machine (W1 + W2)
│   ├── knowledge_agent.py       # RAG specialist — calls rag/ pipeline (W3)
│   ├── research_agent.py        # Web research + rate limiting (W4)
│   └── action_agent.py          # Tools: ticket, calendar, report (W4)
│
├── rag/
│   ├── pipeline.py              # Orchestrates all 7 layers
│   ├── ingestion/               # L1: PDF/docx/txt processing
│   │   └── document_loader.py
│   ├── chunking.py              # L2: semantic chunking
│   ├── embeddings.py            # L3: OpenAI / HuggingFace
│   ├── vector_store.py          # L4: PGVector CRUD
│   ├── query_understanding.py   # L5: reformulation + expansion + intent
│   ├── access_control.py        # L6: RBAC tier filter
│   ├── hybrid_search.py         # L7: BM25 + vector + RRF
│   └── evaluation.py            # P@5 / MRR / NDCG dashboard
│
├── memory/
│   ├── session_memory.py        # Redis sliding window (W3)
│   └── entity_store.py          # PostgreSQL long-term entities (W3)
│
├── tools/
│   ├── web_search.py            # DuckDuckGo + SerpAPI fallback (W4)
│   ├── ticketing.py             # Ticket creation with validation (W4)
│   ├── calendar.py              # Calendar with auth pattern (W4)
│   └── report_generator.py      # Budget-enforced report tool (W4)
│
├── reliability/
│   ├── rate_limiter.py          # Token bucket + exponential backoff (W4)
│   ├── fallback.py              # 3-tier fallback chains (W4)
│   └── cost_tracker.py          # Per-query budget enforcement (W4)
│
├── prompts/
│   ├── supervisor/
│   │   ├── v1.0.0.yaml          # Version 1 (W2)
│   │   └── v1.1.0.yaml          # A/B test variant
│   └── agents/
│       ├── knowledge_agent/v1.0.0.yaml
│       └── research_agent/v1.0.0.yaml
│
├── observability/
│   ├── logger.py                # Structured logging: user, agent, cost (W2)
│   └── metrics.py               # Aggregated cost + quality dashboard
│
├── data/
│   └── sample_knowledge_base/   # 10 sample docs: HR, IT, policy, finance
│
└── tests/
    ├── unit/
    │   ├── test_tools.py        # Tool schema + error handling (W4)
    │   ├── test_rate_limiter.py
    │   ├── test_fallback.py
    │   ├── test_cost_tracker.py
    │   └── test_rag_pipeline.py
    ├── integration/
    │   ├── test_supervisor_routing.py
    │   ├── test_knowledge_agent.py
    │   └── test_full_pipeline.py
    └── evaluation/
        └── test_llm_judge.py    # LLM-as-judge (W4)

Milestone 1 — Core System

Target: End of Week 4 | "It works."

The foundational plumbing. Supervisor routes. Knowledge Agent retrieves. You can have a multi-turn conversation that returns grounded answers from your document set. Memory holds context for the session. Logging tracks costs.

M1 Checklist

• Docker Compose running: PGVector + Redis both healthy
• Sample knowledge base loaded: 10 documents ingested into PGVector
• Supervisor agent implemented: LangGraph graph with 3 nodes (supervisor, knowledge, fallback)
• Knowledge Agent working: runs full 7-layer RAG pipeline end-to-end
• Basic memory: Redis session sliding window stores last 6 messages
• 2 tools functional: knowledge_base_search + web_search
• Structured logging: every query logs user_id, agent, tokens, cost
• Can answer: "What's the PTO policy?" (internal) and "What is RAG?" (web)
• 3 unit tests passing for core components

Milestone 2 — Full Production Stack

Target: 1 Week After M1 | "It doesn't break."

The full reliability and memory stack. All 5 tools. Rate limiting wrapping every external call. 3-tier fallbacks. PostgreSQL entity memory persisting across sessions. Versioned prompts. Budget enforcement. The system survives realistic failure scenarios.

M2 Checklist

• All 5 tools implemented with proper @tool schemas
• Rate limiter wrapping web_search and external API calls
• 3-tier fallback chain: primary -> backup -> graceful default
• PostgreSQL entity store: entities extracted and persisted across sessions
• Versioned prompts: supervisor and all agent prompts in YAML files
• Token budget enforcement: per-query limit, stops before going over
• Research Agent and Action Agent fully functional
• Graceful degradation: if Knowledge Agent fails, system doesn't crash
• 10+ unit tests passing (tools, rate limiter, fallback, cost tracker)
• Server restart test: session restored from Redis, entities from PostgreSQL

Milestone 3 — Portfolio Ready

Target: Before Week 5 | "I can defend every decision."

The evaluation layer, the full test suite, and the portfolio packaging. P@5 and MRR numbers on a test run. LLM judge evaluation scoring response quality. A clean README with real benchmark numbers. An architecture diagram. The project is ready to show to a hiring manager.

M3 Checklist

• Retrieval evaluation: 20-query golden dataset with labelled relevant docs
• P@5 >= 0.70 achieved (if below 0.70 — fix chunking or query expansion first)
• MRR >= 0.65 achieved
• Evaluation dashboard prints on pytest tests/evaluation/ run
• LLM judge test: 5 queries evaluated for faithfulness and relevance
• Full test suite: 18+ tests, all passing
• Architecture diagram in README (draw.io or mermaid)
• README sections: Problem, Solution, Architecture, How to Run, Benchmark Numbers
• Docker: docker-compose up && python main.py starts the full system
• .env.example with all required variables documented
• Video demo (optional stretch): 3-minute Loom showing a full multi-turn conversation

Success Metrics

These are the numbers your README should contain and your test suite should produce automatically.

Retrieval Quality

Reliability Targets

Memory Correctness

Grading Rubric

Maximum: 100 points. Portfolio-worthy threshold: 75 points. Distinction: 90 points.

Bonus Points (up to +10)

• +3 — Prompt A/B testing framework working (two prompt variants, comparison output)
• +3 — Video demo (3-minute Loom showing multi-turn conversation end-to-end)
• +4 — Streamlit or FastAPI front-end for Cortex (even simple chat interface)

Getting Started — Your First 2 Hours

Follow this sequence exactly. Don't skip ahead to the interesting parts before the plumbing is working.

Hour 1 — Infrastructure & Skeleton

# Step 1: Create the project folder and git init
mkdir cortex && cd cortex && git init

# Step 2: docker-compose.yml
docker-compose up -d
docker ps   # Should show pgvector and redis both running

# Step 3: Install dependencies
pip install langchain langgraph langchain-openai langchain-community
pip install pgvector psycopg2-binary redis python-dotenv
pip install rank-bm25 pytest pytest-mock pyyaml

# Step 4: Create .env from example
cp .env.example .env
# Fill in OPENAI_API_KEY — everything else has defaults

# Step 5: Create folder skeleton
mkdir -p agents rag/ingestion memory tools reliability \
  prompts/supervisor observability tests/unit \
  tests/integration data/sample_knowledge_base

Hour 2 — First Working Query

# Step 6: Copy the sample documents (from Solution_Code_Snippets/data/)
# 10 documents: hr_policy.txt, it_runbook.txt, parental_leave.txt, etc.

# Step 7: Ingest documents into PGVector
python -c "from rag.pipeline import ingest_documents; \
  ingest_documents('data/sample_knowledge_base')"

# Step 8: Test a single RAG query (before supervisor)
python -c "
from rag.pipeline import query
result = query('What is the PTO policy?', user_tier='standard')
print(result)
"

# Step 9: Start the supervisor (basic version)
python main.py
# Should respond to: 'What is our parental leave policy?'
# Should respond to: 'What is machine learning?' (routes to web search)