Skip to content

Prerequisites & Setup

Treat setup as the first operational checkpoint: get one clean test run before changing the service. That baseline is what lets later incidents tell you whether your prompt, retriever, authorization rule, or runtime actually improved the system.

What you need

Skills

  • Python 3.11+ at intermediate level (functions, classes, async basics)
  • Comfort with HTTP APIs and JSON
  • Git basics; a GitHub account
  • Optional: Docker, basic Linux CLI

Hardware

Path Minimum Comfortable
API-only (cloud models) 8 GB RAM laptop 16 GB
Local 1–3B Q4 (classify / extract) 8 GB RAM 16 GB
Local 7B–8B Q4 16 GB RAM 32 GB + GPU (8 GB+ VRAM)
Fine-tuning / large RAG 16 GB + GPU 24 GB+ VRAM

Fit weights + KV cache + OS headroom — details in Module 17 §7. A 7B that swaps is worse than a 3B that stays resident.

Cloud-API path = laptop. Learn ML’s no-GPU laptop is that row. A local 7B Q4 uses the 16 GB minimum when the weights stay resident; the comfortable column adds a GPU.

Accounts (pick what you need)

  • At least one LLM provider: OpenAI, Anthropic, Google AI, or free local via Ollama
  • Optional: Hugging Face, Pinecone/Weaviate/Qdrant, cloud host (Fly, Railway, AWS, Azure, GCP)

Model IDs in this course are placeholders

Strings like gpt-4o-mini, claude-sonnet-xxxxxx, or llama3.2 show role (cheap classifier vs stronger generator vs local SLM), not a pin you must use in 2026. The xxxxxx suffix marks a fill-in — it is never a real ID. Check your provider’s current catalog before you copy a snippet.

Environment setup

1. Clone and Python env

Python: 3.11–3.13 recommended (^3.11,<3.14 in pyproject.toml). Core teaching modules are stdlib-only.

git clone https://github.com/<you>/AIEngineering.git
cd AIEngineering

# Poetry (repo default)
poetry config virtualenvs.in-project true
poetry env use python3.11   # if multiple Pythons installed
poetry install --with dev
# Optional stock/data track extras (pandas, numpy, sklearn, …):
# poetry install -E track-data

# Or venv + pip (core tests only)
python3.11 -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -U pip pytest
PYTHONPATH=. pytest tests/ -m "not track_data"

2. Core learning packages

Install as you need modules (not everything on day 1):

pip install openai anthropic python-dotenv httpx tenacity tiktoken
pip install pydantic pydantic-settings
# RAG
pip install numpy scikit-learn  # baselines / embeddings helpers
# Optional later
# pip install chromadb sentence-transformers fastapi uvicorn
# pip install peft transformers accelerate bitsandbytes  # fine-tune

3. Secrets

Create .env (never commit it):

# .env
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
GOOGLE_API_KEY=
# Optional
HF_TOKEN=

Load with python-dotenv or your shell.

4. Smoke test (provider-agnostic idea)

"""setup_smoke.py — adapt to your provider."""
import os
from dotenv import load_dotenv

load_dotenv()

def main():
    # Prefer Anthropic or OpenAI if keys exist; else print local guidance
    if os.getenv("OPENAI_API_KEY"):
        from openai import OpenAI
        client = OpenAI()
        r = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": "Reply with exactly: ok"}],
            max_tokens=16,
        )
        print(r.choices[0].message.content)
        return
    if os.getenv("ANTHROPIC_API_KEY"):
        import anthropic
        client = anthropic.Anthropic()
        r = client.messages.create(
            model="claude-sonnet-xxxxxx",
            max_tokens=16,
            messages=[{"role": "user", "content": "Reply with exactly: ok"}],
        )
        print(r.content[0].text)
        return
    print("No API key found. Install Ollama and run: ollama run llama3.2")

if __name__ == "__main__":
    main()

# macOS / Linux / Windows
# https://ollama.com
ollama pull llama3.2
ollama run llama3.2

Course sandbox package

This repo includes runnable teaching code under src/:

Module Purpose
src.security Sanitization + PII redaction
src.prompts Template render helpers
src.context_memory Session memory + token budget
src.rag TinyRAG + RRF
src.evals Golden-set runner
src.cost Router, cache, spend ledger
src.agents Single-agent loop
src.audit Hashing audit events 
poetry install
poetry run pytest tests/ -v

Guided exercises: Exercises · grading: Assessment.

Capstone starter (no Poetry, no API keys)

The production-AI skeleton lives next to docs/, not inside src/:

cd capstone-starter
pip install -r requirements.txt
uvicorn app:app --reload
pytest tests/test_api.py tests/test_eval.py -v

Walk the five gates. Track day-1 slices: tracks/starters/.

The command-runtime capstone is a second skeleton. A mock diff stands in for a 4B code model:

cd capstone-command
pip install -r requirements.txt
python cmdai.py spec check fixtures/retry_api_client.yaml
pytest tests/ -v

Walk the command-runtime gates. run --intent still returns a patch; Gate 1 closes that.

The third capstone is a task a 20B model fails as one prompt. The starter still trusts that prompt:

cd capstone-decompose
pip install -r requirements.txt
python divide.py check fixtures/refund_task.yaml
python divide.py run fixtures/refund_task.yaml
pytest tests/ -v

Walk the divide-solve-join gates. run prints one_shot and done; Gate 1 makes that a baseline failure, and Gate 4 is the join.

your-work/
  prompts/
  notebooks/          # exploration only
  src/                # or extend this repo's src/
  tests/
  evals/              # golden sets, prompt fixtures
  .env
  PROGRESS.md         # daily log

Windows notes

  • WSL2 is strongly recommended for path parity with tutorials: Install WSL
  • Open the WSL folder in VS Code with the Remote – WSL extension
  • Keep Git line endings consistent (core.autocrlf)

macOS / Linux: native terminal is fine; VS Code or Cursor optional.

Docs site (contributors)

pip install -r requirements-docs.txt
mkdocs serve   # http://127.0.0.1:8000
mkdocs build   # site/ for static export

Next