Build Practical AI Systems

A beginner-first path through tiny local LLMs, RAG, MLflow, DVC, model APIs, and safe terminal agents - paired with offline CPU-conscious labs.

Modules

• Feature Engineering and Data Hygiene - Turn imperfect tables into trustworthy model inputs with Pandas and NumPy.
• Train and Evaluate Classical Models - Fit CPU-fast scikit-learn models and judge them with metrics that match the real cost of errors.
• Drift and Confusion Matrices - Use terminal reports to explain why yesterday's good model fails on today's data.
• How Local LLMs Work - Tokens, parameters, inference, context windows, and why small local models are useful.
• Ollama from the Terminal - Operate a local model inventory and make reproducible CLI and HTTP requests.
• GGUF and Quantization - Understand model files, quantized weights, memory budgets, and safe model selection.
• Embeddings and Similarity - Turn meaning into vectors and retrieve related text correctly.
• ChromaDB and Qdrant Local Mode - Persist vectors locally, attach metadata, query safely, and understand collection lifecycle.
• Build a RAG Pipeline - Chunk documents, retrieve evidence, construct grounded prompts, and cite sources.
• Track Experiments with MLflow - Record parameters, metrics, artifacts, and tracking locations so experiments remain reproducible.
• Version Data with DVC and Git - Track large data by checksum, keep metadata in Git, and detect drift between code and data.
• Serve Models with FastAPI - Design a bounded local inference API with validation, health checks, timeouts, and honest errors.
• Unified LLM Calls with LiteLLM - Use an OpenAI-shaped client across providers while keeping routing and secrets explicit.
• LLMs in Shell Pipelines - Safely connect files, pipes, JSON tools, and local inference in composable commands.
• Build Safe Terminal Agents - Design a bounded observe-plan-act loop with allowlisted tools, approvals, and secret-safe logs.
• LLM Request Tracing and Token Accounting - Follow each request through an LLM application and turn token use into operational cost evidence.
• Automated RAG Evaluation - Turn grounded answers into testable evidence with faithfulness, citation, and support checks.
• Latency and Time to First Token - Benchmark real streamed Ollama responses and diagnose configuration that makes an assistant feel slow.
• Defend Against Prompt Injection - Separate trusted instructions from untrusted user and retrieved content, then test that policy under attack.
• Mask PII in Output Streams - Recognize personal data and credentials, redact streams before storage, and test both leaks and false positives.
• Build Config-Driven Guardrails - Express input and output policy as configuration, enforce it outside the model, and fail closed on bad shapes.
• Continuous Batching and KV Cache - Understand how vLLM-style schedulers interleave requests, reuse attention state, and trade throughput for latency.
• Proxy Routing and Model Fallback - Place a LiteLLM reverse proxy in front of local models and test explicit failure-aware fallback routing.
• Health, Readiness, and Crash Recovery - Expose truthful probes, remove unhealthy instances from traffic, and restart crashed model services safely.