AI Engineering and LLMOps FAQ: Learning, RAG, Agents, and Production

AI Engineering Questions Learners Actually Ask

What is AI engineering?

AI engineering turns models into reliable products and operational systems. It includes data preparation, evaluation, retrieval, APIs, observability, security, deployment, and incident response. Training a model may be one part, but most production work is building and operating the surrounding system.

What is LLMOps?

LLMOps applies production practices to language-model applications: versioning models and prompts, tracing requests, measuring tokens and latency, evaluating retrieval and faithfulness, enforcing guardrails, routing providers, managing capacity, and rolling back safely.

Do I need advanced mathematics to start?

No. Begin with train/holdout splits, leakage, confusion matrices, precision, recall, and why data distribution matters. Learn enough mathematics to reason about the model and metric you use, then deepen it as your role requires. Operational discipline does not wait for a graduate degree.

Can I learn with local CPU-only models?

Yes. Small quantized models are excellent for learning API contracts, streaming, token accounting, RAG, tracing, agents, guardrails, and recovery. They do not reproduce the quality or throughput of large production models, but the engineering boundaries transfer directly.

What is Ollama used for?

Ollama manages and serves local models through a CLI and HTTP API. It is useful for private experiments, offline labs, development, and repeatable local inference. Production suitability depends on scale, hardware, availability, model governance, and required serving features.

What is RAG?

Retrieval-augmented generation searches an external knowledge source and places relevant evidence into the model context. It can improve freshness and grounding, but only if ingestion, embeddings, metadata, retrieval, prompt assembly, and citations are correct. RAG does not guarantee truth.

ChromaDB or Qdrant?

Both can support local vector search. Choose based on deployment model, filtering, persistence, scale, operational ownership, and client ecosystem. For learning, the important skill is preserving IDs, metadata, embedding contracts, and retrieval tests rather than memorizing one API.

Why do embedding mismatches happen?

Teams rebuild queries with a different model, normalization setting, or preprocessing pipeline than ingestion used. A dimension mismatch may fail immediately; equal dimensions can still be semantically incompatible. Version the whole embedding contract and avoid mixed collections.

How are MLflow and DVC different?

MLflow tracks experiment runs, metrics, parameters, tags, and artifacts. DVC versions data and large files alongside Git. Use both when you need to reproduce which code and dataset produced a recorded experiment.

How should RAG be evaluated?

Evaluate retrieval and generation separately. Measure whether relevant chunks are returned, whether the answer is supported by those chunks, whether citations point to real sources, and whether the system refuses when evidence is insufficient. Include changed, ambiguous, adversarial, and no-answer cases.

Are AI agents safe to run in a terminal?

Only with strong deterministic boundaries. Use narrow allowlisted tools, least privilege, isolated workspaces, approvals for high-impact actions, iteration and time limits, independent verification, and protected audit logs. Never treat a model decision as authorization.

Can a system prompt stop prompt injection?

No. A system prompt helps guide behavior but cannot enforce tool permissions or data access. Retrieved documents and tool output remain untrusted. Put authorization, argument validation, secret isolation, and approval in code outside the model.

What should be traced?

Trace request identity, model and prompt version, retrieval IDs, timing spans, token counts, routing decisions, tool calls, guardrail outcomes, and status. Redact or avoid raw sensitive content, restrict trace access, and define retention. Observability must not become a second data leak.

What are guardrails?

Guardrails are checks before or after model execution that enforce product and safety policy. Examples include input size limits, tool allowlists, structured schema validation, PII masking, prohibited-content rules, and faithfulness gates. Define whether each rule blocks, transforms, warns, or requests approval.

Why separate liveness and readiness?

A process can be alive while its model is missing, still loading, or unable to reach required storage. Liveness controls restart behavior; readiness controls traffic. Mixing them causes restart loops or sends requests to an unusable instance.

How do I become job-ready?

Build one small system end to end. Version a dataset, record experiments, serve a model, add a RAG path, trace requests, measure TTFT and tokens, defend a tool boundary, evaluate failures, and write an incident postmortem. The AI course follows that eight-phase path, and the AI labs provide 17 concrete incidents to discuss.