Interview questions for GenAI, LLM engineering, and AI application roles — covering both concepts and practical judgment.
What is RAG and when would you use it over fine-tuning?
RAG grounds an LLM's answers in retrieved external data at query time, without changing the model itself. It's preferable when your data changes frequently or you need traceable sources. Fine-tuning is better suited to changing a model's style or behavior patterns rather than injecting current facts.
How do you evaluate whether a RAG system is working well?
Check retrieval quality first — are the right chunks actually being retrieved for a given query? Then check generation quality — does the model use the retrieved content correctly, or ignore it? Both need separate evaluation, since a good retriever paired with poor prompting still fails.
What causes hallucination and how do you reduce it?
Hallucination happens because LLMs generate statistically plausible text, not verified facts. Reducing it involves grounding responses in retrieved data (RAG), lowering temperature for factual tasks, and explicitly instructing the model to say 'I don't know' when it lacks sufficient information.
Explain the difference between an AI agent and a chatbot.
A chatbot responds to messages in a conversational loop. An agent can autonomously plan multi-step tasks, decide which tools to call, and take actions — the distinction is initiative and tool use, not just conversation.
How would you handle prompt injection in a production AI application?
Treat any untrusted input (documents, web content, user messages) as potentially adversarial — don't let it override system-level instructions, validate and sanitize tool outputs before acting on them, and apply the principle of least privilege to what actions the AI is allowed to take.
Go deeper
These questions are just the surface
Our GenAI & LLM Engineering program builds the hands-on experience behind these answers — not just memorized responses.
