LLM fundamentals

Every AI interview assumes these, and a first screen often tests them directly.

Every AI interview assumes these, and a first screen often tests them directly.

A model predicts the next token. It has no memory between calls, so your app sends the conversation again each time.

A token is about 4 characters of English. Input plus output must fit in the context window, so reserve room for the reply.

It produces plausible text, so it can be confidently wrong. You design around that with grounding, citations and checks in code.

About how many characters is one token in English?
About 4, or about three quarters of a word.
What must fit in the context window?
Input and output together, per request.
Why does a chatbot seem to remember?
The application resends the conversation each time. The model itself keeps no memory between calls.
What does lower temperature do, where a model offers it?
It reduces randomness. It doesn't make answers more true.
What is an embedding?
A list of numbers representing meaning, where similar text gets similar numbers.
What breaks prompt caching?
Any change early in the prompt. Keep fixed text first, and what varies last.
What is prompt injection?
Input that tries to act as instructions. Treat outside text as data, and enforce limits in code.