How LLMs Actually Work

What Is an LLM?

LLM stands for Large Language Model. Strip away the marketing and here's what it does:

You give it text. It predicts the next word.

That's it. It's a text-prediction machine trained on massive amounts of internet text. It learned patterns - which words tend to follow which other words - and it applies those patterns to generate responses.

When you ask "What's the capital of France?" the model isn't looking up an answer in a database. It's predicting that, given the pattern of your question, the most likely next words are "The capital of France is Paris."

Tokens: How the Model Sees Text

LLMs don't read words the way you do. They break text into tokens - small pieces that might be whole words, parts of words, or even single characters.

Every model has a token limit - the maximum number of tokens it can process at once. This isn't just trivia. It's the fundamental constraint that shapes how LLMs work.

The Context Window

The context window is the model's working memory. It's the total amount of text the model can "see" at any one time - your messages, its responses, and any instructions from the developer.

Think of it like a whiteboard with a fixed size. Everything the model needs to know must be written on this whiteboard. When it fills up, old content gets erased to make room.

Typical context window sizes:

• 4K tokens - about 3,000 words (early models)
• 8K–32K tokens - about 6,000–24,000 words (common today)
• 128K+ tokens - about 100,000 words (latest models)

Every Message Is Sent From Scratch

Here's the thing most people don't realize: every time you send a message, the entire conversation is sent to the model from the beginning.

The LLM has no memory. It's completely stateless. It doesn't "remember" what you said two messages ago. Instead, the application (ChatGPT, Claude, etc.) stores your conversation and replays the whole thing on every single turn.

Click through each turn to see what actually gets sent. (The "instructions" row is the developer's hidden setup text - you'll learn about that in the next module.)

Notice how the context keeps growing. Turn 1 sends 2 messages. Turn 2 sends 4 messages. Turn 3 sends 6. The model re-reads everything from scratch each time. It's not "continuing a conversation" - it's receiving the full transcript and predicting what comes next.

This is why long conversations eventually slow down or lose coherence. The context window fills up, and old messages get dropped. The model literally forgets the beginning of your conversation.

Why This Matters for Security

If an LLM is just a pattern-matching text predictor with a finite text window and no memory, then:

• It can't truly "follow rules" - it can only predict text that's consistent with patterns
• It can't "protect secrets" - secrets in the context window are just text it can be prompted to repeat
• It can't distinguish trusted from untrusted input - it's all just tokens

Every AI security vulnerability you'll learn about traces back to these fundamentals.

Next Steps

Now that you know what an LLM actually is, the next question is: how do developers control its behavior?

That's where system prompts come in - and where things start to get fragile.