The model never sees a letter
before a model thinks, it shatters your text into tokens, and that one step explains half its quirks.
The first thing a model does to your sentence is destroy it, neatly, by the rules of a dictionary it learned once and never changes.
Ask a state-of-the-art model how many times the letter 'r' appears in 'strawberry' and it may confidently say two. People take this as proof the thing is dumb. It isn't dumb; it simply never saw the letters. By the time 'strawberry' reaches the part of the model that reasons, it is no longer s-t-r-a-w-b-e-r-r-y. It is two or three numbered chunks, and counting characters inside a chunk is like counting the bricks in a house you were handed as a single photograph.
What a token actually is
A token is a piece of text drawn from a fixed vocabulary: a dictionary of tens of thousands of fragments that was frozen when the model was built. Some tokens are whole common words. Some are single characters. Most useful ones are subword pieces: 'ing', 'tion', ' the'. The model converts each token to an integer, looks up a vector for that integer, and from then on works only with numbers. It has no window back to the original spelling.
Where does the vocabulary come from? Byte-pair encoding. You start with text as raw bytes, then repeatedly find the most frequent adjacent pair of symbols and merge it into one new symbol. Do this thousands of times over a huge corpus and the merges that survive are exactly the patterns that recur, so ' model' becomes a single token because it appeared constantly, while 'qubit' stays in pieces because it didn't. The vocabulary is a frozen record of what was common in the training text.
A token usually carries the space in front of it. To the model, ' cat' and 'cat' are simply two different words.
Why the seams show
Once you see tokens, the quirks stop being mysterious. Casing matters because 'Hello', 'hello', and 'HELLO' can tokenize into different pieces, so they land in different regions of the model's number-space. A stray double space or a tab can flip an answer for the same reason. And languages pay unequal rent: English text was abundant during vocabulary-building, so English packs tightly into few tokens, while Arabic, Thai, or code with long identifiers fracture into many more, meaning the same meaning costs more tokens, more money, and more of your context window.
This is the quiet unit behind everything you measure. A context window of 200,000 'tokens' is not 200,000 words; in English it is roughly 150,000, because a token averages about three-quarters of a word. Prices are per token. Rate limits are per token. When you learn to feel where text fractures, you stop being surprised by the model; you start predicting it, and writing prompts that the tokenizer treats kindly.