What's in a Word—Breaking Texts and

This session introduces the preprocessing behind nearly all text analysis: tokenization, lemmatization, stemming, and stop word removal. We'll also distinguish this sense of "token" from how the word is used in large language models, a difference that reveals a lot about both. You'll leave understanding not just how to prepare text, but what each choice quietly decides.