Lesson 5 (AI): Tokenization
How does an AI read? It splits text into chunks called Tokens.
Split
The split() method divides a sentence into a list of words.
Your Objective
Split the sentence into words.
How does an AI read? It splits text into chunks called Tokens.
The split() method divides a sentence into a list of words.
Split the sentence into words.