S9:E4 - "Confused about compilers?"
Base.cs Podcast
00:00
How to Tokenize a String
We're treating the quote to sleep per chance to dream. Just as a regular sentence, we're not actually interpreting the meaning of the word. So our use of equals is just to simplify things because to take that per chance idea and like codify it would involve randomization and a couple of other things that we just don't need to get into right now. Then from there, we divide them into individual substrings, also called Lexims. And then now we're ready to move on to our next part, which is evaluating.
Transcript
Play full episode
