Compiler From Scratch: Phase 1 - Tokenizer Generator 019: Adding gotos for batch processing

Streamed on 2024-11-22 (https://www.twitch.tv/thediscouragerofhesitancy)

Zero Dependencies Programming!

With newfound confidence because of the new testing capabilities I started in on the goto version of batch processing. A lot of the existing code we have was reusable so it wasn't too long before we got something working. After that it was a matter of testing all the various build combinations, seeing where it broke and fixing it. Eventually it worked and was in the autmatic tests.

There were some more test cases I needed to add, so that's what I did. I added tests for empty strings, strings that only contain whitespace and comments, then some file reading (instead of passing the text directly in) and missing files. Several of the new tests revealed small fixes that needed to be made here and there.

By the end of the day I had 44 test suites were passing for all three encodings and 11 cases each for 1452 texts being tokenized each time we run the whole thing. Not too shabby.