Premium Only Content

Compiler From Scratch: Phase 1 - Tokenizer Generator 009: Generating DFA State code
Streamed on 2024-09-13 (https://www.twitch.tv/thediscouragerofhesitancy)
Zero Dependencies Programming!
I have to confess to cheating ... I did some work on the compiler off-stream. I created an asset pipeline for managing Code Snippets ... large chunks of text that are the same no matter what. These Code Snippets can be divided into chunks with a line of "...", so I can write a chunk, then emit custom code, and move on to the next chunk. The other thing was I refactored TxtBuf and ChrRef to remove the circular dependency that would necessarily exist if I use ChrRef in a TxtBuf::Iterator. And now I have a TxtBuff::Iterator. But that was all the cheating, I swear! Everything else we do on-stream.
Today was starting to emit DFAStates into our Tokenizer. Each DFAState has a list of Transitions and each Transition contains a list of conditions to check, and where the next DFAState is if those conditions are met. All the code emitted today was into simple methods, one per DFAState.
But a lot of the complexity we will face will come in the form of the PerformanceSwitches I am putting in. Whenever I come to a decision that I think will impact performance I am emitting code for each of those decisions, wrapped in a #ifdef/#endif so we can change some defines in our build and test the alternatives. There is some overlap between some of the options I've thought up, so we are going to be emitting way more code than we will end up with. As we go we will confirm correctness of each PerformanceSwitch, but we won't profile until things are working end-to-end and we have a large body of text to test against ... maybe JSON? So, if this stage of development seems a little slow, remember, I am generating 96 (currently) permutations of the way this code could be written.
-
1:25:02
Dear America
2 hours agoHere’s EVERYTHING We Know About The “White” NYC Shooter… + Trump Secures MAJOR Deal With EU!!
112K83 -
LIVE
Badlands Media
4 hours agoBadlands Daily: July 29, 2025
3,965 watching -
1:01:27
Part Of The Problem
12 hours agoDave Smith | The System is Broken | Part Of The Problem 1290
6.09K9 -
LIVE
Wendy Bell Radio
6 hours agoWhen Are "The Experts" Ever Right?
7,848 watching -
LIVE
JuicyJohns
2 hours ago $1.07 earned🟢#1 REBIRTH PLAYER 10.2+ KD🟢 !loadout
127 watching -
LIVE
The Mike Schwartz Show
2 hours agoTHE MIKE SCHWARTZ SHOW with DR. MICHAEL J SCHWARTZ 07-29-2025
4,898 watching -
1:09:29
Dialogue works
13 hours ago $0.34 earnedLarry C. Johnson & Pepe Escobar: Trump's Pressure Backfires, Russia Ignites Unstoppable BRICS Unity
8.51K8 -
1:31:45
Chicks On The Right
5 hours agoNYC Shooter, Trump Trolls UK PM, Swalwell’s TikTok Flop, & Maher Surprises Us | Chicks on the Right
27.4K7 -
1:05:09
Game On!
19 hours ago $2.59 earnedJustin Herbert LEADS Chargers to NFL Glory in 2025?
46.7K2 -
3:58:00
The Bubba Army
1 day ago4 KILLED in Manhattan Shooting Rampage - Bubba the Love Sponge® Show | 7/29/25
56.1K15