Premium Only Content

Compiler From Scratch: Phase 1 - Tokenizer Generator 009: Generating DFA State code
Streamed on 2024-09-13 (https://www.twitch.tv/thediscouragerofhesitancy)
Zero Dependencies Programming!
I have to confess to cheating ... I did some work on the compiler off-stream. I created an asset pipeline for managing Code Snippets ... large chunks of text that are the same no matter what. These Code Snippets can be divided into chunks with a line of "...", so I can write a chunk, then emit custom code, and move on to the next chunk. The other thing was I refactored TxtBuf and ChrRef to remove the circular dependency that would necessarily exist if I use ChrRef in a TxtBuf::Iterator. And now I have a TxtBuff::Iterator. But that was all the cheating, I swear! Everything else we do on-stream.
Today was starting to emit DFAStates into our Tokenizer. Each DFAState has a list of Transitions and each Transition contains a list of conditions to check, and where the next DFAState is if those conditions are met. All the code emitted today was into simple methods, one per DFAState.
But a lot of the complexity we will face will come in the form of the PerformanceSwitches I am putting in. Whenever I come to a decision that I think will impact performance I am emitting code for each of those decisions, wrapped in a #ifdef/#endif so we can change some defines in our build and test the alternatives. There is some overlap between some of the options I've thought up, so we are going to be emitting way more code than we will end up with. As we go we will confirm correctness of each PerformanceSwitch, but we won't profile until things are working end-to-end and we have a large body of text to test against ... maybe JSON? So, if this stage of development seems a little slow, remember, I am generating 96 (currently) permutations of the way this code could be written.
-
29:24
Afshin Rattansi's Going Underground
15 hours agoWas Epstein a Mossad Agent? Will Obama go to Prison? (Afshin Rattansi vs Alan Dershowitz)
6.25K14 -
4:26:54
Nerdrotic
8 hours ago $25.38 earnedFantastic Four Baby Steps V Superman's James Gunn, South Park Returns | Friday Night Tights 364
86.5K5 -
LIVE
megimu32
2 hours agoOFF THE SUBJECT: FAFO Friday! Cops, Crash, Kombat & Chaos!
117 watching -
GrimmHollywood
11 hours ago🔴LIVE • GRIMM HOLLYWOOD • CLIP FARMING 101 •
7.3K1 -
1:07:56
Glenn Greenwald
8 hours agoIsrael-Made Famine Crisis Finally Recognized | SYSTEM UPDATE #493
95K51 -
2:29:42
TheSaltyCracker
3 hours agoGhislaine Maxwell Talks ReEEeStream 7-25-25
74.1K157 -
9:27
MattMorseTV
6 hours ago $2.31 earnedHe just lost EVERYTHING.
20.7K8 -
LIVE
a12cat34dog
5 hours agoSPOOKY ASS 2005 GAME :: F.E.A.R. :: FIRST-TIME FINISHING THIS CLASSIC {18+}
84 watching -
LIVE
WickedVirtue
2 hours agoFortnite w/ Friends
24 watching -
8:11:13
Spartan
8 hours agoSpartan - Pro Halo Player for OMiT | Scrims vs ETHR
17.2K1