Premium Only Content

Compiler From Scratch: Phase 1 - Tokenizer Generator 014: Regex code gen/testing, starting lazy eval
Streamed on 2024-10-18 (https://www.twitch.tv/thediscouragerofhesitancy)
Zero Dependencies Programming!
Last week we had debugged the NFA/DFA not picking up the "ANY" operator correctly and fixed it up through the DFA. I didn't like the way it printed out in the DFA table or DFA graphviz plot, so I fixed that first thing. That was followed up with a small refactor of function names left over from last week's output improvements. Just a couple of small, easy tasks to get warmed up.
After that it was time to do code generation for the ANY operator, which was easier than expected. It was basically putting "true" inside the if condition. A little debugging around the Regex for strings and it was good to go. After that I generated code for the StartOfLine and EndOfLine anchors. The main effort was two functions added to the ChrRef, then that function was called from the tokenizer. Since the anchor operators don't consume a character there was a bit of fiddling to suppress the "munch" function call in these cases. Just a bit more finagling around detecting the end of input and all was well.
I tested a couple of possible error cases with the new processing in place. One turned out to be fine, the other caused a crash as expected (unterminated string at the end of input). This was noted in the TODO for future attention.
And finally it was time to start building out the lazy tokenizer processing. There were a few opportunities to refactor/cleanup other code as we went through this exercise. The high-level methods for doing the lazy processing are put together well enough to compile, but there are still changes needed to get the whole thing working end-to-end, mostly in the helper functions. We'll pick up there next week.
-
54:25
Inverted World Live
8 hours agoThe JFK Assassination and How the CIA Made the Zapruder Film | Ep. 70
35.5K11 -
10:11:25
SpartakusLIVE
15 hours ago$18,000 2v2 WZ Total Frenzy Tournament || #1 Comp Champ is BACK
92.5K4 -
8:11
MattMorseTV
1 day ago $18.18 earnedMacron just SURRENDERED.
81.6K99 -
1:35:03
Glenn Greenwald
11 hours agoTrump DOJ: There's Nothing to the Epstein Story; State Dept: Syria's Al-Qaeda are No Longer "Terrorists;" Trump & Lula Exchange Barbs Over Brazil | SYSTEM UPDATE #482
171K151 -
2:18:39
We Like Shooting
21 hours ago $7.90 earnedWe Like Shooting 618 (Gun Podcast)
48.5K -
2:42:17
Barry Cunningham
10 hours agoHERE'S WHY SOME IN MAGA ARE SO NEGATIVE ABOUT PRESIDENT TRUMP?
89.9K41 -
1:54:30
Joker Effect
8 hours agoWhy Gaming Streamers Aren't Successful and How To Fix It. Tim The Tatman was RIGHT... Kinda
44.7K3 -
3:43:25
megimu32
8 hours agoOTS: From Ozzy to Emo 🎸 The Rise & Fall of the Rockstar
29.2K2 -
1:22:50
Adam Does Movies
11 hours ago $1.60 earnedMovie News Live Taping! + AMA - LIVE!
27K -
1:47:11
EnDuEnDo
7 hours ago🚨Variety Stream 🎮 Push to 600 Followers 🚀 Chill Vibes 😎 Content Creator for @SelfMadeGGs & @SMKAcademy
17.5K1