Q&A #87 (2026-08-25)

In each Q&A video, I answer questions from the comments on the previous Q&A video, which can be from any part of the course.

The questions addressed in this video are:

• [00:03] “Would you agree or disagree that, early abstraction is the root of all evil? I find once I stopped doing that I can come up with much better quality abstractions.”
• [03:07] “I have trouble quantifying how much memory/cache layout actually matters for real-world performance, in contrast to something like SIMD where the 8x speedup is predictable, intuitive and easy to measure … ‘Fit in cache’ is one of the most common phrases in performance circles but at the same time fat structs are often recommended. Anton Mikhailov’s modeler benchmark found no substantial difference until structs exceeded ~1KB … For something like the ~100x gap between Visual Studio and RAD Debugger, how much of that actually comes from cache/allocation patterns specifically, versus general algorithmic/architectural overhead—unnecessary work, abstraction layers, etc.? I keep hearing ‘CPUs have improved by orders of magnitude while memory hasn’t’ and ‘Performance nowadays is mostly about memory/cache” but dont see how that matches my observations above.”
• [17:22] “I would love to practice all the concepts from this course. Any tips what kind of game / genre would naturally force me to write performant code? I wanna make something 2D since iam making it alone and Iam not good at modeling.”
• [22:24] “Do you think there is harm in learning standard data structures standard implementation? or would it be best to wait until the end of this course to be armed with maximum knowledge for how to properly implement them on real hardware to get real speed wins?”
• [27:54] “What do you think of the Steam Machine specs?”
• [28:02] “You’ve mentioned many times that you think AI companies are doing things in an unethical way by not compensating the original content creators explicitly for training, even if they pay to acquire access to the content. For an AI company to get on your “good side,” would they need to make models that are only based on completely open or non-copyrighted works and/or explicitly pay for AI training rights for copyrighted works?”
• [37:25] “You said you’ve personally never seen an heap data structure used in the video game programming industry. I thought video games often need to compute shortest paths, and the main algorithm for that, Dijkstra’s algorithm, is usually implemented with an heap. Am I mistaken in thinking video games often compute shortest paths ? Or is it that you use other algorithms which don’t require heap ? If so, which algorithms are often used in video games ?”
• [40:02] “When a CPU experiences an L1 cache miss but an L2 cache hit, does the data get copied into L1 before the CPU core uses it, or can the core read the data directly from L2?”
• [43:12] “Can you comment on the Cyclomatic Complexity metric (and maybe complexity analysis in general)?”
• [44:37] “What do you think would be a good project to apply performance aware programming to? I've seen the video editor Blick releasing recently and it looks super fast, are there other type of software that have a large gap in their performance and that someone could fix?”
• [51:02] "One of the things I’ve noticed in my own programming, as well as my coworkers, and dev logs on YouTube is that when people self evaluate their code solving a particular problem. There doesn’t seem to be much grounding on what actually worked, and what didn’t. Particularly with my own work, I tend to have rose tinted glasses about what I did, but then a few months later I start running into the limitations and edge cases and I’m no longer happy with it. Have you found this to be true with your own software projects? And do you have any advice or metrics to use to try to help evaluate the code early and get rid of the rose tinted glasses?”
• [53:00] “From the ‘Three Steps from Scalar to SIMD’ video: what is the difference between ‘Reduce’ and ‘Recode’?”
• [54:14] “I’m not really getting why the ‘Reservoir Sampling’ is better than caching all valid elements in an array and just doing a single random generation for the index. I get that probably the random-next function is just a bunch of adds muls mods or even xors (unless you use a Mersenne Twister or something). But if the list is short enough, and assuming the isValid predicate is simple, even the simpler RNGs can’t be faster than just a bunch of movs of indices or pointers pushed to the stack or copied into an array in a reserved page-aligned heap, and then one-shoting the index after.”