<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
    <title>Petros Papatheodorou</title>
    <subtitle>Writing on mezura, nak and the tools around them.</subtitle>
    <link rel="self" type="application/atom+xml" href="https://petros-papatheodorou.dev/writing/atom.xml"/>
    <link rel="alternate" type="text/html" href="https://petros-papatheodorou.dev/writing"/>
    <generator uri="https://www.getzola.org/">Zola</generator>
    <updated>2026-11-16T00:00:00+00:00</updated>
    <id>https://petros-papatheodorou.dev/writing/atom.xml</id>
    <entry xml:lang="en">
        <title>Reinventing CSS progress() with container queries</title>
        <published>2026-11-16T00:00:00+00:00</published>
        <updated>2026-11-16T00:00:00+00:00</updated>
        
        <author>
          <name>Petros Papatheodorou</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://petros-papatheodorou.dev/writing/cq-lerp/"/>
        <id>https://petros-papatheodorou.dev/writing/cq-lerp/</id>
        
        <content type="html" xml:base="https://petros-papatheodorou.dev/writing/cq-lerp/">&lt;p&gt;A friend of mine, &lt;a rel=&quot;external&quot; href=&quot;https://www.zeuscss.com/&quot;&gt;Stavros Lazaris&lt;/a&gt;, released a CSS framework called Zeus. I installed it to see what he had built, and one idea in there led me to build an npm package before I realized the CSS spec already covered half of it.&lt;/p&gt;
&lt;h2 id=&quot;the-idea&quot;&gt;The idea&lt;a class=&quot;anchor&quot; href=&quot;#the-idea&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;Zeus has a fluid engine. Every spacing and type token is a &lt;code&gt;clamp()&lt;/code&gt; that scales between a min and a max as the container grows. The part that got me is the unit: it keys on &lt;code&gt;cqi&lt;/code&gt;, the container query inline unit, with &lt;code&gt;container-type: inline-size&lt;/code&gt; on &lt;code&gt;body&lt;/code&gt;. Tokens follow the container they live in. Put the same card in a sidebar and in a main column and each one sizes itself.&lt;/p&gt;
&lt;p&gt;The formula behind it is the one everybody uses, from Utopia to every fluid typography tutorial out there:&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;css&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;font-size&lt;/span&gt;&lt;span&gt;: clamp(1rem&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 0&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;667rem&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; +&lt;/span&gt;&lt;span&gt; 1&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;667cqi&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 2rem&lt;/span&gt;&lt;span&gt;);&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Read it as: 16px at a 320px container, 32px at 1280px, a straight line between, pinned at both ends. The two odd numbers are an intercept and a slope, worked out once by a preprocessor and pasted in.&lt;/p&gt;
&lt;p&gt;I have written that line hundreds of times, for &lt;code&gt;font-size&lt;/code&gt;, &lt;code&gt;padding&lt;/code&gt;, &lt;code&gt;gap&lt;/code&gt;, &lt;code&gt;border-radius&lt;/code&gt;. I had never once thought about where it stops working.&lt;/p&gt;
&lt;h2 id=&quot;where-it-stops-working&quot;&gt;Where it stops working&lt;a class=&quot;anchor&quot; href=&quot;#where-it-stops-working&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;I wanted a button that leans a few degrees on a narrow container and straightens out on a wide one. Same formula, different unit:&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;css&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;rotate: clamp(-7deg&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; -11&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;667deg&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; +&lt;/span&gt;&lt;span&gt; 1&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;458cqi&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 7deg&lt;/span&gt;&lt;span&gt;);&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That is a type error. &lt;code&gt;cqi&lt;/code&gt; is a length. You cannot add a length to an angle, so the browser drops the whole declaration and the button does nothing.&lt;/p&gt;
&lt;p&gt;The formula fails the same way for any non-length property: &lt;code&gt;transition-duration&lt;/code&gt; in seconds, &lt;code&gt;font-weight&lt;/code&gt; as a number, &lt;code&gt;opacity&lt;/code&gt;, or percentages inside &lt;code&gt;clip-path&lt;/code&gt;. Because the slope carries &lt;code&gt;cqi&lt;/code&gt;, the expression evaluates to a length, and CSS rejects it.&lt;/p&gt;
&lt;p&gt;The workaround we all know is a stack of media queries with hand-picked steps. Six breakpoints to walk a &lt;code&gt;font-weight&lt;/code&gt; from 400 to 620. I have written that too.&lt;/p&gt;
&lt;h2 id=&quot;swap-the-order-of-operations&quot;&gt;Swap the order of operations&lt;a class=&quot;anchor&quot; href=&quot;#swap-the-order-of-operations&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;The fix is a reordering. Divide first, multiply second.&lt;/p&gt;
&lt;p&gt;Dividing one length by another gives a plain number. A bare &lt;code&gt;/&lt;/code&gt; between two lengths only works in Chrome 140 and Safari 26, and Firefox has not shipped it, so the portable spelling is the &lt;code&gt;tan(atan2(a, b))&lt;/code&gt; trick from Jane Ori, which equals &lt;code&gt;a / b&lt;/code&gt; and has worked everywhere since early 2023:&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;css&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;--t: clamp(0&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; tan(atan2(100cqi - 320px&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 960px&lt;/span&gt;&lt;span&gt;)), 1);&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;--t&lt;/code&gt; is 0 at a 320px container, 1 at 1280px, a fraction between, and it has no unit. It is progress along the range, and the browser recomputes it on every layout.&lt;/p&gt;
&lt;p&gt;Now any property that takes &lt;code&gt;calc()&lt;/code&gt; can use it, in its own unit:&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;css&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;rotate:              calc(-7deg &lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;+&lt;/span&gt;&lt;span&gt; 14deg &lt;/span&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;*&lt;/span&gt;&lt;span&gt; var(--t&lt;/span&gt;&lt;span&gt;));&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;transition-duration&lt;/span&gt;&lt;span&gt;: calc(0&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;06s&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; +&lt;/span&gt;&lt;span&gt; 1&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;34s&lt;/span&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt; *&lt;/span&gt;&lt;span&gt; var(--t&lt;/span&gt;&lt;span&gt;));&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;font-weight&lt;/span&gt;&lt;span&gt;:         calc(200 &lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;+&lt;/span&gt;&lt;span&gt; 700 &lt;/span&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;*&lt;/span&gt;&lt;span&gt; var(--t&lt;/span&gt;&lt;span&gt;));&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;letter-spacing&lt;/span&gt;&lt;span&gt;:      calc(-0&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;02em&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; +&lt;/span&gt;&lt;span&gt; 0&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;37em&lt;/span&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt; *&lt;/span&gt;&lt;span&gt; var(--t&lt;/span&gt;&lt;span&gt;));&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;clip-path&lt;/span&gt;&lt;span&gt;:           polygon(calc(24% &lt;/span&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;*&lt;/span&gt;&lt;span&gt; var(--t&lt;/span&gt;&lt;span&gt;)) 0, 100% 0, &lt;/span&gt;&lt;span&gt;calc(100% - 24% &lt;/span&gt;&lt;span class=&quot;z-l-3 z-d-5&quot;&gt;*&lt;/span&gt;&lt;span&gt; var(--t&lt;/span&gt;&lt;span&gt;)) 100%, 0 100%);&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;One &lt;code&gt;--t&lt;/code&gt; for all of them. Declare it on a container and every descendant reads it. Declare it again on an inner container with its own range and everything inside follows the inner width. The &lt;code&gt;cqi&lt;/code&gt; inside the variable resolves against whichever container the value is used in, which is exactly what you want and slightly surprising the first time.&lt;/p&gt;
&lt;p&gt;A bonus I had not planned: descending ranges work. &lt;code&gt;calc(26rem - 24rem * var(--t))&lt;/code&gt; shrinks. The classic form cannot do that, because &lt;code&gt;clamp(26rem, ..., 2rem)&lt;/code&gt; returns 26rem forever when min is bigger than max. My own first helper had that bug, and I only caught it because I measured the computed width and got 26rem at every size. There is a bar on the demo page that shows the stuck version next to the working one.&lt;/p&gt;
&lt;h2 id=&quot;wrapping-it-in-sass&quot;&gt;Wrapping it in Sass&lt;a class=&quot;anchor&quot; href=&quot;#wrapping-it-in-sass&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;I got tired of doing subtraction in my head, so I wrote a few functions:&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;scss&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;@&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;use&lt;/span&gt;&lt;span class=&quot;z-l-2 z-d-6&quot;&gt; &amp;quot;&lt;/span&gt;&lt;span class=&quot;z-l-2 z-d-6&quot;&gt;cq-lerp&lt;/span&gt;&lt;span class=&quot;z-l-2 z-d-6&quot;&gt;&amp;quot;&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; as&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; *&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt;card&lt;/span&gt;&lt;span&gt; {&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;  @&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;include&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt; cq-track&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;240&lt;/span&gt;&lt;span&gt;, &lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;560&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;  padding&lt;/span&gt;&lt;span&gt;:&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;       fluid-on&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;0.6&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;rem&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; 1.4&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;rem&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;  border-radius&lt;/span&gt;&lt;span&gt;:&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; fluid-on&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;0.4&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;rem&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; 1&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;rem&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;  rotate&lt;/span&gt;&lt;span&gt;:&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;        fluid-on&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;-7&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;deg&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; 7&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;deg&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; var&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-9 z-d-12&quot;&gt;--cq-te&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;  font-weight&lt;/span&gt;&lt;span&gt;:&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;   fluid-on&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;400&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; 620&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; var&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-9 z-d-12&quot;&gt;--cq-t&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-2 z-d-6&quot;&gt; &amp;quot;&lt;/span&gt;&lt;span class=&quot;z-l-2 z-d-6&quot;&gt;&amp;quot;&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;cq-track&lt;/code&gt; declares two variables: &lt;code&gt;--cq-t&lt;/code&gt; (linear) and &lt;code&gt;--cq-te&lt;/code&gt; (run through a smoothstep, for things that should settle in gently). &lt;code&gt;fluid-on&lt;/code&gt; takes a min and a max in whatever unit, works out the difference and the sign, and writes the &lt;code&gt;calc()&lt;/code&gt;. &lt;code&gt;fluid()&lt;/code&gt; is the standalone version for when you want one value with its own range baked in.&lt;/p&gt;
&lt;p&gt;Then I built six buttons to test different unit families: padding and sizing lengths, rotation and skew angles, a variable font-weight from 200 to 900, percentages for border-radius and saturation, transition durations in seconds, and a polygon &lt;code&gt;clip-path&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://petros-papatheodorou.dev/writing/cq-lerp/components.webp&quot; alt=&quot;Six buttons, six unit families&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Then a whole app shell, twice. One stylesheet uses this. The other is what we all shipped until this year: &lt;code&gt;clamp()&lt;/code&gt; with &lt;code&gt;vw&lt;/code&gt; for every length, media queries for the font weight, and one fixed transition duration, because nobody writes six breakpoints for a duration. A switch flips between them in place.&lt;/p&gt;
&lt;p&gt;Live, resizable, no JavaScript: &lt;strong&gt;&lt;a rel=&quot;external&quot; href=&quot;https://subamanis.github.io/cq-lerp/&quot;&gt;subamanis.github.io/cq-lerp&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;At this point it covered every unit I needed without width breakpoints or keyframes. Layout changes (stacking, hiding, reordering) still need container or media queries, of course, but the continuous scaling part was handled by one function.&lt;/p&gt;
&lt;h2 id=&quot;then-i-read-the-spec&quot;&gt;Then I read the spec&lt;a class=&quot;anchor&quot; href=&quot;#then-i-read-the-spec&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;Before publishing, I checked whether someone had already done this. It turns out the CSS Working Group had:&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;css&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;--t: progress(100cqi&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 320px&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 1280px&lt;/span&gt;&lt;span&gt;);&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;CSS Values 5. It returns where the first argument sits between the other two, as a number from 0 to 1, clamped. It is my &lt;code&gt;--t&lt;/code&gt; line with the subtraction and division moved into the browser. Chrome 138, Safari 26, Firefox 155.&lt;/p&gt;
&lt;p&gt;I had reinvented &lt;code&gt;progress()&lt;/code&gt;, complete with a Sass wrapper.&lt;/p&gt;
&lt;p&gt;While I was at it I found &lt;a rel=&quot;external&quot; href=&quot;https://github.com/lunelson/sass-lerp&quot;&gt;&lt;code&gt;sass-lerp&lt;/code&gt;&lt;/a&gt;, which has done calc-based interpolation in Sass for years, and posts from &lt;a rel=&quot;external&quot; href=&quot;https://www.madebymike.com.au/writing/interpolation-without-animation/&quot;&gt;MadeByMike&lt;/a&gt; and &lt;a rel=&quot;external&quot; href=&quot;https://typetura.com/blog/unit-division-with-css-and-fallbacks/&quot;&gt;Typetura&lt;/a&gt; circling the same ground. The one piece I could still call mine was putting the halves together around a container unit and seeing that it opens every unit at once.&lt;/p&gt;
&lt;h2 id=&quot;why-it-still-needs-a-package&quot;&gt;Why it still needs a package&lt;a class=&quot;anchor&quot; href=&quot;#why-it-still-needs-a-package&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;There are two reasons the package is still useful: browser support and the interpolation math itself.&lt;/p&gt;
&lt;p&gt;The first is support. Those version numbers mean browsers from 2026 onward. Everything older gets nothing from &lt;code&gt;progress()&lt;/code&gt;. The &lt;code&gt;tan(atan2())&lt;/code&gt; fallback reaches back to early 2023 (Chrome 111, Safari 16, Firefox 110). So the package declares the fallback and upgrades in place:&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;css&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;--cq-t: clamp(0&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; tan(atan2(100cqi - 320px&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 960px&lt;/span&gt;&lt;span&gt;)), 1);&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;@&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;supports&lt;/span&gt;&lt;span&gt; (&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;opacity&lt;/span&gt;&lt;span&gt;:&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; progress&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;px&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; 0&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;px&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt; 2&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;px&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt; {&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  --cq-t: progress(100cqi&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 320px&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; 1280px&lt;/span&gt;&lt;span&gt;);&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;}&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;One catch with that feature test: my first version was &lt;code&gt;@supports (width: progress(...))&lt;/code&gt;, which is always false, because &lt;code&gt;progress()&lt;/code&gt; returns a number and &lt;code&gt;width&lt;/code&gt; rejects numbers. The upgrade never fired once and I did not notice until I measured. Test on &lt;code&gt;opacity&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The second reason is that &lt;code&gt;progress()&lt;/code&gt; hands you a 0 to 1 and stops there. The &lt;code&gt;calc(min + (max - min) * t)&lt;/code&gt; is still yours to write, in the right unit, with the right sign. The Sass functions do that part, catch a &lt;code&gt;rem&lt;/code&gt; mixed with a &lt;code&gt;px&lt;/code&gt;, and turn a bare &lt;code&gt;0&lt;/code&gt; into &lt;code&gt;0deg&lt;/code&gt; when the other end is an angle.&lt;/p&gt;
&lt;h2 id=&quot;the-part-progress-cannot-reach&quot;&gt;The part progress() cannot reach&lt;a class=&quot;anchor&quot; href=&quot;#the-part-progress-cannot-reach&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;progress()&lt;/code&gt; interpolates over whatever you give it. Give it &lt;code&gt;100vw&lt;/code&gt; and you have the viewport, same as the &lt;code&gt;clamp()&lt;/code&gt;-and-&lt;code&gt;vw&lt;/code&gt; genre. Give it &lt;code&gt;100cqi&lt;/code&gt; inside a container and the thing being measured changes.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://petros-papatheodorou.dev/writing/cq-lerp/sidebar.webp&quot; alt=&quot;Dragging the sidebar rescales its contents while the window stays put&quot; loading=&quot;lazy&quot; decoding=&quot;async&quot; /&gt;&lt;/p&gt;
&lt;p&gt;That sidebar has a drag handle. The window never moves. Under cq-lerp the padding, the type size, the letter spacing and the weight of the links all follow the width of the sidebar itself. Under the conventional stylesheet nothing moves, because every viewport-based technique is reading a number that did not change.&lt;/p&gt;
&lt;p&gt;Same story with the three cards on the demo page: identical markup in a 240px, a 380px and a 520px slot, on one screen, at the same time. Each card sizes itself by its slot. A media query has nothing to key off, since all three see the same viewport.&lt;/p&gt;
&lt;p&gt;That is what stays useful no matter how far &lt;code&gt;progress()&lt;/code&gt; support spreads, and it is why the package has &lt;code&gt;cq&lt;/code&gt; in its name.&lt;/p&gt;
&lt;h2 id=&quot;links&quot;&gt;Links&lt;a class=&quot;anchor&quot; href=&quot;#links&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Demo: &lt;a rel=&quot;external&quot; href=&quot;https://subamanis.github.io/cq-lerp/&quot;&gt;subamanis.github.io/cq-lerp&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Source: &lt;a rel=&quot;external&quot; href=&quot;https://github.com/subamanis/cq-lerp&quot;&gt;github.com/subamanis/cq-lerp&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Package: &lt;code&gt;npm i cq-lerp&lt;/code&gt;, with a plain CSS entry at &lt;code&gt;cq-lerp/css&lt;/code&gt; if you do not use Sass&lt;/li&gt;
&lt;li&gt;Zeus, where the container-based fluid engine idea comes from: &lt;a rel=&quot;external&quot; href=&quot;https://www.zeuscss.com/&quot;&gt;zeuscss.com&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content>
        
    </entry>
    <entry xml:lang="en">
        <title>What it takes to count the Linux kernel&#39;s 36M lines of code in under 70ms</title>
        <published>2026-11-16T00:00:00+00:00</published>
        <updated>2026-11-16T00:00:00+00:00</updated>
        
        <author>
          <name>Petros Papatheodorou</name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://petros-papatheodorou.dev/writing/mezura/counting-the-linux-kernel/"/>
        <id>https://petros-papatheodorou.dev/writing/mezura/counting-the-linux-kernel/</id>
        
        <content type="html" xml:base="https://petros-papatheodorou.dev/writing/mezura/counting-the-linux-kernel/">&lt;p&gt;Counting lines of code sounds like &lt;code&gt;wc -l&lt;/code&gt; with a few &lt;code&gt;if&lt;/code&gt; statements. Doing it properly on the Linux kernel in under 70 milliseconds demands a lot more from an 8-core desktop CPU.&lt;/p&gt;
&lt;p&gt;In that time the program walks ~5,900 directories holding ~88,800 files and picks out the ~63,700 source files to count. It opens and reads all 1.37 GB of them from a warm page cache, which works out to roughly 20 GB/s. It validates UTF-8 and settles ambiguous extensions like &lt;code&gt;.h&lt;/code&gt; from the file&#39;s content. Then it lexes 36 million lines, tracking escaped quotes, raw strings and block comments, to tell code from comments and blanks.&lt;/p&gt;
&lt;p&gt;(The tree is Linux v6.15 at commit &lt;code&gt;0ff41df1&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;This post goes through what it took to bring &lt;a rel=&quot;external&quot; href=&quot;https://github.com/subamanis/mezura&quot;&gt;mezura&lt;/a&gt; from about 0.4 s down to 67 ms on that workload:&lt;/p&gt;
&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th style=&quot;text-align: left&quot;&gt;Kernel run on Linux&lt;/th&gt;&lt;th style=&quot;text-align: left&quot;&gt;What got it there&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;about 0.4 s&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;version 2.0.0: &lt;code&gt;str::lines()&lt;/code&gt;, a delimiter search per line, a cleaned copy of every line&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;166 ms&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;still safe Rust only: byte ranges in place of the copies, files handed over in batches&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;88 ms&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;one SIMD pass per 64-byte block, &lt;code&gt;mmap&lt;/code&gt; for large files, the first &lt;code&gt;unsafe&lt;/code&gt;&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;67 ms&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;plain lines classified a block at a time, the hot loop compiled per CPU tier, one lexer&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;41 ms&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;the hardware floor (walk, read, and a simplistic parser)&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Along the way there was hyperfixation on local minima, profiling tools that pointed the wrong way (cutting branch misses by 14% made the program 12% slower), a Windows filter driver that made one drive letter a third slower than the other, and a lexer that got faster and more accurate in the same change.&lt;/p&gt;
&lt;h2 id=&quot;1-the-hardware-floor-as-a-compass-don-t-get-trapped-in-local-minima&quot;&gt;1. The hardware floor as a compass (don&#39;t get trapped in local minima)&lt;a class=&quot;anchor&quot; href=&quot;#1-the-hardware-floor-as-a-compass-don-t-get-trapped-in-local-minima&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;If you optimize only by looking at flamegraphs of your existing code, it is easy to think you are done too early. Once you fix the obvious bottlenecks and no single function dominates the profile, every change starts taking days of work for a 1% or 2% gain. That feels like the limit of the hardware, when most of the time it is a local minimum of your current architecture.&lt;/p&gt;
&lt;p&gt;To know whether you are fast or stuck in a bad design, you need the hardware floor for your problem. For a line counter that is what the machine needs to walk directories, read files from the page cache, and scan bytes in RAM.&lt;/p&gt;
&lt;p&gt;A floor program can have design flaws too, so I kept mine as close to the system calls as I could. It is a single C file (&lt;code&gt;gcc -O3 -march=native&lt;/code&gt;) on &lt;code&gt;opendir&lt;/code&gt;/&lt;code&gt;readdir&lt;/code&gt;, &lt;code&gt;open&lt;/code&gt;, &lt;code&gt;read&lt;/code&gt;, &lt;code&gt;mmap&lt;/code&gt;, &lt;code&gt;pthread&lt;/code&gt; and &lt;code&gt;&amp;lt;immintrin.h&amp;gt;&lt;/code&gt;, with static buffers and no heap allocation in the hot path. I measured it in four steps over the kernel tree on a Ryzen 7 9700X (8 cores, 16 hardware threads, Debian 13, &lt;code&gt;ext4&lt;/code&gt;):&lt;/p&gt;
&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th style=&quot;text-align: left&quot;&gt;Step&lt;/th&gt;&lt;th style=&quot;text-align: left&quot;&gt;What it does&lt;/th&gt;&lt;th style=&quot;text-align: left&quot;&gt;Time, 16 threads&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;1. Memory bandwidth&lt;/strong&gt;&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;The 1.37 GB streamed through memory once, at the ~60 GB/s this machine sustains&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;23 ms, computed from the measured rate&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;2. File I/O, no directory walk&lt;/strong&gt;&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;Every path known in advance, each file opened, read (&lt;code&gt;read&lt;/code&gt; under 256 KiB, &lt;code&gt;mmap&lt;/code&gt; from there) and closed&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;30.4 ms&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;3. Directory walk + file reads&lt;/strong&gt;&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;A concurrent &lt;code&gt;readdir&lt;/code&gt; walk while every matching file is opened and read, with no byte inspected&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;35.9 ms best, 36.5 median&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;4. Walk + read + SIMD + state machine&lt;/strong&gt;&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;Step 3 plus one SIMD pass per 64-byte block and a minimal state machine over the 15.8M bytes where a comment or a string can begin or end&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;40.8 ms best, 42.3 median&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Isolated costs do not add up linearly when they run together. On its own across 16 threads, the directory walk takes 2.9 ms, and &lt;code&gt;open&lt;/code&gt; plus &lt;code&gt;close&lt;/code&gt; on every file takes 14.0 ms. Running that 2.9 ms walk alongside the 30.4 ms file reads takes 35.9 ms rather than 33.3 ms, because walkers and readers contend for the same cores and kernel VFS structures. A realistic floor has to run everything concurrently.&lt;/p&gt;
&lt;p&gt;Running Step 4 with every path known in advance and no directory walk at all took 38.2 ms, and a single pool where every thread both walks and reads landed within a millisecond of separate walker and reader pools (40.8 ms vs 41.1 ms). Neither the walk nor the thread layout was a bottleneck, so a line counter on this machine cannot go much below &lt;strong&gt;41 to 42 ms&lt;/strong&gt; on this tree.&lt;/p&gt;
&lt;p&gt;I wrote this program late, when &lt;code&gt;mezura&lt;/code&gt; already ran the kernel in 88 ms, after I had spent a lot of time profiling the code and just before I convinced myself that it ain&#39;t getting any faster. No single function stood out in the profile by then, and 88 ms looked like a natural place to stop. The floor showed that more than half of that run was still my own code.&lt;/p&gt;
&lt;h2 id=&quot;2-how-to-measure-67-ms-and-how-profiling-tools-mislead&quot;&gt;2. How to measure 67 ms, and how profiling tools mislead&lt;a class=&quot;anchor&quot; href=&quot;#2-how-to-measure-67-ms-and-how-profiling-tools-mislead&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;At tens of milliseconds across 16 threads, casual wall-clock timing is noisy. The machine&#39;s speed drifts during a session, and reversing the order of two commands moved individual means by 1% to 10%.&lt;/p&gt;
&lt;p&gt;To control for that I wrote &lt;a rel=&quot;external&quot; href=&quot;https://github.com/loc-conformance/linebench&quot;&gt;linebench&lt;/a&gt;, a benchmark harness for line counters. It runs every table in both command orders and pools the two passes, checks the background CPU load and the power settings first, and times one control binary at the start and at the end of the session to show how far the machine drifted. It also provides useful performance insights like syscall breakdowns, hardware PMU counters, heap allocation counts, memory curves, core-scaling ladders, and cold-cache runs. It lives under the &lt;code&gt;loc-conformance&lt;/code&gt; organization so that the authors of other counters can join the development, though today I maintain it by myself.&lt;/p&gt;
&lt;h3 id=&quot;where-the-hardware-numbers-pointed-the-wrong-way&quot;&gt;Where the hardware numbers pointed the wrong way&lt;a class=&quot;anchor&quot; href=&quot;#where-the-hardware-numbers-pointed-the-wrong-way&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;A flamegraph shows where time is spent, not how much time you get back if you delete that code. Three times I removed a small operation taking 2% to 5% of flamegraph samples, and wall time did not move at all. Out-of-order cores pipeline cheap, frequent instructions alongside surrounding work, so deleting them frees zero cycles; only removing whole function calls consistently saved time.&lt;/p&gt;
&lt;p&gt;Branch misses were a decoy. I replaced the byte loops that trim whitespace on every line with branchless word-at-a-time (SWAR) arithmetic. Branch misses fell by 14%, instructions rose by 16%, and the program got &lt;strong&gt;12% slower&lt;/strong&gt;. The byte loops were fast simply because they were short, whereas the branchless math saved 0.17 branch misses per line at the cost of roughly 75 extra instructions on every line.&lt;/p&gt;
&lt;h3 id=&quot;elapsed-time-on-a-preempted-thread-is-not-cpu-cost&quot;&gt;Elapsed time on a preempted thread is not CPU cost&lt;a class=&quot;anchor&quot; href=&quot;#elapsed-time-on-a-preempted-thread-is-not-cpu-cost&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;mezura&lt;/code&gt; splits work between producer threads that walk directories and consumer threads that steal batches of file paths to open, read, and parse. Instrumenting both pools with per-thread phase timers (waiting for work, opening, reading, parsing) produced two readings that sent me down the wrong path.&lt;/p&gt;
&lt;p&gt;On Linux the timers showed the directory walkers alive for 90 ms of a run of about 110 ms, so the walk looked slow. Measured on its own, the same walk finishes in 13 ms. The 4 walker threads were just getting preempted by 32 consumer threads that kept all 16 hardware threads busy.&lt;/p&gt;
&lt;p&gt;On Windows the timers put &lt;code&gt;File::open&lt;/code&gt; at 62% to 70% of all consumer time, which made a three-stage pipeline of walkers, openers and parsers look worthwhile. In CPU samples the same call was 17%. The difference was scheduling delay, since a thread preempted inside &lt;code&gt;CreateFileW&lt;/code&gt; keeps its timer running.&lt;/p&gt;
&lt;h3 id=&quot;the-syscalls-no-profile-showed&quot;&gt;The syscalls no profile showed&lt;a class=&quot;anchor&quot; href=&quot;#the-syscalls-no-profile-showed&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;perf stat&lt;/code&gt; counted 66,831 &lt;code&gt;sched_yield&lt;/code&gt; and 18,628 &lt;code&gt;clock_nanosleep&lt;/code&gt; calls in a single run (23% of all syscalls &lt;code&gt;mezura&lt;/code&gt; made), all from worker threads waiting on queues. None of them showed up in CPU flamegraphs (0.00% of samples) because sleeping threads are off-CPU. At the same time, phase timers showed the first consumer finishing 25 ms before the last one, so I let idle consumers steal files from busy ones instead of sleeping.&lt;/p&gt;
&lt;p&gt;The exit gap dropped from 25 ms to 0.3 ms, yet wall time failed to improve at any thread count and got &lt;strong&gt;4% slower&lt;/strong&gt; at the default 64-consumer setup. Idle threads were now spinning over everyone else&#39;s queues on sibling hyperthreads, stealing execution resources from the working thread on the same physical core, while the real tail was a single thread parsing a 24 MB generated AMD GPU register header that could not be split anyway.&lt;/p&gt;
&lt;p&gt;Work stealing between consumers only paid off later, once the default consumer count dropped from 64 to 32 and stealing was restricted to run only after directory walking had finished (taking the kernel run from 93.0 ms to 88.5 ms).&lt;/p&gt;
&lt;h2 id=&quot;3-file-i-o-on-linux-vs-windows&quot;&gt;3. File I/O on Linux vs Windows&lt;a class=&quot;anchor&quot; href=&quot;#3-file-i-o-on-linux-vs-windows&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;p&gt;The same machine gives two different profiles under Linux (&lt;code&gt;ext4&lt;/code&gt;) and Windows 11 (&lt;code&gt;NTFS&lt;/code&gt;). On Linux the 67 ms run spends 0.62 s of CPU in user code and 0.28 s in the kernel, summed over its threads, so it is bound by the parser. On Windows the same run takes 175 ms and spends 1.14 s in user code and 1.30 s in the kernel. More than half of its CPU time is inside the kernel, where files are opened, read and closed and directories are listed.&lt;/p&gt;
&lt;h3 id=&quot;avoiding-63-934-statx-calls-on-linux&quot;&gt;Avoiding &lt;code&gt;63,934&lt;/code&gt; &lt;code&gt;statx&lt;/code&gt; calls on Linux&lt;a class=&quot;anchor&quot; href=&quot;#avoiding-63-934-statx-calls-on-linux&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Rust&#39;s standard library reads a file to completion by asking the OS for its size and then calling &lt;code&gt;read&lt;/code&gt; twice, the second call returning &lt;code&gt;0&lt;/code&gt; to confirm the end of the file.&lt;/p&gt;
&lt;p&gt;On Windows, &lt;code&gt;FindNextFileW&lt;/code&gt; already returns the file size inside each directory entry, so the walk has it for free. A buffer of &lt;code&gt;size + 1&lt;/code&gt; bytes then takes the whole file and confirms its end in one &lt;code&gt;read&lt;/code&gt; call, with no metadata call.&lt;/p&gt;
&lt;p&gt;On Linux, &lt;code&gt;getdents64&lt;/code&gt; returns the inode and the file type (&lt;code&gt;d_type&lt;/code&gt;) and no size. Asking &lt;code&gt;DirEntry::metadata()&lt;/code&gt; for it during the walk costs one &lt;code&gt;statx&lt;/code&gt; per file, 63,934 of them on the kernel tree.&lt;/p&gt;
&lt;p&gt;So on Linux the walkers pass a size of &lt;code&gt;0&lt;/code&gt;, and each consumer issues one 64 KiB &lt;code&gt;read&lt;/code&gt; into a reusable thread-local buffer. 95.7% of the kernel&#39;s files are smaller than that, so the single &lt;code&gt;read&lt;/code&gt; comes back short, giving the full content, its exact length, and proof of &lt;code&gt;EOF&lt;/code&gt; in one syscall. Only the 2,778 files that fill the window call &lt;code&gt;fstat&lt;/code&gt;. Files of 256 KiB and larger are mapped with &lt;code&gt;mmap&lt;/code&gt;, which cut wall time by 9% and page faults in half. Files modified in the last five seconds fall back to &lt;code&gt;read&lt;/code&gt;, since truncating a mapped file mid-scan kills the process with &lt;code&gt;SIGBUS&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id=&quot;the-banshees-of-d-the-ghost-filters-haunting-windows-i-o&quot;&gt;The Banshees of &lt;code&gt;D:\&lt;/code&gt; (the ghost filters haunting Windows I/O)&lt;a class=&quot;anchor&quot; href=&quot;#the-banshees-of-d-the-ghost-filters-haunting-windows-i-o&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Across 16 threads on Windows, opening and closing a single file costs roughly 19 µs (13.7 µs in &lt;code&gt;CreateFileW&lt;/code&gt;, 5.4 µs in &lt;code&gt;CloseHandle&lt;/code&gt;). Trying to bypass Win32 path handling with lower-level NT calls went nowhere. In a standalone benchmark over the kernel&#39;s 88,421 files, directory-relative &lt;code&gt;NtCreateFile&lt;/code&gt; and minimal &lt;code&gt;FILE_READ_DATA&lt;/code&gt; access masks landed within 3% of plain &lt;code&gt;File::open&lt;/code&gt;. Opening files directly by NTFS file ID actually made &lt;code&gt;open&lt;/code&gt; &lt;strong&gt;1.9x slower&lt;/strong&gt; (and the whole run nearly 1.5x slower), because a file opened by ID carries no path string, forcing the filesystem filter drivers on the volume to reconstruct the path from scratch. Whichever API you call on Windows, every open still walks through security checks, handle creation, and every minifilter driver attached to that drive.&lt;/p&gt;
&lt;p&gt;That filter stack turned out to be behind a much weirder mystery on my machine. Counting the exact same warm Linux kernel checkout on &lt;code&gt;D:\&lt;/code&gt; was taking &lt;strong&gt;36% longer&lt;/strong&gt; than on &lt;code&gt;C:\&lt;/code&gt; (611 ms vs 450 ms in controlled back-to-back runs, burning 4.8 s of kernel CPU against 2.9 s). How does a different drive letter burn almost two extra seconds of kernel CPU when the entire repository is already sitting in RAM?&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Slower SSD on &lt;code&gt;D:\&lt;/code&gt;?&lt;/strong&gt; Sampled across 40 warm runs, physical disk reads were &lt;strong&gt;zero&lt;/strong&gt;. Neither drive was being touched.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Windows Defender scanning &lt;code&gt;D:\&lt;/code&gt;?&lt;/strong&gt; &lt;code&gt;mezura.exe&lt;/code&gt; was already excluded by process name, and &lt;code&gt;D:\dev&lt;/code&gt; had a full folder exclusion on top, making &lt;code&gt;D:\&lt;/code&gt; &lt;em&gt;more&lt;/em&gt; exempt from Defender than &lt;code&gt;C:\&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;NTFS formatting differences?&lt;/strong&gt; Cluster size, indexing, compression, and &lt;code&gt;8.3&lt;/code&gt; name generation were identical on both volumes.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Running &lt;code&gt;fltmc instances&lt;/code&gt; in an elevated prompt finally caught it. &lt;code&gt;D:\&lt;/code&gt; had two extra Microsoft minifilter drivers sitting on every file open and directory listing that &lt;code&gt;C:\&lt;/code&gt; did not have, &lt;strong&gt;&lt;code&gt;wcifs&lt;/code&gt;&lt;/strong&gt; (Windows Container Isolation) and &lt;strong&gt;&lt;code&gt;FileCrypt&lt;/code&gt;&lt;/strong&gt;. Months earlier, I had changed Windows&#39;s &lt;em&gt;&quot;Where new content is saved&quot;&lt;/em&gt; setting so Windows Store apps and movies would install on &lt;code&gt;D:&lt;/code&gt; instead of &lt;code&gt;C:&lt;/code&gt;, and Windows had quietly wired container and DRM encryption filters into every I/O call on the drive! Detaching &lt;code&gt;wcifs&lt;/code&gt; alone did nothing, but the moment I ran &lt;code&gt;fltmc detach FileCrypt D:&lt;/code&gt;, kernel CPU time dropped in half (4.9 s to 2.5 s) and &lt;code&gt;D:\&lt;/code&gt; actually beat &lt;code&gt;C:\&lt;/code&gt; by 10%.&lt;/p&gt;
&lt;p&gt;The best part is that you cannot evict them cleanly once they move in. Moving every Windows Store save setting back to &lt;code&gt;C:&lt;/code&gt;, unregistering the volume with &lt;code&gt;Remove-AppxVolume&lt;/code&gt;, and taking ownership of &lt;code&gt;D:\WindowsApps&lt;/code&gt; just to delete it made no difference. On the very next reboot, Windows silently re-attached both &lt;code&gt;wcifs&lt;/code&gt; and &lt;code&gt;FileCrypt&lt;/code&gt; to &lt;code&gt;D:\&lt;/code&gt; anyway. To this day, the only thing keeping &lt;code&gt;D:\&lt;/code&gt; clean is a &lt;code&gt;SYSTEM&lt;/code&gt; scheduled task at boot that runs &lt;code&gt;fltmc detach&lt;/code&gt; on both filters!&lt;/p&gt;
&lt;p&gt;Right alongside that, Windows Defender pulled a second trick during A/B benchmarking. For weeks, every time I copied &lt;code&gt;mezura.exe&lt;/code&gt; to &lt;code&gt;candidate.exe&lt;/code&gt; or &lt;code&gt;control.exe&lt;/code&gt; to test an optimization, the copy ran &lt;strong&gt;24% to 32% slower&lt;/strong&gt; on both drives, sending me looking for some weird Windows penalty on freshly compiled binaries. Then I copied the file into another folder while keeping the name &lt;code&gt;mezura.exe&lt;/code&gt;, and the slowdown vanished. Defender&#39;s process exclusions match the bare filename, so &lt;code&gt;mezura.exe&lt;/code&gt; bypassed real-time scanning on all 63,700 file opens while &lt;code&gt;candidate.exe&lt;/code&gt; got inspected on every single one.&lt;/p&gt;
&lt;h2 id=&quot;4-evolution-of-the-parser-0-4-s-166-ms-88-ms-67-ms&quot;&gt;4. Evolution of the parser (&lt;code&gt;0.4 s&lt;/code&gt; -&amp;gt; &lt;code&gt;166 ms&lt;/code&gt; -&amp;gt; &lt;code&gt;88 ms&lt;/code&gt; -&amp;gt; &lt;code&gt;67 ms&lt;/code&gt;)&lt;a class=&quot;anchor&quot; href=&quot;#4-evolution-of-the-parser-0-4-s-166-ms-88-ms-67-ms&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;h3 id=&quot;stage-1-0-4-s-to-166-ms&quot;&gt;Stage 1 (&lt;code&gt;0.4 s&lt;/code&gt; to &lt;code&gt;166 ms&lt;/code&gt;)&lt;a class=&quot;anchor&quot; href=&quot;#stage-1-0-4-s-to-166-ms&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Version 2.0.0 was straightforward multithreaded Rust. It iterated over &lt;code&gt;str::lines()&lt;/code&gt;, searched each line for comment and string delimiters with &lt;code&gt;memchr&lt;/code&gt; and &lt;code&gt;memmem&lt;/code&gt;, and allocated a cleaned copy of every line to search for keywords. On the kernel tree that took about 0.4 s on Linux and roughly a second on Windows.&lt;/p&gt;
&lt;p&gt;Replacing those per-line string copies with byte-range slices, stealing files in batches, and switching to a depth-first directory walk brought the Linux run down to &lt;strong&gt;166 ms&lt;/strong&gt; (22.4G instructions), still in 100% safe Rust and still four times slower than the 41 ms floor.&lt;/p&gt;
&lt;h3 id=&quot;stage-2-166-ms-to-88-ms-64-byte-bitmasks-and-targeted-unsafe&quot;&gt;Stage 2 (&lt;code&gt;166 ms&lt;/code&gt; to &lt;code&gt;88 ms&lt;/code&gt;, 64-byte bitmasks and targeted &lt;code&gt;unsafe&lt;/code&gt;)&lt;a class=&quot;anchor&quot; href=&quot;#stage-2-166-ms-to-88-ms-64-byte-bitmasks-and-targeted-unsafe&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Up to 166 ms the entire workspace used &lt;code&gt;#![forbid(unsafe_code)]&lt;/code&gt;. Going further required explicit SIMD intrinsics and &lt;code&gt;mmap&lt;/code&gt;, so I kept &lt;code&gt;#![forbid(unsafe_code)]&lt;/code&gt; on the binaries (&lt;code&gt;mezura&lt;/code&gt; and &lt;code&gt;mezura-mcp&lt;/code&gt;) and switched the core library to &lt;code&gt;#![deny(unsafe_code)]&lt;/code&gt;, allowing individual &lt;code&gt;unsafe&lt;/code&gt; blocks only when they gave a clear wall-clock speedup.&lt;/p&gt;
&lt;p&gt;Instead of splitting a file into lines first and scanning each line for delimiters afterward, the parser now runs one &lt;code&gt;AVX2&lt;/code&gt; (&lt;code&gt;x86_64&lt;/code&gt;) or &lt;code&gt;NEON&lt;/code&gt; (&lt;code&gt;aarch64&lt;/code&gt;) pass per 64-byte block to produce two &lt;code&gt;u64&lt;/code&gt; bitmasks, &lt;code&gt;newlines&lt;/code&gt; and &lt;code&gt;candidates&lt;/code&gt; (the bytes where a comment, string, or escape can start or end). I prototyped the SIMD routine with Claude&#39;s help and tested it against a scalar implementation on random inputs. That change alone cut wall time by &lt;strong&gt;15%&lt;/strong&gt; and user CPU by &lt;strong&gt;20%&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;On the Linux kernel, 87.6% of lines contain neither &lt;code&gt;&quot;&lt;/code&gt; nor &lt;code&gt;/&lt;/code&gt;. Whenever no multiline comment or string is open, those plain lines do not need delimiter scanning. I also added an &lt;code&gt;AVX2&lt;/code&gt; ASCII check so pure-ASCII files skip UTF-8 validation via &lt;code&gt;str::from_utf8_unchecked&lt;/code&gt; (-3.2% instructions), and used const generics to compile separate copies of the file loop so files without heredocs, nested languages, or inline tests do not pay branches for those features. Together with &lt;code&gt;mmap&lt;/code&gt; on large files, the run came down to &lt;strong&gt;88 ms&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Not every &lt;code&gt;unsafe&lt;/code&gt; optimization worked. Using &lt;code&gt;from_utf8_unchecked&lt;/code&gt; on individual line slices cut instructions by 7% while increasing CPU cycles by &lt;strong&gt;8.6%&lt;/strong&gt; (IPC dropped from 2.25 to 1.93 because of code-layout changes in the hot loop), so I threw it out.&lt;/p&gt;
&lt;h3 id=&quot;stage-3-88-ms-to-67-ms-carry-ripple-math-cpu-tiers-and-a-table-lexer&quot;&gt;Stage 3 (&lt;code&gt;88 ms&lt;/code&gt; to &lt;code&gt;67 ms&lt;/code&gt;, carry-ripple math, CPU tiers, and a table lexer)&lt;a class=&quot;anchor&quot; href=&quot;#stage-3-88-ms-to-67-ms-carry-ripple-math-cpu-tiers-and-a-table-lexer&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;By default, &lt;code&gt;mezura&lt;/code&gt; groups punctuation-only lines (like a lone &lt;code&gt;}&lt;/code&gt;) with blank lines rather than counting them as code, whereas &lt;code&gt;--counting region&lt;/code&gt; counts any non-blank line as code. Because the engine records all three counts (&lt;code&gt;blank&lt;/code&gt;, &lt;code&gt;punctuation-only&lt;/code&gt;, and &lt;code&gt;word-bearing&lt;/code&gt;) in one pass, the parser still looped over every plain line to trim whitespace and look for word characters.&lt;/p&gt;
&lt;p&gt;My first try added two more &lt;code&gt;AVX2&lt;/code&gt; masks per 64-byte block, &lt;code&gt;words&lt;/code&gt; and &lt;code&gt;content&lt;/code&gt; (non-whitespace bytes), while keeping the per-line loop. Computing those masks took 37 instructions per block, raised kernel instructions by 10% to 18%, and made the 454 MB of 107-byte-line AMD GPU register headers in the tree &lt;strong&gt;34% slower&lt;/strong&gt;.&lt;/p&gt;
&lt;h4 id=&quot;classifying-a-whole-block-of-lines-with-binary-addition&quot;&gt;Classifying a whole block of lines with binary addition&lt;a class=&quot;anchor&quot; href=&quot;#classifying-a-whole-block-of-lines-with-binary-addition&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h4&gt;
&lt;p&gt;Computing &lt;code&gt;words&lt;/code&gt; and &lt;code&gt;content&lt;/code&gt; masks only pays off if you stop looping over individual plain lines. In &lt;code&gt;u64&lt;/code&gt; bitmask form, &lt;code&gt;!newlines&lt;/code&gt; has &lt;code&gt;1&lt;/code&gt; at every byte inside a line and &lt;code&gt;0&lt;/code&gt; at every &lt;code&gt;\n&lt;/code&gt;. Adding &lt;code&gt;words&lt;/code&gt; to &lt;code&gt;!newlines&lt;/code&gt; ripples a binary carry from any word byte in a line up through those &lt;code&gt;1&lt;/code&gt;s and flips the &lt;code&gt;0&lt;/code&gt; bit at that line&#39;s &lt;code&gt;\n&lt;/code&gt; to &lt;code&gt;1&lt;/code&gt;. If a line has no word bytes, no carry reaches its &lt;code&gt;\n&lt;/code&gt; and that bit stays &lt;code&gt;0&lt;/code&gt; (with bit 63&#39;s carry-out passed into the next 64-byte block):&lt;/p&gt;
&lt;pre class=&quot;giallo z-l-code z-d-code&quot; &gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;let&lt;/span&gt;&lt;span&gt; not_newlines&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; =&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; !&lt;/span&gt;&lt;span&gt;newlines&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;let&lt;/span&gt;&lt;span&gt; (&lt;/span&gt;&lt;span&gt;words&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; words_out&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; =&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt; add_with_carry&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span&gt;not_newlines&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; classes&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;.&lt;/span&gt;&lt;span&gt;words&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; carried&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;0&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;let&lt;/span&gt;&lt;span&gt; (&lt;/span&gt;&lt;span&gt;content&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; content_out&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; =&lt;/span&gt;&lt;span class=&quot;z-l-6 z-d-7&quot;&gt; add_with_carry&lt;/span&gt;&lt;span&gt;(&lt;/span&gt;&lt;span&gt;not_newlines&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; classes&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;.&lt;/span&gt;&lt;span&gt;content&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; carried&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;z-l-1 z-d-4&quot;&gt;1&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt;let&lt;/span&gt;&lt;span&gt; (&lt;/span&gt;&lt;span&gt;word_lines&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; content_lines&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; =&lt;/span&gt;&lt;span&gt; (&lt;/span&gt;&lt;span&gt;words&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; &amp;amp;&lt;/span&gt;&lt;span&gt; newlines&lt;/span&gt;&lt;span&gt;,&lt;/span&gt;&lt;span&gt; content&lt;/span&gt;&lt;span class=&quot;z-l-8 z-d-10&quot;&gt; &amp;amp;&lt;/span&gt;&lt;span&gt; newlines&lt;/span&gt;&lt;span&gt;)&lt;/span&gt;&lt;span&gt;;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Bit &lt;code&gt;i&lt;/code&gt; of &lt;code&gt;word_lines&lt;/code&gt; is &lt;code&gt;1&lt;/code&gt; if the line ending at byte &lt;code&gt;i&lt;/code&gt; contained a word character, and &lt;code&gt;content_lines&lt;/code&gt; does the same for non-whitespace characters. Three &lt;code&gt;count_ones()&lt;/code&gt; calls (&lt;code&gt;word_lines&lt;/code&gt;, &lt;code&gt;content_lines &amp;amp; !word_lines&lt;/code&gt;, and &lt;code&gt;plain &amp;amp; !content_lines&lt;/code&gt;) then count all plain lines in the 64-byte block at once. Files whose average line length exceeds 64 bytes fall back to the per-line loop so the long-line AMD headers do not regress. Compared to the scalar fallback, those SIMD classification routines save &lt;strong&gt;38.6% of total wall time&lt;/strong&gt; on the kernel.&lt;/p&gt;
&lt;h4 id=&quot;runtime-cpu-dispatch-target-feature&quot;&gt;Runtime CPU dispatch (&lt;code&gt;#[target_feature]&lt;/code&gt;)&lt;a class=&quot;anchor&quot; href=&quot;#runtime-cpu-dispatch-target-feature&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h4&gt;
&lt;p&gt;Rust&#39;s default &lt;code&gt;x86-64&lt;/code&gt; target does not enable &lt;code&gt;popcnt&lt;/code&gt;, so every &lt;code&gt;count_ones()&lt;/code&gt; on a &lt;code&gt;u64&lt;/code&gt; compiles into a 19-instruction software fallback. Compiling three versions of the hot loop with &lt;code&gt;#[target_feature]&lt;/code&gt; and selecting one per file at runtime (&lt;code&gt;baseline&lt;/code&gt;, &lt;code&gt;AVX2 + popcnt/bmi&lt;/code&gt;, and &lt;code&gt;AVX-512&lt;/code&gt;, where &lt;code&gt;avx512bw&lt;/code&gt; byte shuffles build &lt;code&gt;words&lt;/code&gt; and &lt;code&gt;content&lt;/code&gt; in &lt;strong&gt;8 instructions per block instead of 37&lt;/strong&gt;) saved &lt;strong&gt;5% to 9% of wall time&lt;/strong&gt; in a single portable binary.&lt;/p&gt;
&lt;h4 id=&quot;a-single-table-driven-lexer&quot;&gt;A single table-driven lexer&lt;a class=&quot;anchor&quot; href=&quot;#a-single-table-driven-lexer&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h4&gt;
&lt;p&gt;On the remaining 12.4% of lines that contain candidate bytes, the old parser built four delimiter lists per line and ran 330 lines of index math to resolve overlaps. Replacing that with a single left-to-right cursor stepping through precompiled state tables cut total kernel instructions by &lt;strong&gt;21%&lt;/strong&gt; between &lt;code&gt;3.4.0&lt;/code&gt; and &lt;code&gt;3.5.0&lt;/code&gt;, bringing the PGO release build&#39;s equal-work time from 80 ms (88 ms without PGO) down to &lt;strong&gt;67 ms&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Because two counters pointed at the same folder rarely count the same files or do the same work out of the box, &lt;code&gt;linebench&lt;/code&gt; runs both a default mode and an equal-work mode. Equal-work mode turns off optional features that not every tool implements (ignore files, minified and generated file filters, shebang checks, keyword and test detection, complexity and cost estimates) and sets &lt;code&gt;mezura&lt;/code&gt; to &lt;code&gt;--counting region&lt;/code&gt; to match the others. When a tool runs a check unconditionally with no flag to disable it, such as reading file contents to tell C, C++, and Objective-C &lt;code&gt;.h&lt;/code&gt; headers apart, the other tools that have that check leave it on too. After each run, &lt;code&gt;linebench&lt;/code&gt; checks the JSON outputs to make sure all file and line totals match within 1%.&lt;/p&gt;
&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th style=&quot;text-align: left&quot;&gt;Tool&lt;/th&gt;&lt;th style=&quot;text-align: left&quot;&gt;Linux (&lt;code&gt;ext4&lt;/code&gt;)&lt;/th&gt;&lt;th style=&quot;text-align: left&quot;&gt;Windows (&lt;code&gt;NTFS&lt;/code&gt;)&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;mezura 3.5.0&lt;/strong&gt;&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;67 ms ± 2&lt;/strong&gt; (540.7M lines/s)&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;175 ms ± 15&lt;/strong&gt; (205.6M lines/s)&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;scc 4.1.0&lt;/strong&gt;&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;208 ms ± 3&lt;/strong&gt; (173.1M lines/s)&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;521 ms ± 36&lt;/strong&gt; (69.1M lines/s)&lt;/td&gt;&lt;/tr&gt;
&lt;tr&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;tokei 15.0.0&lt;/strong&gt;&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;411 ms ± 3&lt;/strong&gt; (87.7M lines/s)&lt;/td&gt;&lt;td style=&quot;text-align: left&quot;&gt;&lt;strong&gt;617 ms ± 20&lt;/strong&gt; (58.4M lines/s)&lt;/td&gt;&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Each number is the mean of 30 timed runs after 3 warmups across both command orders, ± one standard deviation. At each tool&#39;s default settings on Linux (with &lt;code&gt;mezura&lt;/code&gt;&#39;s ignore files, keyword counting, and test detection on), &lt;code&gt;mezura 3.5.0&lt;/code&gt; runs in &lt;strong&gt;85 ms ± 2&lt;/strong&gt;, &lt;code&gt;scc 4.1.0&lt;/code&gt; in &lt;strong&gt;290 ms ± 3&lt;/strong&gt;, and &lt;code&gt;tokei 15.0.0&lt;/code&gt; in &lt;strong&gt;418 ms ± 3&lt;/strong&gt;.&lt;/p&gt;
&lt;h3 id=&quot;what-separates-67-ms-from-42-ms&quot;&gt;What separates &lt;code&gt;67 ms&lt;/code&gt; from &lt;code&gt;42 ms&lt;/code&gt;&lt;a class=&quot;anchor&quot; href=&quot;#what-separates-67-ms-from-42-ms&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Part of the difference comes from real work that &lt;code&gt;mezura&lt;/code&gt; does and the minimal floor program skips. &lt;code&gt;mezura&lt;/code&gt; validates UTF-8 on every file, checks the content of ambiguous &lt;code&gt;.h&lt;/code&gt; headers (&lt;code&gt;6.0%&lt;/code&gt; of all kernel instructions, left on in equal-work mode because &lt;code&gt;scc 4.1.0&lt;/code&gt; cannot turn its own check off), and runs a general table-driven parser for 80+ languages instead of hardcoding C. Disabled features do not slow down the equal-work run because const-generic specialization leaves them out of the compiled loop.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;67 ms&lt;/code&gt; is not the ceiling either. Several concrete optimizations are still open without changing behavior, including folding the ASCII check into the 64-byte SIMD pass so mapped files are read from RAM once instead of twice, combining the separate &lt;code&gt;memmem&lt;/code&gt; passes on &lt;code&gt;.h&lt;/code&gt; headers into one multi-pattern scan, and merging the producer and consumer pools into a single worker pool.&lt;/p&gt;
&lt;p&gt;I also tested a pure 64-byte block state machine that did not split lines at all, covering 45 C- and Java-style languages while keeping the line lexer for the rest. It matched every file count and cut instructions on CPython (-14.9%) and the JDK (-5.4%), but ran &lt;strong&gt;5.7% more instructions on the Linux kernel&lt;/strong&gt; because &lt;code&gt;/&lt;/code&gt; characters inside C block comments kept triggering per-block walks where the line lexer jumps straight to &lt;code&gt;*/&lt;/code&gt;. Skipping blocks inside open comments would fix part of that, though maintaining a second 660-line parser for 45 languages was not worth it for a win that depended on the codebase.&lt;/p&gt;
&lt;h2 id=&quot;5-counting-fast-means-nothing-if-the-counts-are-wrong&quot;&gt;5. Counting fast means nothing if the counts are wrong&lt;a class=&quot;anchor&quot; href=&quot;#5-counting-fast-means-nothing-if-the-counts-are-wrong&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h2&gt;
&lt;h3 id=&quot;how-the-faster-lexer-also-fixed-subtle-bugs&quot;&gt;How the faster lexer also fixed subtle bugs&lt;a class=&quot;anchor&quot; href=&quot;#how-the-faster-lexer-also-fixed-subtle-bugs&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Because the old parser gathered all delimiters on a line before tracking state and tried to untangle overlaps afterward, tricky combinations could slip through. A small fuzzer generating random mixes of &lt;code&gt;/&lt;/code&gt;, &lt;code&gt;*&lt;/code&gt;, &lt;code&gt;&quot;&lt;/code&gt;, &lt;code&gt;x&lt;/code&gt;, and spaces against a 20-line reference lexer found &lt;code&gt;/* x /*/&lt;/code&gt; in C within minutes, an overlap bug that caused the rest of the file to be counted as a block comment and had gone unnoticed across 378,000 real files.&lt;/p&gt;
&lt;p&gt;The table-driven lexer reads left to right with explicit state, so once &lt;code&gt;/*&lt;/code&gt; opens a comment, its &lt;code&gt;*&lt;/code&gt; cannot be reused as the start of &lt;code&gt;*/&lt;/code&gt;, and the entire overlap-resolution step disappears. Knowing the current state while scanning also made context-sensitive rules easy to support, such as distinguishing JavaScript regex literals from division, lexing &lt;code&gt;{...}&lt;/code&gt; code expressions inside interpolated strings, and restricting HTML quotes to tags.&lt;/p&gt;
&lt;p&gt;All 80+ supported languages use this same engine. Each language is defined in a plain-text config file (comment pairs, string delimiters, escape rules, nesting, regex contexts, interpolation delimiters) that compiles into transition tables on first use, without per-language branches in the parser. Higher-level features like keyword counting, inline test detection, and nested languages (&lt;code&gt;&amp;lt;script&amp;gt;&lt;/code&gt;/&lt;code&gt;&amp;lt;style&amp;gt;&lt;/code&gt; in HTML, Vue, and Svelte, or HTML around PHP) simply read the clean code byte ranges returned for each line, so each feature is implemented once.&lt;/p&gt;
&lt;h3 id=&quot;what-accuracy-costs-in-speed&quot;&gt;What accuracy costs in speed&lt;a class=&quot;anchor&quot; href=&quot;#what-accuracy-costs-in-speed&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Supporting those edge cases costs real CPU time. A counter can always shave milliseconds by ignoring regex literals (&lt;code&gt;/a/*b/&lt;/code&gt;), skipping &lt;code&gt;{...}&lt;/code&gt; expressions inside interpolated strings (&lt;code&gt;&quot;a ${b /* c */} d&quot;&lt;/code&gt;), treating a trailing &lt;code&gt;\&lt;/code&gt; as a normal character, or counting &lt;code&gt;struct&lt;/code&gt; and &lt;code&gt;class&lt;/code&gt; by raw word matches.&lt;/p&gt;
&lt;p&gt;In 3.5.0, the fixes for regex literals, string interpolation holes, HTML tag quotes, and PHP page boundaries added &lt;strong&gt;2.3%&lt;/strong&gt; to the kernel&#39;s instruction count. Checking that &lt;code&gt;struct&lt;/code&gt; or &lt;code&gt;class&lt;/code&gt; actually starts a type definition adds another &lt;strong&gt;2.6% to 2.9%&lt;/strong&gt; on default runs, which on the kernel&#39;s &lt;code&gt;.c&lt;/code&gt; files is the difference between 1.2 million occurrences of the word &lt;code&gt;struct&lt;/code&gt; and the 33,589 structs actually defined.&lt;/p&gt;
&lt;h3 id=&quot;conformance-testing-with-linejudge&quot;&gt;Conformance testing with LineJudge&lt;a class=&quot;anchor&quot; href=&quot;#conformance-testing-with-linejudge&quot; aria-label=&quot;Link to this section&quot;&gt;#&lt;/a&gt;
&lt;/h3&gt;
&lt;p&gt;Every change runs in CI against &lt;a rel=&quot;external&quot; href=&quot;https://loc-conformance.github.io/linejudge/&quot;&gt;LineJudge&lt;/a&gt;, a conformance suite I built under &lt;code&gt;loc-conformance&lt;/code&gt; (version &lt;code&gt;0.4.0&lt;/code&gt; has 107 tricky edge-case files across 8 structural categories). Even though I wrote both tools, &lt;code&gt;LineJudge&lt;/code&gt; does not test counters against &lt;code&gt;mezura&lt;/code&gt;&#39;s numbers or enforce a single &quot;correct&quot; line count. Each test file only marks the exact byte spans of strings and comments according to the language specification, and each counter is checked against a small declaration of its own counting rules. Two counters that define blank or code lines differently can output different numbers on the same file and both pass. &lt;code&gt;mezura 3.5.0&lt;/code&gt; passes &lt;strong&gt;107/107&lt;/strong&gt; cases, and a follow-up post will cover how that rule engine works.&lt;/p&gt;
&lt;p&gt;For JavaScript regex literals, I also checked &lt;code&gt;mezura&lt;/code&gt; against &lt;code&gt;acorn 8.15.0&lt;/code&gt; across 38,433 real JS files. It matched 48,070 of &lt;code&gt;acorn&lt;/code&gt;&#39;s 48,114 regexes at the exact same byte offsets, and the remaining 44 did not change a single line count.&lt;/p&gt;
</content>
        
    </entry>
</feed>
