<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title>Query-Engine - 标签 - 星河拾贝录</title><link>https://blog.liubang.cc/tags/query-engine/</link><description>Query-Engine - 标签 - 星河拾贝录</description><generator>Hugo -- gohugo.io</generator><language>zh-CN</language><managingEditor>it.liubang@gmail.com (liubang)</managingEditor><webMaster>it.liubang@gmail.com (liubang)</webMaster><copyright>Copyright © 2019-2026 LiuBang. All Rights Reserved.</copyright><lastBuildDate>Sat, 23 May 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://blog.liubang.cc/tags/query-engine/" rel="self" type="application/rss+xml"/><item><title>Table Pipeline：Flux 查询模型如何落到内存执行</title><link>https://blog.liubang.cc/flux/2026-05-23-flux-engine-06-table-pipeline/</link><pubDate>Sat, 23 May 2026 00:00:00 +0000</pubDate><author><name>liubang</name></author><guid>https://blog.liubang.cc/flux/2026-05-23-flux-engine-06-table-pipeline/</guid><description><![CDATA[<p>Flux 查询真正有意思的地方，不在单个表达式求值，而在 table stream。<code>filter</code>、<code>map</code>、<code>group</code>、<code>window</code>、<code>aggregateWindow</code>、<code>join</code> 这些算子处理的都不是一张普通二维表，而是一组带有 group key 的 logical tables。</p>
<p>这篇文章专门讲 <code>cpp/pl/flux</code> 里 table pipeline 的内存执行模型。它是早期 eager interpreter 的核心，也是 connector / physical executor fallback 到 materialized output 时的共同承载格式。理解这一层，后面的 connector pushdown、logical/physical plan 和性能优化才有上下文。</p>
<h2 id="一个贯穿全文的查询" class="headerLink">
    <a href="#%e4%b8%80%e4%b8%aa%e8%b4%af%e7%a9%bf%e5%85%a8%e6%96%87%e7%9a%84%e6%9f%a5%e8%af%a2" class="header-mark"></a>一个贯穿全文的查询</h2><p>先看一个 dashboard 查询。它从 CSV 读入 CPU 数据，按时间过滤，再按主机分组，最后做窗口聚合：</p>
<div class="code-block highlight is-open show-line-numbers  tw:group tw:my-2">
  <div class="
    
    tw:flex 
    tw:flex-row
    tw:flex-1 
    tw:justify-between 
    tw:w-full tw:bg-bgColor-secondary
    ">      
    <button 
      class="
        code-block-button
        tw:mx-2 
        tw:flex
        tw:flex-row
        tw:flex-1"
      aria-hidden="true">
          <span class="tw:group-[.is-open]:rotate-90 tw:transition-[transform] tw:duration-500 tw:ease-in-out tw:print:hidden! tw:w-min tw:h-min tw:my-1 tw:mx-1"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M285.476 272.971L91.132 467.314c-9.373 9.373-24.569 9.373-33.941 0l-22.667-22.667c-9.357-9.357-9.375-24.522-.04-33.901L188.505 256 34.484 101.255c-9.335-9.379-9.317-24.544.04-33.901l22.667-22.667c9.373-9.373 24.569-9.373 33.941 0L285.475 239.03c9.373 9.372 9.373 24.568.001 33.941z"/></svg></span>
          <span class="tw:select-none tw:my-1! tw:block">flux</span>
      </button>

   <div class="tw:flex">
      <button 
        class="
          line-number-button
          tw:mx-2 
          tw:hidden 
          tw:group-[.is-open]:block 
          tw:group-[.show-line-numbers]:text-fgColor-link 
          tw:print:hidden!" 
        title="Toggle line numbers"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M61.77 401l17.5-20.15a19.92 19.92 0 0 0 5.07-14.19v-3.31C84.34 356 80.5 352 73 352H16a8 8 0 0 0-8 8v16a8 8 0 0 0 8 8h22.83a157.41 157.41 0 0 0-11 12.31l-5.61 7c-4 5.07-5.25 10.13-2.8 14.88l1.05 1.93c3 5.76 6.29 7.88 12.25 7.88h4.73c10.33 0 15.94 2.44 15.94 9.09 0 4.72-4.2 8.22-14.36 8.22a41.54 41.54 0 0 1-15.47-3.12c-6.49-3.88-11.74-3.5-15.6 3.12l-5.59 9.31c-3.72 6.13-3.19 11.72 2.63 15.94 7.71 4.69 20.38 9.44 37 9.44 34.16 0 48.5-22.75 48.5-44.12-.03-14.38-9.12-29.76-28.73-34.88zM496 224H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16v-32a16 16 0 0 0-16-16zm0-160H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16V80a16 16 0 0 0-16-16zm0 320H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16v-32a16 16 0 0 0-16-16zM16 160h64a8 8 0 0 0 8-8v-16a8 8 0 0 0-8-8H64V40a8 8 0 0 0-8-8H32a8 8 0 0 0-7.14 4.42l-8 16A8 8 0 0 0 24 64h8v64H16a8 8 0 0 0-8 8v16a8 8 0 0 0 8 8zm-3.91 160H80a8 8 0 0 0 8-8v-16a8 8 0 0 0-8-8H41.32c3.29-10.29 48.34-18.68 48.34-56.44 0-29.06-25-39.56-44.47-39.56-21.36 0-33.8 10-40.46 18.75-4.37 5.59-3 10.84 2.8 15.37l8.58 6.88c5.61 4.56 11 2.47 16.12-2.44a13.44 13.44 0 0 1 9.46-3.84c3.33 0 9.28 1.56 9.28 8.75C51 248.19 0 257.31 0 304.59v4C0 316 5.08 320 12.09 320z"/></svg></button>

      <button 
        class="
          wrap-code-button
          tw:select-none 
          tw:mx-2 
          tw:hidden 
          tw:group-[.is-open]:block 
          tw:group-[.is-wrap]:text-fgColor-link 
          tw:print:hidden!" 
        title="Toggle code wrap"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M16 132h416c8.837 0 16-7.163 16-16V76c0-8.837-7.163-16-16-16H16C7.163 60 0 67.163 0 76v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16z"/></svg></button>
      
      <button 
        class="
          copy-code-button
          tw:select-none
          tw:mx-2 
          tw:hidden
          tw:group-[.is-open]:block
          tw:hover:text-fgColor-link 
          tw:print:hidden!"
        title="Copy code">
          <span class="copy-icon tw:block"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M433.941 65.941l-51.882-51.882A48 48 0 0 0 348.118 0H176c-26.51 0-48 21.49-48 48v48H48c-26.51 0-48 21.49-48 48v320c0 26.51 21.49 48 48 48h224c26.51 0 48-21.49 48-48v-48h80c26.51 0 48-21.49 48-48V99.882a48 48 0 0 0-14.059-33.941zM266 464H54a6 6 0 0 1-6-6V150a6 6 0 0 1 6-6h74v224c0 26.51 21.49 48 48 48h96v42a6 6 0 0 1-6 6zm128-96H182a6 6 0 0 1-6-6V54a6 6 0 0 1 6-6h106v88c0 13.255 10.745 24 24 24h88v202a6 6 0 0 1-6 6zm6-256h-64V48h9.632c1.591 0 3.117.632 4.243 1.757l48.368 48.368a6 6 0 0 1 1.757 4.243V112z"/></svg></span>
          <span class="check-icon tw:hidden"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M173.898 439.404l-166.4-166.4c-9.997-9.997-9.997-26.206 0-36.204l36.203-36.204c9.997-9.998 26.207-9.998 36.204 0L192 312.69 432.095 72.596c9.997-9.997 26.207-9.997 36.204 0l36.203 36.204c9.997 9.997 9.997 26.206 0 36.204l-294.4 294.401c-9.998 9.997-26.207 9.997-36.204-.001z"/></svg></span>
      </button>
        
      <button 
        class="
          tw:select-none 
          tw:mx-2 
          tw:block 
          tw:group-[.is-open]:hidden 
          tw:print:hidden!" 
        disabled
        aria-hidden="true"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M328 256c0 39.8-32.2 72-72 72s-72-32.2-72-72 32.2-72 72-72 72 32.2 72 72zm104-72c-39.8 0-72 32.2-72 72s32.2 72 72 72 72-32.2 72-72-32.2-72-72-72zm-352 0c-39.8 0-72 32.2-72 72s32.2 72 72 72 72-32.2 72-72-32.2-72-72-72z"/></svg></button>
    </div>
  </div>
  <pre style="counter-reset: codeblock;" class="tw:block tw:m-0 tw:p-0"><code 
    id="codeblock-id-1" 
    class="
      chroma 
      tw:block! 
      tw:p-0
      tw:m-0
      tw:transition-[max-height] 
      tw:duration-500 
      tw:ease-in-out 
      tw:group-[.is-closed]:max-h-0! 
      tw:group-[.is-wrap]:text-wrap
      tw:overflow-y-hidden
      tw:overflow-x-auto
      tw:scrollbar-thin
      "><span class="line"><span class="cl">import &#34;csv&#34;
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">csv.from(file: &#34;cpp/pl/flux/examples/ops_dashboard/data/cpu_usage.annotated.csv&#34;)
</span></span><span class="line"><span class="cl">    |&gt; range(start: 2024-01-01T00:00:00Z, stop: 2024-01-01T00:05:00Z)
</span></span><span class="line"><span class="cl">    |&gt; filter(fn: (r) =&gt; r.region == &#34;us-west&#34;)
</span></span><span class="line"><span class="cl">    |&gt; group(columns: [&#34;host&#34;])
</span></span><span class="line"><span class="cl">    |&gt; aggregateWindow(every: 1m, fn: mean, createEmpty: true)</span></span></code></pre>
</div>
<p>从用户角度看，这是一条线性的 pipe。从执行器角度看，每一步都在转换一个 table stream：输入可能包含多张 logical table，输出也可能包含多张 logical table，并且每张表都有自己的 group key、列集合和行集合。</p>]]></description></item><item><title>从 Logical Plan 到 Physical Plan：执行引擎的骨架</title><link>https://blog.liubang.cc/flux/2026-05-23-flux-engine-08-logical-to-physical-plan/</link><pubDate>Sat, 23 May 2026 00:00:00 +0000</pubDate><author><name>liubang</name></author><guid>https://blog.liubang.cc/flux/2026-05-23-flux-engine-08-logical-to-physical-plan/</guid><description><![CDATA[<p>前面两篇分别讲了内存表执行和 connector pushdown。它们看上去像两条路径：一条是 <code>TableValue</code> 上的 builtin 解释执行，一条是 SQLite/MySQL 上的 Page streaming。第 08 篇要回答的就是中间那层问题：同一条 Flux 查询，如何先被表达成可优化的 logical plan，再落成真正可调度的 physical pipeline。</p>
<p>如果项目只有 eager interpreter，执行路径很直接：AST 调 builtin，builtin 操作 <code>TableValue</code>，然后把结果传给下一个 builtin。但 connector、pushdown、多 split、exchange、partial/final aggregate 和 profile 加进来后，直接解释 AST 就不够了。执行器必须先有一个“尚未执行、可以改写、可以解释”的查询表示。</p>
<p>这就是 logical/physical plan 的位置。Logical plan 负责保存用户查询语义，optimizer 负责改写语义等价的计划，physical plan 负责描述执行拓扑，scheduler 再把它变成 driver 和 operator。</p>
<h2 id="一个贯穿全文的查询" class="headerLink">
    <a href="#%e4%b8%80%e4%b8%aa%e8%b4%af%e7%a9%bf%e5%85%a8%e6%96%87%e7%9a%84%e6%9f%a5%e8%af%a2" class="header-mark"></a>一个贯穿全文的查询</h2><p>先用一条查询做主线：</p>
<div class="code-block highlight is-open show-line-numbers  tw:group tw:my-2">
  <div class="
    
    tw:flex 
    tw:flex-row
    tw:flex-1 
    tw:justify-between 
    tw:w-full tw:bg-bgColor-secondary
    ">      
    <button 
      class="
        code-block-button
        tw:mx-2 
        tw:flex
        tw:flex-row
        tw:flex-1"
      aria-hidden="true">
          <span class="tw:group-[.is-open]:rotate-90 tw:transition-[transform] tw:duration-500 tw:ease-in-out tw:print:hidden! tw:w-min tw:h-min tw:my-1 tw:mx-1"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M285.476 272.971L91.132 467.314c-9.373 9.373-24.569 9.373-33.941 0l-22.667-22.667c-9.357-9.357-9.375-24.522-.04-33.901L188.505 256 34.484 101.255c-9.335-9.379-9.317-24.544.04-33.901l22.667-22.667c9.373-9.373 24.569-9.373 33.941 0L285.475 239.03c9.373 9.372 9.373 24.568.001 33.941z"/></svg></span>
          <span class="tw:select-none tw:my-1! tw:block">flux</span>
      </button>

   <div class="tw:flex">
      <button 
        class="
          line-number-button
          tw:mx-2 
          tw:hidden 
          tw:group-[.is-open]:block 
          tw:group-[.show-line-numbers]:text-fgColor-link 
          tw:print:hidden!" 
        title="Toggle line numbers"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M61.77 401l17.5-20.15a19.92 19.92 0 0 0 5.07-14.19v-3.31C84.34 356 80.5 352 73 352H16a8 8 0 0 0-8 8v16a8 8 0 0 0 8 8h22.83a157.41 157.41 0 0 0-11 12.31l-5.61 7c-4 5.07-5.25 10.13-2.8 14.88l1.05 1.93c3 5.76 6.29 7.88 12.25 7.88h4.73c10.33 0 15.94 2.44 15.94 9.09 0 4.72-4.2 8.22-14.36 8.22a41.54 41.54 0 0 1-15.47-3.12c-6.49-3.88-11.74-3.5-15.6 3.12l-5.59 9.31c-3.72 6.13-3.19 11.72 2.63 15.94 7.71 4.69 20.38 9.44 37 9.44 34.16 0 48.5-22.75 48.5-44.12-.03-14.38-9.12-29.76-28.73-34.88zM496 224H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16v-32a16 16 0 0 0-16-16zm0-160H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16V80a16 16 0 0 0-16-16zm0 320H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16v-32a16 16 0 0 0-16-16zM16 160h64a8 8 0 0 0 8-8v-16a8 8 0 0 0-8-8H64V40a8 8 0 0 0-8-8H32a8 8 0 0 0-7.14 4.42l-8 16A8 8 0 0 0 24 64h8v64H16a8 8 0 0 0-8 8v16a8 8 0 0 0 8 8zm-3.91 160H80a8 8 0 0 0 8-8v-16a8 8 0 0 0-8-8H41.32c3.29-10.29 48.34-18.68 48.34-56.44 0-29.06-25-39.56-44.47-39.56-21.36 0-33.8 10-40.46 18.75-4.37 5.59-3 10.84 2.8 15.37l8.58 6.88c5.61 4.56 11 2.47 16.12-2.44a13.44 13.44 0 0 1 9.46-3.84c3.33 0 9.28 1.56 9.28 8.75C51 248.19 0 257.31 0 304.59v4C0 316 5.08 320 12.09 320z"/></svg></button>

      <button 
        class="
          wrap-code-button
          tw:select-none 
          tw:mx-2 
          tw:hidden 
          tw:group-[.is-open]:block 
          tw:group-[.is-wrap]:text-fgColor-link 
          tw:print:hidden!" 
        title="Toggle code wrap"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M16 132h416c8.837 0 16-7.163 16-16V76c0-8.837-7.163-16-16-16H16C7.163 60 0 67.163 0 76v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16z"/></svg></button>
      
      <button 
        class="
          copy-code-button
          tw:select-none
          tw:mx-2 
          tw:hidden
          tw:group-[.is-open]:block
          tw:hover:text-fgColor-link 
          tw:print:hidden!"
        title="Copy code">
          <span class="copy-icon tw:block"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M433.941 65.941l-51.882-51.882A48 48 0 0 0 348.118 0H176c-26.51 0-48 21.49-48 48v48H48c-26.51 0-48 21.49-48 48v320c0 26.51 21.49 48 48 48h224c26.51 0 48-21.49 48-48v-48h80c26.51 0 48-21.49 48-48V99.882a48 48 0 0 0-14.059-33.941zM266 464H54a6 6 0 0 1-6-6V150a6 6 0 0 1 6-6h74v224c0 26.51 21.49 48 48 48h96v42a6 6 0 0 1-6 6zm128-96H182a6 6 0 0 1-6-6V54a6 6 0 0 1 6-6h106v88c0 13.255 10.745 24 24 24h88v202a6 6 0 0 1-6 6zm6-256h-64V48h9.632c1.591 0 3.117.632 4.243 1.757l48.368 48.368a6 6 0 0 1 1.757 4.243V112z"/></svg></span>
          <span class="check-icon tw:hidden"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M173.898 439.404l-166.4-166.4c-9.997-9.997-9.997-26.206 0-36.204l36.203-36.204c9.997-9.998 26.207-9.998 36.204 0L192 312.69 432.095 72.596c9.997-9.997 26.207-9.997 36.204 0l36.203 36.204c9.997 9.997 9.997 26.206 0 36.204l-294.4 294.401c-9.998 9.997-26.207 9.997-36.204-.001z"/></svg></span>
      </button>
        
      <button 
        class="
          tw:select-none 
          tw:mx-2 
          tw:block 
          tw:group-[.is-open]:hidden 
          tw:print:hidden!" 
        disabled
        aria-hidden="true"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M328 256c0 39.8-32.2 72-72 72s-72-32.2-72-72 32.2-72 72-72 72 32.2 72 72zm104-72c-39.8 0-72 32.2-72 72s32.2 72 72 72 72-32.2 72-72-32.2-72-72-72zm-352 0c-39.8 0-72 32.2-72 72s32.2 72 72 72 72-32.2 72-72-32.2-72-72-72z"/></svg></button>
    </div>
  </div>
  <pre style="counter-reset: codeblock;" class="tw:block tw:m-0 tw:p-0"><code 
    id="codeblock-id-1" 
    class="
      chroma 
      tw:block! 
      tw:p-0
      tw:m-0
      tw:transition-[max-height] 
      tw:duration-500 
      tw:ease-in-out 
      tw:group-[.is-closed]:max-h-0! 
      tw:group-[.is-wrap]:text-wrap
      tw:overflow-y-hidden
      tw:overflow-x-auto
      tw:scrollbar-thin
      "><span class="line"><span class="cl">import &#34;sqlite&#34;
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">sqlite.from(path: &#34;metrics.db&#34;, table: &#34;cpu&#34;)
</span></span><span class="line"><span class="cl">    |&gt; range(start: 2024-01-01T00:00:00Z, stop: 2024-01-01T01:00:00Z)
</span></span><span class="line"><span class="cl">    |&gt; filter(fn: (r) =&gt; r.host == &#34;edge-1&#34; and r._value &gt; 80.0)
</span></span><span class="line"><span class="cl">    |&gt; keep(columns: [&#34;_time&#34;, &#34;host&#34;, &#34;_value&#34;])
</span></span><span class="line"><span class="cl">    |&gt; group(columns: [&#34;host&#34;])
</span></span><span class="line"><span class="cl">    |&gt; mean(column: &#34;_value&#34;)
</span></span><span class="line"><span class="cl">    |&gt; sort(columns: [&#34;_value&#34;], desc: true)
</span></span><span class="line"><span class="cl">    |&gt; limit(n: 10)</span></span></code></pre>
</div>
<p>这条查询里同时有 source scan、时间范围、谓词、projection、group aggregate、sort 和 limit。它足够小，但已经覆盖了查询引擎里最核心的几个问题：</p>]]></description></item><item><title>性能优化：从解释执行到查询下推</title><link>https://blog.liubang.cc/flux/2026-05-23-flux-engine-11-performance/</link><pubDate>Sat, 23 May 2026 00:00:00 +0000</pubDate><author><name>liubang</name></author><guid>https://blog.liubang.cc/flux/2026-05-23-flux-engine-11-performance/</guid><description><![CDATA[<p>性能优化不能脱离架构。<code>cpp/pl/flux</code> 早期是 eager interpreter，所有数据尽量变成 <code>TableValue</code>，再由 builtin 一步步处理。这条路径简单、可测、适合小数据，也非常适合把语言语义先跑通。</p>
<p>但查询引擎一旦开始面对 SQLite/MySQL、大表 scan、group aggregate、top-n、join 和 profile，性能问题就不再是“某个函数慢一点”。真正的瓶颈往往来自执行模型：整表 materialization、row object 中间态、重复 key 构造、connector 读了太多列、blocking operator 没有内存边界。</p>
<p>这一篇讲 Flux 性能优化的主线：先用 profile 定位瓶颈，再把热路径从 <code>TableValue</code> 推向 Page streaming，通过 connector pushdown 减少数据移动，用 partial/final accumulator 降低 root 阶段压力，最后用 benchmark 固定同机同口径的比较方法。</p>
<h2 id="一条查询背后的成本" class="headerLink">
    <a href="#%e4%b8%80%e6%9d%a1%e6%9f%a5%e8%af%a2%e8%83%8c%e5%90%8e%e7%9a%84%e6%88%90%e6%9c%ac" class="header-mark"></a>一条查询背后的成本</h2><p>先看一条典型查询：</p>
<div class="code-block highlight is-open show-line-numbers  tw:group tw:my-2">
  <div class="
    
    tw:flex 
    tw:flex-row
    tw:flex-1 
    tw:justify-between 
    tw:w-full tw:bg-bgColor-secondary
    ">      
    <button 
      class="
        code-block-button
        tw:mx-2 
        tw:flex
        tw:flex-row
        tw:flex-1"
      aria-hidden="true">
          <span class="tw:group-[.is-open]:rotate-90 tw:transition-[transform] tw:duration-500 tw:ease-in-out tw:print:hidden! tw:w-min tw:h-min tw:my-1 tw:mx-1"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M285.476 272.971L91.132 467.314c-9.373 9.373-24.569 9.373-33.941 0l-22.667-22.667c-9.357-9.357-9.375-24.522-.04-33.901L188.505 256 34.484 101.255c-9.335-9.379-9.317-24.544.04-33.901l22.667-22.667c9.373-9.373 24.569-9.373 33.941 0L285.475 239.03c9.373 9.372 9.373 24.568.001 33.941z"/></svg></span>
          <span class="tw:select-none tw:my-1! tw:block">flux</span>
      </button>

   <div class="tw:flex">
      <button 
        class="
          line-number-button
          tw:mx-2 
          tw:hidden 
          tw:group-[.is-open]:block 
          tw:group-[.show-line-numbers]:text-fgColor-link 
          tw:print:hidden!" 
        title="Toggle line numbers"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M61.77 401l17.5-20.15a19.92 19.92 0 0 0 5.07-14.19v-3.31C84.34 356 80.5 352 73 352H16a8 8 0 0 0-8 8v16a8 8 0 0 0 8 8h22.83a157.41 157.41 0 0 0-11 12.31l-5.61 7c-4 5.07-5.25 10.13-2.8 14.88l1.05 1.93c3 5.76 6.29 7.88 12.25 7.88h4.73c10.33 0 15.94 2.44 15.94 9.09 0 4.72-4.2 8.22-14.36 8.22a41.54 41.54 0 0 1-15.47-3.12c-6.49-3.88-11.74-3.5-15.6 3.12l-5.59 9.31c-3.72 6.13-3.19 11.72 2.63 15.94 7.71 4.69 20.38 9.44 37 9.44 34.16 0 48.5-22.75 48.5-44.12-.03-14.38-9.12-29.76-28.73-34.88zM496 224H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16v-32a16 16 0 0 0-16-16zm0-160H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16V80a16 16 0 0 0-16-16zm0 320H176a16 16 0 0 0-16 16v32a16 16 0 0 0 16 16h320a16 16 0 0 0 16-16v-32a16 16 0 0 0-16-16zM16 160h64a8 8 0 0 0 8-8v-16a8 8 0 0 0-8-8H64V40a8 8 0 0 0-8-8H32a8 8 0 0 0-7.14 4.42l-8 16A8 8 0 0 0 24 64h8v64H16a8 8 0 0 0-8 8v16a8 8 0 0 0 8 8zm-3.91 160H80a8 8 0 0 0 8-8v-16a8 8 0 0 0-8-8H41.32c3.29-10.29 48.34-18.68 48.34-56.44 0-29.06-25-39.56-44.47-39.56-21.36 0-33.8 10-40.46 18.75-4.37 5.59-3 10.84 2.8 15.37l8.58 6.88c5.61 4.56 11 2.47 16.12-2.44a13.44 13.44 0 0 1 9.46-3.84c3.33 0 9.28 1.56 9.28 8.75C51 248.19 0 257.31 0 304.59v4C0 316 5.08 320 12.09 320z"/></svg></button>

      <button 
        class="
          wrap-code-button
          tw:select-none 
          tw:mx-2 
          tw:hidden 
          tw:group-[.is-open]:block 
          tw:group-[.is-wrap]:text-fgColor-link 
          tw:print:hidden!" 
        title="Toggle code wrap"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M16 132h416c8.837 0 16-7.163 16-16V76c0-8.837-7.163-16-16-16H16C7.163 60 0 67.163 0 76v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16zm0 160h416c8.837 0 16-7.163 16-16v-40c0-8.837-7.163-16-16-16H16c-8.837 0-16 7.163-16 16v40c0 8.837 7.163 16 16 16z"/></svg></button>
      
      <button 
        class="
          copy-code-button
          tw:select-none
          tw:mx-2 
          tw:hidden
          tw:group-[.is-open]:block
          tw:hover:text-fgColor-link 
          tw:print:hidden!"
        title="Copy code">
          <span class="copy-icon tw:block"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M433.941 65.941l-51.882-51.882A48 48 0 0 0 348.118 0H176c-26.51 0-48 21.49-48 48v48H48c-26.51 0-48 21.49-48 48v320c0 26.51 21.49 48 48 48h224c26.51 0 48-21.49 48-48v-48h80c26.51 0 48-21.49 48-48V99.882a48 48 0 0 0-14.059-33.941zM266 464H54a6 6 0 0 1-6-6V150a6 6 0 0 1 6-6h74v224c0 26.51 21.49 48 48 48h96v42a6 6 0 0 1-6 6zm128-96H182a6 6 0 0 1-6-6V54a6 6 0 0 1 6-6h106v88c0 13.255 10.745 24 24 24h88v202a6 6 0 0 1-6 6zm6-256h-64V48h9.632c1.591 0 3.117.632 4.243 1.757l48.368 48.368a6 6 0 0 1 1.757 4.243V112z"/></svg></span>
          <span class="check-icon tw:hidden"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M173.898 439.404l-166.4-166.4c-9.997-9.997-9.997-26.206 0-36.204l36.203-36.204c9.997-9.998 26.207-9.998 36.204 0L192 312.69 432.095 72.596c9.997-9.997 26.207-9.997 36.204 0l36.203 36.204c9.997 9.997 9.997 26.206 0 36.204l-294.4 294.401c-9.998 9.997-26.207 9.997-36.204-.001z"/></svg></span>
      </button>
        
      <button 
        class="
          tw:select-none 
          tw:mx-2 
          tw:block 
          tw:group-[.is-open]:hidden 
          tw:print:hidden!" 
        disabled
        aria-hidden="true"><svg class="icon"
    xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!-- Font Awesome Free 5.15.4 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) --><path d="M328 256c0 39.8-32.2 72-72 72s-72-32.2-72-72 32.2-72 72-72 72 32.2 72 72zm104-72c-39.8 0-72 32.2-72 72s32.2 72 72 72 72-32.2 72-72-32.2-72-72-72zm-352 0c-39.8 0-72 32.2-72 72s32.2 72 72 72 72-32.2 72-72-32.2-72-72-72z"/></svg></button>
    </div>
  </div>
  <pre style="counter-reset: codeblock;" class="tw:block tw:m-0 tw:p-0"><code 
    id="codeblock-id-1" 
    class="
      chroma 
      tw:block! 
      tw:p-0
      tw:m-0
      tw:transition-[max-height] 
      tw:duration-500 
      tw:ease-in-out 
      tw:group-[.is-closed]:max-h-0! 
      tw:group-[.is-wrap]:text-wrap
      tw:overflow-y-hidden
      tw:overflow-x-auto
      tw:scrollbar-thin
      "><span class="line"><span class="cl">import &#34;sqlite&#34;
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">sqlite.from(path: &#34;metrics.db&#34;, table: &#34;cpu&#34;)
</span></span><span class="line"><span class="cl">    |&gt; range(start: 2024-01-01T00:00:00Z, stop: 2024-01-01T01:00:00Z)
</span></span><span class="line"><span class="cl">    |&gt; filter(fn: (r) =&gt; r.region == &#34;west&#34; and r._value &gt; 80.0)
</span></span><span class="line"><span class="cl">    |&gt; keep(columns: [&#34;_time&#34;, &#34;host&#34;, &#34;_value&#34;])
</span></span><span class="line"><span class="cl">    |&gt; group(columns: [&#34;host&#34;])
</span></span><span class="line"><span class="cl">    |&gt; mean(column: &#34;_value&#34;)
</span></span><span class="line"><span class="cl">    |&gt; sort(columns: [&#34;_value&#34;], desc: true)
</span></span><span class="line"><span class="cl">    |&gt; limit(n: 10)</span></span></code></pre>
</div>
<p>它看起来只是几行 pipe，但执行时至少有几类成本：</p>]]></description></item><item><title>项目路线图：从可用到好用</title><link>https://blog.liubang.cc/flux/2026-05-23-flux-engine-12-roadmap/</link><pubDate>Sat, 23 May 2026 00:00:00 +0000</pubDate><author><name>liubang</name></author><guid>https://blog.liubang.cc/flux/2026-05-23-flux-engine-12-roadmap/</guid><description><![CDATA[<p>到目前为止，<code>cpp/pl/flux</code> 已经具备一个语言项目和查询引擎项目的基本骨架：能解析、能执行、能导入标准库、能跑表流查询、能接 SQLite/MySQL、能做部分 pushdown、能输出 explain/profile，也有 LSP 和 conformance tests。</p>
<p>接下来真正重要的问题不是“还能加什么功能”，而是“哪些功能能让它从可用走向好用”。</p>
<h2 id="当前已经稳定的主干" class="headerLink">
    <a href="#%e5%bd%93%e5%89%8d%e5%b7%b2%e7%bb%8f%e7%a8%b3%e5%ae%9a%e7%9a%84%e4%b8%bb%e5%b9%b2" class="header-mark"></a>当前已经稳定的主干</h2><p>语言前端方面，scanner/parser/AST 已经覆盖常见 Flux 查询语法。表达式优先级、函数、pipe、record、array、object、regex、datetime、duration、类型语法都有测试覆盖。</p>
<p>运行时方面，<code>Value</code>、<code>Environment</code>、expression evaluator、statement executor、function closure、pipe 参数、标准库 package 和 table pipeline 已经形成闭环。</p>
<p>查询执行方面，内存 <code>TableValue</code> 路径仍然可用；SQLite/MySQL connector 已经走 metadata/split/page source；optimizer 和 physical executor 主干已经存在；Page streaming、exchange、accumulator 和 profile 能覆盖越来越多真实查询。</p>
<p>工具链方面，CLI、REPL、AST dump、JSON/CSV/human 输出、LSP 和 conformance examples 都已经可用。</p>
<h2 id="路线图的排序原则" class="headerLink">
    <a href="#%e8%b7%af%e7%ba%bf%e5%9b%be%e7%9a%84%e6%8e%92%e5%ba%8f%e5%8e%9f%e5%88%99" class="header-mark"></a>路线图的排序原则</h2><p>后续路线不能只按“哪个功能酷”排序，而要看它是否强化主干。</p>
<p>优先级最高的是会被多个模块共享的基础能力，例如 analyzer/binder、类型诊断、metadata/statistics、Page execution profile、标准库 conformance。这些能力一旦补好，会同时改善 runtime、LSP、optimizer 和文档。</p>
<p>优先级较低的是孤立能力，例如临时加一个外部 IO package、支持某个少见语法糖、或者为一个 demo 写特殊 builtin。它们看起来能快速增加功能列表，但不一定提升项目整体质量。</p>
<h2 id="语言层路线图" class="headerLink">
    <a href="#%e8%af%ad%e8%a8%80%e5%b1%82%e8%b7%af%e7%ba%bf%e5%9b%be" class="header-mark"></a>语言层路线图</h2><p>最值得补的是语义分析和类型检查。当前很多错误要到 runtime 才暴露，例如参数类型不匹配、函数返回值不符合 builtin 预期、对象字段不存在等。</p>
<p>类型检查不一定要一开始追求完整 Hindley-Milner。更现实的方式是先做 binder：</p>
<ul>
<li>绑定 import、package、builtin、变量和函数。</li>
<li>给常见 builtin 建立参数/返回形状。</li>
<li>对明显错误给出诊断。</li>
<li>为 LSP hover/signatureHelp 提供类型信息。</li>
</ul>
<p>其次是 formatter。LSP 已经有 formatter 模块，但要达到日常可用，还需要更多真实文件快照和格式风格约束。</p>]]></description></item></channel></rss>