Malisper

RSS: https://malisper.me/feed/
malisper.me

让 Postgres 分析查询提速 300 倍:批处理、算子融合与 SIMD

pgrust 0.2 发布,分析查询性能比 Postgres 快 300 倍,在 ClickBench 上甚至超过 ClickHouse。核心优化来自查询引擎重构:批处理、算子融合和 SIMD。Volcano 模型每次处理一行带来巨大开销,批处理将 5 亿数求和从 1.3 秒降至 480 毫秒,算子融合进一步降至 358 毫秒,SIMD 优化后仅需 135 毫秒。Postgres 诞生于磁盘 I/O 瓶颈时代,如今数据常驻内存、NVMe 普及,CPU 和内存带宽成为新瓶颈,传统查询引擎架构已不适应。
评论点赞收藏9 天前

PostgreSQL 转写 Rust:在通过全部回归测试前的三个死胡同

Mali 团队耗时重写 PostgreSQL 核心代码至 Rust,经历三次失败并花费十万美元后,最终通过全部回归测试。文章详细记录了从 C 到 Rust 迁移过程中的技术陷阱、性能权衡及具体实现细节,包括内存管理、并发模型适配及编译期检查策略,为大型 C 项目现代化改造提供了极具参考价值的一手工程经验。
评论点赞收藏31 天前

pgrust 构建实录:四次尝试与十万美金投入,用 AI 重写 Postgres

简而言之:经过四次尝试,并投入了10万美元,我们成功利用Opus将Postgres改写成了Rust。 我们明确了若干关键技能,并能够建立一套可重复的流程,将Postgres的文件改写为Rust代码。 在运行过程中…… 阅读其余部分 文章“prust 是如何构建的:用 AI 用 Rust 重写 Postgres 的四次尝试 ”最初发表于。
评论点赞收藏31 天前

pgrust 通过 100% PostgreSQL 回归测试

pgrust 项目宣布通过全部 PostgreSQL 回归测试及隔离测试,兼容 PG 18.3。作为用 Rust 重写 Postgres 的尝试,这一进展验证了其在核心兼容性上的可行性。对于关注数据库底层重构、Rust 在基础设施领域落地以及 Postgres 替代方案的技术读者而言,这是评估该项目成熟度的关键里程碑。
评论点赞收藏36 天前

导致PostgreSQL宕机的四大杀手

作者基于大量初创公司生产事故经验,总结导致PostgreSQL宕机的四大核心原因:VACUUM与事务ID回卷、进程模型导致的连接瓶颈、查询计划器缺乏控制且易出错、JSON字段统计缺失引发的性能灾难。针对这些问题,作者介绍了其重构项目pgrust的技术方案,包括引入64位事务ID、改用线程模型、构建自适应查询计划器以及改进JSON统计与压缩。
评论点赞收藏37 天前

Pgrust: Rebuilding Postgres in Rust with AI

I’ve written dozens of blog posts about Postgres internals. Postgres is one of the greatest databases out there. But from chatting with a lot of my friends at startups, I’ve seen consistent challenges...
评论点赞收藏117 天前

Simple Rules for Building Simple Systems

Once a design of a system meets all the business constraints necessary, my next priority is to make the system as simple as possible. By simple, I mean a system that is both easy to understand and eas...
评论点赞收藏2337 天前

An Algorithm for Passing Programming Interviews

Over the past few years, I’ve interviewed with a dozen or so companies and have completed ~50 or so individual algorithm problems. I’m frequently given feedback that I did a great job at the algorithm...
评论点赞收藏2353 天前

Tools in My Toolbox

One of the most important skills I believe programmers need is to understand when to use certain tools. My typical process for solving a problem is to go through each tool in my toolbox and see which ...
评论点赞收藏2378 天前

Advent of Code: Day 3

Day 3’s problem was an interesting one. We are given two wires and have to find the point where the two wires intersect that’s closest to the origin. As usual, we’ll approach this problem in two steps...
评论点赞收藏2447 天前

Advent of Code: Day 2

Day 2 of the Advent of Code is a classic – implement a virtual machine for a simple assembly like. Eric Wastl, the creator of the AoC, has been writing challenges like this for years before the AoC ev...
评论点赞收藏2448 天前

Advent of Code: Day 1

The Advent of Code is back! In past iterations of the Advent of Code, a bunch of my friends and I would share all our solutions. This year, I thought why not go the extra step and publicly write up … ...
评论点赞收藏2449 天前

Generating Fractals with SQL Queries

I recently gave a talk at!!Con West about how you can use some of the dark corners of SQL to generate fractals. You can watch the recording here: To summarize the video, I write SQL queries that gene...
评论点赞收藏2700 天前

Perfalytics is now Generally Available

After being in beta for several months, Perfalytics is now generally available. Perfalytics is my attempt to bring a new approach to database performance. Perfalytics integrates directly with your da...
评论点赞收藏2706 天前

Postgres Backups with Continuous Archiving

It’s extremely important that you have backups. Without backups, you are at serious risk of losing data. If you don’t have backups, all it takes is one machine failure or fat fingered command to lose ...
评论点赞收藏3093 天前

Postgres JSONB

JSONB is a nifty Postgres type that allows you to store unstructured data inside of Postgres. A common use case of JSONB is to represent a mapping from a set of keys to arbitrary values. JSONB is nice...
评论点赞收藏3334 天前

track_io_timing

The parameter track_io_timing is a relatively unknown, but super helpful parameter when optimizing queries. As the name suggests, when the parameter is turned on, Postgres will track how long I/O take...
评论点赞收藏3335 天前

登录芦苇

登录后关注作者、收藏内容和参与讨论。