完成查询优化:谓词下推、投影下推和常量折叠,配置好了benchmark
This commit is contained in:
1 parent
cae66f0731
commit
0791fe0446
14 files changed
+771
-7
No files matched your search
@@ -0,0 +1,93 @@
|
||||
# ChuSQL 基准测试
|
||||
|
||||
这个目录放**性能基准测试**,和 `chusql-engine`、`chusql-storage` 的功能代码分开,
|
||||
因为它回答的问题不一样:功能代码回答"对不对",基准测试回答"快不快、快多少"。
|
||||
|
||||
```
|
||||
benchmark/
|
||||
├── README.md 本文件:怎么跑、测什么、结果怎么解读
|
||||
├── chusql-engine/ Haskell 查询层:优化器开/关的对比
|
||||
│ ├── stack.yaml 把 chusql-engine 作为本地包引入
|
||||
│ ├── chusql-bench.cabal
|
||||
│ └── src/Main.hs
|
||||
└── chusql-storage/ Rust 存储层:预留(见该目录下的 README)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 怎么跑
|
||||
|
||||
```bash
|
||||
cd benchmark/chusql-engine
|
||||
stack run # 默认 users=500 行、orders=500 行
|
||||
stack run -- 1000 1000 0.5 # 也可以自己指定:users 行数、orders 行数、每批目标秒数
|
||||
```
|
||||
|
||||
第一次跑要编译,之后很快。整个默认跑完约 5 秒。
|
||||
|
||||
---
|
||||
|
||||
## 测什么
|
||||
|
||||
四个用例,**每个都跑两遍**:一遍走"翻译出来的原始工序单",一遍走"优化后的工序单"。
|
||||
两次的结果行数会被打印出来互相对照,**不一致会当场标出来**。
|
||||
|
||||
| 用例 | 想看什么 |
|
||||
| ---- | -------- |
|
||||
| `JOIN` + `WHERE u.age > 90` | 谓词下推的主场:条件能筛掉约 90% 的行 |
|
||||
| `JOIN`(无 WHERE) | 只剩投影下推在起作用,收益应该温和 |
|
||||
| `JOIN` + `WHERE` + `ORDER BY` + `LIMIT` | 规则叠加时的综合效果 |
|
||||
| 单表 `WHERE age > 90` | 对照组:没有下推空间,优化器只剩固定开销 |
|
||||
|
||||
---
|
||||
|
||||
## 计时怎么做(两个坑都踩过了)
|
||||
|
||||
**坑一:Haskell 是惰性的,不强制求值就等于没测。**
|
||||
如果只写 `let r = evalRelOp db plan`,`r` 根本没被算出来,测到的是"定义一个变量"的时间。
|
||||
所以每次必须把结果**算到底**(本项目里是逐行把列数累加起来当校验和)。
|
||||
|
||||
**坑二:Windows 上 CPU 时钟的精度只有约 15.6 毫秒。**
|
||||
优化后的查询只要几毫秒,直接测会得到 `0.0 ms` 这种没意义的数字。
|
||||
|
||||
所以计时用的是**自适应批量**:把同一个用例连续跑 `k` 次测一整批,`k` 不断翻倍,
|
||||
直到一批累计耗时超过目标值(默认 0.3 秒),再折算成单次耗时。
|
||||
这样时钟精度的误差会被 `k` 摊薄,几毫秒的用例也能测准。
|
||||
程序会打印实际用了多大的批次(例如"优化后 64 次/批")。
|
||||
|
||||
跑正式计时之前还会**预热两遍**,避免把首次构造表数据的开销算进去。
|
||||
|
||||
---
|
||||
|
||||
## 实测结果
|
||||
|
||||
**环境**:Windows,GHC 9.10.3,`-O2`
|
||||
**数据**:`users` 500 行(`id / name / age`,`age` 只有 0-99 共 100 种取值),
|
||||
`orders` 500 行(`id / user_id / product`),两表配对共 25 万对。
|
||||
|
||||
| 查询 | 结果行数 | 未优化 | 优化后 | 加速 |
|
||||
| ---- | -------- | ------ | ------ | ---- |
|
||||
| `JOIN` + `WHERE u.age > 90` | 45 | 125.0 ms | **7.1 ms** | **17.7x** |
|
||||
| `JOIN`(无 WHERE) | 500 | 132.8 ms | 105.5 ms | 1.3x |
|
||||
| `JOIN` + `WHERE` + `ORDER BY` + `LIMIT 10` | 10 | 171.9 ms | **10.3 ms** | **16.8x** |
|
||||
| 单表 `WHERE age > 90` | 45 | 0.111 ms | 0.153 ms | 0.7x |
|
||||
|
||||
### 怎么解读
|
||||
|
||||
- **前两行的差距来自谓词下推。** 条件筛掉了约 90% 的用户,配对次数从 25 万降到约 2.25 万,
|
||||
时间直接掉一个数量级。筛得越狠,省得越多。
|
||||
- **第二行只有投影下推。** 配对次数一点没变(没有条件可下推),只是每一行变窄了,
|
||||
所以只有 1.3 倍 —— 收益温和但确实是正的。
|
||||
- **第四行反而略慢。** 单表查询本来就没有下推空间(条件早已贴着扫描),
|
||||
而优化器自己要走一遍规则,有几十微秒的固定开销(0.153 − 0.111 ≈ 0.04 ms)。
|
||||
**优化不是零成本的**:查询越大、配对越多,这笔账才越划算。
|
||||
|
||||
---
|
||||
|
||||
## 加新用例
|
||||
|
||||
在 `src/Main.hs` 的 `queries` 列表里加一条 SQL 字符串就行,其余全自动
|
||||
(会自动翻译、优化、跑两遍、比结果、算加速比)。
|
||||
|
||||
想测别的规模,直接命令行传参;想在文档里引用数字,记得把**数据规模一起写上** ——
|
||||
脱离规模谈"快了几倍"是没有意义的。
|
||||
@@ -0,0 +1,14 @@
|
||||
cabal-version: 2.4
|
||||
|
||||
name: chusql-bench
|
||||
version: 1.0.0
|
||||
synopsis: ChuSQL 查询优化基准测试(对比开/关优化器)
|
||||
build-type: Simple
|
||||
|
||||
executable chusql-bench
|
||||
main-is: Main.hs
|
||||
hs-source-dirs: src
|
||||
build-depends: base >= 4.14 && < 5
|
||||
, chusql-engine
|
||||
default-language: Haskell2010
|
||||
ghc-options: -O2 -Wall
|
||||
@@ -0,0 +1,118 @@
|
||||
module Main where
|
||||
|
||||
import ChuSQL.Algebra.Eval (evalRelOp)
|
||||
import ChuSQL.Algebra.Optimize (optimize)
|
||||
import ChuSQL.Algebra.Planner (translate)
|
||||
import ChuSQL.Model
|
||||
import ChuSQL.SQLSyntax.Parser (parseQuery)
|
||||
import Control.Exception (evaluate)
|
||||
import System.CPUTime (getCPUTime)
|
||||
import System.Environment (getArgs)
|
||||
import Text.Printf (printf)
|
||||
|
||||
mkUsers :: Int -> Table
|
||||
mkUsers n =
|
||||
Table
|
||||
{ tableName = "users"
|
||||
, tableCols = ["id", "name", "age"]
|
||||
, tableRows =
|
||||
[ [("id", VInt i), ("name", VStr ("user" ++ show i)), ("age", VInt (i `mod` 100))]
|
||||
| i <- [1 .. n]
|
||||
]
|
||||
}
|
||||
|
||||
mkOrders :: Int -> Int -> Table
|
||||
mkOrders n m =
|
||||
Table
|
||||
{ tableName = "orders"
|
||||
, tableCols = ["id", "user_id", "product"]
|
||||
, tableRows =
|
||||
[ [("id", VInt j), ("user_id", VInt (1 + (j - 1) `mod` n)), ("product", VStr ("item" ++ show (j `mod` 50)))]
|
||||
| j <- [1 .. m]
|
||||
]
|
||||
}
|
||||
|
||||
checksum :: Either String [Row] -> Int
|
||||
checksum (Left _) = -1
|
||||
checksum (Right rows) = go 0 rows
|
||||
where
|
||||
go acc [] = acc
|
||||
go acc (r : rs) = go (acc + length r) rs
|
||||
|
||||
rowCount :: Either String [Row] -> Int
|
||||
rowCount (Left _) = -1
|
||||
rowCount (Right rows) = length rows
|
||||
|
||||
runOnce :: Database -> String -> Bool -> IO Int
|
||||
runOnce db sql useOpt = do
|
||||
let result = case parseQuery sql >>= translate db of
|
||||
Left e -> Left e
|
||||
Right plan -> evalRelOp db (if useOpt then optimize db plan else plan)
|
||||
forced <- evaluate (checksum result)
|
||||
return forced
|
||||
|
||||
resultRows :: Database -> String -> Bool -> IO Int
|
||||
resultRows db sql useOpt = do
|
||||
let result = case parseQuery sql >>= translate db of
|
||||
Left e -> Left e
|
||||
Right plan -> evalRelOp db (if useOpt then optimize db plan else plan)
|
||||
_ <- evaluate (checksum result)
|
||||
return (rowCount result)
|
||||
|
||||
timed :: IO a -> IO Double
|
||||
timed act = do
|
||||
t0 <- getCPUTime
|
||||
_ <- act
|
||||
t1 <- getCPUTime
|
||||
return (fromIntegral (t1 - t0) / 1e12)
|
||||
|
||||
benchOp :: Double -> IO Int -> IO (Double, Int)
|
||||
benchOp target act = loop 1
|
||||
where
|
||||
loop k = do
|
||||
t <- timed (mapM_ (const act) [1 .. k])
|
||||
if t < target then loop (k * 2) else return (t / fromIntegral k, k)
|
||||
|
||||
report :: Database -> Double -> String -> IO ()
|
||||
report db target sql = do
|
||||
nUn <- resultRows db sql False
|
||||
nOp <- resultRows db sql True
|
||||
(tUn, kUn) <- benchOp target (runOnce db sql False)
|
||||
(tOp, kOp) <- benchOp target (runOnce db sql True)
|
||||
let speedup = if tOp <= 0 then 0 else tUn / tOp
|
||||
mismatch = if nUn == nOp then "" else " <<< 结果不一致!"
|
||||
printf "%s\n" sql
|
||||
printf
|
||||
" 结果行数 %-6d | 未优化 %9.3f ms | 优化后 %9.3f ms | 加速 %6.1fx%s\n"
|
||||
nOp
|
||||
(tUn * 1000)
|
||||
(tOp * 1000)
|
||||
speedup
|
||||
mismatch
|
||||
printf " (实测批次:未优化 %d 次/批,优化后 %d 次/批)\n\n" kUn kOp
|
||||
|
||||
queries :: [String]
|
||||
queries =
|
||||
[ "SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE u.age > 90"
|
||||
, "SELECT u.name, o.product FROM users u JOIN orders o ON u.id = o.user_id"
|
||||
, "SELECT u.name FROM users u JOIN orders o ON u.id = o.user_id WHERE u.age > 90 ORDER BY o.product LIMIT 10"
|
||||
, "SELECT name FROM users WHERE age > 90"
|
||||
]
|
||||
|
||||
warmupSql :: String
|
||||
warmupSql = "SELECT name FROM users"
|
||||
|
||||
main :: IO ()
|
||||
main = do
|
||||
args <- getArgs
|
||||
let pick i d = if length args > i then read (args !! i) else d
|
||||
n = pick 0 500
|
||||
m = pick 1 500
|
||||
target = pick 2 (0.3 :: Double)
|
||||
db = [("users", mkUsers n), ("orders", mkOrders n m)]
|
||||
printf "数据规模:users = %d 行,orders = %d 行\n" n m
|
||||
printf "计时方式:CPU 时间;每个用例自动加倍批量次数,直到一批累计耗时超过 %.2f 秒,再折算成单次耗时\n" target
|
||||
printf "(先跑一遍预热,避免把首次构造数据的开销算进去)\n\n"
|
||||
_ <- runOnce db warmupSql False
|
||||
_ <- runOnce db warmupSql True
|
||||
mapM_ (report db target) queries
|
||||
@@ -0,0 +1,5 @@
|
||||
resolver: lts-24.59
|
||||
|
||||
packages:
|
||||
- .
|
||||
- ../../chusql-engine
|
||||
@@ -0,0 +1,12 @@
|
||||
# This file was autogenerated by Stack.
|
||||
# You should not edit this file by hand.
|
||||
# For more information, please see the documentation at:
|
||||
# https://docs.haskellstack.org/en/stable/topics/lock_files
|
||||
|
||||
packages: []
|
||||
snapshots:
|
||||
- completed:
|
||||
sha256: 0f728e30e843d7f00460ad56858b995a3b227fca683659c7df9f2fc3988a3ce1
|
||||
size: 732465
|
||||
url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/24/59.yaml
|
||||
original: lts-24.59
|
||||
@@ -0,0 +1,36 @@
|
||||
# chusql-storage 基准测试(预留)
|
||||
|
||||
这个目录预留给 **Rust 存储层** 的基准测试代码,目前 `chusql-storage/` 还没有开工,
|
||||
所以这里先占位。
|
||||
|
||||
## 计划放什么
|
||||
|
||||
`chusql-storage` 做的是磁盘那一层的事情,性能指标和查询层完全不同 ——
|
||||
查询层关心"少算几行",存储层关心"少读几次盘"。所以基准要分开测:
|
||||
|
||||
| 待测模块 | 关心的指标 |
|
||||
| -------- | ---------- |
|
||||
| 页管理 | 单页读写延迟、顺序读 vs 随机读吞吐 |
|
||||
| B+ 树 | 插入/查找/范围扫描的耗时随数据量增长的趋势 |
|
||||
| 缓冲池 | 命中率、缺页次数、替换策略(LRU 等)的对比 |
|
||||
| WAL | 追加写吞吐、日志落盘延迟、恢复耗时 |
|
||||
| 崩溃恢复 | 从日志重放的时间、恢复后数据一致性 |
|
||||
|
||||
## 用什么
|
||||
|
||||
Rust 侧的基准测试一般用 [criterion](https://github.com/bheisler/criterion.rs):
|
||||
|
||||
```
|
||||
benchmark/chusql-storage/
|
||||
├── Cargo.toml
|
||||
└── benches/
|
||||
├── page.rs
|
||||
├── btree.rs
|
||||
├── buffer_pool.rs
|
||||
└── wal.rs
|
||||
```
|
||||
|
||||
跑法:`cargo bench`
|
||||
|
||||
另外还要和 Haskell 层对齐口径:查询层的基准是"整条查询的耗时",
|
||||
存储层的基准是"单次 IO / 单次查找的耗时",两边不要混在一起比。
|
||||
Reference in new issue
Block a user