8c3ac22d8d
- Extract bunnative→nativets tag logic into ScriptLang::as_worker_tag() - Add benchmark results for batch pull vs direct SQL (1W and 3W) - Add throughput model script comparing batch vs SQL at scale - Add nativets_sleep benchmark script support Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
94 lines
3.9 KiB
Markdown
94 lines
3.9 KiB
Markdown
# Batch Pull Benchmark Results
|
|
|
|
Date: 2026-03-06
|
|
Setup: 1 server + 1 native worker (8 subworkers), standalone mode
|
|
Hardware: fedora, 1.5TB disk, ~1GB memory usage
|
|
DB: PostgreSQL local, windmill 270 MiB
|
|
|
|
## nativets — 1000 jobs
|
|
|
|
| | Batch Pull | Direct SQL |
|
|
|--|-----------|------------|
|
|
| Duration | 9.2s | 11.4s |
|
|
| **Throughput** | **108 jobs/s** | **88 jobs/s** |
|
|
| Improvement | **+23%** | baseline |
|
|
|
|
### pg_stat_statements
|
|
|
|
| Query | Batch calls | Batch ms | SQL calls | SQL ms |
|
|
|-------|------------|----------|-----------|--------|
|
|
| Native pull (FOR UPDATE SKIP LOCKED) | 2,399 (0.02ms avg) | 47 | 2,849 (0.03ms avg) | 87 |
|
|
| Default worker pull | 520 (0.04ms avg) | 19 | 445 (0.05ms avg) | 23 |
|
|
| DELETE from queue | 1,001 (0.23ms avg) | 231 | 1,001 (0.19ms avg) | 195 |
|
|
| INSERT into completed | 1,001 (0.05ms avg) | 51 | 1,001 (0.04ms avg) | 45 |
|
|
| INSERT job_logs | 2,003 (0.02ms avg) | 44 | 2,003 (0.02ms avg) | 44 |
|
|
| Agent token blacklist | 3,920 (0.00ms avg) | 19 | — | — |
|
|
| **Total** | **9,389** | **1,332** | **8,882** | **1,212** |
|
|
|
|
### pg_stat_database
|
|
|
|
| Metric | Batch Pull | Direct SQL |
|
|
|--------|-----------|------------|
|
|
| Transactions committed | 6,484 | 5,868 |
|
|
| Blocks read (disk) | 101 | 101 |
|
|
| Blocks hit (cache) | 920,257 | 699,032 |
|
|
| Tuples returned | 7,784,044 | 8,987,097 |
|
|
| Tuples fetched | 1,028,819 | 805,100 |
|
|
| Tuples inserted | 6,822 | 6,880 |
|
|
| Tuples updated | 3,377 | 3,090 |
|
|
| Tuples deleted | 2,883 | 2,654 |
|
|
|
|
---
|
|
|
|
## nativets_sleep — 1000 jobs
|
|
|
|
Each job sleeps 300-700ms (random). Theoretical max with 8 workers: ~16 jobs/s.
|
|
|
|
| | Batch Pull | Direct SQL |
|
|
|--|-----------|------------|
|
|
| Duration | 66.7s | 68.2s |
|
|
| **Throughput** | **15.0 jobs/s** | **14.7 jobs/s** |
|
|
| Improvement | ~same | baseline |
|
|
|
|
### pg_stat_statements
|
|
|
|
| Query | Batch calls | Batch ms | SQL calls | SQL ms |
|
|
|-------|------------|----------|-----------|--------|
|
|
| Native pull (FOR UPDATE SKIP LOCKED) | 2,399 (0.02ms avg) | 43 | 1,762 (0.04ms avg) | 75 |
|
|
| Default worker pull | 1,591 (0.07ms avg) | 113 | 1,392 (0.08ms avg) | 115 |
|
|
| DELETE from queue | 1,001 (0.31ms avg) | 308 | 1,001 (0.20ms avg) | 205 |
|
|
| INSERT into completed | 1,001 (0.05ms avg) | 46 | 1,001 (0.04ms avg) | 41 |
|
|
| INSERT job_logs | 2,003 (0.02ms avg) | 45 | 2,003 (0.02ms avg) | 40 |
|
|
| Job runtime ping | 1,490 (0.02ms avg) | 33 | 1,489 (0.02ms avg) | 31 |
|
|
| Worker ping (job) | 1,001 (0.03ms avg) | 32 | 1,001 (0.03ms avg) | 30 |
|
|
| **Total** | **16,523** | **5,798** | **16,364** | **5,717** |
|
|
|
|
### pg_stat_database
|
|
|
|
| Metric | Batch Pull | Direct SQL |
|
|
|--------|-----------|------------|
|
|
| Transactions committed | 13,638 | 13,388 |
|
|
| Blocks read (disk) | 394 | 850 |
|
|
| Blocks hit (cache) | 7,033,158 | 4,932,617 |
|
|
| Tuples returned | 58,424,571 | 57,204,918 |
|
|
| Tuples fetched | 8,776,186 | 6,321,947 |
|
|
| Tuples inserted | 7,500 | 7,531 |
|
|
| Tuples updated | 6,111 | 6,193 |
|
|
| Tuples deleted | 3,006 | 3,196 |
|
|
|
|
---
|
|
|
|
## Analysis
|
|
|
|
**Throughput**: +23% for fast CPU-bound jobs. Negligible difference for I/O-bound jobs.
|
|
|
|
**Pull queries**: Batch pull does MORE pull queries for nativets_sleep (2,399 vs 1,762). The refiller polls every 50ms even when all workers are busy executing jobs. With direct SQL, workers only poll when idle. This is wasted work — the refiller queries DB and gets empty results while jobs are in-flight.
|
|
|
|
**Disk I/O**: Batch pull cuts disk reads in half for nativets_sleep (394 vs 850 blocks). Likely because the batch query locks multiple rows in one pass, reducing index traversal.
|
|
|
|
**Cache hits**: Higher with batch pull (7M vs 4.9M for sleep). More buffer hits from the refiller's repeated empty polls touching the same index pages.
|
|
|
|
**Tuples fetched**: Higher with batch pull (8.7M vs 6.3M for sleep). Same cause — the refiller's empty polls scan the index.
|
|
|
|
**At scale**: With 8 subworkers the differences are small. The real benefit is with many native workers where direct SQL SKIP LOCKED contention grows O(N²).
|