crabgresql

CrabgreSQL

A PostgreSQL-compatible DBMS written in Rust. See docs/ARCHITECTURE.md for the design and ROADMAP.md for the development sequence.

Status: M1 in progress — CRUD. The full parse → bind → plan → execute pipeline runs simple CRUD end-to-end against the in-memory storage engine:

$ cargo run -p crabgresql-server
$ psql -h 127.0.0.1 -p 5433
=> CREATE TABLE crabs (id integer, name text);
=> INSERT INTO crabs VALUES (1, 'ferris'), (2, 'hermit');
=> SELECT id + 1, name FROM crabs WHERE id <> 2;
=> UPDATE crabs SET name = 'red' WHERE id = 1;
=> DELETE FROM crabs WHERE name = 'red';

What exists today:

Tests: cargo test — unit tests per crate plus end-to-end tests that drive a real driver (tokio-postgres) and raw-socket handshake checks.

Configuration

Every environment variable the server reads is declared in one place, crabgresql-config; a value that does not parse falls back to its default rather than failing startup.

Variable Default Range Controls
CRABGRESQL_PORT 5433   TCP port to listen on (also --port)
PGDATA ./pgdata   data directory the durable heap engine is opened in (also --data-dir)
CRABGRESQL_COPY_ALLOW_PATHS (empty)   extra directories a server-side COPY … FROM '<file>' may read, colon-separated (also repeatable --copy-allow-path). The data directory is always readable and is where a relative path resolves; nothing else is, because the read runs with the server’s privileges
RUST_LOG info   tracing filter directives
CRABGRESQL_BUFFER_TABLE_SOFT_BYTES 32MB 1MB–2GB per-relation buffered bytes that make one write buffer flush-eligible
CRABGRESQL_BUFFER_GLOBAL_HARD_BYTES 256MB 1MB–16GB buffered bytes across all relations that make every buffer eligible
CRABGRESQL_BUFFER_MAX_AGE 1m 10ms–24h how long a write buffer may hold rows before being flushed anyway
CRABGRESQL_BUFFER_TICK 1s 10ms–1h how often the background flush worker looks for eligible buffers
CRABGRESQL_SHARED_BUFFERS 128MB 8MB–16GB RAM the buffer pool holds relation pages in, rounded down to whole 8 KiB frames

Sizes take a bare byte count or a binary unit — kB, MB, GB, TB, with the trailing B optional, so 33554432, 32MB and 32m all say the same thing. Durations take a bare count of milliseconds or a unit — ms, s, m (also min), h — so 60000, 60s and 1m are one value. Units are matched case-insensitively; mind that in a duration m is minutes and ms is milliseconds.

A value outside the supported range is clamped to the nearest end of it, and one that cannot be read at all falls back to the default; either way the server logs a warning and starts, because a typo in a tuning knob should not keep it down.

The two *_BYTES knobs count what buffered rows occupy in RAM, not what they would serialize to. A row costs size_of::<Value>() per column whatever that column holds, so a wide analytics table runs several kilobytes a row and a 32 MB buffer holds proportionally fewer rows than its encoded size suggests. Past CRABGRESQL_BUFFER_GLOBAL_HARD_BYTES an autocommit write waits for the flush worker to make room rather than adding to the total; a write inside an explicit transaction block does not, because it already holds the transaction ID that bounds what a flush is allowed to reclaim.

The CRABGRESQL_BUFFER_* knobs are environment variables rather than GUCs because a SET is session-scoped and the flush worker is process-wide; moving them to real storage settings is a follow-up.

PostgreSQL regression tests

The PostgreSQL regression corpus (src/test/regress, pinned to a master commit) is vendored under vendor/postgres/ — populate or bump it with scripts/sync-regress.sh. The pg_regress-style runner in crabgresql-pg-regress executes the scripts against an in-process server, emulating psql -a -q output, and diffs against expected/*.out:

$ cargo run -p crabgresql-pg-regress --bin regress            # full schedule (compat %)
$ cargo run -p crabgresql-pg-regress --bin regress -- --tests boolean,int4
20 of 245 tests passed (8%).
See target/regress/regression.diffs for details.

The score is the compatibility dashboard, so a near-zero percentage at M0 is expected and honest. Regression protection lives in cargo test: the crabgresql-authored smoke suite must always pass, plus every upstream test promoted to crates/crabgresql-pg-regress/suites/upstream_must_pass.txt as coverage grows.

Benchmarks

crabgresql-bench runs published analytical benchmarks — ClickBench for scans and aggregation over one wide table, TPC-H for joins over eight — against an in-process server, or against stock PostgreSQL for comparison. A query that hits an engine gap is reported in place instead of aborting the run, so the results table doubles as a gap list. See crates/crabgresql-bench/README.md.

$ cargo run --release -p crabgresql-bench -- run clickbench --data hits.tsv
43 of 43 queries succeeded, 13.228s total (best runs)

$ cargo run --release -p crabgresql-bench -- run tpch --data tpch/ --timeout 180
22 of 22 queries succeeded, 123.329s total (best runs)  # not a TPC-H result
This site is open source. Improve this page.