v1.4.0 — The Native Core¶

Released: October 2026

One model declaration, compiled once in Rust, drives the ORM, request validation, response serialization and OpenAPI. Design background: the v1.4.0 plan.

Headline numbers¶

Cello 1.3.0 against 1.4.0, both servers running at the same time, wrk -t2 -c32, 10 alternating rounds of 5 s per endpoint, local Postgres (benchmarks/orm/paired.py; raw results in benchmarks/orm/results-v130-vs-v140.json).

Endpoint	1.3.0 rps	1.4.0 rps	Change (mean)
200 rows, `return {"users": qs}` (row → JSON fast path)	112	2,207	+1,869% (19.7×)
200 rows, `[u.to_dict() for u in users]`	111	418	+278% (3.8×)
200 rows, raw `db.fetch`	156	511	+227% (3.3×)
`Model.objects.get(id=…)`	999	5,373	+438% (5.4×)
`Model.objects.create(…)`	1,078	4,977	+362% (4.6×)
`{"ok": true}` control	33,157	35,598	+7.4% (within this box's noise)
`body=` dataclass DTO, 20 fields	10,616	10,577	−0.4% (unchanged)

How to read this:

The box is a shared WSL2 VM. Its A/A noise floor is about ±1.5% on the mean (≈5% on the very fast control endpoint), see benchmarks/orm/RESULTS.md. Only the database-backed endpoints moved by more than the noise, and by far more.
Response bodies are not byte-identical to 1.3.0 on the database endpoints: 1.4.0 fixes NUMERIC (1.3.0 returned a placeholder string) and other value shapes. The harness flags this; it is expected.
The control and DTO endpoints exercise code 1.4.0 did not change on that path, so "unchanged" is the expected result. The native body=Model validator is measured separately in the Phase 03 section (18.1k vs 13.4k rps for Pydantic body=).
These are one paired run on a shared machine. Re-run with the command in benchmarks/orm/README.md before quoting a figure externally.

What is in 1.4.0¶

Area	Pillar	Summary
Prepared-statement cache, async bridge, serializer fast path, row → JSON pipeline	Speed	Sections below
Identifier-injection fix, `write_only`/`read_only`, Postgres TLS, null-byte rejection	Security	Sections below
Schema registry, native `body=Model` validation, OpenAPI from models, Q objects, aggregates, relations, bulk operations, `cello migrate`	Simplicity	Sections below

Deliberate breaking changes (each is listed with its migration in the sections "Deliberate breaks — ORM" and "Value layer"): undeclared column names raise FieldError; NUMERIC/arrays/intervals decode to real values instead of a placeholder; out-of-range integers raise instead of wrapping; fetchrow/fetchval return the first row instead of raising; exclude() is null-safe; filter(x=None) means IS NULL; database errors are DatabaseError subclasses (still RuntimeError); the default Postgres sslmode is prefer.

Also fixed: async def shutdown hooks were skipped (with a "coroutine was never awaited" warning) when the server was stopped with Ctrl-C; SIGTERM was unaffected. This dates from 1.3.0. Ctrl-C reached CPython's signal handler as well as the server's and left a pending KeyboardInterrupt that the hook's first Python call raised and the server then filtered out. The pending interrupt is now consumed before shutdown hooks run (tests/test_shutdown_hooks.py).

Security fix — production error responses: an unhandled exception in a handler used to return its full text to the client in every environment, including env="production". With 1.4.0's database error messages that meant constraint names and row values (DETAIL: Key (email)=(a@b.c) already exists) could reach the client. With debug off (including env="production") the response is now a generic {"error": "Internal Server Error", "status": 500} and the details go to the server log with the request method and path. Development still returns the description (tests/test_error_exposure.py).

Phase 00 — Prepared statement cache and identifier guard¶

The plan schedules both of these for a 1.3.1 backport, ahead of the rest of 1.4.0.

Speed: prepared statements are cached per connection¶

Before: every fetch / fetchrow / fetchval / execute, on the pool and inside transaction(), passed a SQL string to tokio-postgres. Each call prepared an unnamed statement first: a Parse/Describe/Sync round trip to Postgres before the Bind/Execute round trip, followed by a Close. Every ORM call and every raw db.fetch() paid this.

Now: src/db/postgres.rs prepares through the pooled connection's statement cache (deadpool-postgres prepare_cached) and runs the cached Statement. The first call with a given SQL string on a connection does the prepare. Every later call with the same string is a single Bind/Execute round trip. This includes transactions, because a transaction holds a pooled connection.

Parameter binding is unchanged. SqlParam still picks its encoding from the target column type, and those types come from the cached statement.

Measured (loopback Postgres on the dev box, pool size 1, 1,000 sequential fetchval("SELECT $1::int", i)):

	Parse	Describe	Close	Sync
1.3.0	1,000	1,000	999	2,999
Phase 00	1	1	0	1,001

These counts come from a small proxy that counts the frontend protocol messages between Cello and Postgres.

Latency was measured with 2,000 sequential fetchval calls. The 1.3.0 build and the Phase 00 build ran in alternation, 10 runs each, on the same machine under the same load:

	median	min	max
1.3.0	502 µs/query	474	513
Phase 00	385 µs/query	351	401

That is about 23% lower latency per query on loopback. On loopback the network round trip is cheap, so most of the remaining ~385 µs is per-call overhead in the Python↔Rust async bridge, not network. The saving grows with network distance. With 1 ms of round-trip delay injected by a forwarding proxy, we measured 6.3 ms → 2.8 ms per query. The proxy delays every packet separately, so it overstates the baseline. Treat that pair as showing the direction of the effect, not as a benchmark.

Bounded cache. Each connection caches at most 512 statements. When it is full, that connection's cache is cleared, and dropping a statement deallocates it on the server. Code that builds SQL strings with literals in them therefore cannot grow server memory without limit. In our test, 1,300 distinct SQL strings on one connection left 277 server-side statements.

Schema changes (DDL) are handled. A cached statement can go stale. Example: a SELECT * cached before ALTER TABLE ... ADD COLUMN, or before a table is dropped and recreated with different columns. Postgres then rejects the statement with cached plan must not change result type (SQLSTATE 0A000). The same happens after DISCARD ALL / DEALLOCATE ALL (26000). Cello now:

On the pool (db.fetch and the other methods, plus the ORM): clears the statement cache of every connection in the pool, prepares the statement again and retries once. The caller does not see the error. We verified this for drop-and-recreate, ADD COLUMN, ALTER COLUMN ... TYPE and DEALLOCATE ALL.
Inside transaction(): a retry is not possible, because the failed statement has already aborted the transaction. The caches are cleared and the error is raised with an explicit message: "…the schema changed under a cached prepared statement, which aborted this transaction. Statement caches have been cleared — retry the transaction." The next transaction succeeds. This only happens when DDL changes a table between two uses of the same SQL text on the same connection, and the second use is inside a transaction.

Query errors now include the server's message. Previously a server error reached Python as the bare text db error. It now reads, for example, db error: ERROR: relation "x" does not exist.

Connection poolers

A pooler in front of Postgres that runs in transaction pooling mode can route the statements of one session to different server connections. Cello now relies on named prepared statements, which conflicts with that mode. Use session pooling, or a pooler version that supports prepared statements.

Security: identifiers in ORM queries are validated¶

Before: QuerySet._resolve_column() returned any unrecognised key verbatim, and the ORM pasted it between double quotes into SQL. Values were always parameterised, but keys were not. A request that passed user-controlled keys to filter / exclude / order_by / update was injectable. For example, the filter key x" = 1 OR "1 closed the quote.

Now (python/cello/orm.py), following step 1 of the migration path in the plan:

Invalid identifiers are rejected. A key or order_by name that does not match ^[A-Za-z_][A-Za-z0-9_]*$ raises cello.orm.FieldError, which subclasses ValueError and names the bad identifier. The check runs when the query is built, before any SQL reaches the database. The leading - of order_by and the field__lookup suffix are removed first, and what remains is validated. The match is ASCII-only and checks the whole string, so a trailing newline does not pass.
Declared names work without warnings. These are field names, a ForeignKey's <name>_id column (so filter(author_id=5) still works), and any db_column.
Undeclared names that are valid identifiers still work but are deprecated. They emit a DeprecationWarning that points at the calling line, says this becomes an error in 1.4.x, and names the two alternatives: declare the field on the model, or use await db.fetch(sql, *params).
Unsupported lookups on declared fields fail clearly. An unsupported lookup on a declared field, such as name__iexact, now raises a clear FieldError. Before, the whole key was treated as a column name and Postgres failed with an unrelated error.
Names from model declarations are escaped. Table names, db_column values and ForeignKey targets are quoted with embedded " doubled.
on_delete is checked. ForeignKey(on_delete=...) must be one of CASCADE, RESTRICT, SET NULL, SET DEFAULT or NO ACTION (any case).

Tests: tests/test_orm_injection.py (50 tests). Some use a recording stand-in pool to prove that rejected identifiers never reach the database. Others run against live Postgres and are skipped when it is unreachable. The existing ORM suites in tests/test_native_db.py pass unchanged with -W error::DeprecationWarning.

Phase 01 — The schema registry¶

No API change. Every ORM model is now compiled, once, into a Rust description of itself, and the ORM reads identifiers and DDL from that description instead of building them in Python.

Design¶

class User(Model) stays a Python declaration. When the class statement runs, the metaclass in python/cello/orm.py turns each field into a plain descriptor dict and calls cello._cello.schema_register(...). The new src/schema/ module compiles those into a ModelSpec and stores it in a process-wide registry (DashMap<ModelId, Arc<ModelSpec>>).

Type	Holds
`ModelSpec`	class name, table (raw and quoted), fields in metaclass order, primary key, the identifier map used for resolution
`FieldSpec`	name, column (raw and quoted), Postgres type, nullability, `unique`, `max_length`, default, `auto` (database-generated), `auto_now_add`, read/write visibility, foreign-key target and `on_delete`
`PgType`	`SERIAL`, `INTEGER`, `BIGINT`, `DOUBLE PRECISION`, `BOOLEAN`, `VARCHAR(n)`, `TEXT`, `JSONB`, `TIMESTAMPTZ`, or `Custom` (a field's own `sql_type`, kept verbatim)
`DefaultSpec`	none, `now()`, a callable (no DDL default), or a literal (bool / int / float / text / other)
`RelationSpec`	target table, target primary-key column, target `ModelId`, and an `OnDelete` from a closed set

Specs are plain Rust data (Send + Sync, no Python objects), so Phase 02 can read them from Tokio tasks without the GIL. PgType::wire_type() gives the tokio-postgres type a column decodes as, for the row-to-JSON path. Phase 03 reads the same FieldSpec for body validation, serialization and OpenAPI. The write_only visibility flag exists in the spec today and is always off; Phase 03 adds the write_only=True field option that sets it.

Identity. A ModelId comes from a counter and is never reused, unlike Python's id(). The class holds the owning handle at cls._meta.schema. When a class is garbage collected (for example a model defined inside a function or a test), its registry entry is removed, so redefining classes does not grow the registry.

The design is documented in full at the top of src/schema/mod.rs.

Security: identifiers resolve through the registry¶

The Phase 00 rules are unchanged: same FieldError and DeprecationWarning messages, raised in the same cases. What changed is where they are enforced.

The identifier check (^[A-Za-z_][A-Za-z0-9_]*$, ASCII, whole string) and quoting run in Rust (src/schema/ident.rs).
At class definition the registry hands Python a dict of every valid identifier that names a declared field or column, mapped to its quoted column. A field name wins over another field's column of the same name, as before. Declared columns that are not valid identifiers (db_column="my col") are not in the dict, so they still cannot be used as query keys.
A name that is not in the dict goes to ModelSpec.resolve() in Rust. It returns either None (Python raises FieldError) or a quoted undeclared identifier (Python emits the DeprecationWarning, so it still points at the caller's line).
Table names, INSERT/UPDATE column lists and the primary-key column in save() / delete() come from quoted values computed once by the registry. Python no longer quotes identifiers anywhere.
on_delete is checked again in Rust against the closed set, so changing field.on_delete after construction cannot reach DDL.

DDL is rendered in Rust¶

create_table() / drop_table() send ModelSpec.create_table_sql() / drop_table_sql(). Field.column_ddl() still works and renders through the same code. The output is byte-for-byte what 1.3.x produced, including the pinned quirks: a dict default renders as Python's str() of it (so JSONField(default={"a": 1}) still fails at create_table), and a ForeignKey to a model without a primary key still references "None". One hardening: an int/float subclass whose __str__ does not produce a number is now quoted as a string literal instead of being pasted into the DDL. Real ints and floats, including inf and nan, render exactly as before.

Two declaration errors used to surface only at create_table: a ForeignKey whose to is not a model class, and a custom sql_type callable that raises. They still do. The class defines, queries work, and create_table raises the original exception.

Speed¶

Identifier resolution: a paired micro-benchmark in one process, alternating the Phase 00 module and this build, 11 rounds of 300,000 calls each, run twice. Medians from the second run:

`_resolve_column(...)`	Phase 00	Phase 01
field name `title`	498 ns	132 ns
ForeignKey name `author`	536 ns	128 ns
ForeignKey column `author_id`	1,150 ns	133 ns
`db_column` `book_code`	1,303 ns	129 ns
undeclared `nickname` (warns)	2,885 ns	1,468 ns
invalid `a b` (raises `FieldError`)	791 ns	819 ns
`filter(title=…, author_id=…)`	3,276 ns	1,659 ns
`order_by("-author_id", "title")`	2,695 ns	1,154 ns

The Phase 00 cost was mostly the regex match and re-quoting the column on every call. A direct Rust call (schema.resolve) costs about 295 ns, which is still cheaper than Phase 00 but about five times the cost of a dict lookup (56 ns). That is why declared names use the dict the registry builds. The rejection path is about 30 ns (3–5%) slower, because it now misses the dict before reaching Rust. It ends in an exception, so it is not a hot path.

Class definition is a cold path and costs more: defining two models (2 + 8 fields) takes about 93 µs instead of 55 µs.

These numbers measure query building only. A query's database round trip is hundreds of microseconds, so they are not a request-level speedup.

Tests¶

tests/test_schema_registry.py (44 tests, no database needed): registry contents (foreign keys, db_column, subclass and mixin field order, defaults, custom types, models without a primary key), identifier resolution and rejection, _meta caches, DDL compared byte for byte against a copy of the 1.3.x Python renderer, deferred declaration errors, and registry lifetime. The compatibility suites (tests/test_orm_compat_*.py), tests/test_orm_injection.py and tests/test_native_db.py pass unchanged.

Deliberate breaks — ORM¶

1.3.x had ORM behaviours that looked like bugs. The compatibility suites pinned them with # PINNED-QUIRK: so they could not change silently. These are the ones 1.4.0 changes on purpose. Each changed assertion in tests/test_orm_compat_*.py, tests/test_orm_injection.py and tests/test_schema_registry.py is marked 1.4.0 (deliberate break). New tests: tests/test_orm_fixes_v140.py (45 tests).

Unchanged on purpose: the metaclass still removes field attributes from the class, a subclass's own fields still come before inherited ones, get() still replaces a user limit() with LIMIT 2, isnull=None still means IS NOT NULL, and save() on an existing row still writes every column.

1. Undeclared identifiers are an error¶

Step 2 of the plan's migration path. A filter / exclude / order_by / update / values name that is a valid identifier but is not a declared field or column now raises FieldError instead of a DeprecationWarning. The message names the column and points at the two alternatives: declare the field on the model, or use await db.fetch(sql, *params). A ForeignKey's <name>_id column and any db_column are still valid. Nothing is sent to the database.

Migrate: declare the column on the model, or move the query to db.fetch(...).

2. Defaults are applied when the instance is built¶

	1.3.x	1.4.0
`User(name="x").active` (`default=True`)	`None` until saved	`True`
callable default (`default=uuid4`)	never called; no value	called once per instance
`dict` / `list` default	shared object	deep-copied per instance
`create(score=None)`, `score` nullable with `default=5`	wrote `5`	writes `NULL`
`create(active=None)`, `active` NOT NULL with a default	wrote the default	database error (NOT NULL)
`DateTimeField(auto_now_add=True)` (NOT NULL)	`create()` failed: explicit `NULL`	`now()` on INSERT
`DateTimeField(auto_now_add=True, null=True)`	stored `NULL`	`now()` on INSERT

auto_now_add fills the column with the database's now() when the value is None at INSERT. An explicit value is written as given. The new DateTimeField(auto_now=True) writes now() on every INSERT and every save() UPDATE, overwriting the value. Both get DEFAULT now() in the DDL, and the instance gets the stored timestamp string from RETURNING.

Migrate: code that passed None to mean "use the default" should leave the argument out.

3. `create()` always inserts; `save()` inserts a missing row¶

Model.objects.create(...) is always an INSERT, including with an explicit primary key. In 1.3.x create(id=99, ...) ran an UPDATE, matched nothing, and returned an instance that was never stored. Models with a non-AutoField primary key (code = CharField(primary_key=True)) can now be created.
save() inserts when the primary key is None. Otherwise it runs UPDATE ... RETURNING, and if no row has that key it inserts the row. In 1.3.x that was a silent no-op. An AutoField (or a SERIAL / BIGSERIAL key) left as None is omitted from the INSERT, and its value comes back through RETURNING *.

An instance built with a primary key (User(id=1, name="x").save()) still updates that row if it exists. This keeps the pinned "update writes every field" behaviour.

4. `delete()` on a model without a primary key¶

Raises ValueError("<Model> has no primary key ..."), which points at Model.objects.filter(...).delete(). In 1.3.x it was a TypeError from getattr(self, None). delete() on an unsaved instance (pk None) still returns 0, and now does so without sending SQL.

5. ForeignKey¶

Assignment. book.author_id always holds the id. book.author holds what was assigned. Assigning an instance keeps the instance and copies its primary key into author_id. Assigning an id stores the id in both. Assigning author_id directly drops a cached instance whose key differs. After a load, both attributes hold the id. After a save, author keeps an assigned instance while its key still equals author_id. There is no lazy loading. The sync is a __setattr__ installed only on models that declare a ForeignKey, and not on a model whose class or mixins define their own __setattr__.
create(author=obj) and create(author=5) both store the id and set author_id. book.author = other; await book.save() now persists other. In 1.3.x the stale author_id won and the reassignment was silently lost.
filter / exclude / update accept a related instance and use its primary key, including inside __in. An unsaved instance raises ValueError. values("author") returns the id, the same as values("author_id").
DDL type. The FK column type follows the target's primary key: SERIAL → INTEGER, BIGSERIAL → BIGINT, SMALLSERIAL → SMALLINT, and anything else (VARCHAR(10), UUID, ...) as declared. 1.3.x always used INTEGER, so a non-integer key could not be referenced.
A ForeignKey to a model with no primary key raises ValueError when the class is defined. 1.3.x rendered REFERENCES t("None"), which failed at create_table.

6. JSONField defaults render as JSON¶

JSONField(default={"a": 1}) renders DEFAULT '{"a": 1}' (from json.dumps). 1.3.x rendered the Python repr '{''a'': 1}', which Postgres rejected. {} and [] render the same as before. A default that json.dumps cannot encode keeps the old rendering. The registry still reports the default kind as "other".

7. Equality and hashing¶

Two instances are equal when they are the same concrete class and have the same non-None primary key. hash() is hash((class, pk)). Instances without a primary key value (unsaved, or a model with no key) compare and hash by identity. Saving an unsaved instance changes its hash, so do not keep unsaved instances in sets or dict keys across a save().

8. QuerySet¶

	1.3.x	1.4.0
`filter(x=None)` / `exclude(x=None)`	`x = NULL`: matches nothing	`IS NULL` / `IS NOT NULL` (`x__ne=None` is `IS NOT NULL`)
`x__in="abc"` (str / bytes)	iterated per character	`TypeError`
`x__in=<generator / set / range>`	generator: `TypeError`	any iterable, materialized once when the query is built
`contains` / `startswith` / `endswith` / `i*` with `%` `_` `\`	wildcards	matched literally (`ESCAPE '\'`); new `iendswith`
`exclude(a=1, b=2)`	`NOT a AND NOT b`	`NOT (a AND b)`
`exclude(...)` with a NULL column	row dropped	row kept: `(cond) IS NOT TRUE`, correct for every lookup
second `order_by()`	appended	replaces; `order_by()` clears
`count()` / `exists()` after `limit` / `offset`	ignored the slice	counts the slice (subquery); `exists()` uses `EXISTS`
`update()` / `delete()` after `limit` / `offset`	ignored the slice, hit every match	`TypeError: cannot update/delete a sliced queryset`
`update()` with no fields	SQL error	returns `0`, no query
int value outside the column's range (`INTEGER`, `BIGINT`, `SMALLINT`, FK by target key)	wrapped (`231` → `-231`)	`ValueError` naming the column and range, before any SQL (also for `__in` and `update()`)

exclude(qty__gt=5) now includes rows where qty is NULL. To drop them as before, add .filter(qty__isnull=False).

9. ORM calls inside a transaction use it¶

Inside async with db.transaction() as tx: and inside a @transactional handler, ORM calls without using= run on that transaction and commit or roll back with it. In 1.3.x they ran on another pooled connection and survived a rollback. The open transaction is held in a contextvars variable (cello.database.current_transaction()). Tasks created inside the block (asyncio.gather, create_task) inherit it. Concurrent requests and tasks outside the block do not see it. An explicit using(db) always wins, so Model.objects.using(request.database) still reaches the pool from inside a block. Queries a block's tasks run concurrently are serialized on its single connection.

Swallowed errors. When a statement in the block fails and user code catches the error, Postgres aborts the transaction, and the COMMIT on a normal exit silently becomes a rollback. 1.4.0 detects this. After a failed statement the exit sends a probe (SELECT 1). If the transaction is aborted, the exit rolls back and raises cello.TransactionAborted (a RuntimeError), chained to the swallowed error. A failure that did not abort the transaction (a parameter rejected in Python, or a ROLLBACK TO SAVEPOINT recovery) still commits. The probe costs one round trip, and only after a statement in the block has failed.

Implementation: python/cello/database.py wraps the native Transaction's __aenter__ / __aexit__ / execute / fetch / fetchrow at import. This is Python-level wrapping. BEGIN / COMMIT / ROLLBACK and every query are still native.

Speed¶

Query building only, in one process. The 1.4.0-pre module and this build are loaded side by side, with 11 alternated rounds of 200,000 calls each. Medians:

	before	after
`filter(title=…, author_id=…)`	2,841 ns	2,939 ns (+3%)
`filter(pages__isnull=True)`	2,393 ns	2,289 ns (−4%)
`filter(title__icontains=…)` (LIKE escaping)	2,442 ns	2,693 ns (+10%)
`filter(id__in=[1, 2, 3])` (materialize, coerce, range check)	2,683 ns	3,391 ns (+26%)
`exclude(pages__gt=…)` (null-safe conjunction)	2,534 ns	3,122 ns (+23%)
`order_by("-author_id", "title")`	1,991 ns	2,015 ns (+1%)
`filter(…).order_by(…)._select_sql()`	6,933 ns	6,723 ns (−3%)
`_from_row` (5 columns)	2,388 ns	974 ns (−59%)
`to_dict()`	1,745 ns	1,336 ns (−23%)

The added per-condition work is the int range check and the related-instance check (one type() test on the common path), LIKE escaping, and building the exclude conjunction. In absolute terms that is under 1 µs on paths that are followed by a round trip of hundreds of µs. SQL assembly is faster because placeholders are numbered with str.split instead of a per-character loop. Row hydration and to_dict() are faster because the field plan is precomputed once per class, and hydration writes __dict__ directly. Absolute numbers on this box were higher than in the Phase 01 run (the machine was loaded). The before/after pairs were measured side by side.

Response serializer fast path¶

Profiling the 1.3.0 baseline (benchmarks/orm/RESULTS.md) showed write_json_value (src/json.rs) at ~47% of a 200-row list endpoint: it tried extract as bool, i64, u64 and f64 before String, so every string value raised and discarded up to four Python exceptions. It now dispatches on the concrete type (str, dict, list, bool, int, float) first and keeps the old chain as the fallback for subclasses and foreign numeric types (IntEnum, str enums, Decimal, …). python_to_json gets the same fast paths; string escaping copies unescaped runs in bulk.

Output is byte-identical to 1.3.0 (strings with escapes/control chars/non-ASCII, ints up to and beyond u64, NaN/inf, tuples, enum/int/str/float subclasses, Decimal, the non-string-key 500).
Paired wrk (both servers live, alternating, -t2 -c32, 4 × 5 s), 200 rows × 8 fields returned as a dict, against the same tree without the fix: ~316 → ~1,750 rps (≈5.5×). Measured while other builds were running on the box; the per-round ratio stayed between 4.0× and 7.1×.

Value layer — Postgres types, parameters and errors¶

src/db/value.rs (decode and bind), src/db/error.rs (new) and src/db/postgres.rs. This changes the raw app.database / request.database API as well as the ORM. Each change below fixes a 1.3.0 bug and was approved as a deliberate break. Each one was pinned with # PINNED-QUIRK: in tests/test_orm_compat_values.py, and those tests now pin the new behaviour.

Deliberate breaks — value layer¶

1. NUMERIC decodes to its exact decimal string. In 1.3.0 every numeric value, NULL included, came back as the placeholder "<unsupported column type: numeric>". Values now decode from Postgres' binary format (base-10000 digits, weight, sign, display scale) to exactly what the server prints: "12.50", "12.3400", "-0.5", "0", "NaN", and "Infinity" / "-Infinity" on PostgreSQL 14+. The scale is kept, so numeric(10,2) gives "3.14" and never 3.14. NULL decodes to None. A string was chosen over Decimal because it is the frozen shape (see the plan's value-shape table) and the cheapest one to build.

Binding: a numeric parameter accepts decimal.Decimal, int (exact, any size), float (its shortest round-trip digits, so 0.1 binds as 0.1) and str ("12.50", "1e-5", "NaN"). The display scale matches what Postgres' own text input gives (Decimal("1.50") stays 1.50). A Decimal still binds to float4 / float8 as before.

2. NULL of any type decodes to None. 1.3.0 returned the placeholder for a NULL of an unsupported type (NULL::inet, NULL::numeric, …). A non-NULL value of a type that is still unsupported (point, box, ranges, …) still decodes to the placeholder, as the plan requires.

3. Arrays decode to lists, and lists bind to array parameters. 1-D arrays of int2, int4, int8, float4, float8, bool, text, varchar, uuid, numeric, timestamptz, date and jsonb decode to Python lists, with NULL elements as None. Elements decode exactly as the same scalar column would, so a numeric[] gives strings and a jsonb[] gives dicts. Arrays of every other supported scalar type (bytea[], interval[], enum arrays, …) decode the same way.

Multi-dimensional arrays decode to nested lists (ARRAY[[1,2],[3,4]] → [[1, 2], [3, 4]]). Postgres array lower bounds are dropped ('[0:1]={7,8}' → [7, 8]).
An array of an unsupported element type (point[]) decodes to the placeholder for the whole column.
Binding: a list or tuple bound to an array parameter ($1::int4[], = ANY($1), an array column) is encoded as a Postgres array. Nested lists bind as a multi-dimensional array and must be rectangular. A ragged list raises ParameterValueError. For json[] and jsonb[], a nested list is a JSON array element. A list bound to json, jsonb or text still binds as JSON, as it did in 1.3.0.

4. "char", interval, money, inet, cidr, macaddr, timetz decode to text. All of these use binary decoders, with no extra round trip.

Type	1.3.0	1.4.0
`"char"` (the one-byte internal type, e.g. `pg_class.relkind`)	raised `RuntimeError`	one-character `str` (`""` for the empty char)
`interval`	placeholder	Postgres text form for `IntervalStyle = postgres` (the default): `"1 year 2 mons 3 days 04:05:06.5"`, `"-1 days +02:00:00"`, `"00:00:00"`
`timetz`	placeholder	`"13:14:15+02"`, `"13:14:15.25-05:30"`
`inet` / `cidr`	placeholder	`"10.0.0.1"`, `"10.1.0.0/16"`, `"10.0.0.0/8"`, `"::1"`, `"2001:db8::/32"`. The netmask is printed as Postgres prints it, and IPv6 follows RFC 5952.
`macaddr` / `macaddr8`	placeholder	`"08:00:2b:01:02:03"`
`money`	placeholder	plain decimal with two fractional digits: `"12.50"`, `"-0.05"`

Caveats:

money's Postgres text form depends on the server's lc_monetary (e.g. "$12.50"). The binary value has no currency or locale, so Cello returns the plain decimal and assumes two fractional digits.
interval is always rendered in the postgres style, whatever the session's IntervalStyle is.
The tests compare interval, timetz, inet, cidr and macaddr with the server's own output function (format('%s', v)).

Two related decodes changed along with these. Enum values (and enum arrays) decode to their label, where 1.3.0 gave the placeholder. Infinite timestamp, timestamptz and date values decode to "infinity" / "-infinity", where 1.3.0 raised RuntimeError.

5. Integer parameters are range-checked. 1.3.0 truncated an int bound to int2 / int4 (two's-complement wrap: 2**31 → -2**31). An int larger than i64 became a float whose IEEE-754 bits were read as an int8 (2**63 → 4890909195324358656). Now an out-of-range value raises cello.ParameterRangeError (a DataError that is also an OverflowError) before the query is sent:

parameter $1: value 2147483648 is out of range for Postgres type int4

int8 takes the full i64 range, and oid takes 0..=4294967295. An int of any size binds exactly to numeric. This also affects the ORM: an IntegerField given 2**31, or a filter such as qty__gt=2**31 against an INTEGER column, now raises where 1.3.0 wrapped silently. The ORM compat test test_orm_compat_queryset.py::test_large_int_filter_values pinned the wrap and needs updating.

6. bool binds to text as "true" / "false", and mismatches raise a clear error. 1.3.0 sent a bool bound to a text parameter as its binary byte: True read back as "\x01", and False failed as a NUL byte. It now binds as "true" / "false" for text, varchar, bpchar and name.

Any other Python type that cannot be bound to the parameter's Postgres type raises cello.ParameterTypeError before the query is sent. Examples are an int for a text parameter, a float for int4, and a str for int4 or jsonb. The error is a DataError, so also a RuntimeError, and it is also a TypeError. It names the parameter and its type:

parameter $2: expected a value for Postgres type int8, got Python str (cast in SQL to parse text, e.g. $1::text::int8)

In 1.3.0 these reached the server in the wrong binary format and failed as an opaque RuntimeError, or, for some types, were silently misread. A ParameterTypeError is still a RuntimeError, so except RuntimeError still catches it.

A str is still sent as raw text to every type whose binary input is text: text-like types, json, xml, bytea, "char", enums, and extension types such as citext. That is unchanged.

7. Temporal and UUID parameters accept ISO strings and Python objects. A timestamptz, timestamp, date, time or uuid parameter now accepts both forms. Before, a string reached the server in the wrong format (RuntimeError), and a datetime / date / uuid.UUID failed in json.dumps (TypeError). This makes ORM DateTimeField writes work, both create(at="2024-01-01T00:00:00Z") and create(at=datetime(...)).

Parameter	Accepts
`timestamptz`	`datetime` (aware, or naive, which is treated as UTC), `date` (midnight UTC), ISO string (`2024-01-02T03:04:05.5+05:30`, `…Z`, space separator, no offset = UTC, date only), `"infinity"` / `"-infinity"`
`timestamp`	as above. An offset is ignored and the wall-clock time kept, which is how Postgres parses text into `timestamp`.
`date`	`date`, `datetime` (the time part is ignored, as Postgres does), ISO date string, `"±infinity"`
`time`	`time` (any `tzinfo` is ignored, as Postgres does), `"HH:MM[:SS[.ffffff]]"`
`uuid`	`uuid.UUID`, or any string `uuid.UUID()` would accept (hyphenated, 32 hex digits, upper case)

Fractional seconds beyond microseconds are rounded, as Postgres rounds them. An unparsable value ("2024-02-30", "not-a-uuid") raises cello.ParameterValueError, which is a DataError and also a ValueError. A value of the wrong kind (a time for a date parameter, a datetime for text) raises ParameterTypeError. Decoding is unchanged: these types still come back as the frozen strings.

8. Database errors map to an exception hierarchy. 1.3.0 raised every driver and server error as a bare RuntimeError. Errors now map to these classes, which are exported from cello:

RuntimeError
└── DatabaseError                 .sqlstate .detail .constraint .table .column
    ├── IntegrityError            SQLSTATE class 23
    │   ├── UniqueViolation       23505
    │   ├── NotNullViolation      23502
    │   ├── ForeignKeyViolation   23503
    │   └── CheckViolation        23514
    ├── DataError                 class 22 (and parameters Cello cannot bind)
    │   ├── ParameterTypeError    also TypeError
    │   ├── ParameterRangeError   also OverflowError
    │   └── ParameterValueError   also ValueError
    ├── OperationalError          connection / pool failures; classes 08, 40, 53, 57, 58
    └── ProgrammingError          class 42; wrong number of parameters

DatabaseError subclasses RuntimeError, so 1.3.0 code that catches RuntimeError keeps working. A server error keeps its 1.3.0 message (db error: ERROR: … with DETAIL / HINT). The five attributes come from the server's error fields when it sends them, and are None otherwise. They are also None for client-side parameter errors, which never reach the server.

from cello import UniqueViolation

try:
    await db.execute("INSERT INTO users (email) VALUES ($1)", email)
except UniqueViolation as e:
    e.sqlstate     # "23505"
    e.constraint   # "users_email_key"
    e.table        # "users"
    e.detail       # "Key (email)=(a@b.c) already exists."

These are built on the event-loop thread, where the GIL is already held. The I/O thread carries the error as plain Rust data, which keeps the async bridge's no-GIL I/O thread design.

9. fetchrow / fetchval return the first of several rows. In 1.3.0 they raised RuntimeError("query returned an unexpected number of rows") when the query yielded more than one row. They now return the first row, or its first column, and the remaining rows are discarded. That includes transaction().fetchrow. Zero rows still gives None.

The rows after the first are drained by the connection and never reach Python. They are still produced and sent by the server, so add LIMIT 1 when the result is large. An error raised by the server after the first row (e.g. a division by zero in row 2) is not reported.

Unchanged, still pinned: JSON object keys come back sorted, a JSON number outside the f64 range (1e400) fails the query, and duplicate column names collapse to the last value.

Fixes¶

#	1.3.0 behaviour	1.4.0 behaviour	Raised as
1	`numeric` → placeholder (NULL too)	exact decimal string; `Decimal` / `int` / `float` / `str` bind exactly	—
2	NULL of an unsupported type → placeholder	`None`	—
3	arrays → placeholder; lists only bind as JSON	lists, nested for multi-dim; lists bind to array params	`ParameterValueError` (ragged)
4	`"char"` raised; `interval` / `money` / `inet` / `cidr` / `macaddr` / `timetz` → placeholder	text forms (`money` as plain decimal)	—
5	`int` wrapped into `int2` / `int4`; `> i64` garbled as float bits	range-checked	`ParameterRangeError`
6	`bool` → text sent as a binary byte; mismatches failed opaquely	`"true"` / `"false"`; mismatch names `$n` and the Postgres type	`ParameterTypeError`
7	ISO strings and `datetime` / `date` / `UUID` rejected	accepted (naive → UTC for `timestamptz`)	`ParameterValueError` / `ParameterTypeError`
8	every error a bare `RuntimeError`	SQLSTATE-mapped `DatabaseError` subclasses with `.sqlstate` etc.	—
9	`fetchrow` / `fetchval` raised on more than one row	first row	—
—	enum values → placeholder	label `str`	—
—	infinite `timestamp` / `timestamptz` / `date` raised	`"infinity"` / `"-infinity"`	—

The prepared-statement cache and the stale-statement retry (Phase 00) are unchanged. The first-row path of fetchrow / fetchval goes through the same retry.

Speed¶

Decoding now reads each column's raw wire bytes once and builds the Python object directly, without an intermediate Rust String or Vec for text, bytea and uuid. fetch builds each column-name string once per result set instead of once per cell. Binding checks the common types (None, bool, int, float, str, bytes) with exact-type tests before the slower isinstance checks for Decimal, datetime, UUID and list.

Paired benchmark against the Phase 01 build (bench_values.py). Each round runs both builds as separate processes against the same Postgres 16 server, alternating, for 10 rounds, and the whole benchmark was run twice. Each process first runs 100 warm-up fetches. It then times 2,000 sequential db.fetch("SELECT * FROM vl_bench") of a 200-row × 8-column table (int4, int8, text, varchar, float8, bool, timestamptz, jsonb) on a 1-connection pool. Finally it times 5,000 sequential fetchval("SELECT $1::int4 + $2::int8, $3::text", …).

Median of 10 rounds	Phase 01	value layer
2,000 × fetch 200 rows, run 1	2.45 s	2.15 s (−12%)
2,000 × fetch 200 rows, run 2	3.19 s	2.73 s (−14%)
5,000 × fetchval with 3 params, run 1	1.73 s	1.78 s (noise)
5,000 × fetchval with 3 params, run 2	2.11 s	2.10 s (noise)

Run 2 was taken while the machine was under other load, so its absolute numbers are higher, but the direction is the same in both runs. The bind path is dominated by the round trip, and no difference is measurable there.

Tests¶

tests/test_db_values_v140.py (128 tests, live Postgres). Where possible the server is the oracle:

numeric: about 30 fixed literals and 400 random literals, each decoded value compared with the server's ::text. Bind round trips of 300 random Decimals plus edge cases, each compared with the server's own parse of the same text.
arrays: every listed element type decoded and bound, with NULL elements, multi-dim, a non-1 lower bound, ragged / mixed-depth / range / type errors, = ANY($1) and array columns.
text-form types: interval, timetz, inet, cidr, macaddr and macaddr8 compared with the server's output function.
parameters and errors: integer bounds for each type, temporal and UUID coercion including offsets and rounding, the exception classes and their attributes for each integrity violation, data / programming / operational errors, and an integrity error inside a transaction.
fetchrow / fetchval: first-row semantics on the pool and in a transaction, discarded rows not leaking into the next query on a 1-connection pool, and the stale-statement retry on the first-row path.

tests/test_orm_compat_values.py was updated only for the tests that pinned items 1–9. Every other pin in it is unchanged.

Phase 02 — The row pipeline¶

No breaking change. Existing code returns the same bytes; two new shapes are faster, and the shape every doc taught is faster too.

Row → JSON: return the queryset¶

@app.get("/users")
async def users(request):
    return {"users": User.objects.filter(active=True).order_by("id")}

A QuerySet returned from a handler — on its own or anywhere inside the returned dict / list / tuple — is no longer an error. The response serializer (src/json.rs) leaves a hole where the queryset sits and keeps going. src/handler.rs then runs every such queryset on the persistent loop (concurrently when there are several) and splices the results in. The rows of each query are encoded on the cello-io thread, without the GIL, straight from the Postgres wire bytes into one pre-sized buffer (src/db/rows.rs). No Python object is created for any row or value.

The bytes are identical to serializing [u.to_dict() for u in await qs]:

keys and their order come from the compiled ModelSpec (ModelSpec::output_plan / output_columns): field names, <name>_id for a ForeignKey, the dict rule when two fields produce one key, null for a declared column the table lacks, and write_only fields (the registry's readable flag) left out;
values: decoding was restructured so there is one type dispatch (decode_cell in src/db/value.rs) feeding two writers, cell_to_py (the fetch / ORM path) and write_cell_json (this path). The JSON writer uses the response serializer's own string escaping and number formatting (write_json_string, write_json_f64), so NaN / infinity are null, floats print without an exponent, jsonb keys come out sorted, and so on, exactly as before.

await qs.json() does the same inside the handler and returns a cello.RawJSON (pre-serialized JSON the serializer writes verbatim; it cannot be built from arbitrary text). Use it when the query must run in the handler:

Transactions. A returned queryset runs after the handler returns, on the pool (or its using() database), outside any transaction the handler had open. await qs.json() runs inside it.
Errors. A returned queryset that fails becomes a 500 the handler cannot catch.

Fallbacks, same bytes at Python speed: a model that overrides to_dict(), _from_row() or __getattribute__, or shadows a field attribute with a data descriptor such as a @property, is encoded from real instances; so is a queryset bound with using() to something that is not a native Database / Transaction. A bytea column raises TypeError (the serializer never accepted bytes). A queryset inside Response.json(...) or RawJSON.encode(...) raises an error that points at await qs.json().

Rust-backed `to_dict()` and native hydration¶

Model.to_dict() is ModelSpec.to_dict(self) (src/schema/python.rs): the same getattr(obj, name, None) reads, ForeignKey fallback to an assigned instance's primary key, and dict, built through the C API without the GIL pool. write_only fields are omitted.
await qs / all() / get() / first() build instances natively (rows_to_models): cls.__new__(cls), then the _from_row assignments into __dict__, skipping the per-row dict. Instances stay real cls instances; user __new__, methods, @property and isinstance behave as before. A model that overrides _from_row keeps the Python path.
The serializer accepts model instances (return user, return {"users": await qs}) and writes their to_dict() directly.
json/jsonb values and decoded strings are built through the C API with owned references (json_to_py, py_str), which also speeds up db.fetch().

Lazy row handles: not done. The plan allowed eager hydration on the iteration path if a lazy backing store could not keep instance semantics. It cannot cheaply: vars(obj), obj.__dict__, copy, pickling and the ORM's own _load_row / ForeignKey sync read __dict__ directly. The taught pattern (to_dict()) reads every field anyway, so laziness would not help it. Native eager hydration was chosen.

Speed¶

Paired benchmarks/orm/paired.py (both servers live, wrk -t2 -c32, 10 × 5 s ABBA rounds, contaminated rounds re-run). New endpoint orm_list_qs returns the queryset; on a build without QuerySet.json it serves the to_dict() shape, so A/B compares the best shape each build offers.

Phase 02 alone. A = the same wheel with the Phase 02 Python paths patched back (Python to_dict(), all() via db.fetch + _from_row, no QuerySet.json); B = this build. The Rust value layer is identical on both sides.

endpoint	A rps	B rps	delta	paired-round range
control	34,695	34,531	−0.5%	−9.8% … +7.7%
`orm_list` (taught `to_dict()` shape)	510	647	+26.8%	+11.5% … +41.6%
`orm_list_qs` (return the queryset)	568	2,182	+284% (3.8×)	+257.5% … +320.5%
`raw_list` (unchanged code on both sides)	697	641	−8.0% (noise)	−24.3% … +13.1%
`orm_get`	5,650	5,764	+2.0% (noise)	−15.0% … +15.2%

raw_list runs identical code in A and B; its −8% with a −24% … +13% round range is this session's noise, wider than the A/A floor in benchmarks/orm/RESULTS.md (±1.5% on the mean). Only deltas well outside it are claimed: orm_list_qs and orm_list.

Against 1.3.0 (f9119e0, everything in 1.4.0 so far: statement cache, async bridge, serializer fix, value layer, Phase 02):

endpoint	1.3.0 rps	now rps	delta
control	32,409	34,242	+5.7%
`orm_list`	127	546	+330%
`orm_list_qs` (1.3.0 serves `to_dict()`)	150	2,186	+1,354% (14.6×)
`raw_list`	216	857	+298%
`orm_get`	1,160	6,227	+437%

control moved +5.7% in that run, so read the others as ±~6% on top.

Micro (same process, timeit, 200 bench rows): to_dict() × 200 — Python 299 µs, native 148 µs. Loop-thread CPU for a 200-row query: qs.json() ≈ the cost of db.execute() with no rows (≈50–120 µs, per-row work happens on the I/O thread); db.fetch() / await qs ≈ 600 µs, most of it building Python objects for the jsonb column.

Query compiler: not done, by measurement¶

_select_sql() costs 2.9 µs for order_by("id").limit(200), 4.6 µs for the get(id=…) query and 7.1 µs for a three-condition filter with ordering. At the measured service times (≈175 µs per orm_get, ≈460 µs per orm_list_qs request in one saturated process) that is 0.6–2.6% of a request. Moving SQL assembly to Rust could recover at most that share, so it was not rewritten. Identifier resolution already goes through the registry (Phase 01).

Tests¶

tests/test_orm_fastpath.py (132 tests, live Postgres):

Byte parity, raw: ~100 SQL expressions — every decoded type (ints, oid, float4/8 incl. NaN / ±inf / −0.0 / 1e20, numeric incl. NaN / ±Infinity, text with quotes / backslashes / control characters / non-BMP / U+2028, "char", bpchar, uuid, timestamp(tz) / date / time / timetz / interval incl. infinities, json / jsonb with u64 / i64-min / floats / unicode / nesting, 1-D / 2-D / empty / lower-bounded arrays of many element types with NULLs, inet / cidr / macaddr / money, enum, domain, unsupported types → placeholder, NULLs) — _fetch_json vs fetch + serializer, byte for byte.
Byte parity, models: a 20-field model (every ORM field plus numeric, uuid, date, timestamp, interval, int[] / text[], inet, db_column, two ForeignKeys) with value and all-NULL rows: qs.json(), the taught shape, and serialized instances all equal a reference built with the 1.4.0-pre Python _from_row / to_dict(). Also: missing / extra table columns, a field named like a ForeignKey's column, write_only, @property shadowing, overridden to_dict / _from_row / __new__, a non-native database, transactions.
to_dict() in every instance state (unsaved, assigned related instance, unsaved related instance, deleted attribute, extra attributes) vs the reference: values, key order, type(...) is dict.
HTTP: {"users": qs} from async, sync and blocking=True handlers, a top-level queryset, several querysets in nested containers, model instances, RawJSON, a failing deferred query (500), a queryset inside Response.json (clear error).

The compatibility suites, test_orm_fixes_v140.py, test_db_values_v140.py and test_orm_injection.py pass unchanged. Full suite: 1159 passed.

Phase 03 — Native validation, visibility and OpenAPI¶

body=Model with a cello ORM model now validates in Rust against the model's compiled ModelSpec. Pydantic, dataclass and plain-class body= are unchanged, and type-hint validation still returns 422.

Design¶

wrap_handler_with_body (python/cello/validation.py) detects an ORM model and builds one cello._cello.BodyValidator per route at decoration time. Per request it reads the Request's body bytes directly (no Python copy) and runs src/schema/validate.rs:

A serde DeserializeSeed compiled from the spec streams the JSON once. Each known key goes to a per-field seed that type-checks, coerces and checks constraints while parsing. Unknown and read-only keys are skipped with IgnoredAny. There is no intermediate JSON tree and no Python dict.
serde_json's 128-level recursion limit applies everywhere, including inside ignored keys and JSONField values. Bodies over 32 KB are parsed with the GIL released.
Only after the whole body is accepted is the instance built: cls.__new__(cls) and one attribute set per field — the same attributes Model.__init__ sets, with absent fields filled from the model's init plan (literal default, default() called, mutable default deep-copied, else None). If the model overrides __init__, cls(**validated) is called instead, so the 1.3 behaviour is kept.
Errors are written straight to a 400 {"detail": [...]} body.

Decisions¶

Question	Decision
Error item shape	Exactly `{"loc", "msg", "type"}`, the keys every 1.3 `body=` item had. `type`/`msg` use the Pydantic vocabulary (`missing`, `int_parsing`, `int_from_float`, `string_too_long`, `bool_parsing`, `datetime_parsing`, `less_than_equal`, ...). `loc` is `[field]` (or `[column]` when the FK column key was sent), and `["body"]` for whole-body errors. Input values are not echoed, so a rejected password is never reflected.
Coercion	Pydantic lax mode for JSON input (table in `docs/features/advanced/dto-validation.md`). Strings are not coerced from numbers. Integers are range-checked against the column type.
Datetimes	ISO-8601 / RFC 3339, bare dates, Unix timestamps. Naive values are UTC. Stored as the RFC 3339 string a `TIMESTAMPTZ` row decodes to (frozen value shape), not a `datetime`.
Unknown keys	Ignored, matching the 1.3 default for both Pydantic and plain classes.
`AutoField` / `read_only=True`	Ignored on input, like unknown keys. This prevents mass assignment: a body cannot set `id` and turn `save()` into an `UPDATE` of another row.
Required	Accepts input, `null=False`, no default, not `auto_now_add`. Absent optional fields get exactly what `Model(**data)` gives them (see "Defaults are applied when the instance is built").
Duplicate keys	Last wins.
U+0000	Rejected in string fields and anywhere in a `JSONField` value: `{"type": "string_null_byte", "msg": "String should not contain null characters (U+0000)"}`. Postgres refuses it in `text`/`varchar` and in `jsonb`, so it is a 400 before the handler instead of a database error after it. Custom `sql_type` columns are not checked.

Visibility¶

write_only=True (any field): accepted on input, never serialized. It clears FieldSpec.readable, so Phase 02's output plan leaves it out of the native to_dict(), of model instances returned from a handler, of Response.json(model) and of the queryset row → JSON path.
read_only=True: serialized normally, ignored on body= input. It does not affect what the ORM writes.
Both on one field raises ValueError at declaration.
In the spec, write_only clears FieldSpec.readable and read_only clears FieldSpec.writable. Both are input/output visibility only. auto still decides INSERT omission.
No serializer of its own: model instances and querysets are written by Phase 02's paths, which already honour readable. writable is not used to choose INSERT/UPDATE columns anywhere.

OpenAPI¶

/openapi.json documents body= routes. Before this, every POST/PUT/PATCH got {"type": "object"}.

ORM models get a schema generated from the spec by src/openapi.rs::model_schema(): type/format (int32, int64, double, date-time), maxLength, nullable, readOnly, writeOnly, literal defaults, and required. Property keys are the to_dict() keys, in declaration order.
Pydantic models get their own JSON schema, with $defs moved into components/schemas.
Both are added once to components/schemas and shared by every route that uses them. Two different models with the same class name get Name and Name2. These routes also document a 400 response.
Dataclasses, plain classes and routes without body= keep {"type": "object"}.

Speed¶

20-field body (5 strings, 5 ints including one numeric string, 3 floats, 3 bools including "true", 2 datetimes, a JSON object, a text field). Sync handlers. The dev box is a shared aarch64 WSL2 VM, so absolute numbers are low. On it Python's json.loads takes 6.7 µs for this 355-byte body.

HTTP, paired (main build, with Phase 02 and the ORM/value-layer changes). Four servers ran simultaneously; wrk -t2 -c32 -d5s alternated between them, 6 rounds:

Endpoint	mean rps	rounds
Pydantic `body=` (1.3 path, unchanged)	13,391	12.2k–14.2k
ORM model `body=` (native)	18,147	16.8k–19.3k
ORM model `body=` as in 1.3 (`Model(**request.json())`, no validation)	14,341	12.5k–15.2k
`request.json()` only, no model	16,517	16.0k–17.0k

Native was fastest in every round: about 36% over Pydantic body=, and ahead of a bare request.json() because no Python dict is built. (An earlier run on the pre-Phase-02 tree measured 17,627 vs 12,231.)

Validation alone. One process, 11 alternating rounds of 20,000 calls, medians, main build. Each case includes building a Request (9.0 µs here), so subtract it:

	median	minus `Request`
Pydantic `body=` (`request.json()` + `model_validate`)	27.9 µs	~18.9 µs
native `BodyValidator(request)`	16.6 µs	~7.6 µs
native `validate_bytes(raw)` (no `Request`)	6.4 µs	—
Pydantic `model_validate_json(bytes)` (not what `body=` uses)	5.5 µs	—

Of the native ~7 µs, roughly 3 µs is parse + validate in Rust and the rest is creating the Python values and the instance. Pydantic's own bytes-in path is faster at validation alone. It builds datetime objects instead of strings and is not the path body= has ever taken. The error path (17 errors) costs about 2.8 µs.

Tests¶

tests/test_native_validation.py (58 tests, live HTTP) covers:

the coercion and error tables, all-errors collection, exact item keys, and that inputs are not echoed; U+0000 rejection; absent fields filled like Model.__init__ (literal, callable and copied defaults);
invalid-JSON and non-object items, unknown and read-only keys being ignored, and duplicate keys;
100k-deep nesting (in a field and in an ignored key), the 413 limit, and the 32 KB GIL-release path;
400 malformed bodies fuzzed directly and 80 over HTTP (no 500, no panic);
write_only on to_dict(), on returned models, lists and Response.json, and the spec flags;
custom __init__, and unchanged Pydantic/dataclass body= and 422 type-hint validation;
OpenAPI components, dedup and name collisions;
a validated instance saved to live Postgres.

The validator also has Rust unit tests (src/schema/validate.rs). Like the rest of the crate, they cannot link under cargo test --lib.

Phase 05 — Migrations (`cello migrate`, experimental)¶

python -m cello migrate --experimental {make,apply,status,sql} compares the schema registry with the live Postgres catalog. make writes reviewable SQL files (migrations/NNNN_<slug>.sql). apply runs them under an advisory lock and records each one in cello_migrations(id, name, checksum, applied_at). Every command refuses to run without --experimental. See Migrations.

Design¶

Rust decides, Python orchestrates. src/schema/migrate.rs is a pure diff. It takes the registry's ModelSpecs, the live tables and the canonical column forms, and returns ordered operations, each marked destructive or not. It has no I/O, so Rust unit tests cover it. src/schema/migrate_conn.rs adds cello._cello.MigrationConnection: one dedicated tokio_postgres session on a private runtime, driven with the GIL released. That session runs the pg_catalog introspection (columns, defaults, identity and generated flags, PK, UNIQUE and FK constraints with ON DELETE). python/cello/migrate.py handles model discovery, files, checksums, the apply loop and the CLI. python/cello/__main__.py and the cello console script dispatch to it.
Postgres renders both sides. Declared types and defaults are canonicalized by creating the models as temporary tables in a transaction that is always rolled back, then reading back format_type and pg_get_expr. No hand-written alias table can drift from the server, and custom sql_type values work too.
Correctness rules. Destructive statements (DROP TABLE, DROP COLUMN, type changes without a lossless assignment cast) are written commented out unless --allow-destructive is given. apply refuses live destructive statements without the flag. Renames are never guessed: a drop plus an add on the same table prints a warning. A narrowing type change uses the assignment cast, which raises on a value that does not fit, never a truncating USING ::varchar(n). Defaults are dropped and set again around a type change. Applied files are checksummed, and a file edited after it was applied stops apply and make. make refuses to run while files are pending. Each file runs in one transaction together with its bookkeeping row.
Locking. apply holds a session-level pg_advisory_lock on the dedicated connection, so lock, migrations and unlock always use the same session. When several deploys run apply at once, one applies the files and the others wait, then find nothing to do.
Security. Every identifier goes through the registry's quote_ident, including constraint names read from the catalog. The shadow tables are created through the extended protocol, one statement per call, so a declared type cannot smuggle in a second statement.

Hooks outside the new files¶

src/schema/mod.rs: pub mod migrate; pub mod migrate_conn;
src/lib.rs: registers MigrationConnection.
pyproject.toml: [project.scripts] cello = "cello.__main__:main".

Tests¶

tests/test_migrations.py: 26 live-Postgres tests. Each test runs in its own schema, which is dropped afterwards. They cover:
empty → create, with FK ordering
adding a nullable column
adding a NOT NULL column with a default (existing rows are filled)
adding a NOT NULL column without a default (the file rolls back)
widening and narrowing type changes, and flagged type changes
NOT NULL and default changes
FK add, on_delete change and FK drop
unique add and drop, and unique over duplicate values (rolls back)
guarded destructive statements, including commented-out drops beside safe changes and the rename warning
DROP TABLE for a removed model, and a disabled table drop that stays tracked
idempotence: make after apply finds nothing
make refused while files are pending
checksum tampering refused (a CRLF-only change is accepted), a missing applied file, and transaction control in a file body
lock contention: a held lock times out, and 4 concurrent applies apply the file once
round trip: models → make → apply → introspect matches the registry
quoting of identifiers that contain ", and the reserved table name
the CLI end to end in a subprocess, including the refusal without --experimental
Rust unit tests in src/schema/migrate.rs:
type-change classification
creation order, and cycles deferred to ADD FOREIGN KEY
column changes
a flagged type change that disables its default statements
FK and unique drift
table drops ordered by their references
identifier quoting
duplicate table detection

To run them, link libpython: RUSTFLAGS="-L <libdir> -l python3.X" cargo test --lib schema::.

Limitations¶

The diff writes no SQL for:

primary-key changes and SERIAL or identity conversions (it warns)
multi-column constraints (it warns), CHECK constraints and indexes
generated columns (it warns)
statements that cannot run in a transaction
defaults or identifiers that contain a line break (make refuses)
schemas other than current_schema()

Data migrations and renames are written by hand.

Phase 04 — Advanced ORM features¶

Q objects, aggregates and grouping, relations, bulk writes, and the public raw-SQL fast path. Nothing here changes existing behaviour. The compatibility suites, test_orm_fixes_v140.py, test_orm_fastpath.py, test_orm_injection.py, the validation and migration suites all pass unchanged. User-facing documentation is in docs/data-layer.md (sections "Q objects", "Aggregates and GROUP BY", "Relations", "Bulk operations" and "Raw-SQL fast path").

Q objects¶

from cello.orm import Q. Q(**lookups) combines with |, & and ~, nests, and is accepted by filter() / exclude() / get() (also on the manager) alongside keyword lookups. Each lookup inside a Q is compiled by the same _condition as a keyword lookup, so field resolution through the registry, LIKE escaping, x=None → IS NULL, int range checks and related-instance coercion are identical. Negation is (...) IS NOT TRUE, the null-safe form exclude() has used since the ORM deliberate breaks. filter(~Q(x=5)) sends the same SQL text as exclude(x=5), and keeps rows where x is NULL. exclude(q) is filter(~q). ~~q is q: in a WHERE clause only "is it true" matters, and both forms are true for the same rows. An empty Q() adds no condition and is dropped from | / &.

Aggregates and GROUP BY¶

Count, Sum, Avg, Min, Max (all take distinct=True; Count() / Count("*") counts rows):

qs.aggregate(...) returns a dict. It aggregates the slice when limit / offset are set, using the ordered query as a subquery.
qs.values(...).annotate(...) does GROUP BY on the values() fields. order_by() accepts the annotation aliases, and count() counts groups.

Aliases are validated by the registry's identifier rule (new schema_ident, src/schema/python.rs) and quoted. Aggregate fields and group-by fields resolve through _meta.lookup. Value shapes are the value layer's, unchanged: sum(bigint) and avg(<integer>) are Postgres numeric, returned as a decimal string. sum(integer) is bigint, an int. Documented in a table in docs/data-layer.md.

API change: values() returns a queryset. It is still awaited exactly as before (await qs.values("a") gives the same list, from the same SQL). It is now a ValuesQuerySet (a QuerySet subclass) instead of a coroutine, so it can be chained (annotate, order_by, limit, ...) and returned from a handler. Field errors are raised when values() is called, not when it is awaited. A returned or json()-encoded values queryset is encoded natively by _fetch_json(None, ...), each key aliased to its name. The bytes are identical to serializing the awaited list (tested).

Relations¶

Reverse accessors. A ForeignKey installs a non-data descriptor on its target: <source>_set, or ForeignKey(..., related_name=...). It returns Source.objects.filter(fk=instance). Collisions:
an explicit name that clashes with a field, an attribute or another model's accessor raises ValueError when the class is defined;
two default names from one model are made ambiguous (AttributeError with a hint);
a default name that clashes with an attribute is skipped;
redefining a model (same module and qualname) replaces its accessors.

Only ForeignKeys declared on the class itself install accessors. - select_related("author__publisher", ...). One LEFT JOIN per ForeignKey over the base query as a subquery, so unqualified WHERE / ORDER fragments stay unambiguous. The outer ORDER BY repeats the ordering on "_c0". Model column segments are separated by NULL AS "__cello_join__". Database._fetch_related / Transaction._fetch_related (rows_to_model_graph in src/db/rows.rs) hydrate each segment with its own model's plan through the same Hydrator that rows_to_models now uses. They attach a related instance to its parent's __dict__ only when its primary key is non-NULL, so a NULL or dangling ForeignKey keeps the id. A database without _fetch_related, or a model in the graph that overrides _from_row, gets the same instances from one = ANY($1) query per relation. - prefetch_related(...). Each name is a ForeignKey or a reverse accessor, chained with __. Each path step is one WHERE key = ANY($1) query: a single array parameter, so the statement text is constant and cached. Reverse results are stored as a QuerySet holding the rows in the instance __dict__, so awaiting the accessor runs no query. Children get their ForeignKey attribute set to the parent instance. - Fast path kept. to_dict() never nests related objects, so json() / a returned queryset with select_related or prefetch_related runs the base query alone on the native path. The bytes are identical to [b.to_dict() for b in await qs] with the joins (tested, including _run_querysets). Models that fall back to instance encoding keep their relations, in case a custom to_dict() reads them. - A ForeignKey with db_column equal to its name keeps the id under the field name, so it cannot hold an instance. select_related / prefetch_related reject it with FieldError.

Bulk operations¶

Model.objects.bulk_create(objs, batch_size=None, using=None): one multi-row INSERT ... RETURNING * per batch. The write plan is the one save() uses:
an AutoField / serial left as None is DEFAULT, and the column is omitted when no row sets it;
auto_now is now();
auto_now_add is now() when unset;
ForeignKey instances are written as their primary key.

Each instance is refreshed with _load_row from its returned row, relying on Postgres returning a multi-row INSERT ... RETURNING in VALUES order. A row-count mismatch raises. - bulk_update(objs, fields, batch_size=None, using=None): one UPDATE t SET col = "_v"."cN" FROM (VALUES ($1::type, ...), ...) AS "_v" (...) WHERE t.pk = "_v"."c0" per batch, returning the number of rows updated. - Field names resolve through the registry. - auto_now fields are set to now() and read back. - Casts come from the registry's column type. Serial types map to their integer type, and length / precision modifiers are dropped, so VARCHAR(10) overflow raises instead of being truncated by an explicit cast. - Duplicate or missing primary keys raise ValueError before any SQL. - Batches are capped at 65535 // params_per_row rows. Several batches on a pool run in one transaction, so the call is all-or-nothing (tested). On a transaction, they use it.

Raw-SQL fast path: `db.fetch_json(sql, *params)`¶

The public form of _fetch_json(None, ...) on Database and Transaction. It returns a cello.RawJSON whose bytes equal serializing await db.fetch(...). A failure inside a transaction is tracked like fetch (TransactionAborted). _fetch_related and fetch_json were added to the transaction hooks in python/cello/database.py.

Speed¶

All runs are on one box against local Postgres, in alternated rounds, and report medians. The scripts were run ad hoc from the session scratchpad and are not committed.

	median	range
1,000 rows, loop of `create()`, autocommit (7 rounds)	2,082 ms	1,802–2,362
1,000 rows, loop of `create()` inside one transaction (9 rounds)	498 ms	433–719
1,000 rows, `bulk_create` (two separate runs)	44–65 ms	37–83
200 books + author, N+1 loop (21 rounds)	116 ms	55–141
200 books + author, `select_related`	1.94 ms	0.99–2.90
200 books + author, `prefetch_related`	2.21 ms	1.16–2.95

bulk_create is about 11× the transactional loop and about 32× the autocommit loop. select_related is about 60× the N+1 loop.

QuerySet._clone keeps an attribute-by-attribute path for plain QuerySet and a generic __dict__ copy only for subclasses. A plain generic copy measured +260 ns per filter(). With the specialised path, filter(title=…, author_id=…) is 2,660 ns against 2,715 ns before (11 alternated rounds, within noise).

Tests¶

tests/test_orm_advanced.py has 28 tests, live Postgres plus recording stand-ins:

Q: SQL equality of ~Q and exclude, nesting, empty Q, registry errors, and live NULL semantics.
Aggregates: alias and field injection attempts (no SQL sent), SQL shape, every value shape (including empty sets and slices), GROUP BY with ordering on aliases, count of groups, and JSON parity for values() querysets.
Relations:
validation and joined-SQL shape;
native hydration, including NULL and nested-NULL ForeignKeys and equal column names;
fallback (plain database, overridden _from_row) equals native;
the JSON fast path is byte-identical and skips the join;
transactions;
reverse accessors and every collision rule;
prefetch with query counting (cached accessors run no query), nested and empty.
Bulk:
batches and statement counts, defaults, auto_now*, mixed explicit primary keys, a non-serial primary key;
the 65,535-parameter split (7,000 rows × 10 parameters → 2 statements);
multi-batch atomicity;
bulk_update values, casts (jsonb, FK, NULL, db_column name), auto_now read back, every error, and no truncation.
fetch_json: parity with fetch, inside a transaction, and abort tracking.

Full suite: 1271 passed.

Not done¶

Lookups across relations (filter(author__name=...)), HAVING on annotations, and per-object annotate() without values(). Each needs a join planner for WHERE clauses, not only for SELECT.
select_related on reverse relations (that is prefetch_related).
Prefetch keys are bound as one array parameter. Integer and text keys are tested. A uuid key, decoded as a string, was checked by hand to bind to uuid[]. Other key types are untested.

Postgres TLS¶

Postgres connections (the enable_database() pool and the cello migrate session) now support TLS. Before this release both connected with tokio_postgres::NoTls, so every query, row and password crossed the network in plaintext, whatever the server was configured to do.

What changed¶

libpq sslmode: disable, prefer (the default, as in libpq), require, verify-ca and verify-full. allow is treated as prefer. The option is accepted in the URL query string, in libpq key=value DSNs and as DatabaseConfig(sslmode=...). The DatabaseConfig option overrides the URL.
sslrootcert: a PEM CA file; when it is given, only those CAs are trusted. When it is omitted or set to system, the bundled Mozilla roots (webpki-roots) are used. sslcert / sslkey: PEM client certificates. As in libpq, require + sslrootcert verifies the chain (verify-ca).
One TLS stack: this is a small MakeTlsConnect implementation over the existing rustls 0.22 / tokio-rustls 0.25 (src/db/tls.rs), so it adds no OpenSSL or native-tls. It also implements SCRAM channel binding (tls-server-end-point), so scram-sha-256 logins over TLS use SCRAM-SHA-256-PLUS, and channel_binding=require works.
Unix-socket hosts never use TLS, and sslmode is ignored for them, as in libpq.
Errors: bad certificate paths, non-PEM files, an unknown sslmode, and sslcert without sslkey raise ValueError from enable_database() (or from MigrationConnection(...)). Handshake failures raise OperationalError. Pool-checkout errors now include their cause chain (... invalid peer certificate: NotValidForName, ... FATAL: no pg_hba.conf entry ... no encryption). Before, they showed only the driver's generic "error performing TLS handshake" or "db error".
app.database.sslmode reports the effective mode, and the enable_database() banner prints it.

Behaviour change¶

prefer is the new default, so an app pointed at a server with ssl = on now gets an encrypted connection where it used to get a plaintext one. This includes localhost on Debian/Ubuntu, whose packaged clusters ship with the snakeoil certificate. That matches what psql does. To keep plaintext, pass sslmode="disable". In production, use sslmode="verify-full": prefer and require do not authenticate the server.

Performance¶

The handshake runs once per pooled connection, on the cold path. Measured on localhost (aarch64 WSL2, PG 16, interleaved runs):

	`disable`	`require` (TLS 1.3)
New connection + first `SELECT 1`, p50	11.1 ms	18.7 ms
`SELECT 1` on a warm connection, p50	413–596 µs	439–591 µs (≈0–6 %)
1,000 rows × ~100 B, p50	1.84–2.58 ms	1.99–2.76 ms (≈7–8 %)

Tests¶

tests/test_db_tls.py (23 tests) starts two throwaway clusters, using initdb into a temp dir on non-default ports, and removes them afterwards:

a TLS cluster (ssl=on, a hostssl-only pg_hba, cert auth for one role) with an EC CA and server/client certificates generated by openssl
a plaintext cluster (ssl=off)

The tests cover:

verify-full with the right CA, and with the wrong hostname (NotValidForName)
the wrong CA, and public roots against a private CA (UnknownIssuer)
verify-ca ignoring the hostname
require without a CA
the require + sslrootcert → verify-ca upgrade
the prefer default
disable against a hostssl-only server (rejected)
prefer falling back on a non-TLS server
require against a non-TLS server (fails)
a config option overriding the DSN
channel_binding=require (SCRAM-SHA-256-PLUS)
client-certificate auth
text, jsonb, bytea and a 200 KB value round-tripping over TLS, plus a transaction
pg_stat_ssl.ssl = true for the session
the migration session over TLS, and its hostname failure
configuration errors at enable_database() time, and Unix-socket hosts disabling TLS

The tests skip with a reason when initdb/pg_ctl (also looked up in /usr/lib/postgresql/*/bin) or openssl are missing. When the tests run as root, the clusters run as the postgres user.

Rust unit tests in src/db/tls.rs cover:

DSN splitting, for URLs and key=value strings with quoting and escapes
mode parsing and override precedence
the Unix-socket rule
bad-path errors
the channel-binding hash selection

Limitations¶

There is no implicit ~/.postgresql/{root.crt,postgresql.crt,postgresql.key} lookup.
There are no CRLs.
Encrypted private keys and sslpassword are not supported.
prefer does not retry in plaintext after a failed handshake. It still falls back when the server declines TLS.
Channel binding is offered only for RSA PKCS#1 and ECDSA certificate signatures. With RSASSA-PSS and Ed25519 certificates it falls back to plain SCRAM, so channel_binding=require fails with those.

v1.4.0 — The Native Core¶

Headline numbers¶

What is in 1.4.0¶

Phase 00 — Prepared statement cache and identifier guard¶

Speed: prepared statements are cached per connection¶

Security: identifiers in ORM queries are validated¶

Phase 01 — The schema registry¶

Design¶

Security: identifiers resolve through the registry¶

DDL is rendered in Rust¶

Speed¶

Tests¶

Deliberate breaks — ORM¶

1. Undeclared identifiers are an error¶

2. Defaults are applied when the instance is built¶

3. create() always inserts; save() inserts a missing row¶

4. delete() on a model without a primary key¶

5. ForeignKey¶

6. JSONField defaults render as JSON¶

7. Equality and hashing¶

8. QuerySet¶

9. ORM calls inside a transaction use it¶

Speed¶

Response serializer fast path¶

Value layer — Postgres types, parameters and errors¶

Deliberate breaks — value layer¶

Fixes¶

Speed¶

Tests¶

Phase 02 — The row pipeline¶

Row → JSON: return the queryset¶

Rust-backed to_dict() and native hydration¶

Speed¶

Query compiler: not done, by measurement¶

Tests¶

Phase 03 — Native validation, visibility and OpenAPI¶

Design¶

Decisions¶

Visibility¶

OpenAPI¶

Speed¶

Tests¶

Phase 05 — Migrations (cello migrate, experimental)¶

Design¶

Hooks outside the new files¶

Tests¶

Limitations¶

Phase 04 — Advanced ORM features¶

Q objects¶

Aggregates and GROUP BY¶

Relations¶

Bulk operations¶

Raw-SQL fast path: db.fetch_json(sql, *params)¶

Speed¶

Tests¶

Not done¶

Postgres TLS¶

What changed¶

Behaviour change¶

Performance¶

Tests¶

Limitations¶

3. `create()` always inserts; `save()` inserts a missing row¶

4. `delete()` on a model without a primary key¶

Rust-backed `to_dict()` and native hydration¶

Phase 05 — Migrations (`cello migrate`, experimental)¶

Raw-SQL fast path: `db.fetch_json(sql, *params)`¶