Fix Sync Streams NOT null semantics

[sravan27]

Summary

This fixes Sync Streams JavaScript/static evaluator divergences from SQLite around NULL in boolean expressions.

SQLite uses three-valued logic:

• NOT NULL evaluates to NULL
• NULL NOT IN (...) evaluates to NULL
• 0 OR NULL evaluates to NULL
• 1 AND NULL evaluates to NULL

In a WHERE clause, NULL should not admit the row. The current JavaScript/static evaluator paths collapse NULL to false too early in a few places, which can turn expressions such as NOT nullable_flag, state NOT IN (...), or NOT (nullable_flag OR 0) into matches that SQLite would filter out.

Changes

• Preserve NULL for unary NOT in the shared SQL support helper.
• Preserve SQLite three-valued semantics for AND / OR in the JavaScript evaluator operator implementation.
• Route row/parameter boolean filter composition through the shared operator implementation so static filter paths use the same NULL semantics.
• Add Sync Streams regression coverage for:
  • WHERE NOT nullable_flag with a null value
  • WHERE state NOT IN '[...]' with a null left operand
  • WHERE NOT (nullable_flag OR 0) with a null operand
• Add JavaScript engine coverage for AND / OR with NULL.

Verification

Passed locally:

corepack pnpm --filter @powersync/service-sync-rules exec vitest --run test/src/sync_plan/evaluator/null_not_semantics_repro.test.ts test/src/sync_plan/evaluator/not_in_type_affinity.test.ts
corepack pnpm --filter @powersync/service-sync-rules exec vitest --run test/src/sync_plan/engine/engine.test.ts -t "javascript"
corepack pnpm --filter @powersync/service-sync-rules run build:tsc

Note: the local node:sqlite engine half of engine.test.ts fails on my Node 23 runtime because stmt.all() returns named row objects / parameter binding behavior that does not match the existing test expectations. The JavaScript evaluator path and new Sync Streams regressions pass.

[changeset-bot]

🦋 Changeset detected

Latest commit: e7ccf58

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages

Name	Type
@powersync/service-sync-rules	Patch
@powersync/service-core	Patch
@powersync/lib-services-framework	Patch
@powersync/service-module-mongodb-storage	Patch
@powersync/service-module-mongodb	Patch
@powersync/service-module-mssql	Patch
@powersync/service-module-mysql	Patch
@powersync/service-module-postgres-storage	Patch
@powersync/service-module-postgres	Patch
@powersync/service-module-core	Patch
@powersync/service-image	Patch
test-client	Patch
@powersync/service-rsocket-router	Patch
@powersync/lib-service-mongodb	Patch
@powersync/lib-service-postgres	Patch
@powersync/service-schema	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[CLAassistant]

All committers have signed the CLA.

[rkistner]

The change looks good in principle, but there is a significant risk of breaking existing queries that rely on NOT NULL evaluating to 1 / TRUE instead of NULL - see the failing tests as examples.

I think we'd need to introduce a new compatibility option here to avoid changing behavior in existing queries.

[simolus3]

I think we'd need to introduce a new compatibility option here to avoid changing behavior in existing queries.

I wonder if it could make sense to add an (experimental) compatibility option to actually use the SQLite-based evaluator instead of our JavaScript re-implementation. That should hopefully fix all of these issues at once.

The main reason I haven't explored this further so far is that we might need to refactor how we manage loaded sync rules to close the underlying SQLite database once we're done with them. But since we'll only use in-memory databases as an evaluator, a finalizer would probably work. SQLite in node also technically isn't stable yet, so the option would have to be experimental until that happens (I don't think the trouble of including better-sqlite3 in the OCI image for different architectures is worth it).

[sravan27]

Pushed an update addressing the compatibility concern: the SQLite-compatible NULL boolean behavior is now behind a new strict_null_boolean_semantics compatibility option enabled for compiled streams, so legacy behavior is preserved by default.\n\nFocused validation under Node 24.15.0 now passes:\n\nvitest test/src/sync_plan/engine/engine.test.ts test/src/sync_plan/evaluator/null_not_semantics_repro.test.ts test/src/sql_operators.test.ts --run\n\nResult: 54/54 tests passed.

[simolus3]

I've verified that unstable_sqlite_expression_engine: true also covers this, so I'm not sure if we should add another compatibility option here (to be fair, this one wouldn't be unstable). I'll let @rkistner make that call.

[simolus3]

This can probably use the syncTest helper