fix minor bugs in tests, docs and examples related to the new 1.0.0 major release#32
Merged
fix minor bugs in tests, docs and examples related to the new 1.0.0 major release#32
Conversation
* New architecture WP 1 * New architecture WP 2 * WIP 3 * Optimized cloudsync_table_context interaction * New architecture WP 4 * Refactored begin/commit ALTER * New architecture WP 6 * New architecture (WP 5) * Minor changes * New architecture WP 6 * New architecture WP 7 * Small compilation issue fixed * New architecture WP 9 * fix: minor compilation issue * fix: database_* functions must call cloudsync_memory_* functions instead of direct sqlite3_* memory function to support the memory debugger module call cloudsync_memory_mprintf and cloudsync_memory_free instead of direct sqlite3_mprintf and sqlite3_free memory functions from database_create_insert_trigger otherwise the memory debugger would report "Pointer being freed was not previously allocated." * fix(memdebug): fix pointer returned by memdebug_zeroalloc * fix(unittest): avoid a memory leak from do_test_dbutils * test: add unittest target to Makefile Introduces a 'unittest' target to run only unit tests and updates the help output accordingly. * Update unit.c * Refactored SQL in dbutils (related to settings) * Updated static statements in cloudsync.c (WP 1) * Replaced name escape with a function (WP 2) * Completed SQL refactoring * fix: minor compilation issue * test: re-add integration test use `make unittest` to run only the unittest * fix(network): fix the network layer after database-api refactoring * Several compilation warnings fixed * Cleaned-up 64bit types * fix(lcov): exclude sql_sqlite.c from code coverage, it just contains query strings * Update cloudsync_sqlite.c * Use int64_t for version variables in network.c Replaces sqlite3_int64 with int64_t for new_db_version and new_seq variables to standardize integer type usage and improve portability. Was giving a compile error on linux musl * Update vtab.c * chore: remove warnings * Update vtab.c * test: fix compile errors on linux musl * Replaced all %lld (except one) * fix: minor compilation error * refactor: new directory structure to separate multi-platform code from database-specific implementations Refactor the codebase to separate multi-platform code from database-specific implementations, preparing for PostgreSQL extension development. vtab.c/h has been renamed to sqlite/cloudsync_changes_sqlite.c/h * fix(workflow): add CFLAGS for CURL Android builds and clean up Android build files * fix(android): add -fPIC to CFLAGS * fix(android): use OpenSSL specific version * fix(android): update OpenSSL install path to a local one, instead of a system wide path * db_version must be int64_t in network.c * fix: avoid crash on postgres when these functions are called with NULL values for db and data * improved error logs, fix some debug messages * New postgresql extension WIP 1 * Updated SQL_DATA_VERSION and added a new sql_build_select_nonpk_by_pk function to database.h * fix: remove obsolete property * chore * fix: improved SQL queries (WIP) * implement SQL_SCHEMA_VERSION with app_schema_version table and event trigger * fix: fix PG SQL queries used in cloudsync_init * fix: avoid a segfault crash in cloudsync_init Allocate in TopMemoryContext to survive SPI cleanup * test: calling twice the cloudsync_init function on postgresql is failing (WIP) * Code simplification and memory cleanup (wp) * Minor fixes * test: add debug PostgreSQL devcontainer and Docker setup Introduces a VS Code devcontainer configuration for PostgreSQL development with CloudSync, including a debug Dockerfile, docker-compose file, and Makefile changes to support debug builds. This setup enables easier debugging and development of the CloudSync extension for PostgreSQL in a containerized environment. * chore: update docker/README.md with VS Code Dev Container Debugging instructions * test: add .vscode/launch.json with the "Attach to Postgres (gdb)" configuration * Added more explicit string_dup functions * Fixed network * Error returned by sqlite3_extension_init function must be dynamically allocated * test: add debug symbols and src code of postgresql server to dev container * fix: add SPI_connect and SPI_finish to _PG_init function * dbutils.c removed db_t * Minor changes * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) Removed cloudsync_private.h and db_t * Refactoring (pg wp) * Various PostgreSQL fixes * Refactoring PG code * chore: minor fixes for when debug macros are enabled * fix: use global memory when the DBFLAG_PERSISTENT flag is set to avoid crash for double free * fix(cloudsync): guard against errmsg aliasing in error formatting It was fixed by snapshotting db_error into a local buffer when it aliases data->errmsg, then formatting from the copy. This avoids undefined behavior. * fix(postgresql): fix error message in cloudsync_init_internal, the error message must be copied before database_rollback_savepoint reset the error message * fix(postgresql): skip SPI_cursor_open for non-cursorable plans - avoid trying to open a cursor on INSERT/UPDATE/DELETE plans without RETURNING - use SPI_is_cursor_plan to decide whether to use a portal or execute once - fixes “cannot open INSERT query as cursor” during cloudsync_init paths (for example with SQL_INSERT_SITE_ID_ROWID) * Minor fixes * Improved SPI’s memory ownership rule and prevents accumulation across large result sets * dbmem apis now use pg native memory functions * More memory related fixes * Apparently repalloc doesn't like a NULL ptr * Update database_postgresql.c * get_cloudsync_context must be allocated in a global context * Revert "get_cloudsync_context must be allocated in a global context" This reverts commit 4e614f1. * Revert "Update database_postgresql.c" This reverts commit 13367c1. * Revert "Apparently repalloc doesn't like a NULL ptr" This reverts commit ffd587a. * Revert "More memory related fixes" This reverts commit f44c161. * Revert "dbmem apis now use pg native memory functions" This reverts commit 4523e42. * Several memory related issues fixed * Fixed memory allocations in PG BLOB functions * pgvalue_vec_push can fails (it now return a bool) * Improved memory handling for dbvalue_t * fix(sql_postgresql): fix placeholder for cloudsync_memory_mprintf in SQL_PRAGMA_TABLEINFO_LIST_NONPK_NAME_CID * fix(database_postgresql): refactor error handling in PostgreSQL database functions to always call PG_END_TRY, this fix a SIGSEGV error for corrupted stack * fix(dbutils): dbutils_table_settings_get_value and dbutils_settings_get_value must return NULL in case no rows or error Caller code can check if the return value is NULL or not * fix: remove unnecessary switch to top memory context for text_to_cstring * fix: if databasevm_bind_text size argurmnt is negative, then the length of the string is the number of bytes up to the first zero terminator * refactor(pgvalue): always alloc values owned by pgvalue struct in the PG memory context to simplify the memory management * chore * chore: remove unused SQL queries from sql_postgresql.c * fix(sql_postgresql): fix SQL_PRAGMA_TABLEINFO_LIST_NONPK_NAME_CID * fix(sql_postgresql): SQL_PRAGMA_TABLEINFO_LIST_NONPK_NAME_CID * fix(postgresql): implement triggers and functions called by triggers, fix metatable's schemas, fix cloudsync_pk_encode/decode functions (use bytea instead of text for pk col values) * test(pg smoke test): add tests for cloudsync_pk_encode and for insert/delete triggers and for the content of the metatable * Minor changes * Payload encoding sanity check added * Improved dbutils_settings_get_value * fix(cloudsync_postgresql): cloudsync_init returns the site_id as bytea type * chore * fix(postgresql): lazy-init cloudsync context per call Avoid relcache/snapshot leaks during CREATE EXTENSION by moving SPI-dependent init to normal function calls. Add cloudsync_pg_ensure_initialized helper, drop SPI work from _PG_init, and wire initialization into SQL entry points so context loads on demand. * Improved dbutils_table_settings_get_value * fix: solve a compile error. cloudsync_context is opaque in this compilation unit, so data->site_id isn’t accessible, use the public accessor instead. * feat(postgresql): implement cloudsync_update * chore * fix(postgresql): fix the PG_TRY/PG_CATCH exception stack, must not return inside PG_TRY block * Various improvements to encoding/decoding functions * Improved cloudsync_pg_context_init * Better network memory management * Implemented cloudsync_changes * fix: sql_build_rekey_pk_and_reset_version_except_col has different parameters for sqlite and postgresql * fix(cloudsync_postgresql.c): fix implementation of cloudsync_update aggregate function and cloudsync_delete * ci: add new target to build and run the debug version for vscode and the standalone asan version * fix(cloudsync_postgresql): fix memory context for cloudsync_changes (read) and change the col_value type to bytea to store pk_encoded value preserve SQLite-compatible payloads by encoding `col_value` with the same pk wire format before it reaches the SRF/view layer. With the bytea value for col_value we can reuse the same existing columns from the sqlite extension to encode/decode the type of the value and the value itself, and reuse the same query `SELECT cloudsync_payload_encode(tbl, pk, col_name, col_value, ...) FROM cloudsync_changes`, otherwise we should add a new column for the type and use a cast to the specific type * Added new pk_encode_value * fix(sql_postgresql): fix SQL_BUILD_UPSERT_PK_AND_COL query used for col_merge_stmt * add a skip_decode_idx argument to pk_decode, used in postgresql * fix(cloudsync_postgresql): fix cloudsync_changes_insert_trg to use col_value as bytea with the pk encoded value * test: add read and write tests for cloudsync_changes * Renamed PG changes functions and removed unused variable * No real changes * Improved cloudsync_changes SELECT * Updated cloudsync_changes (wp) * Finished implementing cloudsync_changes * fix(pk): the original blob value for the undecoded col_value for the skipped column was missing the first type byte * fix(pk): add the skip_idx argument for the pk_encode and pk_encode_size just like I already added to pk_decode, needed by cloudsync_payload_encode on postgresql * fix(cloudsync): fix the buffer len value (blen) after decompressing a compressed payload * fix(sql_postgresql): fix the SQL_CLOUDSYNC_UPSERT_RAW_COLVERSION for postgresql * fix(sql_postgresql): fix placeholder from ? to $<n> notation for postgresql * refactor(postgresql): add pgvalue_free function to make it clear how to free pgvalue object from internal functions * chore * fix(postgresql/coydsync--1.0.sql): fix arguments for cloudsync_payload_encode aggregate function * Update cloudsync--1.0.sql * fix(postgresql): fix cloudsync_changes_insert_trigger for TOMBSTONE rows * fix(postgresql): trying to fix relcache/plancache/snapshot leaks that occurs when an exception is thrown and catched inside cloudsync_changes_insert_trigger (WIP) * test(postgresql/smoke_test): add a test for payload roundtrip to another database * test: minor changes to smoke_test.sql * Added bounds check to pk and checksum to payload * test: update include directive for integration test when run with CLOUDSYNC_LOAD_FROM_SOURCES from Xcode project * Checksum is checked only if payload version is >= 2 * Fixed compilation issue * fix(android): renamed endian.h to cloudsync_endian.h to avoid android ndk clang to have conflicts with sys/endian.h * ci: update the dockerfile configurations to use postgresql 17 instead of 16, the same version used by supabase * build(supabase): build a custom supabase/postgres:17.6.1.071 docker image to be used from the `supabase start` stack * test(postgres/smoke_test): update the test to create different databases to simulate different peers * Several issues fixed and optimizations added (reported by Claude) * skip the schema hash check for now, we cannot compare the hash between sqlite and postgres like we were doing for sqlite * feat(network)!: support the v2 endpoints exposed by the new sqlite-sync-server * fix: free SPI_tuptable (if exists) after each invocation of SPI_execute to avoid memory leaks and to optimize memory usage * fix: update the return type for the cloudsync_payload_apply function, it returns the number of applied rows * Update database_postgresql.c * fix(supabase): prevent a unhealthy status during the restart of the supabase stack The error occurs because the event trigger function inserts into app_schema_version without schema qualification, failing due to missing table in the search_path used by Supabase realtime (which connects to the "postgres" database with a different schema context). Always create app_schema_version in public and updating the function’s insert statement to reference public.app_schema_version * docs(docker/README.md): added a troubleshooting note about the app_schema_version/Realtime migration error * Several memory related issues fixed * fix(network): token size when calling cloudsync_network_set_token before cloudsync_network_init, if the token was greater that 256 chars it was truncated * fix: bind null values for col_value column in INSERT INTO cloudsync_changes with type bytea to avoid "failed to find conversion function from unknown to bytea" error * fix(postgresql): return raw column in SQL_BUILD_SELECT_COLS_BY_PK_FMT instead of the encoded value This query is used during merge conflict resolution to compare local values against incoming changes. Returning encoded bytea caused type mismatches and order-dependent winners in multi-db tests, failing 03_3db_multiple_roundtrip.sql. * test(postgresql): move the smoke test to the test/postgresql dir Also split the smoke test into different test files, all these test files are called by smoke_test.sql * bump version * test(postgresql): new multi-db tests * chore: add docs with analysis on the open issues * test(postgresql): make 02_roundript.sql test executable as a standalone test * Update ISSUE_WARNING_resource_was_not_closed.md * Added support for schema * release wip-pg-extension branch node and expo packages to npmjs with the "pg" tag * renamed workflows for OIDC publishing issues, to revert before merging to main * Added new database_internal_table_exists function to make sure to check for system tables in the public schema (PG only) * fix(workflow): node packages to use pg tagged version * fix(packages/node): wrong fat binary artifact folder * Bump version to 0.9.63 * fix(database_postgresql): fix database_select1_value to make it work with text result of type name (for example the result of SELECT current_schema();) * fix(cloudsync_postgresql): only free quoted identifiers if they're different from the input * test(postgresql): add tests for multi-schema scenario * fix(postgresql): prevent duplicate primary keys when tables exist in multiple schemas When a table name exists in multiple schemas (e.g., public.users and auth.users), SQL queries joining information_schema.table_constraints with information_schema.key_column_usage were returning duplicate primary key columns. Solution: Added "AND tc.table_schema = kcu.table_schema" to all JOIN conditions to ensure primary key information is only retrieved from the target schema specified by cloudsync_schema() or current_schema(). * fix(cloudsync): avoid a crash when setting the cloudsync_set_schema to the the same previous pointer * test(postgresql): improved tests * Added new define for schema literal * fix: skip the decode step regardless of the data type (for the col idx specified by skip_decode_idx) we still need to parse the value depending on the data type to increment the bseek value for the next loop * Several minor issues fixed * Several other issues fixed * Update network.m * fix: preserve prepared plan across databasevm_reset() in PostgreSQL backend Previously, databasevm_reset() called databasevm_clear_bindings() which destroyed the SPIPlanPtr on every reset, forcing a full SPI_prepare on each bind/step cycle. This negated the benefit of caching statements in cloudsync_table_context. Now reset() only clears parameter values while keeping the plan, types, and nparams intact for reuse. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Added new CLOUDSYNC_CHANGES_NCOLS constant * database_value_text returns NULL also in PostgreSQL implementation * fix(postgres): fix current memory context to avoid crashes on PG_CATCH, the CopyErrorData func must not be called from a error context * test(postgres): add a test similar to the sport_tracker app * Fixed allocation in value returned after SPI_finish * Updated databasevm_step0, databasevm_step and databasevm_clear_bindings. Fixed warnings in test (due to uint64_t usage in a signed BIGINT) * Fix for 11_multi_table_rounds.sql * Several minor issues fixed * fix(packages/node): broken dynamic import for platform specific package in ESM * test(postgresql): improved test for multi_table_multi_columns_rounds and move the test for repeated_table on multiple schemas to a separated test file * Fixed cloudsync_network_logout * fix(network): cleanup the network configuration during network logout * fix: use the correct schema for previously initialized tables on new connections to the database * test(postgres): build the debug image with no-optimization flag * Fixed some issues related to escaping * Quoting and memory issues fixed Several quoting issues fixed. Added pfree(elems) and pfree(nulls) after the loop to free memory allocated by deconstruct_array. Moved CStringGetTextDatum allocations before PG_TRY and pfree calls after PG_END_TRY. A need_schema_param flag determines whether 1 or 2 Datums are allocated/freed. This ensures the Datum memory is cleaned up on both the success path and the PG_CATCH error path. * Removed unused files * Delete .github/workflows/main.yml * Rename rename_to_main_before_merge_to_main_branch.yml to main.yml --------- Co-authored-by: Andrea Donetti <andinux@gmail.com> Co-authored-by: Gioele Cantoni <gioele.cantoni@gmail.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Introduces the SUPABASE_POSTGRES_TAG variable to allow explicit control over the Supabase Postgres base image tag in Docker builds. Updates the Makefile, Dockerfile, and documentation to support and document this new build argument, improving flexibility for building custom images.
…C "any" Changed PostgreSQL functions from VARIADIC anyarray to VARIADIC "any" to support composite primary keys with mixed data types (e.g., TEXT + INTEGER). PostgreSQL arrays require all elements to be the same type, causing errors like "function cloudsync_insert(unknown, text, integer) does not exist" for heterogeneous composite keys. VARIADIC "any" accepts arguments of any type. This ensures pk_encode_prikey() receives correct type information for encoding, maintaining compatibility with SQLite.
Added 15_datatype_roundtrip_unmapped.sql to test roundtrip encoding/decoding of unmapped PostgreSQL types (JSONB, UUID, INET, CIDR, RANGE) and 16_composite_pk_text_int_roundtrip.sql to test roundtrip and bidirectional sync with composite primary keys mixing TEXT and INTEGER. Updated full_test.sql to include the new composite PK test. 15_datatype_roundtrip_unmapped.sql still fails, so it is not yet included to full_test.sql
When upgrading from an older version, the schema hash algorithm may differ. Call cloudsync_update_schema_hash before updating the stored version to ensure hash compatibility across synced devices.
Add 7 new SQLite unit tests and 7 new PostgreSQL test files covering: - DWS/AWS algorithm rejection (unsupported CRDT algos return clean errors) - Corrupted payload handling (empty, garbage, truncated, bit-flipped) - Payload apply idempotency (3x apply produces identical results) - Causal-length tie-breaking determinism (3-way concurrent update convergence) - Delete/resurrect with out-of-order payload delivery - Large composite primary key (5-column mixed-type PK roundtrip) - PostgreSQL-specific type roundtrips (JSONB, TIMESTAMPTZ, NUMERIC, BYTEA) - Schema hash mismatch detection (ALTER TABLE without cloudsync workflow)
Add a postgres-build matrix job that compiles the PostgreSQL extension for 5 platform/arch combinations (linux-x86_64, linux-arm64, macos-arm64, macos-x86_64, windows-x86_64) and includes them as release assets. Add postgres-package Makefile target and Windows platform support (PG_EXTENSION_LIB, -lpostgres linking).
…dows macOS: install gettext for libintl.h and pass include path via PG_EXTRA_CFLAGS. Windows: skip POSIX defines (_POSIX_C_SOURCE, _GNU_SOURCE) so PG headers use Winsock instead of netinet/in.h.
macOS: guard Security.h include behind !CLOUDSYNC_POSTGRESQL_BUILD to avoid type conflicts (Size, uint64) with PostgreSQL headers, use getentropy instead. Windows: dropped from postgres-build matrix — MSYS2 PostgreSQL headers expect Unix includes incompatible with MinGW.
Replace _GNU_SOURCE with _DARWIN_C_SOURCE on macOS to expose preadv/pwritev declarations required by PostgreSQL headers. _GNU_SOURCE is now Linux-only.
… linking macOS linker requires explicit handling of undefined symbols. PostgreSQL extensions resolve symbols at load time, so -undefined dynamic_lookup is needed. Also consolidate UNAME_S detection to a single location.
macos-13 runners are deprecated. Cross-compile x86_64 on macos-15 ARM runner via -arch x86_64 passed through PG_EXTRA_CFLAGS. Also pass PG_EXTRA_CFLAGS to linker for arch flag propagation.
Add is_altering flag to cloudsync_table_context to prevent errors when alter functions are called multiple times on the same table. begin_alter returns early if already altering; commit_alter returns early if not.
…into sqliteai-sync-to-dev
andinux
approved these changes
Mar 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.