Skip to content

feat(puffin): add format constants, utilities, and JSON serialization#603

Open
zhaoxuan1994 wants to merge 1 commit intoapache:mainfrom
zhaoxuan1994:feat/puffin-format-utils
Open

feat(puffin): add format constants, utilities, and JSON serialization#603
zhaoxuan1994 wants to merge 1 commit intoapache:mainfrom
zhaoxuan1994:feat/puffin-format-utils

Conversation

@zhaoxuan1994
Copy link
Contributor

Second PR in the Puffin series (following #588).

puffin_format.h/.cc: format constants, footer flag operations, little-endian I/O, compress/decompress dispatch (LZ4/Zstd stubbed as NotSupported)
puffin_json_internal.h/.cc: JSON serialization for BlobMetadata and FileMetadata

Copilot AI review requested due to automatic review settings March 24, 2026 07:06
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds core Puffin file-format helpers and JSON (de)serialization support for Puffin metadata, plus unit tests and build-system wiring to compile/run the new components.

Changes:

  • Introduce puffin_format constants/utilities (footer flags, little-endian I/O, compression dispatch stubs).
  • Add puffin_json_internal JSON serialization/deserialization for BlobMetadata and FileMetadata.
  • Add puffin_test and register it in both CMake and Meson test targets.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/iceberg/puffin/puffin_format.h Declares Puffin format constants and helper APIs (flags, endian I/O, compress/decompress).
src/iceberg/puffin/puffin_format.cc Implements flags/endian helpers and compression dispatch stubs.
src/iceberg/puffin/puffin_json_internal.h Declares JSON (de)serialization APIs for Puffin metadata.
src/iceberg/puffin/puffin_json_internal.cc Implements JSON (de)serialization and string parsing for FileMetadata.
src/iceberg/test/puffin_test.cc Adds unit tests for endian/flags/compression stubs and JSON round-trips.
src/iceberg/test/CMakeLists.txt Registers puffin_test in CMake-based test build.
src/iceberg/test/meson.build Registers puffin_test in Meson-based test build.
src/iceberg/meson.build Adds new Puffin .cc sources to the main Meson library build.
src/iceberg/CMakeLists.txt Adds new Puffin .cc sources to the main CMake library build.
src/iceberg/puffin/meson.build Updates Puffin header installation list.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.


/// Default compression codec for footer payload.
static constexpr PuffinCompressionCodec kFooterCompressionCodec =
PuffinCompressionCodec::kLz4;
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PuffinFormat::kFooterCompressionCodec is set to kLz4, but Compress/Decompress currently return NotSupported for LZ4. This constant is likely to be used as the default when writing/reading footer payloads, and will cause immediate failures once that code lands. Consider defaulting to kNone until LZ4 support is implemented, or clearly documenting that the default codec is not yet supported and must not be used for writing.

Suggested change
PuffinCompressionCodec::kLz4;
PuffinCompressionCodec::kNone;

Copilot uses AI. Check for mistakes.
@zhaoxuan1994 zhaoxuan1994 force-pushed the feat/puffin-format-utils branch from a9a1d22 to 18a4dcd Compare March 24, 2026 07:29
@zhaoxuan1994 zhaoxuan1994 force-pushed the feat/puffin-format-utils branch from 18a4dcd to 39d4a91 Compare March 24, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants