Factor out readable function signatures to avoid duplication by swolchok · Pull Request #5857 · pybind/pybind11

swolchok · 2025-10-09T00:18:45Z

Description

This should provide a nice to have size optimization for programs that use pybind11 in general. I wrote it because it seems to reduce size costs of adding enum_-specific implementations of dunder methods, but since it seems generally useful, I thought it might be a good idea to send as a separate PR.

Concretely, for the pybind11_tests.cpython-313-darwin.so file built as part of pybind11's tests, I see the following size delta (output is sdiff of size -m):

pybind11_tests.cpython-313-darwin.so:                           pybind11_tests.cpython-313-darwin.so:
Segment __TEXT: 5668864                                       | Segment __TEXT: 5619712
        Section __text: 4660144                                         Section __text: 4660144
        Section __stubs: 4068                                           Section __stubs: 4068
        Section __stub_helper: 4008                                     Section __stub_helper: 4008
        Section __init_offsets: 288                                     Section __init_offsets: 288
        Section __const: 277048                               |         Section __const: 232528
        Section __gcc_except_tab: 543816                                Section __gcc_except_tab: 543816
        Section __cstring: 57906                                        Section __cstring: 57906
        Section __ustring: 32                                           Section __ustring: 32
        Section __unwind_info: 103928                                   Section __unwind_info: 103928
        Section __eh_frame: 280                                         Section __eh_frame: 280
        total 5651518                                         |         total 5606998
Segment __DATA_CONST: 98304                                     Segment __DATA_CONST: 98304
        Section __got: 968                                              Section __got: 968
        Section __const: 92840                                |         Section __const: 83016
        total 93808                                           |         total 83984
Segment __DATA: 16384                                           Segment __DATA: 16384
        Section __la_symbol_ptr: 2656                                   Section __la_symbol_ptr: 2656
        Section __data: 595                                             Section __data: 595
        Section __thread_vars: 120                                      Section __thread_vars: 120
        Section __thread_bss: 40 (zerofill)                             Section __thread_bss: 40 (zerofill)
        Section __bss: 2960 (zerofill)                                  Section __bss: 2960 (zerofill)
        total 6371                                                      total 6371
Segment __LINKEDIT: 114688                                      Segment __LINKEDIT: 114688
total 5898240                                                 | total 5849088

Full disclosure: Not all of the other .sos improved (though none regressed). For this to show a benefit, users' pybind11 extensions would have to define multiple pybind11 functions or methods with the same signature.

Suggested changelog entry:

Reduced size overhead by deduplicating functions' readable signatures and type information.

This seems to reduce size costs of adding enum_-specific implementations of dunder methods, but also should provide a nice to have size optimization for programs that use pybind11 in general.

…g 17+

swolchok · 2025-10-10T21:09:49Z

Hmm, I could've sworn I saw Apple Clang 15 raise -Wdeprecated-redundant-constexpr-static-def, but here it is, claiming not to know about that warning: https://github.com/pybind/pybind11/actions/runs/18418181095/job/52486700109?pr=5857#step:11:123
Perhaps the existence of the warning flag is gated on whether we're in C++17 mode or C++11.

Also, here is clang 16 calling it a plain old -Wdeprecated: https://github.com/pybind/pybind11/actions/runs/18418181095/job/52486699145?pr=5857#step:6:70

…s well.

rwgk

Looks fine to me. My suggestions are purely about naming and comments.

rwgk · 2025-10-12T20:56:11Z

include/pybind11/pybind11.h

+#define PYBIND11_READABLE_SIGNATURE_EXPR                                                          \
+    detail::const_name("(") + cast_in::arg_names + detail::const_name(") -> ") + cast_out::name
+
+// We centralize readable function signatures in a specific template


The word "centralize" here was throwing me off when I first looked at this PR. It led me to think you're making the source code more DRY, but apparently that's not what this PR does?

How about writing

// We factor out readable function signatures ...

?

// so that we don't duplicate them across different ...

Is it really "we"? It'd be the compiler? But a smart-enough compiler could do that regardless, maybe that's why some .sos are still the same size?

How about

// so that we can be sure compilers don't duplicate them across different ...

// We factor out readable function signatures ...

Yep, I think that is much more straightforward and standard wording. Thank you. Also revising the PR title.

a smart-enough compiler could do that regardless

While this is technically true (c.f. the LLVM MachineOutliner), I think automatic outlining of large portions of templated functions is generally beyond reasonable expectations for C++ compilers, and so we shouldn't blame them for duplication happening. Having to extract portions of templates to non-generic code to avoid bloat is usual practice (and pybind11 itself does that with initialize_generic in this same codepath!); the difference here is we are extracting to less-generic code (fewer template arguments) rather than non-generic code.

How about a neutral framing like

// so that they don't get duplicated across different...

rwgk · 2025-10-12T20:58:28Z

include/pybind11/pybind11.h

+};
+#undef PYBIND11_READABLE_SIGNATURE_EXPR
+
+// Prior to C++17, we don't have inline variables, so we have to provide an out-of-line definition


Wow, that's a pretty big block of pure ugliness, even by the usual C++ standards ...

Could you please add a comment to prominently suggest that this block is removed when C++11 and C++ 14 support is dropped?

rwgk · 2025-10-12T21:00:17Z

include/pybind11/pybind11.h

 #    define PYBIND11_COMPAT_STRDUP strdup
 #endif

+#define PYBIND11_READABLE_SIGNATURE_EXPR                                                          \


PYBIND11_READABLE_FUNCTION_SIGNATURE_EXPR

for consistency. — Since this is used only in a couple places, I think consistency is more valuable than trimming the length of the name.

rwgk · 2025-10-12T21:06:04Z

include/pybind11/pybind11.h

+template <typename cast_in, typename cast_out>
+class ReadableFunctionSignature {
+public:
+    using readable_signature_type = decltype(PYBIND11_READABLE_SIGNATURE_EXPR);


Within this scope shorter names would make this code more readable, e.g. simply sig_type here, and static constexpr sig_type sig(), kSig or similar below.

Good point. ReadableFunctionSignature::kReadableSignature is redundant.

rwgk · 2025-10-12T21:10:33Z

include/pybind11/pybind11.h

+
+public:
+    static constexpr readable_signature_type kReadableSignature = readable_signature();
+#if !defined(_MSC_VER)


Could you please add a comment to explain (or at least hint) why this works only with compilers other than MSVC?

rwgk

Looks good to me, thanks!

@oremanj It'd be great if you could give this a second-set-of-eyes review. (Unless you see a problem, I think it'll be a super quick review.)

oremanj · 2025-10-15T21:53:33Z

include/pybind11/pybind11.h

+};
+#undef PYBIND11_READABLE_FUNCTION_SIGNATURE_EXPR
+
+// Prior to C++17, we don't have inline variables, so we have to


C++17 also made static constexpr variables inline by default. Could you suppress the definitions in C++17 mode rather than adding the warnings? I think that might make this clearer

good idea. I will try that now.

…++17 compat code

swolchok added 5 commits October 8, 2025 17:01

Centralize readable function signatures to avoid duplication

058a02a

This seems to reduce size costs of adding enum_-specific implementations of dunder methods, but also should provide a nice to have size optimization for programs that use pybind11 in general.

gate disabling of -Wdeprecated-redundant-constexpr-static-def to clan…

3fa3d55

…g 17+

fix gating to include Apple Clang 15

cd1f09c

Make GCC happy with types

d3db87b

fix apple clang gating again. suppress -Wdeprecated for GCC

cb1027f

swolchok added 2 commits October 10, 2025 14:14

Gate warning suppressions to C++17. Suppress -Wdeprecated for clang a…

c5cbec7

…s well.

hopefully fix last straggler CI job

98bee98

rwgk reviewed Oct 12, 2025

View reviewed changes

swolchok added 2 commits October 15, 2025 11:10

Merge branch 'master' into centralize-function-signatures

b543acc

attempt to address readability review feedback from @rwgk

7af710e

swolchok changed the title ~~Centralize readable function signatures to avoid duplication~~ Factor out readable function signatures to avoid duplication Oct 15, 2025

swolchok requested a review from rwgk October 15, 2025 18:31

rwgk approved these changes Oct 15, 2025

View reviewed changes

oremanj approved these changes Oct 15, 2025

View reviewed changes

drop warning suppressions and instead just gate compilation the pre-C…

cc36bce

…++17 compat code

rwgk merged commit 1e5bc66 into pybind:master Oct 16, 2025
85 checks passed

github-actions bot added the needs changelog Possibly needs a changelog entry label Oct 16, 2025

rwgk removed the needs changelog Possibly needs a changelog entry label Jan 21, 2026

BrewTestBot mentioned this pull request Feb 17, 2026

pybind11 3.0.2 Homebrew/homebrew-core#267908

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Factor out readable function signatures to avoid duplication#5857

Factor out readable function signatures to avoid duplication#5857
rwgk merged 10 commits intopybind:masterfrom
swolchok:centralize-function-signatures

swolchok commented Oct 9, 2025

Uh oh!

swolchok commented Oct 10, 2025 •

edited

Loading

Uh oh!

rwgk left a comment

Uh oh!

rwgk Oct 12, 2025

Uh oh!

swolchok Oct 15, 2025

Uh oh!

rwgk Oct 12, 2025

Uh oh!

rwgk Oct 12, 2025

Uh oh!

rwgk Oct 12, 2025

Uh oh!

swolchok Oct 15, 2025

Uh oh!

rwgk Oct 12, 2025

Uh oh!

rwgk left a comment

Uh oh!

oremanj Oct 15, 2025

Uh oh!

swolchok Oct 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

swolchok commented Oct 9, 2025

Description

Suggested changelog entry:

Uh oh!

swolchok commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rwgk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rwgk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

swolchok commented Oct 10, 2025 •

edited

Loading