Skip to content

fix: exclude paused sections from instrumentation measurement#44

Open
not-matthias wants to merge 1 commit intomainfrom
cod-2033-codspeed-cpp-includes-setupteardown-in-instrumentation-mode
Open

fix: exclude paused sections from instrumentation measurement#44
not-matthias wants to merge 1 commit intomainfrom
cod-2033-codspeed-cpp-includes-setupteardown-in-instrumentation-mode

Conversation

@not-matthias
Copy link
Member

PauseTiming/ResumeTiming now stop/start Callgrind instrumentation in
CODSPEED_ANALYSIS mode, ensuring setup/teardown code is not included
in benchmark measurements.

@codspeed-hq
Copy link

codspeed-hq bot commented Mar 13, 2026

Merging this PR will degrade performance by 71.94%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 5 improved benchmarks
❌ 76 regressed benchmarks
✅ 379 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime BM_Vector_Reserve[1000] 3.2 µs 3.6 µs -12.82%
WallTime BM_Vector_Reserve[10000] 31.3 µs 36 µs -13.28%
WallTime BM_Vector_PushBack[10000] 33.7 µs 39.4 µs -14.47%
WallTime BM_Vector_Reserve[100] 350.6 ns 394.1 ns -11.04%
WallTime BM_Vector_PushBack[1000] 3.7 µs 4.2 µs -11.99%
WallTime BM_RLE_Decode[10000] 24.7 µs 22.1 µs +11.77%
Simulation BM_FibonacciIterative[50] 173.9 ns 300 ns -42.04%
Simulation FooTest[MyFixture] 90.8 ns 187.8 ns -51.63%
Simulation DoubleTest[MyTemplatedFixture, double] 90.8 ns 187.8 ns -51.63%
Simulation BM_with_multiple_args[100/200] 150.3 ns 247.2 ns -39.21%
Simulation TestB[MyTemplate2, int, double] 90.8 ns 187.8 ns -51.63%
Simulation TestA[MyTemplate1, int] 90.8 ns 187.8 ns -51.63%
Simulation BM_Capture[int_string_test] 60.6 ns 215.8 ns -71.94%
Simulation BM_Capture_int[int_test] 61.1 ns 158.1 ns -61.34%
Simulation BM_String_Concatenation[10] 526.9 ns 594.7 ns -11.4%
Simulation BM_Capture_int_string[int_string_test] 60.8 ns 216.1 ns -71.85%
Simulation BM_Capture[int_test] 61.1 ns 158.1 ns -61.34%
Simulation BarTest[MyFixture] 90.8 ns 187.8 ns -51.63%
Simulation BM_StringCopy 398.6 ns 466.4 ns -14.53%
Simulation BM_custom_args[1000] 90.8 ns 187.8 ns -51.63%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing cod-2033-codspeed-cpp-includes-setupteardown-in-instrumentation-mode (696eaa1) with main (d6b4111)

Open in CodSpeed

@avalanche-staging
Copy link

avalanche-staging bot commented Mar 16, 2026

Congrats! CodSpeed is installed 🎉

🆕 468 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Open in CodSpeed

@not-matthias not-matthias force-pushed the cod-2033-codspeed-cpp-includes-setupteardown-in-instrumentation-mode branch from 5b52b9e to 646ed4c Compare March 16, 2026 18:17
PauseTiming/ResumeTiming now stop/start Callgrind instrumentation in
CODSPEED_ANALYSIS mode, ensuring setup/teardown code is not included
in benchmark measurements.
@not-matthias not-matthias force-pushed the cod-2033-codspeed-cpp-includes-setupteardown-in-instrumentation-mode branch from 646ed4c to 696eaa1 Compare March 18, 2026 09:52
@not-matthias not-matthias marked this pull request as ready for review March 18, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants