memory leak / retention

[nilbus]

Your environment

• ruby -v: ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
• rdbg -v: rdbg 1.7.1

Describe the bug
After shipping a Rails app with the debug gem bundled and required via debug/open_nonstop, memory usage grew with each request.

Expected behavior
Memory is not retained during the request cycle

---

Our goal is to have the Rails process ready for a debugger process to attach in staging environments.

Do you have an idea of what memory might be allocated and retained across requests? Can this be worked around?

[jaynetics]

i can confirm this is a problem.

i compared multiple memory dumps over time as described here and found these leaks:

Retained 49881 ARRAY objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/thread_client.rb:111
Retained 49867 HASH objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/thread_client.rb:119
Retained 49858 OBJECT objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/session.rb:1488
Retained 49853 HASH objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/thread_client.rb:113
Retained 49849 DATA objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/session.rb:1488
Retained 49840 HASH objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/thread_client.rb:114
Retained 49835 DATA objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/thread_client.rb:112
Retained 49809 ARRAY objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/session.rb:1488
Retained 33265 IMEMO objects at: /app/vendor/bundle/ruby/3.1.0/gems/mini_magick-4.11.0/lib/mini_magick/shell.rb:45
Retained 33259 ARRAY objects at: /app/vendor/ruby-3.1.2/lib/ruby/3.1.0/open3.rb:223
Retained 33251 ARRAY objects at: /app/vendor/bundle/ruby/3.1.0/gems/mini_magick-4.11.0/lib/mini_magick/shell.rb:45
Retained 33241 DATA objects at: /app/vendor/bundle/ruby/3.1.0/gems/mini_magick-4.11.0/lib/mini_magick/shell.rb:46
Retained 33231 ARRAY objects at: /app/vendor/bundle/ruby/3.1.0/gems/mini_magick-4.11.0/lib/mini_magick/shell.rb:46
Retained 33229 DATA objects at: /app/vendor/bundle/ruby/3.1.0/gems/mini_magick-4.11.0/lib/mini_magick/shell.rb:45
Retained 16650 DATA objects at: /app/vendor/ruby-3.1.2/lib/ruby/3.1.0/open3.rb:223
Retained 16641 ARRAY objects at: /app/vendor/bundle/ruby/3.1.0/gems/mini_magick-4.11.0/lib/mini_magick/tool.rb:111
Retained 16634 FILE objects at: /app/vendor/ruby-3.1.2/lib/ruby/3.1.0/open3.rb:97
Retained 16633 FILE objects at: /app/vendor/ruby-3.1.2/lib/ruby/3.1.0/open3.rb:93
Retained 16628 OBJECT objects at: /app/vendor/bundle/ruby/3.1.0/gems/mini_magick-4.11.0/lib/mini_magick/tool.rb:91

there is some symmetry of the debug object numbers with the mini_magick + open3 numbers, so maybe there is a memory leak in some gems, such as mini_magick (see also minimagick/minimagick#544) and that leak is greatly exacerbated by the debug gem? or debug itself is leaky.

either way, debug should probably include big warnings so people don't load it in production...

edit: i didn't require 'debug/open_nonstop' like OP. having gem 'debug' in the Gemfile was sufficient to cause the problem.

[ko1]

Thank you for the information.

Retained 49809 ARRAY objects at: /app/vendor/bundle/ruby/3.1.0/gems/debug-1.6.1/lib/debug/session.rb:1488
-> added at thread creation. it should be free'ed when the thread is free'ed

And maybe all of them are based on same reason.

Maybe your program making many threads. I'll think about it.

[nilbus]

Nice discovery! I'll try to confirm this fix later today or sometime this week.