Add buffer pool for socket buffers to reduce memory for idle connections

[carlhoerberg]

Summary

• Introduce a buffer pool for IO::Buffered read/write buffers to save memory when handling many concurrent connections
• Most connections are idle most of the time, so releasing buffers back to a shared pool when not actively in use allows memory to be reused across connections
• Write buffers are released after flush, read buffers released on close

Test plan

• All existing tests pass (1218 examples, 0 failures)
• Manual testing with many concurrent idle connections to verify memory reduction

🤖 Generated with Claude Code

[claude]

Claude finished @carlhoerberg's task —— View job

---

PR Code Review - Buffer Pool Implementation

Todo List:

• Read and analyze changed files for potential issues
• Check for bugs, security vulnerabilities, performance issues
• Verify error handling and Crystal anti-patterns
• Report findings

Issues Found

1. Race Condition in Buffer Pool (src/stdlib/io_buffered.cr:88-108)

The fill_buffer method has a race condition where it releases the buffer before waiting for readability, then acquires a new one. Between the release and acquire, another fiber could take the released buffer, forcing allocation of a new one:

# Release existing buffer before waiting
if use_pool && self.is_a?(Socket)
  if (p = pool) && (in_buf = @in_buffer) && !in_buf.null?
    p.release(in_buf)
    @in_buffer = Pointer(UInt8).null  # ← Buffer released here
  end
  Crystal::EventLoop.current.wait_readable(self)  # ← Yields to other fibers
end

in_buf = if use_pool && (p = pool)
           @in_buffer ||= p.acquire  # ← May allocate new buffer

2. Missing Error Handling (src/stdlib/io_buffered.cr:107)

No error handling for unbuffered_read which could raise exceptions, potentially leaving the IO in an inconsistent state with a partially initialized buffer.

3. Potential Memory Leak (src/stdlib/io_buffered.cr:190-192)

In the close method, if flush raises an exception, the read buffer won't be released back to the pool:

flush if @out_count > 0  # ← Could raise exception
ensure
  unbuffered_close

The buffer release happens before the ensure block.

4. Performance Issue - Socket Type Check (src/stdlib/io_buffered.cr:93)

The self.is_a?(Socket) check happens on every fill_buffer call for socket connections. This runtime type check could be expensive in high-throughput scenarios.

5. Crystal Anti-pattern - Buffer Size Mismatch (src/stdlib/io_buffered.cr:89,137)

The condition @buffer_size == pool.buffer_size is checked multiple times without caching. If buffer sizes don't match, the pool isn't used but there's no logging/warning about this misconfiguration.

---

[carlhoerberg]

Dropping memory usage for 10.000 connections, each with one channel, from 637MiB to 413MiB, a 35% reduction.