Documentation.

Author: ioquatix

context/getting-started.md

@@ -0,0 +1,145 @@
+# Getting Started
+
+This guide explains how to use `io-stream` to add efficient buffering to Ruby IO objects.
+
+## Overview
+
+`io-stream` provides a buffered stream wrapper for any IO-like object in Ruby. It wraps standard Ruby IO instances (files, sockets, pipes) and adds buffering for both reading and writing operations, significantly improving performance for applications that perform many small reads or writes.
+
+## Installation
+
+Add the gem to your project:
+
+~~~ bash
+$ bundle add io-stream
+~~~
+
+## Core Concepts
+
+### Buffered Streams
+
+`io-stream` provides buffering through the {IO::Stream::Buffered} class, which wraps any IO object. Buffering reduces the number of system calls by accumulating data in memory before actually reading from or writing to the underlying IO.
+
+### Read and Write Buffers
+
+The stream maintains separate buffers for reading and writing:
+
+- **Read buffer**: Accumulates data from the underlying IO, allowing multiple small reads without system calls
+- **Write buffer**: Accumulates data to write, flushing to the underlying IO only when the buffer is full or explicitly flushed
+
+## Basic Usage
+
+### Wrapping an IO Object
+
+You can wrap any IO-like object using {IO::Stream}:
+
+~~~ ruby
+require 'io/stream'
+
+# Wrap a file
+file = File.open("data.txt", "w+")
+stream = IO::Stream(file)
+
+# Wrap a socket
+require 'socket'
+socket = TCPSocket.new("example.com", 80)
+stream = IO::Stream(socket)
+~~~
+
+### Opening Files Directly
+
+You can also open files directly as buffered streams:
+
+~~~ ruby
+require 'io/stream'
+
+# Open a file for reading
+stream = IO::Stream::Buffered.open("data.txt", "r")
+data = stream.read
+stream.close
+
+# Open with a block (auto-closes)
+IO::Stream::Buffered.open("data.txt", "w") do |stream|
+	stream.write("Hello, World!")
+	stream.flush
+end
+~~~
+
+### Reading Data
+
+The {IO::Stream::Readable} module provides various methods for reading:
+
+~~~ ruby
+require 'io/stream'
+
+IO::Stream::Buffered.open("data.txt", "r") do |stream|
+	# Read entire stream
+	content = stream.read
+	
+	# Read specific number of bytes
+	chunk = stream.read(1024)
+	
+	# Read a line
+	line = stream.gets
+	
+	# Read all lines
+	lines = stream.readlines
+	
+	# Check for end of stream
+	if stream.eof?
+		puts "Reached end of file"
+	end
+end
+~~~
+
+### Writing Data
+
+The {IO::Stream::Writable} module provides methods for writing:
+
+~~~ ruby
+require 'io/stream'
+
+IO::Stream::Buffered.open("output.txt", "w") do |stream|
+	# Write data (buffered)
+	stream.write("Hello, ")
+	stream.write("World!")
+	
+	# Write with automatic newline
+	stream.puts("This is a line")
+	
+	# Flush buffer to ensure data is written
+	stream.flush
+end
+~~~
+
+## Important Behaviors
+
+### Automatic Flushing
+
+The write buffer automatically flushes when:
+
+- The buffer size reaches the minimum write size (default: 64KB).
+- You call {IO::Stream::Writable#puts} (always flushes immediately).
+- You call {IO::Stream::Writable#flush} explicitly.
+- The stream is closed.
+
+### Manual Flushing
+
+For applications that need precise control over when data is written:
+
+~~~ ruby
+stream.write("Important data")
+stream.flush  # Ensure data is written immediately
+~~~
+
+### Buffer Sizes
+
+You can customize buffer sizes when creating streams:
+
+~~~ ruby
+# Smaller buffer for interactive applications
+stream = IO::Stream::Buffered.new(io, minimum_write_size: 4096)
+
+# Larger buffer for bulk operations
+stream = IO::Stream::Buffered.new(io, minimum_write_size: 256 * 1024)
+~~~

context/high-performance-io.md

@@ -0,0 +1,225 @@
+# High Performance IO
+
+This guide explains how to achieve optimal performance when using `io-stream` by understanding and controlling flush behavior.
+
+## Overview
+
+The key to high-performance IO with `io-stream` is understanding when and how to flush your write buffer. Improper flush timing can significantly impact throughput, latency, and CPU usage. This guide helps you choose the right buffering strategy for your application.
+
+## Why Buffering Matters
+
+Every write to an underlying IO object (file, socket, pipe) involves a system call, which has overhead:
+
+- **Context switching**: Transferring control between userspace and kernel space.
+- **System call overhead**: The cost of invoking kernel functions.
+- **Network packet overhead**: For sockets, each small write may trigger a separate packet.
+
+Buffering solves this by accumulating data in memory and performing larger, less frequent writes. However, buffering introduces latency - data sits in memory until flushed.
+
+Use buffering when you need:
+- **High throughput**: Maximize data transfer rate for bulk operations.
+- **Reduced CPU usage**: Minimize system call overhead when writing many small pieces.
+- **Efficient network utilization**: Avoid sending many tiny packets.
+
+## The Flush/Throughput Tradeoff
+
+There's a fundamental tradeoff between responsiveness and throughput:
+
+```mermaid
+graph LR
+    A[Immediate Flush] -->|Low Latency| B[Responsive]
+    A -->|Many System Calls| C[Lower Throughput]
+    D[Delayed Flush] -->|Higher Latency| E[Buffered]
+    D -->|Fewer System Calls| F[Higher Throughput]
+```
+
+**Immediate flushing** (after every write):
+- ✅ Data is sent immediately - low latency.
+- ✅ Simple mental model - predictable behavior.
+- ❌ High system call overhead.
+- ❌ Lower maximum throughput.
+- ❌ More CPU usage.
+- ❌ Network inefficiency (many small packets).
+
+**Buffered flushing** (accumulate before sending):
+- ✅ Fewer system calls - higher throughput.
+- ✅ Better CPU efficiency.
+- ✅ More efficient network packet utilization.
+- ❌ Data is delayed - higher latency.
+- ❌ Requires careful flush management.
+
+## Automatic Flush Behavior
+
+`io-stream` automatically flushes in these situations:
+
+~~~ ruby
+# 1. Buffer reaches minimum_write_size (default: 64KB)
+stream.write("x" * 65536)  # Automatically flushes
+
+# 2. Using puts() always flushes
+stream.puts("This is flushed immediately")
+
+# 3. Closing the stream
+stream.close  # Flushes any remaining data
+~~~
+
+## Choosing Your Flush Strategy
+
+### Strategy 1: Let Automatic Flushing Handle It
+
+Best for: Bulk data transfer, file processing, log writing.
+
+~~~ ruby
+require 'io/stream'
+
+# Default behavior - automatic flush at 64KB
+stream = IO::Stream::Buffered.open("large_file.dat", "w")
+
+# Write lots of data
+1000.times do |i|
+	stream.write("Record #{i}\n" * 1000)
+end
+
+stream.close  # Final flush on close
+~~~
+
+**When to use:**
+- Writing large amounts of data continuously.
+- Throughput is more important than latency.
+- You don't need interactive feedback.
+
+### Strategy 2: Manual Flush at Logical Boundaries
+
+Best for: Request/response protocols, transaction processing, structured logging.
+
+~~~ ruby
+require 'io/stream'
+require 'socket'
+
+socket = TCPSocket.new("example.com", 80)
+stream = IO::Stream(socket)
+
+# Build complete HTTP request
+stream.write("GET / HTTP/1.1\r\n")
+stream.write("Host: example.com\r\n")
+stream.write("Connection: close\r\n")
+stream.write("\r\n")
+
+# Flush after complete request
+stream.flush  # Send request as one operation
+~~~
+
+**When to use:**
+- Message-based protocols (HTTP, Redis, etc.)
+- You need to send complete "units" of data
+- Each logical operation should complete atomically
+- Balance between throughput and responsiveness
+
+### Strategy 3: Immediate Flush for Interactive Applications
+
+Best for: Chat applications, streaming responses, real-time dashboards.
+
+~~~ ruby
+require 'io/stream'
+
+# Use smaller buffer for more frequent automatic flushes
+stream = IO::Stream::Buffered.new(
+	socket,
+	minimum_write_size: 512  # Smaller buffer = more responsive
+)
+
+# Or flush after every message
+stream.write(message)
+stream.flush  # Ensure immediate delivery
+~~~
+
+**When to use:**
+- Real-time user interaction required.
+- Low latency is critical.
+- Data arrives in small, discrete chunks.
+
+### Strategy 4: Time-Based Flushing
+
+Best for: Streaming data, progress updates, monitoring
+
+~~~ ruby
+require 'io/stream'
+
+stream = IO::Stream::Buffered.open("stream.log", "w")
+last_flush = Time.now
+
+loop do
+	stream.write(generate_log_entry)
+	
+	# Flush every second or when buffer is large
+	if Time.now - last_flush > 1.0
+		stream.flush
+		last_flush = Time.now
+	end
+end
+~~~
+
+**When to use:**
+- Ensuring regular progress visibility.
+- Protecting against data loss (periodic flush to disk).
+- Streaming applications with real-time monitoring.
+
+### Strategy 5: Readiness based flushing
+
+Best for: interactive protocols, terminal applications, chat servers.
+
+~~~ ruby
+require 'io/stream'
+
+stream = IO::Stream::Buffered.new(socket, minimum_write_size: 1024)
+
+loop do
+	# Blocking read from a queue of messages to send:
+	chunk = queue.pop
+	stream.write(chunk)
+	
+	if queue.empty?
+		# Flush when we are likely to block on the queue:
+		stream.flush
+	end
+end
+~~~
+
+**When to use:**
+- When you have unpredictable message arrival patterns.
+- When you want to ensure the lowest possible latency while still benefiting from buffering when messages arrive in bursts.
+
+## Buffer Size Configuration
+
+The `minimum_write_size` parameter controls when automatic flushing occurs:
+
+~~~ ruby
+# Very small buffer - more responsive, lower throughput
+stream = IO::Stream::Buffered.new(io, minimum_write_size: 1024)
+
+# Default - balanced (64KB)
+stream = IO::Stream::Buffered.new(io)
+
+# Large buffer - maximum throughput, higher latency
+stream = IO::Stream::Buffered.new(io, minimum_write_size: 512 * 1024)
+~~~
+
+### Choosing Buffer Size
+
+**Small buffers (1-8KB):**
+- Interactive protocols (terminal, chat).
+- Real-time data visualization.
+- Acceptable: Lower throughput.
+
+**Medium buffers (8-64KB):**
+- Web servers (default is good).
+- Application servers.
+- Database connections.
+- Balance of throughput and responsiveness.
+
+**Large buffers (64KB-1MB):**
+- File processing.
+- Bulk data transfer.
+- Video encoding.
+- Logging systems.
+- Only latency-insensitive applications.

context/index.yaml

@@ -0,0 +1,16 @@
+# Automatically generated context index for Utopia::Project guides.
+# Do not edit then files in this directory directly, instead edit the guides and then run `bake utopia:project:agent:context:update`.
+---
+description: Provides a generic stream wrapper for IO instances.
+metadata:
+  documentation_uri: https://socketry.github.io/io-stream/
+  source_code_uri: https://github.com/socketry/io-stream.git
+files:
+- path: getting-started.md
+  title: Getting Started
+  description: This guide explains how to use `io-stream` to add efficient buffering
+    to Ruby IO objects.
+- path: high-performance-io.md
+  title: High Performance IO
+  description: This guide explains how to achieve optimal performance when using `io-stream`
+    by understanding and controlling flush behavior.