README.md

gbuf

Generic buffers, containers, and buffered I/O for Go

---

Concept

Go's standard bytes, bufio, and container packages provide essential building blocks for in-memory buffering, tokenized scanning, and data structures—but bytes and bufio are restricted to []byte, while container/{heap,list,ring} rely on untyped any values and manual type assertions.

gbuf generalizes in-memory buffering, specialized buffer topologies, container structures, and buffered I/O to operate over any data type T ([]T and T), built natively on top of gio.

Key design principles:

1. Idiomatic Standard Library Parity & Beyond: gbuf.Buffer[T], gbuf.Reader[T], gbuf.Repeat[T], and the gbufio subpackage (Reader[T], ComparableReader[T], Writer[T], ReadWriter[T], Scanner[T]) preserve the familiar API and semantics of Go 1.27's bytes and bufio packages while working seamlessly with any gio.Reader[T] and gio.Writer[T].
2. Unified Bufferer[T] Contract Across Topologies: Whether your workload requires a linear growable buffer (Buffer), a fixed-capacity circular ring (RingBuffer, RingFilter), an in-memory random-access file (SeekBuffer), a double-ended queue (DequeBuffer), or a zero-reallocation slab chain (ChunkedBuffer), they all satisfy the common Bufferer[T] interface.
3. GC-Safe Memory Lifecycle: Consumed, truncated, discarded, and reset slots across all buffer and gbufio types are explicitly zeroed via Go's built-in clear(), ensuring pointer-bearing T elements (structs containing slices, maps, pointers, or interfaces) are never pinned in backing arrays after consumption.
4. Zero-Allocation Fast Paths: AvailableBuffer(), Next(), Value(), ReadSlice(), Peek(), ReaderFrom, WriterTo, and size-capped Pool[T] enable high-throughput streaming without unnecessary heap allocations.

---

Installation

go get github.com/zalgonoise/gbuf

---

Features

• Unified Bufferer[T] Interface:
  • Composes gio.ReadWriter[T], gio.ReaderFrom[T], gio.WriterTo[T], gio.ItemScanner[T], and gio.ItemWriter[T] with Value(), Next(n), Len(), Cap(), Truncate(n), and Reset().
• Linear, Peekable & Seekable Buffers:
  • Buffer[T] — variable-sized linear buffer of T items (equivalent to bytes.Buffer) with Discard, Clone, Peek, PeekFrom, PeekRange, ReadItems, and ReadSlice.
  • PeekBuffer[T] — wrapper around *Buffer[T] with nil-safe auto-initialization for lookahead workflows.
  • SeekBuffer[T] — in-memory read-write-seek buffer implementing gio.ReadWriteSeeker[T], gio.ReaderAt[T], and gio.WriterAt[T] (in-memory file semantics).
  • Reader[T] — read-only seekable view over an existing []T slice (equivalent to bytes.Reader).
  • Repeat[T] — exponential-copy slice repeater (equivalent to bytes.Repeat).
• Circular & Concurrent Ring Buffers:
  • RingBuffer[T] — fixed-capacity circular buffer that overwrites the oldest unread items when full.
  • RingFilter[T] — RingBuffer[T] extension that invokes a filter/interceptor callback func([]T) error on every write.
  • SyncRingBuffer[T] — thread-safe circular buffer (gio.ReadWriteCloser[T]) supporting both non-blocking overwrite mode (NewSyncRingBuffer) and bounded blocking producer-consumer mode (NewBlockingRingBuffer).
• Specialized Structural Buffers:
  • DequeBuffer[T] — power-of-two circular double-ended queue supporting both FIFO (Write/Read) and O(1) LIFO/lookahead operations (WriteFront, WriteItemFront, ReadBack, ReadItemBack).
  • ChunkedBuffer[T] — linked slab-allocated buffer that grows without reallocating or copying existing items and reclaims drained slabs immediately.
  • PriorityBuffer[T] & SliceHeap[T] — heap-backed buffer (NewPriorityBuffer, NewOrderedBuffer) that orders written items by a comparator or cmp.Ordered and yields them in priority order on Read/ReadItem/WriteTo.
• Buffer Pooling & Stream Transformers:
  • Pool[T] — generic, size-capped sync.Pool for *Buffer[T] that automatically resets buffers and discards oversized capacities.
  • TransformReader[In, Out] & TransformWriter[In, Out] — streaming map+filter adapters converting between gio.Reader[In] $\rightarrow$ gio.Reader[Out] and gio.Writer[Out] $\rightarrow$ gio.Writer[In].
• Generic container Data Structures:
  • Heap[T] (Init, Push, Pop, Remove, Fix) — strongly-typed generic replacement for container/heap.
  • List[T] & Element[T] — doubly linked list replacement for container/list.
  • Ring[T] — circular doubly linked ring replacement for container/ring.
• Buffered I/O (gbufio subpackage):
  • gbufio.Reader[T] — buffered gio.Reader[T] with Peek, Discard, ReadItem, UnreadItem, and WriteTo.
  • gbufio.ComparableReader[T] — extends gbufio.Reader[T] for comparable types with delimiter-based ReadSlice, ReadLine, and ReadItems.
  • gbufio.Writer[T] — buffered gio.Writer[T] with Flush, AvailableBuffer, WriteItem, and ReadFrom.
  • gbufio.ReadWriter[T] — composite buffered reader and writer.
  • gbufio.Scanner[T] — token scanner with ScanLines, ScanItems, and custom SplitFunc[T] support.

---

API Reference

Unified Buffer Interface (Bufferer[T])

Interface	Embedded / Methods	Description
Bufferer[T any]	gio.ReadWriter[T], gio.ReaderFrom[T], gio.WriterTo[T], gio.ItemScanner[T], gio.ItemWriter[T], Value() []T, Next(n int) []T, Len() int, Cap() int, Truncate(n int), Reset()	Common contract implemented by *Buffer[T], *PeekBuffer[T], *RingBuffer[T], *RingFilter[T], *SeekBuffer[T], *DequeBuffer[T], and *ChunkedBuffer[T].

Buffers & Stream Types (package gbuf)

Type / Constructor	Key Methods	Description
Buffer[T any] NewBuffer[T](buf []T)	Read, Write, ReadItem, UnreadItem, WriteItem, ReadFrom, WriteTo, Value, AvailableBuffer, Available, Len, Cap, Next, Truncate, Discard, Reset, Clone, Grow, ReadItems, ReadSlice, Peek, PeekFrom, PeekRange	Variable-sized linear buffer of T items. Zero value is ready to use.
PeekBuffer[T any] NewPeekBuffer[T](buf []T) AsPeekBuffer[T](b *Buffer[T])	All Buffer[T] methods (Peek, PeekFrom, PeekRange) with nil-safe initialization	Extension wrapper around *Buffer[T] for non-consuming lookahead reads.
SeekBuffer[T any] NewSeekBuffer[T](buf []T)	Read, Write, Seek, ReadAt, WriteAt, ReadItem, UnreadItem, WriteItem, ReadFrom, WriteTo, Value, All, Len, Size, Cap, Next, Truncate, Reset	In-memory read-write-seek buffer with random-access offset reads and writes.
Reader[T any] NewReader[T](b []T)	Read, ReadAt, ReadItem, UnreadItem, Seek, WriteTo, Len, Size, Reset	Read-only seekable reader over an immutable []T slice.
RingBuffer[T any] NewRingBuffer[T](size int)	Read, Write, ReadItem, UnreadItem, WriteItem, ReadFrom, WriteTo, Seek, Value, Len, Cap, Next, Truncate, Discard, Reset, ReadItems	Fixed-size circular buffer that overwrites the oldest unread elements when full.
RingFilter[T any] NewRingFilter[T](size int, fn func([]T) error)	All RingBuffer[T] methods; invokes fn on Write, WriteItem, and ReadFrom	Circular buffer that runs a processing/validation callback on written items.
SyncRingBuffer[T any] NewSyncRingBuffer[T](size int) NewBlockingRingBuffer[T](size int)	Read, Write, ReadItem, WriteItem, Close, Reset, Value, Len, Cap	Thread-safe ring buffer supporting either non-blocking overwrite mode or bounded blocking mode.
DequeBuffer[T any] NewDequeBuffer[T](items []T)	Bufferer[T] plus WriteFront, WriteItemFront, ReadBack, ReadItemBack	Dynamically growing power-of-two double-ended circular queue (FIFO + LIFO).
ChunkedBuffer[T any] NewChunkedBuffer[T](chunkSize int)	Bufferer[T] plus Chunks() int, Discard(n int) int	Linked-slab buffer that grows without copying and frees drained slabs immediately.
PriorityBuffer[T any] NewPriorityBuffer[T](less, items...) NewOrderedBuffer[T cmp.Ordered](items...)	Read, Write, ReadItem, WriteItem, PeekItem, ReadFrom, WriteTo, Value, Next, Len, Cap, Reset	Min-heap buffer that yields written items in priority order.
Pool[T any] NewPool[T](initCap, maxCap int)	Get() *Buffer[T], Put(b *Buffer[T])	Size-capped sync.Pool wrapper for *Buffer[T] with automatic GC-safe reset.
TransformReader[In, Out any] NewTransformReader[In, Out](src, fn)	Read(p []Out) (int, error), ReadItem() (Out, error)	Pull-based stream adapter applying TransformFunc[In, Out] to a gio.Reader[In].
TransformWriter[In, Out any] NewTransformWriter[In, Out](dst, fn)	Write(p []In) (int, error), WriteItem(item In) error	Push-based stream adapter applying TransformFunc[In, Out] before writing to gio.Writer[Out].
Repeat[T any]	Repeat[T any](b []T, count int) []T	Returns a new slice consisting of count copies of b.

Generic Containers (package gbuf)

Container	Types & Functions	Description
Heap	Heap[T any] interface (sort.Interface, Push(T), Pop() T) SliceHeap[T any] (NewSliceHeap(less, items...)) Init[T](h), Push[T](h, x), Pop[T](h) T, Remove[T](h, i) T, Fix[T](h, i)	Generic min-heap operations and ready-to-use slice-backed heap.
List	List[T any] (NewList[T]()) & Element[T any] (Next(), Prev(), Value) Init, Len, Front, Back, PushFront, PushBack, PushFrontList, PushBackList, InsertBefore, InsertAfter, MoveToFront, MoveToBack, MoveBefore, MoveAfter, Remove	Generic doubly linked list.
Ring	Ring[T any] (NewRing[T](n int)) Next, Prev, Move, Link, Unlink, Len, Do(func(T))	Generic circular doubly linked list.

Buffered I/O (package gbufio)

Type / Constructor	Configuration Options (cfg.Option[Config[T]])	Key Methods	Description
Reader[T any] NewReader[T](rd, opts...)	WithSize[T](size int)	Read, ReadItem, UnreadItem, Peek, Discard, Buffered, Size, WriteTo, Reset	Buffered wrapper around gio.Reader[T].
ComparableReader[T comparable] NewComparableReader[T](rd, opts...)	WithSize[T](size int), WithDelim[T](delim T), WithExcluded[T](item T)	All Reader[T] methods plus ReadSlice(delim), ReadLine(), ReadItems(delim)	Buffered reader for comparable types with delimiter search and line reading.
Writer[T any] NewWriter[T](w, opts...)	WithSize[T](size int)	Write, WriteItem, Flush, Available, AvailableBuffer, Buffered, Size, ReadFrom, Reset	Buffered wrapper around gio.Writer[T].
ReadWriter[T any] NewReadWriter[T](r, w)	—	Embeds *Reader[T] and *Writer[T]	Composite buffered gio.ReadWriter[T].
Scanner[T comparable] NewScanner[T](r, opts...)	WithSize[T](size int), WithDelim[T](delim T), WithExcluded[T](item T)	Scan() bool, Value() []T, Err() error, Buffer(buf, max), Split(SplitFunc[T]) Built-in splits: ScanLines[T], ScanItems[T]	Token scanner over a gio.Reader[T].

---

Usage Examples

1. Linear Buffer[T], Lookahead PeekRange, and Pool[T]

package main

import (
	"fmt"

	"github.com/zalgonoise/gbuf/v2"
)

func main() {
	pool := gbuf.NewPool[int](64, 4096)
	buf := pool.Get()
	defer pool.Put(buf)

	_, _ = buf.Write([]int{10, 20, 30, 40, 50})

	// Non-consuming reverse peek of indices [1:4] -> [40, 30, 20]
	peeked := make([]int, 3)
	_, _ = buf.PeekRange(4, 1, peeked)
	fmt.Println("Peeked reverse:", peeked)

	// Read until delimiter predicate matches
	slice, _ := buf.ReadItems(func(v int) bool { return v == 30 })
	fmt.Println("ReadItems:", slice) // [10, 20, 30]
	fmt.Println("Remaining:", buf.Value()) // [40, 50]
}

2. Double-Ended DequeBuffer[T] & Ordered PriorityBuffer[T]

package main

import (
	"fmt"

	"github.com/zalgonoise/gbuf/v2"
)

func main() {
	// DequeBuffer: mix FIFO writes and front prepends
	dq := gbuf.NewDequeBuffer([]string{"middle", "back"})
	_ = dq.WriteItemFront("front")

	first, _ := dq.ReadItem()     // "front"
	last, _ := dq.ReadItemBack()  // "back"
	fmt.Println(first, last, dq.Value()) // front back [middle]

	// PriorityBuffer: automatic min-heap ordering on Read
	pb := gbuf.NewOrderedBuffer[int]()
	_, _ = pb.Write([]int{42, 7, 19, 3, 25})

	fmt.Println("Ordered:", pb.Next(3)) // [3, 7, 19]
}

3. Concurrent Producer-Consumer with SyncRingBuffer[T]

package main

import (
	"fmt"

	"github.com/zalgonoise/gbuf/v2"
	"github.com/zalgonoise/gio/v2"
)

func main() {
	ring := gbuf.NewBlockingRingBuffer[int](4)

	go func() {
		for i := 1; i <= 10; i++ {
			_ = ring.WriteItem(i)
		}
		_ = ring.Close()
	}()

	all, err := gio.ReadAll[int](ring)
	if err != nil {
		panic(err)
	}
	fmt.Println("Received:", all) // [1 2 3 4 5 6 7 8 9 10]
}

4. Token Scanning with gbufio.Scanner[T]

package main

import (
	"fmt"

	"github.com/zalgonoise/gbuf/v2"
	"github.com/zalgonoise/gbuf/gbufio"
)

func main() {
	// Stream of integers separated by 0 as a record delimiter
	src := gbuf.NewReader([]int{1, 2, 0, 3, 4, 5, 0, 6})

	scanner := gbufio.NewScanner(src, gbufio.WithDelim(0))
	for scanner.Scan() {
		fmt.Println("Token:", scanner.Value())
	}
	// Output:
	// Token: [1 2]
	// Token: [3 4 5]
	// Token: [6]
}

---

License

This project is licensed under the MIT License — see the LICENSE file for details.