I Built a Concurrent Resource Scheduler in Go Using Sharded Priority Heaps

Support on GitHub: github.com/phero20/concurrent-resource-scheduler (Give it a star if you find it useful!) View Docs: pkg.go.dev/github.com/phero20/concurrent-resource-scheduler What happens when thousands of concurrent requests compete for a small pool of reusable resources? You can put a mutex around a slice and hope for the best. Or you can design the scheduler around concurrency from the beginning. I chose the second option. I built Concurrent Resource Scheduler (CRS) , a domain-agnostic Go library for selecting, prioritizing, routing, and maintaining reusable resources under heavy concurrent load. The core idea is simple: MANY CONCURRENT REQUESTS │ ▼ ┌───────────────────┐ │ Resource Scheduler│ └─────────┬─────────┘ │ ┌──────────────┼──────────────┐ │ │ │ ▼ ▼ ▼ Priority Acquire State Heap Strategy Management │ │ │ └──────────────┼──────────────┘ │ ▼ BEST AVAILABLE RESOURCE But making that work correctly under concurrency is where things get interesting. CRS is designed for use cases such as: LLM/API gateways API key pools proxy rotation database replicas GPU workers backend pools worker resources connection pools rate-limited providers reusable compute resources The scheduler itself does not know what a resource means. It only knows: "I have resources. I need to safely maintain them, prioritize them, and return an appropriate one to a concurrent caller." Table of Contents The Problem The Naive Approach Why a Global Mutex Becomes a Problem The Core Idea Behind CRS Architecture at a Glance Sharded Priority Heaps Why Sharding Helps The O(1) Lookup Map Priority and Acquire Are Different Problems Acquire Strategies Round Robin Weighted Acquire Adaptive Acquire Affinity Routing Shared vs Exclusive Acquisition Resource Lifecycle Atomic State Transitions The Inactive Store Batch Operations Updates Without Destroying Heap Ordering Cooldowns Asynchronous Events Observability Prometheus Integration Concurrency Model Complexity Testing the Library Race Detector Validation