Part 8 · 1 chapters · ~8 min

Tokio and Tower Internals

Tokio's scheduler (worker threads, local and global queues, work stealing, the LIFO slot), cooperative budgeting, the IO driver and timer wheel, Tower's Service and Layer traits, backpressure through poll_ready, and how Axum composes them.

Services all the way down

code

// Tower's core abstraction: a Service is an async function from Request to Response, with backpressure
pub trait Service<Request> {
    type Response; type Error; type Future: Future<Output = Result<Self::Response, Self::Error>>;
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>>;   // "can you take a request?"
    fn call(&mut self, req: Request) -> Self::Future;
}
// a Layer wraps a Service in another Service (timeout, rate limit, retry, concurrency limit)
let svc = ServiceBuilder::new()
    .concurrency_limit(100)              // poll_ready returns Pending past 100 in-flight requests: backpressure
    .timeout(Duration::from_secs(5))
    .service(inner);

Tokio internal	why it matters
work-stealing scheduler with a LIFO slot	a just-spawned task runs next on the same thread (cache-warm), idle workers steal
cooperative budget	tasks yield after a budget of operations so one busy task cannot starve others
IO driver	epoll, kqueue or IOCP readiness wakes tasks (OS part 5)
timer wheel	millions of timeouts cheaply (Backend Algorithms course)
`tokio-console`	see tasks, polls and wake-ups live while debugging