Data Engineering
Dataflow Worker Sizing Calculator
A stream-processing job fails in two opposite ways. Too few workers and the backlog grows until latency becomes a queueing problem. Too many workers and the pipeline pays for idle capacity all day. Five inputs above describe the arrival rate and service capacity of the job closely enough to choose a starting worker count before the pipeline sees production traffic.
Calculations run locally in your browserTool
Affiliate disclosure
Uptime, incident, and on-call management. Better Stack provides status pages, incident management, and on-call scheduling for engineering teams.
View options with Better Stack →
External site · Independent provider · We may receive a commission · Not a recommendation
About this tool
Dataflow Worker Sizing Calculator
Size a stream-processing worker pool from events per second, average processing time, concurrency per worker, and headroom. Browser-only — no data sent.
• Choose a starting worker count for a stream-processing job
• See how latency tail and headroom change the pool size
• Size workers before the pipeline sees production traffic
FAQ
What does this tool tell you?
Size a stream-processing worker pool from events per second, average processing time, concurrency per worker, and headroom. Browser-only — no data sent.
What affects the result most?
Size a stream-processing worker pool from events per second, average processing time, concurrency per worker, and headroom. Browser-only — no data sent.
How should I use the result?
The calculation is deterministic — the same inputs always produce the same output — so the most useful workflow is to vary one input at a time and see which factor moves the result most. That tells you where to focus your attention before committing to a decision.
Related tools