ML Engineering Tools
Learning Rate Scheduler Reference
Search learning rate scheduling strategies. Covers step decay, cosine annealing, linear warmup, OneCycleLR, and the learning rate finder method.
Calculations run locally in your browserTool
About this tool
Learning Rate Scheduler Reference
The Learning Rate Scheduler Reference covers step decay, cosine annealing, linear warmup + cosine decay, OneCycleLR, and the LR finder method with use-case guidance.
• Choose cosine vs step decay for a new training run
• Look up warmup params for transformer fine-tuning
• Reference OneCycleLR config for PyTorch
• Find LR finder implementation guidance
Next step
Batch Size Memory Calculator — Calculate GPU memory for training from model size, precision, and optimizer.
Open Batch Size Memory Calculator →
FAQ
What does this tool tell you?
The Learning Rate Scheduler Reference covers step decay, cosine annealing, linear warmup + cosine decay, OneCycleLR, and the LR finder method with use-case guidance.
What affects the result most?
Step decay: multiply by γ every N epochs — simple, discontinuous. Cosine annealing: smooth decay → cos curve — transformer standard, no hyperparameter cliff. Linear warmup + cosine decay: warm W steps then anneal — standard for all transformer fine-tuning.
How should I use the result?
Use this tool to orient quickly to the concepts, field names, or values you are about to look up in a full specification or vendor documentation. It summarizes the common cases; the authoritative source remains whichever standard or vendor doc defines the values themselves.
Related tools