ML Engineering Tools
Learning Rate Scheduler Reference
Search learning rate scheduling strategies. Covers step decay, cosine annealing, linear warmup, OneCycleLR, and the learning rate finder method.
No data is transmitted — everything runs locallyTool
About this tool
Learning Rate Scheduler Reference
The Learning Rate Scheduler Reference covers step decay, cosine annealing, linear warmup + cosine decay, OneCycleLR, and the LR finder method with use-case guidance.
• Choose cosine vs step decay for a new training run
• Look up warmup params for transformer fine-tuning
• Reference OneCycleLR config for PyTorch
• Find LR finder implementation guidance
Next step
Batch Size Memory Calculator — Calculate GPU memory for training from model size, precision, and optimizer.
Open Batch Size Memory Calculator →
FAQ
What does this tool tell you?
The Learning Rate Scheduler Reference covers step decay, cosine annealing, linear warmup + cosine decay, OneCycleLR, and the LR finder method with use-case guidance.
What affects the result most?
Step decay: multiply by γ every N epochs — simple, discontinuous. Cosine annealing: smooth decay → cos curve — transformer standard, no hyperparameter cliff. Linear warmup + cosine decay: warm W steps then anneal — standard for all transformer fine-tuning.
How should I use the result?
Use this tool to orient quickly to the concepts, field names, or values you are about to look up in a full specification or vendor documentation. It summarizes the common cases; the authoritative source remains whichever standard or vendor doc defines the values themselves.
Related tools