megatron.optimizer.clip_grads

Description

Gradient clipping.

Functions

clip_grad_norm_fp32(parameters, ...[, ...])	Clips gradient norm of an iterable of parameters whose gradients
count_zeros_fp32(parameters, ...)