megatron.model.glu_activations.SwiGLU#

class megatron.model.glu_activations.SwiGLU#

Bases: _GLUBaseModule