megatron.model.module

Description

Megatron Module

Classes

Float16Module(module, args)
MegatronModule([share_word_embeddings])	Megatron specific extensions of torch Module with support for pipelining.

Functions

conversion_helper(val, conversion)	Apply conversion to val.
float16_to_fp32(val)	Convert fp16/bf16 val to fp32
fp32_to_float16(val, float16_convertor)	Convert fp32 val to fp16/bf16
param_is_not_shared(param)