megatron.model.bert_model

Description

BERT model.

Classes

BertLMHead(mpu_vocab_size, hidden_size, ...)	Masked LM head for Bert
BertModel([num_tokentypes, add_binary_head, ...])	Bert Language model.

Functions

bert_extended_attention_mask(attention_mask)
bert_position_ids(token_ids)
post_language_model_processing(lm_output, ...)