megatron.tokenizer.bert\_tokenization.BasicTokenizer
====================================================

.. currentmodule:: megatron.tokenizer.bert_tokenization

.. autoclass:: BasicTokenizer
   :members:
   :special-members:
   :show-inheritance:
   :exclude-members: __weakref__, __init__