Language Generation Specifics
The Language Generation model provides an easy way to use a trained Transformer model for language generation. Unlike the other models in Simple Transformers, the LanguageGenerationModel does not support training of any kind. If you wish to train or fine-tune models for language generatin, please see the Language Modeling section.
Tip: This Medium article provides more information on fine-tuning language models and language generation.
Usage Steps
The process of performing Language Generation in Simple Transformers consists of initializing a model and generating sequences.
- Initialize a
LanguageGenerationModel - Generate text with
generate()
Supported Model Types
New model types are regularly added to the library. Language Modeling tasks currently supports the model types given below.
| Model | Model code for LanguageGenerationModel |
|---|---|
| CTRL | ctrl |
| GPT-2 | gpt2 |
| OpenAI GPT | openai-gpt |
| Transformer-XL | transfo-xl |
| XLM | xlm |
| XLNet | xlnet |
Tip: The model code is used to specify the model_type in a Simple Transformers model.