# Knowledge-enhanced Representations

LUKE is a Transformer-based pre-trained model specifically designed for words and entities, providing deep contextual representations through entity-aware self-attention mechanisms.