2. Transformer model
2.1 Attention, a fundamental mechanism
With these first concepts of probabilistic modeling established, in this section we introduce the Transformer model, which relies on a more general mechanism for encoding the context of each decision.
SCROLL TO TOP2.1.1 Context vector calculation
The central idea of the Transformer model
Exclusive to subscribers. 97% yet to be discovered!
You do not have access to this resource.
Click here to request your free trial access!
Already subscribed? Log in!
The Ultimate Scientific and Technical Reference
This article is included in
Technological innovations
This offer includes:
Knowledge Base
Updated and enriched with articles validated by our scientific committees
Services
A set of exclusive tools to complement the resources
Practical Path
Operational and didactic, to guarantee the acquisition of transversal skills
Doc & Quiz
Interactive articles with quizzes, for constructive reading
Transformer model
Bibliography
- (1) - AHARONI (R.), JOHNSON (M.), FIRAT (O.) - Massively multilingual neural machine translation. - Proceedings of the 2019 conference of the north American chapter of the association for computational linguistics : Human language technologies, volume 1 (long and short papers), Association for Computational Linguistics, p. 3874-3884 (2019)....
Exclusive to subscribers. 97% yet to be discovered!
You do not have access to this resource.
Click here to request your free trial access!
Already subscribed? Log in!
The Ultimate Scientific and Technical Reference