Encoder-decoder architectures can handle inputs and outputs that both consist of variable-length sequences and thus are suitable for sequence-to-sequence problems such as machine translation.

Let’s instantiate an encoder–decoder model by following the Transformer architecture. Here we specify that both the Transformer encoder and the Transformer decoder have two layers using 4-head …

Originally proposed for machine translation, the Transformer architecture in Fig. 11.7.1 consists of an encoder for representing input sequences and a decoder for generating target sequences.

When we encountered machine translation in Section 10.7, we designed an encoder–decoder architecture for sequence-to-sequence learning based on two RNNs (Sutskever et al., 2014).

In this section, we will demonstrate the application of an encoder–decoder architecture, where both the encoder and decoder are implemented as RNNs, to the task of machine translation (Cho et al., 2014, …

为了处理这种类型的输入和输出， 我们可以设计一个包含两个主要组件的架构： 第一个组件是一个 编码器 （encoder）： 它接受一个长度可变的序列作为输入， 并将其转换为具有固定形状的编码状态。

In the end, the encoder-decoder architecture contains both an encoder and a decoder, with optionally extra arguments. In the forward propagation, the output of the encoder is used to produce the …

The core idea behind the Transformer model is the attention mechanism, an innovation that was originally envisioned as an enhancement for encoder–decoder RNNs applied to sequence-to …

We studied the machine translation problem in Section 9.7, where we designed an encoder-decoder architecture based on two RNNs for sequence to sequence learning.

As an instance of the encoder-decoder architecture, the overall architecture of the transformer is presented in Fig. 10.7.1. As we can see, the transformer is composed of an encoder and a decoder.