← All posts← 全部文章

Encoder-decoder transformer编码器-解码器 Transformer

Published发布于
Tags标签
Transformer
Reading time阅读时间
2 min2 分钟

An encoder-decoder transformer is the original kind of transformer, built to turn one sequence into another, such as a sentence into its translation.

Figure 1: The Transformer - model architecture.
From Vaswani et al. (2017), Attention Is All You Need.

It has two stacks of blocks:

  • Encoder: Reads the whole input. Each block has self-attention, then a feed-forward layer
  • Decoder: Writes the output token by token. Each block adds cross-attention to the encoder's output between those two¹

Encoder-decoder Translation Process

The encoder reads all of I love you at once, not word by word.

The decoder starts from [start] and writes one token per round:

Round The decoder reads Next token Cross-attention looks most at
1 [start] 我 I
2 [start] 我 爱 love
3 [start] 我 爱 你 you
4 [start] 我 爱 你 [end]

The decoder appends each new token to its input. The next round reads it. This loop is Autoregressive generation.

[end] stops the loop: The decoder appends nothing more.


How the two stacks connect

The figure goes through seven steps:

  • Step 1: The encoder reads I love you. Each layer runs self-attention, then the MLP.
  • Step 2: Only the top encoder layer's output goes on. The outputs of layers 1 and 2 never reach the decoder.
  • Step 3: That one output goes to the cross-attention of every decoder layer.
  • Steps 4 to 7: The decoder predicts one token per step. Each layer runs masked self-attention, cross-attention, then the MLP. The decoder appends each token to its input.

The encoder runs once. The decoder runs once per token. The encoder's output stays the same.

All three arrows carry the same output. Each decoder layer reads it with its own weights.

The dashed line beside each sublayer is a residual connection. It adds the sublayer's input back to its output (the +), then LayerNorm runs.

A real model has 6 layers in each stack. The figure draws 3 to stay small.


¹ The decoder generates the output one symbol at a time, auto-regressively, and each decoder layer adds a sub-layer of attention over the encoder stack's output: Vaswani et al. (2017), Attention Is All You Need, §3 and §3.1

编码器-解码器 Transformer 是最初的那一种 Transformer,用来把一个序列变成另一个序列,比如把一句话变成它的译文。

图 1:Transformer 模型架构
出自 Vaswani 等人(2017),Attention Is All You Need。

它有两摞块:

  • 编码器:读完整个输入。每个块先做自注意力,再过一层前馈层
  • 解码器:一个 token 一个 token 地写出输出。每个块在这两层之间多加一层对编码器输出的交叉注意力¹

编码器-解码器的翻译过程

编码器一次读完整句 I love you,不是一个词一个词地读。

解码器从 [start] 开始,每轮写出一个 token:

轮次 解码器读入 下一个 token 交叉注意力最关注
1 [start] 我 I
2 [start] 我 爱 love
3 [start] 我 爱 你 you
4 [start] 我 爱 你 [end]

解码器把每个新 token 接到自己的输入后面,下一轮再读它。这个循环就是自回归生成。

[end] 让循环停下:解码器不再往后接任何东西。


两摞块怎么连起来

这张图分七步:

  • 第 1 步:编码器读入 I love you。每一层先做自注意力,再过 MLP。
  • 第 2 步:只有编码器最顶层的输出继续往下传。第 1、2 层的输出到不了解码器。
  • 第 3 步:这一份输出送进每一个解码器层的交叉注意力。
  • 第 4 到 7 步:解码器每步预测一个 token。每一层依次做掩码自注意力、交叉注意力,再过 MLP。解码器把每个 token 接到自己的输入后面。

编码器只运行一次。解码器每个 token 运行一次。编码器的输出始终不变。

三支箭头带的是同一份输出。每个解码器层用自己的权重去读它。

每个子层旁边的虚线是残差连接:把子层的输入加回它的输出(图里的 +),然后做 LayerNorm。

真实模型每摞有 6 层。图里只画了 3 层,好让它小一点。


¹ 解码器一次生成一个符号,是自回归的;每个解码器层都多加一个子层,对编码器那一摞的输出做注意力:Vaswani et al. (2017), Attention Is All You Need, §3 和 §3.1