moyin/fzu-product
1
0
Fork 0
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

chore: turn cos to cdn

Browse Source
This commit is contained in:
camera-2018
2023-07-25 02:23:45 +08:00
parent e81e23ef85
commit 3716e9f77c
98 changed files with 522 additions and 522 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
4.人工智能/4.6.7.2BERT.md
View File

@@ -30,13 +30,13 @@ mlp 的重点和创新并非它的模型结构，而是它的训练方式，前
BERT 模型的输入就是上面三者的和，如图所示：
![](https://pic-hdu-cs-wiki-1307923872.cos.ap-shanghai.myqcloud.com/boxcngc1a7cWapQA9rSLXYqUvkf.png)
![](https://cdn.xyxsw.site/boxcngc1a7cWapQA9rSLXYqUvkf.png)
## 模型结构
简单来说，BERT 是 transformer<strong>编码器</strong>的叠加，<strong>也就是下图左边部分</strong>。这算一个 block。
![](https://pic-hdu-cs-wiki-1307923872.cos.ap-shanghai.myqcloud.com/boxcnPg8594YzCdnX6KZxpEYYod.png)
![](https://cdn.xyxsw.site/boxcnPg8594YzCdnX6KZxpEYYod.png)
说白了就是一个 多头自注意力=>layer-norm=> 接 feed forward(其实就是 mlp)=>layer-norm，没有什么创新点在这里。因为是一个 backbone 模型，它没有具体的分类头之类的东西。输出就是最后一层 block 的输出。