https://chengaoshen.com/en/tags/transformer/ Recent content in Transformer on ChengAo Shen Hugo en Mon, 23 Oct 2023 00:00:00 +0000 https://chengaoshen.com/en/posts/transformer/ Mon, 23 Oct 2023 00:00:00 +0000 https://chengaoshen.com/en/posts/transformer/ <blockquote> <p>Transformer is a really popular method in modern neural networks. We have BERT or GPT to process the natural language and ViT to deal with computer vision. In this essay, you will understand what is the transformer and why the transformer works. But be careful, limited by my knowledge, I can’t show some mathematical theories or code of transformer for you.</p> </blockquote> <h2 id="why-do-we-need-the-transformer"> Why do we need the Transformer? <a class="heading-link" href="#why-do-we-need-the-transformer"> <i class="fa-solid fa-link" aria-hidden="true" title="Link to heading"></i> <span class="sr-only">Link to heading</span> </a> </h2> <p>In the NLP( Natural Language Processing) field, the text dataset always has some obvious features that prevent us from using MLP.</p>