RoFormer: Enhanced transformer with Rotary Position Embedding

Su, Jianlin; Ahmed, Murtadha; Lu, Yu; Pan, Shengfeng; Bo, Wen; Liu, Yunfeng

doi:10.1016/j.neucom.2023.127063

Record: found
Abstract: not found
Article: not found

RoFormer: Enhanced transformer with Rotary Position Embedding

Author(s): Jianlin Su , Murtadha Ahmed , Yu Lu , Shengfeng Pan , Wen Bo , Yunfeng Liu

Publication date Created: February 2024

Publication date (Print): February 2024

Journal: Neurocomputing

Publisher: Elsevier BV

Read this article at

ScienceOpenPublisher

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Related collections

Most cited references 43

Record: found
Abstract: not found
Book: not found

Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Volume 1 (Long and Short Papers)

J Devlin, M-W Chang, K Lee … (2019)

0 comments Cited 175 times – based on 0 reviews

Record: found
Abstract: not found
Book: not found

Attention is all you need

A. Vaswani, Vaswani, A Vaswani … (2017)

0 comments Cited 107 times – based on 0 reviews

Record: found
Abstract: not found
Book: not found

Language models are few-shot learners.

T. BROWN, B. Mann, N Ryder … (2020)

0 comments Cited 72 times – based on 0 reviews

All references

Author and article information

Contributors

Murtadha Ahmed: (View ORCID Profile)

Journal

Title: Neurocomputing

Abbreviated Title: Neurocomputing

Publisher: Elsevier BV

ISSN (Print): 09252312

Publication date Created: February 2024

Publication date (Print): February 2024

Volume: 568

Page: 127063

Article

DOI: 10.1016/j.neucom.2023.127063

SO-VID: 26d67d15-8bfb-46b2-8376-26d8ee4ae02b

License:

https://www.elsevier.com/tdm/userlicense/1.0/

https://doi.org/10.15223/policy-017

https://doi.org/10.15223/policy-037

https://doi.org/10.15223/policy-012

https://doi.org/10.15223/policy-029

https://doi.org/10.15223/policy-004

History

Data availability:

RoFormer: Enhanced transformer with Rotary Position Embedding

Read this article at

Related collections

ScienceOpen Research

Most cited references 43

Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Volume 1 (Long and Short Papers)

Attention is all you need

Language models are few-shot learners.

Author and article information

Contributors

Journal

Article

History

Comments

Comment on this article

Similar content 1,987

Cited by 38

Most referenced authors 487