The language of proteins: NLP, machine learning & protein sequences

Dan Ofer, Nadav Brandes*, Michal Linial

*Corresponding author for this work

Research output: Contribution to journalReview articlepeer-review

118 Scopus citations

Abstract

Natural language processing (NLP) is a field of computer science concerned with automated text and language analysis. In recent years, following a series of breakthroughs in deep and machine learning, NLP methods have shown overwhelming progress. Here, we review the success, promise and pitfalls of applying NLP algorithms to the study of proteins. Proteins, which can be represented as strings of amino-acid letters, are a natural fit to many NLP methods. We explore the conceptual similarities and differences between proteins and language, and review a range of protein-related tasks amenable to machine learning. We present methods for encoding the information of proteins as text and analyzing it with NLP methods, reviewing classic concepts such as bag-of-words, k-mers/n-grams and text search, as well as modern techniques such as word embedding, contextualized embedding, deep learning and neural language models. In particular, we focus on recent innovations such as masked language modeling, self-supervised learning and attention-based models. Finally, we discuss trends and challenges in the intersection of NLP and protein research.

Original languageAmerican English
Pages (from-to)1750-1758
Number of pages9
JournalComputational and Structural Biotechnology Journal
Volume19
DOIs
StatePublished - Jan 2021

Bibliographical note

Publisher Copyright:
© 2021 The Author(s)

Keywords

  • Artificial neural networks
  • BERT
  • Bag of words
  • Bioinformatics
  • Contextualized embedding
  • Deep learning
  • Language models
  • Natural language processing
  • Tokenization
  • Transformer
  • Word embedding
  • Word2vec

Fingerprint

Dive into the research topics of 'The language of proteins: NLP, machine learning & protein sequences'. Together they form a unique fingerprint.

Cite this