#nlp

The original paper Attention is a mechanism that lets neural networks focus on specific parts of an input sequence.

CLIP (Contrastive Language-Image Pre-training) is a neural net model developed by OpenAI that efficiently learns visual concepts from natural language supervision.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that allows to adapt large pre-trained models to specific tasks while minimizing computational resources.

Loss functions commonly used in NLP tasks. For general losses (Cross-Entropy, MSE, KL Divergence), see General losses.

NLP metrics evaluate tasks from machine translation and summarization to question answering and sequence labeling.

Most of the information is available in the BERT paper. Key details: Multi-head attention. Transformer encoder.

BERTopic is a modular topic modeling pipeline.

GloVe (Global Vectors for Word Representation) is a word embedding technique developed in 2014.

Latent Dirichlet Allocation (Blei, Ng, Jordan, 2003) is the canonical probabilistic topic model.

Retrieval-Augmented Generation (RAG) is a hybrid approach that combines LLM’s generative abilities with real-time information retrieval from external knowledge sources​.

Recurrent Neural Networks are a type of neural networks designed for processing sequential data.

Term Frequency-Inverse Document Frequency (TF-IDF) is a numerical statistic used in NLP to show how important a word (term) is to a document in a corpus.

A survey of the main topic modeling methods, ordered roughly by historical development (matrix factorization → probabilistic generative models → neural → embedding-based).

Topic modeling is an unsupervised technique for discovering abstract themes in a document collection, where a document is whatever unit of text the project treats as one (article, review, tweet, paragraph, support ticket).

The first Transformer was introduced in the Attention Is All You Need paper, soon after that BERT was published.

Word embedding is a representation of a word, usually with a vector of values.

Word2Vec is a approach for learning word embeddings.

fastText is a library for efficient text classification and word representation learning by Facebook.