#architecture

The original Transformer paper. Neural sequence attention predates it: Bahdanau et al., 2014 introduced soft alignment for machine translation.

CLIP (Contrastive Language-Image Pre-training) is a neural net model developed by OpenAI that efficiently learns visual concepts from natural language supervision.

Deep & Cross Network (DCN) is a neural network architecture developed by Google that combines a cross network with a deep network to efficiently learn explicit and implicit feature interactions.

Deep Learning Recommendation Model is a neural network architecture developed by Facebook to handle large-scale recommendation tasks efficiently.

Neural Collaborative Filtering (NCF) is a deep learning approach to collaborative filtering for recommendation systems that aims to learn the complex user-item interaction function using neural networks.

Two-tower architecture is a neural network approach used in recommendation systems for candidate generation and retrieval.

Wide & Deep Learning is a joint model architecture developed by Google for recommendation systems and search ranking.

Most of the information is available in the BERT paper. Key details: Multi-head attention. Transformer encoder.

Recurrent Neural Networks are a type of neural networks designed for processing sequential data.

The first Transformer was introduced in the Attention Is All You Need paper, soon after that BERT was published.