Foundation Models
The ascendance of computer programs that simulate and process human conversations has taken the industry by storm. With 100+ million users, and the website traffic of 1.5 billion visitors per month, ChatGPT is an AI sensation of sorts. In order to get an appropriate response from ChatGPT, all one needs is a clear prompt with sufficient details. The impressive language capabilities of ChatGPT arises from two main AI model types: Foundational Models and Large Language Models.
Foundational models represent a category of artificial intelligence (AI) models trained on extensive un-labelled datasets encompassing both text and code. The AI algorithm that underpin foundation models learn using un-supervised training on deep leaning neural network architectures. Their purpose is to grasp the fundamental patterns and connections within language, leveraging this understanding to execute various tasks, including text generation, language translation, and answering queries.
Large Language Models
A Large Language Model is an instance of a Foundation Model. The ‘Large’ in large language models (LLMs) refer to the massive datasets (apart from the number of parameters and weights) that they are trained on. As an example, a 1 GB sized text file could have about 178 million words. ChatGPT was trained on 45TB of data. LLMs can understand the statistical relationships between words and phrases, allowing them to generate grammatically correct and semantically meaningful text, translate text, summarize, answer question and more.
A Large Language Model is an instance of a Foundation Model. The ‘Large’ in large language models refer to the massive datasets (apart from the number of parameters and weights) that they are trained on. As an example, a 1 GB sized text file could have about 178 million words. ChatGPT was trained on 45TB of data. LLMs can understand the statistical relationships between words and phrases, allowing them to generate grammatically correct and semantically meaningful text, translate text, summarize, answer question and more.
Generative AI (GenAI)
Generative AI is an umbrella term for AI algorithms and models that can create text, code, images, video, and music. Generative AI can use information from LLMs and other types of AI models to generate new content. Generative Ai models are trained on large datasets and learn to generate content that is similar to the examples they were exposed to during training. One of the key features of generative AI is its ability to produce novel and diverse outputs.
Are they bedfellows?
Are the buzz words FM, LLM and Generative AI related? The short answer is that not all Generative AI tools are built on LLMs. However, LLMs are a form of Generative AI. LLMs create text-only outputs and focus more on text-based content creation. A Large Language Model is an instance of a Foundation Model.
