Large Language Models

We have already talked about Large Language Models (LLMs) in the previous weeks as we discussed AI. However, today we dive deeper into understanding LLMs.

Continuing with my readings of ChatGPT and Google Colab: The Easiest, Quickest Way to Start Learning AI by Dr. Kelsey, we get to a chapter dedicated solely to LLMs. Dr. Kelsey explains that "LLMs are artificial intelligence m models that are designed to understand and generate text." LLMs actually form part of the Artificial Neural Networks (ANNs) family we touched in my blog post What Even is Machine Learning? and in more detail in Using AI to Explore Deep Learning. Examples of LLMs I have used just in this class are ChatGPT and DALL-E. 

LLMs fall under the unsupervised learning as text is subjective. There is no one correct answer when interpreting text. "What is salt?" does not have a universal answer unlike 2 + 2, which will always be 4. Interpreting text depends on the context and historical data. This is what training models do. They prepare the algorithm to respond appropriately.

I asked ChatGPT, a LLM how LLMs work. First, training of the model is essential to achieve proper results. According to ChatGPT, 

LLMs are trained on vast datasets of text [which] can include books, articles, websites and other forms of written language.

LLMs are a type of Artificial Neural Networks and Deep Learning. To try and keep things straight, I've simplified the levels under AI that highlight the path to LLMs here:

We've already discussed Deep Learning in a previous post, so we will continue onto the next level: Transformer Networks, or Transformers.

I asked ChatGPT again to describe Transformer Networks in simple terms. What I understood is that Transformers are similar to speed reading on asteroids. Transformers models intake data and information and determine the words, sentences, and paragraphs most important to the story, but instead of going word by word and line by line, Transformers can analyze several paragraphs at a time. ChatGPT used the following metaphor to describe Transformers and I think it really puts everything into perspective:

It's like having a smart highlighter that marks which words in a sentence are most important for understanding the overall meaning.

Moreover, Transformer Networks are versatile as they can read text as well as write it, translate it into different language, analyze it, summarize it, and even follow through complex tasks like answering questions about Machine Learning.

Transformer Networks are the predecessor of Large Language Models. LLMs represent the most advanced to date application of deep learning technologies, as LLMs have the capacity to handle large and high-dimensional datasets. 

These technologies are still in their infancy and despite the incredible advancements LLMs have had in our world today, there are still several hurdles to overcome.

Dr. Kelsey highlights three main challenges LLMs face today: computational costs, data quality, and ethical concerns. I own a Mac and I do most of my coding work in R in my Mac. I wanted to share a screenshot of an error code I got while trying to create a Decision Tree algorithm.

This is a Mac... Please engage with Apple and ask them for support... After several years of many reports of installation problems on Mac, it's time to gingerly point out that there have been no similar problems on Windows or Linux.

Needless to say, developers definitely don't use Mac computers! Jokes aside, the computer power needed to train and maintain these models does not come cheap. Not to mention the high energy consumption these models take plus the carbon footprint this implies.

Data quality is also subject to the data fed to the model. Moreover, the inherit biases that can filter through from different cultural, gender-based, racial, or related attributes will play a role on the end result. There is a reason ChatGPT has a disclaimer stamped at the bottom of the screen: "ChatGPT can make mistakes. Consider checking important information."

Lastly there is the ethical concerns with using LLMs. Not only do you have to think about biases and misinformation, there is also the issue of dependence and over reliance. Just this year, my sister was no longer allowed to use digital textbooks in her community college course as faculty is now facing the issue of having college students plagiarizing ChatGPT. Moreover, jobs like marketers that depend on LLMs to create content and are not fact-checking the information they post could face serious ramifications.

References:

ChatGPT and Google Colab: The Easiest, Quickest Way to Start Learning AI

ChatGPT