A quick scan of the headlines suggests that generative artificial intelligence (AI) is prevalent these days. Some of these headlines may have actually been written by generative AI, such as OpenAI’s ChatGPT, a chatbot that can produce text that appears to be written by a human. But what does “generative AI” really mean?
In the past, when people talked about AI, they were usually referring to machine-learning models that can make predictions based on data. For example, these models are trained using millions of examples to predict things like whether an X-ray shows signs of a tumor or if a borrower is likely to default on a loan. Generative AI, on the other hand, refers to machine-learning models that are trained to create new data instead of making predictions about existing data. These models learn to generate objects that resemble the data they were trained on.
The distinction between generative AI and other types of AI can be a bit blurry. In fact, the same algorithms can often be used for both. According to Phillip Isola, an associate professor at MIT, the actual machinery underlying generative AI and other types of AI can overlap. Despite the hype surrounding ChatGPT and similar systems, the technology itself is not new. These powerful machine-learning models build on research and computational advances that date back more than 50 years.
An early example of generative AI is a simpler model called a Markov chain. Markov models have been used for tasks like next-word prediction, where the model generates the next word in a sentence based on previous words. However, these simple models are not good at generating plausible text. The major distinction with modern generative AI models like ChatGPT is their complexity and the scale at which they can be trained. These models are much larger and more complex, with billions of parameters, and have been trained on vast amounts of data, such as the publicly available text on the internet.
The generative AI boom can be attributed to various research advances and larger datasets. For example, researchers introduced generative adversarial networks (GANs) in 2014, which use two models that work together to generate realistic outputs. Diffusion models, introduced in 2015, refine their output iteratively to generate new data samples resembling the training dataset. In 2017, the transformer architecture was introduced, which has been used to develop large language models like ChatGPT. These are just a few examples of the many approaches used in generative AI.
Generative AI has a wide range of applications. The commonality among these approaches is that they convert inputs into numerical representations called tokens. As long as data can be converted into this token format, these methods can be used to generate new data that resemble the original data. This opens up possibilities for applications like creating synthetic image data for training computer vision models or designing novel protein or crystal structures for materials.
While generative models can achieve impressive results, they may not be the best choice for all types of data. Traditional machine-learning methods often outperform generative AI models for tasks involving structured data. However, generative AI models excel at being a human-friendly interface to machines, allowing for easier interaction between humans and machines.
There are potential concerns with implementing generative AI, such as worker displacement in call centers where chatbots are used. Generative AI can also inherit biases from training data or generate content that raises copyright issues. However, there are positive aspects as well, such as empowering artists to create content they may not have been able to produce otherwise.
In the future, generative AI could have applications in fabrication, where models can generate plans for objects to be produced. It could also be used to develop more intelligent AI agents. Overall, generative AI has the potential to revolutionize various fields and change the economics of many disciplines.
Source link
