OpenAI has just made GPT-3 generally available to all developers (in supported countries). What is GPT-3? Wikipedia defines Generative Pre-trained Transformer 3 as an autoregressive language model that uses a deep learning neural network to produce human-like text. In other words, GPT-3 can automatically write news stories and articles, create blog posts, compose poetry, and more, in addition to generating working program code in more than a dozen programming languages. GPT-3 is also being used to develop conversational models which can respond to comments or questions with replies that fit the context of the conversation.
There are a number of known weaknesses in GPT-3 though:
- The model is pre-trained - it doesn't continue to learn from each interaction.
- If the Internet material that was used to train GPT-3 included biased content, GPT-3 will tend to exhibit those same biases.
- GPT-3 can only accept a limited amount of input for each interaction, so the input has to adequately describe the desired result.
Plus GPT-3 also has a dark side. According to an article on VentureBeat.com, "the Middlebury Institute of International Studies’ Center on Terrorism, Extremism, and Counterterrorism found that GPT-3 can generate “influential” text that could radicalize people into far-right extremist ideologies. A group at Georgetown University has used GPT-3 to generate misinformation, including stories around a false narrative, articles altered to push a bogus perspective, and tweets riffing on particular points of disinformation."
OpenAI does have guidelines that specify what type of content GPT-3 can and can't be used to create and filters to try to detect models that violate those guidelines. However, like many other advances in technology, it's still an open question as to whether tools like GPT-3 can be supervised effectively and what their overall effect will be.
So what's the future of GPT-3 and similar deep learning models? The best answer is that OpenAI and other organizations specializing in neural network applications are already hard at work attempting to develop even more advanced deep learning models. There's no turning back at this point, so hopefully GPT-3 and its successors will avoid the "AI is going to take over" stigma and turn out to produce more benefits than problems.