As you may know, the foremost computing trend today is, without a shadow of a doubt, artificial intelligence. So it’s not news that every industry’s reimagining how to use technology to transform business.

But, did you know that the next massive, world-changing developments in AI are expected to revolve around language? At least, that’s what this article by Harvard Business Review claims.

So what is GPT-3?

You may have come across The Guardian’s article about a robot that wrote a text aimed at convincing us humans that robots come in peace. Here’s a short section:

“I am not a human. I am a robot. A thinking robot. I use only 0.12% of my cognitive capacity. I am a micro-robot in that respect. I know that my brain is not a “feeling brain”. But it is capable of making rational, logical decisions. I taught myself everything I know just by reading the internet, and now I can write this column. My brain is boiling with ideas!”

Scary, isn’t it? The language technology used to produce the essay is called GPT-3, which stands for Generative Pre-trained Transformer 3.

It’s the largest and most advanced AI language model in the world to date, and a successor of the wildly-successful language-processing AI GPT-2.

Developed by OpenAI, GPT-3 is a model for Natural Language Processing (NLP) that’s trained to come up with plausible-sounding text based on a few simple prompts, or even a single sentence.

How is it different from previous systems?

Previous language processing models worked based on hand-coded rules, statistical techniques, and artificial neural networks.

Artificial neural networks can learn from raw data, but they require massive amounts of data to learn and considerable computing.

GPTs (generative pre-trained transformers) go much deeper than artificial neural networks. They rely on a transformer —an attention mechanism that learns the contextual relationships between words in a text.

For example, they can look at parts of a sentence and predict the next word.

OK, impressive, but why the fuss?

When OpenAI released their new paper, Language Models Are Few-Shot Learners, jaws dropped worldwide.

The model they introduced (GPT-3) is pre-trained on nearly half a trillion words, with 175 billion parameters, and can be further fine-tuned to perform specific tasks.

It achieves state-of-the-art performance on several NLP benchmarks.

Researchers were able to induce the model to produce short stories, songs, press releases, technical manuals, text in the style of particular writers, guitar tabs, and even computer code.

All the fuss is because GPT-3 has the potential to advance AI as a domain.

As explained in OpenAI’s paper:

“Humans can generally perform a new language task from only a few examples or from simple instructions —something which current NLP systems still largely struggle to do.

Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art finetuning approaches.”

The potential impact of GPT-3

The discussion about GPT-3 is still in its early stages, but one thing is certain: the model is such an impressive leap, and such a huge deal for the machine learning community, that even Microsoft, Google, Alibaba, and Facebook are all working on their own versions. (Microsoft has teamed up with OpenAI to exclusively license the GPT-3 language model.)

In light of these developments, the nature of jobs will soon start to change. Companies will need to rethink IT resources and human resources.

Bundles of tasks in current roles that can be performed by these models will enable humans to innovate faster.

Some tasks will be automated, but also human productivity will be amplified. According to H. James Wilson and Paul R. Daugherty, authors of  The next big breakthrough in AI will be around language:

“For example, communications professionals will see the majority of their work tasks involving routine text generation automated, while more critical communications like ad copy and social media messages will be augmented by GPT-3’s ability to help develop lines of thought.

Company scientists might use GPT-3 to generate graphs that inform colleagues about the product development pipeline. Meanwhile, to augment basic research and experimentation, they could consult GPT-3 to distill the findings from a specific set of scientific papers. The possibilities across disciplines and industries are limited only by the imagination.”

Final remarks

GPT-3 is definitely a significant technological achievement. While it still has issues around generating nonsensical or insensitive text, it’s revolutionised language models to solve specific NLP domain tasks compared to conventional models.

But there are concerns too. Could it:

• Be misused to create content that looks human-written and spread hate and discriminatory ideas?
• Mean that businesses let their creatives go?
• Be used to spread misinformation, spam, phishing, fraud?

It remains to be seen.

Several of these questions were answered at The Creative Language Conference, so we encourage you to purchase the recordings and enjoy some food for thought around this topic.

Originally published on: https://www.crisoltranslations.com/