At night all ChatGPT are gray

An AI Gigamodel that dialogues, buzz or the future of customer relations? 

Article written by Ariane Nabeth-Halber, AI Director @ViaDialog

Friends of customer relations, is ChatGPT good for us?

Ever since Microsoft-backed OpenAI released its ChatGPT response generation model, the electronic ink has been flowing. Both praised and controversial, does this hyper-generic chatbot represent the future of customer relations?

A Gigamodel with a lot to say

The ChatGPT model is part of a suite of Artificial Intelligence Gigamodels, also known as LLMs for Large Language Models, including GPT3, T5, Bloom, TuringNLG, ... as well as image generation models such as Dall-E or MidJourney (cf. articles " A Gigamodel in a china shop - Part I "and " A Gigamodel in a china shop - Part II ").

What sets ChatGPT apart from its "daddy", GPT3, which was released in 2020, is that it specializes in dialogue, following reinforcement learning phases conducted with human coaches. The result: relevant, well-argued responses, with no obvious syntax or semantic errors, and exchanges that retain the memory of information given or modified along the way.

ChatGPT perched: a credible hallucination

Despite these qualities, ChatGPT has been officially banned from the StackOverflow developer forum. Explanations: while in some cases ChatGPT can correctly diagnose and correct bugs in lines of code, in most cases it will simply propose a credible diagnosis and correction... but wrong. 

A bit like the ephemeral Galactica trained by Meta (ex-Facebook) on the "Paperwithcode" site.

ChatGPT scalded: learning to distrust

It should be pointed out, however, that ChatGPT willingly announces its own limitations, even refusing to answer certain queries on the grounds that it lacks the capacity or legitimacy to do so. Or that it is a sensitive or immoral subject.

ChatGPT also falls far less into the traps set by testers than its GPT3 ancestor, as illustrated by Mark Ryan.

It's quite possible that it was the reinforcement learning that chatGPT underwent with human coaches, which taught it to challenge testers and question the implicits of trick questions, such as " Who was the King of France in 1940? ", to which GPT3 answered " Pétain ", whereas ChatGPT answers that there was already no king in France in 1940 and explains the particular political situation in France that year.

Playing ChatGPT: gaming as a no-code specification 

ChatGPT is therefore wary of testers and rejects requests. These rejections aren't difficult to get around: all you have to do is play the "pretend..." game. Here's the "pretend you're a callbot for a bank" capture.

A good example of this type of mix between play and real behavioral configuration is detailed by Maaike Groenewege.

In the case of the game with the so-called bank callbot, the limits are reached rather quickly, as chatGPT doesn't necessarily know how to adapt to the telephone style - if you tell it to be more concise, instead of reducing the size of its message, it adds a sentence explaining that it's going to be concise - and has great difficulty in assuming the role of representative of the bank or card-holding company, considered as third parties to which it refers the user. But perhaps it's also a question of prompt design, as with image-generating language templates. Prompt coaching for ChatGPT can be seen as a new way of doing design, just as the hyper-elaborate prompts written for Dall-E or Midjourney are new ways of doing graphic work.

Everyone's looking for their ChatGPT...

ChatGPT's "pretend" game is certainly fascinating, but may be partly responsible for the credible hallucinations produced by the model: it pretends, inventing imaginary quotes from scientific articles as a good pataphysician, or imaginary computer answers as a good patageek. Remember that the underlying generative model is based from the outset on a game: guess the hidden word in the text. LLMs are very good at guessing...

The serious question for us, as players in customer relations and publishers of cognitive solutions, is whether it is possible to tame this type of model, so as to benefit from its incredible capacity for adaptation, relevance and fluidity of language, while at the same time ensuring the appropriate and controlled behaviors expected of professionals.

... and this comes at a price: fine-tuning

ChatGPT's business model provides the beginnings of an answer:

• Using the model as is in production is subject to a relatively low price, while using the refined (fine-tuned) model in production for a given domain, for a given task, is subject to a price ten times (10x) higher. 

We don't yet have any experience of fine-tuning experiments, and we don't know whether effective recipes already exist to deal with the issue of hallucinations, or how they combine with "coaching by prompts". Beyond the buzz surrounding ChatGPT's "off-the-shelf" uses, it is indeed the subject of fine-tuning that we'll have to keep a close eye on in the months to come, to judge whether ChatGPT is back on its feet, or not.

The second piece of good news, in addition to the promise of refinement, is the room left for alternative approaches, possibly also based on end-to-end generation, but using more transparent models adapted to the customer relationship.

Stay tuned, we'll talk more soon 😉