November 30, 2022: «We’ve trained a model called ChatGPT which interacts in a conversational way… ».
It was this announcement of OpenAI’s newest product, a chatbot belonging to the generative AI space, that changed humanity’s perception of the potential of Artificial Intelligence. A subject considered mythical until that day and to a certain extent distant from the general public, was now just a click away.
It is a fact that generative AI has benefited from large investments in recent times, just as an example, according to an article in the “Open Data Science Conference”, Microsoft plans to invest $10 billion in OpenAI, after having already invested over 1Bi$.
But after all: What is ChatGPT? What is it for? How it works? It’s trustable?
Unlike Google, which is a model created to retrieve information, ChatGPT is a language model and in fact it is the latest in the research lab’s lineage of large language models using Generative Pre-trained Transformer (GPT) technology.
ChatGPT has the ability to generate text for a variety of purposes with skill, detail, and consistency. The model used in ChatGPT was fine-tuned with the aim of reducing bias or false responses in its outputs, like other Machine Learning models, except that in this case a combination of Supervised Learning was used in conjunction with Reinforcement Learning with Human Feedback.
In other words, we can say that ChatGPT is a somewhat sophisticated way of learning links between words.
A technology that we have known for some time, but which gains its maximum expression now, taking into account fundamentally the amount of text that was used to train it (the exact size of the dataset is not publicly disclosed, but OpenAI has stated that its models are trained on “billions of words”) and the conjugation of scales (from words to sentences, from sentences to paragraphs, from paragraphs to text …) that it manages to output.
Another important advantage of these models is their ability to learn and adapt to new tasks and languages without any changes to the architecture, just by “feeding” new training data to the model.
Going a little deeper into the technology behind ChatGPT, it is important to note that generative AI is based on large language models, a type of artificial intelligence that is trained to process and generate natural language. Meaning that it is designed to “understand” and to generate human-like language.
In addition to GPT there are other great language models like BERT (Bidirectional Encoder Representations from Transformers), or RoBERTa (Robustly Optimized BERT Approach), from OpenAI competitors, namely from Google, the company that started the snowball process with the article “Attention Is All You Need”, which introduced the concept of “Transformers”. These are neural network architectures that can achieve incredible performances on these types of tasks, but requiring several orders of magnitude less computational power than previous approaches such as Recurrent Neural Network (RNNs).
RNNs were basically unfeasible for real-world tasks, while Transformers have clearly overcome this limit, although they still require very large volumes of high-quality data for training, as well as large computational resources to train and adjust the models, which is expensive and time-consuming. These difficulties justify the need for a large amount of financial investment in this type of technology.
So far, we know that ChatGPT is a free (chatbot) capable of producing text, with a complicated and expensive technology as a base and bringing information on different subjects, but how “diverse” can subjects be? It is capable of producing wedding speeches, corporate emails, journalistic format texts, organization lists, and computer code, you just have to know how to ask it correctly, and it can do it in different languages, including: English, Spanish, German, French, Italian, Portuguese, Dutch, Russian, Arabic, Chinese and Japanese.
In an educational way and seeking to understand the future prospects of the most famous bot nowadays, we tested its capacity by asking: “Describe the future of ChaGPT”, and this was the model’s response:
3. Increased personalization: As language models like ChatGPT learn more about individual users and their preferences, they may be able to offer increasingly personalized and relevant responses, improving the overall user experience.
4. More advanced capabilities: In the future, language models like ChatGPT may develop new capabilities such as the ability to generate text in a specific writing style, or to summarize and condense large amounts of text.
However, it’s also important to note that language models like ChatGPT are not perfect and still have limitations, such as biases in the training data, difficulty in handling complex and nuanced situations, and potential ethical and privacy concerns. As the technology continues to evolve, it will be important for researchers and developers to address these issues and strive for responsible and ethical AI development.”
Finally, it is important to note that ChatGPT is not a replacement for people doing truly intelligent tasks, because there is no guarantee that the generated text is “true” or that it solves the problem we need to solve.
In that sense, it is fundamentally different from Google Search, although both use almost the entire Web as a source. While ChatGPT generates new text, which could be present in the source (but probably isn’t…), Google Search points us to the real information, with context.
This is a profound difference.
ChatGPT’s ability to put facts and text together is practically non-existent, as pointed out in the words of OpenAI CEO Sam Altman: “ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. It’s a mistake to be relying on it for anything important right now. It’s a preview of progress; we have lots of work to do on robustness and truthfulness.”
ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. It’s a mistake to rely on it for anything important right now. It’s a preview of progress, we have a lot of work to do in terms of robustness and veracity.
