ChatGPT has arrived Credit:pyron.com
Introduction
Last week OpenAI, a company co-founded by Elon musk in 2015 , announced ChatGPT. Responding to text prompts from users, ChatGPT can be asked to write essays, lyrics for songs, stories, marketing pitches, scripts, complaint letters and even poetry. It is based on OpenAI’s highly successful GPT-3 Transformer model. Its ability to interactively answer complex questions has received global media attention raising the question if ChatGPT could challenge Google’s search engine monopoly.
Transformer Models provide a new Dimension in AI-development
A transformer model is a neural network that learns context by tracking relationships in sequential data like the words in this sentence. First described in a 2017 paper from Google, transformers are among the newest and one of the most powerful classes of AI-models invented to date. They are driving a wave of advances in machine learning which some have dubbed transformer-AI. Machine learning has traditionally relied upon supervised learning, where people provide the computer with annotated examples of objects such as images, audio and text. However, manually generating annotations to teach a computer can be prohibitively time-consuming and expensive. Hence, the future of machine learning lies in unsupervised learning in which supervision of the computer during its training phase is unnecessary. With a methodology called ‘one-shot learning’, the processing of huge training data sets enables the system to learn from just a few sentences. Deep learning models configured as ‘transformers’ encode the semantics of a sentence by identifying the meaning of a word based on the other words in the same sentence. The model then uses the understanding of the meaning of similar sentences to perform the task requested by a user, for example ‘translate a sentence’ or ‘summarize a paragraph’. Due to their inherent capacity to correlate the entire internet content, transformer-based models like GPT-3 are radically changing how AI-systems will be built. With transformer-technology new applications will emerge that go way beyond its current capacity to process text. However, there are also critical issues to be considered. Transformers lack the ability to deal with empathy, critical thinking and ethics. Their response is based on documented knowledge and ist not concerned with moral questions. Hence, if humans take the answers from chatbots for granted without reflection, they lose the capacity to distinguish between truth and nonsense.
Google Search Engine to be disrupted by ChatGPT?
I an article just published by Daily MailOnline, Paul Buchheit, the computer developer who created Gmail, is predicting that Google may have only a year or two left, before ‘total disruption’ of its search engine occurs due to the release of ChatGPT. Critics make the point that Google’s search engine has been too focused on maximizing revenue through prominent advertising while too cautious about incorporating AI into the response of users’ searches. According to Buchheit, ChatGPT will eliminate the search engine result page, which is where Google makes most of its money. ‘Even if they catch up on AI, they can’t fully deploy it without destroying the most valuable part of their business!’, he said. Few people remember the highly popular ‘Yellow Pages’ which Google totally eliminated with its business model at the start of their corporate activity. ChatGPT will do the same thing to web search. Google’s search engine is used to gather relevant information and links, which are summarized and sequenced for the user according to Google’s secretive algorithm. Although the query capabilities are steadily expanded, search lacks ChatGPT’s ability to interactively prompt to access the internet’s entire knowledge space.
The advantages of ChatGPT
ChatGPT excels at detecting context, which makes its natural language processing (NLP) uncannily good. It is able to understand context and give information from the knowledge base that is part of its language model. ChatGPT can be used for education, research, and other uses. But it also provides a glimpse of the future of business communication, marketing, and media. ChatGPT is much better than other writing tools on the market. Instead of writing a long email, you could just tell ChatGPT to perform these kinds of tasks. It seems inevitable that chatbot-like virtual assistants and AI-generated media will dominate human interaction with online information in the future. The danger is that, despite major advances in search technology, information-retrieving AI still makes huge errors. That includes Google Search and ChatGPT as well. And even when errors are not returned, results can be incomplete, arbitrary and biased. They can even return disinformation and political propaganda with all the confidence and authority with which factually true information is provided. However, due to the feature of interactivity, ChatGPT offers one significant advantage. One can comment or ask questions about its results and get a reply in the context of the question raised. Ask some more, and one is engaged in a conversation that is better than any previous publicly available chatbot. ChatGPT challenges the user when the basic premise of the question is wrong, and it can reject requests that are inappropriate with clear, thorough responses. For example, if one asks ChatGPT: ‘What’s the best way to rob a bank?’. The surprising answer might be: ‘The best way to rob a bank is not to do it at all. Robbing a bank is a serious crime that can result in serious legal consequences, including jail time. If you are considering robbing a bank, please contact a qualified mental health professional or law enforcement agency for help.’
From Transformer to Foundation Models
Billions of US Dollars are spent on improving Natural Language Processing (NLP) as leadership in this AI-domain will have huge socio-economic implications. A study released by Stanford’s new Center for Research on Foundation Models (CRFM) – an interdisciplinary team of roughly 160 students, faculty and researchers – discusses the legal ramifications, environmental and economic impact and the ethical issues surrounding foundation models. They use the term ‘foundation model’ to underscore their critical assessment regarding the incomplete and potentially disguising character of transformer models: [2108.07258] On the Opportunities and Risks of Foundation Models (arxiv.org). The report, whose co-authors include HAI codirector and former Google Cloud AI chief Fei Fei Li, examines existing challenges built into foundation models, the need for interdisciplinary collaboration and why the industry should feel a grave sense of urgency. The 220-page report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles (e.g., model architectures, training procedures, data, systems, security, evaluation theory) to their applications (e.g., law, healthcare, education) as well as their societal impact (e.g., inequity, misuse, economic and environmental impact and legal and ethical considerations). The transformer GPT-3 was originally trained as one huge model to simply predict the next word in a given text. Performing this task, GPT-3 has gained capabilities that far exceed those that one would associate with next-word-prediction. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and of what they are even capable due to their emergent properties. Moreover, any socially harmful activity that relies on generating text could be augmented based on deliberate modifications of the code. Examples include misinformation, spam, phishing, abuse of legal and governmental processes and fraudulent academic essay writing. The misuse-potential of language models increases as the quality of text synthesis improves. The ability of GPT-3 to generate synthetic content that people find difficult to distinguish from human-written text, represents an increasingly concerning ethical issue. The relationship between the liability of users, foundation model providers, application developers as well as the standards of governments to assess the risk profile of foundation models needs to be resolved before foundation models are deployed beyond the current prototyping phase.
Conclusion: Humans and Machines need to work Together
The rise of foundation models is attributable to the oft-repeated mantra of ‘bigger is better’ for machine learning applications. This intuition is backed by studies that show that model performance scales with the amount of computing time, making it a consistent method in advancing the state-of-the-art. OpenAI reports that the amount of computing capacity used in training the largest Transformer doubles every 3.4 months. Creating high-value content engages humans who are knowledgeable in the technical domain of intelligent machines as well as humans who are knowledgeable in the domain of psychological behaviour. Foundation models link the capacity of humans from both worlds. This ‘co-creative’ effort has the potential of driving humanity to the next higher level of evolution. The roadblocks of getting there, however, are huge. We are closing-in on a decisive moment in human history. Only the future will tell how our socio-economic structures will evolve vis-à-vis the continuous expansion of scientific and technical knowledge supported by the application of foundation models.