Meta creates AI language bot use external software tools

ChatGPT has revolutionized natural language processing, yet it still struggles with arithmetic and fact-checking. Meta researchers unveiled Toolformer, an AI language model that can learn to use search engines, calculators, and calendars without compromising its language modeling abilities, on Thursday. Toolformer's strength is its usage of APIs (application programming interfaces), which allow applications to communicate seamlessly and automatically. Toolformer was trained by giving it a few human-written samples of how each API is used and then letting it annotate a massive language modeling dataset with prospective API calls. It "self-supervised" it's learning. Photo Credit: Jesus Rogriguez The algorithm predicted text-based API calls like any other text. It can inject calls when creating text from human input. Toolformer can also "choose" the right tool and how to utilize it. Toolformer can contact APIs for search engines, calculators, language translators, and factual references. LLMs are notoriously bad at arithmetic. Toolformer can use a calculator to overcome that limitation. Toolformer could add a date to a calendar via an API link to a calendar app for an LLM-based assistant. Toolformer uses a 6.7 billion-parameter pre-trained GPT-J model. The researchers found that Toolformer outperforms the 175 billion-parameter GPT-3 model in tool-using tasks. Language model restrictions have been addressed before. Bing Chat, which made headlines last week, can execute web searches on its own, while others have integrated with browsers, calculators, and search engines. Meta's researchers said most methods for incorporating tools into language models have depended on vast amounts of human annotations or been limited to task-specific settings. Toolformer may handle a variety of tools without specialized expertise. Photo Credit: CNET With techniques like Toolformer, LLMs that can leverage external apps could become more versatile and trustworthy aides (ostensibly). But the capacity to conduct API calls may boost an LLM's potential to harm user data (in apps) or cause havoc in the outside world (via a web browser or communications tools)—abilities they may unwittingly engage while answering.