Wider availability of ChatGPT-style systems, and release of open-source versions, would make it more difficult to limit research or wider deployment. And the competition between companies large and small to adopt or match ChatGPT suggests little appetite for slowing down, but appears instead to incentivize proliferation of the technology.
Last week, LLaMA, an AI model developed by Meta—and similar to the one at the core of ChatGPT—was leaked online after being shared with some academic researchers. The system could be used as a building block in the creation of a chatbot, and its release sparked worry among those who fear that the AI systems known as large language models, and chatbots built on them like ChatGPT, will be used to generate misinformation or automate cybersecurity breaches. Some experts argue that such risks may be overblown, and others suggest that making the technology more transparent will in fact help others guard against misuses.
Meta declined to answer questions about the leak, but company spokesperson Ashley Gabriel provided a statement saying, “While the model is not accessible to all, and some have tried to circumvent the approval process, we believe the current release strategy allows us to balance responsibility and openness.”
ChatGPT is built on top of text-generation technology that has been available for several years and learns to mirror human text by picking up on patterns in enormous quantities of text, much of it scraped from the web. OpenAI found that adding a chat interface and providing an additional layer of machine learning that involved humans providing feedback on the bot’s responses made the technology more capable and articulate.
The data provided by users interacting with ChatGPT, or services built on it such as Microsoft’s new Bing search interface, may provide OpenAI a key advantage. But other companies are working on replicating the fine-tuning that created ChatGPT.
Stability AI is currently funding a project investigating how to train similar chatbots called Carper AI. Alexandr Wang, CEO of Scale AI, a startup that carries out data labeling and machine-learning training for many technology companies, says many customers are asking for help doing fine-tuning similar to what OpenAI did to create ChatGPT. “We’re pretty overwhelmed with demand,” he says.
Wang believes that the efforts already underway will naturally mean many more capable language models and chatbots emerging. “I think there will be a vibrant ecosystem,” he says.
Sean Gourley, CEO of Primer, a startup that sells AI tools for intelligence analysts, including those in the US government, and an adviser to Stability AI, also expects to soon see many projects make systems like ChatGPT. “The watercooler talk is that this took about 20,000 hours of training,” he says of the human feedback process that honed OpenAI’s bot.
Gourley estimates that even a project that involved several times as much training would cost a few million dollars—affordable to a well-funded startup or large technology company. “It’s a magical breakthrough,” Gourley says of the fine-tuning that OpenAI did with ChatGPT. “But it’s not something that isn’t going to be replicated.”
By Wired, March 25, 2023
