There is now an open source alternative to ChatGPT, but good luck running • Zoo House News

The first open-source equivalent of OpenAI’s ChatGPT is here, but good luck running it on your laptop — or at all.

This week, Philip Wang, the developer responsible for reverse engineering closed-source AI systems including Meta’s Make-A-Video, released PaLM + RLHF, a text generation model that behaves similarly to ChatGPT. The system combines PaLM, a large language model from Google, and a technique called Reinforcement Learning with Human Feedback – RLHF for short – to create a system that can do pretty much any task ChatGPT can do, including composing emails and suggesting computer code.

But PaLM + RLHF is not pre-trained. That is, the system has not been trained with the sample data found on the Internet that is required for it to actually work. Downloading PaLM + RLHF doesn’t magically install a ChatGPT-like experience – that would require compiling gigabytes of text for the model to learn from and finding hardware powerful enough to handle the training load.

Like ChatGPT, PaLM + RLHF is essentially a statistical word prediction tool. When fed an enormous number of samples from training data – e.g. B. Posts from Reddit, news articles and e-books – PaLM + RLHF learns how likely words are to occur based on patterns such as the semantic context of the surrounding text.

ChatGPT and PaLM + RLHF share a special sauce in Reinforcement Learning with Human Feedback, a technique that aims to better align language models with what users want. RLHF involves training a language model—in the case of PaLM + RLHF PaLM—and fine-tuning it against a data set, the prompts (e.g., “Explain machine learning to a six-year-old”), coupled with what human volunteers expect from the model , includes saying (e.g., “Machine learning is a form of AI…”). The above prompts are then fed into the fine-tuned model, which generates multiple responses, and the volunteers rank all responses from best to worst. Finally, the rankings are used to train a “reward model” that takes the original model’s answers and sorts them by preference, filtering for the best answers to a given prompt.

It is an expensive process to collect the training data. And the training itself is not cheap. PaLM has a size of 540 billion parameters, where “parameters” refer to the parts of the language model learned from the training data. A 2020 study put the cost of developing a text generation model with just 1.5 billion parameters at up to $1.6 million. And to train the open-source Bloom model with 176 billion parameters, it took three months with 384 Nvidia A100 GPUs; A single A100 costs thousands of dollars.

Running a trained model of size PaLM + RLHF is also non-trivial. Bloom requires a dedicated PC with around eight A100 GPUs. Cloud alternatives are expensive, with the cost of running OpenAI’s text-generating GPT-3 — which includes around 175 billion parameters — on a single Amazon Web Services instance being around $87,000 per year.

Sebastian Raschka, an AI researcher, points out in a LinkedIn post about PaLM + RLHF that scaling the required development workflows could also prove to be a challenge. “Even if someone gives you 500 GPUs to train this model, you still have to deal with the infrastructure and have a software framework that can handle it,” he said. “It’s obviously possible, but at the moment it’s a big effort (of course we’re developing frameworks to make that easier, but it’s not trivial yet).”

That’s all to say that PaLM + RLHF will not replace ChatGPT today – unless a well-funded company (or person) takes the trouble to train it and make it publicly available.

Fortunately, several other efforts to replicate ChatGPT are progressing rapidly, including one led by a research group called CarperAI. In partnership with open AI research organization EleutherAI and startups Scale AI and Hugging Face, CarperAI plans to release the first ready-to-run ChatGPT-like AI model trained with human feedback.

LAION, the nonprofit that provided the first dataset to train stable diffusion, is also leading a project to replicate ChatGPT using the latest machine learning techniques. LAION has the ambitious goal of building an “assistant of the future” – one that not only writes e-mails and letters, but also “does meaningful work, uses APIs, dynamically researches information and much more”. It’s in the early stages. But a GitHub page with resources for the project went live a few weeks ago.