Technically possible and not expensive — Stanford rebuilt ChatGPT for $600 last week.
That’s the headline version, and it’s close enough to the truth to take seriously. The model is called Alpaca. Stanford’s researchers took Meta’s seven-billion-parameter LLaMA model and fine-tuned it on 52,000 instruction-following examples, examples they generated by having OpenAI’s own text-davinci-003 write them. That data cost “less than $500 using the OpenAI API.” The fine-tuning took three hours on eight A100s, “which costs less than $100 on most cloud compute providers.” The team’s own verdict is that Alpaca “behaves qualitatively similarly to OpenAI’s text-davinci-003, while being surprisingly small and easy/cheap to reproduce.”
So for under $600, a university lab stood up something that talks a lot like the thing everyone has been talking about all winter. Strictly it’s a cousin of ChatGPT rather than a copy, and a small one — but the point stands.
I think the difference may be between personal use and corporate use, though. For one person’s daily work, a model like this may already be plenty. Corporate may need massive services in healthcare research, energy, R&D… and I’m not sure what else. Note too that Alpaca “is intended only for academic research and any commercial use is prohibited,” so no company gets to find out with this particular model anyway.
But note, they’ve taken it down, as it was too costly to host. The demo went up on March 13 and was gone within about a week. Stanford’s statement, as The Register reported it, was that “given the hosting costs and the inadequacies of our content filters, we decided to bring down the demo.” Building the thing turned out to be the cheap part. Serving it to strangers was not.
It was also more prone to hallucinations, and the researchers were candid about it: Alpaca “exhibits several common deficiencies of language models, including hallucination, toxicity, and stereotypes,” and hallucination in particular, they say, “seems to be a common failure mode.”
And yet it could be run on a mobile phone’s computing power. The same Register piece notes that developers reportedly got the code up and running on Raspberry Pi computers and even a Pixel 6.
So that’s the picture, a week in: cheap to make, expensive to host, shaky on facts, and small enough to fit in a pocket.
