I’ve been sending a video around to a bunch of people. It’s a tutorial from a channel called Chat with data, posted at the end of March, on using GPT-4 to chat with roughly a thousand pages of Tesla’s annual reports — you ask a question in plain English and it answers from the filings.
One of the best AI minds I know found it and set about building a knowledge ingestion system. Basically, you input information, and the output is an AI that knows everything about that knowledge. It knows your documents, and only your documents, and answers in their terms.
The technique underneath is older than the excitement. A 2020 paper by Patrick Lewis and colleagues described models that “combine pre-trained parametric and non-parametric memory for language generation,” where the non-parametric memory “is a dense vector index” of documents. In English: the model does the reading and writing, and a searchable store of your own material supplies the facts. The model doesn’t need to have been trained on your data. It needs to be handed the right page at the right moment.
At the other end of the scale, you can train on the data itself. Bloomberg announced BloombergGPT at the end of March, “a 50 billion parameter language model that is trained on a wide range of financial data,” built on “a 363 billion token dataset based on Bloomberg’s extensive data sources.” They report that it “outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks.” Proprietary data going in, and a model nobody else can build coming out.
Between those two — the weekend tutorial and the fifty-billion-parameter flagship — sits every company with a filing cabinet. Manuals, contracts, support tickets, call notes, decades of email. The general models are extraordinary and everyone has them, which is precisely why they confer no advantage.
Whether you build it or the proverbial WE do, it’s likely that proprietary data can create an AI worth some level of investment.
