At the moment, nothing is as useful as a GPT-4 subscription, even though its context memory is currently limited. When that memory is unlimited, we’ll all have new best friends and personal C-suite teams.
Some numbers on the limit. GPT-4 reaches most of us through ChatGPT Plus, the $20-a-month tier, where it has been available since March 14 “with a usage cap.” The model’s standard context window is 8K tokens, with a 32K version in the API. Call it a few thousand words at the low end. That’s the whole of what it can hold in mind at once. Everything you told it before that has, as far as the model is concerned, never happened.
I’ve been researching Pinecone, a vector database. It’s possible that linking an OpenAI “user” with a Pinecone database could allow a GPT-like service to offer effectively infinite memory and context to each user. The model’s own window stays small. The database holds everything else, and hands the model only the pieces that matter for the question in front of it.
If that holds, a service could be built to help load all kinds of context a user might want — every email, every personal document, all their location data — into Pinecone, and have it available to improve GPT’s answers. I’m skipping the privacy concerns for the moment, and they are not small.
I think we could build a “personalized” GPT-4 that has access to your entire digital life when weighing its responses, using Pinecone. Approaching perfect memory.
Samuel Johnson wrote, in 1759, that “the true art of memory is the art of attention.” He meant that we forget what we never properly attended to. The interesting thing about this architecture is that it splits the two: the database remembers everything indiscriminately, and the model’s whole job becomes attention — deciding which of ten years of email matters right now.
There’s a business point too. It is still possible to fund your way to growth and build a brand name and traffic toward a derivative, but comparable, AI service.
We should consider this. Pinecone has a chatbot tutorial that’s the natural starting point.
