The programming prompt is everything. Everything else in the project is plumbing by comparison.
Where the project stands: currently it works only with PDFs that don’t have images. We’re also developing a way to upload spreadsheets, but that’s going to take longer.
The job itself is one half the industry is chasing this spring — take a document, let a person ask it questions, and have the model answer from the document rather than from its own imagination. OpenAI’s own open-source retrieval plugin describes the same task as “semantic search and retrieval of personal or organizational documents using natural language queries.” The models behind projects like this got cheap in March, when OpenAI opened the ChatGPT API at $0.002 per 1,000 tokens, or about 750 words, with Snap, Quizlet, Instacart and Shopify among the first to build on it. Cheap enough that the interesting problem moved upstream, to what you tell the model before anyone asks it anything.
That’s the programming prompt — the system message, in OpenAI’s terms. It tells the AI who it is and what it’s supposed to do with the docs you upload. OpenAI’s cookbook puts it mildly: the system message “can be used to prime the assistant with different personalities or behaviors.” The same page admits that the first ChatGPT API model, gpt-3.5-turbo-0301, “does not generally pay as much attention to the system message” as GPT-4 does, and suggests putting important instructions in the user message instead. So the prompt is everything, and the model doesn’t always listen to it. That’s the craft.
So we built three demo projects, all pointed at the same document. One uses the prompt we’ve engineered, which seems to work. Two has no prompt and no guardrails. Three lets you insert your own. Two is the honest one — it shows what the model does when nobody tells it who to be.
Clearly, it’s better to experiment with the tech and see what other minds figure out.
