I’ve been saying for weeks that knowledge ingestion is the killer AI app nobody’s talking about. Well, now I’m putting my money where my mouth is.
I’ve been building a tool that lets you upload a PDF and immediately start querying it — asking questions, extracting insights, pulling structure out of unstructured documents. And after weeks of development, I can tell you with absolute certainty: the tech works. But the thing that ACTUALLY matters isn’t the model, the vector database, or the chunking strategy.
It’s the prompt.
The Prompt Is the Product
Here’s what I mean. You can take the same PDF — say, a 40-page contract or a dense research paper — and feed it into the same AI pipeline twice. Change nothing but the system prompt. The difference in output quality is staggering. One prompt gives you generic, surface-level summaries that miss the point. The other gives you precise, contextual answers that feel like talking to someone who actually READ the document.
This is the part most people building in this space haven’t figured out yet. They’re obsessing over model selection, fine-tuning, retrieval architectures — and yeah, those matter. But the programming prompt is EVERYTHING. It’s the difference between a tool that feels like a toy and one that feels like a junior analyst.
I’ve been experimenting heavily with this, and the results are pretty eye-opening. The same underlying technology, with a well-crafted prompt, can handle legal documents, financial reports, technical specifications — all with surprisingly competent output. With a lazy prompt? It hallucinates, misses key clauses, and gives you the kind of vague answers you could’ve gotten from skimming the executive summary yourself.
Why Playing Around Beats Planning
Here’s the thing I keep coming back to: clearly, it’s better to play around with the tech and see what other minds figure out than to sit in a room and try to architect the perfect solution from scratch.
I wrote a few weeks ago about knowledge ingestion being the killer AI app and the startup implications. What I’ve learned since then is that the real unlock isn’t technical — it’s creative. The people who are going to win in this space are the ones who experiment relentlessly with HOW they instruct the AI, not just what they feed it.
Think about it this way. When ChatGPT launched, the first wave of users typed in basic questions and got basic answers. Then the prompt engineering crowd showed up and started getting 10x better results from the exact same model. The same thing is happening now with document-based AI tools, except the stakes are higher because you’re working with YOUR data, YOUR documents, YOUR business context.
The PDF Problem (And What’s Next)
Right now, most of these tools — including what I’ve been building — work best with text-heavy PDFs. Documents with lots of images, charts, and embedded visuals are still a challenge. The AI can’t “see” a graph inside a PDF the way it can parse text. That’s a solvable problem, but it’s not solved yet.
Spreadsheets are another frontier. On the surface, uploading a CSV or Excel file seems simpler than a PDF — it’s already structured data, right? But the reality is more complicated. Spreadsheets carry implicit logic. The relationships between columns, the formulas, the way a human reader understands that row 47 is a subtotal and not a data point — teaching an AI to parse that context is genuinely hard. It’s going to take longer to get right, and anyone who tells you otherwise is either oversimplifying or selling something.
What This Means for Builders
If you’re building anything in the AI-plus-documents space right now, here’s my take:
Stop optimizing infrastructure and start optimizing instructions. Your prompt strategy will drive more output quality improvement than your next three engineering sprints combined.
Let other people use your tool as early as possible. I’ve been opening up access to what I’m building, and the most valuable thing isn’t the feedback on bugs or UI — it’s watching how OTHER people prompt the system. They think of use cases I never would have. They phrase questions differently. They stress-test assumptions I didn’t know I’d made.
Accept the current limitations honestly. Images in PDFs, complex spreadsheets, multi-modal documents — these are real constraints. Build for what works NOW and be transparent about what doesn’t. Users will forgive limitations; they won’t forgive overpromising.
The Bigger Picture
We’re still in the very early innings of this. The combination of large language models and personal or business document collections is going to reshape how people interact with their own information. I genuinely believe that. But the gap between “this technology exists” and “this technology is useful” is almost entirely a prompt engineering gap right now.
And that’s actually exciting. Because it means the barrier to building something genuinely useful isn’t capital or compute — it’s creativity and iteration. Play around with the tech. See what works. Let other people surprise you with what they figure out.
That’s where the real breakthroughs are going to come from.
