I’m working on a project to get GPT to work with other tools — update Salesforce, say, or summon a human into the chat. The trouble is that a chat AI can’t do this directly.
What’s required is an intermediate observer layer, an IOL: something that monitors the conversation between a human and GPT and infers when it, the observer, should act, because the conversation suggests that action is needed. The layer then has to insert itself, and the tools it judges necessary, into the human/GPT conversation, holding GPT in reserve in case it’s needed again. Or it pops up a second dialog and gives the human direct access to the tool they seem to have asked for.
As far as I can tell, an IOL of this kind is what’s initiating the calls to OpenAI’s plug-ins. Those launched in March as an alpha, with a dozen partners including Expedia, Instacart, Klarna and Zapier — a dozen tools, and a model that has to decide when to reach for one.
Second task for the day. One reason we haven’t released the personalized AI engine is that it struggles, as GPT-4 does, to work with long-term memory stored in a separate database. It’s slow, and it takes perfect prompts for GPT-4 to pull the correct data on each request. A paper posted last month, ReWOO, explains why and proposes a fix: detach the reasoning from the tool calls, and chain several LLMs — a Planner, a Worker and a Solver. The authors report “5x token efficiency and 4% accuracy improvement on HotpotQA,” and this write-up explains the why and the how. As I see it working for our engine, the planner intuits a request for additional resources, the worker (a separate instance) researches solutions, and the solver works with those solutions to devise a course of action and hands it back to the planner to work again with the human.
Put the two together — an IOL watching the conversation, Planner, Worker and Solver behind it, and third-party tools linked into the process — and you get a more efficient Auto-GPT that can accomplish real-world (digital) tasks for humans.
Has anyone heard of this concept, or have code for a universal IOL?
