I’ve been liking Gemini. Google opened Gemini 1.5 Pro to developers in 180-plus countries on Tuesday, in public preview, with the million-token context window that had been limited to a private preview since February.
And a little update from the team at Ragu. This is a GPT-4 / Mistral combo product — LLMs working together. Ragu now has a couple of pipelines that use GPT-4 for hierarchy, Perplexity for research, Claude for finishing, and Mistral, powered by Groq, to verify the outputs. Basically, drawing the best from all of them.
Some context on the cast. Claude 3 arrived on March 4 in three sizes — Haiku, Sonnet and Opus — and Anthropic says the family “sets new industry benchmarks across a wide range of cognitive tasks.” Mistral’s Mixtral 8x7B is “a sparse mixture of experts model (SMoE) with open weights,” released in December under the Apache 2.0 license, and Mistral claims it “matches or outperforms GPT3.5 on most standard benchmarks.” Groq is the hardware underneath: in February an independent benchmark clocked Groq serving Llama 2 70B at 241 tokens a second, “more than double the speed of other hosting providers.”
The division of labor is the point. Hierarchy — deciding what matters and in what order — is a judgment task. Research wants a model that reads the live web. Finishing wants the cleanest writer. Verification wants something fast, open and cheap that owes nothing to the model it’s checking.
Nobody’s model is best at everything, whatever the launch posts say.
