Recently, Google released a couple new Gemini models. They were Gemini 3.6 Flash, 3.5 Flash Lite, and 3.5 Flash Cyber. Many people on the internet roasted Gemini Flash and Flash Lite for their less-than-frontier performance. This makes sense when you consider that OpenAI and Anthropic have been sparring back and forth for first place for quite a while but Gemini almost never gets that first place position. After all, this is Google we are talking about. Many people assume they have been scraping the world’s internet for decades and should have a lead based on that. They even created the Transformer before OpenAI realized its value. So, it has become a bit of a meme to roast Google’s models when they release.

The Performance Results

If we look at the results on Artificial Analysis, we see that the latest Gemini models are truly behind when it comes to general intelligence. In this image you can see 3.6 Flash ranking in at 12th place.

That is certainly not ideal. When you switch to the Coding Index, it improves to #11. Still not great.

More to AI Than Coding

I think most of the loudest voices on Twitter/X are judging it for its coding ability. While that is a valid concern, many people are overlooking its speed. Keep in mind that not all the numbers you see are true in all environments though. I seem to remember seeing a graph of the newer Gemini models approaching 300 TPS (tokens per second). From what I’ve seen, that number is only possible in heavily optimized environments. If you use OpenRouter, that number is significantly different.

When we compare the new Flash models to some of the top models from OpenAI and Anthropic, we see that Gemini is indeed faster by a large margin and with lower latency and at a much cheaper cost. But, is comparing Flash models to the flagship models actually a fair comparison? Not really. If we compare the Flash models to similar models from the same providers, we see a bit less of a lead.

The Gemini models still have a 50% higher throughput and the latency is still improved as well. But the cost difference starts to get way closer. In general, you can think of Gemini 3.6 Flash as being comparable to Claude Sonnet and OpenAI’s Terra. Gemini 3.5 Flash Lite should probably be compared to Claude Haiku and OpenAI’s Luna. So, we still see faster latency and higher throughput.

But when does latency and throughput matter? If we aren’t going to use Gemini Flash models for coding, what should we be using them for. Think about their ability to summarize large documents. You might want to plug them into programmatic workflows via Langchain or other flow based tools. For Google, it makes sense that they would use them to provide AI summaries above your search results. If you expand your thinking a bit, you might find more use cases that fit your workflows. Not everything is about coding and not everything needs the absolute highest “intelligence” scores.

Latency and Throughput At a Cost

But, latency and throughput can’t be looked at in a vacuum. When people look at Artificial Analysis, they tend to stick to the intelligence scores. If you keep scrolling, you will see other graphs that show the amount of tokens used, per-task time averages, end-to-end timing, as well as price graphs.

The heading of this section mentions cost, but that shouldn’t be taken literally as only meaning dollars spent. In this graph, we see that the token usage is remarkably high compared to the models from the other providers.

At the same “high” reasoning effort levels, Gemini Flash 3.6 consumes more tokens than the other flagship models and even more than much lower tier models from OpenAI. If the output tokens are inflated this much, we lose out on almost all of the gains from higher throughput. But how does token usage impact actual cost per task?

If we compare the cost to run each task, we at least see that 3.6 Flash is cheaper than Opus. But, it ends up more expensive than any OpenAI model. 3.5 Flash Lite maintains a pretty low per-task cost hitting the same price point as Luna. However, if you were to go back up to the top you would notice that Luna is still in a significant lead when it comes to intelligence.

If token usage is high, that surely means that the speed of the models is impacted, right? We can look at the per-task time values and see that Flash Lite is indeed fast compared to other models.

Flash Lite is fast per task while 3.6 Flash loses all of its throughput gains because of the token usage. The thing to remember is that this measurement is excluding the TTFT (time to first token) as well as overhead time. What if we add in those values? We can do that by looking at the end-to-end measurement, we see the time to output 500 tokens, including reasoning model ‘thinking’ time as well as TTFT.

Flash Lite is right around the Luna level and 3.6 Flash is in a similar ranking as it had in the per-task timing. Inexplicably, Terra is the fastest as a mid tier model.

Thinking in 2d

The previous graphs have been thinking in a single dimension/measurement, which has value but doesn’t paint a full picture. If we want to consider these things more fairly, we can map the results in a 2d graph. This first graph shows the intelligence vs speed index and we see Flash Lite as being the closest to the “optimal quadrant”.

Another thing we can look at is cost vs speed. When we do so, we see Flash Lite getting closest to optimal once again.

If we only care about one metric, Gemini models tend to be a poor choice most of the time, but these 2d graphs give use a bit more perspective.

Considering Micro Benchmarks

Here at GitKraken, we created GitBench to try and see which models are better at doing git related tasks. This micro benchmark paints a wildly different picture when we think about what intelligence means. We can see Google models being 6 of the top 10 models when it comes to their pass rate on git focused benchmarks. Gemma clocking in at #6 as an open-weight model, might be the most surprising part.

Circling back to the throughput and TPS of Gemini models we can do a 2d comparison against Gitbench pass rate and see that Gemini models are 3 of the top 6 for that level of efficiency.

Wrapping Up

Most of this post has been purely observational. But, I think by the end we have shown that the Gemini models probably don’t deserve the amount of hate and derision we see on social media. In our own micro benchmarks we see them perform admirably. It would be a meme at this point to say that people are “holding it wrong” in their critiques, but I think we need to assess every model in ways that matter the most to us. If predictions are correct and subsidized tokens are going to end, we need to be thinking about more and more about the ratios between intelligence, cost, and speed depending most upon the task we are trying to accomplish.