Here’s my view: lets assume the above is true and the average serve with a delay of 6-9 months. Given that, its fair to expect them to start serving DS flash v4, glm 5.3, mimo 2.6 pro or kimi 3.1 pro in a matter of months. DS flash is already fast to the point I’ve became a bottleneck. But my point is: Cerebras should soon release better models that are not that far from frontier ones, at a fraction of cost and with multiples of the speed.

Just be patient and plan for the future.

Openrouter on nitro gets me there. Although the TPS is 5X less still.

I have tried GPT OSS 120b on cerebras and it flies. But it's mostly wrong.

They don't really show it here, but supposedly a Cerebras' chip is sub-divided into a 2D mesh of what they call PEs (cores). So data flow generally flows in two axis, and maintains consistent timings to its 4 neighboring cores. They show the layers flowing from left-to-right being streamed by PEs, but I wonder how true that ends up being: maybe the communications are easier to stream via diagonals, or some other shape that has better topology for data flow.

Then, even from die shots of CPUs and GPUs we often see that cores are tiled as squares, and those don't have the same cross-core connection and streaming requirement. Now, I assume that some parts of what we call a core can be moved around more freely while maintaining whatever power/heating/timing requirements we have, so perhaps it's easier to 'force' a square by moving those loose parts around. But why do we confine those to squares? Is super-core layout just something not worth investing time into, like how interconnects are timed, non-convex tilings, and distances to I/O?

And for streaming architectures like Cerebras, why aren't other types of tilings considered, something like a hexagon grid? This could have 3 inputs / 3 outputs, which could allow more complex data flow (?). Similarly, we could have 8 connections with longer diagonals in a square tiling, etc.

I guess, what are the tradeoffs in place such that so many chip designs seem to reuse square tilings both locally and globally?

The total number of concurrent requests represents the number of simultaneous requests moving through the pipeline at any point in time. Unlike a GPU batch, those requests don’t wait for one another to finish, they stream through the pipeline continuously.

What do they mean by “unlike a GPU batch”? It’s not a hardware limitation of GPUs right? It’s just the scaffolding to do this in software isn’t available right

For ultrafast latency, only activations need to be passed between the wafers, and this can be done via RDMA over Ethernet.

I believe it for decode but the article says nothing about how they make prefill fast on that interconnect story :/.

(though the agents also spend a lot of time in testing and building, which cannot be improved in speed a lot, I suppose)

Such a weird company with a terminally online on linkedin ceo, garbage devrel on X. Cant really be friends with CSPs coz everyone has their own chip, no consumer solution either, no neocloud would want to onboard them either coz its baggage no one wants to risk dev time/customer money with.