AMD Buys Taalas for AI Chip Technology

AMD announced it will acquire Taalas, a company that specializes in model-specific AI inference chips. These chips are designed to run a single model, rather than being programmable to run multiple models like other AI inference chips on the market.

Taalas’ approach is to “burn” the model into CMOS, rather than loading model weights from memory and using programmable portions of chips to handle specific matrix and compute needs. This approach can result in significant performance gains compared to more programmable solutions.

Model-Specific Chips

The basic concept behind Taalas is that each chip is designed to run a specific model, and if you want to change the model being run, you need to change the chip. This is in contrast to more programmable chips that can run multiple models.

Taalas’ current-generation hardware is the HC1 technology demonstrator, which is a 6nm chip with an 815 square millimeter die and 53 billion transistors. The company claims that the HC1 can run Llama 3.1 8B at up to 17,000 tokens per second per user.

Performance Comparison

Taalas compares the HC1 to other chips on the market, including the Nvidia H200 and B200, as well as chips from Groq, SambaNova, and Cerebras. However, it’s worth noting that these comparisons are based on Taalas’ own measurements.

One of the challenges of using model-specific chips is that they can be difficult to manufacture and test. If a single chip is used to run a model, and that chip needs to be changed, it can be a complex and time-consuming process. However, Taalas says that changing weights, matrix dimensions, and other important bits only requires changing two mask layers, which can simplify the process.

A model-specific chip trades flexibility for efficiency, which can be beneficial for stable, high-volume inference workloads. This approach is most attractive for applications where a single model is used for an extended period.

AMD’s Plans for Taalas

AMD plans to fold Taalas’ technology into its accelerator roadmap and build system-level products around its Instinct accelerators. This move strengthens AMD’s full-stack AI platform, which also includes Helios rackscale systems, EPYC CPUs, and ROCm software. Having an in-house model-specific inference engine gives AMD another option for customers who want maximum efficiency with a single stable model.

In practice, this development means that companies using AMD’s AI platform may be able to achieve faster and more efficient inference workloads, which can be particularly beneficial for applications where speed and accuracy are critical. The acquisition of Taalas is a significant move for AMD, and it will be interesting to see how quickly the company can integrate Taalas’ technology into its products and bring them to market.

The acquisition is a big move for AMD.