Toronto’s Untether straps in for growth selling AI chips - but can it avoid getting crushed by Nvidia?

Open this photo in gallery:

Toronto semi-conductor startup Untether AI Corp. is gearing up for a big push into the AI market with the launch of a new chip, backed by big-name investors and a strategy to steer clear of leviathan Nvidia Corp.’s NVDA-Q key business.

Untether says its second-generation chips can deliver vastly better performance than their competition, and has announced key partnerships with General Motors Co. GM-N and Arm Holdings PLC. ARM-Q But the technology still has to be proven in the market.

The company in January appointed former Intel Corp. INTC-Q senior executive Chris Walker as chief executive and brought on Google’s former head of silicon, Amir Salek, as senior technical adviser.

It’s also raising another nine-figure financing after landing $125-million in a 2021 venture capital deal that was backed by Intel Capital, Tracker Capital, Radical Ventures, GM Ventures and Canada Pension Plan Investment Board. That financing marked one of the few direct investments in a Canadian startup by CPPIB, the country’s largest institutional investor.

The investment is one of the largest for Radical, which has established itself as one of Canada’s most heavily financed venture capital firms, with key stakes in the country’s most promising AI companies including Waabi and Cohere.

Untether “is at an inflection point for growth,” said Mr. Walker, who runs the 170-person company from Silicon Valley with a senior leadership team split between Toronto and California. “All people have to do is look at Nvidia’s earnings to know there is a need for alternative solutions to meet demand or for people that want a different approach.”

The AI boom has created a bonanza for makers of sophisticated chips that power deep learning. Nvidia has become one of the world’s most valuable companies owing to massive demand for its graphic processing units, which are typically housed in data centres and used to train large language models and other AI systems.

But some argue there is a much bigger opportunity in powering inference – the process of using AI in a machine or application after the model has been trained. An autonomous vehicle or other AI-enabled equipment will be used repeatedly, whereas training a model is more of a one-time event, with start and end dates. Inference will amount to a US$100-billion market, Mr. Walker said.

That’s where Untether comes in. It makes inference chips it says are far more efficient and faster than what’s available today for guiding vision-based systems including robots, autonomous vehicles, tractors and drones.

But Untether is competing against many other upstarts plus giant incumbents, including Nvidia and Arm. For prospective customers to take a chance, Untether must deliver a chip that is orders of magnitude better than what competitors offer.

“Nvidia will still be the dominant training platform,” Radical co-founder Tomi Poutanen said. “But for running models efficiently, Untether has a much more effective product.”

Inference is a “very fragmented market, which is a great thing for a small company because you have multiple entry points,” Mr. Salek said.

While some semi-conductor startups are going after both training and inference, Mr. Salek said he was drawn to Untether’s “very focused” sales and marketing strategy to stick to inference, where Nvidia isn’t as dominant. “Training has so far been the shiny object because that’s where Nvidia is making a ton of money. But going head-to-head against Nvidia is not the right recipe for success for a small company.”

The growth in AI-enabled devices could also lift the fortunes of Altera, a California company recently spun out of Intel that employs 200-plus people in Toronto.

Altera specializes in a type of chip that can be reprogrammed to perform a variety of functions, including accelerating computationally intensive tasks, such as in AI. This type of chip is embedded in a range of devices across industries including aerospace and defence, automotive, communications and industrial automation.

“Every single segment that we operate in, we have our customers telling us that in the next three to five years, up to 80 per cent of them are looking to introduce more AI capability in computing platforms,” Altera CEO Sandra Rivera said in an interview. “A larger part of the growth is happening on the inference side over the next five years,” she said. Altera estimates the inference market could grow by around 40 per cent, compared with 25 per cent for training.

Untether is one of several Canadian semi-conductor companies fuelling a revival in the sector here. It grew out of the research of co-founder Martin Snelgrove, who earned a PhD in electrical engineering from the University of Toronto and taught there for about a decade. He has served as both CEO and CTO at Untether and remains a consultant.

For years, the dominant approach to chip-making has followed a framework outlined by Hungarian-American mathematician and physicist John von Neumann in 1945. The problem was that design wastes a lot of energy shuttling data around.

Untether has instead developed a new architecture that cuts the distance data must travel by placing memory and processing units side-by-side on the hardware.

That speeds up the processing required for AI, reducing the amount of data movement by a factor of six and boosting computational efficiency by up to nine times, resulting in more economical energy use. (The chip itself is manufactured by Taiwan Semiconductor Manufacturing Co., the world’s largest chip fabricator.)

Untether’s first chip, introduced in 2021, was intended to prove its utility among early adopters. It is working with GM on perception systems for autonomous vehicles and providing AI capabilities to Arm to help make products for advanced driver assistance systems.

“This company,” Mr. Poutanen said, “is about to demonstrate a new way of doing deep learning inference that’s measurably more efficient than existing ways of doing it, an innovation that has been incubated here in Canada and taken years to get to this place.”