A hunk of material bustles with electrons, one tickling another as they bop around. Quantifying how one particle jostles others in that scrum is so complicated that, beginning in the 1990s, physicists developed an esoteric mathematical structure called a tensor network just to describe it. A decade or so later, when quantum physicist Román Orús began studying tensor networks, he didn’t envision applying them to the seemingly unrelated concepts of artificial intelligence.
But with the advent of enormous, energy-hogging large language models like those behind ChatGPT, “we realized that by using tensor networks we could address some of the bottlenecks,” says Orús, of Donostia International Physics Center in San Sebastián, Spain. Tensor networks can help squish bloated AI models down to a more manageable size, cutting energy use and improving efficiency without sacrificing accuracy. That’s Orús’ aim in his work at Multiverse Computing, a startup he cofounded. It’s an appealing prospect: AI currently gobbles so much energy that tech companies are hatching plans for a future generation of small nuclear power plants. And the need to power AI data centers may already be helping to drive up electricity costs in some areas.
Smaller models also boast the potential to be crammed onto personal devices like cell phones or household appliances. The ability to put AI on the devices themselves — rather than running it through the cloud — means users wouldn’t need an internet connection to use the AI.
There are other ways to compress AI models. But tensor network proponents argue that the technique’s basis in physics and math can provide more of a guarantee that the compressed model will perform as well as — or even better than — its big sibling. “It seems like kind of a slam dunk every time people try it,” says physicist and tensor network enthusiast Miles Stoudenmire of the Flatiron Institute in New York City.
But Stoudenmire wants to push tensor networks even further.
To read more, click here.