Last Updated: 11 December 2024
We know the internet depends on fiber optic cables to move data at high speed over long distances. But once that information reaches a data center, it often slows down because the last steps are still handled by copper wires. For complex, AI-driven tasks—like training giant generative AI models—this slowdown can waste time, money, and a lot of energy. IBM says it has found a new way to fix this problem.
Instead of relying on slower electrical signals, IBM’s new co-packaged optics (CPO) technology uses beams of light to connect crucial components inside a data center.
By designing and assembling the first publicly announced polymer optical waveguide (PWG) to make this possible, IBM wants to change how chips, circuit boards, and servers communicate with each other. Think of it as replacing a series of outdated local roads with multi-lane highways of light—speeding up traffic and cutting down on wasted fuel (in this case, energy).
Right now, powerful graphics processing units (GPUs) inside data centers often spend more than half their time waiting around for data. That’s because the information they need has to crawl along copper wires before it reaches them. This waiting game is a big deal if you’re training large AI models—think the kind that handle advanced language tasks or make complex predictions—because every minute counts and energy use adds up fast.
IBM’s new approach aims to bring fiber-like speed and efficiency right inside the data center. With optical connections, GPUs could get the data they need sooner, reducing their idle time.
The end result? Faster training of big AI models, less energy wasted, and—over time—significant cost savings. It’s the difference between taking hours to get something done versus finishing it in a fraction of the time, all while using fewer resources.
IBM’s latest research suggests that this optical approach could cut power consumption for data transmission by more than five times compared to typical electrical interconnects. It could also stretch the reach of data cables from just one meter to hundreds of meters without sacrificing speed.
The upshot is that developers could train large language models five times faster than before. Imagine shortening a three-month training process down to three weeks—freeing up talent and resources for other creative tasks.
From an energy standpoint, IBM claims these improvements could save as much electricity as what 5,000 U.S. homes use in a year for each AI model trained. That’s not only good for the bottom line but also a win for anyone concerned about the environmental footprint of growing AI demands.
“With this breakthrough, tomorrow’s chips will communicate much like how fiber optics cables carry data in and out of data centers, ushering in a new era of faster, more sustainable communications that can handle the AI workloads of the future.”
Dario Gil, SVP and Director of Research at IBM
These words point to a bigger idea: if AI keeps expanding, the underlying infrastructure must adapt. IBM’s approach gives decision-makers one more tool in figuring out how to scale AI effectively without pushing energy consumption and costs through the roof.
The push for optical connections inside the data center also opens the door to huge increases in data bandwidth—up to 80 times more than what current electrical links offer between chips. This bigger “data pipeline” means devices can talk to each other faster, handle more tasks at once, and potentially deliver insights sooner. By combining multiple wavelengths of light into a single channel, and adding more fibers at once, IBM’s design aims to break past the limits that have slowed down electronics for years.
Ultimately, IBM’s co-packaged optics is one step toward making data centers more efficient, powerful, and ready for tomorrow’s AI challenges.
It’s not a one-size-fits-all solution, but rather a new tool that could help the industry strike a smarter balance between speed, cost, and energy use.
As the world relies more on AI, having this option on the table could allow companies and organizations to scale their capabilities without paying a sky-high price in power bills or hitting performance roadblocks.
In other words, IBM is working to clear the path, so data and AI can travel at the speed of light—literally—inside the heart of the data center.
