AI Data Centers May Start Using Radio Instead of Copper

AI data centers are running up against the physical limits of copper for transmitting data, so radio is being considered as a replacement.

When data centers want to expand, they follow two strategies, scaling up or scaling out. Scaling out means linking more computers together to tackle a problem in chunks, while scaling up involves cramming as many GPUs together into the same computer as possible, combining their power to act as one, big GPU.

The two domains rely on two different physical connections. Scaling out mostly relies on photonic chips and optical fiber, which together can sling data hundreds or thousands of meters. Scaling up, which results in networks that are roughly 10 times as dense, is the domain of much simpler and less costly technology—copper cables that often span no more than a meter or two.

As you can imagine, these large combinations of GPUs require a lot of bandwidth between them in order to function, so much that copper can’t keep up.

As data centers continue to try and pack as much compute power as they possibly can, the throughput of copper wire isn’t enough anymore.

Copper cables are subject to a lot of factors that can cause issues with data transmission.

Copper can pick up interference from nearby electrical equipment, for example through electromagnetic interference or crosstalk. This is because copper wires act as antennas and can pick up signals around them: the longer the cable, the stronger the antenna. In order to combat this, you need more shielding or shorter cables.

Copper cables can also distort the signal as it travels through them. Over time, they become corroded and will need to be replaced to prevent signal distortion.

The final problem with copper is that they also give off electromagnetic radiation, which can be captured and decoded out of the air. As you can imagine, this can be an issue for a data center.

The solution that’s been utilized to solve this problem is fiber optic cables, which don’t suffer from the same issues. Typically these days, data centers use fiber optic cables to communicate between server racks. These cables can span tens or hundreds of meters in length.

Fiber optic cables work great for this purpose, but they have their issues. They’re power hungry, with data centers already using about 10% of their compute power just for fiber optic connections, according to Nvidia. They are finicky and can’t be bent very far before affecting the signal. They are very sensitive to temperature and they’re notoriously unreliable.

So, although companies are making fiber optic GPU interconnects available, they’re not without drawbacks.

AttoTude and 2Point are looking to replace these interconnects with radio instead, with waveguides that guide the waves to where they need to go. 2Point claims “3x Lower Power, 1000x Lower Latency, and 3x Lower Cost Than Optical Cabling. 10x Cable Reach at Similar Cost of Copper.” Quite impressive.

However, whenever new technology like this rolls out, you have to wonder about possible security issues. I couldn’t find any info on how effectively they’re able to prevent RF data leakage or if they even try to, and also how resilient they are to electromagnetic interference. There are standards such as NIST SP 800-53 and exist but they only cover protection against electromagnetic interference (EMI), not really against data leakage from RF.

RF shielding is already a concern for data centers, so shielding solutions are already available. It’s possible to pull a video signal wirelessly from HDMI, for example, and there‘s plenty of other examples of data leakage over RF that you can point to in electronics

The ICD 705 only covers frequencies between 10 kHz and 10 GHz, which is smaller than the 70 GHz frequency of the new interconnect cables.

As more and more of our life happens in the cloud and through cloud-based AI models, hopefully there will be stricter requirements for RF shielding in data centers. This also highlights the importance of E2EE: there are so many possible vectors of data leakage that we don’t even think about normally. Encrypting data so that it never touches any server in the clear protects not just from malicious service providers but from accidental data leakage as well.

While it’s not performant enough yet for prime time, homomorphic encryption could provide similar E2EE protections for cloud-based AI models in the future.