As artificial intelligence (AI) and related technology becomes increasingly prevalent, it’s impossible to avoid acknowledging the gaps in our current digital infrastructure capabilities. Few existing cloud deployments or data centers were designed with the density and latency requirements of AI and other high-performance computing applications in mind, so as investments and demands in this compute-intensive arena continue to skyrocket, IT leaders are running harder and faster into one major question: How do they adapt and expand infrastructure quickly, efficiently, and effectively?
Organizations across industries have announced ambitious AI roadmaps, but whether or not their infrastructure can support them in staying the course remains to be seen. In order to understand how organizations are responding to these challenges, Flexential surveyed 350 IT leaders at organizations with over 100 million USD in annual revenue and compiled their feedback into a 2024 State of AI Infrastructure Report. In this article, we’ll run through the key findings across four main areas of consideration: execution, organizational barriers, optimization, and sustainability.
Infrastructure Investments Are Integral to Executing AI Roadmaps
Nearly all survey respondents (99 percent) state their organization has a documented AI roadmap, and 59 percent are actively planning to increase infrastructure investments to account for increased AI workloads. While 53 percent of respondents are extremely confident in their organization’s ability to execute AI roadmaps, 46 percent express some level of doubt, and 36 percent worry their organization may be playing catch-up in building out AI capabilities.
One thing most respondents (93 percent) agree on: There’s a greater expectation that IT leaders minimize time-to-revenue for AI-driven IT infrastructure. IT leaders have a board-level mandate to invest significant resources in executing AI roadmaps, but this pressure to achieve speedy implementation and ROI clashes with the reality that significant infrastructure investments are required to support complex AI use cases.
Skills Gaps, Performance Issues, and Security Concerns Are Roadblocks to AI Progress
Multiple challenges impede organizations’ ability to execute their AI roadmaps. Almost all respondents (91 percent) report a staffing gap related to AI in the last 12 months, and more than half (53 percent) report skills gaps or staffing shortages related to the management of specialized computing infrastructure, which makes it difficult to meet the pressing need for high-density infrastructure. Simultaneously, 82 percent of respondents have encountered some kind of performance issue with their AI workloads (e.g., bandwidth shortages, unreliable connections, scaling difficulty, excessive latency), which in turn impacts overall AI efficiency and delays innovation and new product launches. Data privacy and security are also top of mind, as 95 percent of respondents believe more AI investment means more vulnerability to cyberthreats, and 40 percent aren’t confident their existing cybersecurity teams have the ability to protect AI applications and workloads.
In order to anticipate and navigate around roadblocks while meeting ambitious AI goals, most organizations will need to draw on third-party expertise and specialized infrastructure. Adopting flexible solutions is key as needs continue to change.
Optimization Is a Driving Factor in AI Workload Deployment
With security, cost efficiency, and application performance driving where AI workloads are being deployed, many organizations are leveraging colocation to meet AI infrastructure challenges; 60 percent have had to pull back AI applications and workloads from public cloud to private cloud, on-premise data center, or third-party data center locations. However, organizations could use colocation and other AI infrastructure optimization strategies more effectively. For example, only 24 percent of respondents are deploying the most GPUs at the edge, and 34 percent still have AI applications access data via public Internet, which increases latency and security vulnerabilities. Shifting to private connections would mitigate these risks, but better still would be a hybrid or multi-cloud approach, which offers the flexibility to integrate multiple network options and establish reliable connectivity.
Improving Data Center Sustainability Is Increasingly Critical
Like AI, sustainability is a board-level concern, and most respondents (97 percent) feel some level of pressure to improve IT sustainability. Particularly as AI workloads increase exponentially—another recent analysis predicts that by 2027, AI infrastructure could consume roughly the same amount of energy as Sweden uses in a year—increasing investment in energy-intensive AI infrastructure means increasing focus on improving data center sustainability. Currently 63 percent of respondents are not satisfied with their organization’s progress with IT sustainability, and 94 percent would pay a significant premium for better sustainability outcomes from third-party data centers or cloud vendors.
Experimentation Alone Is Not Enough
Unfortunately, all the enthusiasm and excitement in the world is not enough to drive the potential of AI forward into a reality. To harness the AI revolution’s full potential—reliably and successfully—organizations must level up their IT infrastructure to match their AI ambitions and vision. Based on the findings from this report, this process of reimagining infrastructure architecture includes exploring and embracing the complexity of AI infrastructure challenges by taking a more strategic, proactive approach to AI workload deployment informed by third-party expertise. Such an approach means:
- sourcing high-density compute capacity that can scale with AI workloads while flexibly integrating new tools and technologies as organizational needs evolve;
- securing the sensitive data processed in AI applications through robust security measures that extend throughout IT architecture and ensure compliance with ever-evolving privacy rules;
- leveraging software-led interconnection to support real-time collaboration across IT ecosystems and large, distributed workloads; and
- deploying liquid-cooling technologies and other specialized equipment and strategies to address sustainability priorities without sacrificing performance.
Whatever your business priorities with AI, planning and achieving them depends your organization’s abilities to make these core infrastructure investments. An experienced data center partner like Flexential can swiftly interconnect and optimize your IT infrastructure to deliver the high bandwidth, consistent throughput, and low-latency private connections you need to scale your most crucial AI workloads.
