Nordic State of AI - Central Observations 4/5: AI Infrastructure

Adopting AI and scaling its use requires several new capabilities from organizations, starting with a systematic approach to data use, new management practices, and new types of infrastructure. The Nordic State of AI report highlights the transformative power of functional AI infrastructure. As a rapidly evolving concept, AI infrastructure is not just an investment but a catalyst for scaling AI, disseminating related knowledge among various stakeholders, and ensuring operational readiness.

In this five-part blog series, we will dive into the central observations of the Nordic State of AI report, an annual report examining AI adoption across leading Nordic multinational companies by surveying their current use of AI. In this fourth part, we will explore companies' challenges and opportunities regarding AI infrastructure – what it stands for and the role and importance of the right AI infrastructure choices.

The backbone of AI: understanding and implementing effective AI infrastructure

At its core, AI infrastructure comprises the various tools, resources, and systems required to build, run, and maintain AI models. This encompasses on-premise hardware, private and public cloud services, high-performance computing resources such as supercomputers, exclusive datasets, and commercial and open-source development tools.

Every part of AI infrastructure is crucial. On-premise hardware and private clouds provide control and security essential for industries dealing with sensitive data. Public cloud services offer scalability and flexibility, enabling organizations to manage large-scale AI workloads without significant upfront investments. Supercomputers and specialized hardware, often publicly funded, support the training of complex models that require immense computational power.

The strategic importance of AI infrastructure investments

Investing in AI infrastructure is more than just acquiring the latest technology. It involves strategically building a robust foundation to support long-term AI initiatives. Our most recent Nordic State of AI report shows that 57% of companies invest in AI infrastructure, recognizing its strategic importance in scaling AI and ensuring operational readiness.

Companies’ investments in AI Infrastructure have notably increased within the previous year.

Improving success with AI through evaluation frameworks and well-built infrastructure

Slightly over a quarter of Nordic State of AI survey respondents have a framework in place to evaluate the success of AI projects in terms of ROI or other metrics. Notably, 37,5% of those who are satisfied with the outcomes of their AI experiments and projects have a framework, while only 15.7% of those who are neutral at best have one.

As companies and organizations mature in using AI, we expect the satisfaction rate to increase. 

Satisfaction will be achieved through carefully considered investments in AI infrastructure, which, over time, will lower the investments needed on a project basis. By establishing robust data practices and scalable computational resources, companies can avoid repetitive investments for individual projects. Instead, they can leverage a well-built infrastructure to quickly adapt and implement new AI solutions, increasing their efficiency and return on investment.

Overcoming challenges in scaling AI with effective AI infrastructure

Scaling AI across an organization often presents challenges such as a lack of talent, fragmented data practices, and unclear business strategies, which can hinder progress.

Effective AI infrastructure helps mitigate these issues by providing a cohesive framework within which AI development can thrive. A solid infrastructure supports seamless data integration, robust model training, and efficient deployment, crucial for transforming AI from isolated projects to enterprise-wide capabilities.

The lack of scalable infrastructure remains one of the biggest challenges in scaling AI use across the company.

Choosing the right AI infrastructure

The choice of AI infrastructure heavily depends on the specific needs of an organization and the nature of the AI applications they build. Companies must consider intellectual property, budget constraints, and the complexity of the AI models they plan to deploy.

For instance, a company focusing on developing proprietary AI models for competitive advantage might heavily invest in on-premise hardware and private clouds. On the other hand, those who leverage existing AI tools for productivity gains might find public cloud services more cost-effective. Additionally, organizations can benefit from publicly funded resources like supercomputers, which provide the computational power required for training large-scale AI models without the high internal cost of building such infrastructure.

AI infrastructure: The LLM example

For most companies, the best path will be a combination of private and public cloud, on-premise for certain use cases, proprietary models, and existing AI tools. Generative AI and large language models (LLMs) have come to play a big role in infrastructure discussions over the last few years. 

Similar to other AI technologies, questions arise regarding how companies should incorporate these models into their infrastructure choices. Should companies use available general-purpose models, host their own proprietary LLMs, or acquire LLMs through SaaS?

Using general-purpose LLMs leaves companies with little control over the models, high costs, and minimal differentiation, as many competitors will use the same models. Hosting proprietary LLMs offers more flexibility and the ability to customize the models but is a very resource-intensive option. With LLMs through SaaS, companies can own their model and related data IPs while focusing on their core expertise and outsourcing the model operation. These LLMs can be built on top of existing open source base models, allowing companies full control, transparency, and flexibility. Silo AI offers models, tools, and expertise enabling LLMs through SaaS.

Flexibility and futureproofing

Rapid advancements and continuous innovation characterize the AI landscape, and AI infrastructure is an essential part of the AI value chain. Companies should adopt modular architectures that easily integrate new technologies and methods. This flexibility ensures the infrastructure can evolve alongside AI advancements, maintaining its relevance and effectiveness.

By strategically investing in robust, flexible, and scalable infrastructure, companies can unlock AI's full potential and drive significant value in their industries. Companies that recognize and act on the growing need for a solid AI infrastructure will be well-positioned to lead in the AI-driven future. By focusing on the foundational aspects of AI infrastructure, organizations can ensure they are not just keeping pace with technological advancements but are at the forefront, driving innovation and achieving sustained success.