Artificial intelligence (AI) is changing the IT landscape in radical, unprecedented ways. Organizations are rewriting the rules of code generation, automating complex customer service interactions, and extracting deep data insights that were impossible to uncover just a few years ago. However, for technology specialists and IT leaders responsible for keeping the lights on, artificial intelligence represents a massive shift in infrastructure requirements.
As I’ve mentioned in my previous blog posts, you can’t build a reliable, secure AI strategy on legacy infrastructure that was not built for high-density, GPU-accelerated workloads. The conversation is quickly shifting from the software layer to the bedrock of your data center. Most conversations about artificial intelligence focus on the models themselves, such as large language models (LLMs) and neural networks. Yet the primary reason AI projects stall is a lack of suitable infrastructure.
Broadcom recently introduced VMware Cloud Foundation 9 (VCF 9), which incorporates groundbreaking technology designed specifically to support artificial intelligence. This release signals a pivotal moment for private cloud infrastructure. It moves LLMs and generative AI from hardware-intensive experiments into scalable, manageable standard resources.
For IT professionals, the challenge is adopting these capabilities without incurring massive capital expenditures or introducing unmanageable complexity.
Breaking the proof-of-concept loop
Most organizations experimenting with LLMs find themselves stuck in a proof-of-concept loop. Models live on isolated servers or developer laptops, data pipelines are brittle, and teams don’t have a standardized way to govern, version, or securely expose models to applications. This disjointed approach creates shadow AI, unclear ownership, and severe risks around security, data privacy, cost overruns, and inconsistent behavior across business units.
Public AI models are fantastic for general tasks, but most organizations are uncomfortable sending their code, customer data, or financial records into a public model.
Training AI models on public cloud platforms increases the risk of accidentally exposing sensitive proprietary data, which can create serious privacy, compliance, and intellectual property issues.
Beyond privacy, selecting the right LLM is critical for meeting specific use cases and compliance goals. The AI landscape is evolving quickly, with constant introductions of new vendors and components. This complexity drives up costs and presents significant performance challenges.
To bridge this infrastructure gap, you need a platform that treats AI resources more like traditional virtualized resources. This means adopting pooled capacity, implementing strict policy controls, and establishing clear governance instead of relying on one-off hardware silos.
VCF 9 as an AI-native platform
VCF 9 fundamentally transforms private cloud into a unified platform where AI workloads can coexist with traditional enterprise applications, managed by the tools you use today. Broadcom makes private AI services a standard part of VCF 9, effectively turning it into an AI-native private cloud platform.
This sophisticated infrastructure approach aligns the competitive advantages of AI with strict privacy and compliance requirements that are essential for modern organizations. VCF 9 allows you to bring the AI model to your data, instead of sending your sensitive data to a public AI service.
Through VMware Private AI Foundation with NVIDIA, organizations can fine-tune LLMs, deploy retrieval-augmented generation workflows, and run inference workloads completely within their secure data centers. This joint platform combines innovations from both Broadcom and NVIDIA to unleash productivity with a much lower total cost of ownership (TCO).
VCF 9 key capabilities for enterprise AI workloads
Deploying an AI model is not a simple task. Models are massive files that need to be versioned, secured, and distributed efficiently.
VCF 9, combined with VMware Private AI Services on VMware Private AI Foundation with NVIDIA, introduces several powerful capabilities to streamline this process.
Model Store and Model Runtime
A key feature is the way VCF 9 combines a Harbor-based Model Store for AI artifacts with vSphere content libraries for deep learning virtual machine images. In the past, models were scattered across the network. Now, you can govern models and AI images with the same rigor you apply to traditional templates. They are versioned, approved, and delivered from a central, controlled repository.
The Model Runtime service handles the versioning, deployment, and scaling of these models as shared services. It allows data scientists to create and manage model endpoints for their applications easily. These endpoints simplify the model instance, allowing users to send inputs and receive outputs without needing to understand the underlying mechanics. This approach also lets you scale deployment by efficiently handling many different concurrent requests.
Agent Builder and integrated data services
AI agents are autonomous software entities that use AI techniques to perceive environments, make decisions, and achieve specific goals. The Agent Builder service allows application developers to create these AI agents using the Model Store and Model Runtime.
VCF 9 with Private AI Services also features a Data Indexing and Retrieval Service, which lets enterprises chunk and index private data sources like PDFs, CSV files, and internal wiki pages. This service vectorizes the data and makes it available through specialized knowledge bases. As your proprietary data changes, these knowledge bases update on a schedule or on demand, ensuring your generative AI applications always access the most current information.
Kubernetes blueprints for AI
Modern AI platforms are increasingly containerized. Whether you are running training jobs or deploying inference endpoints, Kubernetes is the control plane of choice alongside virtual machines. As I explained in my last post, managing Kubernetes has historically been a complex task.
VCF 9 deepens VMware integration with the vSphere Kubernetes Service. This brings Kubernetes management directly into the interface that IT professionals already know and trust. VCF Automation ships with blueprints and deep learning images tailored for AI, including workloads that stand up Kubernetes clusters, GPU operators, and ready-to-use stacks with automated cloud initialization. Your teams can use Kubernetes as the control plane for distributed training and inference, keeping everything under a unified management umbrella.
GPU as a Service
One of the most significant hurdles in enterprise AI is the cost and management of graphics processing units. In a physical environment, a GPU is often tied to a single server. VCF 9 fundamentally changes how enterprises consume this capacity, as I also highlighted in another post.
Powered by VMware Private AI Foundation with NVIDIA, VCF 9 makes GPU as a Service a governed platform capability. IT teams can virtualize GPU resources, publish them as catalog-driven services, and allocate them on demand to different teams. This dramatically improves utilization across the organization while maintaining centralized control. Additional capabilities monitor GPU slowdown temperatures, memory usage, and compute utilization to optimize performance and prevent hardware damage.
Performance and TCO
A common concern when virtualizing intensive workloads is the potential for performance degradation. But putting AI workloads on virtualized solutions preserves performance while adding the benefits of virtualization, such as ease of management and enterprise-grade security. This is particularly crucial for training and running LLMs, which demand incredible compute resources.
Independent and VMware-published benchmarks show that AI and ML workloads can achieve near bare-metal performance on VCF 9, while gaining virtualization benefits like simplified operations and stronger security. Combined with per-core subscription pricing instead of opaque, token-based LLM billing, this can deliver materially lower and far more predictable TCO than public-cloud LLM services or bespoke bare-metal stacks.
The combination of resource sharing, a unified architecture, and consistent operations results in significantly lower TCO. VMware relies on a per-core pricing model, steering clear of the unpredictable token-based billing models used by most cloud providers for LLM services. The platform achieves up to 90 percent lower TCO compared to public clouds, and up to 29 percent lower compared to bare metal solutions.
This means your IT team can deploy private and independent AI environments and dedicated LLMs for many different tenants or business units. Each organization operates within a dedicated, isolated environment, ensuring security and autonomy while optimizing costs across the board.
Security and compliance
AI and LLMs introduce new attack vectors and compliance challenges. Many businesses don’t want to use public LLMs because of data privacy and intellectual property concerns. If you’re running AI on-premises or in a private cloud to protect your data, the underlying infrastructure has to be a fortress.
With VCF 9, organizations benefit from air-gapped support. VMware Private AI Foundation with NVIDIA can now be deployed in fully air-gapped environments. This ensures data confidentiality and isolation for critical workloads, including proprietary data used to fine-tune your LLMs. Air-gapping the most sensitive assets slashes cyber-risk exposure, maintains strict environmental compliance, and safeguards your revenue and reputation.
Modernize your infrastructure for the AI era with VCF 9
The introduction of VCF 9 is a clear signal that the future of IT is intertwined with artificial intelligence. GPU as a Service, managing Kubernetes natively, and governing AI models efficiently will soon be table stakes for enterprise IT. But technology is only as good as the platform it runs on.
As you navigate your modernization journey, you have to consider how you’ll power, secure, and scale these workloads. Procuring, then racking and stacking high-performance servers can take months. And this shouldn’t be a bottleneck that stifles innovation.
Partner with 11:11 as you build your AI strategy. With our VMware-based solutions, you can be sure that the infrastructure supporting your AI workloads meets the rigorous standards of your business, and your industry. Security, performance, and true resilience are embedded in our DNA.
Additional Resources:
Unify Kubernetes, VMs, and AI with VCF 9
Unlock AI with GPU as a Service in VCF 9
VCF 9, Infrastructure, and the AI Revolution
