What Nutanix’s latest Enterprise Cloud Index tells MSPs about shadow AI, sovereignty, and the infrastructure shift ahead

Robert Dutt: Hello and welcome to In The Channel from ChannelBuzz.ca, bringing news and information to Canadian IT solution providers for 16 years now. I’m Robert Dutt, editor of ChannelBuzz.ca, and as always your host for the show.

If you’re an MSP, there’s a good chance your customers are already using AI tools that your team doesn’t know about. Nutanix recently released the 8th annual Enterprise Cloud Index, their big annual survey of how organizations are building and managing infrastructure. And this year, the data paints a picture that would be uncomfortable for anyone who thinks they’ve got a handle on where AI is running in their environment. Nearly 80% of respondents say they’ve encountered AI tools or agents deployed outside IT’s control. Data sovereignty has become a top priority, and containers are quietly becoming the default for new applications.

My guest today is Lee Caswell, Nutanix’s senior vice president of product and solutions marketing. Lee came to Nutanix from VMware, so he’s been watching the infrastructure market reshape itself from a vantage point that very few people have. We dig into what the survey data actually says, where the contradictions are, and what it means for MSPs and solution providers. Here’s our conversation.

Robert Dutt: Lee, thanks for taking the time.

Lee Caswell: Well, Robert, thank you.

Robert Dutt: You come to Nutanix from VMware, and your CEO now, Rajiv Ramaswami, he was the COO over there. Now you’re running this survey while the virtualization market is being reshaped by Broadcom’s changes. How does sitting where you sit now, having been kind of on both sides of that fence, shape how you look at this year’s data?

Lee Caswell: Well, I think it’s fascinating that for years, maybe 20 years, people just assumed that the underlying virtualization layer was fixed. That vSphere was well established, super product, exciting. A lot of people built their careers, frankly, on learning the ins and outs of vSphere. And to a lesser extent, some of the later add-on products. But the idea that the underlying virtualization layer has changed has, for the first time in years, had people reconsidering how they will build out their IT infrastructure for the next 10 years.

Robert Dutt: And we’ll circle back to that theme and that infrastructure theme a little later. But I wanted to dive in off the top into shadow AI, because it’s something that we’ve been talking about a fair bit on the podcast, and it’s something that a lot of partners are thinking about and trying to get their heads around how to deal with it. According to the survey, 79% of your respondents say they’re encountering AI tools or agents that are deployed outside the purview of IT. That’s a striking number. I’m curious, though, about the quality of the problem. Is this mostly folks who are using ChatGPT carelessly or without permission, or are we talking about the worst-case scenario of actual AI agents making business decisions willy-nilly without oversight?

Lee Caswell: Well, we’ve certainly seen some of those later examples, but I think the majority of this is rational decision-making on IT and developer teams. Thinking about the fact that AI infrastructure itself can be relatively expensive. GPUs, new servers, new hardware. You’re generally bringing new hardware into the mix to start with. And what customers have been doing is before they go and make their investment strategy, and particularly in on-prem environments, they’ve been trying things out in the cloud where you can rent infrastructure, you can basically start something up, spin it down. That’s kind of a classic test-dev model, by the way, not different from what we’ve experienced in the past.

And yet, when you look at how you’re going to deploy AI long-term with considerations around sovereignty and privacy, and particularly around predictable and lower costs, you start thinking about how you can take your on-prem infrastructure skills, which could include a data center but might also include the edge, and start thinking about how do you bring your already-strapped IT teams into this? And from a channel perspective, it’s how do you leverage some of the skills where people have been trained, particularly on virtualization. We’ll come back to this in just a minute. And basically apply this now into the new world of AI LLMs, AI hardware, and containerized infrastructure running on VMs.

Robert Dutt: So if I’m an MSP supporting that kind of mid-market client, the 200 to 1,000 seat kind of space, what does a practical response to shadow AI look like at this moment in time? Because, you know, “implement an AI governance framework,” that’s great in concept, but that’s the kind of consulting engagement that’s a little hard for a lot of MSPs to deliver.

Lee Caswell: Well, first off, you want to start thinking about what are the risks you’re trying to address. One is you want to look carefully at what LLMs your user base is actually using. One of the things that we’ve been able to do, for example, is have an audit trail, so you can look at who’s using DeepSeek, for example. Who’s using OpenAI? Who’s using some of the Llama 2, Llama 3 models, for example, or NVIDIA models? So the ability to go and look into the user base and get an assessment of that.

Secondly, you’re looking at how do you make sure you don’t have a runaway cost model? This was one of the risks in the early cloud days, you remember. You had users getting shocked with the amount of unplanned, unmanaged cloud costs. And so you’ve got this opportunity now to look at how do you manage a brand new metric of consumption, by the way, called a token. I defy you to find somebody who knows exactly how tokens are created and the like. That’s a very difficult challenge. If you can provide a predictable way to manage, monitor, and control the usage of tokens, we do that as a way to basically protect against runaway costs.

And then finally, the idea of sovereignty. So where is your data? Specifically, as you look at geopolitical considerations, we have, I think, a stunning finding that showed that 57% of our respondents said that they wanted their AI workloads to be within a sovereign country. Now, that doesn’t mean a single location necessarily, but it does show the concerns around where’s my data? Who can subpoena my data? Who’s got access to my data? And it may be, Robert, that the data model is more sensitive than the data itself, because the data model shows how you’re interpreting the data. And that’s actually a really interesting finding, I think, for a lot of folks, as AI takes hold so quickly.

Robert Dutt: And data sovereignty is an area that we want to drill down on. It’s an area that’s of key interest to our audience, obviously. You touch on the 57% number in terms of how customers want infrastructure in a single country. 80% say it’s a high priority. You wrote recently about what you called the “sovereign edge,” the idea that AI is forcing compute closer to data within sovereign boundaries. For a Canadian audience that’s been navigating this between different regulation at different levels, the US hyperscalers and the CLOUD Act, for years, what’s new here? Is this kind of validation that what they’re seeing is real, or is the ground really shifting here?

Lee Caswell: I think the sensitivity is a continuation of the trends that we’ve seen in the past. What’s changed is the understanding that in an AI world, data will be more distributed than it is today. And so imagine if you’re a hydro company, let’s say. And you’ve got different dams and facilities and hydro control points. These are distributed. They need to be able to run in a disconnected manner. You want to have AI applied locally. If you’re doing things around video processing, you don’t want to send all that data back to a central location.

And so the ability to have a distributed model where your data and apps are more distributed and yet be connected so that you can do patching, for example, day-two operations, security updates, and push those out to a distributed environment. Now the realization is sovereignty has grown in importance, and at the same time, my data and applications will be more distributed. That’s a double stressor for IT teams looking at how to maintain that control and let the agility of distributed operations continue on.

Robert Dutt: So are you seeing organizations redefine sovereignty in terms of operational control rather than just “the data lives here”? Because I think that distinction can matter pretty significantly for how MSPs ultimately architect their solutions and try to address this challenge.

Lee Caswell: Yeah, I think for MSPs, there’s a few important areas to think through. One is that customers who were looking for, let’s say, an infrastructure link are now looking for an AI dial tone. They’re expecting to have AI available, always on, no matter where they are, accessing it for their users. Because AI is quickly, as you can see from the data here, becoming a top corporate priority. So that’s one thing.

The second one is that the sovereignty means you need to make sure you’re controlling where is your data replicated to? Where does DR happen? How do you fail back within sovereign boundaries? Being able to establish that, something where the data services, something that you can establish or set as a differentiated capability, has been extremely important.

And then lastly, you start thinking about what about within the MSP? There’s a noisy neighbor issue, but there’s a nosy neighbor issue, which is how do I make sure that someone inside can’t cross boundaries internally in an MSP and look at your data being hosted in a common location? This is an area that you’re going to want to look carefully at multi-tenancy and how the infrastructure protects your data even when some of the infrastructure is shared across users.

Robert Dutt: So let’s shift and talk about containers, because I think that’s one of the areas that’s impactful but kind of hard for the audience to act on immediately. You have 87% increasing their containerization, 83% building new apps in containers. For MSPs who are still living in a virtual machine-centered world, which is probably a lot of them at this point in time, what’s the practical on-ramp? And honestly, how urgent is this? Do they have years to kind of figure this out and re-strategize, or is this a situation where if you’re not there, you’re already behind?

Lee Caswell: I think for many customers who are running traditional applications and let’s say they move from an owned data center into a service provider model, the idea is that the applications may not be changing as fast as the container world might have them think. However, what you’re seeing is that new applications are built with containers because developers benefit from running in containers.

What we’re finding though is most customers, the far majority of containers, are running in VMs. And they run in VMs because you’re able to now get the benefits of software agility – develop apps faster, eliminate testing dependencies, be able to run in distributed environments more quickly. Those benefits are married or matched with the resiliency of the underlying infrastructure so that individual components can fail. You can have day-two operations intact, and you’ve got integrated privacy and security and sovereignty.

The idea that you’re going to run these both – it turns out we allow customers to run containers depending on their use case. If they were going to run on a bare metal instance, they can. If they want to run in the public cloud on EKS, for example, they can run our container Kubernetes stack, take advantage of our orchestration capabilities, but they don’t have to. For many customers, the fastest path to adopting containers will be to run containers in VMs, very familiar to our users and to the service provider base.

What we’re encouraging them to get ready for is that even if they weren’t considering containers for traditional workloads, the fast adoption of AI workloads will bring a requirement for supporting containers. Think carefully around how do you leverage the training you already have, resilient infrastructure, all the things that our teams have been able to protect their downside, and still get access to the upside of new AI applications. We think running containers in VMs actually makes that the fastest path to container adoption.

Robert Dutt: On AI agents, the survey shows a great deal of optimism around them. The productivity gains, the new revenue streams, all of that. But you also note that, as we talked about before, 79% of organizations can’t quite figure out how to manage the tools their employees are already using. Can you walk me through that disconnect? How do you go from “we can’t govern what we have” to “let’s deploy autonomous agents”?

Lee Caswell: Yeah. Well, as you start thinking about what people have realized about AI, first, most customers have figured out that AI training will happen in the public cloud and that training requires huge investments, large power outlays that can only be taken on by the development of the models by the largest hyperscalers and some sovereign nations themselves. And so customers have been looking at, “I’m going to take models,” but then they quickly realize that the ability to have these models be useful in a particular company environment is dependent on having access to proprietary data.

Think of support. If you want to support a product, it’s not interesting to have support in a general sense. You want to have support for your products, things that you may not want to expose, internal documents that are proprietary and private to your specific company. So now what you’re doing is basically taking these models, giving them access to your private data. And now the idea is, “I’m going to be able to take that inferencing model,” which is what this is called. Taking inferencing means you can take advantage of a software platform that abstracts the new hardware that’s required, GPUs, and abstracts the different types of models that you may choose over time.

And so this is where you have these different LLMs. The ability to access those – we certify and validate the leading models so that they will run on the GPUs that are certified by our OEM partners. And so what we’re doing is taking out the risks. Effectively, what you do is leverage all the expertise you have for building an enterprise-level application today, and now be able to assimilate GPUs at the hardware layer and new LLMs at the software layer. And we’ll make it operate exactly the same as what you have today.

Robert Dutt: 65% say that their AI applications are running today via managed service providers. That’s a pretty validating number for our audience, except for maybe the few who are going to say, “Well, what about the other 35%?” But, you know, can’t please everyone every time. I want to push though on what running AI via MSP actually means in practice. Are we talking about infrastructure hosting, model development and management, governance and compliance? What’s the service that MSPs are actually delivering today versus what they should be thinking about building towards for the future as this evolves?

Lee Caswell: Yeah, I think the numbers overstate a little bit about how much training and skill building has actually happened already, because this would include things like SaaS-delivered services. And as you think of SaaS-delivered services like Copilot or ServiceNow or Salesforce, you’ll have AI-enhanced SaaS services that can be delivered by a service provider. What we’re anticipating and preparing service providers for is the idea that customers will, as they have private data to run their private models, be requiring dedicated equipment or provided services that give access to GPUs and LLMs that are beyond a SaaS-level model and now are actually specific applications for specific customer use case models.

Robert Dutt: We talked about shadow AI a little earlier. I’m curious, speaking of future states for MSPs, is there a world where the MSP becomes kind of the governed alternative to shadow AI? Essentially the sanctioned AI service layer? Because that seems like a bigger play and a little bit harder to get your head around, but a bigger opportunity than just, “Hey, we host applications on GPUs now as well as CPUs.”

Lee Caswell: I think so. And I think there’s a terrific both revenue and profit opportunity for service providers around this. First, there’s a services aspect of thinking about where do these applications run? Do they run in one location? Do they run across the hybrid cloud? So for anyone who’s working with cloud providers, how do I bridge this world out to this sovereign edge as we talked about? So that idea of how do I optimally locate applications, AI applications, and their associated data – that’s a very interesting workflow model to start with.

And then next up, I think, is the idea of, well, where and how do I maintain sovereignty within this model? Service providers have a terrific opportunity to say, “Here are the limits within which your data and applications can move. And I’m going to provide that and give you some audit capabilities to manage any compliance risks that you have.” So terrific opportunity, I think, for service providers to become, as you mentioned, that governed alternative.

And then finally, the idea that you would have a predictable cost model with tokens that allow you to share GPU resources means not just predictable, but lower cost than having an unpredictable model from the hyperscalers. We think this is actually a really compelling opportunity for service providers going forward.

Robert Dutt: Can’t let you go without asking this one directly. A lot of our audience is in the middle of evaluating their virtualization platforms because of what’s happened with Broadcom and VMware. Within the survey data, is there anything about how those infrastructure decisions intersect with AI and sovereignty, the things we’ve been talking about, that you’d like to share? Are organizations treating this transition and the AI buildout as separate projects, or do things start to connect in an overall infrastructure refresh rethink?

Lee Caswell: Well, I think some of the excitement from a service provider standpoint should be based on modeling or following what’s happening with the largest hyperscalers. I mean, you’re watching hyperscalers build out tens of billions of dollars of capital per month. We’ve never seen anything like this happen. And so that model, at a hyperscaler level, now what you’re thinking about is 82% from the survey of our respondents felt that their infrastructure was not fully ready for AI.

And so building this out – I called this an AI dial tone earlier. The idea that similar to how you remember, Robert, how when you went to hotels, when Wi-Fi came along, all of a sudden Wi-Fi became de rigueur. You had to have it. If it wasn’t fast enough, people knew right away and responded very quickly. My view is we’re going to have exactly the same response to having fast, secure, and managed AI dial tones, if you will, for AI workloads, where you can apply your custom data or your private data and do that quickly using skills that you already have.

For me, that means using a platform based on servers, based on certified GPUs, getting access to a changing set and world of LLMs. And being able to abstract both the hardware elements and the software elements means that you’re going to have customers be able to take all of the fast-changing AI world and bring it to their business problems more quickly.

Robert Dutt: Before we wrap, a couple of lightning round questions, if you will. If a Canadian MSP is listening to this and thinking, “Okay, I need to do something differently,” what’s kind of the one thing based on what this data is showing that you’d tell them to prioritize in the next 12 months in terms of transforming their business?

Lee Caswell: Yeah, I’d say number one is AI is coming. So prepare yourself. If you think you can get started nicely with small clusters, for example – one of the nice things about the Nutanix model is you can start small and grow from there. So start small, get a usable cluster ready for customers so they can try out how they can assimilate new GPU hardware, new AI LLMs. I think that’s essential.

Also, in the process, what will happen is they’ll get experience with this new world of containers without giving up their virtualization expertise. That’s an extremely important step. If you try and do everything at once, it can be a lot. There are competitive solutions that force you to go to a Kubernetes-oriented management model. That’s a step too far for most service providers. If you think now what you could do instead is leverage your familiar virtualization skills, bring in the containerization, and allow customers to get started on shared infrastructure with a predictable cost. That’s a winning strategy for providing an on-ramp to AI with the lowest risk and a fast uptake.

Robert Dutt: All right. And finally, so that the MSP audience can kind of keep an eye on what they need to keep in mind on the customer side, what’s the most dangerous assumption that you see IT leaders making right now about AI infrastructure?

Lee Caswell: I think the most concerning thing I see is customers who are racing to be AI fast without being AI smart. And we saw some of this in the early days of the cloud. We remember “cloud first” versus “cloud smart.” And what happened was you had blown-up costs, you had programs that weren’t successful. But I’d say the most important thing actually has nothing to do with the infrastructure itself. It has to do with corporate management making sure that the application of AI is tied to a specific business problem. That’s the most important element.

This is the thing I look for first. If you’re trying to solve an important business problem where you can ideally show that you can save money, generate more revenue, or do things more efficiently, those are the areas where you say AI is going to help here. Don’t just apply AI because it’s cool. Apply it because it’s going to solve a business problem, and you’ll find that you can actually move any infrastructure. We’ll bring that and make that work for you.

Robert Dutt: Once again, it all kind of flows back to business outcomes. That’s great advice. I love that. Lee, thanks so much for taking the time. I appreciate it.

Lee Caswell: Robert, I really appreciate it. Thank you.

Robert Dutt: There you have it. Lee Caswell from Nutanix on their 8th annual Enterprise Cloud Index. A couple of things I’d like to flag from that conversation. Lee’s distinction between a noisy neighbor and a nosy neighbor when it comes to multi-tenant environments and data sovereignty – that’s a framing worth sitting with if you’re thinking about how to position managed services around compliance. And his point about organizations racing to be AI fast without being AI smart – that’s one you can take directly to client conversations.

We’ll have a link to the full Enterprise Cloud Index report in the show notes, as well as a full transcript of the conversation. Tomorrow on the show, AWS Canada celebrates 20 years of the cloud. I sat down with Eric Gales to talk about what that milestone looks like from a Canadian perspective, and we’ll be back next Monday to catch you up on the headlines with In Case You Missed It. Between now and then, we’d invite you to subscribe to or follow the podcast in your podcast app of choice. And if it lets you, please do leave a review. Until next time, I’m Robert Dutt for ChannelBuzz.ca, and I’ll see you in the channel.