CONTRIBUTOR
Chief Content Officer,
Techstrong Group

Synopsis

In this Digital CxO Insights Leadership video, Mike Vizard interviews Wallaroo CEO Vid Jain about how and why The U.S. Space Force is investing in AI.

 

Transcript

Mike Vizard: Hello, and welcome to the latest edition of the Digital CxO Leadership Insights series. I’m your host Mike Vizard. Today we’re with Vid Jain who is the CEO for Wallaroo. And we’re talking about AI at the edge. And the ultimate use case at the edge is the space orbits that we’re going to start seeing a lot of use cases for AI. And Wallaroo is working with the US Space Force, and specifically their SpaceWorks arm, which is the innovation group within there, about how to deploy AI models way out at the edge. Vid, welcome to the show.

Vid Jain: Thank you. Thanks for having me onboard.

Mike Vizard: So what exactly are you doing with SpaceWorks in Space Force? And what’s the mission? And what’s the goal? Because how much can we get AI all the way out at the edge? What should people expect?

Vid Jain: Yes. So one way to think about the Space Force is, it is the edge, but it’s actually data in all sorts of different places, right? And so they have, obviously, things in space, they have land-based systems, which are communicating with devices, and space or running all sorts of other operations. They’ve got centralized coordinated, things going on via NASA, etc. So actually, the Space Force is a very modern example of a data factory, right? They, in fact, they have data being generated pretty much everywhere. And some of that data has implications for, you know, predictive maintenance, or like, being able to make sure that the devices and the infrastructure that the Space Force relies on is working correctly; doesn’t have any faults or isn’t about to fail. Some of it has to do with things like predicting, you know, future events, like looking at debris or stuff like that. And so, they have a lot of different use cases. And it’s really about working across all these different domains, centralized land based systems, and space based systems, and coordinating across all of those things. So it’s a very exciting opportunity. For us, it’s, it’s also going to be obviously very interesting and challenging.

Mike Vizard: One of the issues that people encounter a lot so far when they invest in AI is it’s hard to get that ROI. It’s hard to get it to work. There’s a lot of people involved with different cultures as data scientists, as DevOps teams, engineers of all given types. What do you guys see as the major challenges that people run into?

Vid Jain: Well, there’s – there’s a lot of challenges that kind of pile up, right? And obviously, before you even get to deploying a machine learning, there’s challenges around the data foundation; cleansing the data, etc. And I think the US military has made amazing progress over the last two years, when we started working with them. A couple of years ago, I think they were not nearly as far along in their journey. And they were still dealing with a lot of data foundation issues and data, quality and data access issues. A lot of those, not to say that all those issues have gone away, but most of those have been mitigated or a lot of them have been mitigated. The second issue is obviously around talent. And and building those models, right, building the data science teams and machine learning, engineering teams. That’s a hard thing to do sometimes, especially in this environment where there’s a shortage of those people. But I do think also over the last couple of years, in our interactions with with the US Air Force and Space Force, they’ve hired some amazing people, and they’ve gotten much stronger teams. So I think, you know, they’re on to the point where the challenges are now focused on what I would call that last mile, which is, how do you actually get that data and those machine learning models integrated into a production environment? And how do you iterate very quickly? How do you monitor how those applications are working those AI applications, and just make sure that they’re actually generating the outcomes that you’re hoping for? And that’s really that last mile of getting these models into production, optimizing them, monitoring them, getting them to the point of having an impact on the mission? That’s where we’re hoping to help the US Space Force

Mike Vizard: Do you think that organizations in general, underestimate how often those models need to be updated and may change over time? And I guess there’s new data sources that emerge? Or sometimes the algorithm itself just is out of touch with how the business process has changed or evolved? So do we need to kind of think more about what that workflow is going to look like?

Vid Jain: Absolutely. That’s one of the major issues is that people quite often think of deploying machine learning applications as though it was very similar to what they’re already used to, with, you know, standard applications. The problem is that the data is with it, right? So if the data was fixed for all time and was the same data over and over again, then that would work much better. But the problem is the data changes. So, if you think about the consumers, like, if you’re, if you’re a Walmart, with your shopping application, the consumer behavior changes; people start buying one day, they’re buying toilet paper, next day, they’re buying, who knows what? You know, one day, blue jeans are all the rage, the next day is green jeans. So that it’s not like the models to predict every possible thing and know how to deal with every possible thing there. They’re trained, and on a certain kind of data, a certain population of behavior, and actions. And so when when the incoming data no longer reflects, that models don’t work as well. And so the issue, that’s what the issue is big time. And the other issue is, is as you mentioned, you have new sources of data, and they need to be integrated; you have changes in the business itself, or the business processes that need to be taken care of. So it is a constant process, people need to get used to the fact that this is not a one time thing, this is an ongoing process that they need to monitor, manage update on a frequent basis. This is the reality of it.

Mike Vizard: Is there are also cultural issues at play here. Because we see, DevOps teams are updating applications a couple of times a week, and the data science guys are trying to build models. You know, it takes them a few months sometimes to get those together. And then I need to insert the model into the application at the production point. And they’re just not on the same cadence. So how do we kind of bring those folks together?

Vid Jain: Yeah, so you hit on a really important point. And, you know, a very good example of that would be like something like credit card fraud, right? Where the way that the attackers are trying to take advantage of the system changes quickly if you change it in a few weeks. And if it takes the DevOps team a month or two months to actually get a model into production, by the time it’s in production, it’s like looking at the wrong types of attacks, right? Is that going to be effective? So getting them aligned, and getting it to be much more frequent, is really critical. And I think part of the way that happens is, you’re right, it’s cultural, it’s education, it’s making sure that – one of the issues I see over and over again, in teams is that it’s not one team, it’s two separate teams that don’t really speak to each other very frequently. And so the data scientists is job is to build models, it’s not to operationalize them, and then they throw it over the wall. The DevOps team’s job is not to deliver business value, it’s just to take this thing and stick it somewhere in a production environment. And so because they’re not aligned to the overall goal of delivering business value, often that thing failed, because they’re not, they’re not going to talk to each other, working out or understanding what the process needs to be. So the first thing that needs to happen is is that those teams need to be aligned in a much stronger way, towards the overall goal of being able to rapidly deploy, monitor and iterate on the models, right? They, both the data scientists and the engineers and DevOps people, need to be aligned to that outcome. And once that alignment happens, then you can start designing processes, and try to start bringing in tooling that enables that to happen, but the most significant barriers are actually structural in most organizations.

Mike Vizard: As part of that whole conversation, there’s this ongoing debate between, do we need MLOps, or is essentially an AI model, just another type of software artifact that will be inserted into my GIT repository and managed alongside everything else? So you know, are these things really fundamentally different? Or over time is the AI model just going to be an artifact that we manage alongside everything else? And maybe that’s how we bring everybody together?

Vid Jain: Well, you do need tooling to be able to run AB tests, to be able to monitor how the models are doing. Right? So what you said just focuses on like the one part of that, which is really like how do I bring some code, some model code, Python code or some other model code into the standard kind of deployment practices that folks might be using for other applications? Unfortunately, that’s not sufficient to generate business value creation or mission outcomes or whatever; you have to have all the other tooling to be able to do rapid AB testing to be able to iterate on those models quickly, and to be able to understand if those models are actually generating the outcome they want. So it doesn’t matter how much you streamline the deployment of the machine learning code into production. If you don’t have the other tooling, you’re still basically flying blind. You could be having models in there that are losing you money or losing value or failing your mission, and you don’t know it.

Mike Vizard: Is there a smart way for Digital CxOs to kind of put all those processes together short of maybe just throwing everybody involved and locking them in a room and hoping that something good happens? Or is there a set of best practices that are starting to emerge that people should think about?

Vid Jain: Yeah, so I think the the whole thing here about how do I get to business value is in many places left an afterthought. Right? And so I think, the way I would start is by designing the teams by integrating the teams and by designing the processes of how you get from data to business value; starting with how am I going to get business value? Not in piecemeal pieces, like how am I going to get, you know, spend my data or I’m going to make a data lake or how am I going to provide this – I think it has to start with the fundamental rethinking of what what the process needs to be like and how the teams need to be integrated. The second thing is, you need as as if you’re running this process; it’s your mandate to do this at some company or some organization within DOD, then you need to think hard about what you want to build versus what you’re going to partner with vendors or other technologies to get done. Right? You cannot deal with everything you need, you need to be fundamentally focused on the things that you’re good at, that your teams are good at. And then you need to bring in partners and bring in tooling where it’s not going to be a lot of work for little return. Right? And so the other mistake that I see a lot of folks making is that they think they can, they can build everything from scratch, right? And this is why machine learning is hard. It requires a lot of different components working together carefully, and in a coordinated fashion. And if you think you’re going to make this build the entire stack, hire, everybody that has all those different skill sets, you’re talking about taking a very long time to get to your values; talking about huge investments, and you’re talking about high risk of pain now. But if you want to go down that path, that’s fine. But you need to make that decision on a conscious basis. Like, hey, not without knowing where you’re getting into. And then if you decide, look, you know what, I don’t need to build all this stuff, then you need to, you know, make sure that you’re partnering with and bringing in the right technologies, and you have a you have a strategic goal plan of how you get there. A lot of people are not doing this strategically; they’re doing it ad hoc piece at a time making it up as they go along. And then you find yourself in a bit of a mess.

Mike Vizard: Do you think people have reasonable expectations of what to expect from their AI investment or do they all think it’s all going to be instantaneous magic?

Vid Jain: It’s all over the place. It’s all over the place. I think people that have not done it before, think, “Oh, my competitor is doing it, or this group over here is doing it.” And they just hired these two data scientists. Let’s go do it. People that have done this for the last 10 years will tell you how hard it is. Right? We’ll I’ll tell you what’s involved, how hard it is, how long it takes. And if you’re doing it on industrial scale, right? If you’re doing little prototypes, and you got one use case and a little bit of data and a simple model, you can probably figure it out. But what we’re talking about here, which is the case with large enterprises, or with the US Air Force, or the Space Force, or other parts of the government is doing it industrial scale. Right? That’s the difference between baking three cupcakes, right? I can probably figure out how to bake three cupcakes versus like selling, you know, 10,000 cupcakes a day through my deal with Whole Foods. Right? Right. The US military, the US government needs to operate at that industrial scale. Right? And so are you going to build that factory? Are you building that AI factory all by yourself from scratch? That’s the question. You need to think whether you’re in the government or you’re in a large enterprise, right? What scale do you need to operate at? What tooling do you need to operate an industrial scale? Right? The cupcake analogy is great. You’re like, okay, can you build everything to crank out and keep track of 10,000 cupcakes a day, and make sure you’re earning a profit on it for a while? Is what you’re building only going to work with three cupcakes that you’re selling in your little car?

Mike Vizard: Alright, well, hopefully we don’t wind up with too many cooks in the kitchen as they say. Hey, Vid, thanks for being on the show.

Vid Jain: Okay, thanks for having us.

Mike Vizard: All right. Thank you all for watching this latest episode of the Digital CxO leadership insights series. You can find this episode on digitalcxo.com. We invite you to check out this one and all the other ones we have there, and hopefully we’ll see you again soon next time.

Vid Jain: Thanks, Michael.