Navigating the Path to DevOps Maturity in the Real World

Charlene:Good afternoon, good morning, or good evening, depending on where you are in the world. Welcome to today’s DevOps.com webinar, Navigating the Path to DevOps Maturity in the Real World, definitely a topic that I’m sure is near and dear to many of our hearts. Today’s webinar is hosted by CloudBees. Today, we’ve got some really great information that we’re going to be presenting, but first, before we get started, I do wanna go through a couple housekeeping items.

First of all, my name is Charlene O’Hanlon, I’m an editor at DevOps.com, and I am the moderator for today’s event. And, this event today is being recorded, so fret not if you miss part of it. The recording should be available in about 24 hours or so, so it will be on the DevOps.com website, so feel free to check back, maybe this time tomorrow, if you need to listen to the webinar again. Also, we will have the slides available to download at that point. And, we are going to be taking questions at the end of the webinar, so if you have a question for any of our panelists today, please feel free to send in your questions at any time, using the control panel that’s on your screen.

And, we will get to as many questions as we can, before the end of the webinar, but don’t wait. Please go ahead and get your questions in early. And, with that, we’ll go ahead and advance onto the next slide. As I said, today’s webinar title is, Navigating the Path to DevOps Maturity in the Real World, and then, today’s presenters include Michael Facemire, who’s VP and Principal Analyst at Forrester. Sanil Pillai, Director of Apexon Labs, and Brian Dawson, who is the DevOps Evangelist at CloudBees. Welcome to all of you, gentlemen. Thanks for joining me, today.

Multiple Speakers:Thank you.

Charlene:Michael, why don’t you go ahead, and we’ll start with you, if you could just kind of introduce yourself to the audience, and give us a little information about yourself?

Michael:Absolutely. So, thanks for joining today, and thanks for having me. As, so my name is Mike Facemire, I’m the analyst at Forrester, that covers digital experience delivery, and so that is web, mobile, IOT, connected devices. All these things that companies are using to build their digital experience. My background is, I’m a computer engineer, I’m a developer, I still write code. So, a lot of the challenges that we’ll talk about today, I’ve had the fortune or misfortune of living through myself.

Charlene:Great, thanks Mike. Sanil, tell us about yourself, please.

Sanil:Sure, good morning and good afternoon everyone. I am the director for Apexon Labs at Apexon. Apexon Labs is responsible for various innovation initiatives, for initiatives covering mobile, and especially around DevOps. So, my responsibility is to kind of lead those initiators, and also lead the DevOps practice at Apexon. And, my background is heavily in computer engineering, mobile and DevOps, and I couldn’t be more excited to be here today.

Charlene:Wonderful, thanks Sanil. And, Brian, please, tell us about yourself.

Brian:Hello, again, thanks everybody for joining. I’m Brian Dawson, I’m a DevOps evangelist with CloudBees. CloudBees is often known as the Enterprise Jenkins company. We are major contributors to the Jenkins Project, as well as we offer Enterprise-scale solutions for Jenkins. I come to this role in this webinar by way of years of software development, with a focus on optimizing the software development process, on the planes of technical process or people, and most recently, I’ve been a continuous integration, continuous delivery and DevOps consultant.

Charlene:Thank you very much. Okay, let’s go ahead and advance the slide.

Brian:So, today, we’re going to cover, and I hope you guys find value in, is first, we’ll hear from Mike, and he’ll discuss research that Forrester has done in analyzing the software market, or the technology markets today, and how continuous delivery and DevOps play a role in those markets. I’ll then introduce a simplified four-quadrant model for measuring your DevOps maturity. We’ll then talk about some of the hurdles that you encounter, when trying to implement DevOps, and we’ll put those in the context of a real-world Enterprise CD implementation, that was done in a large financial software company, to be revealed later.

And then, Sanil will discuss how you apply the four quadrants model, to facilitate implementation of DevOps in the real world. So, with that, I’ll go ahead and hand it over to Mike.

Mike:Absolutely, now, thanks all, thanks for joining. I actually enjoyed how Brian said he spent a good part of his career optimizing the software development patterns, and software development life cycle. I’ll tell you that, as I talk to clients today, and do research, our clients, and the development organizations within our clients’ shops are evolving, and it’s kind of a forced evolution. As I mentioned, I cover a lot of the digital experience development and creation space, and that evolution is forced, because we are now having to build things for web, for mobile, for IOT, for ALEXA – some folks are building for cars. A lot of folks are building for the machines that make their businesses work. And, this is a far reach from what we used to have to do. When I was building software professionally, we built websites.

And, the only thing we had to worry about was one web browser versus another. So, things like IE6, we had to worry about if IE6 was going to work, and that was the biggest challenge that we had. But, now, it’s how do we make sure that we have a consistent experience, across all of these devices, and across all of our different lines of business? And, this is what’s forcing the evolution of our organizations. And so, as I talked to them, one of the biggest challenges they have is, how do we keep up with the pace?

Because the question that I get so often, or the demand I get is, how do we build things quickly? And, in my career, we started out once upon a time – and once upon a time for me, folks, isn’t that long ago, where the overall software development lifecycle was roughly 12 to 18 months long. And, that afforded us an opportunity to have design take their time, and do their job, and their role, and then kind throw it over the wall to development, and then development does their thing, and throws it over the wall to QA. QA throws it over the wall to Ops, and then Ops to kind to APM, and production monitoring, and that all took 12 to 18 months.

Well, now, when we talk to stakeholders, and we talk to customers, they want the software delivered in a 2 to 4-month time frame. And so, moving from 12 to 18 months down to 2 to 4 months, screams for optimization and screams for efficiency. Now, it screams for that, but what a lot of folks simply do is, they just cut off the parts that fall outside of those four months. So, if they’re able to do design, and some of development in four months, well then, that’s when they stop, and they push the market. And, unfortunately, that doesn’t work, because both your customers and your employees still have this understanding of quality, and they still want these experiences to look and work well.

Now, this trend isn’t stopping. We’re still moving towards this zero-day event for software, where your customers, or your business stakeholders are gonna come to us, and say, I have an idea for something, can I have it now? Or, can I have it this week? And, if the answer is no, there are so many other opportunities for them to get that. Something close to what they need, or close enough. And so, we need to work on optimizing how we build software. And, that’s where I spend a lot of my time, researching here at Forrester. Now, we also have a ton of data, and this is some very, very recent data that we have, based on a survey that we run every year.

We asked folks how often does your team release applications? And, if you look here in the middle, the one release per month, is going up. And so, two years ago, in 2014, about one out of every five shops were releasing once a month. Now, we’re up to about one every four shops. Or, one shop out of every four, are releasing once a month, and that’s a lot different than that 12 to 18-month life cycle. So, can you release one time a month? In a lot of folks, the answer is no, and so, they ask me, do I need to blow up my organization and build it again? And, that’s quite extreme, and I – the answer to that is no. We can optimize within what you already have. There are tools and technologies and processes to enable you to do this.

A couple other really neat pieces of data, the one release a week is actually going down. So, folks realize once a week might be great for that brand new startup in the Valley, or startup in the Bay Area, but, for a lot of folks, once a month seems to be the sweet spot that we’re landing on. And then, if we go industry by industry, financial services and insurance – we’ve mentioned, we’ve got a case study coming up on that in a few minutes. Nearly half – actually, a little bit more than half the shops in there, in the financial services and insurance industry, release once a month or faster.

So, this is kinda the goal that we’re looking at. How do we build our process, so that we can go once a month, or faster? And, a great example here, is Netflix. Netflix had a need to be able to deliver their entertainment, deliver their content to a world of devices, because they have to handle not only web browsers and desktops and mobile devices, but they also have to handle game devices, televisions, televisions with extremely low memory, and an extremely constrained development space. And so, they now deliver – they now have a presence on over 200 devices, and I love this quote from Marie Hastings, their CEO, they said, “We realized we learned best by getting in the market and then learning, even if we’re less than perfect…”

And so, it’s that idea, it’s that mantra, of how do we build something that still has quality, but we can get it into the market quickly, and then quickly iterate? This is kinda the marching order that we are now given, as we build software. And so, what does that actually look like? You know, what are the big challenges here? The biggest one that I come across is this. We have all these disparate backend data stores, that we need to provide access to. Whether it’s your CRM system, your content management, your directory servers. All of these things have different data, they’re different types of data, and then, we have to deliver them on all of these disparate and different front ends.

And, the types of front ends seem to be exploding every day. We have the mobile devices, and the web that we still deliver on, and now, we’re seeing an explosion of chat platforms, things like Facebook Messenger, and WeChat, and even iMessage coming out. We have things like the Amazon Echo. And, how do we deliver an experience on that that keeps up with the expectations that we’ve established with all of these other channels? And then, finally, things like the mobile platforms themselves. How do we integrate with Siri and Google Now, and Cortana? All of these things. And so, there’s this disparate array of front end things that we have to deliver on. And so, the true challenge here is, how do we bridge these two?

And, this is just development. This doesn’t speak to the entire software development life cycle. We still have design, and QA and Ops. And so, how do we address this challenge that we see here, as well as the other, individual discipline challenges, and then, make them all work together and come together well, so that we release software on that cadence that we’ve seen before? This is what we’re starting to see, we’re starting to see a change in the modern software life cycle. And, you see on the bottom right, I’ve highlighted on this diagram whose fingers are in that part of the discussion, or who’s the heavy influence in that part of the discussion?

So, initially, when we define the objectives that we need for a digital experience, that’s heavily reliant on the business. We see a little bit of dev in there, but it’s mostly the business that’s defining that. Then, we start to see that both business and development establish a success criteria. What does success look like for this digital experience? We then, within development, create a minimum viable product. That minimum viable product is put into the market, and through operations, and things like APM, and analytics, that we’ve built into this product, we quantify the feedback that we get from our experience.

And, that quantified feedback is important, because in the past, we would put software out with no analytics, and no real feedback mechanisms. But, today, with mobile, and the web, and Alexa, by definition, by platform design, we have mechanisms by which our customers and users can give us feedback. And so, we need to quantify that feedback, so that we can then align that feedback with the KPIs and success criteria that we established in Step 2, and you see here, we have heavy influence from all three parts of both business development and operations. And then, we rinse and repeat. And so, what we’re seeing here is not that we need to tell developers to build things faster, or tell dev and ops to get together, and magically solve this DevOps challenge.

But, instead, our business, our entire business, including development, including the lines of business, need to be on the same page, such that this software life cycle is inclusive of them all, but can also perform efficiently. And, that efficiency means let’s get rid of the friction between each of the disciplines of the SDLC, and the friction between the SDLC and the rest of the business. So, with that in mind, what does that all mean? What are we seeing coming out of this? No. one, digital experience creation is getting more complex, yet the demands on time to market are getting shorter. So, we have to build more, in a shorter period of time. No. two, normalization across the entire SDLC is a must, and this can only be done by eliminating friction between disciplines, and between our software organizations and our business as well.

And, finally, this modern development life cycle is changing. Adapting to this at scale is really the big challenge. And, what do we mean by scale? That means, not only do we have to do this for more than one experience, but we also have to do this for more than one line of business within an organization, and we have to do this within IT. And so, it’s an M by N by X type matrix, that gets more and more complex with every front end channel, but also gets more and more complex with every part of the business that starts to create experiences that help define our overall company’s digital experience, and what we deliver to customers and employees.

And so, that’s what I see right now, in the industry, as the biggest challenge we have. So, with that in mind, with that kinda framework, let me pass it over to Brian from CloudBees, to talk about what they’re doing and what they’re seeing in the market as well.

Brian:Excellent, thank you, Mike. Before I sort of take off into my piece, I did wanna highlight the fact that you called out that when we talk DevOps, DevOps is not just about dev and ops, but as you said, removing friction through all parts of the process, but not only the core development and delivery process, but also with the business. So, we’ll talk a bit to that, by reviewing a simplified model for describing, defining and driving your path to DevOps maturity. This model came into play because, as demonstrated by Mike, there’s a lot of complexity in starting and succeeding in a DevOps journey, so that you can deliver better software faster.

It crosses silos, it crosses lines of business, and ideally scales across your entire organization. What I have found in my experience is that, oftentimes, that complexity creates misalignment, creates additional friction, and ultimately contributes to the failure of one to achieve their goals sought through DevOps. So, through my practice, and our practice at CloudBees, in collaborating with Apexon, we’ve tried to simplify the problem space around the DevOps journey, into a four quadrants maturity model. So, again, you could define and describe where you’re at, and then ultimately, start from a high-level perspective, before you drilled down.

So, to provide a little more context to this model, before I actually step into it, I want to talk a bit about the components of DevOps. So, as Mike highlighted, with the Netflix quote, they – to paraphrase, they discovered that it was better to deliver, and then learn from what they delivered, which you guys may identify really as an agile principle. Really, we wanna deliver fast, to get feedback from the customer early, so we can correct, reduce waste, and increase value. And, this speaks to the overall definition of DevOps, which I like to describe as a culture that you subscribe to, and aspire to, which is supported by practices and technology. And, to highlight this, if we look at the components of it –

If you’re going to implement a DevOps solution, you need to focus on getting a continuous flow from the upstream development, which is on the left-hand side, to downstream delivery, which is on the right-hand. So, we need to be able to go from definition, plan and code, to release, deploy and monitor, in short cycle times. To achieve this, we deploy practices such as agile or incremental project planning and execution, continuous integration of the changes that we make – so, again, we can find problems fast, and correct them more easily, with less expense. Extension of those continuous integration practices, further into the life cycle, via continuous delivery.

And, for some select companies, they also seek to, and successfully achieve continuous deployment, where changes are deployed as soon as they are made. I think it’s important to note that not all companies need to implement continuous delivery to achieve DevOps. Rather, if it’s not a fit for your technology, and it’s not a fit for a customer, then you don’t need to feel the need to achieve it, but you should achieve continuous delivery, to ensure that your software is always ready for release. And, once you’ve done this, you’re able to make rapid changes, get user feedback in a production environment, including bugs, feature responses, etc. Performance, incorporate that in your change management process, and iterate on changes rapidly.

So, let’s take a look at the market. So, market data indicates that only 33 percent of organizations have implemented agile upstream. And, what this means is, they’ve adopted iterative, or lean project planning and execution practices, and/or adopted fast, modern development approaches, such as continuous integration. Now, if we look at who has been able to scale that, only 22 percent of organizations have managed to successfully scale those practices across the enterprise, so they’re getting those benefits across multiple teams. Moreover, and where we start to see what we need to focus on, is only 13 percent of organizations have managed to connect their upstream, agile upstream development processes, with agile downstream delivery processes.

And, for these organizations that haven’t implemented this, what you’re going to find is that they’re not truly seeing the value of being able to define, plan, code and build software more quickly, because they’re not able to get it out to market via an agile delivery process, such that they could get that fast feedback. Now, if we look at the upper right-hand space, we find, by triangulating data, that roughly only 10 percent of organizations claim to have achieved Enterprise CD or DevOps. And, that is where they’ve crossed the chasm from agile upstream, to agile downstream, and connected those, and then managed to scale those across their entire organization, and codify them as standard practices.

And, one may ask, if it’s only 10 percent there, why do we care, why do we wanna get there? Well, companies like Netflix have proven that if you’re able to deliver faster, at Enterprise scale, as Mike noted, across 200 platforms, then you can innovate faster, you can respond to the market, in terms of competitor advantage, security notifications, bugs, or any market conditions, and this allow you to maintain or gain competitive advantage. It’s also been seen that we’re able to not only increase quality of software, but we’re able to increase the productivity of the teams that deliver that software. And, ultimately, that results in us being able to maintain and gain top talent, by ensuring that we have satisfied employees.

And, that cyclically contributes to our productivity, our quality, and ultimately, our competitive advantage. So, this is why we wanna get there. Well, if there’s so much value that has been proven in getting there, why aren’t we there? Well, first, I’d like to highlight what I call the trinity. The DevOps trinity. The DevOps trinity consists of people and culture, process and practices, tools and technology. And, to achieve continuous flow from upstream development to downstream delivery, it requires that you establish flow on the plains of DevOps trinity, all the way from define to operate, or concept to customer. But, in most organizations, there’s an inherent problem with this, based on the way that we’ve sort of defined our legacy, practices and culture.

There is a chasm, or disconnect, between upstream and downstream, that we have to cross. For example, upstream culture tends to be more focused on innovating, getting a number of changes and moving fast. While the downstream culture tends to focus more on maintaining quality, stability, and uptime of the application or services that you provide. Upstream, we’re adopting more agile practices. Downstream, there’s still a prevalence of legacy practices. When we talk tools and technology, we’re seeing a space where open source is dominating the market, people are quickly and rapidly adopting point solutions, and downstream, we’re still seeing a focus on the traditional Enterprise Class tools, procured through a corporate procurement process, tying back to that cultural focus on stability.

So, what we need to figure out how to do, is how do we cross these chasms? And, I’m gonna make the argument that, based on the industry data, and based on a recognition of these chasms, that exist in legacy or mature software organizations, we could divide the problem space into four quadrants. The lower two quadrants, Quadrant 1 and 2, represent the team level, or work group level path, from definition to operation. With Quadrant 1 being an organization that practices team-level agile, or agile upstream, Quadrant 2 being organizations that have managed to adopt team-level continuous integration, across one or a few teams.

A Quadrant 3 company being a company that has scaled their agile upstream practices across multiple workgroups, into the Enterprise, and agile is practiced on most to all teams. A Quadrant 4 organization being an organization that has codified those agile upstream and downstream practices, across their entire organization. So, to drill down a little bit further into this concept, we’ll take a look at some of the characteristics of a company in each of these quadrants. So, for instance, if we look at a Quadrant 1 company, a Quadrant 1 company will tend to have one to a few teams that have adopted agile development practices, however, they’ve probably individually deployed continuous integration, and they have only team-level tools integration, between sort of the processes that happen to the left in the business, and any connectivity to the downstream on the right.

They have team-level project management and development practices, and you can usually identify a Quadrant 1 organization by the fact that they practice a water-scrum-fall approach. And, one of the reasons for this is, not only do we have the chasm between upstream and downstream, but traditionally, the people that orchestrate the definition planning, coding, building, testing and other phases of the SDLC, not only sometimes have different priorities, but they are actually organizationally separated, and they report to different managers, they have different budgets. So, what you will find is, people will tend to do what they can to modernize the area they can, with the least resistance.

So, we often see that dev adopts scrum practices within the code and build, but they resist the impedance that they’re going to encounter when they try to affect or influence that change across the organizational silos. In Quadrant 2, let’s get back to Quadrant 2, here. In Quadrant 2, we start to see some improvement, right? We see some, or a few teams, that have specific applications, supported by agile downstream practices, such as agile test, dynamic provisioning of environments, infrastructure as code Teams have begun to cross that chasm, to extend continuous integration to continuous delivery. But, the problem is, is these implementations and any ops alignment are usually project-specific one-offs.

So, we haven’t determined how to scale that across multiple teams, and codify that as part of our practices. When we look at Quadrant 3, which is where a lot of companies are today, we’ve scaled our agile development practices across the organization, we’ve started to embark on business alignment, with agile specification, and iterative development. But, we still have team-level dev tools supporting the development and delivery process, and we still only have team-level tools integrations. As we move on to Quadrant 4, where we’re recognizing that innovation, fast time to market. We see standardization on tools and process, for things like automated tests and deployment, dynamically provisioned environment, containers, infrastructure as code.

What’s also interesting is that the business and the customers have been brought along with this iterative process for agile definition, development and delivery, and in our organization, we’ve begun to embody a culture of collaboration, learning, and innovation. Okay, so that’s to set the stage for the quadrants, which Sanil will review. I now wanna take an opportunity to walk you through the path that Intuit took, in pursuing Enterprise adoption of the CloudBees-Jenkins platform, to build CI and CD as a service, across the organization. And, I attribute this content and the work that they did to Wen Gu, of Intuit, who is now at Twitter.

So, first, some background. When Intuit decided they wanted to implement Enterprise CI and CD as a service, there were specific business drivers that inspired the initiative. With Intuit, they had a One-Intuit initiative, where they wanted to transform from a product-centric company, where you had separated teams, separated knowledge bases and knowledge stores, delivering each of the product lines, such as TurboTax or QuickBooks, and realign it to be an experience-centric company, where instead of looking at a specific product, you were looking at consumer products, small business products, and Enterprise products. And, therefore, the customer experience say, for consumer products, or personal products, such as TurboTax, would be more aligned across the entire Intuit portfolio.

Instead of getting a different experience, depending on which product you use. This required cross-business unit integration, that didn’t exist on a tools and technology process, or cultural tier at this point. They figured that the best place to start was gonna be with Enterprise CI and CD. And, their drivers were focused not only on do we wanna adopt DevOps, but rather, how do we connect people, or our employees, how do we align our products, and how do we do that such that it improves our profit for our shareholders?

And so, they defined some goals. One is, the driving mantra is, we need to support the overall corporate goal of a One-Intuit, and this realignment. In order to do this, they sought to establish leadership at a central level, because at the end of the day, you needed somebody mediating or driving that had a cross-functional, and cross-team perspective. They figured the lowest hanging fruit, sort of the foundation for the transformation would be to approve the adoption of CI and CD DevOps practices, across the entire enterprise, and in order to do that, they had to win goodwill, and they had to start at grassroots, and solve the problem for the developer, such that they provided a reliable, high quality and consistent infrastructure, that developers would trust, and ultimately use, and that would act as the connective tissue for the development teams to take us across business units. So, our journey was focused on first things first. Start with alignment, and this is one of the things that I can’t emphasize enough, based on my practices. All too often, a specific portion of that delivery process, or the organization, with good intent, starts off on their own, to begin to improve processes.

But, as I like to say, whether we’re a business analyst, a developer, or operations engineer, we are all part of the business, and at the end of the day, our activities, and the output of those activities need to be aligned with the business. So, aligned with the business, aligned with stakeholders across the development life cycle, to be sure that you guys have common objectives, and you define a common path to that. Next, you wanna look to build a team, and that team needs to consist of sort of your central team, that is helping drive the transformation, as well as all of the stakeholders, that are affected by the transformation. Everybody needs a seat at the table at the beginning of the process, again, so you can align on expectations and shared objectives.

The next step was to select pilot teams. I know, oftentimes, you’ll find that in a shared tools and technology service, we like to start the transformation, sort of within the vacuum, as a skunkworks project, within a non-production team. There are some advantages to that, but the disadvantages are one, it’s in a cleanroom environment, two, doing it in that cleanroom environment – so you’re taking the happy path. Second, doing it in a cleanroom environment means you’re not inherently sharing knowledge, and developing support in buy-in. So, it’s important to select a pilot team that is a production team that, ideally, has a low risk and high reward product that they are going to deliver, so you can experiment, you can fail, adjust and learn, without compromising the business.

But yet, that is balanced by the central oversight, so they can address things in light of production needs and production concerns. You also should look to select a range of teams, different team size, different technology, different hosting model. So, Intuit chose to SaaS teams. Two micro-services teams, two mobile teams, focused on iOS, two mobile teams focused on Android, and two platform teams. So that they were sort of vetting out the full experience across technology stacks, across BUs, and across cultures. Once they selected the pilot teams, they needed to identify some starting metrics, and use those to establish key metrics that they would measure, to see if they were improving, or marking progress.

The top engineering metrics were the availability of the platform that they would use. So, kind of if we were using the shared infrastructure, is it reliable and accessible? They also wanted to measure the number of builds. We wanna deliver better software faster, and with less incident, going back to Mike’s slides, we wanna go from 12 to 18 months, to 2 to 4 months, to ultimately make software delivery a zero-event. One of the starting metrics that we could track there was number of builds that were happening per application. But, when we looked at those goals via business metrics, we also then extend that, and look at cycle time, right?

How long does it take to go from definition, or user story, to delivery of that user story, and then, in consideration of that cycle time, how often can teams release? If we’re able to have shorter to equal times, and release more frequently, we’re meeting that goal of getting to market faster, to maintain our competitive advantage. So, what were some of the challenges that they faced? Well, as we said, they faced the challenges of culture, technology and process. When it came to cultural challenges, there was a resistance to the idea of a cultural corporate technology group. And, they had to navigate the feeling that it might be a top-down mandate, where they would take control, versus sharing a platform, and there was fear of loss of that control.

And the fact that, by handing off their resources, or their platform to a central team, they would move too fast or too slow. They addressed that by ensuring that they took a collaborative approach, they innovated incrementally, and developed trust and respect with the stakeholders. That trust and respect then built perseverance. So, look, everything was not going to be perfect out of the box, but based on the trust and the respect, and the involvement in the innovation through collaboration, that built a culture of perseverance, as they iterated through problems. Their process challenge was a concern that one size doesn’t fit all, right, and also, what is the scope?

Does CD apply to every team? Certain teams have different processes, they have different disciplines, and the approach to solve this was to start with a value-stream based design of the process, first at each team, and then merging the sort of value stream per team, so that they could standardize as much as possible, but respect the fact that they would need to divert where needed, because teams legitimately have their own technology process, as well as cultural needs. So, when we take a look at the Enterprise model, what they established is that, for some teams, they would have CI, which would go from build to varying levels of integration tests, with deploy and smoke, to manual and automated tests, and then CD, which would encompass the entire life cycle.

So, going from build with things like unit tests and quality scans to deploy and smoke tests, manual tests and automatic tests, and ultimately, deployment, all the way out to production and test in a production-like environment. And, they needed to support mandatory gates, and application team defined gates, and be able to do all of this in a way that was consistent, reproducible, and traceable. And, their technology challenges were largely focused on, in a large, mature organization, there are inherent differences. And, they need to be able to account for those differences, and begin to align teams on best of breed tools, and proven industry best practices, with support of open source, but not only open source, but open communication. So that they can arrive at the best balance of the standardized platform, and some lowest-common denominator standardized practices, while still supporting and respecting the differences in the team.

And, to sort of highlight those challenges, you can see that there’s different application types, languages, build frameworks, test frameworks, deployment frameworks, hosting needs, etc. And, all of these, as they were supported or deployed by the central tools and technology group, on the Enterprise CD platform, needed to be secure, scalable, trustworthy and reliable, yet still support the One-Intuit goal, of standardization across business units. So, they proceeded with the process, and they were able to recognize some pretty powerful outcomes across the various business drivers that they had for people, product, and profit.

A very interesting one, again that I think is often not focused on, but extremely important, is that from a people standpoint, a KPI they ended up measuring was, what was the employee satisfaction rate? Literally, within weeks of onboarding teams into the common CD as a CI, and CD as a service platform, they’re engagement or employee satisfaction score went up 15 percent. From a product standpoint, they were able to identify via the established metrics that they were able to deliver software with shorter, faster cycle times, and more frequently without compromising reliability. And then, ultimately, for their shareholders, the common platform allowed them to save on licensing costs, administration costs, and infrastructure costs, as well as to be able to deliver faster, they found that they can improve their process, as opposed to scaling headcount.

And, they’ve identified that, by using Jenkins as the tools and technology platform, that provided the orchestration backbone, they saw some benefits, such as the quality and expansibility of Jenkins, Enterprise plugins, which supported security and isolation that the teams needed, via folders and RBAC. The ability to increase reliability, via long running jobs, high availability and backup, and the ability to provide an SLA, which is really a production SLA which became a mission-critical production platform, with support from CloudBees. As they move forward, they sought to leverage more of the Enterprise features, such as the API integration, and license management, to scale the implementation across teams.

So, with that, I’d like to hand it over to Sanil, to build on the Intuit implementation, and the four-quadrants, and provide you guidance on how they can help you implement this in the real world.

Sanil:Thank you, Bri. Hopefully you can see the slides. So, what Mike started the webinar with was the motivation for DevOps, and Brian followed up with a great framework, that can be used for implementing a DevOps journey. What I would like to spend the next few minutes on, are on some practical implementation strategies for DevOps. What does it really mean, and what [inaudible] [00:45:50].

So, DevOps is, it’s not a monolithic concept. It’s got several moving parts, and it works best when all the different aspects are taken care of. So, what that implies, really, is that DevOps can get complicated too soon. And, hence, it is important to evaluate where one is in the DevOps journey, and what should be the guardrail that should be maintained, to have a bridge here. Specifically, I think it’s an assessment that’s required, that would address some key aspects of DevOps. And, if you look at, the assessment really needs to cover things like value today, and what are your three-month goals, and what are your six-month goals. Are you aware of the infrastructure requirements, how are your environments set up, today, what is your org structure? Does it help, or does it hinder a DevOps implementation? What are your existing tools in your tool chain, and does it need to be revamped to the latest in our tool chain, to implement the DevOps strategy?

And, later on I’ll show you an actual assessment, that will actually help you define some of the – get answers to some of these and put you in a specific CloudBees DevOps culture. So, Brian did give an excellent overview of the CloudBees DevOps quadrant, and what you see here is a glimpse of the actual, specific implementation, or implementation implications for each quadrant. This is, of course, a generic version, and through an assessment, we can easily derive the special recommendation for each of the quadrants. So, as an idea, you would see that if you are in the Team Agile quadrant, at the bottom left, what one could benefit from is, are some strategies around – and the system would help them figure out, okay, do we need a workshop, or do we need a jumpstart program?

Which helps to kind of build a proof of concept, of CI and CD, so that they can really see your maturity level in each of these quadrants, and what should be your next step that you should take are. Or, if you’re in the top right corner, which is the area that everyone aspires to be, you can really leverage some of the more advanced concepts of DevOps, and have very well-defined templates, and actually the frameworks that can actually help you take your DevOps even further than where you are at that point of time. And, specifically, if you look at the DevOps journey, there are two parts to it. One is, if you are in a particular quadrant, as an assessment would identify, one part is, how do you mature in that quadrant further, and then, the second part is really, how do you move from one quadrant to the other, which is an adoption strategy.

So, what you see as an example is, if you’re in Team Agile, or you aspire to be in Team Agile, you could actually take care of some of the basic concepts, like assessment, coded pipelines, etc. Or, even SMEs, who could be part of your team, who can actually help evangelize what should be the best practices that are needed across the enterprise. And, if you want to move, let’s say, from Enterprise Agile to Enterprise DevOps, that is the top left corner to the top right corner, you can really now take advantage of some of the more detail advanced concepts in DevOps, to actually take it further, to that level.

So, depending on where you are, a definition of your journey is also defined in terms of the kind of tools that will actually help you get there. Now, the next slide will actually show you a different kind of journey that you could actually take. You could decide that, okay; you are in Team CD, which is the bottom right corner, where you have a very mature deployment set up in your organization. You will decide to look at this as a good time to go into Enterprise Agile, which is to have federated CI across your entire organization, and what should really be your focus, to actually go from that to the next strategy?

So, what quadrants allows you to do is to have you kind of decide what maturity level you are in the particular cycle of your DevOps journey, and then also, defines what your options strategy should be, to move from one quadrant to the other. And, at this point, I’d like to actually show you – this is just a snapshot of what an actual assessment is, but I’m quickly gonna show you how the assessment program is run. So, that’ll give you a good feel for the assessment. So, I believe you can see the assessment spreadsheet. And, so what we have here, essentially, is a list of questions, that you would walk a prospect or a customer through, and the questions are actually organized, top to bottom, in terms of different quadrants.

And, this is answered by a prospect or a practitioner, in the organization. And, as a result of the answers or responses given to each of these questions, which are very specific implementation-level questions. What ends up is that the organization gets mapped into particular quadrants. For example, in this particular setup, we can see that it seems the organization is premature in Team Agile and Team level CD, and based on this, we could give other recommendations, and say what should really they be deploying further, to kind of further their movement in the DevOps journey? So, if I make a change, for example, let’s say that the answer of this question is, do developers integrate code from several developers once a day, and the answer is no, you can see that the DevOps quadrant, that the customer right now is marked as amber, which essentially means that they have some aspects of Team Agile in place.

But, they really could do more to actually mature, and the recommendations that come up, based on that level, would actually indicate – if I click on recommendations, it will real time indicate as to what should be the implementation strategy for each of these levels. Now, obviously, this is not to indicate that everything that’s in here should be implemented. The idea is that you can take some aspects of this concept of individual advance, for example, and start implementing it and then, move along the journey. So, this can be used in multiple different facets, and actually, this gives you a very easy way to decide where you are, and what should really be the next step that you could take in the next six weeks, to actually mature yourself in a particular quadrant.

With that, I’ll just move back to the slides again, So, hopefully, that gave you an idea of how an assessment can help indicate where you are in a particular quadrant, and then take further, and then using the motivation that Mike mentioned, and the actual DevOps quadrant concept, that Brian mentioned, I think we have a very clear path to actually implementing concepts to ideas. And, you can really start that pretty soon.

Charlene:Great. Thank you, all three of you, for that outstanding presentation. We have some questions and answers, or at least some questions, and hopefully you guys have some answers. We have about five or six minutes’ left before we have to close out the webinar, so why don’t we go ahead and take some questions now, if you guys are ready for me?

Mike:Absolutely.

Charlene:Okay, great, so the first question we have is from Arhoul, who asks, what is the level of automation in the continuous integration/continuous delivery space for mobile projects, that you see in the markets? And then, he asks, are customers moving to licensed products in this space, or relying more on open source options?

Mike:Yeah, so this is Mike, I’ll jump in here first, and then let the other guys comment. We’re definitely seeing a desire for automation around mobile, with CI, CD, but as far as the implementations go, they’re still a bit few and far between. One of the challenges is that the mobile OS’s, to truly automate on top of them, whether it be automating on a mobile app, or automating on a mobile web experience, it’s not exactly normalized, and so, if it’s the mobile web, if I build everything in a way that I can kind of attach to the DOM, attach my automation things to the DOM, I can automate there a bit. But, I have to build it the right way, and that includes all the JavaScript libraries, and everything that I use.

For an app, it gets a little bit easier, because both X-Code and Android Studio afford us an option to attach enough testing frameworks that we can use for CI/CD. But, with all the differences, across the individual OS’s and across apps to web, that’s what’s really holding folks back, because the tooling that we use is still a little bit brittle, such that as we change the front end, oftentimes, we have to rewrite or rebuild some of the automation that we’ve already done, and therefore, the ROI goes down pretty significantly

Sanil:And, I can jump in quickly, just to add a few more things than that. So, mobile has definitely been late to the game to adopt DevOps, but what we have seen in a couple of implementations that we’re doing, we have been able to successfully implement a coded pipeline for mobile, for iOS and for Android applications. And, I think what we have learned over a period of time is that the tools are now kind of evolving itself to kind of be linked to orchestrators, like Enterprise Jenkins, or even open source. I think what we learned is, it is helpful to have a template translated for each different operating system that can be leveraged across, because each has its own idiosyncrasies, and that is more prevalent in mobile, probably, than the other platforms. But, definitely, there is automation that exists today, for DevOps and mobile.

Charlene:Okay, great. Thanks very much for the answer. I’ve got a question here from Ken, and Michael, I believe this one is directed at you. Normalization was mentioned, and actually you just mentioned it again. But, how do you see this goal, across different languages and environments such as Cobalt, Java, EE, and Force.com?

Michael:Yeah, great question, and a challenge. And so, it comes down to how do we do that? Obviously, it’s incredibly difficult to normalize across languages, and so, therefore, we try to normalize across the output of those languages, the binary output, or otherwise. And so, that’s why we’re seeing, across the back end, so many folks implementing restful API architectures, so regardless of what language we use, we have restful API endpoints, and so we can kind of normalize there. And so, when I showed you the slide that had the bridge in the middle?

We can normalize that left-hand side, all those disparate things, behind REST. So, that solves the left-hand side, and the right-hand side goes to kind of what we just touched on, which is how do you automate when all these things are built so differently? And so, we’re still working there, and as was mentioned, there’s starting to become some mechanisms by which we can kind of abstract away the nuances of Android versus iOS, and the nuances of web implementation to web implementation. But, the frontend is still lagging a little bit, behind the back end. So, it’s – so, I don’t see the normalization coming language to language, but instead, at an output, perspective, and at a kind of an architectural perspective.

Brian:Yeah, and I’d like to chime in, if I may, Charlene?

Charlene:Absolutely.

Brian:Look, it is, as Mike said, there are inherent challenges in achieving continuous delivery, and continuous deployment on a single plane, even something with smaller and newer content, such as mobile, or SaaS. But, it is doable, it may be painful at times. I would say it needs to be done incrementally, not in a big bang manner, and we should also be aware that at various paces, strides – progress has been made in all of the languages, and environments, and technologies that you mentioned. I would recommend a couple of things. First, if you do know that you plan to normalize this process across different stacks, then you need to identify where you standardize and where you divert. You need to select tools that normalize as Mike said, that output, in an agnostic way, and allow you to, at least for that output, the binary and delivery and testing and delivery of those binaries, allow you to establish a supply chain methodology.

Then, begin to sort of plug in the lowest hanging fruit, right? Your Greenfield SaaS application. Learn best practices from that, then go reach out to your full Java Stack team. And, start to begin to integrate them on that process, and deviate to make it fit, and continuous scale that across the organization, learning and adjusting incrementally, as you move along.

Charlene:Great, thank you, Brian, for adding that. I appreciate it, I’m sure the audience does as well. And, unfortunately, that is all the time that we have for the questions. If you did not – if you submitted a question, and did not get your answer, I’m sure the panelists will be more than happy to address your questions offline. So, with that, I would like to thank Michael, Brian and Sanil, for taking part in today’s webinar. Again, please feel free to check on DevOps.com, for an archived version of this webinar, probably starting tomorrow about the same time, and also check the website for upcoming webinars that we have planned. But, for now, thank you very much for attending today’s webinar. I’m Charlene O’Hanlon, signing off.