When I heard about the CA DevOps and Cloud Forum regional event here in Seattle, I decided this would be a great opportunity to stop by the EMP museum and hear about the state of continuous delivery from CA Technologies, their customers and Forrester analysts, and maybe catch a little of the Star Trek exhibit.

CA is continuing with its brand mantra of Digital Transformation, and advancing that on June 15 they recently announced an Open Ecosystem for Continuous Delivery that incorporates their product suite, along with containerization (Docker), CI (Jenkins, etc.) other common tools (JIRA, git, etc.), as well as cloud service providers that can host elements of the solution.

“DevOps is the new factory driving business transformation” said Kieran Taylor, CA’s product marketing head for the division. Rather than focus on known disruptors like AirBnB and Uber, Taylor presented several customer examples of more established companies like GE, Nike and Bosch that are building innovative practices such as deep analytics and IoT devices through better automation and more nimble release timelines.

The solutions map is quite broad now – encompassing their well-established CA Release Automation (formerly Nolio), Service Virtualization (ITKO LISA from my alma mater), API management (formerly Layer7) and Application Performance Management solutions, as well as the more recently named solutions of CA Test Data Manager (formerly Grid-Tools TDM), CA Agile Requirements Designer and Agile Management (formerly Rally), and a Mobile Cloud for building/testing mobile apps.

Stephen Feloney, CA’s product management VP for the unit, described how the new Continuous Delivery toolchain is not just about deploying faster, but automating testing with test data and services across every phase of the SDLC to avoid risk. “94% of executives face pressure to release faster, but you can’t claim ‘Assume the Risk’ as a badge of courage if automated testing is not built into every release.”

Forrester analyst Milan Hanson framed the current market for more agile development. “Simply driving IT costs down is no longer the top priority – 68% of companies now rate customer experience (CX) highly.” Success in CX is measured not just by satisfying customers with business technology, but through growth delivered by delighting customers.

The need for speed in delivering applications customers want can negatively impact customer experience.  “Many companies are basically doing faster releases, with QA in production, atop constantly changing environments that are hard to replicate.” Even if faster releases are done as quick canary deployments with rollback capability, that can lead to costly customer losses, and demoralizing extended-hour break-fix exercises and war room scenarios for IT teams.

Then Forrester presented some TEI (Total Economic Impact) studies they conducted with a sampling of several large deployed customers using CA’s TDM, service virtualization and release automation solutions. [Reports available lower on the release page here.]

The payback on these ranged from 3-6 months from implementation, with 3-year ROIs ranging from 292% to 389% per solution. Release automation reduced deployment times by as much as 20X, and the use of service virtualization and test data management created some equally astounding results – saving 640 developer hours per release, finding more than 150 defects in earlier phases…

Man, I have been either marketing or writing about software for a long time, and have never seen a major analyst present those kinds of numbers for me. The results make sense though, when you visit the customers who have fully embraced and championed the value of these solutions for their SDLC.

My favorite part of the program was a customer Q&A which could have used more time on the agenda, in my opinion. Practitioners from a major state healthcare payer, and the online automotive service AutoTrader.com fielded questions from CA and the audience.

Adam Mills of AutoTrader said they used to spend 2 weeks out of every 6 week test cycle waiting for environments to be ready, and now they are not only out of that game, they are doing some cool what-if testing scenarios, including something like NetFlix’s famous “Chaos Monkey” project.

“We Set up ‘Chaos as a Service’ to simulate the behavior of systems working improperly in our testing – slow performance, no response, multiple responses, garbled data,” said Mills. “We immediately found we were breaking things like error handling that you can’t test without generating that kind of data. We get a lot of benefit from testing what third parties might do. Now that we can simulate whatever we want – it’s a lot of fun.”

A post-session reception was perched in the fantastic little Blue Lounge atop the EMP theater room. Looking at some APM demos and talking to some of their current and potential customers there, I definitely felt the presence of a “chicken or the egg” dilemma for established IT shops in prioritizing which aspects of their software delivery toolchain to modernize first.

One thing is for sure. All established companies are struggling with test environments, and the time it takes to get them provisioned well at each phase. Should they start with a move to cloud-based labs or containers, or by making the assets themselves leaner and more repeatable with test data management and virtual services? Should more performance and test insight be embedded into the software itself so real-time feedback occurs and problems are found earlier? All I can say is yes – start somewhere! 

Here's a fun complete breakfast I prepared, photographed, ate and then wrote for the ServiceVirtualization.com community site sponsored by CA, ran Dec. 16, 2015. (See original post at http://servicevirtualization.com/247-complete-devtest-breakfasts-service-virtualization-in-cloud-environments/) - JE

It’s been a while since Service Virtualization (and this SV.com site, for that matter) came out, both as a practice and a technology. Since this site was launched back in 2010, it seems like another trend has occurred: fast food restaurants selling breakfast food 24/7. I don’t know about you, but breakfast is still by far my favorite meal. If everything you want is there, no meal beats breakfast. So why not have a complete breakfast there whenever you want it?

The invention of Service Virtualization in 2007 was huge for resolving dependencies in development and testing, so those teams could move forward without the “complete breakfast” available. At the same time, SV inadvertently resolved some of the primary constraints to serious enterprise adoption of public dev/test cloud environments. We used to describe this phenomenon of constrained components that you can’t simply import as “wires hanging out of your cloud.”

Take any system that you need to have ready for testing, but is not readily available. It could be a heavy mainframe that is too bulky to image as a VM, or a third party service you don’t have the access permission to copy. It would be much easier if you could realistically simulate just the behavior and data you need to run tests with those components.

So SV gave us a lightweight way to eliminate these constraints by replacing them with Virtual Services. This new technology is now a standard practice in large enterprises, with several major vendors offering solutions in the space. SV is proven to “cut the wires” of dependencies in dev/test environments.

That’s great for traditional on-premise environments, but it is especially useful in cloud dev/test scenarios, where dynamic availability – anywhere, anytime — is of the essence. Cloud infrastructure has come a long way in the last few years as well – offering increased capacity and performance at decreasing cost. We're seeing some huge environments running in cloud labs now at every phase of the lifecycle, including CA applications, SAP test regions, Oracle RAC servers, the list goes on. You can even invoke and specify these environments via API with a host of new IaC (Infrastructure as Code) and automated deployment solutions like CA’s Release Automation and CI/CD tools like Jenkins and many others. Containers have also now become a new, happy meal player in this complete SLDC breakfast.

Having the real thing in a 24/7 cloud is great, but who really wants a Coke with breakfast? In many cases, even if you could get the whole production-style application architecture imported, you may not always want the real thing in your dev/test cloud. Production systems may respond and perform unpredictably. If you are developing an application that will talk to production systems, you will likely need to suss out all the boundary conditions in your battery of tests. For instance, what if the mainframe responds in 30 seconds instead of 3 seconds, or .3 seconds? What if my partner’s service returns my form request with an unknown error, or a bunch of SQL hack statements? 

It takes too much work and coordination to try and make every other team’s system respond exactly as you want. But you can easily make a virtual service learn the behavior, and get Test Data Management (TDM) to supply the exact right data, so that you can have it all dynamically provisioned in a cloud environment that does exactly what you want, on-demand. And other teams can get that same level of customization and on-demand environments that are separate so they can work in parallel, without having to become experts in how the whole application infrastructure and network is set up. Better to focus on the aspects of development testing, integration and performance testing that are in the scope of your requirements, and simulate the rest with SV. 

The same concern applies to virtualizing test data. Unless you are getting close to production phases of your SDLC, you shouldn’t have to deal with extracting and loading huge volumes of production data into dev/test environments. When policy demands test data be scrubbed of sensitive info before being loaded, TDM provides another lightweight asset that can provide valid “virtual test data” and be spun up in a public dev/test cloud.

What’s cool is that all of these modern development and test technologies can be loaded into an environment in Skytap, along with all the pre-configured servers, network and domain settings, and management controls needed. Then you can instantly stamp out clones of the exact environments needed to advance development, test and release activities faster than we ever imagined.

Yes, SV and Environments-as-a-Service in cloud infrastructure are great technologies separately, but when pulled together, and given automation, they make up a complete breakfast of dev/test champions that is available 24/7. Don’t just settle for serial, or cereal breakfast just because it is late. Customers won’t wait for it. So why should you?