When I heard about the CA DevOps and Cloud Forum regional
event here in Seattle, I decided this would be a great opportunity to stop by
the EMP museum and hear about the state of continuous delivery from CA Technologies, their customers
and Forrester analysts, and maybe catch a little of the Star Trek exhibit.
CA is continuing with its brand mantra of Digital
Transformation, and advancing that on June 15 they recently announced an
Open Ecosystem for Continuous Delivery that incorporates their product suite, along with containerization (Docker), CI
(Jenkins, etc.) other common tools (JIRA, git, etc.), as well as cloud service providers
that can host elements of the solution.
“DevOps is the new factory driving business transformation”
said Kieran Taylor, CA’s product marketing head for the division. Rather than
focus on known disruptors like AirBnB and Uber, Taylor presented several
customer examples of more established companies like GE, Nike and Bosch that are building
innovative practices such as deep analytics and IoT devices through better
automation and more nimble release timelines.
The solutions map is quite broad now – encompassing their
well-established CA Release Automation (formerly Nolio), Service Virtualization
(ITKO LISA from my alma mater), API management (formerly Layer7) and
Application Performance Management solutions, as well as the more recently named
solutions of CA Test Data Manager (formerly Grid-Tools TDM), CA Agile
Requirements Designer and Agile Management (formerly Rally), and a Mobile Cloud
for building/testing mobile apps.
Stephen Feloney, CA’s product management VP for the unit,
described how the new Continuous Delivery toolchain is not just about deploying
faster, but automating testing with test data and services across every phase
of the SDLC to avoid risk. “94% of executives face pressure to release faster,
but you can’t claim ‘Assume the Risk’ as a badge of courage if automated
testing is not built into every release.”
Forrester analyst Milan Hanson framed the current market for more agile development. “Simply driving IT costs
down is no longer the top priority – 68% of companies now rate customer
experience (CX) highly.” Success in CX is measured not just by satisfying customers
with business technology, but through growth delivered by delighting customers.
The need for speed in delivering applications customers want
can negatively impact customer experience. “Many companies are basically doing faster
releases, with QA in production, atop constantly changing environments that are
hard to replicate.” Even if faster releases are done as quick canary
deployments with rollback capability, that can lead to costly customer losses,
and demoralizing extended-hour break-fix exercises and war room scenarios for
Then Forrester presented some TEI (Total Economic Impact)
studies they conducted with a sampling of several large deployed customers
using CA’s TDM, service virtualization and release automation solutions.
[Reports available lower on the release page here.]
The payback on these ranged from 3-6 months from
implementation, with 3-year ROIs ranging from 292% to 389% per solution. Release
automation reduced deployment times by as much as 20X, and the use of service
virtualization and test data management created some equally astounding results
– saving 640 developer hours per release, finding more than 150 defects in
Man, I have been either marketing or writing about software for a long time, and have
never seen a major analyst present those kinds of numbers for me. The results
make sense though, when you visit the customers who have fully embraced and
championed the value of these solutions for their SDLC.
My favorite part of the program was a customer Q&A which
could have used more time on the agenda, in my opinion. Practitioners from
a major state healthcare payer, and the online automotive service AutoTrader.com
fielded questions from CA and the audience.
Adam Mills of AutoTrader said they used to spend 2 weeks out
of every 6 week test cycle waiting for environments to be ready, and now they
are not only out of that game, they are doing some cool what-if testing scenarios,
including something like NetFlix’s famous “Chaos Monkey” project.
“We Set up ‘Chaos as a Service’ to simulate the behavior of
systems working improperly in our testing – slow performance, no response,
multiple responses, garbled data,” said Mills. “We immediately found we were
breaking things like error handling that you can’t test without generating that
kind of data. We get a lot of benefit from testing what third parties might do.
Now that we can simulate whatever we want – it’s a lot of fun.”
A post-session reception was perched in the fantastic
little Blue Lounge atop the EMP theater room. Looking at some APM demos and talking to some of their current and potential
customers there, I definitely felt the presence of a “chicken or the egg”
dilemma for established IT shops in prioritizing which aspects of their
software delivery toolchain to modernize first.
One thing is for sure. All established companies are
struggling with test environments, and the time it takes to get them
provisioned well at each phase. Should they start with a move to cloud-based
labs or containers, or by making the assets themselves leaner and more
repeatable with test data management and virtual services? Should more performance
and test insight be embedded into the software itself so real-time feedback
occurs and problems are found earlier? All I can say is yes – start somewhere!