Devops.com – An Engineering Think Tank to Change the World

We are happy to see our contribution at the global DevOps.com blog! :-)

An Engineering Think Tank to Change the World

 

 

DevOps and beyond – a forecast on upcoming generations of software production lines (SPL)

Introduction

Techniques proven and routine in other engineering
disciplines 
are considered radical innovations
in software engineering.
(Fred Brooks, ‘Mythical Man Month’ (1975) [1]

 

As the team of the open think tank SIGSPL.org aiming to foster industrial excellence in software delivery methods, we talk to quite many organizations who deal with software delivery. Surprisingly, quite many of these organizations still have to do their homework in terms of lean analysis [2] of their enterprise’s value chain [3]  and digital transformation [4] towards in the meantime broadly known best practices like Continuous Integration [5], Continuous Delivery [6] and DevOps [7] as mission critical core business assets.

But what if we forget the daily struggle bound to technical Change Management [8] and look into our crystal ball? What if today’s systems – which we recognize as software production lines – built around Continuous Delivery and DevOps principles, namely CALMS (Culture, Automation, Lean, Measurement and Sharing) [9], would evolve from one generation to next one, and so forth?

To answer this question, we glance back to what has happened in the software industry in our perception during previous decades, and then proudly present our SIGSPL.org technology forecast on upcoming generations of software production lines (SPL). To illustrate current market dynamics, Figure 1 depictures most prominent software tools and companies as drivers of this development.

figure_1

Figure 1 – converging trends in software technology (grouped around the generic domains labour automation, analysis and advanced visualization). Source: SIGSPL.org (2016)

For every described period, we consider industrialization trends and advancing technologies in the following generic domains, which we consider to be the pillars of the upcoming SPL paradigm:

  • Labour Automation: new test and simulation approaches, agents, machine learning and artificial intelligence;
  • Analysis: natural language processing and process management;
  • Advanced Visualization: new interfaces and immersive virtual reality technologies.

SPL Level Zero (before 2004)

The time before the raise of the Continuous Integration practice and related tools, the automation of machine code production and delivery was limited to the binary code compiler and some satellite tools. We consider this a time before the 3rd industrial revolution [11] in software engineering industry. Maturity of test automation was quite low, and status reporting of technical teams based mainly on established trust.

SPL Level One (2004-2010):
Continuous Integration era

The industrial concept of supply chain management (SCM) [12] is yet quite new to software engineering domain [13]. One of the clues about mass production in software context is not only increasing variety of software products, but repetitive compilation and composing of complex product’s parts every time the product needs to be delivered, or its change and status tracked internally.

The era of Continuous Integration began with convenient automation tools for the processes described above naturally appearing in bigger teams around bigger products. These tools enabled then an exponential growth of open source code base to millions of standard software components [14]. This period ends for us with the prominent educative books ‘VELOCITY’ [15] and ‘Continuous Delivery’ [8].

SPL Level Two (2010-2018):
DevOps era

In these years, Continuous Integration evolves to Continuous Delivery, and the world starts to buzz about DevOps, reaching also less technical mass media like Forbes [16]. Docker [17] appears on the market and becomes a boom in just one or two years. Lean methods finally penetrate the domain of Information Technology after their first appearance coined as Agile last decade, and some high-end enterprises recognize the necessity of strategic investments to deploy Six Sigma methodology for mission critical software delivery.

From the SPL point of view, Docker can be emphasized as a key organization disruption tool on the delivery end, as it enables standardized virtual integration testing and sharing of integration scenarios, e.g. via so called image bakery [18].

All possible building pieces for industrial software production lines (SPL) are now (as of February 2016) there and well in use [19, 20], what is often missing, is an organized business-driven strategy to put DevOps philosophy tools and related production data spread across many departments in the context of the value chain [21], removing organizational silos. The barrier to a game changing performance and excellence leap already passed by industry leaders like Google and Amazon, is not the technology but lacking management awareness that the hyping Digital Transformation has not yet reached the business processes of departments responsible for the enterprise information technology from research and development to operations – this setting reminds us of crime fiction; guess why.

SPL Level Three (2018-2022):
Post-DevOps era

From here, we start our actual forecast. Let’s imagine, DevOps principles rule the organizations delivering software, and first Software Production Lines are now in operation. How would this world look like?

We think, that while in 2016 Google or Amazon can deploy new features to their operative environments many times a day, we can be quite sure that delivery of new software artifacts e.g. new business functions for enterprise IT will soon touch real time, and this with an increasing technical quality.  Advanced SPL tools will start to support natural language processing (NLP) to enable formal quality assurance for written requirements and specifications as part of the value chain [22]. Standard productivity key performance indicators (KPIs)  for software teams will become  state of the art [23]. Business processes and artifact maps will appear on large interactive dashboards [24] based on captured process relevant Big Data forming the KPIs, maybe in 3D where it will make sense. Figure 2 gives an example of one such visualization, generated by the software analytics tool Seerene.

figure_2Figure 2 – Automated Visual Software Analytics. Source: Döllner/hpi.de (2015)

SPL Level Four (2022-2025):
Digital Value Chain Dominance era

By this time, we can expect that the digital value chain loop will be closed, and mostly any industrial value will find its origin in abstract digital models and software code, these assets cascading towards goods and services perceivable to human beings. For a detailed overview of specific trends, see the World Economic Forum report ‘Deep Shift’. [25]

For software production lines, we expect in this time higher process parallelism and further advance of productivity. Natural language processing (NLP) will experience deep integration to measure and prevent impacts of immature or poor requirements, even to discover ambiguous, or flawed business processes and rule sets. As we assume, this development could become one possible branch of the IBM Watson technology [26]. This approach will lead to deep organizational transformation even before a software project would be anticipated. Advanced machine learning and other artificial intelligence (AI) approaches will serve to optimize product and process quality. We expect by this time also tight integration with Virtual Reality technology, that is, software engineers and testers will operate in virtual environments using fluid complex representations of software assets and related infrastructure [27, 28].
Our expectation is also that further standardization of software components will lead to significant drop in diversity on the software market. Like in many other domains, also specialized commercial SPL cloud services [29] will face the challenge of abundant [30], freely available computational power [31]. Quality measurements and evaluations of prominent software components will become much more transparent towards business users [32] and will benefit from leveraging distributed trust mechanisms like Blockchain [33] in terms of finding common sense across communities; software supply chains and intellectual property business might benefit from smart contract technology [34].

SPL Level Five (2026-2030):
IoT SPL era

While as of 2016 we experience a big hype regarding Internet of Things (IoT), the technologies behind are not in their mature phase yet as we suppose. Technologies behind IoT will become far more advanced within the next decade.  We think that in the SPL domain we have to learn best practices from the mature industry, we assume that the Machine-to-Machine (M2M) [35] and Internet of Things concepts will find their impact in the world of the SPL technology only in the later next decade as the 4th industrial revolution becomes daily reality [36]. In our perception, we talk here about something like the second order IoT, or Internet of Virtual Things [37], as SPL sensors do not digitize physical processes, but furthermore capture already existing digital data relevant for software production. Requirements input interfaces will evolve to interactive NLP powered tools, that is, engineers and domain experts will be able just to talk to computer as peer to induce software specification, modeling and also development in many cases. Industrial software agents [38] could also jump into partially fulfill roles of technical project coordinators and release managers. Quality management will be performed through very advanced simulations and predictive analytics – possibly, using specialized high performance computing (HPC) hardware. VR immersion could advance to augmented VR, that is software teams will find themselves in Holodeck like specialized working environments along with haptics and immersive interactive gamified programming [39].

SPL Level Six (after 2030):
Era of Industry Z

Given the previous rapid pace of digital technology, and as we look back to year 2000, we know that it is almost impossible to predict what could happen until 2030, or whether this amazing development could just stop somewhere.

Therefore, now we aim to describe something so ultimate that further development of SPL technology will remain completely obscure despite of all our imagination strength.

Likewise, in case brain machine interfaces (BMI) [40] become really advanced by 2030, maybe some programmers will try out a direct connection to machine, by this time maybe bidirectionally (Fred Brooks mentions, that productivity level of programmers can range 1:10 [1], so probably best programmers might have neuronal patterns, reusable for design of even more advanced software agents, as already demonstrated in industrial robotics [41]). Even without this type of approach, we are quite sure that dynamic SPL programming with AI support will become possible, e.g. an AI will analyze demand and shape an SPL as a part of the software product. Enterprises will use agent-based technology to create new, more advanced software agents and induce new quality standards in a self-propelling, artificial life manner. Likewise the industrial vision that machines could build their next generations by themselves, so why should same be not possible, even more viable in the virtual digital world – the Cyberspace?

Considering the impacts on the totally digitized… everything, we think that by then, the Industry Z will be born, Z for an maturity state which could dominate the rest of the 21st century and goes beyond our imagination limits, also considered as Technological Singularity [42].

Summary

This white paper gives a retrospective how industrial methods for automation, mass production and quality assurance have influenced the software engineering domain before 2016 resulting in Continuous Delivery and DevOps, and establishes a prognosis and a vision on how the Special Interest Group for Software Production lines (SIGSPL.org) estimates these fields can evolve until 2030, considering known trends.

As closing words, we quote Eric Ries, possible one of the most prominent industrial think leaders of our time: “The big question of our time is not Can it be built? but Should it be built? This places us in an unusual historical moment: our future prosperity depends on the quality of our collective imaginations.” [43]

 

List of Figures

Figure 1 – converging trends in software technology.
Based on: Watch multi-billion capital: SIGSPL.org Magic Triangle reveals how 21st century information technology will look like.
http://sigspl.org/watch-billion-capital-sigspl-org-magic-triangle-reveals-how-21st-century-information-technology-will-look-like  
retrieved on 2016-02-12

 

Figure 2Automated Visual Software Analytics.
Source: Döllner/hpi.de (2015)
https://open.hpi.de/courses/softwareanalytics2015#135
retrieved on 2016-02-12

References

[1] Mythical Man-Month. Brooks (1975)

[2] Lean Software Development: An Agile Toolkit. Poppendieck (2003)

[3] Competitive Advantage: Creating and Sustaining Superior Performance. Porter (1985)

[4] Digital Enterprise Transformation: A Business-Driven Approach to Leveraging Innovative IT. Uhl, Gollenia (2014)

[5] Continuous Integration: Improving Software Quality and Reducing Risk. Duvall et al. (2007)

[6] Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Humble, Farley (2010)

[7] DevOps: A Software Architect’s Perspective. Bass et al. (2015)

[8] Organizing ITSM: Transitioning the IT organization from the silos to services with practical organizational change. Steinberg (2015)

[9] Is the S (Sharing) in CALMS redundant? Devopsguys.com (2014)
https://blog.devopsguys.com/2014/08/08/is-the-s-sharing-in-calms-redundant/
retrieved on 2016-02-12

[10] Horizon 2020 Extract from Part 19 Commission Decision C(2014)4995, Annex G. Technology readiness levels (TRL). European Commission (2014)
http://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/annexes/h2020-wp1415-annex-g-trl_en.pdf
retrieved on 2016-02-12

[11] Revealing the German plan on the future ‘Industry 4.0’ SIGSPL.org (2015)
http://sigspl.org/2015/10/14/revealing-the-german-plan-on-the-future-industry-4-0-part-13-terminology/
retrieved on 2016-02-12

[12]  Supply Chain Engineering. Goetschalckx (2011)

[13] A Newcomer’s Perspective: Software Supply Chains Sonatype (2015)
http://blog.sonatype.com/2015/07/a-newcomers-perspective-software-supply-chains/#.VrerR_nhAdg
retrieved on 2016-02-12

[14] Better and Fewer Suppliers (2015 Software Supply Chain Report). Nexus (2015)
http://www.sonatype.org/nexus/2015/06/17/better-and-fewer-suppliers-2015-software-supply-chain-report/
retrieved on 2016-02-12

[15] VELOCITY: Combining Lean, Six Sigma and the Theory of Constraints to Achieve Breakthrough Performance – A Business Novel. Jacob et al. (2009).

[16] DevOps and ITIL: Friends or Enemies? Forbes (2015)
http://www.forbes.com/sites/jasonbloomberg/2015/11/13/devops-and-itil-friends-or-enemies/#7e946ee6467b
retrieved on 2016-02-12

[17] 5 Reasons Why Docker is a Billion Dollar Company. Forbes (2015)
http://www.forbes.com/sites/mikekavis/2015/07/16/5-reasons-why-docker-is-a-billion-dollar-company/#46a5146f19c5
retrieved on 2016-02-12

[18] FI-PPP FICONTENT2 public report, Chapter 5 “Packaging and delivery of server side Specific Enablers (SEs)”  (2015)
http://mediafi.org/wp-content/uploads/2013/07/FI-CONTENT-2_WP6-008_D6.4.2_V1.0.pdf
retrieved on 2016-02-12

[19] Periodic table of DevOps tools. XebiaLabs (2015)
https://xebialabs.com/periodic-table-of-devops-tools/
retrieved on 2016-02-12

[20] Software Measurement: Establish – Extract – Evaluate – Execute. Christof Ebert (2007)

[21] The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win. Kim Gene (2013)

[22] Kof, 2007. Text Analysis for Requirements Engineering- Application of Computational Linguistics.

[23] Integrated Measurement – KPIs and Metrics for ITSM (Stories in transforming ITIL® best practice into operational success). D. McLean (2013)

[24] 70 MP Delight: Amazing 15000 x 5000 Pixels LHRD wall at the GI VR/AR workshop. SIGSPL.org (2015)
https://www.linkedin.com/pulse/70-mp-delight-impressive-15000-x-5000-pixels-lhrd-wall-muryshkin?trkSplashRedir=true&forceNoSplash=true
retrieved on 2016-02-12

[25] Deep Shift. Technology Tipping Points and Societal Impact. Survey Report. Global Agenda Council on the Future of Software & Society to World Economic Forum (2015)
http://www3.weforum.org/docs/WEF_GAC15_Technological_Tipping_Points_report_2015.pdf
retrieved on 2016-02-12

[26] Cognitive Computing and Big Data Analytics. Hurwitz, Kaufman (2015)

[27] A new UI for Docker? Docker.com (2015)
https://blog.docker.com/2015/11/dockercraft/
retrieved on 2016-02-12

[28] Build for VR in VR. Unrealengine.com (2016)
https://www.unrealengine.com/blog/build-for-vr-in-vr
retrieved on 2016-02-12

[29] That Pivot Seems To Have Worked Out OK–CloudBees Picks Up $23.5M. Forbes (2015)
http://www.forbes.com/sites/benkepes/2015/01/27/that-pivot-seems-to-have-worked-out-ok-cloudbees-picks-up-23-5m/
retrieved on 2016-02-12

[30] Abundance: The Future Is Better Than You Think. Peter H. Diamandis (2014)

[31] Travis CI – Overview. CrunchBase.com (2016)
https://www.crunchbase.com/organization/travis-ci#/entity

[32] CTO Advisory: towards a benchmark for software quality you could show your CEO because it creates value through sustainability. SIGSPL.org (2015)
http://sigspl.org/2015/09/21/cto-advisory-towards-a-benchmark-for-software-quality-you-could-show-your-ceo-because-it-creates-value-through-sustainability  retrieved on 2016-02-12

[33] The Science of the Blockchain. Wattenhofer (2016)

[34] Great Chain of Numbers: A Guide to Smart Contracts, Smart Property and Trustless Asset Management. Swanson (2014)

[35] From Machine-to-Machine to the Internet of Things: Introduction to a New Age of Intelligence. Tsiatsis et al. (2014)

[36] The Fourth Industrial Revolution. Schwab (2016)

[37] IoT 2.0 – the next Internet of Things! SIGSPL.org (2015)
retrieved on 2016-02-12

[38] Industrial Agents: Emerging Applications of Software Agents in Industry. Leitão, Karnouskos (2015)

[39]  Magic Leap – A startup is betting more than half a billion dollars that it will dazzle you with its approach to creating 3-D imagery. TechnologyReview.com (2015)
https://www.technologyreview.com/s/534971/magic-leap/
retrieved on 2016-02-12

[40] Beyond Boundaries: The New Neuroscience of Connecting Brains with Machines – And How It Will Change Our Lives. Nicolelis (2012)

[41] Will robots take our jobs? MIT building worker robots that learn by operating beside humans. ibtimes.co.uk (2015)
http://www.ibtimes.co.uk/will-robots-take-our-jobs-mit-building-worker-robots-that-learn-by-operating-beside-humans-1490318
retrieved on 2016-02-12

[42] The Technological Singularity. Shanahan (2015)

[43]  The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Ries (2011)

Header image source: http://img1.123freevectors.com/wp-content/uploads/portal/spaceship-illustration-free-vector-839.jpg