The quest for the holy grail of software

Hardware pieces have long assembled into functional products on the assembly line. With software, we’re finally ready to create a similar revolution.

 

It seems no coincidence that the term “software engineering” first appeared in an actual engineering context. Margaret Hamilton used it to describe her work on the Apollo Guidance Computer for the NASA Apollo Mission.

Inside the Apollo Guidance Computer was a core rope memory weaved by professional weavers. The whole construction was brittle and hard-coded: any mistake would entail the collapse of the entire mechanism.

The software itself was brittle as well. When Hamilton used the word engineering, then, she did not refer to the ease with which the software was replicated but merely to the precision and diligence with which it was built. She was well aware, in other words, that there was no assembly line for software products yet.

Perhaps, however, a veritable assembly line was exactly what the software industry needed.

The battle for a world of components

In 1968, with the Apollo mission still in progress, at NASA’s Software Engineering Conference in Garmisch, Germany, software engineer Kenneth Kolence spoke up.

»The basic problem is that certain classes of systems are placing demands on us which are beyond our capabilities and our theories and methods of design and production«.

»It is large systems that are encountering great difficulties. We should not expect the production of such systems to be easy«.

In 1968, building large, reliable software systems such as those required for the Apollo Mission in a controlled, cost-effective way was an unsolved challenge. Before a veritable assembly line for software – one capable of constructing such large systems – could become a reality, the right reusable parts had to be discovered. 

One person who was present at the conference dreamed of a world where such components would exist.

In his talk, “Mass Produced Software Components”, Malcolm Douglas McIlroy, then of Bell Labs, laid out a vision. A vision where the construction of large systems of software was handled with ease.  

Douglas McIlroy's vision

»The purchaser of a component from a family will choose one tailored to his exact needs. He will consult a catalogue offering routines in varying degrees of precision, robustness, time-space performance, and generality. He will be confident that each routine in the family is of high quality – reliable and efficient. He will expect the routine to be intelligible, doubtless expressed in a higher-level language appropriate to the purpose of the component… He will expect families of routines to be constructed on rational principles so that families fit together as building blocks«
Douglas McIlroy

Software Engineer

The failure to turn McIlroy’s vision into reality

Ever since McIlroy laid out his vision of a world of reusable components, the software industry has struggled to create it.

Several factors spoke against McIlroy’s ideas being instantly implementable. To work, reusable software components had to be discovered by large software companies. The larger those companies grew, the quicker their internal feedback loops deteriorated. A company of 20 people knew each other’s tasks, making double-work unlikely. In a company of 20.000, significant investment had to be put into maintaining strong feedback loops, and significant resources had to be put into maintaining and developing a common resource of reusable components. If no common resource was maintained, reuse would be reduced to opportunistic, scavenging approaches. To put a final nail in the coffin, psychological impediments such as the “not invented here” and “uniqueness bias” syndromes enforced barriers to successful reuse at large scales. 

In 1999, in a 1999 issue of C++ Report Magazine, developer Douglas C. Schmidt declared McIlroy’s vision anything but dead – or at least not yet awakened. Yet, human history was full of examples where reuse had worked.

Most familiar to all, alphabets had allowed for the “infinite use of finite means”, as naturalist Wilhelm van Humboldt put it.  

Later, the hardware industry made reuse work. Why then, had the software industry consistently failed to replicate earlier successes?  

One provocative possibility was that no one had really tried. Just like the reusable parts of language used to consist of entire symbols of objects in the world, only to emerge as right-sized letters after thousands of years of trial and error, perhaps getting reuse to work would require a similar significant investment by one or more software companies. 

In the 2010s, Netcompany decided it was time to make this investment. 

Netcompany enters reuse era

»Our way of thinking was that, if you’re a small business, a reusable component will likely not be used enough before its outdated. But as a large-scale business, it will. This simple logic made us decide to invest heavily in software reuse at scale«
Jesper Molbo

Principal

At Netcompany, the process of discovery took place over millions of hours of project work. Netcompany Foundations (LINK), spanning error handling, logging, caching, Web API, relational databases, transaction management, were discovered to be reusable components across nearly all IT projects. Components in Netcompany Platforms (LINK) were discovered to be reusable across specific domains. Certain IT projects were discovered to be so similar that entire reusable Standard Products could be developed. 

»The components were never born to be reusable, but they were discovered to be so«, Martin Ellebæk, Principal at Netcompany, says. 

With a veritable assembly line for software, rapid deployment was possible. 

»With our platforms, we could start a project almost instantly and make a PoC with a Platform by two people in a couple of weeks«, Ellebæk says.  

Since McIlroy laid out his vision of a world of components, his vision remained just that – a vision. It turned out, however, that his vision was not impossible, it was just hard.

What MIlroy called for, in essence, was an industrial revolution for software. As is well known, the assembly line played a crucial part in the industrial revolution in the 18th century. This did not reduce product diversity but increase it. The same will happen with software, eventually. We’re not there yet. But we might soon be if the rest of the software industry follows suit.  As McIlroy noted, »there are considerable areas of software ready, if not overdue, for this approach«. 

Benefits from software reuse

Balance innovation and stability

Reuse leads to quickly implementing and integrating componentized platforms with underlying SOR-systems, thus maintaining stability and decreasing time-to-market.

Lower risk

Only tried-and-truly components become reusable, reducing inherent risks associated with IT-transformations.

Focus on what matters

Using primarily reusable components to drive their transformation, customers get to spend valuable time on the few percent that set them apart from competitors.

Innovate consistently

Platforms of reusable components are modular and reconfigurable. This approach ensures that businesses continue to deliver additive value to their customers while still managing risks.

Want to learn more about our approach to reuse?

Reach out to

Jesper Molbo