Disrupted Software the Disrupted Software Industry Uses to Build Disruptive Software
In the previous article I looked the ways and reasons the software industry disrupts the very tools it uses to write software, and the results in terms of disrupting itself.
In this article I’d like to look at the nature of the development tools themselves and how disruption has affected the way they, and the developers who must use them, work.
First, though, to give a basis for assessing these tools, I’d like to give an example of a tool set that, due to both being niche and the specific nature of that niche, have not been subject to that kind of disruption.
The niche involved is not a specific type of software, say software for a specific industry, or software at an unusual micro or macro scale, though there is such software and in a related way it’s far less subject to disruption. This software development language and environment can be used to create any general-purpose software. Its niche is in developing software that must work. Since it must work (because it’s often used for situations where an error could mean the difference between life and death), engineers who write software in that area cannot allow it to be disrupted by radical ongoing changes in the core language, base tools and core frameworks.
Due to that circumstance, it becomes both possible and worthwhile to spend significant time writing tools, and even significant time writing core frameworks that are designed to make writing developer tools easier and the resulting tools more powerful. The vicious circle I analyzed on the industry scale in the previous article and will look at on the scale of actual tools later in this one can also be a virtuous one, but it requires specific conditions for that to occur, and those specific conditions only exist in niches.
By looking at one language and environment and the tools it provides, one that while niche is not a niche requiring a specific type of software, but general-purpose software with a specific requirement, namely that it must be reliable, it’s possible to see the ways the disruptive tendencies in the industry prevent mainstream development tools from achieving a similar level of maturity and usability even though the goals of such tools are ostensibly the same.
It happens that the base language was one of the first I learned, at the time it was a common ‘academic’ language, and still is in most of the world, though pressure from industry in North America has largely forced it to be replaced by a more mainstream language. And it’s the tools available in the latter, and how they have changed, that I will compare with it. Given that I used it in university, which was some time ago now, it has obviously been around for a while.
Although it is niche, and anything but an intrinsically embedded language, most of which lack things like GUI’s since they are intended for environments without visual interfaces of any kind, only Java, the language in which more code has been written than all others combined, is more common in embedded systems, and even then, not by a huge margin.
The difference is in the types of embedded systems, taking embedded systems as an example, that each tend to be found in. Java tends to be found in TV set top boxes, smart phones, and other user systems, things that an occasional hiccup or crash aren’t of any great significance to if they mostly work. By contrast, the language and environment I’m looking at tends to run medical devices such as MRI’s and fMRI’s, production automation equipment such as that found in semiconductor fabs, and military equipment such as automated defense systems, things where a problem can result in an unrecoverably negative result.
For no other real reason than my own liking for the environment, I created a version of software that I was writing in a more mainstream environment in it. However, without the limitations of the mainstream environment and available tools, the use cases covered by the niche version comprise more than five times the use cases of the mainstream version. In terms of time to write, by contrast, it took a total of 22 hours of development, compared with nearly three months in the mainstream environment. There are, of course, good reasons for the discrepancy: the base language itself is simple, flexible and powerful, and contains both a powerful metaphor and strong restrictions that allow the creation of software to solve a wide variety of problems in a safe, reliable way; the stability of the core language and base tools (which are themselves defined in the language spec) made it worthwhile to create not only a wide variety of first class development tools, but a full UI framework layer specifically designed for developer tools, which enhances the power and usefulness of those tools; that framework, along with a visualization framework also intended primarily for development tools, was the main basis on which I built the software I was writing, and the two in combination provided most of the facilities I needed ready-made. The other tools, built largely on the same frameworks, provided a wealth of functionality that the software I was writing can invoke to accomplish things based on the analysis it performs, which resulted in the variety and depth of use cases it can cover.
Those things are not available in the environment I wrote the original version in, and since they’re largely specific to the niche language as well as environment, they can’t be used from that environment to assist with development in other languages to nearly the same effect (although, with strict limitations, some of them can be used to analyze software written in other languages). Although similar frameworks and the tools built on them could, in theory, be written in most languages, in fact they don’t exist in any other environment.
One of the reasons other environments lack such features is that the developer UI toolkit itself took over 10 years to write, built on what was already a very good language and UI framework, while the analysis and visualization toolkits built on it took a further 11 years. My ~ 22 hours was therefore based on nearly 22 years of previous work. The mainstream language, by contrast, has only existed in a reasonably usable form for 21 years, and the most common environment for 16 years. Over that period, the language, build tools, and core frameworks have changed quite radically, so the environment has had to change quite radically as well. Despite the effort that has been put into it, and its phenomenal complexity, the constant disruption from the base language up has resulted in it still not having all the features of its predecessor, which was written in just over a year. It’s predecessor, however, was written in the niche language and environment I was just discussing.
In talking about it with a former colleague, who couldn’t understand initially why the capabilities couldn’t be implemented in the mainstream version in any reasonable period of time, I pointed out that it lacked the developer UI toolkit, and lacked the flexibility in the core UI toolkit to build anything similar, implying that it would take as long or longer to build the prerequisites, and that since it lacked many of the tools the analysis calls in the niche version to automatically fix certain things, those would have to be written as well. Yet since the environment has existed for 16 years, and is one of the most used environments in the most used language, it seemed strange to her that those things would exist in a niche environment used and developed by relatively few people.
The reason that’s the case is precisely the self-disruptive tendencies in the industry, tendencies that mirror reflexively the disruptive nature of software, as virtual, in other industries. In addition, the conflation of developer and user tools common in the industry results in the environment being built with tools that themselves are built only with user tools in mind. Further, this is exacerbated by a common trope that GUI tools such as development environments are for ‘newbies’, thus designing them as developer tools and the thinking that needs to go into that doesn’t happen, and they do become far less powerful than they ought and need to be.
There’s no reason to think that will change, since nothing suggests that the disruptive tendencies, more potent in mainstream environments because there’s nothing to keep them in check and more developers to promote them, and because competition within the mainstream developer tool space, not only by private companies but within the open source community as well, prevents the kind of community focus on creating a shared set of core technologies necessary.
In turn, this makes the spending the kind of time necessary to build such tools virtually unthinkable, since there’s no reason to believe that even if the language and environment are still available by the end of the necessary development time, they’ll be much if anything like they are now, and the continuing changes from the base language up will only lengthen that time due to the need to rewrite already written code to be compatible with each new version.
