My first experience with programming occurred at a Cub Scout meeting. To build our visions of the future, our den leader brought in parents to talk about the work that they did. My father came in with a set of 3×5 cards. We sat in a circle and he handed each of us a card. Following the instructions on the card, we were able to perform an operation of binary logic on two numbers. Believe me, we didn’t have a clue what we were doing, but the insights we gained into the nature of computing sure made us feel smart.
Perhaps forty-five years later, I now go by the airy title of “Software Engineer.” This is actually a fraud. By law, “engineer” is a designation reserved for technologists that have passed a rigorous assessment of capability. No such test exists for software. As one consequence, the lack of reliable practices for predicting cost and schedule for software development is a scandal. At one point, surveys of software projects reported that almost half failed.
Given the importance of software, a number of serious efforts have been made to attack the problem, spanning the full spectrum from machine design all the way up to the executive suites. Methods and tools, team dynamics, and project management were all brought under scrutiny and overhauled. Nothing has worked.
Looking back at that Cub Scout meeting, I must admit to some bemusement, because in fact what goes on in computers is exactly what we did with our 3×5 cards, except on a much larger scale and at incomprehensibly faster pace. It’s just a bunch of 0’s and 1’s flying around, with some conversion going on to allow people to digest the results of the calculation. Why should it be so hard to get a handle on the process?
Among the reasons are those that we might hope would be resolved in the foreseeable future. First is the enormously rapid pace of change in the industry. Moore’s Law meant that the difficulty of the problems that we could address grew by a factor of two every eighteen months over a span of thirty years – compounding the growth, that becomes a million times! In data storage and access, the factors are even larger. No other technology discipline comes even close to boasting these kinds of numbers. The rapid pace of change means that what was important and interesting five years ago is meaningless today. So how can we certify practices at the forefront of technology?
Of course, as has been reported in the technology press, Moore’s Law has progressed to the extent that fundamental laws of physics will impose limits on computing power in the foreseeable future. That may allow the test designers some time to catch up with progress.
A second cause for hope is that pattern processing algorithms are becoming sophisticated enough to effectively interpret human behavior. Some of those algorithms are used for gesture and speech recognition. Some are used to recognize our habits and track our schedule, allowing digital assistants to interpret our gestures and speech as specific instructions appropriate to the context. The near-term impact of this capability will be that more and more of us will become programmers, because the barrier to entry – learning how to “speak” languages understood by computers – will be removed.
But there are certain aspects of computer programming that will remain intractable, the foremost of them being trying to visualize clearly the outcome of a program’s execution.
My father Karl was given a consulting assignment in the ‘60s at a large defense contractor. The president had committed to automation of production operations, and the project was nearing completion. But while the developers were confident that the software would perform as specified, nobody had performed a comprehensive assessment of the impact on current operations. That was the task presented to Karl.
Of particular concern was the production scheduling office. The process transformed his understanding of programming. The walls of the large room were covered with clipboards hanging on hooks. Colored sheets were used to identify time-sensitive assemblies. Written manuals defined the procedures for movement of the clipboards on the walls. In many respects, it resembled the process he guided the Cub Scouts through a few years later. It was a program, and there was no way that the production automation system would be able to replicate its level of sophistication. When presented with the assessment, the president chose to force the scheduling team to use the new software, and production collapsed.
Fundamentally, software is just manipulations of zeros and ones. It is useful only to the extent that we can use the zeros and ones to represent the real world. For most of the history of computing, that meant that the principle role of the software developer was to translate the language of the human expert into terms that could be executed by a computer. When that knowledge was packaged and distributed, it meant that every local expert was replaced by a world’s expert, improving the decisions made by others, who then commissioned software solutions that stimulated improvements elsewhere, all at an ever increasing pace that frustrated the attempts by managers to actually control the end result.
What Karl realized in the scheduling office was that programs exist all around us. Some of them we call “habits”, others we call “software”, others we call “symphonic scores.” To realize the intentions of the people that commission the creation of the program, the programmer must describe the actions to be taken in terms understood by the executor of the program – whether a mechanic, a computer or a musician. What is common to each of those situations, however, is the fact that the program only specifies a pattern of execution. A spark plug can be changed in a driveway or in a repair shop. Microsoft Word can be used to write a love letter or a subpoena. A symphony can be performed on period or modern instruments. The result in each case may be subtly or grossly different, but the pattern of the execution is the same.
And there was no good way of representing such patterns. It wasn’t science, and it wasn’t engineering. “Programming” was as good a word as any to use.