From that ‘self-assessment’, specific guidance is then offered (’survival skills’) in the following areas:

Planning

• Hitting a moving target

• Requirements development

• Quality assurance

• Architecture

Phased Development

• Detailed design

• Construction

• System testing

• Release

My favorite part of this book is the ‘crib notes’ section, in which Steve summarizes a set of “dos and don’ts” from the Software Engineering Laboratory at NASA’s Goddard Space Flight Center:

Do’s for software success

1. Create and follow a plan

2. Empower team

3. Minimize bureaucracy

4. Define the requirements baseline and manage changes to it

5. Perform periodic reviews of endeavor’s health and progress, and adjust approaches as necessary

6. Re-estimate system size, effort, and schedules periodically

7. Define and manage phase transitions

8. Foster a team spirit

Don’ts for Software Success:

1. Don’t let team members work in an unsystematic way

2. Don’t set unreasonable goals

3. Don’t implement changes without assessing their impact and obtaining approval of the change board

4. Don’t gold-plate

5. Don’t overstaff, especially early in the project

6. Don’t assume that a schedule slip in the middle of a phase will be made up later

7. Don’t relax standards in order to cut costs or shorten a schedule

8. Don’t assume that a large amount of documentation assures success

In it, he proposes that an organization of any significant scale cannot be understood through the typical approach of functional decomposition. Instead, Beer suggests examining living systems in order to better understand how the form, functions, fitness, and interactions of their parts contribute to the purposes and fitness of the system as a whole. He argues that in the absence of meaningful adaptive control, bureaucracy emerges from the logic of collective action over time.

Beer's later works followed on this theme and developed it further. In particular, Diagnosing the System for Organizations provides a more succinct summary of an underlying method to deploy his ideas, while in The Brain of the System, he dives more into the theory behind these methods. Both build upon his work on Operations Research, and his book, "Decision and Control" (a lost art except in production control situations).

Recursion

In The Brain of the System, Beer spends quite a bit of time explaining Figure 1, one of many diagrams which he uses to explain his notion of a viable system. Each such system is grounded in recursion, or as Beer says, "Every viable system contains viable systems and is contained in a viable system." One of his students recovered an audio recording from one of his lectures in 1990 (part 1 and part 2) on Intelligent Organizations. These lectures provide a good overview of both books and are worth sampling before committing the time it takes to digest the books themselves.

Beer uses Figure 1 as a notational metalanguage to describe the minimum orientations necessary for effective organizational design. In this notation, operations are depicted as circles, a control hierarchy (aka management) is portrayed using squares, and governance is represented by triangles. If you examine this figure, you can see that recursion (in which each immediately lower level contains all the levels below it), and all can be subsumed in a higher aggregation. This enables his model to be self-referencing, which characterizes situations in which each part makes sense precisely in terms of the other parts.

This means the whole cannot be completely defined by the parts themselves, but rather by their interactions. This complexity and associated unknowns of the associated activities, despite best efforts, is what makes planning and tracking tasks themselves complex. Such systems also are susceptible to oscillation, a phenomenon demonstrated in systems thinking workshops that employ devices like the Beer Distribution Game.

Beer introduces this metalanguage and its associated concepts gradually, relating each to the structures and functions of the brain and nervous systems, and observations about their design and behaviors. He suggests the overall purpose of a living system is survival of its species, rather than of a particular individual, which may be unsettling for selected agents. His notion of survival involves retaining identity as an independent entity over extended periods, since survival requires adaptation to environmental changes; he expects the forces of fitness for use, learning, adaptation, and creative destruction will shape the situation at any point over this journey. Though he doesn't quote Galileo Galilei directly, his argument rests on a principal Galileo first stated as "Nature... doth not that by many things, which may be done by few."

Beer goes on to directly confront the underlying constraints encountered by most reorganizations - their fuzzy scope, political gaming, periods of uncertainty, and cultural resistance to change. He also relates the goal of survival to a firm’s need to protect its ‘seed corn’ and the need to optimize its transfer functions to improve its performance over time; in other words, transformational operations cannot consume all available resources for itself.

His depiction of how hierarchical structures, intelligently designed, enable efficiencies in searching for solutions, and mirrors a pattern recently noticed elsewhere, such as in an MIT article relating the physics of the fabric of the universe, and in recent breakthroughs in machine learning. The shortest path for reducing uncertainty to its minimum size involves uses of binary chop algorithms to evaluate binary classifiers for recognition, selection, and decisions.

Stanford argues for the design of a control system for organizations involving a focus on inputs that are optimally converted to outputs, and in which sensors, transforms, and redundancy are used to offset noisy communications channels, limited bandwidth, and communications latencies. He notes that our own central and peripheral nervous systems (see Figure 2) are a mixture of such central and distributed control.

He suggests his principles are applicable to smaller organizations, entire firms, and even societies, and even attempted this computationally with far less computing power than a single desktop offers today. He introduces a useful term (the 'system of study') which is like the architectural concept of a system of interest, and which he uses to place observers in a variety of hierarchical contexts. This reorienting frame of reference allows one to 'zoom in and out'; there is always a higher authority, and there is always a changing environment, all the way down.

He argues that in this progressive descent, each system of interest must autonomously strive to achieve homeostasis, a stable, self-regulating state of balance within its internal environment (interfacing with its peer organizations) while adapting to the external environment (where it has kinship with its peers):

It is necessary that large areas of any such complex organization should in fact be autonomous. If every aspect of the business, every smallest decision, had to be thought about consciously at the senior management level, then the firm would grind to a halt rather quickly. It is the same in the body and the same reasons apply. Both systems operate autonomic control, which is to say a level of management which does not involve conscious direction by the organism as a whole.

From the point of view of the whole organism, whether body or firm, the autonomic function is essentially to maintain a stable internal environment. This idea, called "homeostasis', really is necessary in any viable system. Neither brain nor board could press on with prosecuting a deliberate policy if the internal organs were running amuck. The well-ordered production machinery must not overheat, whether in terms of men or machines; cost and quality must be kept within physiological limits, which is to say that they must vary within a range narrow enough for the health of the whole organism to tolerate; and stocks of inter-process materials must be kept small enough to avoid idle time. The company board expects that its internal management can cope with these matters, and the conscious part of the brain expects the same mutatis mutandis of its autonomic nervous system.

Design issues

He starts his exposition on designing organizations by cataloging the problems most businesses face, and by poking holes in conceptual schemes:

The primitive act in all conceptual schemes for choosing sets is the simple, finite act of enumeration - we "set" them down as it were. We may do this by actually producing all the members of the set for inspection, as a chess set or a set of teeth. Usually, however, we specify a set of names, which is taken to represent some set of "things". We rarely enumerate the collections for any which forms the conceptual basis for our thinking. Enumeration, which forms the conceptual basis for the other operations, has perils of its own, but these are as naught compared with the possibilities for mischief in derived methods.

and heuristics:

We have a scale of ascending values for heuristic devices, depending on how far you go before you must stop. Going from the narrow to the broad, we find ideas, concepts, rules, principles, laws, reality, and truth. The further along this scale, the less we notice that a heuristic device is a device.

The nastiest of these methods is the representation of a set by a typical member. This method rests on the assumption that the set can be typified, an idea that goes back at least to Plato. Platonists argued that the ideal type is a better representation of the set than any enumeration could be, since the actual members of a set could at best be faulty realizations of the ideal type. The ideal type, however, is strictly an observer's mental construction, which may be a useful way to summarize a mass of data, but as taxonomists have discovered, it may simply be the path to a decomposition fallacy.

Beer also argues that oversight has limits:

If I want you to know in complete detail what I have been doing for the last hour, then I shall want exactly one further hour to explain. If I have ten colleagues, and cannot see on average more than two of them at once, then I shall need five hours to explain my one hour. It is just the law of requisite variety. If can afford no more than ten minutes in explaining myself for every hour worked, then I shall devote two minutes per colleague, and there will be a ratio of 30:1 in the reduction of variety between myself and them. Some lethargic managerial societies seem to work quite smoothly on this basis, for the very simple reason that for them this turns out to be also the ratio of clock time to useful working activity. But a man who really is doing an hour's work in every hour bound to lose in intelligibility.

Next, we must note that this applies in the best of all worlds - one in which people love each other, and are completely determined to share their understanding. But human nature is not like this. Even the most willing of us find himself antipathetic, in varying degrees, to some of his colleagues. Even the most innocent of us occasionally succumbs to political motives which make him a deliberately poor communicator.

The conservatively minded business men among us will have none of this somewhat ruthless analysis. There is no need, for goodness sake, for me to tell everyone else in detail what i am doing. I am paid my job properly and all that anyone to needs to know is what I think they ought to know about the results of my work. And yet this is just the trouble. A viable organism works as an integral whole. A typical business is integrated too late, and too little.

and being mindful of innovation dilemmas:

Adventurous ideas soon become constrained by the observed fact that their proponents cannot think them through to a proper conclusion, it would seem, and the old ideas prevail. It is not because they are successful: they are not, and the world is in a worse mess to prove it.

The first reason as to why adventurous ideas often fail, and old ideas prevail is benign. It reflects merely on human weakness and inadequacy. It is not a theory of conspiracy, which declares that sinister forces are mustered against any kind of innovation. And yet there is a malign explanation as well... The word malign means simply inimical to viability. We do not have to be paranoid to recognize that we are actually ill.

The second explanation says that the new idea is not only beyond the comprehension of the existing system, but that the existing system finds it threatening to its own status quo. Of course it does. That is not necessarily because of its determination to hang on to power, although that is often a factor. It is mainly because the existing system does not know what will happen if the new idea is embraced.

The first explanation said that the innovator fails to work through the systemic consequences of the new idea. The second explanation says that the Establishment cannot either, and with better reason and, what is more, that it has no incentive whatsoever to do so. It was not its own idea for heaven’s sake. The onus is on the innovator. That seems perfectly reasonable, until we remember the power equation: it turns out that the Establishment controls the resources that the adventurous idea needs.

Too often, he argues, such new gloss - call it the new miracle technology - on old ideas:

It is against this background that management confronts the electronic computer. This instrument offers management its own technology B, something which makes the managerial world utterly different. But management has addressed itself to the possibilities in a way which virtually precludes the emergence of a new managerial order. It has tried instead to assimilate the computer into managerial technology A - improving, or let us say simply souping-up, the ways of regulating matters with which managers are already familiar.

What could they really be expected to do? This question was answered according to temperament, but many managers suspected two things. Computers might turn out to be incomprehensible to the manager himself, and therefore a substantial personal threat; in any case, the cost might be ruinous. But the good manager is made of sterner stuff than this. In the second stage he very properly came to grips with the nature of the machine and made a serious effort to understand its basic method of operation. He soon found out that the machine is a moron. Not only did this discovery remove unjustifiable fears, but it took away all sense of wonder, and that was a pity.

Although present-day computers fall very far short of the human brain in many capabilities, they are in just as many ways very much superior to the computers in our skulls. But in this second phase people lost sight of the fact they fell to discussing rather trivial problems about the relation between and consulting office machines and scientific machines in terms, for example, of the input/output requirement. Thus the managerial issues rapidly became political, because people used these trivial arguments to justify different computers in the office and the research laboratory, and a different computer again in the production context. Anything which inflames the appetite for empire building not only becomes a vice, but detracts from the issues which ought to be discussed.

For the manager, this was to be the age of electronic data processing referred to by the slick acronym EDP. Regardless of the purposes to which processed data would be put, all effort was now focused on the argument whether more and better data could be provided faster and more cheaply by installing a computer or by streamlining orthodox clerical procedures. This done (and of course this is a process that still goes on) some managers decided to go ahead and install computers. And that brought us to the third phase, in which most businesses remain. There is a rather widespread use of computers in the role of new lamps for old. Routine office work is done by machines; sometimes staff have been saved, sometimes not. More and better output has been obtained; sometimes people have known how to make use of it, and sometimes not. A variety of benefits has been sought; sometimes money has been saved, but all too often the pay-off has been negligible. Many who introduced computers during phase two became disappointed in phase three while many who did not came to feel that they were well out of it.

Generally, these evangelized miracles have limitations that constrain realizable benefits, leaving those who chased a particular technology epoch stuck with the bill. This phenomenon is similar to ownership of units in a condominium which may be stuck with escalating costs, should the property developer be unable to sell enough units of the units, leaving a fraction of the people stuck with paying for all of the support and maintenance.

Beer then introduces his viable system model for organizations, and his representation of a theory of viable systems, which each share the collective purpose of survival, even at the expense of their individual elements. Firms, he suggests, are such constructs. Adaptability is crucial to their survival over the long term, and this is an essential characteristic of their design. Yet the anatomy of management limits the attention paid to each part of the organization. Because of variety, one of his foundational arguments, and a measure of the complexity of the interactions in the system, the absence of a learning model such as pattern recognition requires excess and unnecessary variety. This leads to the combinatorial explosion of states across all parts of a system, which are impractical to even count, though their ideal state is captured by Ashby's Law of Requisite Variety. Despite this complexity, each part may still be comparable with others to rank which have more variety than others, and through feedback, determine whether signal attenuation or amplification is required.

He uses the triggering and responses of an organism as an analogy, as it responds to situations through a set of voluntary and involuntary actions. Vertebrate species employ sensing, cognition, decisions, and action using the pathways of the central nervous system, which includes the brain, spinal cord, and peripheral nervous system, to connect it to every other part of the body. The peripheral system is, in fact, a system of systems, which consist of:

• a somatic nervous system, which mediates voluntary movement

• an enteric nervous system, which controls the internal systems such as the gastrointestinal system.

• an autonomic nervous system, which is further classified into two subsystems: a sympathetic nervous system, which normally maintains homeostasis, and provides fight or flight responses involving rapid mobilization of energy; and a parasympathetic nervous system, which operates when organisms are in a relaxed state, to "rest-and-digest" or "feed-and-breed".

The primary function of the central nervous system is to send signals from one cell to another, and through that mechanism, from one part of the body to another, with feedback on the results. Both the autonomic and enteric nervous system function involuntarily, whereas the somatic nervous system controls the muscles to make voluntary movements and provide the reflex arcs of the body's neural pathways. Signals travel over separate pathways depending upon which direction information is being sent; those sent from the brain pass over efferent nerves, while information transmitted to the brain from the body is transmitted over afferent nerves. Signal transport itself is performed by special cells called neurons, in waves passing over thin fibers called axons, and using signals generated by electrochemistry.

Signal modulation

Each junction, or synapse, excites, inhibits, or modulates these signals, providing either amplification or attenuation. Collectively, these subsystems form circuits and neural networks that form an organism's perception of the world.

In his viable system model, he deliberately reorganizes hierarchies of command:

An algorithm is a technique, or a mechanism, which prescribes how to reach a fully specified goal. A typical air pilot's flight plan is an algorithm. The instruction: turn left at the crossroads, take the second turning on the right, turn left at the Red Lion and our house is 120 yards up on the right' is an algorithm. A method for finding the square root is an algorithm, and so is a computer program. This last is important, because we shall soon have to clear up some confusion about the capabilities of computers. A computer can do only what it is precisely told to do. The programmer has to write an algorithm, then, which will exactly determine the computer's next move in any set of circumstances whatever.

An heuristic specifies a method of behaving which will tend towards a goal which cannot be precisely specified because we know what it is but not where it is. Suppose you are trying to reach the peak of a conical mountain enveloped in a cloud. It must have a highest point, but you do not know the compass bearing. The instruction: "keep going up' will get you there, wherever ‘there' is. ‘Take care of the pence and the pounds will look after themselves' is an attempted heuristic for "being wealthy". Heuristics prescribe general rules for reaching general goals, and do not typically prescribe an exact route to a located goal as does an algorithm. There are after all an infinite number of paths up the mountain, and it does not matter much which path is taken (although some routes may be quicker than others).

These two techniques for organizing control in a system of proliferating variety are really rather dissimilar. The strange thing is that we tend to live our lives by heuristics, and to try and control them by algorithms. Our general endeavor is to survive, yet we specify in detail... how to get to this unspecified and unspecifiable goal. We certainly need these algorithms, in order to live coherently; but we also need heuristics and we are rarely conscious of them. This is because our education is planned around detailed analysis: we do not really understand things unless we can specify their infrastructure. The point came up before in the discussion of transfer functions, and now it comes up again in connection with goals. "Know where you are going, and organize to get there' could be the motto foisted onto us and on to our firms. And yet we cannot know the future, we have only rough ideas as to what we or our firms want, and we do not understand our environment well enough to manipulate events with certitude.

Birds evolved from reptiles, it seems. Did a representative body of lizards pass a resolution to fly? lf so, by what means could the lizards have organized their genetic variety to grow wings? One has only to say such things to recognize them as ridiculous, but the birds are flying this evening outside my window.

If the goal is not recognized in detail, an heuristic is required, so the computer must be supplied with an algorithm determining an heuristic. That is a basic trick. Suppose we say: ‘The computer must learn from its own experience, as do we ourselves.' Learn what? We do not know; what we meant was that the computer must find out over a period, by trial and error, the courses of action which lead to better results of control. We shall say what is a better and what a worse result, but the computer has to determine a better strategy, a better control system, than we ourselves know. And of course it can do it. Because its algorithm, what it is programmed to do, specifies an heuristic. Alter the solution you are now using a little bit, says the algorithm, and compare the outcome with the erstwhile outcome. If this is more profitable, or whatever else we say, adopt it. Go on like this until variation you make leads to a worse result than you already have. Then any you may hang on to this solution, until the situation changes; whereupon you may do better once again by producing a new variation.

This example could be continued indefinitely. The point is that heuristic techniques are determined within a framework specifying the mode, the limits, and the criteria of search. And if that framework is itself an heuristic, then it too requires a framework; and so on indefinitely. At some point the nth framework must be reached which, from this system's internal standpoint at least, will have to be declared an absolute framework. In good logic, this cannot be done; but in all practice it has to be done. Hence all finite systems are limited and incomplete. We ourselves, our firms, our economies all suffer from this limitation. And because we do, the best possibility for change directed towards ever more successful adaptation lies in a reorganization of these hierarchies of command.

He strives to dampen oscillations, accommodate limitations in information exchanges over channels, and reinforce the accountability of agents. The individual 'systems' are conceptualized as multinodes which he paints as elaborate functioning collections of unreliable elements interoperating to deliver results.

At each level in such sense and respond systems, a basic set of decision elements must be attended to:

It is interesting to begin the analysis of hierarchical control structures by asking about the basic decision elements of which ranks and orders of command are in general composed. In nature, if we consider that most sophisticated control system, the brain, this element might be identified as a single nerve cell or neuron. In industry or government indeed in any strongly cohesive social group the element is some sort of manager.

Both the neuron and the manager have one really basic task to perform: to decide. In the neuron's case, a pulse must either be triggered down the output nerve or not. For the manager, the fundamental task is also to say yes or no. It true that managers do not spend their lives uttering these two words; they may never utter them. None the less, this is their task, and the subtleties, the nuances, the might-I-suggests and the perhaps-you-woulds are really socially intricate ways of saying yes or no.

In order to reach a binary decision, the decision element has to establish a threshold of decision. We may think of it as saying 0 until it is prompted to say 1 instead. This would be a permissive kind of management, in which the decision element does nothing until activated. It must not be activated by any stray impulse or noisiness that happens to be floating around the system, and this fact establishes the need for a threshold. Over-sensitive neurons would soon send either men or firms mad. When things really begin to happen, the decision element accumulates its evidence. When it is sure that there is real evidence demanding action, which is to say when the sum of inputs exceeds a threshold value, it fires.

This doesn't mean organic systems are perfect. Malfunctions arise because of genetic defects, physical damage, infection, and age; each of these has their complements in dysfunctional organizational behaviors. Our bodies protect against such malfunctions by redundancy and resiliency. For example, brains are amazingly capable of re-purposing components to bilateral functions when needed. However, as argued elsewhere, 'thinking fast and slow' mechanisms act differently, and are wired differently, with ramifications that must be factored into the effective design of systems.

Effective multi-level control

Beer suggests an alternative measurement approach that relies upon a multi-level structure to inject adjustments which will counter oscillations, tune performance, and provide for the greatest utility. These measures are depicted in Figure 3, and described here:

The dynamics of this whole structure depend on the quantification of its performance. Hitherto business has used the measure of money and price, the direction and rate of cash flow. Thus we have come to the quantification of business activity, in the corporations we know, the cost-accounting function. This is because cost accountancy provide lingua franca by which the disparate activities of unlike divisions may compared and aggregated. There is no reason why this should be so, beyond its historicity and (alleged) familiarity.

Divisional performance is about both short-and long-term viability. The notion that cost should be minimized or profit maximized within a fixed epoch leaves right out of the count other factors which are vital to the future viability of the business contained within the division. They are the latent capabilities of the firm, which may be built up and metabolized by wise management, or squandered recklessly by stupid management, without in either case procuring a change which is reflected in costings. For costings are short-term control instruments, and will not detect the mismanagement of latent resources. By definition, this mismanagement will not be detected until it is too late.

We need a measure of achievement which is both less 'loaded’, in terms of profits, and which is more comprehensive, in terms of the real resources at risk. If money is not the unit, then we must think in terms of pure numbers. There is a classic measure of productivity which can be extended. It expresses the ratio of what is possible to what is actual. Now we may deal with the problem of incorporating latent resources by a slight elaboration of this theme, which requires us to define three (rather than two) levels of achievement. They are:

• actuality: This is simply what we are managing to do now, with existing resources, under existing constraints.

• capability: This is what we could be doing (still right now) with existing resources, under existing constraints, if we really worked at it.

• potentiality: This is what we ought to be doing by developing our resources and removing constraints, although still operating within the bounds of what is already known to be feasible..

There can be no argument about the numbers used to measure actuality. There will be severe arguments about the other two sets of numbers. But if we use good operational research it becomes possible to gain agreement that the numbers used are sensible and the process of investigation and discussion is itself highly beneficial. What matters is that capability and potentiality measures, though somewhat arbitrarily fixed, cannot then be arbitrarily changed. Hence although the absolute values of the productivity and latency indices provide only approximate assessments, movements of these indices over time provide the information that we really need.

Now of course we may project our future plans on the basis of any one of these notions of achievement, or indeed have three sets of plans which employ these three criteria. Planning on the basis of actuality I call programming. Planning on the basis of capability I call planning by objectives. Planning on the basis of potentiality I call normative planning. The first of these is simply a program because it accepts the inevitable shortcomings of the situation, and does not admit that anything can imminently be done about them. Programming is a tactical ruse. We move to genuine planning only when we set new objectives and try to achieve them. This is the strategic planning level.

Normative planning sets potentiality as its target and incurs major risks and penalties, although it also offers major and perhaps decisive benefits. But, however we plan, what really matures is always called actuality, and the measures of achievement proposed relate capability and potentiality to whatever may be actual at the time. Here are some more definitions:

• productivity: the ratio of actuality and capability;

• latency: the ratio of capability and potentiality

• performance: both the ratio of actuality and potentiality, and also the product of latency and productivity.

Potentiality is always better than capability, which is always better than actuality. But if we are talking about profit, for example, 'better' means 'more', whereas if we talk about the number of men required to do a job, 'better' means 'less'. Consider, then , the question of what happens to the achievement indices when we go on doing what we have always done when the ultimate possibilities remain the same. This it may do by undertaking work study of processes, negotiating new agreements with the unions, raising the morale or improving the quality of managers, and so forth. What happens is that the latency measure improves (because capability is approaching potentiality) and productivity is lowered. But if potent management can in these circumstances improve the actual performance, as obviously it should, all three measures of achievement will rise.

The nodes of these systems of interest must be designed with the means to use such information, using what he calls algedonic feedback, in the form of alarms and triggers that cascade up through a command backplane anytime a request for information or reporting has timed out, fails to produce needed results, or exceeds the performance limits established in resource bargains which previously were struck.

Beer warns that without such a performance envelope, nodes or larger entities may end up competing with each other, rather than pulling together; at any scale, this quickly leads to an unstable overall system. Note that portions may still do well because of fitness to a particular set of circumstances, but may not be able to adapt to emerging circumstances:

The viable system is a system that survives. It coheres; it is integral. It is homeostatically balanced both internally and externally, but it has none the less mechanisms and opportunities to grow and to learn, to evolve and to adapt - to become more and more potent in its environment. In all of this the viable system may succeed sensationally, spectacularly fail or it may muddle along. The amoeba succeeded, the dinosaur failed, the coelacanth muddles along.

You and I have our own problems of survival. As to the firm, as to government, as to society, as to the future of mankind all viable systems we shall see. Structural change is so potent a business, and so traumatic to undergo, that people prefer to pretend they cannot see what their own eyes insistently report, rather than commit themselves to the re-shaping which is necessary.

Organizations cannot face up to more than a quarter of the reshaping that their long-term viability demands. This is of course the reason why so many enterprises are in a state of continuous, never mind continual, reorganization. People pretend that the great upheaval is almost complete: it never is, and the viable system becomes increasingly unstable as a result. This in turn makes every enterprise vulnerable to attack. When the management or the government has fallen, when its policies are in disarray and its people in despair, we can pretend no longer.

Beer is a critic of attempts to use computers as change agents; instead of clarity, he argues, such efforts often lead to information overload and measurement without control, which just produces waste and distraction. His insights have relevance to other general-purpose technologies, such as AI, as well:

It is against this background that management confronts the electronic computer. This instrument offers management its own “technology B”, something which makes the managerial world utterly different. But management has addressed itself to the possibilities in a way which virtually precludes the emergence of a new managerial order. It has tried instead tried to assimilate the computer into managerial technology A - improving, or let us say simply souping-up, the ways of regulating matters with which managers are already familiar.

What could they really be expected to do? This question was answered according to temperament, but many managers suspected two things. Computers might turn out to be incomprehensible to the manager himself, and therefore a substantial personal threat; in any case, the cost might be ruinous. But the good manager is made of sterner stuff than this. In the second stage he very properly came to grips with the nature of the machine, and made a serious effort to understand its basic method of operation. He soon found out that the machine is a moron. Not only did this discovery remove unjustifiable fears, but it took away all sense of wonder, and that was a pity.

Although present-day computers fall very far short of the human brain in many capabilities, they are in just as many ways very much superior to the computers in our skulls. But in this second phase people lost sight of the fact they fell to discussing rather trivial problems about the relation between and consulting office machines and scientific machines in terms, for example, of the input/output requirement. Thus the managerial issues rapidly became political, because people used these trivial arguments to justify different computers in the office and the research laboratory, and a different computer again in the production context. Anything which inflames the appetite for empire building not only becomes a vice, but detracts from the issues which ought to be discussed.

For the manager, this was to be the age of electronic data processing referred to by the slick acronym EDP. Regardless of the purposes to which processed data would be put, all effort was now focused on the argument whether more and better data could be provided faster and more cheaply by installing a computer or by streamlining orthodox clerical procedures. This done (and of course this is a process that still goes on) some managers decided to go ahead and install computers. And that brought us to the third phase, in which most businesses remain. There is a rather widespread use of computers in the role of new lamps for old. Routine office work is done by machines; sometimes staff have been saved, sometimes not. More and better output has been obtained; sometimes people have known how to make use of it, and sometimes not. A variety of benefits has been sought; sometimes money has been saved, but all too often the pay-off has been negligible. Many who introduced computers during phase two became disappointed in phase three while many who did not came to feel that they were well out of it.

Enough has happened in the computer world to demonstrate that these machines are now permanently with us. History has painfully demonstrated that once mankind knows how to perform a function by machine, the machine is in and man is out. And yet there is disappointment, and the economics of the whole business look somewhat rocky. The answer to the dilemma is becoming clear. Too many managers have been dazzled in EDP terms by the “more and quicker argument, with the result that little fresh thought has been given to the purposes which the information duly handled is supposed to serve. This... is management information. And so, the magic letters EDP are being replaced by the equally magic letters MIS, management information systems. This certainly appears to be another advance. It looks as if it takes seriously the question about the purposes of EDP.

But in reality, we become more and more embedded in managerial philosophies of the past. We continue to replace one thing by another which is indeed more effective, and now we have a great vision whereby all these bits and pieces will be integrated in a vast informational network. The whole firm will run on a basis of ‘instant fact’ because managers will draw any item of knowledge they require from a huge database into which all the facts about the business will be poured. I shall show explicitly why this vision of the future is actually incapable of fulfillment. The argument for the present rests on the fact that. even if these prognoses were reasonable, we should still have missed the point.

Items of fact about a business are profuse. They proliferate with every second that passes. Most of them are worthless in the sense that they have no bearing on managerial decision. By recording them, sorting them in different ways and printing out huge quantities of tables, nothing useful is accomplished. On the contrary, managers become engulfed in a sea of useless facts. Doubtless some valuable facts may be included, but if so they will be lost without trace. The manager wants information, not facts, and facts become information only when something is changed. The manager is the instrument of change (otherwise what is he doing?) which is to say his job is that of control. This means that the job is not to design a data-processing system at all, but to design a control system. And if we use the computer simply to undertake a souped-up version of the old kind of control system, which was inadequate simply because we did not have computers, we are no better off than before. It is the same with our planning techniques, which are part of the manager’s control armory, and which so desperately needs to be in the context of technological change. For again we are concentrating on ways of doing things rather than on what we do. What is the use of the ever-faster, ever-slicker, more nearly perfect implementation of rotten plans?

Even when measurements are intelligently designed, research indicates you'll only get what you measure, Worse, the measures will get gamed, leaving you worse than before.

Beer died in 2003, though his methods live on, as does the pathfinding he performed in introducing mathematical decision support systems. These begat management information systems, which begat executive information systems. then data analytics, and now reinforcement learning. Each is a shiny object marketed as the next new thing yet quickly becomes one more challenging thing to try to get working adequately for some constrained mission.

In roughly the last third of the book, Beer describes efforts to prototype these concepts in a decision support system for a state-run enterprise, which was launched during the presidency of Salvador Allende, who led Chile in the early 1970s. The endeavor, Project Cybersyn, had the goal of managing Chile's political system within their socialist framework that provided protections for the autonomy of workers, rather than imposing a smothering, top-down command-and-control system. He anticipated the failures of the Soviet system yet offered a model that would work at all the levels depicted in figure 4.

One of Beer's associates at the time was Fernando Flores, who would go on to become a pioneer in the field of artificial intelligence, along with Terry Winograd at Stanford; both have been at the forefront of progress in cognitive science over the last thirty years. Beer recognized that recursion was the only structure possible by which to manage the complexity necessary to sustain living systems and argued that the trends in increasing complexity would not end well unless appropriate shock absorbers were designed into these systems from the start.

Barry Diller, the chairman of IAC/Interactive Corp., was at Harvard Business School explaining the rationale behind the mosaic of interactive commerce companies he had assembled at IAC, such as Ticketmaster, Hotels.com, Match.com, and LendingTree.com. One of the students pointed out that these various businesses seemed to be operating independently, not in a coordinated synergistic fashion.

Diller erupted in mock anger. "Don't ever use that word synergy. It's a hideous word," he said. "The only thing that works is natural law. Given enough time, natural relationships will develop between our businesses."

I agree. What applies to disparate parts of a giant company also applies to disparate people in an organization. You can't force people to work together. You can't mandate synergy. You can't manufacture harmony, whether it's between two people or two divisions. You also can't order people to change their thinking or behavior. The only law that applies is natural law.

The only natural law I've witnessed in three decades of observing successful people's efforts to become more successful is this: People will do something - including changing their behavior - only if it can be demonstrated that doing so is in their own best interests as defined by their own values.

New product development efforts are motivated by the belief that a common solution can be profitably developed to solve a problem that users need to have solved. Synergy initiatives attempt to extend this concept, and base their value proposition on two related beliefs: aggregation of needs from multiple domains will achieve greater economies of scale), and greater reuse of knowledge and assets can be applied to service those needs more effectively (typically by being smarter at avoiding sources of unnecessary cost, removing redundant solutions and excess capacity, and by exploiting the best available solutions and strategies available within the community). The elements targeted for this reuse can take many forms, including the sharing of tools, technology, processes, concepts, components, people, and information.

The unique aspect which most synergy efforts introduce is that this sharing is often pursued through novel organizational schemes. These new approaches attempt to provide multi-dimensional focus on products, architectures, value streams, processes, and time horizons. The work inherent in this new focus often must be done in parallel (or at a lower priority) than the activities which implement the organization's continuing primary mission. Unrealistic expectations may be set to just 'do it all', without providing any additional resources for providing solutions which satisfy these added dimensions, or without the disciplines that would accompany normal project efforts. Both of these situations ignore the fact that the bigger and more complex the work is that an organization must do, the lower the overall productivity of the people who will be in pursuing it; as an example, consider this analysis of software estimating factors, which documents that as much as an order of magnitude reduction in productivity can occur on large projects over small ones.

The ability and time horizon to realize value from such synergy initiatives rests upon a number of important assumptions:

1. A catalog of reusable components, products and services exists that can satisfy the aggregated needs for a target market with only incremental investments. If this assumption is not true, you do not have a synergy project, but instead, have a large and complicated development program with many (and likely changing) dynamic constraints.

2. Requirements exist which accurately characterize this catalog; these requirements are coherent and have complete overlap with end-user needs. Such requirements are necessary to provide a well-formed basis to evaluate gaps between the 'as-is' and the 'to-be' perspectives of stakeholders. As a result of this overlap (often traceable to a common heritage from which these items were designed), prior investments have produced results that are collectively useful and affordable (and thus can be expected continue to do so in the future).

3. These requirements enable decision-making on investments, design, integration, utilization, and support of solutions. Each of those decisions is essential to enable stakeholders to shape the fitness of existing and proposed solutions to satisfy end-user needs.

4. Components from this reuse catalog are compatible with technical and business architecture. This architecture is essential because it is the glue that will enable components (both organizational and physical) to be made sufficiently robust to enable 'plug and play' engagements with the user community in delivering value to them over time. The less robust or homogenous this architecture is, or the more difficult it is for contributing organizations to 'plugin' their various parts, products, and services to deliver value to customers, the more variants that will have to be accommodated in operational configurations (typically through special, costly interface layers), or by implementing additional tooling, training, and translations, as 'value-harvesting' is attempted by developers and users of these components.

5. There are redundant or underutilized resources available while operating and evolving the system within the scope which the synergy effort is intended to exploit. While it may be possible to do more with less over time, it is only possible if there are people and money available to make these changes.

6. Stakeholders will agree to commit adequate investment funds to align their own resources, support development activities, evaluate candidate products, and align strategies to the new (interdependent) way of doing business. This includes the leadership in all affected organizations and the customers of those organizations, who must 'buy-in' to the benefits of the value propositions which the effort intends to pursue. You can't just roll out products - the community of customers must be willing to invest in new ways of doing business.

7. Governance is sufficiently robust or can be developed to efficiently allocate resources and manage planning, development, distribution, and support so the goals of the synergy effort can be achieved. This governance will need to rapidly respond to changes in demand, must allocate limited resources to the most important work over time, and must ensure that investments will reliably return expected business results once an endeavor is selected and launched. Conditions are dynamic, and no one will be smart enough to allocate resources properly upfront. So the means of distributing resources for operations and improvements must be eff

8. There must be an evaluation approach that facilitates the efficient and effective selection of the best approaches to satisfy stakeholder needs (both supplies of talent, suppliers of new components, and technology) from among available alternatives.

9. There are efficient and effective mechanisms in place to track benefits, evaluate tradeoffs, shape strategies, and accurately distribute their associated costs to the organizations responsible for these decisions, so the underlying business drivers of the synergy efforts can be managed and operated as a closed-loop system. Such mechanisms enable stakeholders to meaningfully invest in capabilities beyond the existing features needed for local users, and manage varying rates of products and services, so each provider of parts, services, and architectures can continue to service the collective organization's and broader user community's needs over time, without unnecessarily burdened 'network' costs.

If these assumptions are not valid, it is possible for the appearance of synergy to occur ('Look, everyone's working together!'), while underlying inefficiencies continue to persist. Without these features in the environment in which synergy is expected to emerge, extra efforts will be required (requirements elicitation, refactoring, etc) to realize benefits from a synergy initiative. This additional effort will erode the value which can be realized and may even tip the endeavor over from being useful to being harmful. This value erosion is not immediately apparent because synergy is usually just an abstract concept; as a result, while it may sound from the 'buzz' of an initiative that you are generating a lot of synergies, you may not actually be able to harvest the commensurate value you thought you could once the details are all accounted for, and all the required transformations are made.

This is why establishing a basis for measuring results is so critical for all commitments related to synergy initiatives, and why disciplined follow-up and accountability must accompany commitments to pursue all synergy endeavors. Since the assumptions upon which commitments are made may not be valid, the resulting activities should include special efforts towards validating the endeavor's assumptions, or mitigating them as risks, since these risks may severely limit the returns which can be harvested from the synergy initiatives themselves.

The articles in this series consider the above assumptions about synergy, explore the synergy value proposition (and the dynamics which it can exhibit over time) in more depth, discuss case studies and examples of pursuing synergy in practice, and make recommendations on approaches which can enhance the feasibility of achieving your target objectives from such 'common benefit' solutions.

• discover and connect to customers

• manage the resources necessary for activities in development, production, and operations

• organize these activities into effective flows

• perform the work necessary to accomplish those activities quickly and efficiently

• evaluate and improve how this work is being performed, and

• enhance the effectiveness of running the business overall

Even the owner of a lemonade stand needs to attract seed money (or a sponsor) to cover pre-launch expenses; once launched, enough capital must continue to be attracted to warrant continued investments in labor and materials for operating expenses. In parallel, the means of production must be established, maintained, and evolved at a steady pace for growth to occur, or at a minimum, for mere survival. Herbert Simon, computing's first economist, describes the steep path this involves in his book, Sciences of the Artificial:

The idealization of human rationality is enshrined in modern economic theories, particularly those called neoclassical. These theories are an idealization because they direct their attention primarily to the external environment of human thought, to decisions that are optimal for realizing the adaptive system’s goals (maximization of utility or profit). They seek to define the decisions that would be substantively rational in the circumstances defined by the outer environment.

Economics exhibits in purest form the artificial component in human behavior, in individual actors, business firms, markets, and the entire economy. The outer environment is defined by the behavior of other individuals, firms, markets, or economies. The inner environment is defined by an individual’s, firm’s, market’s, or economy’s goals and capabilities for rational, adaptive behavior. Economics illustrates well how outer and inner environment interact and, in particular, how an intelligent system’s adjustment to its outer environment (its substantive rationality) is limited by its ability, through knowledge and computation, to discover appropriate adaptive behavior (its procedural rationality).

The question of maximizing the difference between revenue and cost becomes interesting when, in more realistic circumstances, we ask how the firm actually goes about discovering that maximizing quantity. Cost accounting may estimate the approximate cost of producing any particular output, but how much can be sold at a specific price and how this amount varies with price (the elasticity of demand) usually can be guessed only roughly. When there is uncertainty (as there always is), prospects of profit must be balanced against risk, thereby changing profit maximization to the much more shadowy goal of maximizing a profit-vs.-risk “utility function” that is assumed to lurk somewhere in the recesses of the entrepreneur’s mind.

In real life the business firm must also choose product quality and the assortment of products it will manufacture. It often has to invent and design some of these products. It must schedule the factory to produce a profitable combination of them and devise marketing procedures and structures to sell them. So we proceed step by step from the simple caricature of the firm depicted in the textbooks to the complexities of real firms in the real world of business. At each step toward realism, the problem gradually changes from choosing the right course of action (substantive rationality) to finding a way of calculating, very approximately, where a good course of action lies (procedural rationality). With this shift, the theory of the firm becomes a theory of estimation under uncertainty and a theory of computation—decidedly non-trivial theories as the obscurities and complexities of information and computation shift.

Achieving this fiscal acumen requires those with a fiduciary responsibility to intelligently juggle priorities and allocate resources properly across the many demands which compete for those resources. This requires business intelligence of sufficient fidelity and relevance for the above to occur across many large organizations, to empower all employees to act in the business's interests as well as their own.

The importance of this information is highlighted by Lawler, Mohrman and Ledford in their landmark 1995 publication, Creating high performance organizations:

Without (..) information about business performance, it is difficult for individuals to understand how the business is doing and to make meaningful contributions to its success. In addition, participation in planning and setting direction is impossible for employees to make good suggestions about how products and services can be improved and about how work processes in their area can be done more effectively. Finally it is also difficult for employees to alter their behavior in response to changing conditions and receive feedback on the effectiveness of their performance and that of the organization. In the absence of business information, individuals are usually limited simply to carrying out prescribed tasks and roles in a relatively automatic bureaucratic way.

Any established business is likely to have many different projects in their portfolio. Each effort is likely to be at different stages of maturity in each product's lifecycle. The attention and perspectives of the stakeholders of these projects vary over their period of performance. As a result, portfolios are effectively bundles of focused bets. Each bundle presents collections of risks and rewards, with outcomes that may or may not be realizable or deliver the value intended by the original sponsors.

In such aggregations, success will depend on many factors, including the stability of requirements, the amount of effort that can be focused on each activity, and how well the work is performed. Amazon is a classic example of such a bundle, which is one of the reasons their customers viewing them with which a broad spectrum of aggregated love and scorn.  When viewing work in these bundles, each project competes with their peers within the bundle they are a part of and offers different value propositions, bargaining positions, and margins for the shaping of the business and alignment with its environment with both evolving over the time. James Grier Miller, in his book Living Systems, described the push for economic analysis in this context:

All adjustment processes have their costs, in energy of nonliving or living systems, in material resources, in information (including in social systems a special form of information often conveyed on a marker of metal or paper money), or in time required for an action. Any of these may be scarce. (Time is a scarcity for mortal living systems.) Any of these is valued if it is essential for reducing strains. The costs of adjustment processes differ from one to another and from time to time. They may be immediate or delayed, short-term or long-term.

How successfully systems accomplish their purposes can be determined if those purposes are known. A system's efficiency, then, can be determined as the ratio of the success of its performance to the costs involved. A system constantly makes economic decisions directed toward increasing its efficiency by improving performance and decreasing costs. Economic analyses of cost effectiveness are equally important in biological and social science but much more common and more sophisticated in social than in biological sciences. In social systems such analyses are frequently aided by program budgeting. This involves keeping accounts separately for each subsystem or component that carries out a distinct program. The matter-energy, information, money, and time costs of the program in such analyses are compared with various measures of the efficiency of performance of the program. How efficiently a system adjusts to its environment is determined by what strategies it employs in selecting adjustment processes and whether they satisfactorily reduce strains without being too costly. This decision process can be analyzed by a mathematical approach to economic decisions, or game theory. This is a general theory concerning the best strategies for weighing "plays" against "payoffs," for selecting actions which will increase profits while decreasing losses, increase rewards while decreasing punishments, improve adjustments of variables to appropriate steady-state values, or attain goals while diminishing costs. Relevant information available to the decider can improve such decisions. Consequently such information is valuable. But there are costs to obtaining such information. A mathematical theory on how to calculate the value of relevant information in such decisions was developed by Hurley. This depends on such considerations as whether it is tactical (about a specific act) or strategic (about a policy for action), whether it is reliable or unreliable, overtly or secretly obtained, accurate, distorted, or erroneous.

Investments should thus be considered through the lens of the cost structure of an endeavor - the production function of products, services, and capabilities - and the business models which monetize these outputs over a competitive landscape. Candidate investments must be reconciled with realistic expectations of how the business will realistically be shaped by each capability increment going forward.

To properly account for the effects of products and markets in different stages of maturity, these costs and benefits should be grouped into several high-level categories, to account for the inherent uncertainty characteristic of lifecycle phases. These accounting structures should simplify the effective management of resources as new capabilities are designed, introduced, grow in usage, and reach a mature level of adoption within targeted business segments. 

Figure 1 provides a conceptual depiction of the financial aspects of an offering over these phases in its design and service life. This representation suggests consideration of alternative investments within a similar reference frame to aid in determining whether adjustments to the business’s strategies should be implemented. Unfortunately, those with self-interest in investment decisions inevitably are subject to biases and thus will manipulate factors in directions they believe to be favorable from their points of view. Until a new product enters service (when benefits begin to accrue) the resources consumed by an endeavor may have provided better returns had they instead been allocated to alternative endeavors or investments.

The cumulative cost of these activities is depicted by the green dashed line in figure 1. A dotted red line is used to represent the aggregate capability available to users, as the product is launched, gains acceptance, achieves maturity, and declines in usage over the product’s lifecycle. A baseline business plan for a representative environment before and after realization must be available to provide a reference point that reflects the situation had no costs been incurred, and no benefits were realized. This kind of representation is also useful to highlight the continuing future obligations that may be implied by near-term commitments.

Capability development phases

An example of these groupings is depicted as blue boxes in Figure 1 above:

• Inception - Customer development is performed and launch customers are selected. The business context, intentions, and ways and means by which these needs can be satisfied are defined. An endeavor to address these needs is scoped, alternative approaches are explored, and an opportunity evaluation and initial approach are defined. The cost drivers of variability in anticipated activities are considered.

• Elaboration - The details required for realization are explored and evaluated. The available options are refined into definitions of a vision, requirements, and high-level architecture, within a planning baseline of the product breakdown structure. Each option is evaluated for technical and business feasibility. A down-selected set is then further elaborated into work packages, statements of work, and defined specifications so the resulting architectural elements can be concurrently realized. In parallel, mitigation options are outlined that will credibly bound uncertainty to acceptable levels. The cost drivers for this realization include the complexity and stability of the existing environment and the fitness of the business and technical architecture for these situations.

• Construction -​ Work packages are initiated and results are verified for functional, logical, and physical design elements within the capability’s structure. The cost of developing each work package is a function of the amount of new, changed, and deprecated features which must be incorporated into the increment.

• Integration- Logical and physical design elements are woven together and refined into cohesive solutions. The cost of this activity is typically driven by the quality of these design elements and the clarity of their requirements and design definitions.

• Production- Physical elements of realization are sufficiently stable for replication in larger quantities. Costs are often driven by whether an effective means of late-stage customization for distinct operational configurations is in place. Note that in the PIANOS model, the above can all be considered production, when connected with the associated fields of that model.

• Transition- Operational configurations are fully deployed, delivered, and readied for use within each customer environment. A service level plan is established for ongoing support. The cost driver for this activity is established by the effectiveness of upstream processes.

• Sustainment- Services are provided as described in the service level plan, including support, training, troubleshooting, maintenance, and enhancements;

• Upgrades- Periodic technology updates and modernization activities are performed to assure continued utility and acceptable operational performance within the bounds originally authorized.

Investors in for-profit businesses expect profits back from their investments that exceed the minimum acceptable rate of return. According to Mike Cowell, getting to cash flow neutral ((when the green line crosses the red one in figure 1) is thus an essential survival instinct for a business to develop:

What causes most businesses to fail is running out of cash. In putting together a financial plan for your startup, the primary goal is to determine when or even if the business will get to cash flow neutral. This is the point when there is the same amount of cash coming into the business as is going out...

Cash flow is the lifeblood of your business. The cash flow statement allows you to see what your cash balance will look like month to month through the duration of your plan. When looking at this statement, keep in mind you need to keep a substantial amount of cash on hand at all times as a safety buffer. It is not unusual to keep at least six months of operating expenses on hand in the form of cash. Most businesses fail because they run out of cash before they achieve positive cash flow.

There are always limits to how much cash a business will be able to gain access to, whether through sales, borrowing, or tactical adjustments to working capital. Available cash is thus the lifeblood of businesses and serves as an important measure of the health for any business's portfolio. This cash is generated from the many alternative sources (over different time horizons), as depicted in Figure 2.

Each business needs these reserves to invest in new product development and buffer the affordable capacity of the business against the cyclic demands placed on its resources. Once a particular product has matured in the marketplace, it becomes possible to re-purpose those resources away from mature product lines towards future ones, though achieving this goal is a long-term play. 

New product development is the seed corn for future products. Ideally, this process would be predictable and painless; the quality of construction would always exceed customer expectations; wasteful activities would occur so infrequently and have such little impact that the drain they present could be ignored.

Unfortunately, in the real world, development inevitably takes longer than forecast, risks become issues which quickly drain available resources, and quality compromises result in traveled work, cost overruns, and dissatisfied customers. As a result, each endeavor requires placing and winning a series of bets.  Each product in its embryonic beginnings can therefore be characterized as a hypothesis that investments will deliver sufficient value to the business to provide a self-sustaining source of revenue for the future. All such bets are made with lofty expectations but uncertain payoffs.

Until realization is actually achieved, costs inevitably accumulate, and overruns push break-even points further and further into the future, extending the continual risk of plan continuation bias further. When seen from this lens, each product development (ad)venture exerts a tax (or drag) on a business's resources with an expectation that these investments will result in worthwhile payoffs over a longer time horizon. Uncertainty must be factored into resource forecasts over this timeframe so that the range of likely performance is appropriately characterized. If uncertainty hasn't been properly accounted for, or discipline wanes, gaps in performance can easily take longer to be noticed, since aggregation can dilute the signals available to those paying attention. Industrial models of productivity can provide a stopgap basis until more effective estimation processes are developed.

By the time data is available and understood, it may be very difficult to avoid deterioration that can accelerate nonlinearly. For example, when we examine the robust estimate produced for a moderately complex development project for an embedded system (see below), the relative effects that different factors have on performance can be studied. Such estimates helpfully demonstrate how confidence levels should be accounted for using the productivity data that estimates are derived from.

Resource expenditures are typically managed separately as direct and indirect costs. Negotiations are often necessary to pro-rate these indirect costs to organizations and infrastructure across the business's direct base. Each planning cycle must accommodate 'carryovers' of expenditures made in a prior accounting period but not paid until the current period is underway. As a result, each accounting method must also 'settle accounts' periodically.  Without such reconciliation, accounting systems struggle to provide meaningful accumulations of 'total project cost' that are immune to gaming by those who wish to obfuscate outside interference.

A common accounting mechanism allows certain kinds of major investments that exceed an established threshold to be treated as capital expenditures. These are distinguished from annualized expenses by their multi-year distribution. Additional resources in addition to this capital investment are often necessary to acquire or prepare an asset for use, to repair or improve it for a new application, or to extend its service life. If the asset retains most of its value over an extended period after the year of acquisition, the costs may be depreciated over time and retained on the businesses' balance sheet.  If an asset has no material value shortly after its initial acquisition, it is instead treated as an item that is expensed in the accounting period in which the work was performed.

In Knowledge and Decisions, Sowell discusses how challenging this can be, since accounting for operating expenses using unit costs and marginal costs are quite different:

When people casually speak of "the" cost of producing something, they usually mean the average cost - that is, the total cost of running the enterprise divided by the number of units of output it produces. But for actual decision-making purposes at any given time, the incremental cost is more crucial. The total cost of running an airline obviously includes the cost of airplanes, but in deciding whether or not to make a particular flight, what matters at that point is whether the incremental cost of that flight will be covered by its incremental value to the passengers, as revealed by what they are willing to pay for it....

An airplane idle on the ground during a particular time has a very low cost in the economic sense of cost as a foregone alternative. If a plane that would otherwise remain in a hangar overnight is instead brought out at midnight to fly a party of vacationers to a nearby resort, the cost of this short flight that does not interfere with its other schedule of flights is much less than the "average" cost of an airplane flight. In this case, the incremental cost of the flight is little more than the cost of fuel and a flight crew, since the plane itself is there for another purpose anyway.

The allocation of indirect costs attempts to walk a fine line between concepts that legal theory describes as fairness and efficiency. Different stakeholders typically benefit at different times, and to different degrees, from applying these two concepts. Let's define efficiency as that which maximizes aggregate welfare, and fairness as a morally defensible treatment of or distribution among stakeholders. Achieving both goals can be quite difficult and usually requires compromises to be formed. Such compromises require trust to be developed. It can be far easier to just grant one class of stakeholder access to resources to pursue their grand vision, despite the opportunity cost that may represent to others.

Literature on complex adaptive systems has suggested that bottom-up approaches to pursuing fairness and efficiency create structures that are more resilient to volatility. Other studies have found that the bottom-up activity-based costing approach is more agile and less limiting than the top-down theory of constraints approach. Clearly, a balance must be struck.

Many of these accounting methods focus primarily on costs, rather than on assuring that claimed benefits are actually realistic and can be achieved under nominal circumstances. The problem with cost-focused methods is they have a built-in bias towards cutting costs and reducing expenses, rather than providing more selective shaping of investments, in concert with more effective accountability.

Enhanced collective returns are often available by reallocating existing resources from currently assigned work to pursue longer-term, more credible opportunities. A newly engineered must enter service before its non-recurring costs can be properly accounted for as unit costs across the product's lifecycle. The costs incurred for supporting different configurations will still need to be fairly assigned to the particular party who required those features. Profit centers and cost centers are both alternative accounting methods to achieve such a strategy, though incorporating such changes usually requires a considerable period of adjustment before the revised allocations can be fairly and efficiently distributed, and the quality of the data will be adequate for decision-making across candidate future scenarios.

Ideally, tradeoffs between alternatives should be performed consistently across projects and endeavors, by converting benefits to dollars using consistent and credible assumptions, meaningful measures of effectiveness, and active risk management. In practice, accounting systems may not provide data in the form needed for a meaningful analysis of alternatives. In this context, productivity measures are essential tools to stamp reality onto this ideal canvas, and to evaluate organizational performance as a whole. The underlying units of measure should be tied to the 'real' completion of each job, rather than just a checkmark on a schedule (which often obscures a large iceberg of remaining work). 

Until a business can make credible projections of its lifecycle costs and what benefits can be expected from its products and services over realistic forecasts of the future, the yield recoverable from investments may be quite uncertain. Without a credible story of how such investments will pay off, this environment may make it increasingly challenging to secure the investments required for sustaining endeavors of significance over time. For this reason, cash surpluses should be channeled into productive uses of capital for long-term business perpetuation, such as investments to improve efficiency, accelerate the capture of new sources of revenue, and improve focus on the bottom line. While sunk costs can never be recovered, neither is it wise to bet that the cards will turn in your favor, presuming you have actually been counting them along the way.

Software Intensive System Estimate

160KB ∙ PDF file

Download

Download

• drives or shapes your subsequent actions over a planning horizon (typically multi-year)

• are not easily changed once made, and

• will have the greatest impact at this point in time on your long-term success

This handful of decisions includes:

• selecting your strategic posture with respect to your customers for the products and services which you intend to offer them

• identifying the value propositions which you will pursue with those customers through a coherent concept of operations

• developing approaches to create or align projects with these sources of value

• constructing tailored delivery systems to realize benefits from these projects as soon and broadly as possible

In synthesizing these strategies over time, customer-driven value should be pursued by extending:

1. Speed: the ability to deliver results faster than before, and respond rapidly to the emerging needs of the business

   • Approaches to quickly and successfully launch activities for the most frequently occurring customer engagement scenarios

   • The ability to rapidly and reliably reallocate resources to new situations with the minimum of friction

   • Responsive and effective infrastructure for satisfying operational service requests

2. Capacity: assuring a supply of products and services that is sufficient to meet the needs of all customers

   • Capacity that is scalable to meet current and future requirements cost effectively (up and down)

   • Strategies to extend the reach of products and services so that unit costs can be driven down

3. Efficiency: More products for less resources across the life cycle of your products and services

   • Managing assets and execution resource consumption to extend the leverage of investments

   • Eliminating chronic patterns of waste

4. Delivery: Products and services that accelerate delivery of results to the customer

   • Architectures that enable responsive, cost-effective solutions for customer’s needs - now and in the future

   • Scalable solutions which offer value and choices to the customer for different levels of investment

   • Approaches which insulate customers from the underlying means of production (‘handle the details’)

5. Quality: Reliable realization and cost-effective satisfaction of quality expectations for the customer

   • Assuring that the quality of products and services is adequate to support customer needs

   • Coordinating infrastructure changes so they have with minimal impact to customers

   • Keeping systems operating at peak performance

   • Ensuring that infrastructure provides security controls and maintains the integrity of the organization’s information

   • Managing corrective action and restoring services when outages or problems occur

Opportunities to pursue such value will only be worthwhile if they are appealing to your customers, can be monetized, and if the resulting revenue or resource authorizations can be redistributed among the organizations to reimburse the costs of implementing the required changes, and sustain and support the changes over time. All of these require anticipating the customer’s needs based upon insights derived from working together over time and developing an understanding of their preferred patterns, chronic issues, and future direction.

Regardless of how they are communicated, key success criteria for evaluating and selecting them are as follows:

1. They must promote a consistent understanding of their intent, scope, and reach across multiple stakeholders

2. They must integrate and exploit the conceptual knowledge available within the organization

3. They must provide utility for the organization’s business and its evolving environment under multiple possible future scenarios

4. They must facilitate exploration and trade-offs of candidate approaches to implement the strategies as the environment evolves over time

1. All work must be clearly defined as to content, sequence, timing, and outcome.

2. Every customer-supplier connection must be direct, explicit, and transparent

3. An effective protocol must be used to communicate requests and information about deliveries and priorities.

4. The pathway for delivering every product and service must be simple, seamless, and well understood by the responsible agents.

5. Improvements must be guided through carefully controlled experiments, under the guidance of an experienced coach, at have ownership at the lowest possible levels in the responsible groups.

Lean means the elimination of waste. In manufacturing, this typically means reduction of material in inventory, since that material is expensive, and since such reductions cause efficiencies in other important aspects, such as cycle time. In knowledge-based industries, waste is not extra people, as you might expect, but rather a reduction in the information and decisions that are ‘in inventory’, or essentially, in processing. This waste reduction is usually further augmented with additional efficiency improvements, by balancing capacity against demand and prioritizing work queues.

Lean practices have not yet achieved the penetration within engineering groups that they have achieved in manufacturing. I believe this is for several reasons:

1. Engineers don’t always understand why cycle time is important to the business.

2. A lack of a clear and brief explanation of how lean principles can be applied to the engineering environment.

3. There haven’t yet been enough good examples and ‘how-to’ guidance available or shared on Engineering Lean practices that have been relevant to engineering fields.

As a result, many engineering groups think Lean practices are not relevant to what they do, believe that Lean is a way of thinking, rather than a collection of techniques, or do not know what steps they might take to employ these techniques.

In manufacturing environments, variation typically arises when recurring, mechanistic operations are implemented inconsistently. Within engineering, such routine operations do not exist, since each design problem is unique, requiring searches through unfamiliar territory for potential solution elements.

Identifying waste

What’s common to both environments is a mandate to improve, and Lean can help in that regard. To accelerate outcomes, waste is anything that stands in the way of that objective. Pursuing waste reduction unlocks access to the next goal - removing these obstacles from the pathway of future endeavors.

For example, if regular meetings don’t seem to be helping move towards these goals, then in the spirit of removing waste, the design of these meetings should be reconsidered. Streamline the communications path by separating those critical for decision-making from those who merely want to be kept informed.

Avoiding rework, upping quality, and root-cause analysis

If a feature has an unusually high defect density, it may contribute to wasted time and effort. This wasted time doesn’t help in getting software in the hands of the customers, therefore, an effort should be made to reclaim it. The only way to do that would be to - get it right the first time. To do that, one has to ensure that no (or as few as possible) bugs are found during the final testing, instead, everything ought to be caught and fixed earlier.

This means two things - one, that testing needs to be done early and often through all stages of the game. Two, each time a non-trivial issue is discovered, the root causes should be investigated to prevent the same kind of thing from happening again. This is the equivalent of the ‘pull the Andon cord’ philosophy of Lean manufacturing.

Getting better at the difficult stuff

One reason why certain teams to achieve frequent releases is because deployment is hard. It can be an arduous process with lots of moving parts. That is the kind of situation that is ripe for fresh eyes and new thinking. Thet technical debt that inevitably piles up when not doing this boldly, simply must be avoided at all costs - else it will kill even the possibility of speedy delivery.

Multi-skilled people

When tasks require unique skills, those skills can become single points of failure should those individuals be unavailable for any reason. Such bottlenecks must be resolved. Having redundancy of skills across a Lean team can help mitigate this risk.

A culture of excellence

A culture of excellence can contribute to reducing cycle-time further. These include limiting the size of queues, achieving single-piece flow, using pull scheduling systems, and proactively discovering the next opportunity that can improve throughput of the team.

Each team-member should have a caring attitude for the customer and a curious attitude. One set of questions can be directed towards the customers themselves, to gain a fuller understanding of what they’re trying to achieve, so that the best, most creative, and efficient solution can be found. Another set of questions can be directed to the team and its processes. By following these practices, things become better for the customer, for the company, and for the team as well.

This culture of excellence judges its results by one metric alone: throughput. It realizes that any step taken to improve things should be measuring its effects. This culture should focus on how to optimize the whole system, and not just a part of it.

Once offerings enter into service, adequate resources must be invested to improve the quality and functionality of the capability. These improvements become the focus of subsequent iterative cycles to refine, shape, and extend its value for a broader range of customers. As figure 1 shows, the value of incremental content delivered in each cycle, or batch, has a direct relationship on the holding costs of the business, which influence the availability of the information, material, and resources necessary for continued investments.

As an example, in Managing the Design Factory, Don Reinertsen highlights how such batch size tradeoffs can play out in protracted test cycles:

Let us define the batch size of our testing process to be the amount of design work that we complete before we start testing. The largest batch size would be waiting for the entire design to be complete before we start the test. This will minimize the number of tests, but we will discover our defects very late in the design process, when it is expensive to react to them. In contrast, if we test a partially complete design, we deliver the information associated with the test earlier. Our problem is to balance the cost of extra test cycles against the cost of making expensive changes late in the design cycle.

As products come and go, those that glitter may not subsequently be found to contain gold. The evaluation of these opportunities must be adequately framed, so that considerations and prioritization can be performed competently. Implementation then requires a series of transformations across multiple planning horizons, disciplines, organizations, product breakdown structures, and work breakdown structures. Orchestration of batch sizes and cycle patterns enables timely feedback to be exploited and incorporated within and across each cycle of necessary transformations. Control of these batch sizes can unlock tremendous efficiencies and form the basis of faster iterations.

In the initial iterative cycle, the initial conditions, transformations, and context may all be flawed, but it is necessary to be able to tell the difference. We would like to be able to reduce uncertainty and improve our projections of the work to go until our original commitments will be achieved; the earlier and more frequently we obtain information about that target and our path towards it, the better off we will be.

Stepwise Refinement

Erhardt Reichlin, author of The Art of Systems Architecting, describes how to achieve this progressive elaboration - by reducing batches and increasing iterations - as follows:

Stepwise refinement is the progressive removal of abstraction in models, evaluation criteria, and goals. It is accompanied by an increase in the specificity and volume of information recorded about the system, and a flow of work from generalized to specialized design disciplines. Within the design disciplines the pattern repeats as disciplinary objectives and requirements are converted into the models of form of that discipline. In practice, the process is neither so smooth or continuous. It is better characterized as episodic, with episodes of abstraction reduction alternating with episodes of reflection and purpose expansion.

Since the value of information uncovered during this pursuit quickly decays, it must be acted on quickly. While more information seems preferable to less, in practice, information is only trusted in decision-making when it is accurate, meaningful, and relevant to the situation. The groups responsible for accomplishing this stepwise refinement need sufficient capacity to respond to latent demand while having effective channels to return work packages that are not sufficiently developed for realization.

• who are the customers?

• what do they need to realize business objectives?

• what functions must be provided to fulfill these needs?

• how well must capabilities perform for these functions to meet expectations for cost, quality, and schedule?

• which requirements are necessary for these capabilities in order to focus and manage development?

• how will the uncertainty of achieving these requirements within the negotiated resource bargain be managed?

Here is Stanford Beer on such resource bargains:

The Resource Bargain is the ‘deal’ by which some degree of autonomy is agreed between the Senior Management and its junior counterparts. The bargain declares: out of all the activities that System One elements might undertake, THESE will be tackled (and not THOSE), and the resources negotiated to those ends will be provided.

However autocratic or democratic (or even anarchic) your Resource Bargaining proves to be, the governing mode of management is Accountability. Please think about this responsibility for resources provided in terms (not of financial probity, not of emotional dependency, but) of variety engineering. Can you possibly itemize every single thing that the subsidiary does, demand a report on it, and expect a justification? Obviously not. Therefore accountability is an attenuation of high-variety happenings [and] investment is a variety attenuator.

EXAMINE precisely how accountability is exercised and especially what attenuators (totals, averages, key indicators) are used. If in the end, you are appalled to discover that the machinery is inadequate, that Senior Management just does not have Requisite Variety, then you had better own up.

The discipline of triage is essential to close any gaps discovered between the identified problem and solution spaces and establish situational awareness for aligning the viewpoints of design, production, and support systems. Gaps have different levels of importance for different stakeholders, and these stakeholders have different interests at different points in their system’s lifecycle.

Faster iterations also enable concurrent searches for value across parallel options, with each decision cycle framed by their circumstances. Timeboxing these parallel iterations can facilitate expanded exploration of more alternatives within a given timeframe and provides opportunities for more effective evaluations to be performed earlier; this can reveal which ones are worth continued investments, so the rest can be pruned out without a lot of collateral damage. Such adaptation tactics form an essential practice for businesses which enables convergence of fitness for use through incremental cycles of learning. This convergence is effective even when performed within a constantly changing environment, as long as the team’s agility is faster than the pace of environmental change.

Alignment

OODA loops have proven to be effective constructs to shape a cyclic set of activities for organizing the pursuit of agility over these cycles, especially in high-pressure environments. Chet Richards positioned this framework as a strategic framework to progressively align actions to changing environments: “Strategy is a mental tapestry of changing intentions for harmonizing and focusing our efforts as a basis for realizing some aim or purpose in an unfolding and often unforeseen world of many bewildering events and many contending interests.”

Each cycle of iteration of these transformations conceptually can be divided into 4 phases:

• Orientation

  Orientation begins by performing a situational assessment which involves the exploration and framing of the perceptions of actors, the states of objects they are focused on, and the significance of events which shape those states, as each object interacts within a broader context. The goal should be to develop a neutral understanding of the collective intentions of stakeholders relative to the environment and conditions which present themselves. It is usually helpful to use a set of organizing questions to help shape the collection of this information in order to establish connections between actions and consequences, by projecting not only what is happening, but how the situation is likely to unfold over time under different scenarios.

• Observation

  Given this orientation, observations are collected by actively acquiring information from available sources. In philosophical terms, it is the process of filtering sensory information through a mental process. In science, observation additionally involves the recording of data via the use of instruments. Nearly all observations will be a mixture of objective and subjective assessments; these observations should include context, learnings, and an innate “feel” for situations, people, and events. It should be recognized that each of these is influenced by a mix of variables, some within the subject’s control, and others outside it. The underlying goal of both activities is to synthesize a mental operational definition that is grounded in facts and whose interpretation (or meaning) is consistent with observations. There is sometimes a question in data processing as to where “observing” ends and “drawing conclusions” begins, though the boundary typically involves recognizing intensions, and evaluating options for aligning behaviors and intentions. Casual analysis can also be helpful, if time permits.

• Decision-making

  Decisions require appropriate framing in order to make balanced trade-offs between target outcomes, time, resources, and risk. A major part of this decision-making involves identification and analysis of a finite set of alternatives that can be evaluated using a limited set of criteria The speed within which decisions must be made is likely to determine the decision process involved; decision methods such as that used by the military are thorough but time-consuming. When the observer is the decision-maker, this task might vary according to whether the decision must be made intuitively or intellectually. If the latter, one must rank these alternatives in terms of how they are perceived to impact target outcomes, both in isolation and as interactions between the criteria when all are considered simultaneously. When decisions instead are to be made by a group, adoption of a more formal decision process should be considered.

• Taking Action

  Actions are necessary to establish momentum and begin achieving progress towards objectives. In military operations, the objectives are sent as an operations order. The sooner one can begin getting started, the faster one can begin learning. Of course, one can’t initiate just any action; it must be purposeful, conscious, and focused on making progress. Tyler Cowen stresses the importance of taking action even in the face of limited data: “The more information that’s out there, the greater the returns from just being willing to sit down and apply yourself. Information isn’t what’s scarce; it’s the willingness to do something with it.”

As Kyle Eschenroeder says, “we need to rebalance our appreciation for the power of action with our tendency to overthink, over-plan, and otherwise waste our energies in abstraction.”

The work of Fernando Flores has been instrumental in guiding how to transform an existing chain of transactions into a lean and efficient set of organizational capabilities through learning. The pace of this learning depends on how quickly and effectively provider and customer concerns about the necessity and sufficiency of provided capabilities can be fulfilled:

A process can be improved if it can be understood; it can be understood only if it has a consistent structure [Senge 1990]; and its structure can be consistent only after the first steps of process improvement have reduced process variability... Many processes exhibited such broad variation in behavior that it was difficult for process specifiers to agree on a process that represented the typical scenario. Many organizations informally built process specifications from anecdotal process experience instead of developing the baseline process model with empirical models and data. Many organizations... created an ideal specification instead of capturing empirical practices, and organizations used these specifications as a baseline for improvement despite the mismatch. Because many process specification models were divorced from empirical practice, they could not reliably drive real development practice.

Organizations can track their effectiveness in this decision-making by measuring the momentum of benefits realized from investments, rather than by measuring the velocity of the teams who are doing the work. Preparation for this realization capture requires orientation to both the context of customer concerns, the utility of capabilities being deployed, and the uncertainties involved in action.

Multinodes

In Stafford Beer’s Viable System Model, the term multinode is used to describe the structural and cognitive configuration of the organization’s “identity function” when decision-making is not the result of a single person but a collective mind. It is Beer’s way of describing how multiple individuals can jointly enact the role of ultimate policy, coherence, and long‑term direction. His comments on this notion illuminate convergence:

We claim we know how the whole thing works. The problem is to make it work more quickly. That must surely mean the introduction of discipline and order, of some sort, into the situation. It also means, however, that no measures may be adopted which would at the same time put the remarkable freedom of action and the wonderful flexibility of the multinode in jeopardy. If people could see how to do this, without putting themselves and their organization into a strait-jacket, there is some chance that they would adopt new techniques. One method, we ought now to agree, must be excluded although it is the one method most usually attempted in practice, because no one can think of anything else. This is the method of rigorous protocol. Explicitly: it denatures the system itself - with all its in-built capacity to generate the right answers.

The first difficulty is to know what kind of problem the multinode actually solves. It does not devote itself, its seniority and power, to the determination of trivial outcomes or it ought not to do so. It is likely to be settling a policy of great importance and therefore considerable complexity. Thus it is that people think of thinking as a process of synthesizing an integral but elaborate conclusion from a large number of component parts. The decision is seen as a rococo edifice built up clause by legal clause. This is perhaps why there are endless drafting problems facing anyone trying to promulgate an agreed decision.

The cybernetician adopts the contrary position. The output of the thinking process, the decision, has the following form: do this (rather than anything else). When the process of thinking originally starts, the multinode is faced not indeed with a number of building bricks in an edifice but with a seemingly infinite number of possible outcomes between which it must choose. It is the existence of this plethora of possibilities that cries out for decision in the first place. Then, under this model, the process we seek to assist is one of chopping down ambiguities and uncertainties until we may say: do this. In short, we would like to measure the variety of the complex decision at the start, measure the reduction of variety brought about by each conclusion reached in the process of thinking things out, and in general monitor the entire operation of the multinode as the variety comes down to a value of the decision itself. To do this we shall need two tools: a paradigm of logical search, and an actual metric - a rule and a scale - for measuring uncertainty.

The speed of this search is the critical discriminator between groups that can deliver sustainable, exemplary performance and those who just get by. Those in the latter class are implicitly condemned to a death by a thousand cuts.

Operational excellence is not a single methodology or a set of best practices; it’s a discipline of coherence: the ongoing ability of an endeavor to align intent, capability, and execution so that work flows smoothly from concept to completion. At its core, operational excellence depends on three things:

• Clarity of purpose (what the system is trying to achieve)

• Visibility of work (how value is actually produced)

• Coherence of behaviors (how agents act within the system)

When these elements reinforce each other, endeavors develop a kind of operational gravity: work moves in the right direction with less friction, fewer surprises, and more resilience. But this gravity only emerges when the underlying orientations of the people doing the work are recognized and integrated rather than left to operate in isolation.

The Role of Agent Orientations

Every agent in a system operates from a viewpoint. Some are Process‑oriented, optimizing flow and reducing variation. Others are Insight‑oriented, probing for meaning, patterns, and strategic leverage. Still others are Action‑oriented, driving momentum and execution. These orientations are inherent to how agents approach complex work.

Operational excellence emerges when these orientations are deliberately orchestrated. A system that overweighs process becomes rigid. One that overweighs insight becomes paralyzed by analysis. One that overweighs action becomes chaotic. The craft of operational leadership lies in creating a work operating system where each orientation plays its role without dominating the others.

The Pitfalls That Undermine Operational Excellence

Most endeavors don’t fail because they lack intelligence or effort. They fail because they fall into predictable traps. Some of the most common include:

1. The Illusion of Alignment

Teams often believe they share a common understanding of goals, constraints, and priorities. In reality, each orientation interprets the mission differently. Without explicit alignment rituals, these differences compound into drift, rework, and conflict.

2. Process Overreach

In the name of efficiency, organizations often add layers of process that obscure rather than illuminate the work. This creates a brittle system where compliance replaces judgment and where deviations (often the source of innovation) are treated as defects.

3. Invisible Work and Hidden Queues

When work is not made visible, bottlenecks form silently. People optimize locally, unaware of the system‑level consequences. Operational excellence requires exposing the true shape of work, even when that visibility is uncomfortable.

4. Over‑reliance on Heroics

High performers can temporarily mask systemic flaws, but heroics are not scalable. Systems that depend on exceptional individuals inevitably collapse under load. Excellence comes from designing systems where ordinary performance reliably produces extraordinary outcomes.

5. Fragmented Feedback Loops

When insights from the front lines don’t reach decision‑makers—or when decisions don’t translate into operational behavior—the system loses its ability to learn. Excellence requires tight, bidirectional feedback loops that connect strategy to execution.

6. External perturbations

Changes are inevitable, and the dynamics they generate amplify the effort required to resolve known problems and obfuscate the clarity of purpose essential for coherence.

The Real Work: Integrating Perspectives Into a Coherent System

Operational excellence is ultimately a systems integration problem. It requires building a work operating system where:

• Strategy is translated into actionable constraints

• Processes are designed to support - not constrain - human judgment

• Insights flow freely across boundaries

• Execution is measured in terms of outcomes, not activity

• Agents understand not just their role, but how their orientation contributes to the whole

This is why operational excellence is so difficult: it demands continuous negotiation between competing goods; efficiency vs. adaptability; standardization vs. autonomy; speed vs. quality. The organizations that excel are those that treat these tensions not as problems to eliminate but as dynamics to manage.

The PIANOS model exposes these viewpoints and operates within a macro-level flow depicted in Figure 2:

A narrative of these viewpoints in action

Here lies the Territory occupied by endeavors pursuing excellence. Its aspirations are charted as requirements, the sacred pact made among team members and the endeavor’s stakeholders. The endeavor’s Audience - the sum of all our stakeholders, vests a Navigator role with fiduciary trust, expecting that the stakeholder’s needs and intentions will stand as the blueprint for endeavors as they unfold. The interactions with this Audience often shape risk profiles, priorities, and cultural dynamics.

Yet in every endeavor, risks lurk - wily shadows that threaten our course and anticipate unforeseen detours. Opposite them, opportunities wait in latent promise, ready to be elaborated by those daring enough to shoulder the inevitable responsibilities. Navigation allocates resources to fuel learning and knowledge capture which provide a foundation for improvements that will stick. Despite this knowledge, the impacts of changes continue to emerge, perturbations that nudge our venture from stillness into motion.

In parallel, Adaptation lays down Structure, carving channels for Alignment. It’s the architect of our resilience. It beckons Orchestration, that deft hand which conveys vision and assures cohesion, ensuring each resource finds its place.

Actions take form, as each team member uses their competency and capabilities to execute their Jobs to Do. These tasks are fed by ceaseless feedback loops. With each iteration, outputs materialize, as capabilities updates are deployed, releases are delivered, and customers are served.

Between each node, flawed intentions and slips also contaminate our aspirations and plans. Defects emerge as intentions falter, and power is typically pulsed to correct our course. The crucible of alignment serves to confirm that business objectives are being effectively translated into operational targets and to trigger adjustments when warranted.

Within this Territory of intentions therefore lies both our greatest potential and our most familiar pitfalls. Only by tracing these threads - and by orchestrating Adaptation and Navigation - can we shift from planning to effective transformation of intentions into impacts.

1. what outcomes are expected from this work?

2. why is this work required?

3. who is going to do and use the work?

4. when must this work be completed?

5. how will the work be performed?

6. where will this work be reviewed and reported on?

The ‘what’ question should be answered by writing down the accomplishments and completion criteria which must be satisfied when performing this work. These criteria should enable an evaluation of whether an event has been completed or not, with the determined result producing a binary choice. While activities may be represented in relative terms (like “50% complete), accomplishments are either achieved or they are not. A good plan needs a description of exactly how this doneness will be determined. A good way to do this is to document the tangible evidence that is expected to demonstrate satisfactory completion, such as the examination or evaluation of an artifact, rather than just tracking the passage of time, or the consumption of resources. Events described in this way provide the basis for assessing progress while the work is being performed.

Answers to the ‘why’ question reveal the underlying intentions for performing the work. If the team’s performance ever strays too far from the path of these intentions, a record of the underlying motives is helpful in providing a north star for reorienting the team as they consider whether to adjust their plan or their execution.

Answers to the ‘who’, ‘when’, and ‘where’ questions are typically required to divide up the work into executable chunks by performing units. For major endeavors, this partitioning is often done by defining distinct work packages that can be separately estimated and planned at lower levels. These work packages are natural subdivisions of lower-level activities that can be assigned to specific organizations for allocating responsibilities, lower-level planning, and monitoring progress during execution.

A work package is completed when all accomplishments assigned to that work package have been satisfied. To answer these three questions, the description of these work packages should record the work that is to be performed, the deadlines by which that work must be completed, and the skills and expertise required for performing that work. Once the work package is framed in this way, an estimate of resources required for this execution. and an assessment of the risks that may impede progress along the way should be developed. For example, work packages might be defined for performing a design task, developing tooling, making an initial manufacturing production run, performing a shop order, purchasing an off-the-shelf component, or any other definable collection of activities which can be assigned to another organization or a sub-unit of an internal organization.

The ‘how’ question is answered by describing the activities which will achieve these outcomes. These activities often require elaboration at different levels of detail over time, though it is unusual for all of the required details to be known before execution must begin. Since planning for complicated endeavors are required at many different levels, the associated plans and schedules are established and managed in a hierarchical fashion. When we use the adjective ’master’ with the noun ‘plan’, we are mean the top-level elements of this collection, which are the principal, driving forces, and controlling authority, for lower-level activities. In this context, the top and upper-level elements will likely be established using different methods, and will be negotiated by different parties, than lower-level plans. Detail is added, and accountability is provided, as the schedule, functional, and performance requirements are flowed down, and managed within a well-defined set of organizational interfaces. When we add the adjective ‘integrated’ to our ‘master plan’, we have an ‘integrated master plan’, which describes the details and information relationships across these multiple levels, and assures that they have been verified to be sufficiently consistent and complete for execution to begin, and refinement of planning to occur over progressively longer time horizons.

While planning can occur in people’s heads, and in conversations among team members, none of that has meaning unless it is written down and approved. An integrated master plan must be written down, reviewed, and approved. This suggests using an outline such as that provided below:

• Introduction

  • A short description of the endeavor

  • A definition of the scope for the work

  • An analysis of the progressive flow and maturity of work products. There should be a particular focus on adopting conventions and expectations for maturity definitions such as draft, preliminary, written, submitted, accepted, and approved.

• Assumptions and Ground Rules

• Events, Accomplishments, and Criteria Dictionary

  • Events, which are the logical points at which to assess progress

  • Accomplishments, which are the desired results to be achieved by a specific event

  • Criteria, which is the evidence required to demonstrate that each accomplishment has been complete

• Work packages

  • Organizational responsibilities for producing the work package

  • The primary organization responsible for negotiating additional details about the interactions between this work package and other work packages (often called the work package owner).

  • A narrative describing the work package activities, with a focus on the handoffs into or out of the work package:

    • what inputs are needed and by when

    • which work package those inputs are needed from

    • what outputs will be produced

    • when (relative the start of the work package) key decisions need to be made

  • The definition of done

The intent of this information is to lay the groundwork for producing a schedule which is tethered to a start date, i.e. weeks from launch, months from go-ahead, etc.

To avoid confusion about whether a particular named activity is located in the plan, or will instead be found in its accompanying schedule, an effective convention for naming is often adopted; this convention involves describing activities in an integrated plan using past tense verbs, since these activities describe assessment points that should have been completed when the event itself occurs. For example a key event might be described using accomplishment named ’Systems Requirements Review completed’. In contrast, present tense verbs should be used to describe activities when they are instead in the integrated master schedule (e.g., Develop Software Specification, Perform Requirements Analysis). This change in verb tense reinforces that there is a relationship between planned activities and scheduled activities and makes it easier to integrate activities and events across plans and schedules. Consistent use of such constructs helps to emphasizes these important distinctions and provides a very simple naming convention that signals where to find things.

If the definition or development of any portion of a work package has not yet been sufficiently defined to allow planning to be done, a mechanism should be adopted which captures the associated activities in a ‘parking lot’ construct. The need for this deliberate delay in detailed planning typically arises when the work is sufficiently far in the future that it is dominated by unknowns, being deliberately performed in a series of implementation phases, or if scouting explorations to discover which rainbow to follow in order to accomplish that work have not yet been accomplished. Ideally, a rough estimate of the time and resources required for accomplishing this future work can also be made, though this many not always be possible. One way of addressing uncertainty in this context is by producing an estimate with both an upper and lower bound combined with a confidence level associated with this prediction.

The collection of these ‘future planning packages’ should be time-phased in accordance with known need dates and historical rates of execution for similar endeavors. Refined planning of these future planning packages can then be accomplished once requirements and the environment become clearer, and the timeframe for performing the work draws nearer. The work required to refine the plans for these future planning packages should be incorporated as a part of the baseline integrated master plan itself, to avoid painting the team into a corner.

In small bites, a schedule depicts activities and their dependencies for an endeavor. But when endeavors are complicated and involve many pieces, it is likely that a particular kind of schedule is used - an integrated master schedule which links the information in an accompanying integrated master plan for an endeavor, instantiates that information through the network logic established by lower-level schedules, and optimizes the allocation of available resources and time. Through this means, the integrated schedule establishes dates when key events are expected to occur, and weaves together the work packages necessary to achieve the corresponding accomplishments.

An integrated master schedule is thus a time-based depiction of the information contained within the integrated master plan, with substantiation of this depiction through lower-level schedules which implement that plan, and depict how the work will be performed. The use of the adjectives ‘integrated’ and ‘master’ in this context have the same meaning as they do in the corresponding integrated master plan.

A good schedule requires elaboration of details and concurrent sizing of the work to be done. Each require collection of facts and data about that work, so that appropriate modeling of this work can be performed. The organization which charters the development of such an integrated master schedule must commit to proactively use the resulting information (instead of optimism, opinions, or ill-defined targets); it must also then manage the lower-level schedules, or the resulting top-level schedule will be meaningless. While many think schedules are a joke, that attitude is a self-fulfilling prophecy.

In reality, schedules are just a tool, and should not be confused with the challenges that may actually be encountered while performing the work. The process of developing a schedule can be as important as the schedule which results from that process, since this journey itself often serves as a catalyst for discussions which may not otherwise have happened. In the absence of sufficient attention to planning and scheduling, precious time and resources may be wasted on relatively unimportant activities as the work unfolds, and many options for mitigating risk will be missed until it is too late to exploit them.

Work packages owners (i.e. those primarily responsible for producing the results expected from the work packages) must work together to translate the activities, events, and accomplishments described in the integrated master plan into specific dates by:

• aligning with the equivalent planning & schedule models of the higher and lower level schedules for an effort

• connecting with upstream, concurrent, and downstream performing organizations

• working through the needs of customers, including those of downstream performing organizations

• negotiating interactions with other work packages owners

• taking account of estimates for the demand and the availability of critical resources

An integrated master schedule should be presented as a structured hierarchy which make the key activities, milestones, and interdependencies between the constituent work packages visible. An integrated master schedule models the logic of these interactions (i.e., their predecessors, successors, and associated lag or float time) in order to depict how accomplishments will be achieved with the passage of time, and through the expenditure of resources. This hierarchy should also align with realistic start dates and needed end dates to lower level planning efforts. When seen from this perspective, an integrated master schedule is really a network of commitments. The combination of these commitments for individual work packages, their predecessor and successor relationships, and their triggering conditions, are what enable a critical path analysis to be performed, and resources to be optimized across a portfolio.

The integrated master schedule should be a living document that is regularly updated to reflect the progress of the program or project. The information in the integrated schedule network should be logically consistent, and the collection of resources required should sum to the budget agreed to for the endeavor. This collection should also:

• Maintain consistency with the integrated master plan

• Illustrate the interrelationships among events, accomplishments, criteria, and tasks

• Maintain consistency with the work package definitions

• Indicate the start and completion dates and duration for each event, accomplishment, criterion and task;

• Provide the ability to sort schedules multiple ways (e.g., by event, by work package owner, by type of funding required,?etc.

• Provide for critical path analysis;

• Provide the means to perform ?what if? schedule exercises

• Provide for updates on a regular basis, and record completed actions, schedule slips, and rescheduled actions from the previous schedule for reference

This network should include all the elements necessary for achieving the accomplishments. For example, if the project in question is an engineering development project, it should include requirements elaboration and clarification, design and development, verification and validation, production or modification, and delivery of the total system. The work required for these activities should be described within a context which is meaningful to running the business. This network should provide the basis for critical path analysis, establish a framework for identification and assessment of disconnects, and allow alternative scenarios to be explored. The progressively increasing level of detail in these schedules across the schedule network or hierarchy should include:

• A summary schedule, typically depicted in a single view, which highlights the period of performance, key milestones and/or deliveries, and other significant events

• Intermediate schedules, which represent work package or project activities, milestones and phases at a level of detail which is below, but vertically traceable to, the summary schedule. Intermediate schedules are used within performing organizations to manage their commitments and communicate their progress

• Detailed schedules at the level that the work is being done within the formal network, and are the sources of the data that drives the summary and intermediate schedules. Detailed schedules elaborate authorized work into a logical sequence of time-phased tasks and interrelationships, and provide meaningful indicators of work being accomplished at levels below the formal reporting threshold. For example, such detailed schedules may not track individual change requests, but instead would the activities which must be implmemented to deliver a collection of these work requests over time.

As this elaboration is performed, the resources required to accomplish these work packages can begin to be rolled up and examined in aggregate. It is at this point that the total demand on available resources will become apparent. Generally, adding resources to meet the demand should be a last resort, because most resources are not interchangeable, and growing the team will increase the costs of coordination, communications, and decision-making. A preferred alternative is for the scheduling process to rearrange the order in which these work packages will be executed; however, this process should not try to change the underlying structure or approach to doing the work, unless the associated integrated plans are themselves updated.

Negotiations typically begin in earnest when such resource or time conflicts become apparent. A strategy will need to be developed so that tradeoffs among priorities and alternatives can be performed. Ideally, these tradeoffs should avoid erosion of contingency buffers which are necessary to absorb the inevitable risks that will arise. To the extent possible, this reconciliation should be performed so as to preserve the ability to accomplish events in the higher-level schedules, though often, the results of these discussions may also rippleinto requested changes to the integrated master plan itself.

For complex projects or portfolios of related projects, use of a work breakdown structure (WBS) will be helpful to organize the work packages and activities associated with an entire portfolio of projects associated with some endeavor. This work breakdown structure provides a framework and context for collecting costs, reporting progress, and providing oversight by different levels of management, and across different levels of this schedule hierarchy, with respect to the actual status of the individual elements contained in this structure. The work breakdown structure may also organize this work across major phases, such as pre-acquisition, acquisition, and sustainment, to emphasize the necessary lifecycle implications of an endeavor over some planning horizon.

Events and deadlines are typically represented on a schedule using a milestone notation. All such notational conventions should be established up front, for consistency across the schedule network. These standards should also describe how the planned start and finish dates of activities, and the relationships between these activities, should be depicted, as well as provide a means of communicating and managing schedule baselines. As work is performed, a similar convention should also be adopted to depict progress which has been achieved. This convention should also draw attention to the impact of prior delays on work in progress, as well as work planned in the future.

The basis of estimates for the durations of activities should be traceable to realistic projections of effort for the size and complexity of all products which must be produced. Whenever possible, the analysis behind these projections should be calibrated with data from prior experience. While an integrated master schedule should be an event-driven depiction which has been derived from the projected durations of the underlying activities, there may be specific constraints on when individual events must be accomplished by (i.e. no later than dates). A periodic schedule analysis should be performed to evaluate the realism of any sequence of activities relative to such constraints, as progress unfolds.

The path originally envisioned by the integrated master schedule may turn out to be unrealistic, once execution begins; those who are responsible for managing to changes to the schedule should keep in mind that the map is not the terrain. High-risk, concurrent activities may be attempted, and aggressive estimates may be prescribed to meet emerging conditions. For an integrated master schedule to be effective, it should not override what can realistically be done; instead, schedule risks should be identified, and timely mitigation efforts should be put in place to attempt to shorten time flows which don’t fit into overly ambitious targets. The integrated master schedule should then explicitly include these known risk reduction activities which are expected to mitigate these risks, and include the associated resources required for performing this mitigation.

Scheduling tools and the specialists who use them can steer projects into trouble if these tools are used to spread resources in artificial ways, such as by capping resource consumption at some level or forcing the spread of resources to match available funding. Handoffs between individuals are rarely clean unless there is a robust process in place which communicates the protocols of these exchanges. For example, while gantt charts can be useful representations, the activities which they depict can be misleading unless they are integrated, and their start and end dates match the criteria established in an integrated master plan. The resulting allocation may not actually be practical or realistic, but those reading the schedule will not be able to tell the difference. People can’t multitask from one task to another without reorientation and loss of efficiency. Another pattern often observed is for teams to locally authorize shifts to events for their own convenience, without requiring corresponding reviews and authorization of the direct and indirect impacts on other work packages.

There are many practical scheduling guides available that offer good advice about how to satisfy the above provisions. One of the better ones is the Planning and scheduling excellence guide, which is available from the NDIA. This guide provides eight over-arching tenets for building, maintaining, and using schedules using what is described as ‘Generally Accepted Scheduling Principles (GASP)’.

“Planning” is a concept with baggage. For many, it calls to mind a passive activity: sitting, thinking, staring into space, abstracting what you’re going to do. In its more institutional form, planning is a bureaucratic exercise in which the planner writes reports, colors maps and charts, programs activities, and fills in boxes on flowcharts. Such plans often look like train schedules, but they’re even less interesting. Much planning does fit that bill.

And that’s a problem because it’s a serious mistake to treat planning as an exercise in abstract, bureaucratic thought and calculation. What sets good planning apart from the rest is something completely different. It is captured by a Latin verb, experiri. Experiri means “to try,” “to test,” or “to prove.” It is the origin of two wonderful words in English: experiment and experience.

Plan-driven methods

Professional societies offer plenty of guidance about this experiential experimentation. Since 1969, the Project Management Institute has championed plan-driven methods that promise to produce a robust project plan. Unfortunately, the robustness often drags along an accompanying bureaucratic overhead with it. Many other disciplines, like systems engineering and software engineering, echo significant elements from PMI’s framework.

Figure 2 captures the major processes in the Project Management Body of Knowledge Guide (PMBOK), with planning processes getting most attention, and plans being the primary output.

There are ten core elements of the planning process: scope planning, scope definition, activity definition, resource planning, activity sequencing, activity duration estimating, cost estimating, schedule development, cost budgeting, and project plan development. In comparison, there is only one executing process and two controlling processes, so the primary output of project management is the project plan, which itself risks becoming bureaucratic. Quoting the PMBOK:

The definition and management of processes is key to plan-driven methods. For that reason, plan-driven methods are almost always associated with process improvement. The processes need to be defined, standardized, and incrementally improved to provide the data needed to control and manage their operation. Such processes generally include detailed plans, activities, workflow, roles and responsibilities, and work product descriptions.

Thus, learning and improvement are expected to be a fundamental feature built into project plans, though these attributes often rely on activities outside the scope of the resulting plan.

In its most recent incarnation, the PMBOK has placed an increased emphasis on principles, which are worthwhile, unsurprising, and easier said than done. This is a positive step away from a strict control orientation (note, for example, the guidance on the project manager chief role responsible for coordinating project activities). Several concerns about the PMBOK remain:

1. Structural rigidity and lack of flexibility

   1. Prescriptive nature: PMBOK has traditionally emphasized rigid processes and documentation, which can hinder adaptability in dynamic or innovative projects.

   2. Slow to adapt: Critics argue that PMBOK struggles to keep pace with rapidly changing business environments, especially in tech and creative sectors.

2. Documentation focus and administrative burden

   1. Heavy emphasis on documentation: While useful for compliance and audits, extensive paperwork can create unnecessary overhead, particularly for small endeavors.

   2. Process-centric bias: The focus on process groups and knowledge areas may lead to bureaucratic inefficiencies in fast-moving projects.

3. Insufficient emphasis on sustainability

   1. Sustainability gaps: While PMBOK mentions sustainability, it lacks actionable frameworks for integrating related concepts into project success criteria.

   2. Lifecycle sustainability: Critics call for sustainability to be embedded throughout all project phases, not just acknowledged in theory.

4. Underutilization of AI and Data-Driven Decision Making

   1. AI as an afterthought: PMBOK has been slow to embrace AI as a strategic partner in project planning, risk assessment, and communication.

   2. Missed opportunities: Tools like generative AI and machine learning could enhance forecasting, stakeholder engagement, and adaptive planning, but are not yet central to the PMBOK framework.

5. Challenges with Distributed Teams and Modern Collaboration

   1. Limited guidance for remote and distributed teams: PMBOK has traditionally focused on co-located teams and hierarchical structures, which may not reflect today’s collaborative, networked environments.

6. Disconnects between conceptual and practical elements

   1. Theory-heavy: Some practitioners find PMBOK too abstract or academic, lacking practical tools for real-world challenges.

   2. Principles vs. execution: The shift to principles-based guidance in newer editions aims to address this, but implementation clarity remains a concern

Plan-driven methods are themselves production processes which are expected to facilitate tradeoffs across alternative approaches for activities, confront tensions between autonomy and control, and transform inputs into outputs within whatever organizing framework has been adopted for activities. The Origins of Efficiency outlines the challenging circumstances of production:

Production processes are sociotechnical systems, embedded in human organizations and subject to the vagaries of human decisions, which are often guided by considerations other than cost. These decisions can and do influence the trajectory of production process development, by adding costs in service of achieving other goals like increasing safety or reducing environmental harms, or by limiting the sorts of improvements that can be made.

Information is an input needed, and an output produced, by most production processes. When we think of planning as itself a production process, the is often operating in the realm of ideas, narratives, and concepts, often (much of it unreliable), assumptions (when information is unavailable), and predictions of forward-looking behaviors (which are subject to biases), all under time pressure that often limits a necessary understanding of the presenting situations of the endeavor.

Agile

Agile software development emerged in 2001 and provides an iterative, incremental, and evolutionary framework which embraces values and principles appeal to small teams. Since then, many separate methods have emerged that share DNA with these ideas, while seeking differentiation across the competitive environment they operate within; an ocean liner of businesses (including many led by Agile Alliance members) emerged ready to offer their services and combat the common pitfalls encountered with these methods. Yet studies on failed transformations highlight preparation issues: - lack of leadership, inadequate sponsorship, unclear decision rights, misaligned incentives, rigid governance structures, ow psychological safety, and poor cross-functional collaboration.

Many Agile-focused methods have also been embraced in applications outside software development, with limited success. Scaling up agile to major development efforts (100+ developers working on a specific product) and in regulated, safety-focused endeavors has also proved challenging.

Steve McConnell’s book, More Effective Agile offers the following contrasts between plan-based, sequential methods like PMBOK and adaptive methods like Agile:

Comparisons of Agile and plan-driven methods often contrasted failed plan-driven projects with successful Agile projects, or vice versa. Since all successful projects rely upon effective organization and oversight, customer collaboration, and robust transformations, such comparisons are rarely useful.

One of my favorite books evaluating improvement frameworks is Balancing Agility and Discipline. It describes the tension between these two competing worldviews:

The agilists rail against the traditionalists and lament the dehumanization of software development by “Taylorian” reductionists who worship process. The establishment has responded with accusations of hacking, poor quality, and having way too much fun in a serious business. True believers on both sides have emerged to proclaim their convictions with near-messianic stridency, raising the perplexity level of software developers and managers trying to evolve their success strategies.

At the ten-year anniversary of Agile’s Manifesto, Philippe Kruchten captured the ‘elephant in the room’ nature of issues which the participants reflecting on. A sample of his thoughts deserve repeating:

While the community has been good at amplifying what works, it often has not been good at dampening what did not work, analyzing why it did not work, or under which conditions it did not work (context!)…

In most instances, when practices are described too little of the actual context is described, leaving an impression of very wide applicability; even if this wide applicability is not actually claimed. Sometimes this is just pure dogmatism, or bigotry…

Unlike other disciplines, like medicine for instance, there is very little reporting of failures or limitations in software, and no clear venue or even templates or exemplars of such reporting. And again the fear that this would reflect badly on the person reporting, or the “victim” or “culprit” organization. A good template for such report would include a detailed description of the context…

There are still too few people gathering objective, significant, scientific evidence on our practices, except for a very few that are easy to instrument (pair programming, for example)…

The agile movement is in some ways a bit like a teenager: very self-conscious, checking constantly its appearance in a mirror, accepting few criticisms, only interested in being with its peers, rejecting en bloc all wisdom from the past, just because it is from the past, adopting fads and new jargon, at times cocky and arrogant.

Continuous flow

The book The Origins of Efficiency highlights one of the primary constraints limiting both plan-driven methods and Agile: they simply do not scale:

When we take a 30,000-foot view at where and how efficiency has improved, one recurring theme is the importance of scale. Economies of scale are not only a major driver of efficiency improvements in and of themselves, but they are also a gating mechanism that unlocks other efficiency improvements by making it possible to amortize the large fixed costs that such improvements often require over a sufficiently large output…

When we can’t repeat a process, or when there’s significant variation in the production environment or in the product itself, many paths toward increased efficiency are blocked. Without repetition, we can’t take advantage of economies of scale. A constantly changing process or a highly variable environment is harder to predict and control, making it more difficult to increase yield and decrease waste. It’s also harder to automate or mechanize in such circumstances, which limits our ability to reduce labor costs. This is one reason why services like medical care, car repair, and construction, which rely on skilled labor that can flexibly adapt to highly variable conditions, remain expensive. Finding ways to make these things cheap will require escaping the constraints of scale to get the benefits of a continuous process—that is, a constant transformation of inputs into a finished product without waiting, buffering, failures, or wasted actions—without needing to repeat that process over and over again…

The development of continuous processes has historically been most easily achieved with a carefully orchestrated arrangement of workers and special-purpose equipment, which is costly to build and operate and, therefore, requires the cost to be spread over a very large output. It takes time, effort, and expense to make the steps in a process work together in sync, a result that can often only be achieved with dedicated equipment and justified with large production volumes…

What if it was quick and easy to figure out the order of steps in a process and arrange them so that the transformation is swift and steady? And what if this process didn’t require special equipment but could use flexible, adaptable machinery able to produce a variety of different items? Then it would be possible to achieve a smooth continuous process that worked in a highly variable environment, bringing the benefits of efficiency improvement to domains that have traditionally resisted it…

For production technology to be able to flexibly adapt to a highly variable environment, there are two broad requirements. First, it requires [methods and] equipment that is physically capable of responding flexibly—that can take a broad range of possible actions, or a small number of actions that can be combined to produce a wide variety of outputs…

The second requirement for flexible, automated production is that the equipment has some method of determining the sequence of the proper actions to take—an information-processing component that can, based on data from the environment, break a desired output into a series of achievable production steps.

The capabilities available from satisficing these requirements will be considerably enhanced as artificial intelligence technologies are incorporated into activities and allow dynamic reallocation of efforts to maximize achievable outcomes.

Operations focus

Neither Agile nor plan-driven methods provide the universal flow-oriented viewpoint for production that is offered by Lean (an extension of the Toyota Production System). Lean and JIT (Just-In-Time) methods emerged from industrial engineering with a queueing-theory backbone, pushing teams toward dynamic flow while retaining and customer focus.

However, Lean lacks the disciplines of plan-oriented methods, since it is designed for improving operations; further, its focus on elimination of waste can shortcut strategic efforts. As a result, Lean-Agile hybrids have been effective in industries that require both operational excellence and rapid innovation. These hybrids typically feature adaptive planning with feedback loops, pull-based scheduling, visual management techniques, and continuous improvement concepts with a customer-value focus across cross-functional teams.

Analysis

Well-established professions typically build their concepts around theoretical foundations. In The underlying theory of project management is obsolete, the authors recognize that production has many manifestations:

… competing theories of production (projects are just special instances of production) have existed even before the emergence of project management. Another concept of production was presented already in the framework of early industrial engineering. The flow view of production … is embodied in JIT and lean production. In a breakthrough book, Hopp and Spearman (1996) show that by means of the queueing theory, various insights, which have been used as heuristics in the framework of JIT, can be mathematically proven.

While Lean concepts rely on this theoretical foundation, and have recently gained traction among startups, they have had their greatest success fighting ‘big company disease’, through a focus on customer concerns and a systems approach across collaborative teams. Their biggest contribution has been helping to change thinking from a static plan-oriented focus to a dynamic flow-oriented model.

Let’s break down the features of each of these frameworks, drawing from this analysis:

If the reader studies this table, they may conclude no single option provides what they need.

Hybrid approaches

Hybrid approaches, while appealing, aren’t a panacea; common pitfalls include overhead and complexity, fragmented communications, dilution of core principles, tooling challenges, inconsistent metrics, and the ever-present resistance to change. Examples cited in the literature include:

• Teams using sprints but still following rigid upfront planning, losing Agile’s adaptability

• Isolated Agile adoption in which Agile is adopted locally but is not supported or compatible with other departments frameworks, creating delivery misalignment

• Organizations cherry-picking practices and adopting Agile rituals (like stand-ups) without embracing its principles, often resulting in limited adaptability

• Lean’s reliance on accurate forecasting clashes with Agile’s responsiveness, especially in volatile markets.

This is made worse when one realizes that few endeavors fully implement all the features in their situation, so relying on a hybrid puts the user in the difficult role of integrating parts designed for separate purposes on their own.

The bottom line:

The idea of a fixed method, or a fixed theory of rationality, rests on too naive a view of man and his social surroundings - Paul Feyerabend, in Against Method

In the next post in this series, we will confront the inevitable tensions between desires for autonomy and control.

Profit Beyond Measure details how two extremely profitable manufacturers, Toyota and the Swedish truck maker Scania, have rejected the traditional mechanistic mindset of managing by results that generates waste. Johnson and Broms explain how Toyota and Scania achieve their legendary cost advantage through a revolutionary concept they call managing by means (MBM). Instead of being driven to meet preconceived accounting targets, the production systems of Toyota and Scania are governed by the three precepts that guide all living systems: self-organization, interdependence, and diversity.

Amid a wealth of new insights into Toyota’s vaunted system, Johnson and Broms introduce the tools of MBM to show how design, production, and profitability analysis are done to customer order. They demonstrate that by following the principles that emulate life systems, even a lean and profitable company can organize work to greatly lessen its long-term earnings instability and sharply reduce its short-run operating costs.

Scania has achieved sixty-five years of financial stability and longevity in the face of fierce competition. Toyota has amassed a market value since 1988 that has rivaled - or sometimes surpassed - the American “Big Three” automakers combined. The principles that Johnson and Brams set forth in Profit Beyond Measure can guarantee the same richer, longer life to any company that applies them.

Just which jobs we consider complicated and which ones we consider elementary, even crude, is a judgment too often made not on the true nature of the work but on the things that attend it — the pay, the title, whether it’s performed in a factory or an office suite, in blue jeans or a gray suit. And while those are often reasonable yardsticks — a constitutional scholar may indeed move in a world of greater complexity than a factory worker — just as often, they’re misdirections, flawed cues that lead us to draw flawed conclusions about occupations we don’t truly understand. We continually ignore the true work people and companies do and are misled, again and again, by the rewards of that work and the nature of the place it is done.

It may not be much of a surprise that bosses and economic theorists don’t fully appreciate the complexity in many jobs. Both groups, after all, are interested mostly in performance, never mind how it’s achieved. More surprising is the fact that coworkers are often just as poorly informed about what the person one desk over or two spots down on the assembly line actually does all day. When they do give the matter any thought at all, they almost always conclude that the other person’s job is far less complicated than it is.

Feedback is essential for any performer to adjust their actions and converge on the output needed:

The longer it takes for the complexity intrinsic in the work to become apparent, the less useful feedback will be in guiding implementation and incorporating changes that will have lasting effects on performance. People are motivated to change their behaviors when their own interests are connected to business needs. Causes must be connected to effects so they can make sense of information and how it relates to a particular context and framing of their work. In The Principles of Product Development Flow, Don Reinertsen describes the role feedback plays in enabling teams to establish control over their environment:

Rapid feedback causes us to subconsciously associate our delivery of a work product and the beginning of the next process step. This attribution of causality gives us a sense of control, instead of making us feel we are the victim of a massive bureaucratic system.

In the book Improving Performance, Brache and Rummler describe the factors which affect the ability to perform work.

Collectively, these factors determine the extent to which planned work can be successfully executed and determine the extent to which the means of performing such work can be improved over time. These factors include:

1. Job Preconditions

   1. Are the inputs accessible and suitable for processing?

   2. Can the task be performed without interference?

   3. Are the procedures and workflow logical?

   4. Is help available if required?

2. Performance specifications

   1. Do job standards exist?

   2. Are they attainable?

   3. Are they accepted by the people who must perform the work?

3. Ability to execute

   1. Are the roles that individuals are assigned to perform consistent with their experience and preparation?

   2. Do the performers have sufficient capacity available to implement these roles within the time allocated?

4. Feedback

   1. Will feedback information be relevant, accurate, specific, understandable, and timely?

   2. Will the feedback be provided in a way that is useful to the performer and so that adjustments can be incorporated within the required period of performance?

5. Motivation

   1. Have incentives and consequences been established to support desired performance?

   2. Are the criteria for reporting progress and estimates of remaining work well designed, agreed to, and appropriate to the situation?

Simply put, the degree to which work is actionable determines the pace at which work can be acted on, and the extent to which rework can be avoided. Jobs are 'actionable' if and only if they satisfy all the above attributes. When some portion of a work package does not satisfy these attributes, the time it will take to perform that work will be more variable, since work products that will be produced will be more likely to have less utility than desired, and risks associated with achieving target outcomes will be increased. Teams select the most actionable work from a work package to perform first, even though this work may not be the most important work from the perspective of the customer; while it may enhance the team's ability to demonstrate progress in the short term, this approach may not produce the best long-term results.

Purpose

A Concepts of Operations artifact (aka "ConOps") should provide a useful overall depiction of needed capabilities using compelling visual representations. The information should provide a user-centric bridge from the operational goals and objectives of the system of interest to an operational view of how the system will be used within the environments targeted for deployment.

The narrative developed in the ConOps should limit the problem space for defined audiences over distinct time periods. It should document the sponsor and customer assumptions and intentions with respect to each operation or series of operations, especially when a series of connected operations must be carried out simultaneously or in succession. The resulting narrative perspective weaves the associated concepts for a system into a story about how the anticipated physical and technical capabilities of the system will be used to provide selected capabilities, such as resource or information aggregation, computation, synthesis, navigation, production, distribution, or monitoring over time.

A ConOps should provide powerful graphics that reinforce a written narrative that describes a 'big picture' perspective to the designers of the system. Its depiction should amplify and clarify the purpose of the endeavor. This perspective (often composed of multiple viewpoints) should help explain how a proposed solution is expected to work 'holistically' once it enters service. The interactions between solution elements should be collectively introduced in a way that captures the voice of the customers and emphasizes how the delivered capability will enable the harvesting of value by its stakeholders. Use case diagrams can supplement a ConOps further and frame the problem at a lower level but should not be necessary to characterize the essential attributes of the proposed solutions.

Expected Outcomes

A ConOps puts these interactions into a meaningful framework by describing:

• What outcomes are intended from using the system?

• What novel concepts are crucial to understanding how the system will be used to realize these outcomes?

• How will these concepts be utilized throughout the system's lifecycle?

What resources will users need to operate and maintain the system?

Content description

The following outline provides a suggested template for effectively communicating such information. Content described will vary depending upon whether for a new system of interest that is being developed from scratch, when a major upgrade to an existing system is to be performed, or whether the system is intended to replace an existing system.

• Current System or Situation

• Background, Objectives, and Scope

• Assumptions and constraints

• Operational policies and constraints

• Description of the current system or situation

• Stakeholders and personnel affected by change

• Justification for and nature of changes

  • Reason changes are necessary

  • Description of needed changes

  • Priorities of changes

  • Changes considered but not included

• Description new and changed system elements

  • Block diagram

  • Description of critical functions

  • Description of primary interfaces

  • Key timeline and interactions

  • Measures of effectiveness

  • Operational scenarios & modes

    • Initialization

    • Operations

    • Support

    • Modification

    • Service restoration after failures

• Summary of capabilities provided

  • System execution

  • Data management

  • Operations management

  • Systems management

• Analysis of the proposed system

  • Summary of benefits

  • Summary of limitations

  • Summary of alternatives and tradeoffs considered

Planned capabilities need to be oriented within the missions of each affected organization, with particular focus on:

• A delineation of the areas of responsibility, especially at the boundaries

• A timeline which details time budgets for activities

• A description of how progress will be tracked with respect to performance targets

• the resources allocated to perform each aspect of this mission

• how these resources scale under different scenarios, especially:

  • Innovation: the ability to introduce new materials, technologies, and methods, as well as to enhance traditional work in new ways

  • Robustness: the ability to maintain effectiveness across a range of tasks, situations, and conditions, including pivots in the face of unusual (and potentially unanticipated) states. These should include:

    • Resiliency: the ability to recover from or adjust to misfortune, damage, or a destabilizing perturbation in the environment

    • Responsiveness: the ability to react to changes to the environment in a timely manner

Flexibility should be provided to enable actors to employ multiple pathways to success and the capacity to move seamlessly between them. This contingent flexibility should include the following:

• The ability to make sense of the situation in which their role is to be performed

• The means for coordination with other actors, to enable them to work to together efficiently within the operational environment

• Knowledge, skills, and experience sufficient to recognize and respond to each situation

• The ability to select and execute a sequence of steps that will achieve the intended outcomes

While this information does not all need to be covered within the CONOPs artifact itself, it will enable the organizations involved to evaluate the clarity of the CONOPS, and its readiness to guide the subsequent efforts transforming the CONOPS information into a working system.

Summary

This ConOps information will be useful regardless of the domain or disciplines which will be involved in its implementation. If more projects used a ConOps approach, their probability of success will be increased, since the follow-on requirements elicitation process will have a better focus, allow clear delineation of responsibilities, and will be reinforced by a description of what “done” looks like. It is also particularly useful in onboarding new personnel to the endeavor.

For more information about using a CONOPS, see MITRE's guidance on this subject.

In Federalist paper #51, James Madison writes:

In framing a government which is to be administered by men over men, the great difficulty lies in this: you must first enable the government to control the governed; and in the next place oblige it to control itself. A dependence on the people is, no doubt, the primary control on the government; but experience has taught mankind the necessity of auxiliary precautions.

Madison was expressing concerns about limits to the ‘wisdom of the crowds’. In The Square and the Tower, Niall Ferguson highlights further the weaknesses of people in these situations:

…the defect of autocracy is obvious, too. No individual, no matter how talented, has the capacity to contend with all the challenges of imperial governance, and almost none is able to resist the corrupting temptations of absolute power.

Sometimes failures arise as a result of defects injected by agents themselves. But too often, it is a failure to understand the weaknesses of the methods they use and the dynamics of the collective games that are being played while using them. Let’s consider how frequent these problems arise.

A sampling of failures

The author of the book How Big Things Get Done, Bent Flyvbjerg (a professor at Aalborg University in Denmark) has been tracking over 16000 projects across 20+ different fields, and across 136 countries for over twenty years. His findings:

Only 8.5 percent of projects hit the mark on both cost and time. And a minuscule 0.5 percent nail cost, time, and benefits. Or to put that another way, 91.5 percent of projects go over budget, over schedule, or both. And 99.5 percent of projects go over budget, over schedule, under benefits, or some combination of these. Doing what you said you would do should be routine, or at least common. But it almost never happens.

Such measures depend upon when one begins counting. If you start when ideas are first generated, according to the article 3,000 raw ideas = 1 commercial success, it is even worse (see Figure 1):

Steve McConnell’s book More Effective Agile records his consulting business’s experience in over one thousand company engagements with Agile endeavors. As you can see in Figure 2, the performing organizations - on average - failed to apply Agile key practices consistently and effectively across this landscape, even when blessed with the benefits of experience from experts. Unless there is a ‘closed loop’ approach to such practices, this is exactly what one might expect given the inevitable cultural resistance that accompanies introducing such practices into routine practice. And without buy-in and practice, everyone hunkers down until another batch of promising silver bullets attracts attention yet again.

Several large endeavors are particularly noteworthy across the landscape of my experience. Such endeavors tend to be ambitious projects that fear delivering reduced capabilities which can erode trust and accountability in their sponsors, providers, and methods. When ‘selling’ the endeavor, the underlying mindset is to offer optimistic projections drawn from limited surveys while downplaying risks and uncertainty. Their common maladies typically involve overreach and minimal learning:

1. Transportation

   1. Puget Sound’s mass transit project, Sound Transit - according to Bloomberg - was to be a showcase that would show that providing better service would attract more riders and generate a virtuous cycle of growth. The current endeavor was approved by 54 percent of voters in 2016 and was to include sixty-two miles of light rail throughout the region, to be completed by 2041; the delivery was phased, and construction was well underway when the Bloomberg article was written in 2018. It has not gone well.
      
      One of the key segments was across the I-90 bridge, which moves with wind, water, and traffic. In 2022, it was discovered much of the bridge installation required extensive demolition and rebuilding (much as other segments have encountered, suffering of all things rail gauge incompatibilities). These issues pushed the original 2023 opening of cross-bridge traffic to 2026. The original $55B budget, which voters (paying over $1700 per household per year in new taxes) were assured had more than adequate contingency for any uncertainty, is now looking at between a $30-40B overrun (likely a conservative estimate). Five other segments to suburbs have also suffered delays of up to seventeen years according to current guesses, while their taxes, and additional overruns, continue unabated. Meanwhile, the current system carries less than 1% of all trips in the region, has failed to offload overcrowded roads, and can be twice as slow as buses over select routes.
      
      Such problems are always viewed as financial, with advocates arguing for additional revenue and deferring implementation still further, ignoring the root cause pathways adopted decades ago. Meanwhile, revenues from riders have tanked, as the adopted hub-and-spoke design reduces rider convenience and its required transfers lengthen end-to-end times. Luckily, self-driving cars are beginning to get approval for ride sharing in limited areas, and by the time the Sound Transit plans are completed, this more effective and less costly alternative will likely be ubiquitous.

   2. But it’s hardly the worst transportation endeavor currently in work. Consider California’s High-speed rail effort, authorized by a 2008 statewide initiative to deliver continuous service between San Francisco and Los Angeles with a nonstop travel time of one hundred sixty minutes. Phase 1 (five hundred miles long) was to be delivered by 2020, at a cost of $33B. Phase 1 is expected to share tracks with conventional trains, operating at a higher average rate than France’s fastest TGV trains. They were warned this was infeasible back in 2008. Costs to date have tripled and projected completion is thirteen years after original commitments. It’s no wonder that funding is questionable since the initial construction package will only connect Fresno and Bakersfield, over which little traffic is expected.

   3. How’s NASA doing in transporting astronauts to outer space? We are no longer hitching rides on Russia launch capability, but our astronauts still need a place to ride. The Orion Capsule is NASA’s crew spacecraft capable of supporting a crew of four beyond low earth orbit and has been under development since 2006. Its specifications call for twenty-one days of undocked service and six months of docked service, and are to be used by the Artemis program, NASA’s current lunar exploration endeavor. Orion has already spent $31B (2025 dollars) and struggles to recreate the mission first accomplished over sixty years ago (the total expenditures including the Space Launch System approach $100B). The maladies of the doomed effort are well chronicled here; suffice to say that if only cost and schedule were the problem, one could still kindle some hope, but it’s far, far worse than that. Meanwhile, SpaceX’s private space efforts continue to astound, with a vision pointing to establish a permanent Mars settlement, rather than completing another trip to the moon.

2. Government services

   1. Health care

      The U.S. health care system is innovative but fiscally unsustainable. It delivers cutting-edge treatments and short wait times for specialized care, but at the cost of uneven access and worse population health outcomes than other wealthy nations. For example, according to The Origins of Efficiency, the cost to develop a new drug has increased by a factor of 14 - in inflation adjusted terms - over the last 4 decades. These problems have worsened since the launch of the Affordable Care Act, which has relied on escalating infusions of subsidies, rather than effective cost control.

   2. Education

      Since the founding of the Department of Education:

      1. $3 trillion spent

      2. Reading scores for 13-year-olds have not improved

      3. Only 27% of 8th graders proficient in math

      4. Only 31% of 8th graders are proficient in reading

      5. 47 cents of every federal education dollar is spent on compliance without accountability

      
      If you’re defending the Department of Education, you’re defending a failure. For example, five years ago, according to a recent report from UC San Diego faculty, as few as 30 students arrived with math skills below high-school level. Now, that number is now more than 900, with most failing to fully meet middle-school math standards. These students can be expected to earn 8% less than graduates 12 years ago as a result of this shortfall.

   3. Megaprojects

      Look no further than The Line and its collapse.

Traditional frameworks thus appear to have had inadequate influence on outcomes for many types of endeavors.

Reality always requires investments in understanding

Existing frameworks focus on prescriptive content definitions for expected artifacts (many of which are rarely read or maintained in practice). When an endeavor is large, complex, and ‘too big to fail’, political will inevitably contaminates prudent judgement, a recipe for failure rather than success. A library of excuses - environmental reviews, inflationary costs, adjustments in scope, externalities - is quickly designated the cause, without confronting the uncertainty inherent in snapshots of reality against plans. Plans have at their core imperfect forecasts of the future based upon limited knowledge and less understanding, so assumptions must be made. These assumptions can inevitably unravel in long-lived projects, those guilty of overreach may have moved on to their next adventures, and a new wave of leaders emerge to claim that a comprehensive reset (with new assumptions) will curtail further sins - if stakeholders will just wait a while longer and shell out a lot more. Until they won’t.

True understanding is achieved by through adequate preparation in the ways things can go wrong, as described in the next post in this series.

The tension

Miranda’s musical celebrates Hamilton’s ambition and legacy, framing him as a hero of order and national coherence by favoring centralized, expert-led authority to tackle national problems like infrastructure, regulation, and technocratic governance. This pursuit often occurs in an atmosphere of Jeffersonian skepticism which champions local autonomy and individual liberty and is wary of elite control and centralized power. It’s a romanticized origin story of American governance, emphasizing the creative tension that forged the Constitution and early institutions.

The musical is an entertaining story, but reality is, as usual, more complicated than that. Marc J. Dunkelman’s Why Nothing Works explores why America, once capable of building massive infrastructure and social programs, now struggles to “get things done.” Everyone needs power but would prefer someone else to shoulder the burden of making that happen (Figure 1).

A case study

Dunkelman argues that endeavors often try to embody both of these impulses, which can lead to incompatible goals and conflicts. These conflicts can produce a vetocracy, where too many actors can veto action, stall progress and erode public trust. It creates a dynamic in which no one has the power or motivation to say “yes”, while anyone can delay and block action. The inevitable outcomes of these forces lead to repeated delays that can accumulate and result in cancellations.

This dynamic is highlighted in Chapter 8 of Dunkelman’s book, which uses electrical power distribution as a metaphor for the broader dysfunction in American governance. It illustrates how the tension between centralized control and decentralized autonomy undermines systemic reliability and reform. It also highlights the challenge of big ideas - for example, demanding national action on climate change (a Hamiltonian tendency) while resisting higher-level mandates that impinge on personal rights (the Jeffersonian approach).

In order for alternative energy sources to be developed, the means to transport that power from source to destination is crucial. The U.S. power grid evolved from a patchwork of local systems into a vast, interconnected infrastructure. Initially, power was generated and distributed locally, reflecting Jeffersonian ideals of community autonomy and responsiveness. But as demand grew and industrialization accelerated, the need for centralized coordination became unavoidable. High-voltage transmission networks and regional interconnects were introduced - hallmarks of Hamiltonian planning - and enabled bulk power delivery across states, improving efficiency and reliability, and striving for balance between the benefits of competition and the stability of monopolies that are core characteristics of public utilities.

However, Dunkelman argues that this centralization of planning and control came at a cost, as local actors lost control of their property and resources, and the system became vulnerable to fragmentation and political gridlock. Regulatory agencies like FERC and NERC were created to oversee reliability and standards for these assets, but their authority is often undermined by state-level resistance, utility lobbying, environmental causes, and fragmented jurisdiction. The result is a paradox: while the grid requires Hamiltonian coordination to function, Jeffersonian impulses - manifested in local vetoes, fragmented oversight, and resistance to federal mandates -prevent necessary upgrades, especially in integration and resilience planning for renewables.

This tension is especially evident in ongoing efforts to modernize the grid. For example, building interstate transmission lines for wind and solar power requires federal coordination, but local communities often block projects due to land use concerns or political ideology. Since these sources of clean power require these transmission lines to transport power from producer to consumer, the dynamic erodes action on climate change while resisting the centralized authority needed to implement it. This contradiction mirrors the broader theme of the book: America’s inability to reconcile its founding impulses now paralyzes its capacity to solve big problems.

Progress, far from consisting in change, depends on retentiveness. When change is absolute there remains no being to improve and no direction is set for possible improvement: and when experience is not retained, as among savages, infancy is perpetual. Those who cannot remember the past are condemned to repeat it. In the first stage of life the mind is frivolous and easily distracted, it misses progress by failing in consecutiveness and persistence. This is the condition of children and barbarians, in which instinct has learned nothing from experience. - George Santayana, The Life of Reason

Santayana highlights the importance of focus, experience, and learning over instinct. Just as Hamilton and Jefferson clashed over the role of government, modern America struggles to balance expert-led coordination with grassroots autonomy. Until this tension is acknowledged and addressed, Dunkelman warns, the nation will continue to suffer from systemic breakdowns - not just in electricity, but in governance itself.

A scorecard for traditional frameworks

There are plenty of competing frameworks that presume a control orientation, but all have limitations, and none provide explicit guidance for all the forces captured in the PIANOS model. It is worth stepping back and putting our system’s thinking hat on to understand why.

In their article Applying Systems Thinking to Engineering and Design, Monat and Gannon highlight the key principles of system thinking for production:

1. Holistic Perspective/Proper Definition of System Boundaries

2. Awareness that Events and Patterns are caused by Underlying Structures and Forces

3. Systemic Root Cause Analysis

4. Sensitivity to Emergent Properties

5. Sensitivity to Unintended Consequences

6. Dynamic Modeling

7. Focus on Relationships

8. Sensitivity to Feedback

Many endeavors employ these principles in practice, but none of the traditional frameworks explicitly seek to leverage these principles.

The dynamics

Let’s consider how dynamics of traditional frameworks unfold using the causal loop diagram shown in Figure 2. Such diagrams highlight the interactions of factors involved in achieving some result, in this case, the capacity of a production system. These interactions (labeled by the red arcs) have a polarity associated with each relationship, labeled with a plus or minus sign. This polarity indicates the influence one factor has on another - positive means ‘moves in the same direction’ and negative means ‘moves in the opposite direction. For example, the leftmost arc connects the cost of capital to the economy with a minus sign, meaning that in general (without considering other factors), when the cost of capital increases, the net effect will cause the economy to deteriorate. Alternatively, when the cost of capital decreases, the net effect will enhance the economy.

These factors also collectively form loops which accumulate the impacts of this polarity, and feed back to influence subsequent performance, leading to further growth or decline. These loops are labeled in the diagram with a prefix of R as reinforcing (creating growth) or B for balancing (leading to decline).

When building a simulation of these dynamics, the extent of each factor’s influence can be calibrated to match what is common knowledge, as long as the natural variation that accompanies each factor is taken account of.

The figure highlights the effects of resource contention - resources can be anything with enduring value, such as a home, a person’s health, and even their education. The ‘centralization of control’ factor in the lower left-hand corner is examined in more detail here. The running narrative for these interactions - applicable to any production system (whether governmental or industrial) - follows:

Loop R1: The Capability Investment Flywheel

At the heart of the diagram is a reinforcing feedback loop: production capacity increases → enabling more investments → which fund capability enhancements and learning opportunities → boosting labor productivity → which further expands production capacity. This loop reflects a virtuous cycle of growth, where reinvestment in human and technical capital drives compounding returns. It’s a Hamiltonian vision of centralized scaling through expertise and infrastructure.

However, when the economy takes a downturn - whether from a national crisis or demographic decline - this loop runs in reverse with limited options for recovery, all relying on the ability to attract buyers willing to finance additional investments with higher risk and lower returns.

Loop B2: Human Capital Foundations

This loop highlights how willing and able workers depend on upstream factors like:

• Effectiveness of education

• Effectiveness of public health

• Immigration (as a labor supply enhancer)

These elements feed into labor productivity and ultimately production capacity. But they’re also shaped by regulations, tax burden, and centralization of control, which can either enable or constrain public investment. This loop shows how social systems and governance deeply affect economic output, underscoring the Jeffersonian tension between local autonomy and centralized policy.

Loop B3: Resource Competition Dynamics

Here we see a more complex, potentially destabilizing loop:

• Rising production capacity boosts the economy, which increases resource demands → driving up prices.

• Higher prices may stimulate resource supply, but also increase debt and inflation, which can negatively impact cost of capital and reduce investment.

This loop introduces negative feedback which can increase systemic fragility. It shows how economic growth can trigger affordability crises and financial instability, especially if regulatory and monetary systems fail to adapt.

Loop B4: Resource Supply and Regulatory Friction

Demand on production increases pressure on the supply of resources, which can be constrained by regulations and tax burden.

• If these constraints are too tight, they reduce investment and slow the capability loop.

• If too loose, they risk environmental degradation or inequality.

This loop captures the dilemma this poses for governance: balancing growth with sustainability and equity. It’s where Hamiltonian efficiency meets Jeffersonian caution.

Loop B5: Popping economic bubbles

Once irrational exuberance begins fueling a speculative bubble, the threat of future price increases begins to spur investor enthusiasm, causing a contraction in the economy. There will be winners and losers around this transition.

The dynamic tensions in systems

The diagram doesn’t offer a single solution—it models a dynamic equilibrium shaped by competing feedback loops.

• Positive loops drive growth and innovation.

• Negative loops introduce checks, risks, and social costs.

• Governance levers (regulation, centralization, taxation) mediate these forces but are themselves contested and subject to dynamics.

The lower right factor Consensus on policies deserves elaboration. Consensus - the collective agreement or unified opinion reached by a group - doesn’t require unanimity, but it implies broad support or alignment. That only happens when leadership can orchestrate stakeholder negotiations and interests until a broadly acceptable solution emerges that everyone can live with. That relies upon a common understanding, acceptance, and support of the required tradeoffs which are necessary to realize the goal being pursued - all infused with patience for the dynamics to produce those results.

Achieving balance

In the Incremental Commitment Spiral Model, Barry Boehm emphasizes what he describes as The Fundamental System Success Theorem, which states: “A system will be successful if and only if it makes winners of its success-critical stakeholders.”:

One implication of the Fundamental System Success Theorem is that it is risky to use terms such as “optimize,” “minimize,” and “maximize” in project guidance. As Nobel Prize winner Herbert Simon showed in his book Models of Man, success in multiple-stakeholder situations is achieved not by optimizing, minimizing, or maximizing of individual criteria, but rather by satisficing with respect to the stakeholders’ multiple criteria.

To be accountable for a commitment, it is important to neither promise nor expect more than can be delivered. In a world of rapidly changing mission priorities, new technologies, competitive challenges, and organizational relationships, total commitment to the full details of a product to be delivered five years later is a very risky bet.

The term “satisficing” here means not everybody gets everything they want, but everybody gets an outcome that they can be satisfied with. Lacking such consensus, convergence on solutions may be fleeting, disappointing, and potentially overridden as power shifts with improved situational awareness, emerging political alliances, and new externalities drive dynamics. Adaptation is key to weathering this storm.

In the next post of this series, the scaffolding underneath these tensions is explored to lay the groundwork for understanding the common failure modes in endeavors. With these insights, you will be ready to practice how to avoid them.

The ideas behind the model are not intuitive, and the handbook lays out a strategy to introduce its ideas within an operational context. The graphics have been updated considerably from his prior works, which introduce the model's ideas but may not reveal its nuances.

Businesses have traditionally adopted a top-down command structure and flowed down strategic plans with a cascade of instructions through the ranks; unfortunately, many of these efforts lack effective feedback or the means to exploit it when available. This structure is simply too slow and inflexible to deal with environments presenting increasing rates of change and levels of complexity. It turns out that reporting relationships matter less than understanding how the complex parts of a business fit together to form a synergistic whole.

Yet when more flexible structures are adopted to better enable autonomy, this same freedom may reduce cohesion, meaningful cooperation, and synergy, which constrain the sharing of knowledge and information necessary to respond to situations rapidly and effectively across organizational boundaries. It is not possible to share or digest enough data; leaders can only recognize patterns sufficiently within their locus of control.

In their book, Profit Beyond Measure, authors H. Thomas Johnson and Anders Broms describe how blunt our tools are when we attempt to translate goals into the actions to be taken by others:

Probably the ultimate root of the modern habit of using quantitative abstractions to represent, or stand in place of, concrete things is language, "the mode of behavioral coordination through which humans bring forth the world they create with each other thing from its context". By using words, human beings give abstract shape to the world we occupy. Language lets human being separate a particular thing from its context. Formulated in a phrase, a tangible thing can then be analyzed intellectually, as if it existed apart from its physical context in the real world. Because of language, ?there is no limit to what we can describe, imagine, and relate [as mental abstractions]".

Since language allows humans to objectify and depersonalize particular things, its power is formidable. When we use language to create abstractions and to generalize, we remove and separate particular things from their context and treat them as if they were not part of a pattern in a living system. Thus, the first use of a simple word as "tree" begins to separate us from the natural habitat we share with other living beings. Once we objectify "tree? with a word we stop living among the lungs of the earth in the wild, and begin to see trees as the source of results such as fruit, shelter, tools, transportation, shade, decoration, wind barrier, writing surfaces and toilet paper. By the twentieth century, trees, almost extinct as species, became a harvestable crop in tree farms, where the primary result is to maximize the projection of "fiber" for industrial and management purposes. Thus, managing by result - managing to secure a "sustainable" supply of fiber- subject to the the web of relationships in the natural forest system to the stresses and strains of clear cutting, thereby threatening the long term viability of the natural system.

In the same way that language lets us think abstractly about fiber and trees, language also allows us to treat the abstract concept of the business result as if it were an actual object that existed separately from the messy, organizational context which gives rise to it. Viewed as an independent entity, this abstraction, this quantifiable business result that we entertain only in our minds, seems to be more real and concrete than does the real situation from which it emanated in the first place.

It is not a great leap, then, from seeing abstract results as concrete realities, to trying to manage them by arbitrarily manipulating the relationships from which they emanate. Such manipulation involves separating ends from means, or goals from the acts that achieve them. In Western culture the separation of means (the way our goal was achieved) from ends (the goal or what is to be achieved) has often been accompanied by a belief that the end or goal is immutable and prominent, and the means are ephemeral and changing. Therefore ends seem more "real" and more "valuable" than the means.

The underlying elements described by Beer are collectively holistic and autonomic. Small businesses may exhibit these functions with direct reporting relationships with the owner, whereas in large organizations they may require examination of organization charts with titles and responsibilities to discern their interrelationships.

Key concepts

Variety

In cybernetics, the number of distinguishable items (or distinguishable states of some item) is called the “variety’. So we may sum up by saying that the output variety must (at least) match the input variety for the system as a whole and for the input arrangement and the output arrangement considered separately. This is a vitally important application of Ashby’s Law of Requisite Variety, which says that control can be obtained only if the variety of the controller (and in this case of all the parts of the controller) is at least as great as the variety of the situation to be controlled. This, like all profound statements of natural law, is perfectly obvious once it has been pointed out.

There is no great difficulty, however, in finding examples of attempted control systems which disobey this law quite flagrantly, and therefore do not succeed. From traffic control to the control of the national economy, this fallacy is apparent; indeed, this is one of the key problems of control in a firm. For management always hopes to devise systems that are simple and cheap, but often ends up by spending vast sums of money to inject requisite variety which should have been designed into the system in the first place.

Multinodes

We claim we know how the whole thing works. The problem is to make it work more quickly. That must surely mean the introduction of discipline and order, of some sort, into the situation. It also means, however, that no measures may be adopted which would at the same time put the remarkable freedom of action and the wonderful flexibility of the multinode in jeopardy. If people could see how to do this, without putting themselves and their organization into a straight-jacket, there is some chance that they would adopt new techniques. One method, we ought now to agree, must be excluded although it is the one method most usually attempted in practice, because no one can think of anything else. This is the method of rigorous protocol. Explicitly: it denatures the system itself - with all its in-built capacity to generate the right answers.

The first difficulty is to know what kind of problem the multinode actually solves. It does not devote itself, its seniority and power, to the determination of trivial outcomes or it ought not to do so. It is likely to be settling a policy of great importance and therefore considerable complexity. Thus it is that people think of thinking as a process of synthesizing an integral but elaborate conclusion from a large number of component parts. The decision is seen as a rococo edifice built up clause by legal clause. This is perhaps why there are endless drafting problems facing anyone trying to promulgate an agreed decision.

The cybernetician adopts the contrary position. The output of the thinking process, the decision, has the following form: do this (rather than anything else). When the process of thinking originally starts, the multinode is faced not indeed with a number of building bricks in an edifice but with a seemingly infinite number of possible outcomes between which it must choose. It is the existence of this plethora of possibilities that cries out for decision in the first place. Then, under this model, the process we seek to assist is one of chopping down ambiguities and uncertainties until we may say: do this. In short, we would like to measure the variety of the complex decision at the start, measure the reduction of variety brought about by each conclusion reached in the process of thinking things out, and in general monitor the entire operation of the multinode as the variety comes down to a value of the decision itself. To do this we shall need two tools: a paradigm of logical search, and an actual metric - a rule and a scale - for measuring uncertainty.

Viability

An organization is viable if it can survive in a particular sort of environment.

When people refer to the firm, or any other institution, as ‘viable’, they are often referring to economic viability. From this preoccupation with the economic dimension stems the assumption that most of our problems are economic too.

Solvency, it is true, is a prerequisite of business activity- to trade while insolvent is illegal. Profitability, too, is a prerequisite- lack of business confidence is not illegal, but it is lethal. So these affairs are of primary concern. But they do not (as many suppose) constitute the goals of the enterprise. Rather are they the constraints under which it operates. So of course the regulation of cash-flow (for example) is an important managerial concern. And of course profit-consciousness in the private sector, or the consciousness of social benefit in the public sector, has to apply to expenditure; and proper returns on capital are required.

However: the tendency in ‘the city’, and among financial journalists, to treat all this as the essence of viability, is to mistake the epiphenomena of the system - the appearances and flurries of activity that prove it is actually there- for the system itself. Which of the firm’s workers, or even middle-managers, could recognize the old place by its indices of ROI, P/E ratios, and the rest? These things are abstractions, and very useful ones too, if we want to manipulate successfully our economic constraints.

Requisite variety

A measure of the complexity with which management has to deal.

You may well say that the number of possible states in a complicated entrepreneurial system is not precisely countable. That is surely correct. However, it is countable in principle: it is therefore amenable to the making of comparative statements (this has more variety than that), and to the arithmetic of ordinal numbers (this product is the fifth most profitable).

Resource bargains

The Resource Bargain is the ‘deal’ by which some degree of autonomy is agreed between the Senior Management and its junior counterparts. The bargain declares: out of all the activities that System One elements might undertake, THESE will be tackled (and not those), and the resources negotiated to these ends will be provided.

Principles of organization

1. Managerial, operational, and environmental varieties diffusing through an institutional system tend to equate; they should be designed to do so with minimal damage to the people and to cost

2. The four directional channels carrying information between the management unit, the operation, and the environment must each have a higher capacity to transmit a given amount of information relevant to variety; selection in a given time than the originating subsystem has to generate it in that time.

3. Wherever the information carried on a channel capable of distinguishing a given variety crosses a boundary, it undergoes transduction; the variety of the transducer must be at least equivalent to the variety of the channel.

4. The operation of the first three principles must be cyclically maintained through time without hiatus or lags

Two aphorisms

• It is not necessary to understand the workings of a black box to understand the nature of the function it performs.

• It is not necessary to understand the workings of a black box to calculate the variety that it potentially may generate.

Axioms

1. The sum of horizontal variety disposed by n operational elements (systems one) equals the sum of the vertical variety disposed by the six vertical components of corporate cohesion. (The six are from Environment, System Three*, the System Ones, System Two, System Three and Algedonic alerts.)

2. The variety disposed by System Three resulting from the operation of the First Axiom equals the variety disposed by System Four.

3. The variety disposed by System Five equals the residual variety generated by the operation of the Second Axiom.

System types

Using the architecture of the human cortex as a pattern, diverse range of specialization structures and backgrounds, and weave them into an ability to mitigate the constraints of bounded rationality, and getting reliable results out of unreliable components, especially under conditions of high uncertainty and complexity, by provide redundant inputs and assuring timely closure in decision-making processes.

• What is the purpose?

• What is the direction?

The types are numbered, with lower level (higher numbered) systems using algedonic signals to ‘wake up’ higher systems. Investments are used as variety attenuators.

Multiple 3-2-1 systems operate as peers (and on a traditional organization, would be depicted horizontally), providing separate functions, such as engineering, marketing, production,

Performance components

System 1 - Processes

The primary purpose of system 1 is to sense and make sense of what has been sensed in order to do something that will be rewarded by its environment. Operations are managed through a regulatory center, which fans out its inputs to a variety of measuring points, which are incapable of measuring everything. The overall processes must establish:

• What activities are to be done?

• What resources are to be allocated to perform these activities?

• Who will be held accountable if resource bargains are broken?

• How will operational exceptions be managed, whether to triage issues or adjust behaviors?

• How will business opportunities be sanctioned and managed?

System 2 - Coordination & Orchestration

System 1 activities may get in each other’s way. This can cause oscillations that result from unconstrained schedule glitches, budget disruptions and problems with enabling infrastructure. Operational units should comply with the various policies and standards, not because they are ordered to do so, but because they offer tangible benefits/control/cohesion, resource management, performance management, compliance management, monitoring/audit, or coordination.

Iterations clarify:

• How is coordination or orchestration provided appropriate to the situation?

• What protocols must each multinode perform to support the higher regulatory authority?

System 3 - Internal monitoring and control

This requires self-organization and autonomic regulation, making inside and now decisions across the collection of System 1 activities within their emerging situation, allocating time, money, space and attention. It also must clean up excess variety that that falls through the cracks of regular arrangements.

If things go wrong and levels of risk increase the System 3 asks for help or puts it to colleagues for a remedy. This is the pain of an algedonic alert, which can be automatic when performance fails to achieve capability targets. The autonomic 3?2?1 homeostatic loop's problem is absorbed for a solution within the autonomy of its metasystem.

Measurements are used to match people, machines, and money to jobs that produce useful products or services. In a set of processes, some jobs are done by one person. Some are done by many and often many processes are done by the same person. Throughout the working day a participant, in completing a task, may find the focus shifts between internal and external Systems 1?5 from moment to moment.

The choices, or decisions discriminated against, and their cost (or effort) defines the variety and hence resources needed for the job. The processes (Systems 1) are operationally managed by System 3 by monitoring performance and assuring (System 2) the flow of product between System 1s and out to users.

System 3 is able to audit (via 3*) past performance so "bad times" for production can be compared to "good times". If things go wrong and levels of risk increase the System 3 asks for help or puts it to colleagues for a remedy.

Whilst specialist groups may be used to help provide focus and impetus for the functions of System 4, they should be augmented by a range of integrative mechanisms to align the separate narratives competing for attention.

Includes additional feedback on the true state of operations across the business entities. via independent audits. they must be sporadic and infrequent, since routine audits relinquish much of the variety they generate.? Second, they must be openly declared mechanisms to avoid concerns about excessive corporate control.? Third, they must only link two adjacent levels of recursion to avoid a breakdown in trust or an overload of unnecessary information.

• How are activities to be defined, managed and monitored?

• How will assurance activities be used to verify performance?

System 4 - Outside and future

The Intelligence function is the two-way link between the primary activity (ie. Viable System) and its external environment. Intelligence is fundamental to adaptivity; firstly, it provides the primary activity with continuous feedback on marketplace conditions, technology changes and all external factors that are likely to be relevant to it in the future; secondly, it projects the identity and message of the organization into its environment. These loops must operate in balance, to avoid either overloading the system with a swamp of external research data without the capacity to interpret and act on that data; or the alternative risk of communicating outwards in a strong fashion, without having a corresponding means to listen for feedback from the marketplace.? The intelligence function is strongly future-focused. It is concerned with planning the way ahead in the light of external environmental changes and internal organizational capabilities so that the organization can invent its own future (as opposed to being controlled by the environment). To ensure that its plans are well-grounded in an accurate appreciation of the current organizational context, the intelligence function also needs to have at its disposal an up-to-date model of the organization.

The Intelligence function is the two-way link between the primary activity (ie. Viable System) and its external environment. Intelligence is fundamental to adaptivity; firstly, it provides the primary activity with continuous feedback on marketplace conditions, technology changes and all external factors that are likely to be relevant to it in the future; secondly, it projects the identity and message of the organization into its environment. These loops must operate in balance, to avoid either overloading the system with a swamp of external research data without the capacity to interpret and act on that data; or the alternative risk of communicating outwards in a strong fashion, without having a corresponding means to listen for feedback from the marketplace.? The intelligence function is strongly future-focused. It is concerned with planning the way ahead in the light of external environmental changes and internal organizational capabilities so that the organization can invent its own future (as opposed to being controlled by the environment). To ensure that its plans are well-grounded in an accurate appreciation of the current organizational context, the intelligence function also needs to have at its disposal an up-to-date model of the organization.

Pattern recognition in chess is possible without considering and grading every possible move. A filter should be used that can exploit pattern recognition capable of recognizing patterns in the developing, imminent future. It must acquire relevant criteria from knowledge about the rules (or corporate ethos). The typical strategy one adopts until a depth picture emerges is to strengthen oneself. Remember the decision not to act is a current action.

System 5 - Identity and policy

The last function, giving closure to the system as a whole, is the policy-making function. This function is by definition low-variety (in comparison with the complexity of the rest of the organizational unit and the even larger complexity of the surrounding environment); it, therefore, needs to be highly selective in the information it receives. This selectivity is largely achieved through the activities and interactions of the Intelligence and Control functions.? The main roles of Policy are to provide clarity about the overall direction, values, and purpose of the organizational unit, and to design, at the highest level, the conditions for organizational effectiveness.

Miller argues that living systems exist at eight nested hierarchical levels, not unlike matryoshka dolls. These levels begin at the cellular level and build into organs, organisms, groups, organizations, communities, societies, and supranational systems. Miller argues that each of these levels can be studied from a common viewpoint, that of nineteen comparable subsystems which each interact within their enclosing and subordinate structure to form a holistic system, as Figure 1 depicts. Each of these subsystems operates within a context which is appropriate to the level of the system. Collectively, all achieve transformations of space and time, matter, energy or information, performing functions such as locomotion, manipulation, communications, and expulsion.

Regardless of scale, state, or perspective, these living systems have both form and function and have properties that can be observed and categorized. As these patterns are recognized and decomposed, a foundation for reasoning can be formed on topics such as the best possible arrangement and provisioning of parts for achieving some purpose. To pursue this purpose, each entity must make effective choices from real options available to influence its future, given a set of initial conditions, so that beneficial emergent properties can unfold across space, scale, and time.

He drills into each of these subsystems as both a categorization scheme (or logical architecture) and with a specialist's love of details, revealing both the structure and processes of each subsystem. He also goes to a great deal of effort to explain key characteristics of these patterns across all levels; for example, he devotes one whole chapter (of eighty pages) to the topic of information overload, a topic essential to understanding the forces and constraints at play. He also provides a full-color plate in which he points out common characteristics of these subsystems within the level being studied (three are depicted below).

An Organ

An Organism

An Organization

Highlights

3. Subsystems

3.1 Subsystems which process both matter-energy and information
3.1.1 Reproducer
3.1.2 Boundary

3.2 Subsystems which process matter-energy
3.2.1 Ingestor
3.2.2 Distributor
3.2.3 Converter
3.2.4 Producer
3.2.5 Matter-energy storage
3.2.6 Extruder
3.2.7 Motor
3.2.8 Supporter

3.3 Subsystems which process information
3.3.1 Input transducer
3.3.2 Internal transducer
3,3,3 Channel and net
3.3.4 Decoder
3.3.5 Associator
3.3.6 Memory
3.3.7 Decider
3.3.8 Encoder
3.3.9 Output transducer

4. Relationships among subsystems or components

4.1 Structural relationships
4.1.1 Containment
4.1.2 Number
4.1.3 Order
4.1.4 Position
4.1.5 Direction
4.1.6 Size
4.1.7 Pattern
4.1.8 Density

4.2 Process relationships
4.2.1 Temporal relationships
4.2.1.1 Containment in time
4.2.1.2 Number in time
4.2.1.3 Order in time
4.2.1.4 Position in time
4.2.1.5 Direction in time
4.2.1.6 Duration
4.2.1.7 Pattern in time

4.2.2 Spatial-temporal relationships
4.2.2.1 Action
4.2.2.2 Communication
4.2.2.3 Direction of action
4.2.2.4 Pattern of action
4.2.2.5 Entering or leaving containment

4.3 Relationships among subsystems or components which involve meaning

5. System processes

5.1 Process relationships between inputs and outputs

Matter-energy inputs related to matter-energy outputs

1. Matter-energy inputs related to information outputs

2. Information inputs related to matter-energy outputs

3. Information inputs related to information outputs

5.2 Adjustment processes among subsystems or components, used to maintain stability
5.2.1 Matter-energy input processes
5.2.2 Information input processes
5.2.3 Matter-energy internal processes
5.2.4 Information internal processes
5.2.5 Matter-energy output processes
5.2.6 Information output processes
5.2.7 Feedback

5.3 Evolution - Emergence, growth, cohesiveness, and integration

5.5 Pathology

1. Lack of matter-energy inputs

2. Excesses of matter-energy inputs

3. Inputs of inappropriate forms of matter:energy

4. Lack of information inputs

5. Excesses of information inputs

6. Inputs of maladaptive genetic information in the template

7. Abnormalities in internal matter processes

8. Abnormalities in internal information processes

5.6 Decay and termination

As system engineers are prone to say, the rest is just details, though that would be missing Miller's point. His conclusions emphasize (especially at the level of supranational systems), that it is far better to recognize such patterns and deal with them proactively than to wait until decay and termination set in. This insight relates to our responses to threats such as cybersecurity and related changes in our current social, technological, and political environment, though only if the mitigations are appropriate, timely, and effective.

Economists’ thinking about production has evolved from simple input–output rules to today’s nuanced, multi-factor frameworks. These conceptual shifts include transitioning:

1. From an engineering metaphor to an economic optimization tool

2. From fixed technologies to technology itself as an endogenous, evolving factor

3. From two-factor simplicity to multi-factor, multi-output complexity

4. From static efficiency to dynamic productivity and innovation analysis

This evolution in thinking originally rested on the concept of production as a black box and examination of what powers it. Let’s open this box and see what’s inside.

Means of production

The concept of the means of production, depicted here in Figure 1, is widely used to signify the relationship between things used as inputs to a production system and the constituent mechanisms needed to provide those inputs - factories, tooling, systems, services, and methods - whether in a system of interest, or within an economy or society.

Karl Marx focused on these means of production to distinguish them from contributions by labor. Elon Musk sees them as levers for exponential innovation and automation. Two components of these means - resources and capabilities - deserve further detailing. For orientation for my intended use of the term power, see here.

Resources as power

Production for a system of interest consumes resources provided by upstream providers (who produce them) and produces resources for downstream consumers (who consume them). Energy is one of the most critical resources - the capacity to do work - since it fuels all actions, from powering a data center, lifting an object, or running a machine.

Energy comes in various forms - kinetic, potential, thermal, electrical, etc. Power is the rate at which these forces can be applied in accomplishing work or transferring resources. Think of it as the speed of doing work. It can tell us how effectively resources are being used to accomplish useful work. Energy is a stock, while power is a flow. If unfamiliar with this terminology, see this post.

The three primary resources, as shown in Figure 2, are time, talent, and treasure (capital), as those allow effort to be applied, capabilities to be matured, and objects to be transformed. While the selection of the other resources - forces, information, and raw materials - are important for efficiency, the primary forces enable better objects to be used in assembly, greater competency to be incorporated into capabilities, and better domain knowledge to be leveraged in performing the work.

Capabilities as power

Capabilities (Figure 3) provide the ability to execute specified courses of action, and guide and constrain agents in performing these activities.

The concepts underpinning capabilities have had the greatest reach in military applications today, reflecting how armed forces think about applying power in conflicts while keeping a mission focus. The applications of these concepts focus on preparation - to understand how well forces can reliably employ resources within anticipated situations. This focus goes beyond traditional planning, which focuses on raw numbers of resources and equipment, to consider the capacity to successfully achieve desired courses of action. When considered in this context, capabilities are sources of power, but unlike resources, are not consumed in usage (though they can be degraded with time).

Maladies in production

Many things inevitably go wrong in accomplishing this work, especially when affected by complexity, external influences, resource shortages, quality breakdowns, and garbled communications. In this post we are focusing on the implicit structure for each production operation when encountering these situations. These are only apparent when observing patterns of operations across the requisite variety of contexts. When they arise, they lead to internal pivots in straightforward processing, or off-ramps from further processing. Such maladies are chronic, problematic conditions that introduce disruptions in production equilibrium, whether biological, psychological, or systemic. They often signal:

• A misalignment between internal states and external demands

• Flaws in communications and exchanges

• Breakdowns in incorporating information from feedback loops

• An inability to effectively adjust resource allocations as adverse conditions arise

Let’s examine the many types of pitfalls that are at the root of these production maladies:

1. Shortfalls or incorrect resources (time, talent, treasure, information) provided for the work

2. Breakdowns in preparation (orientation, architecture, stakeholder engagement) or orchestration of required actions

3. Inadequate attention to risks or change management

4. Slips and lapses in performing the work (resulting in defects that require rework)

   1. Mistakes in analyzing what is needed for the work to be performed effectively

   2. Performing work out of sequence

   3. Mistakes in performing the work itself

   4. Failure to properly document the work

5. Externalities adversely injecting change into endeavor

6. Improper use of tools - including automation - to situation

7. Incomplete testing of functions, performance, structures, or failure modes

8. Procedural disconnects (mismatches between situations and domain knowledge)

Each of these maladies threatens the orderly flow of work and thus adversely impacts throughput across each step of production.

The Minimum Viable Production Flow

If we are going to orchestrate production adequately, we must account for all these steps and transformations that occur, track the frequency, pace, and rework of each step, and use that information to learn to apply and mature capabilities and means of production to minimize non-value-added detours through this flow. The underlying steps and feedback loops are depicted in a stock and flow model, in which flows are represented by valve symbols representing transfer functions and stocks are represented as boxes depicting reservoirs. The processing steps are described generically here but are mappable to most situations. Note that inputs and outputs required by each step are not shown here but are implied in separate writings about portfolio management.

Once jobs are committed, the work is distributed across performing agents, depicted in Figure 6 as multiple parallel efforts for each step (when steps indicate depth):

We want our model of these dynamics to be viable for representing as many situations as possible, while being as simple as possible. Performance of production systems is non-linear and classically follows what is described as an S-curve - the sigmoid function whose graph is shown in Figure 7. This pattern is consistent with the elements described in figure 3 of this post.

In Brian Potter’s book The Origins of Efficiency, these dynamics are described as follows:

When we talk about production methods, we’re really talking about technologies… a production process can be thought of as a large collection of different technologies strung together to accomplish a particular goal… For individual technologies, progress tends to follow an S-shaped curve, with time or effort on the horizontal axis and technological performance on the vertical axis.

Early on, the technology performs extremely poorly, if it works at all. The phenomenon at play in the technology may have only recently been discovered and as a result is poorly understood. It might not yet be clear how it behaves under different conditions, or what arrangement of components can best take advantage of it, or even for what purposes it might be harnessed. Fixing one problem with a nascent technology tends to simply reveal more problems, so significant time and effort might be invested without any noticeable increase in performance.

But over time, as scientists, engineers, and tinkerers explore different ways of implementing it, the technology’s characteristics become better understood. As people begin to figure out what works and what doesn’t, the search space of the technology is narrowed, and it attracts more talent and funding. As attention converges on the most promising avenues for advancement, performance improves more quickly… This gradual refinement that leaves the basic nature of the technology unchanged is often called incremental or evolutionary improvement. During this period, the technology might converge on a dominant design: a specific way of implementing the technology that can be easily adapted to serve the needs of many potential users.

Eventually, a technology’s performance approaches some natural limit: the maximum level of performance that the given effect or principle at work or the structure of the dominant design can achieve. As the technology approaches this limit, gains in performance are harder and harder to achieve, and the rate of improvement slows.

As the legend in Figure 6 indicates, the steps in our production process manifest performance through four phases (estimates below are approximations for a typical small to medium-sized endeavor):

1. Inception: the scope, goals, expected benefits, risks, and feasibility of the endeavor is assessed. The effort (~5%) and schedule (~10%) at this point are small but establish the baseline for future acceleration.

2. Elaboration: The requirements, strategy, and structure of the work are explored, and the necessary details are captured, organized into jobs ready for assignment, and baselined. The curve begins to steepen as more resources are engaged, and the definition of jobs and methods stabilizes. This phase involves ~20% of the overall effort and ~30% of the schedule.

3. Construction: As jobs are committed and prepared, actionable elements of the work elements are initiated, work in process is generated, results of that work are stabilized, and solution elements are aligned, with a focus on functionality and performance. This represents the bulk of cost (~65%) and schedule (~50%) consumption, powered by peak learning and velocity.

4. Transition: The solution is validated and made available for delivery, with training, validation, and final adjustments. Effort (~10%) flattens and schedule (~10%) reflects wrap-up activities.

The pathway through this landscape is challenging. Each step’s performance has a probability distribution reflecting the natural variation of processing. This variation is amplified as this variation, and its corresponding limits of capacity, inject uncertainty, incorporate feedback from downstream steps, as these functions interact, and adjust to maladies encountered along the way, rinsing, and repeating with each time increment.

Transfer functions

Transfer functions characterize each action’s output for each input. There will be many complex inputs in addition to what is shown in Figure 6; those collectively influence decisions that are abstracted into the Power value that feeds the rates of each step in the flow. Such functions can be implemented by human agents or agentic transformers.

The steps depicted in Figure 6 are outlined below, with flows (the transfer functions) stylized in bold, stocks in italics, and the transition indicated by an arrow character in the following lists. The normal flow (aka “happy path”) is described in the outermost, numbered outline level. These steps typically have one or more feedback loops that are captured underneath each step as indented, exception processing pathways which inject rework into its upstream steps, resulting in delays.

Step sequences

1. Observation: Signals → Opportunities

   Observation is the act of noticing, watching, or measuring something to gather information. These observations can be passive or active; each involves:

   • Perception: Detecting the signals through sensing and communications

   • Attention: Focusing on specific signals while filtering out noise

   • Interpretation: Deriving meaning from perception free of influence from prior knowledge or bias

   Signals come in many flavors - orders, resource shifts, support obligations, and expedite requests among them - but each requires ingredients in amounts appropriate to each situation. Noise can obfuscate these signals; indeed, when the level of noise exceeds the signal, chaos dominates the search for worthwhile opportunities, and can increase the likelihood of subsequent pivots, or influence decisions about needing to ignore opportunities. Incentives must be sufficient to amplify effort sufficiently to effectively capture the opportunities despite the noise.

2. Orientation: Opportunities → Goals

   Goals are a bridge between intentions and execution. Orientation must validate the feasibility of each opportunity (considering timing, constraints, and achievable throughput), confirm relevance with priorities and values, and translate visions into outcomes that realize the intended successes. Goals should follow best practices and describe what success looks like in unambiguous terms, anchoring opportunities with timing to reinforce urgency, provide room for course corrections, and enable evolution as feedback reveals new insights or conditions.

   1. Ignoring Opportunities

      Not all opportunities are deemed worthwhile to pursue, so some number will out of necessity be ignored. Delays in processing are an indicator that such disposition should be considered.

3. Analysis: Goals → Requirements

   Goals are analyzed and translated into requirements through the capture, refinement, and elaboration of context, intentions, and constraints into specific, testable conditions that must be satisfied.

   1. Pivoting towards different Opportunities

      Some goals are worthwhile in pursuing an opportunity but require revision prior to analysis.

   2. Discarding Goals

      Other goals are not critical to pursuing the opportunity and can be discarded.

4. Organization: Requirements → Jobs

   In his book Competing against Luck, Clayton Christensen describes jobs from a customer’s perspective; progress involves understanding what that customer is trying to do in particular circumstances, and what decisions they must make. This step elaborates statements of work, allocates responsibilities, and distributes requirements across jobs for responsible agents.

   1. Revisiting Requirements with respect to Goals

      As requirements are fully considered, the need for clarification, conflict resolution, and addressing missing elements arises, forcing further work on the related requirements.

5. Commitment: Jobs → Preparation

   Commitments require a psychological and relational progression through

   three phases:

   • Understanding: Gaining clarity about the context and expectations for the jobs

   • Acceptance: Acknowledging and embracing the reality of these inputs and their relationship to the jobs being assigned (including responsibilities, relationships, limitations, and challenges)

   • Support: Agreeing to accept these responsibilities and collaborate upstream and downstream to ensure adequate progress is achieved

   Preparation ensures actions connect to broader goals or narratives and empowers responsible agents to navigate this progression so that work can proceed in a straightforward fashion. This preparation is the traditional role that planning performs and enables parallel action in subsequent steps if sufficient agents and resources are available.

   1. Refactoring Jobs given new Requirements

      Requirements changes can necessitate adjustments to job structure considering those changes.

6. Provisioning: Preparation → Actionable work

   Once the parties responsible have committed to perform the necessary work, preparation is performed to define the steps necessary, required interdependencies, and secure the resources necessary for implementing these steps. Work becomes actionable by provisioning the inputs necessary for execution.

   1. Enhancing Preparation given new Jobs

7. Initiation: Actionable work → Work in process

   Although work may be actionable, many factors can delay launching the effort:

   • Cognitive friction: Complexity can overwhelm; goals and requirements can lack clear definition; and uncertainty around prioritization and sequencing can paralyze action.

   • Emotional and relational factors: Anxiety, low motivation or energy, and insufficient alignment on values can burden progress with negotiation or avoidance.

   • Structural barriers: Poor integration of tools, unclear feedback mechanisms, multitasking, and resource shortages (promised but unfulfilled) can erode progress.

   To optimize flow, these constraints should be addressed rather than passed downstream. Initiating action converts potential into momentum, shifting from planning to doing.

   1. Augmenting Requirements after consideration of Actionable work

      Revisions to requirements are typically triggered by new insights into stakeholder needs, technical feasibility, risk exposure, or strategic alignment. These insights often emerge from real-world feedback, evolving conditions, or deeper understanding gained in implementing these requirements.

8. Generation: Work in Process → Results

   Each action is primed by clear intentions and requirements which set the direction and define what success looks like. Interactions with other actions and related systems feed checkout, incorporate feedback, and enable progress to be accumulated.

   1. Discovering new Requirements while performing Work in Process

      See 7.a

9. Inspection: Work in Process → Reviews

   Reviews are a mechanism to generate signals that inform whether to continue, refine, or pivot work in process and candidate solutions. The entry conditions for such reviews are an important consideration in organizing the work.

   1. Abandoning Work in Process

      Effort invested in preparing for and performing work accumulates; when work is abandoned, those efforts are wasted and contribute to the opportunity costs of the endeavor.

10. Incorporation: Review results into Work in Process

    Typical dispositions of review findings include acceptance, conditional acceptance, revision requests, rejection, and escalation for further action. These outcomes vary depending on the context - legal, academic, regulatory, or organizational - but they reflect the reviewer’s judgment on the adequacy, accuracy, and implications of the material reviewed. Additional actions required to achieve this disposition are added to the work in process.

11. Stabilization: Results → Solution elements

    Elaboration, trial, and error powers the maturation of results into candidate solution elements.

    1. Correcting Results

       When results require correction relative to

12. Alignment: Solution elements → Candidate solutions

    Holistic solutions require that the pieces interact properly and collectively satisfy requirements.

    1. Diagnosing problems with Solution elements

13. Validation: Candidate solutions → Progress

    Each proposed solution must be evaluated against real-world conditions before being released into service.

    1. Isolating problems with Candidate Solutions through Reviews

The gears driving production rates

As Figure 6’s legend indicates, each step can be characterized into one of three categories, using the metaphor of a funnel:

1. Exploration: High Variability, Low Throughput

   In exploration, we are in the wide part of the funnel - the scope of searches is wide, and the entropy is high.

   1. Characteristics

      1. Purpose: Discover options, surface unknowns, generate hypotheses

      2. Inputs: Often ambiguous or incomplete

      3. Outputs: Possibilities, insights, candidate pathways

   2. Throughput Implications

      1. Rate: Low and uneven; progress is nonlinear and often recursive

      2. Variation: High given the low quality of information available; cognitive load, novelty, and branching paths reinforce this unpredictability

      3. Constraints: Bottlenecks often stem from lack of clarity, insufficient framing, or premature convergence.

2. Evaluation: Moderate variability and throughput

   Evaluations narrow the funnel through the incorporation of structured filtering.

   1. Characteristics

      1. Purpose: Compare, prioritize, validate, and assess trade-offs

      2. Inputs: Structured options or hypotheses

      3. Outputs: Ranked choices, decisions, or filtered paths

   2. Throughput Implications

      1. Rate: Moderate. Can accelerate with clear criteria and decision matrices.

      2. Variation: Medium; depends on complexity of criteria and stakeholder alignment

      3. Constraints: Bottlenecks arise from unclear metrics, cognitive bias, or decision fatigue

3. Transformation: Low Variability, High Throughput

   These steps exploit upstream processing by operating as the nozzle of a funnel does, allowing agents to ‘turn the crank’ and deliver at high rates.

   1. Characteristics

      1. Purpose: Execute, build, incorporate, implement, or convert

      2. Inputs: Validated decisions or designs

      3. Outputs: Tangible results suitable for follow-on steps

   2. Throughput Implications

      1. Rate: High. Execution benefits from repeatability and automation

      2. Variation: Low; standardized inputs yield predictable outputs

      3. Constraints: Bottlenecks shift to resource availability, tooling, or integration points

Each exception processing step in the step sequences above are explorations for context by searching for the relevant details of the presenting conditions so the underlying problem can be accurately diagnosed. Transformations and evaluations have their own puzzles to be solved, but the context in both those cases is known - to produce a useful result within the attack surface of the territory and goal in question.

Wrap-up

Now that you understand this DNA of production generically, it should become apparent that even straightforward pathways can quickly become quite complicated. Such complications can be difficult to unwind, especially for the inexperienced. Practice and learning are the best medicine for such dilemmas. Such learning requires understanding each of the writings in this series, animating the above dynamics under alternative initial conditions and injections by externalities, and rehearsing the actions expected by each of the

Figure 1 is a depiction of a real machine which was used around 60 A.D. for removing standing water from underground mines. These machines are known as reverse overshot water-wheels, and one of them was recovered from a copper mine at Rio Tinto in Spain and is now at the British Museum.

We know about these devices from the writings of Vitruvius, the Roman architect. He described a specific installation consisting of thirty-two water wheels, mounted and stacked above one another within the vertical shaft of a Roman mine. Each wheel was turned by one or more individuals who walked along the top of the water wheel to spin them.

A batch of water was raised up a specified distance as each worker turned their assigned wheel. This water then became a queue of work for the next higher wheel and worker to process. Each worker could only consume the output of the prior worker after enough water had accumulated at their assigned stage.

The pace of the slowest worker in the chain would thus constrain the end-to-end throughput potential across the entire device. Despite this limitation, their efforts were collectively able to lift the underground water about 200 vertical feet from the mines where they were installed, which was an impressive accomplishment for that time, and which gave the endeavors who used this technology advantage over other competing approaches, such as a bucket brigade.

This device can be considered from two distinctive perspectives. The first viewpoint consists of its capability to raise a column of water by a particular distance; we'll call this a water elevation system. The second perspective uses such a capability in the context of a larger ore extraction system; its primary goal is to economically produce ore which can then be provided as a resource for many other purposes that are not relevant to our example. The extraction of ore was financed by investors who saw this potential. Not all mines had water draining into the tunnels being mined, but when it did, the water represented a constraint which blocked the potential realization of value that could be achieved.

The economics of such ore extraction is determined by the value that ore has in the marketplace, the number of resources necessary to produce that ore, and how long it takes to produce it. Each mine would have unique characteristics, such as the volume of water flowing into the mine, and the height to which the water needed to be raised. The operation of an ore extraction system is only economically viable within a performance envelope and operating context which each device was designed to serve. The throughput of this water extraction system had to be sufficient to consume inflows from all sources of water and any accumulated pool of water present at startup.

Note that the ore extraction system and the water elevation system are interdependent since each requires the other to provide value. The costs to develop and operate the water elevation system will be bounded by the economics of the ore and its transport to a location where the ore is used as an input to another system that would consume the ore, such as a smelter.

The water elevation system would require water to be drained sufficiently so that ore could be extracted from the mine. One can envision water seeping into these underground mines from many sources, such as from rainfall, springs, or underground streams. Each of these inflows may have a varying rate over time, such as when a sudden rainstorm occurred. It might not take many people to extract water during a dry period, and it might take a long time after a rainy period to remove sufficient water for the ore extraction to be resumed. Delays in developing or operating the water elevation system could lead to delays in the time it takes to perform ore extraction, and thus significantly reduce the time-adjusted value of that ore. The actions and skills required to produce operational outputs are fundamentally different from those necessary to develop new capabilities that could extract water more efficiently and effectively.

In this example, the ore is an output of a collection of activities we can call a mining value chain. The water wheels which were used to raise this water required personnel, equipment, information, and tools to realize this needed capability, as determined by many factors, including the volume of water which each wheel could consume, the pace at which workers could operate the system, and the leakage which occurs across the elements of the system itself.

During operation, pools of water formed at the boundaries between each processing stage performed by each wheel. The size of these pools can provide a visual indicator of the backlog of the work cells that are accomplishing that work. Such queues are inevitable in any development or production process, due to variation in the quality of requirements, the capacity of team members, and the information available to the team.

Development environments share most of the features of the water elevation system described above. In development settings, a cascading process is typically employed, as depicted in Stuart Pugh's core design activity model of the interactions of technologies and technique shown in figure 3; note that in this depiction, the vertical flow is depicted top to bottom, unlike in the water wheels, which are subject to gravity, but may also be distributed horizontally.

Each stage requires horizontal integration of the elements relevant to the structure of activities and material relevant to that stage, though the translation into wedges of activity depicted may vary by stage. The flow among team members involved in such designs can be thought of as echoing our cascading water elevation system.

The overall throughput of a team is a function of the interactions between the team members. Feedback can be used to manage the rates of injecting work (be it new or rework) into the system. The queues of work in process between team members can be used to provide visual indicators of the flow of this work as it moves through the system. The size of these queues will be dependent upon whether a batch production, job production, or continuous production technique is being used. A description of managing such flow is here.

Production decisions regarding the allocation of resources across work stages are often driven by the robustness of the forecasting information available, the cycle time variation inherent in each operation, and the quantity necessary for efficient economic operations.

The size of each batch to be processed across each stage of such an operation is critical for many reasons:

• Communications costs grow as connections are n**n for n units, and many interactions are often necessary to coordinate the work across these channels

• Large batch sizes limit efficiencies that may be achieved by performing similar and routine operations together

• Large batch sizes also reduce the usefulness and effectiveness of feedback to improve the system, which complicates connecting cause and effect when something goes wrong.

The value of a batch of work can only be evaluated within the context of the environment within which that work is being performed. A tool (like the water wheels used to provide the capability to elevate the water in question) is only a means to an end; in the case of mining, the immediate mission is to extract marketable ore from geological deposits. Far too often, we are inclined to measure value by quantifying the resources which are visible, or which are easy to measure, rather than by the flow of this work in the context in which value is realized. This can cause us to measure work by the number of people who descend into the mines each day, rather than considering the efficiencies being realized in the various stages of the flow. As Donald Reinertsen indicates:

Product developers create a large number of proxy objectives: increase innovation, improve quality, conform to plan, shorten development cycles, eliminate waste, etc. Unfortunately, they rarely understand how these proxy objectives quantitatively influence profits. Without this quantitative mapping, they cannot evaluate decisions that affect multiple interacting objectives...

Life would be quite simple if only one thing changed at a time. We could ask easy questions like, "Who wants to eliminate waste?" and everyone would answer, "Me!" However, real-world decisions almost never affect only a single proxy variable. In reality, we must assess how much time and effort we are willing to trade for eliminating a specific amount of waste.

A modern, kid-powered version of one stage of this water wheel is in use today for pumping water in underdeveloped countries. In field use, such devices have often failed to meet expectations, and have proved to be more costly, less reliable, and harder to use than the child's play they were marketed to be.

Regardless of the outcomes expected from such production systems - be they technological maturity, new product development, or service excellence - value can only be realized once solutions have been completed and have been placed into service. This can only occur at the rate of acceptance of the methods by users and the throughput of the system overall.

Until then, investments are just holding costs for an uncertain future. They merely represent an inventory of work which has begun but which has not yet been completed. This inventory can be particularly large for software-intensive efforts since the inventory is less tangible than it is for physical assets. This means that investments in labor may exceed the costs of materials, especially when the costs of paying for this labor pool over time are made visible. The size of this inventory is equally critical, since the value they offer may become stale over time, as opportunities for usage and customer enthusiasm for requested work tend to evaporate during dry seasons.