More than some assembly required
A jigsaw puzzle is a classic afternoon diversion for one or more people. Variants are now available in three dimensions and are of many shapes and sizes. The biggest commercially available puzzle is about 18,000 pieces. These puzzles are solvable because the pieces are designed to fit together. Considerable assembly is required, but fans of jigsaw puzzles can enjoy this diversion because they have a high degree of confidence that the whole thing can be made from the parts provided, and that these parts will mesh functionally, esthetically, and mechanically. But today’s large-scale systems are far bigger, and with that scale brings a host of other complications.
For a thought experiment, consider the added challenge for our puzzle scenario that would arise if you had hundreds of thousands of pieces to assemble, rather than hundreds or thousands, and imagine having hundreds - or even thousands - of different people putting these parts together, after design and manufacture elsewhere.
Consider the challenges of integrating these parts together after contracting separately for boxes of pieces to be produced independently based upon their color, rather than having them all laser cut from a single tool by a single method. Try to wrap your head around the realization that this puzzle and its pieces might need integration across more than three dimensions. Parts interact in many ways besides just spatially, due to their dynamic properties, persistent operational modes and states, the failure modes of elements and contribution to the whole’s integrity, and the intrinsic material and electrical properties of the pieces which the puzzles must be built from to illuminate the pieces individually.
Now confront the added challenge when the puzzle, its constituent parts, and their interfaces, are all undergoing constant refinements, and are being produced at different rates, according to the capabilities of the producers involved. Everything is in motion as it evolves towards a usable condition. Pieces arrive for assembly at various times and have been built to different versions of the interface definitions, and interpretation of those interfaces. Some of this is because of delays in transporting required knowledge, and other delays can occur due to communications delays and breakdowns as the responsible groups learn of and try to incorporate that information.
Even though the pieces may all have been specified as being woven from the same material, in practice, the materials obtained from separate sources are not particularly uniform and have constrained availability. As a result, some parts are thicker than expected, and some are thinner than you need. Some are stiffer than your process can tolerate, and some may be unacceptably flimsy. One supplier's blue is what another considers teal. In another, when a tab doesn't really seem to quite fit into the matching slot.
You have a dilemma - if you use a lot of pressure, you find it sort of can be forced to fit - is that good enough? Determining whether such a match is correct or not may take much longer than you'd like. Isolating this flaw to the appropriately offending piece - the tab or the slot, the specification, or its realization - is time-consuming, yet necessary to support the rates of integration required to meet schedules.
What if the puzzle is so big that no one can really see the whole thing at once. Different people would be working on various parts of the puzzle at separate times, sometimes in parallel efforts. As they try to share insights, they are calling the puzzle piece makers, trying to understand their vision, and how these parts are all supposed to fit into the whole picture. The people putting sections of the puzzle together are talking to the people who developed the picture as it was envisioned at one point in time, and trying to understand whether they have the right pieces, whether they are trying to insert them in the right place, and trying to figure out why things don't seem to quite fit. Under fatigue, everyone's vision is a little blurry, and everyone is prone to slips and oversights, exacerbating the communications problems that need to be resolved in order to be successful.
This is something like what large-scale integration is like if expressed in metaphorical, 'puzzly' terms. In practice, many additional integration challenges are still not fully represented in this analogy, especially the need to craft a balanced solution across stakeholders with different requirements, needs, and viewpoints of the overall situation. To make it fully representative, our analogy would need to be a puzzle as big as a building, with interactions involving power, cooling, structural integrity, maintenance, and aesthetics, while still answering questions such as the following:
Who are the intended users of the puzzle, and in what ways will their usages be made most productive?
Who will be the owner of the finished product, and how can their specific desires for uniqueness be achieved?
Who will be the responsible parties for developing, integrating, and supporting the components of the finished product over its anticipated performance envelope and lifecycle?
How will the businesses who are involved in this creation extract value in this endeavor over time?
This is why integration activities require a viable platform, a robust strategy to organize the work flowing through that platform, and a credible tracking system to measure progress and highlight disconnects. If your table or floor (the platform) keeps collapsing, you will not be able to make much real progress, and perceived progress may be deceiving if you end up having to move things to a different platform later.
How might you estimate how long it will take to create and roll out this product, and have it look like what it's supposed to look like when you are done. Do you estimate based upon the number of pieces?
Consider the impact that inaccurate estimates might have on the overall effort, especially when the endeavor is unprecedented. Consider the constraints that must be confronted:
the confidence in specifications and material properties
the robustness of component designs and interaction scenarios
the capabilities of the team
the stability and strength of the means of production
the complexities of the supply chain
Indeed, all these might represent risks that could be difficult to properly take account of, until relevant experience has been accumulated.
This is why it's hard to predict how long it will take to develop large, complex systems, especially in unprecedented ways, even if there is a good understanding of how to chunk the work, what each part should cost to produce, and with commitments to produce all the pieces.
Once you finally think you have a picture in mind that's stable (at least according to your vision and beliefs), it's easy to assume that the challenges are behind you. However, until an accurate picture of the integrated pieces emerges, no one can know how long things will take, because putting together puzzles of increasing size doesn't scale any better than putting together large software projects, especially as things are still changing, and people are still sorting out how parts should fit together.
This series offers key strategies to consider for any large-scale systems integration effort that will enable driving towards a stable implementation.