risk management | Life, Project Management, and Everything

Shake it Out – Embracing the Future of Program Management – Part Two: Private Industry Program and Project Management in Aerospace, Space, and Defense

In my previous post, I focused on Program and Project Management in the Public Interest, and the characteristics of its environment, especially from the perspective of the government program and acquisition disciplines. The purpose of this exploration is to lay the groundwork for understanding the future of program management—and the resulting technological and organizational challenges that are required to support that change.

The next part of this exploration is to define the motivations, characteristics, and disciplines of private industry equivalencies. Here there are commonalities, but also significant differences, that relate to the relationship and interplay between public investment, policy and acquisition, and private business interests.

Consistent with our initial focus on public interest project and program management (PPM), the vertical with the greatest relationship to it is found in the very specialized fields of aerospace, space, and defense. I will therefore first begin with this industry vertical.

Private Industry Program and Project Management

Aerospace, Space & Defense (ASD). It is here that we find commercial practice that comes closest to the types of structure, rules, and disciplines found in public interest PPM. As a result, it is also here where we find the most interesting areas of conflict and conciliation between private motivations and public needs and duties. Particularly since most of the business activity in this vertical is generated by and dependent on federal government acquisition strategy and policy.

On the defense side, the antecedent policy documents guiding acquisition and other measures are the National Security Strategy (NSS), which is produced by the President’s staff, the National Defense Strategy (NDS), which further translates and refines the NSS, and the National Military Strategy (NMS), which is delivered to the Secretary of Defense by the Joint Chiefs of Staff of the various military services, which is designed to provide unfettered military advise to the Secretary of Defense.

Note that the U.S. Department of Defense (DoD) and the related agencies, including the intelligence agencies, operate under a strict chain of command that ensures civilian control under the National Military Establishment. Aside from these structures, the documents and resulting legislation from DoD actions also impact such civilian agencies as the Department of Energy (DOE), Department of Homeland Security (DHS), the National Aeronautics and Space Administration (NASA), and the Federal Aviation Administration (FAA), among others.

The countervailing power and checks-and-balances on this Executive Branch power lies with the appropriation and oversight powers of the Congress. Until the various policies are funded and authorized by Congress, the general tenor of military, intelligence, and other operations have tangential, though not insignificant effects, on the private economy. Still, in terms of affecting how programs and projects are monitored, it is within the appropriation and authorization bills that we find the locus of power. As one of my program managers reminded me during my first round through the budget hearing process, “everyone talks, but money walks.”

On the Aerospace side, there are two main markets. One is related to commercial aircraft, parts, and engines sold to the various world airlines. The other is related to government’s role in non-defense research and development, as well as activities related to private-public partnerships, such as those related to space exploration. The individual civilian departments of government also publish their own strategic plans based on their roles, from which acquisition strategy follows. These long terms strategic plans, usually revised at least every five years, are then further refined into strategic implementation plans by various labs and directorates.

The suppliers and developers of the products and services for government, which represents the bulk of ASD, face many of the same challenges delineated in surveying their government counterparts. The difference, of course, is that these are private entities where the obligations and resulting mores are derived from business practice and contractual obligations and specifications.

This is not to imply a lack of commitment or dedication on the part of private entities. But it is an important distinction, particularly since financial incentives and self-interest are paramount considerations. A contract negotiator, for example, in order to be effective, must understand the underlying pressures and relative position of each of the competitors in the market being addressed. This individual should also be familiar with the particular core technical competencies of the competitors as well as their own strategic plans, the financial positions and goals that they share with their shareholders in the case of publicly traded corporations, and whether actual competition exists.

The Structure of the Market. Given the mergers and acquisitions of the last 30 years, along with the consolidation promoted by the Department of Defense as unofficial policy after the fall of the Berlin Wall and the lapse of antitrust enforcement, the portion of ASD and Space that rely on direct government funding, even those that participate in public-private ventures where risk sharing is involved, operate in a monopsony—the condition in which a single buyer—the U.S. government—substantially controls the market as the main purchaser of supplies and services. This monopsony market is then served by a supplier market that is largely an oligopoly—where there are few suppliers and limited competition—and where, in some technical domains, some suppliers exert monopoly power.

Acknowledging this condition informs us regarding the operational motivators of this market segment in relation to culture, practice, and the disciplines and professions employed.

In the first case, given the position of the U.S. government, the normal pressures of market competition and market incentives do not apply to the few competitors participating in the market. As a result, only the main buyer has the power to recreate, in an artificial manner, an environment which replicate the market incentives and penalties normally employed in a normative, highly diverse and competitive market.

Along these lines, for market incentives, the government can, and often does, act as the angel investor, given the rigorous need for R&D in such efforts. It can also lower the barriers to participation in order to encourage more competition and innovation. This can be deployed across the entire range of limited competitors, or it can be expansive in its approach to invite new participants.

Market penalties that are recreated in this environment usually target what economists call “rent-seeking behavior.” This is a situation where there may be incumbents that seek to increase their own wealth without creating new benefits, innovation, or providing additional wealth to society. Lobbying, glad-handing, cronyism, and other methods are employed and, oftentimes, rampant under monosponistic systems. Revolving-door practices, in which the former government official responsible for oversight obtains employment in the same industry and, oftentimes, with the same company, is too often seen in these cases.

Where there are few competitors, market participants will often play follow-the-leader and align themselves to dominate particular segments of the market in appealing to the government or elected representatives for business. This may mean that, in many cases, they team with their ostensible competitors to provide a diverse set of expertise from the various areas of specialty. As with any business, profitability is of paramount importance, for without profit there can be no business operations. It is here: the maximization of profit and shareholder value, that is the locus of power in understanding the motivation of these and most businesses.

This is not a value judgment. As faulty and risky as this system may be, no better business structure has been found to provide value to the public through incentives for productive work, innovation, the satisfaction of demand, and efficiency. The challenge, apart from what political leadership decides to do regarding the rules of the market, is to make those rules that do exist work in the public interest through fair, ethical, and open contracting practices.

To do this successfully requires contracting and negotiating expertise. To many executives and non-contracting personnel, negotiations appear to be a zero-sum game. No doubt, popular culture, mass media and movies, and self-promoting business people help mold this perception. Those from the legal profession, in particular, deal with a negotiation as an extension of the adversarial processes through which they usually operate. This is understandable given their education, and usually disastrous.

As an attorney friend of mine once observed: “My job, if I have done it right, is to ensure that everyone walking out of the room is in some way unhappy. Your job, in contrast, is to ensure that everyone walking out of it is happy.” While a generalization—and told tongue-in-cheek—it highlights the core difference in approach between these competing perspectives.

A good negotiator has learned that, given two motivated sides coming together to form a contract, that there is an area of intersection where both parties will view the deal being struck as meeting their goals, and as such, fair and reasonable. It is the job of the negotiator to find that area of mutual fairness, while also ensuring that the contract is clear and free of ambiguity, and that the structure of the instrument—price and/or cost, delivery, technical specification, statement of work or performance specification, key performance parameters, measures of performance, measures of effectiveness, management, sufficiency of capability (responsibility), and expertise—sets up the parties involved for success. A bad contract can no more be made good than the poorly prepared and compacted soil and foundation of a house be made good after the building goes up.

The purpose of a good contract is to avoid litigation, not to increase the likelihood of it happening. Furthermore, it serves the interests of neither side to obtain a product or service at a price, or under such onerous conditions, where the enterprise fails to survive. Alternatively, it does a supplier little good to obtain a contract that provides the customer with little financial flexibility, that fails to fully deliver on its commitments, that adversely affects its reputation, or that is perceived in a negative light by the public.

Effective negotiators on both sides of the table are aware of these risks and hazards, and so each is responsible for the final result, though often the power dynamic between the parties may be asymmetrical, depending on the specific situation. It is one of the few cases in which parties having both mutual and competing interests are brought together where each side is responsible for ensuring that the other does not hazard their organization. It is in this way that a contract—specifically one that consists of a long-term R&D cost-plus contract—is much like a partnership. Both parties must act in good faith to ensure the success of the project—all other considerations aside—once the contract is signed.

In this way, the manner of negotiating and executing contracts is very much a microcosm of civil society as a whole, for good or for bad, depending on the practices employed.

Given that the structure of aerospace, space, and defense consists of one dominant buyer with few major suppliers, the disciplines required relate to the details of the contract and its resulting requirements that establish the rules of governance.

As I outlined in my previous post, the characteristics of program and project management in the public interest, which are the products of contract management, are focused on successfully developing and obtaining a product to meet particular goals of the public under law, practice, and other delineated specific characteristics.

As a result, the skill-sets that are of paramount importance to business in this market prior to contract award are cost estimating, applied engineering expertise including systems engineering, financial management, contract negotiation, and law. The remainder of disciplines regarding project and program management expertise follow based on what has been established in the contract and the amount of leeway the contracting instrument provides in terms of risk management, cost recovery, and profit maximization, but the main difference is that this approach to the project leans more toward contract management.

Another consideration in which domains are brought to bear relates to position of the business in terms of market share and level of dominance in a particular segment of the market. For example, a company may decide to allow a lower than desired target profit. In the most extreme cases, the company may allow the contract to become a loss leader in order to continue to dominate a core competency or to prevent new entries into that portion of the market.

On the other side of the table, government negotiators are prohibited by the Federal Acquisition Regulation (the FAR) from allowing companies to “buy-in” by proposing an obviously lowball offer, but some do in any event, whether it is due to lack of expertise or bowing to the exigencies of price or cost. This last condition, combined with rent-seeking behavior mentioned earlier, where they occur, will distort and undermine the practices and indicators needed for effective project and program management. In these cases, the dysfunctional result is to create incentives to maximize revenue and scope through change orders, contracting language ambiguity, and price inelasticity. This also creates an environment that is resistant to innovation and rewards inefficiency.

But apart from these exceptions, the contract and its provisions, requirements, and type are what determine the structure of the eventual project or program management team. Unlike the commercial markets in which there are many competitors, the government through negotiation will determine the manner of burdening rate structures and allowable profit or margin. This last figure is determined by the contract type and the perceived risk of the contract goals to the contractor. The higher the risk, the higher the allowed margin or profit. The reverse applies as well.

Given this basis, the interplay between private entities and the public acquisition organizations, including the policy-setting staffs, are also of primary concern. Decision-makers, influences, and subject-matter experts from these entities participate together in what are ostensibly professional organizations, such as the National Defense Industrial Association (NDIA), the Project Management Institute (PMI), the College of Scheduling (CoS), the College of Performance Management (CPM), the International Council on Systems Engineering (INCOSE), the National Contract Management Association (NCMA), and the International Cost Estimating and Analysis Association (ICEAA), among the most frequently attended by these groups. Corresponding and associated private and professional groups are the Project Control Academy and the Association for Computing Machinery (ACM).

This list is by no means exhaustive, but from the perspective of suppliers to public agencies, NDIA, PMI, CoS, and CPM are of particular interest because much of the business of influencing policy and the details of its application are accomplished here. In this manner, the interests of the participants from the corporate side of the equation relate to those areas always of concern: business certainty, minimization of oversight, market and government influence. The market for several years now has been reactive, not proactive.

There is no doubt that business organizations from local Chambers of Commerce to specialized trade groups that bring with them the advantages of finding mutual interests and synergy. All also come with the ills and dysfunction, to varying degrees, borne from self-promotion, glad-handing, back-scratching, and ossification.

In groups where there is little appetite to upend the status quo, innovation and change, is viewed with suspicion and as being risky. In such cases the standard reaction is cognitive dissonance. At least until measures can be taken to subsume or control the pace and nature of the change. This is particularly true in the area of project and program management in general and integrated project, program and portfolio management (IPPM), in particular.

Absent the appetite on the part of DoD to replicate market forces that drive the acceptance of innovative IPPM approaches, one large event and various evolutionary aviation and space technology trends have upended the ecosystem of rent-seeking, reaction, and incumbents bent on maintaining the status quo.

The one large event, of course, came about from the changes wrought by the Covid pandemic. The other, evolutionary changes, are a result of the acceleration of software technology in capturing and transforming big(ger) dataset combined with open business intelligence systems that can be flexibly delivered locally and via the Cloud.

I also predict that these changes will make hard-coded, purpose-driven niche applications obsolete within the next five years, as well as those companies that have built their businesses around delivering custom, niche applications, and MS Excel spreadsheets, and those core companies that are comfortable suboptimizing and reacting to delivering the letter, if not the spirit, of good business practice expected under their contracts.

Walking hand-in-hand with these technological and business developments, the business of the aerospace, space and defense market, in general, is facing a window opening for new entries and greater competition borne of emergent engineering and technological exigencies that demand innovation and new approaches to old, persistent problems.

The coronavirus pandemic and new challenges from the realities of global competition, global warming, geopolitical rivalries; aviation, space and atmospheric science; and the revolution in data capture, transformation, and optimization are upending a period of quiescence and retrenchment in the market. These factors are moving the urgency of innovation and change to the left both rapidly and in a disruptive manner that will only accelerate after the immediate pandemic crisis passes.

In my studies of Toynbee and other historians (outside of my day job, I am also credentialed in political science and history, among other disciplines, through both undergraduate and graduate education), I have observed that societies and cultures that do not embrace the future and confront their challenges effectively, and that do not do so in a constructive manner, find themselves overrun by it and them. History is the chronicle of human frailty, tragedy, and failure interspersed by amazing periods of resilience, human flourishing, advancement, and hope.

As it relates to our more prosaic concerns, Deloitte has published an insightful paper on the 2021 industry outlook. Among the identified short-term developments are:

A slow recovery in passenger travel may impact aircraft deliveries and industry revenues in commercial aviation,
The defense sector will remain stable as countries plan to sustain their military capabilities,
Satellite broadband, space exploration and militarization will drive growth,
Industry will shift to transforming supply chains into more resilient and dynamic networks,
Merger and acquisitions are likely to recover in 2021 as a hedge toward ensuring long-term growth and market share.

More importantly, the longer-term changes to the industry are being driven by the following technological and market changes:

Advanced aerial mobility (AAM). Both FAA and NASA are making investments in this area, and so the opening exists for new entries into the market, including new entries in the supply chain, that will disrupt the giants (absent a permissive M&A stance under the new Administration in Washington). AAM is the new paradigm to introduce safe, short-distance, daily-commute flying technologies using vertical lift.
Hypersonics. Given the touted investment of Russia and China into this technology as a means of leveraging against the power projection of U.S. forces, particularly its Navy and carrier battle groups (aside from the apparent fact that Vladimir Putin, the president of Upper Volta with Missiles and Hackers, really hates Disney World), the DoD is projected to fast-track hypersonic capabilities and countermeasures.
Electric propulsion. NASA is investing in cost-sharing capabilities to leverage electric propulsion technologies, looking to benefit from the start-up growth in this sector. This is an exciting development which has the potential to transform the entire industry over the next decade and after.
Hydrogen-powered aircraft. OEMs are continuing to pour private investment money into start-ups looking to introduce more fuel-efficient and clean energy alternatives. As with electric propulsion, there are prototypes of these aircraft being produced and as public investments into cost-sharing and market-investment strategies take hold, the U.S., Europe, and Asia are looking at a more diverse and innovative aerospace, space, and defense market.

Given the present condition of the industry, and the emerging technological developments and resulting transformation of flight, propulsion, and fuel sources, the concept and definitions used in project and program management require a revision to meet the exigencies of the new market.

For both industry and government, in order to address these new developments, I believe that a new language is necessary, as well as a complete revision to what is considered to be the acceptable baseline of best business practice and the art of the possible. Only then will organizations and companies be positioned to address the challenges these new forms of investment and partnering systems will raise.

The New Language of Integrated Program, Project, and Portfolio Management (IPPM).

First a digression to the past: while I was on active duty in the Navy, near the end of my career, I was assigned to the staff of the Office of the Undersecretary of Defense for Acquisition and Technology (OUSD(A&T)). Ostensibly, my assignment was to give me a place to transition from the Service. Thus, I followed the senior executive, who was PEO(A) at NAVAIR, to the Pentagon, simultaneously with the transition of NAVAIR to Patuxent River, Maryland. In reality, I had been tasked by the senior executive, Mr. Dan Czelusniak, to explore and achieve three goals:

To develop a common schema by supporting an existing contract for the collection of data from DoD suppliers from cost-plus R&D contracts with the goal in mind of creating a master historical database of contract performance and technological development risk. This schema would first be directed to cost performance, or EVM;
To continue to develop a language, methodology, and standard, first started and funded by NAVAIR, for the integration of systems engineering and technical performance management into the program management business rhythm;
To create and define a definition of Integrated Program Management.

I largely achieved the first two during my relatively brief period there.

The first became known and the Integrated Digital Environment (IDE), which was refined and fully implemented after my departure from the Service. Much of this work is the basis for data capture, transformation, and load (ETL) today. There had already been a good deal of work by private individuals, organizations, and other governments in establishing common schemas, which were first applied to the transportation and shipping industries. But the team of individuals I worked with were able to set the bar for what followed across datasets.

The second was completed and turned over to the Services and federal agencies, many of whom adopted the initial approach, and refined it as well to inform, through the identification of technical risk, cost performance and technical achievement. Much of this knowledge already existed in the Systems Engineering community, but working with INCOSE, a group of like-minded individuals were able to take the work from the proof-of-concept, which was awarded the Acker in Skill in Communication award at the DAU Acquisition Research Symposium, and turn it into the TPM and KPP standard used by organizations today.

The third began with establishing my position, which hadn’t existed until my arrival: Lead Action Officer, Integrated Program Management. Gary Christle, who was the senior executive in charge of the staff, asked me “What is Integrated Program Management?” I responded: “I don’t know, sir, but I intend to find out.” Unfortunately, this is the initiative that has still eluded both industry and government, but not without some advancement.

Note that this position with its charter to define IPM was created over 24 years ago—about the same time it takes, apparently, to produce an operational fighter jet. I note this with no flippancy, for I believe that the connection is more than just coincidental.

When spoken of, IPM and IPPM are oftentimes restricted to the concept of cost (read cost performance or EVM) and schedule integration, with aggregated portfolio organization across a selected number of projects thrown in, in the latter case. That was considered advancement in 1997. But today, we seem to be stuck in time. In light of present technology and capabilities, this is a self-limiting concept.

This concept is technologically supported by a neutral schema that is authored and managed by DoD. While essential to data capture and transformation—and because of this fact—it is currently the target by incumbents as a means of further limiting even this self-limited definition in practice. It is ironic that a technological advance that supports data-driven in lieu of report-driven information integration is being influenced to support the old paradigm.

The motivations are varied: industry suppliers who aim to restrict access to performance data under project and program management, incumbent technology providers who wish to keep the changes in data capture and transformation restricted to their limited capabilities, consulting companies aligned with technology incumbents, and staff augmentation firms dependent on keeping their customers dependent on custom application development and Excel workbooks. All of these forces work through the various professional organizations which work to influence government policy, hoping to establish themselves as the arbiters of the possible and the acceptable.

Note that oftentimes the requirements under project management are often critiqued under the rubric of government regulation. But that is a misnomer: it is an extension of government contract management. Another critique is made from the perspective of overhead costs. But management costs money, and one would not (or at least should not) drive a car or own a house without insurance and a budget for maintenance, much less a multi-year high-cost project involving the public’s money. In addition, as I have written previously which is supported by the literature, data-driven systems actually reduce costs and overhead.

All of these factors contribute to ossification, and impose artificial blinders that, absent reform, will undermine meeting the new paradigms of 21^st Century project management, given that the limited concept of IPM was obviously insufficient to address the challenges of the transitional decade that broached the last century.

Embracing the Future in Aerospace, Space, and Defense

As indicated, the aerospace and space science and technology verticals are entering a new and exciting phase of technological innovation resulting from investments in start-ups and R&D, including public-private cost-sharing arrangements.

IPM to Project Life-Cycle Management. Given the baggage that attends the acronym IPM, and the worldwide trend to data-driven decision-making, it is time to adjust the language of project and program management to align to it. In lieu of IPM, I suggest Project Life-Cycle Management to define the approach to project and program data and information management.
Functionality-Driven to Data-Driven Applications. Our software, systems and procedures must be able to support that infrastructure and be similarly in alignment with that manner of thinking. This evolution includes the following attributes:
- Data Agnosticism. As our decision-making methods expand to include a wider, deeper, and more comprehensive interdisciplinary approach, our underlying systems must be able to access data in this same manner. As such, these systems must be data agnostic.
- Data neutrality. In order to optimize access to data, the overhead and effort needed to access data must be greatly reduced. Using data science and analysis to restructure pre-conditioned data in order to overcome proprietary lexicons—an approach used for business intelligence systems since the 1980s—provides no added value to either the data or the organization. If data access is ad hoc and customized in every implementation, the value of the effort cannot either persist, nor is the return on investment fully realized. It backs the customer into a corner in terms of flexibility and innovation. Thus, pre-configured data capture, extract, transformation, and load (ETL) into a non-proprietary and objective format, which applies to all data types used in project and program management systems, is essential to providing the basis for a knowledge-based environment that encourages discovery from data. This approach in ETL is enhanced by the utilization of neutral data schemas.
- Data in Lieu of Reporting and Visualization. No doubt that data must be visualized at some point—preferably after its transformation and load into the database with other, interrelated data elements that illuminate information to enhance the knowledge of the decisionmaker. This implies that systems that rely on physical report formats, charts, and graphs as the goal are not in alignment with the new paradigm. Where Excel spreadsheets and PowerPoint are used as a management system, it is the preparer is providing the interpretation, in a manner that predisposes the possible alternatives of interpretation. The goal, instead, is to have data speak for itself. It is the data, transformed into information, interrelated and contextualized to create intelligence that is the goal.
- All of the Data, All of the Time. The cost of 1TB of data compared to 1MB of data is the marginal cost of the additional electrons to produce it. Our systems must be able to capture all of the data essential to effective decision-making in the periodicity determined by the nature of the data. Thus, our software systems must be able to relate data at all levels and to scale from simplistic datasets to extremely large ones. It should do so in such a way that the option for determining what, among the full menu of data options available, is relevant rests in the consumer of that data.
- Open Systems. Software solution providers beginning with the introduction of widespread CPU capability have manufactured software to perform particular functions based on particular disciplines and very specific capabilities. As noted earlier, these software applications are functionality-focused and proprietary in structure, method, and data. For data-driven project and program requirements, software systems must be flexible enough to accommodate a wide range of analytical and visualization demands in allowing the data to determine the rules of engagement. This implies systems that are open in two ways: data agnosticism, as already noted, but also open in terms of the user environment.
- Flexible Application Configuration. Our systems must be able to address the needs of the various disciplines in their details, while also allowing for integration and contextualization of interrelated data across domains. As with Open Systems to data and the user environment, openness through the ability to roll out multiple specialized applications from a common platform places the subject matter expert and program manager in the driver’s seat in terms of data analysis and visualization. An effective open platform also reduces the overhead associated with limited purpose-driven, disconnected and proprietary niche applications.
- No-Code/Low-Code. Given that data and the consumer will determine both the source and method of delivery, our open systems should provide an environment that supports Agile development and deployment of customization and new requirements.
- Knowledge-Based Content. Given the extensive amount of experience and education recorded and documented in the literature, our systems must, at the very least, provide a baseline of predictive analytics and visualization methods usually found in the more limited, purpose-built hardcoded applications, if not more expansive. This knowledge-based content, however, must be easily expandable and refinable, given the other attributes of openness, flexibility, and application configuration. In this manner, our 21^st century project and program management systems must possess the attributes of a hybrid system: providing the functionality of the traditional niche systems with the flexibility and power of a business intelligence system enhanced by COTS data capture and transformation.
- Ease of Use. The flexibility and power of these systems must be such that implementation and deployment are rapid, and that new user environment applications can be quickly deployed. Furthermore, the end user should be able to determine the level of complexity or simplicity of the environment to support ease of use.

Focus on the Earliest Indicator. A good deal of effort since the late 1990s has been expended on defining the highest level of summary data that is sufficient to inform earned value, with schedule integration derived from the WBS, oftentimes summarized on a one-to-many basis as well. This perspective is biased toward believing that cost performance is the basis for determining project control and performance. But even when related to cost, the focus is backwards. The project lifecycle in its optimized form exists of the following progression:

Project Goals and Contract (framing assumptions) –> Systems Engineering, CDRLs, KPPs, MoEs, MoPs, TPMs –> Project Estimate –> Project Plan –> IMS –> Risk and Uncertainty Analysis –> Financial Planning and Execution –> PMB –> EVM

As I’ve documented in this blog over the years, DoD studies have shown that, while greater detail within the EVM data may not garner greater early warning, proper integration with the schedule at the work package level does. Program variances first appear in the IMS. A good IMS, thus, is key to collecting and acting as the main execution document. This is why many program managers who are largely absent in the last decade or so from the professional organizations listed, tend to assert that EVM is like “looking in the rearview mirror.” It isn’t that it is not essential, but it is true that it is not the earliest indicator of variances from expected baseline project performance.

Thus, the emphasis going forward under this new paradigm is not to continue the emphasis and a central role for EVM, but a shift to the earliest indicator for each aspect of the program that defines its framing assumptions.

Systems Engineering: It’s not Space Science, it’s Space Engineering, which is harder.
The focus on start-up financing and developmental cost-sharing shifts the focus to systems engineering configuration control and technical performance indicators. The emphasis on meeting expectations, program goals, and achieving milestones within the cost share make it essential to be able to identify fatal variances, long before conventional cost performance indicators show variances. The concern of the program manager in these cases isn’t so much on the estimate at complete, but whether the industry partner will be able to deploy the technology within the acceptable range of the MoEs, MoPs, TPPs, and KPPs, and not exceed the government’s portion of the cost share. Thus, the incentive is to not only identify variances and unacceptable risk at the earliest indicator, but to do so in terms of whether the end-item technology will be successfully deployed, or whether the government should cut its losses.

Risk and Uncertainty is more than SRA. The late 20^th century approach to risk management is to run a simulated Monte Carlo analysis against the schedule, and to identify alternative critical paths and any unacceptable risks within the critical path. This is known as the schedule risk analysis, or SRA. While valuable, the ratio of personnel engaged in risk management is much smaller than the staffs devoted to schedule and cost analysis.

This is no doubt due to the specialized language and techniques devoted to risk and uncertainty. This segregation of risk from mainstream project and program analysis has severely restricted both the utility and the real-world impact of risk analysis on program management decision-making.

But risk and uncertainty extend beyond the schedule risk analysis, and their utility in an environment of aggressive investment in new technology, innovation, and new entries to the market will place these assessments at center stage. In reality, our ability to apply risk analysis techniques extends to the project plan, to technical performance indicators, to estimating, to the integrated master schedule (IMS), and to cost, both financial and from an earned value perspective. Combined with the need to identify risk and major variances using the earliest indicator, risk analysis becomes pivotal to mainstream program analysis and decision-making.

Conclusions from Part Two

The ASD industry is most closely aligned with PPM in the public interest. Two overarching trends that are transforming this market that are overcoming the inertia and ossification of PPM thought are the communications and information systems employed in response to the coronavirus pandemic, which opened pathways to new ways of thinking about the status quo, and the start-ups and new entries into the ASD market, borne from the investments in new technologies arising from external market, geo-political, space science, global warming, and propulsion trends, as well as new technologies and methods being employed in data and information technology that drive greater efficiency and productivity. These changes have forced a new language and new expectations as to the art of the necessary, as well as the art of the possible, for PPM. This new language includes a transition to the concept of the optimal capture and use of all data across the program management life cycle with greater emphasis on systems engineering, technical performance, and risk.

Having summarized the new program paradigm in Aerospace, Space, and Defense, my next post will assess the characteristics of program management in various commercial industries, the rising trends in these verticals, and what that means for the project and program management discipline.

Over at AITS.org — Failure is not Optional

My latest is at this link at AITS.org with the provocative title: “Failure is not Optional: Why Project Failure is OK.” The theme and specifics of the post, however, are not that simple and I continue with a sidebar on Grant’s conduct of the Overland Campaign entitled “How Grant Leveraged Failure in the Civil War.” A little elaboration is in place once you read the entire post.

I think what we deal with in project management are shades of failure. It is important to understand this because we rely too often on projections of performance that oftentimes turn out to be unrealistic within the framing assumptions of project management. In this context our definition of what defines success turns out to be fluid.

To provide a simplistic example of other games of failure, let’s take the game of American baseball. A batter who hits safely more than 30% of the time is deemed to be skilled in the art of hitting a baseball. A success. Yet, when looked at it from a total perspective what this says is that 70% failure is acceptable. A pitcher who gives up between 2 and 4 earned runs a game is considered to be skilled in the art of pitching. Yet, this provides a range of acceptable failure under the goal of giving up zero runs. Furthermore, if your team wins 9-4 you’re considered to be a winning pitcher. If you lose 1-0 you are a losing pitcher, and there are numerous examples of talented pitchers who were considered skilled in their craft who had losing records because of lack of run production by his team. Should the perception of success and failure be adjusted based on whether one pitched for the 1927 or 1936 or 1998 Yankees, or the 1963 Dodgers, or 1969 Mets? The latter two examples were teams built on just enough offense to provide the winning advantage, with the majority of pressure placed on the pitching staff. Would Tom Seaver be classified as less extraordinary in his skill if he averaged giving up half a run more? Probably.

Thus, when we look at the universe of project management and see that the overwhelming majority of IT projects fail, or that the average R&D contract realizes a 20% overrun in cost and a significant slip in schedule, what are we measuring? We are measuring risk in the context of games of failure. We handle risk to absorb just enough failure and noise in our systems to both push the envelope on development without sacrificing the entire project effort. To know the difference between transient and existential failure, between learning and wasted effort, and between intermediate progress and strategic position requires a skillset that is essential to ultimate achievement of the goal, whether it be deployment of a new state-of-the-art aircraft, or a game-changing software platform. The noise must pass what I have called the “so-what?” test.

I have listed a set of skills necessary to the understanding these differences in the article that you may find useful. I have also provided some ammunition for puncturing the cult of “being green.”

River Deep, Mountain High — A Matrix of Project Data

Been attending conferences and meetings of late and came upon a discussion of the means of reducing data streams while leveraging Moore’s Law to provide more, better data. During a discussion with colleagues over lunch they asked if asking for more detailed data would provide greater insight. This led to a discussion of the qualitative differences in data depending on what information is being sought. My response to more detailed data was to respond: “well there has to be a pony in there somewhere.” This was greeted by laughter, but then I finished the point: more detailed data doesn’t necessarily yield greater insight (though it could and only actually looking at it will tell you that, particularly in applying the principle of KDD). But more detailed data that is based on a hierarchical structure will, at the least, provide greater reliability and pinpoint areas of intersection to detect areas of risk manifestation that is otherwise averaged out–and therefore hidden–at the summary levels.

Not to steal the thunder of new studies that are due out in the area of data later this spring but, for example, I am aware after having actually achieved lowest level integration for extremely complex projects through my day job, that there is little (though not zero) insight gained in predictive power between say, the control account level of a WBS and the work package level. Going further down to element of cost may, in the words of the character in the movie Still Alice, where “You may say that this falls into the great academic tradition of knowing more and more about less and less until we know everything about nothing.” But while that may be true for project management, that isn’t necessarily so when collecting parametrics and auditing the validity of financial information.

Rolling up data from individually detailed elements of a hierarchy is the proper way to ensure credibility. Since we are at the point where a TB of data has virtually the same marginal cost of a GB of data (which is vanishingly small to begin with), then the more the merrier in eliminating the abuse associated with human-readable summary reporting. Furthermore, I have long proposed through this blog and elsewhere, that the emphasis should be away from people, process, and tools, to people, process, and data. This rightly establishes the feedback loop necessary for proper development and project management. More importantly, the same data available through project management processes satisfy the different purposes of domains both within the organization, and of multiple external stakeholders.

This then leads us to the concept of integrated project management (IPM), which has become little more than a buzz-phrase, and receives a lot of hand waves, mostly by technology companies that want to push their tools–which are quickly becoming obsolete–while appearing forward leaning. This tool-centric approach is nothing more than marketing–focusing on what the software manufacturer would have us believe is important based on the functionality baked into their applications. One can see where this could be a successful approach, given the emphasis on tools in the PM triad. But, of course, it is self-limiting in a self-interested sort of way. The emphasis needs to be on the qualitative and informative attributes of available data–not of tool functionality–that meet the requirements of different data consumers while minimizing, to the extent possible, the number of data streams.

Thus, there are at least two main aspects of data that are important in understanding the utility of project management: early warning/predictiveness and credibility/traceability/fidelity. The chart attached below gives a rough back-of-the-envelope outline of this point, with some proposed elements, though this list is not intended to be exhaustive.

PM Data Matrix

In order to capture data across the essential elements of project management, our data must demonstrate both a breadth and depth that allows for the discovery of intersections of the different elements. The weakness in the two-dimensional model above is that it treats each indicator by itself. But, when we combine, for example, IMS consecutive slips with other elements listed, the informational power of the data becomes many times greater. This tells us that the weakness in our present systems is that we treat the data as a continuity between autonomous elements. But we know that the project consists of discontinuities where the next level of achievement/progress is a function of risk. Thus, when we talk about IPM, the secret is in focusing on data that informs us what our systems are doing. This will require more sophisticated types of modeling.

Repeat after me — Excel is not a project management solution

Aside from dealing with organizations that oftentimes must use Excel as workarounds due to limitations of legacy software systems, I was reminded of the ubiquity of Excel in a recent article by my colleague Dave Gordon at AITS on the use and misuse of RAID (Risk Assumptions, Issues, and Decisions). His overall assessment of the weakness of how RAID can be applied is quite valid. But the literature on risk is quite extensive. The article “Risk Management Is How Adults Manage Projects” at Glen Alleman’s Herding Cats blog is just one quick overview of a very mature process that has a large amount of academic, statistical, mathematical, and methodological grounding.

The dangers that Dave is identifying are not implicit in RAID so much as they are implicit in anyone who uses Excel. It is not that Excel is a bad tool. It is very useful for one-off spreadsheet problems. It is not a software solution and is not meant to be one. Going to Microsoft’s site on the progression of Excel to Access to SQL Server clarifies the differences.

Note that in my title I didn’t use the word tool. This word has been the source of great confusion. To the layman it seems to be a common-sense descriptive. When you physically work on something–a home repair, an automobile, a small bit of machinery–you have a toolbox and you have a set of tools to address the problem. So the analogy is that projects (and other processes) should be approached the same way. Oftentimes this is done in the shorthand of marketing. It’s easier to explain something complex through a simple analogy.

But no descriptive has been more harmful or destructive in preventing people from fully conceiving of an understanding of the needed solutions in the project management community than the word tool when applied to software applications. A project is a complex system–a complex adaptive system. What is called for are software applications that fit into the manner that project systems operate.

Back in the 1980s and 1990s there was a great deal of justification to take the spreadsheet software that came with the operating environment or the Office Suite and adapt it administrative needs. There were a great number of individual tasks and processes that needed to be automated, and the market had not yet responded to that demand. In some cases the demand was only vaguely understood, and the solutions were not fully conceptualized.

Then, as software applications became more sophisticated to begin to replace manual processes, they oftentimes could not address all of the corollary operations that need to be performed in those systems. Oftentimes, the solutions addressed the 80% need of these requirements, and so one-off workarounds until the software was developed to be more comprehensive, were applied. The very process of automating previously manual tasks had an effect on organizational processes, driving them toward greater sophistication. The phenomena of Knowledge Discovery in Databases (KDD) is but one of these effects.

But note that when relying on Excel for important processes, such as risk handling, that KDD is impossible. An accountant or head of finance in a corporation would not use Excel to keep the company’s books, even though that was the original target audience back in the 1980s for spreadsheet software. No doubt that financial personnel use Excel to supplement their work, but using it as the primary solution to constitute the system of record would be foolhardy. Even very small businesses today use more sophisticated financial management solutions.

The reason why one must be extra careful with any process when using Excel as the solution is that the person is still to a large extent, a part of the computer. The person must perform operations that a spreadsheet application cannot perform. This is why in risk management that assumptions, issues, decisions, and handling are tied to the work. The origin of all work decomposed from the requirements are the plan and then the detailed schedule. The detailed schedule, consisting of activities, is then further decomposed into the work organization: tasks, resources, etc. There may or may not be a WBS or OBS that ties performance reflected in terms of earned value.

Using Excel as an external tool in addressing this important and essential process separates it from the other elements of project management systems. It thus creates a single point of failure in the process–the mind of the individual keeping the Excel spreadsheet. It also containerizes the information, preventing it from being mainstreamed into the larger organization, and thus being a source of KDD.

Over at AITS.org — Black Swans: Conquering IT Project Failure & Acquisition Management

It’s been out for a few days but I failed to mention the latest article at AITS.org.

In my last post on the Blogging Alliance I discussed information theory, the physics behind software development, the economics of new technology, and the intrinsic obsolescence that exists as a result. Dave Gordon in his regular blog described this work as laying “the groundwork for a generalized theory of managing software development and acquisition.” Dave has a habit of inspiring further thought, and his observation has helped me focus on where my inquiries are headed…

To read more please click here.

Margin Call — Schedule Margin and Schedule Risk

A discussion at the LinkedIn site for the NDIA IPMD regarding schedule margin has raised some good insight and recommendations for this aspect of project planning and execution. Current guidance from the U.S. Department of Defense for those engaged in the level of intense project management that characterizes the industry has been somewhat vague and open to interpretation. Some of this, I think, is due to the competing proprietary lexicon from software manufacturers that have been dominant in the industry.

But mostly the change in defining this term is due to positive developments. That is, the change is due to the convergence garnered from long experience among the various PM disciplines that allow us to more clearly define and distinguish between schedule margin, schedule buffer, schedule contingency, and schedule reserve. It is also due to the ability of more powerful software generations to actually apply the concept in real planning without it being a thumb in the air-type exercise.

Concerning this topic, Yancy Qualls of Bell Helicopter gave an excellent presentation at the January NDIA IPMD meeting in Tucson. His proposal makes a great deal of sense and, I think, is a good first step toward true integration and a more elegant conceptual solution. In his proposal, Mr. Qualls clearly defines the scheduling terminology by drawing analogies to similar concepts on the cost side. This construction certainly overcomes a lot of misconceptions about the purpose and meaning of these terms. But, I think, his analogies also imply something more significant and it is this: that there is a common linkage between establishing management reserve and schedule reserve, and there are cost/schedule equivalencies that also apply to margin, buffer, and contingency.

After all, resources must be time-phased and these are dollarized. But usually the relationship stops there and is distinguished by that characteristic being measured: measures of value or measures of timing; that is, the value of the work accomplished against the Performance Management Baseline (PMB) is different from the various measures of progress recorded against the Integrated Master Schedule (IMS). This is why we look at both cost and schedule variances on the value of work performed from a cost perspective, and physical accomplishment against time. These are fundamental concepts.

To date, the most significant proposal advanced to reconcile the two different measures was put forth by Walt Lipke of Oklahoma City Air Logistics Center in the method known as earned schedule. But the method hasn’t been entirely embraced. Studies have shown that it has its own limitations, but that it is a good supplement those measures currently in use, not a substitute for them.

Thus, we are still left with the need of making a strong, logical, and cohesive connection between cost and schedule in our planning. The baseline plans constructed for both the IMS and PMB do not stand apart or, at least, should not. They are instead the end result of a continuum in the construction of our project systems. As such, there should be a tie between cost and schedule that allows us to determine the proper amount of margin, buffer, and contingency in a manner that is consistent across both sub-system artifacts.

This is where risk comes in and the correct assessment of risk at the appropriate level of measurement, given that our measures of performance are being measured against different denominators. For schedule margin, in Mr. Qualls’ presentation, it is the Schedule Risk Analysis (SRA). But this then leads us to look at how that would be done.

Fortuitously, during this same meeting, Andrew Uhlig of Raytheon Missile Systems gave an interesting presentation on historical SRA results, building models from such results, and using them to inform current projects. What I was most impressed with in this presentation was that his research finds that the actual results from schedule performance do not conform to any of the usual distribution curves found in the standard models. Instead of normal, triangle, or pert distributions, what he found is a spike, in which a large percentage of the completions fell exactly on the planned duration. Thus, distribution was skewed around the spike, with the late durations–the right tail–much longer than the left.

What is essential about the work of Mr. Uhlig is that, rather than using small samples with their biases, he using empirical data to inform his analysis. This is a pervasive problem in project management. Mr. Qualls makes this same point in his own presentation, using the example of the Jordan-era Chicago Bulls as an example, where each subsequent win–combined with probabilities that show that the team could win all 82 games–does not mean that they will actually perform the feat. In actuality (and in reality) the probability of this occurring is quite small. Glen Alleman at his Herding Cats blog covers this same issue, emphasizing the need for empirical data.

The results of the Uhlig presentation are interesting, not only because they throw into question the results using the three common distributions used in schedule risk analysis under simulated Monte Carlo, but also because they may suggest, in my opinion, an observation or reporting bias. Discrete distribution methods, as Mr. Uhlig proposes, will properly model the distribution for such cases using our parametric analysis. But they will not reflect the quality of the data collected.

Short duration activities are designed to overcome subjectivity through their structure. The shorter the duration, the more discrete the work being measured, the less likely occurrence of “gaming” the system. But if we find, as Mr. Uhlig does, that 29% of 20 day activities report exactly 20 days, then there is a need to test the validity of the spike itself. It is not that it is necessarily wrong. Perhaps the structure of the short duration combined with the discrete nature of the linkage to work has done its job. One would expect a short tail to the left and a long tail to the right of the spike. But there is also a possibility that variation around the target duration is being judged as “close enough” to warrant a report of completion at day 20.

So does this pass the “So What?” test? Yes, if only because we know that the combined inertia of all of the work performed at any one time on the project will eventually be realized in the form of a larger amount of risk in proportion to the remaining work. If the reported results are pushing risk to the right because the reported performance is optimistic against the actual performance, then we will get false positives. If the actual performance is pessimistic against actual performance–a less likely scenario in my opinion–then we will get false negatives.

But regardless of these further inquiries that I think need to be made regarding the linkage between cost and schedule, and the validity of results from SRAs, we now have two positive steps in the right direction in clarifying areas that in the past have perplexed project managers. Properly identifying schedule reserve, margin, buffer, and contingency, combined with properly conducting SRAs using discrete distributions based on actual historical results will go quite far in allowing us to introduce better predictive measures in project management.

Out of Winter Woodshedding — Thinking about Project Risk and passing the “So What?” test

“Woodshedding” is a slang term in music, particularly in relation to jazz, in which the musician practices on an instrument usually outside of public performance, the purpose of which is to explore new musical insights without critical judgment. This can be done with or without the participation of other musicians. For example, much attention recently has been given to Bob Dylan’s Basement Tapes release. Usually it is unusual to bother recording such music, given the purpose of improvisation and exploration, and so few additional examples of “basement tapes” exist from other notable artists.

So for me the holiday is a sort of opportunity to do some woodshedding. The next step is to vet such thoughts on informal media, such as this blog, where the high standards involved in white and professional papers do not allow for informal dialogue and exchange of information, and thoughts are not yet fully formed and defensible. My latest mental romps have been inspired by the movie about Alan Turing–The Imitation Game–and the British series The Bletchley Circle. Thinking about one of the fathers of modern computing reminded me that the first use of the term “computer” referred to people.

As a matter of fact, though the terminology now refers to the digital devices that have insinuated themselves into every part of our lives, people continue to act as computers. Despite fantastical fears surrounding AI taking our jobs and taking over the world, we are far from the singularity. Our digital devices can only be programmed to go so far. The so-called heuristics in computing today are still hard-wired functions, similar to replicating the methods used by a good con artist in “reading” the audience or the mark. With the new technology in dealing with big data we have the ability to many of the methods originated by the people in the real life Bletchley Park of the Second World War. Still, even with refinements and advances in the math, they provide great external information regarding the patterns and probable actions of the objects of the data, but very little insight into the internal cause-and-effect that creates the data, which still requires human intervention, computation, empathy, and insight.

Thus, my latest woodshedding has involved thinking about project risk. The reason for this is the emphasis recently on the use of simulated Monte Carlo analysis in project management, usually focused on the time-phased schedule. Cost is also sometimes included in this discussion as a function of resources assigned to the time-phased plan, though the fatal error in this approach is to fail to understand that technical achievement and financial value analysis are separate functions that require a bit more computation.

It is useful to understand the original purpose of simulated Monte Carlo analysis. Nobel physicist Murray Gell-Mann, while working at RAND Corporation (Research and No Development) came up with the method with a team of other physicists (Jess Marcum and Keith Breuckner) to determine the probability of a number coming up from a set of seemingly random numbers. For a full rendering of the theory and its proof Gell-Mann provides a good overview in his book The Quark and the Jaguar. The insight derived from the insight of Monte Carlo computation has been to show that systems in the universe often organize themselves into patterns. Instead of some event being probable by chance, we find that, given all of the events that have occurred to date, that there is some determinism which will yield regularities that can be tracked and predicted. Thus, the use of simulated Monte Carlo analysis in our nether world of project management, which inhabits that void between microeconomics and business economics, provides us with some transient predictive probabilities given the information stream at that particular time, of the risks that have manifested and are influencing the project.

What the use of Monte Carlo and other such methods in identifying regularities do not do is to determine cause-and-effect. We attempt to bridge this deficiency with qualitative risk in which we articulate risk factors to handle that are then tied to cost and schedule artifacts. This is good as far as it goes. But it seems that we have some of this backward. Oftentimes, despite the application of these systems to project management, we still fail to overcome the risks inherent in the project, which then require a redefinition of project goals. We often attribute these failures to personnel systems and there are no amount of consultants all too willing to sell the latest secret answer to project success. Yet, despite years of such consulting methods applied to many of the same organizations, there is still a fairly consistent rate of failure in properly identifying cause-and-effect.

Cause-and-effect is the purpose of all of our metrics. Only by properly “computing” cause-and-effect will we pass the “So What?” test. Our first forays into this area involve modeling. Given enough data we can model our systems and, when the real-time results of our in-time experiments play out to approximate what actually happens then we know that our models are true. Both economists and physicists (well, the best ones) use the modeling method. This allows us to get the answer even if not entirely understanding the question of the internal workings that lead to the final result. As in Douglas Adams’ answer to the secret of life, the universe, and everything where the answer is “42,” we can at least work backwards. And oftentimes this is what we are left, which explains the high rate of failure in time.

While I was pondering this reality I came across this article in Quanta magazine outlining the new important work of the MIT physicist Jeremy England entitled “A New Physics Theory of Life.” From the perspective of evolutionary biology, this pretty much shows that not only does the Second Law of Thermodynamics support the existence and evolution of life (which we’ve known as far back as Schrodinger), but probably makes life inevitable under a host of conditions. In relation to project management and risk, it was this passage that struck me most forcefully:

“Chris Jarzynski, now at the University of Maryland, and Gavin Crooks, now at Lawrence Berkeley National Laboratory. Jarzynski and Crooks showed that the entropy produced by a thermodynamic process, such as the cooling of a cup of coffee, corresponds to a simple ratio: the probability that the atoms will undergo that process divided by their probability of undergoing the reverse process (that is, spontaneously interacting in such a way that the coffee warms up). As entropy production increases, so does this ratio: A system’s behavior becomes more and more “irreversible.” The simple yet rigorous formula could in principle be applied to any thermodynamic process, no matter how fast or far from equilibrium. “Our understanding of far-from-equilibrium statistical mechanics greatly improved,” Grosberg said. England, who is trained in both biochemistry and physics, started his own lab at MIT two years ago and decided to apply the new knowledge of statistical physics to biology.”

No project is a closed system (just as the earth is not on a larger level). The level of entropy in the system will vary by the external inputs that will change it: effort, resources, and technical expertise. As I have written previously (and somewhat controversially), there is both chaos and determinism in our systems. An individual or a system of individuals can adapt to the conditions in which they are placed but only to a certain level. It is non-zero that an individual or system of individuals can largely overcome the risks realized to date, but the probability of that occurring is vanishingly small. The chance that a peasant will be a president is the same. The idea that it is possible, even if vanishingly so, keeps the class of peasants in line so that those born with privilege can continue to reassuringly pretend that their success is more than mathematics.

When we measure risk what we are measuring is the amount of entropy in the system that we need to handle, or overcome. We do this by borrowing energy in the form of resources of some kind from other, external systems. The conditions in which we operate may be ideal or less than ideal.

What England’s work combined with his predecessors’ seem to suggest is that the Second Law almost makes life inevitable except where it is impossible. For astrophysics this makes the entire Rare Earth hypothesis a non sequitur. That is, wherever life can develop it will develop. The life that does develop is fit for its environment and continues to evolve as changes to the environment occur. Thus, new forms of organization and structure are found in otherwise chaotic systems as a natural outgrowth of entropy.

Similarly, when we look at more cohesive and less complex systems, such as projects, what we find are systems that adapt and are fit for the environments in which they are conceived. This insight is not new and has been observed for organizations using more mundane tools, such as Deming’s red bead experiment. Scientifically, however, we now have insight into the means of determining what the limitations of success are given the risk and entropy that has already been realized, against the needed resources that are needed to bring the project within acceptable ranges of success. This information goes beyond simply stating the problem, leaving the computing to the person and thus passes the “So What?” test.

More on Excel…the contributing factor of poor Project Management apps

Some early comments via e-mails on my post on why Excel is not a PM tool raised the issue that I was being way too hard on IT shops and letting application providers off the hook. The asymmetry was certainly not the intention (at least not consciously).

When approaching an organization seeking process and technology improvement, oftentimes the condition of using Excel is what we in the technology/PM industry conveniently call “workarounds.” Ostensibly these workarounds are temporary measures to address a strategic or intrinsic organizational need that will eventually be addressed by a more cohesive software solution. In all too many cases, however, the workaround turns out to be semi-permanent.

A case in point in basic project management concerns Work Authorizations Documents (WADs) and Baseline Change Requests (BCRs). Throughout entire industries who use the most advanced scheduling applications, resource management applications, and–where necessary–earned value “engines,” the modus operandi to address WADs and BCRs is to either use Excel or to write a custom app in FoxPro or using Access. This is fine as a “workaround” as long as you remember to set up the systems and procedures necessary to keep the logs updated, and then have in place a procedure to update the systems of record appropriately. Needless to say, errors do creep in and in very dynamic environments it is difficult to ensure that these systems are in alignment, and so a labor-intensive feedback system must also be introduced.

This is the type of issue that software technology was designed to solve. Instead, software has fenced off the “hard’ operations so that digitized manual solutions, oftentimes hidden from plain view from the team by the physical technological constraint of the computer (PC, laptop, etc.), are used. This is barely a step above what we did before the introduction of digitization: post the project plan, milestone achievements, and performance on a VIDS/MAF board that surrounded the PM control office, which ensured that every member of the team could see the role and progress of the project. Under that system no one hoarded information, it militated against single points of failure, and ensured that disconnects were immediately addressed since visibility ensured accountability.

In many ways we have lost the ability to recreate the PM control office in digitized form. Part of the reason resides in the 20th century organization of development and production into divisions of labor. In project management, the specialization of disciplines organized themselves around particular functions: estimating and planning, schedule management, cost management, risk management, resource management, logistics, systems engineering, operational requirements, and financial management, among others. Software was developed to address each of these areas with clear lines of demarcation drawn that approximated the points of separation among the disciplines. What the software manufacturers forgot (or never knew) was that the PMO is the organizing entity and it is an interdisciplinary team.

To return to our example: WADs and BCRs; a survey of the leading planning and scheduling applications shows that while their marketing literature addresses baselines and baseline changes (and not all of them address even this basic function), they still do not understand complex project management. There is a difference between resources assigned to a time-phased network schedule and the resources planned against technical achievement related to the work breakdown structure (WBS). Given proper integration they should align. In most cases they do not. This is why most scheduling application manufacturers who claim to measure earned value, do not. Their models assume that the expended resources align with the plan to date, in lieu of volume-based measurement. Further, eventually understanding this concept does not produce a digitized solution, since an understanding of the other specific elements of program control is necessary.

For example, projects are initiated either through internal work authorizations in response to a market need, or based on the requirements of a contract. Depending on the mix of competencies required to perform the work financial elements such as labor rates, overhead, G&A, allowable margin (depending on contract type), etc. will apply–what is euphemistically called “complex rates.” An organization may need to manage multiple rate sets based on the types of efforts undertaken, with a many-to-many relationship between rate sets and projects/subprojects.

Once again, the task of establishing the proper relationships at the appropriate level is necessary. This will then affect the timing of WAD initiation, and will have a direct bearing on the BCR approval process, given that it is heavily influenced by “what-if?” analysis against resource, labor, and financial availability and accountability (a complicated process in itself). Thus the schedule network is not the only element affected, nor the overarching one, given the assessed impact on cost, technical achievement, and qualitative external risk.

These are but two examples of sub-optimization due to deficiencies in project management applications. The response–and in my opinion a lazy one (or one based on the fact that oftentimes software companies know nothing of their customers’ operations)–has been to develop the alternative euphemism for “workaround”: best of breed. Oftentimes this is simply a means of collecting revenue for a function that is missing from the core application. It is the software equivalent of division of labor: each piece of software performs functions relating to specific disciplines and where there are gaps these are filled by niche solutions or Excel. What this approach does not do is meet the requirements of the PMO control office, since it perpetuates application “swim lanes,” with the multidisciplinary requirements of project management relegated to manual interfaces and application data reconciliation. It also pushes–and therefore magnifies–risk at the senior level of the project management team, effectively defeating organizational fail safes designed to reduce risk through, among other methods, delegation of responsibility to technical teams, and project planning and execution constructed around short duration/work-focused activities. It also reduces productivity, information credibility, and unnecessarily increases cost–the exact opposite of the rationale used for investing in software technology.

It is time for this practice to end. Technologies exist today to remove application “swim lanes” and address the multidisciplinary needs of successful project management. Excel isn’t the answer; cross-application data access, proper data integration, and data processing into user-directed intelligence, properly aggregated and distributed based on role and optimum need to know, is.

Frame by Frame: Framing Assumptions and Project Success or Failure

When we wake up in the morning we enter the day with a set of assumptions about ourselves, our environment, and the world around us. So too when we undertake projects. I’ve just returned from the latest NDIA IPMD meeting in Washington, D.C. and the most intriguing presentation at the meeting was given by Irv Blickstein regarding a RAND root cause analysis of major program breaches. In short, a major breach in the cost of a program is defined by the Nunn-McCurdy amendment that was first passed in 1982, in which a major defense program breaches its projected baseline cost by more than 15%.

The issue of what constitutes programmatic success and failure has generated a fair amount of discussion among the readers of this blog. The report, which is linked above, is full of useful information regarding Major Defense Acquisition Program (also known as MDAP) breaches under Nunn-McCurdy, but for purposes of this post readers should turn to page 83. In setting up a project (or program), project/program managers must make a set of assumptions regarding the “uncertain elements of program execution” centered around cost, technical performance, and schedule. These assumptions are what are referred to as “framing assumptions.”

A framing assumption is one in which there are signposts along the way to determine if an assumption regarding the project/program has changed over time. Thus, according to the authors, the precise definition of a framing assumption is “any explicit or implicit assumption that is central in shaping cost, schedule, or performance expectations.” An interesting aspect of their perspective and study is that the three-legged stool of program performance relegates risk to serving as a method that informs the three key elements of program execution, not as one of the three elements. I have engaged in several conversations over the last two weeks regarding this issue. Oftentimes the question goes: can’t we incorporate technical performance as an element of risk? Short answer: No, you can’t (or shouldn’t). Long answer: risk is a set of methods for overcoming the implicit invalidity of single point estimates found in too many systems being used (like estimates-at-complete, estimates-to-complete, and the various indices found in earned value management, as well as a means of incorporating qualitative environmental factors not otherwise categorizable), not an element essential to defining the end item application being developed and produced. Looked at another way, if you are writing a performance specification, then performance is a key determinate of program success.

Additional criteria for a framing assumption are also provided in the RAND study. These are that the assumptions must be determinative, that is, the consequences of the assumption being wrong significantly affects the program in an essential way. They must also be unmitigable, that is, the consequences of the assumption being wrong are unavoidable. They must be uncertain, that is, the outcome or certainty of it being right or wrong cannot be determined in advance. They must be independent and not dependent on another event or series of events. Finally, they must be distinctive, in setting the program apart from other efforts.

RAND then applied the framing assumption methodology to a number of programs. The latest NDIA meeting was an opportunity to provide an update of conclusions based on the work first done in 2013. What the researchers found was that framing assumptions which are kept at a high level, be developed early in a program’s life cycle, and should be reviewed on a regular basis to determine validity. They also found that a program breached the threshold when a framing assumption became invalid. Project and program managers, and requirements personnel have at least intuitively known this for quite some time. Over the years, this is the reason given for requirements changes and contract modifications over the course of development that result in cost, performance, and schedule impacts.

What is different about the RAND study is that they have outlined a practical process for making these determinations early enough for a project/program to be adjusted with changing circumstances. For example, the numbers of framing assumptions of all MDAPs in the study could be boiled down to four or five, which are easily tested against reality during the milestone and other reviews held over the course of a program. This is particularly important given the lengthened time-frames of major acquisitions from development to production.

Looking at these results, my own observation is that this is a useful tool for identifying course corrections that are needed before they manifest into cost and schedule impacts, particularly given that leadership at PARCA has been stressing agile acquisition strategies. The goal here, it seems, is to allow for course corrections before the inertia of the effort leads to failure or–more likely–the development and deployment of an end item that does not entirely meet the needs of the Defense Department. (That such “disappointments” often far outstrip the capabilities of our adversaries is a topic for a different post).

I think the court is still out on whether course corrections, given the inertia of work and effort already expended at the point that a framing assumption would be tested as invalid, can ever truly be offsetting to the point of avoiding a breach, unless we then rebrand the existing effort as a new program once it has modified its structure to account for new framing assumptions. Study after study has shown that project performance is pretty well baked in at the 20% mark. For MDAPs, much of the front-loaded efforts in technology selection and application have been made. After all, systems require inputs and to change a system requires more inputs, not less, to overcome the inertia of all of the previous effort, not to mention work in progress. This is basic physics whether we are dealing with physical systems or complex adaptive (economic) systems.

Certainly, more efficient technology that affects the units of measurement within program performance can result in cost savings or avoidance, but that is usually not the case. There is a bit of magical thinking here: that commercial technologies will provide a breakthrough to allow for such a positive effect. This is an ideological idea not borne out by reality. The fact is that most of the significant technological breakthroughs we have seen over the last 70 years–from the microchip to the internet and now to drones–have resulted from public investments, sometimes in public-private ventures, sometimes in seeded technologies that are then released into the public domain. The purpose of most developmental programs is to invest in R&D to organically develop technologies (utilizing the talents of the quasi-private A&D industry) or provide economic incentives to incorporate technologies that do not currently exist.

Regardless, the RAND study has identified an important concept in determining the root causes of overruns. It seems to me that a formalized process of identifying framing assumptions should be applied and done at the inception of the program. The majority of the assessments to test the framing assumptions should then need to be made prior to the 20% mark as measured by program schedule and effort. It is easier and more realistic to overcome the bow-wave of effort at that point than further down the line.

Note: I have modified the post to clarify my analysis of the “three-legged stool” of program performance in regard to where risk resides.

Better Knock-Knock-Knock on Wood — The Essential Need for Better Schedule-Cost Integration

Back in early to mid-1990s, when NSFNET was making the transition to the modern internet, I was just finishing up my second assignment as an IT project manager and transitioning to a full-blown Program Executive Office (PEO) Business Manager and CIO at a major Naval Systems Command. The expanded potential of a more open internet was on everyone’s mind and, on the positive side, on how barriers to previously stove-piped data could be broken down in order to drive optimization of the use of that data (after processing it into useable intelligence). The next step was then to use that information, which was opened to a larger audience that previously was excluded from it, and to juxtapose and integrate it with other essential data (processed into intelligence) to provide insights not previously realized.

Here we are almost 20 years later and I am disappointed to see in practice that the old barriers to information optimization still exist in many places where technology should have long ago broken this mindset. Recently I have discussed cases at conferences and among PM professionals where the Performance Management Baseline (PMB), that is, the plan that is used to measure financial value of the work performed, is constructed separately from and without reference to the Integrated Master Schedule (IMS) until well after the fact. This is a challenge to common sense.

Project management is based on the translation of a contract specification into a plan to build something. The basic steps after many years of professional development are so tried and true that it should be rote at this point: Integrated Master Plan (IMP) –> Integrated Master Schedule (IMS) with Schedule Risk Assessment (SRA) –> Resource assignments with negotiated rates –> Develop work packages, link to financials, and roll-up of WBS –> Performance Management Baseline (PMB). The arrows represent the relationships between the elements. Feel free to adjust semantics and add additional items to the process such as a technical performance baseline, testing and evaluation plans, systems descriptions to ensure traceability, milestone tracking, etc. But the basic elements of project planning and execution pretty much remain the same–that’s all there is folks. The complexity and time spent to go through the steps varies based on the complexity of the scope being undertaken. For a long-term project involving billions or millions of dollars the interrelationships and supporting documentation is quite involved, for short-term efforts the process may be in mental process of the person doing the job. But in the end, regardless of terminology, these are the basic elements of PM.

When one breaks this cycle and decides to build each of the elements independently from the other it is akin to building a bridge in sections without using an overarching plan. Result: it’s not going to meet in the center. One can argue that it is perfectly fine to build the PMB concurrent with the IMS if the former is informed by the latter. But in practice I find that this is rarely the case. So what we have, then, is a case where a bridge is imperfectly matched when the two sections meet in the middle requiring constant readjustment and realignment. Furthermore, the manner in which the schedule activities are aligned with the budget vary from project to project, even within the same organization. So not only do we not use a common plan in building our notional bridge, we decide to avoid standardization of bolts and connectors too, just to make it that more interesting.

The last defense in this sub-optimized environment is: well, if we are adjusting it every month through the project team what difference does it make? Isn’t this integration nonetheless? Response #1: No. Response #2: THIS-IS-THE-CHALLENGE-THAT-DIGITAL-SYSTEMS-ARE-DESIGNED-TO-OVERCOME. The reason why this is not integration is because it simultaneously ignores the lessons learned in the SRA and prevents insights gained through optimization. If our planning documents are contingent on a month-to-month basis then the performance measured against them is of little value and always open to question, and not just on the margins. Furthermore, utilization of valuable project management personnel on performing what is essentially clerical work in today’s environment is indefensible. If there are economic incentives for doing this it is time for project stakeholders and policymakers to end them.

It is time to break down the artificial barriers that define cost and schedule analysts. Either you know project and program management or you don’t. There is no magic wall between the two disciplines, given that one cannot exist without the other. Furthermore, more standardization, not less, is called for. For anyone who has tried to decipher schedules where smiley-faces, and non-standard and multiple structures are in use in the same schedule, which defy reference to a cost control account, it is clear that both the consulting and project management communities are failing to instill professionalism.

Otherwise, as in my title, it’s like knocking on wood.