cost management

More on Excel…the contributing factor of poor Project Management apps

Some early comments via e-mails on my post on why Excel is not a PM tool raised the issue that I was being way too hard on IT shops and letting application providers off the hook.  The asymmetry was certainly not the intention (at least not consciously).

When approaching an organization seeking process and technology improvement, oftentimes the condition of using Excel is what we in the technology/PM industry conveniently call “workarounds.”  Ostensibly these workarounds are temporary measures to address a strategic or intrinsic organizational need that will eventually be addressed by a more cohesive software solution.  In all too many cases, however, the workaround turns out to be semi-permanent.

A case in point in basic project management concerns Work Authorizations Documents (WADs) and Baseline Change Requests (BCRs).  Throughout entire industries who use the most advanced scheduling applications, resource management applications, and–where necessary–earned value “engines,” the modus operandi to address WADs and BCRs is to either use Excel or to write a custom app in FoxPro or using Access.  This is fine as a “workaround” as long as you remember to set up the systems and procedures necessary to keep the logs updated, and then have in place a procedure to update the systems of record appropriately.  Needless to say, errors do creep in and in very dynamic environments it is difficult to ensure that these systems are in alignment, and so a labor-intensive feedback system must also be introduced.

This is the type of issue that software technology was designed to solve.  Instead, software has fenced off the “hard’ operations so that digitized manual solutions, oftentimes hidden from plain view from the team by the physical technological constraint of the computer (PC, laptop, etc.), are used.  This is barely a step above what we did before the introduction of digitization:  post the project plan, milestone achievements, and performance on a VIDS/MAF board that surrounded the PM control office, which ensured that every member of the team could see the role and progress of the project.  Under that system no one hoarded information, it militated against single points of failure, and ensured that disconnects were immediately addressed since visibility ensured accountability.

In many ways we have lost the ability to recreate the PM control office in digitized form.  Part of the reason resides in the 20th century organization of development and production into divisions of labor.  In project management, the specialization of disciplines organized themselves around particular functions: estimating and planning, schedule management, cost management, risk management, resource management, logistics, systems engineering, operational requirements, and financial management, among others.  Software was developed to address each of these areas with clear lines of demarcation drawn that approximated the points of separation among the disciplines.  What the software manufacturers forgot (or never knew) was that the PMO is the organizing entity and it is an interdisciplinary team.

To return to our example: WADs and BCRs; a survey of the leading planning and scheduling applications shows that while their marketing literature addresses baselines and baseline changes (and not all of them address even this basic function), they still do not understand complex project management.  There is a difference between resources assigned to a time-phased network schedule and the resources planned against technical achievement related to the work breakdown structure (WBS).  Given proper integration they should align.  In most cases they do not.  This is why most scheduling application manufacturers who claim to measure earned value, do not.  Their models assume that the expended resources align with the plan to date, in lieu of volume-based measurement.  Further, eventually understanding this concept does not produce a digitized solution, since an understanding of the other specific elements of program control is necessary.

For example, projects are initiated either through internal work authorizations in response to a market need, or based on the requirements of a contract.  Depending on the mix of competencies required to perform the work financial elements such as labor rates, overhead, G&A, allowable margin (depending on contract type), etc. will apply–what is euphemistically called “complex rates.”  An organization may need to manage multiple rate sets based on the types of efforts undertaken, with a many-to-many relationship between rate sets and projects/subprojects.

Once again, the task of establishing the proper relationships at the appropriate level is necessary.  This will then affect the timing of WAD initiation, and will have a direct bearing on the BCR approval process, given that it is heavily influenced by “what-if?” analysis against resource, labor, and financial availability and accountability (a complicated process in itself).  Thus the schedule network is not the only element affected, nor the overarching one, given the assessed impact on cost, technical achievement, and qualitative external risk.

These are but two examples of sub-optimization due to deficiencies in project management applications.  The response–and in my opinion a lazy one (or one based on the fact that oftentimes software companies know nothing of their customers’ operations)–has been to develop the alternative euphemism for “workaround”: best of breed.  Oftentimes this is simply a means of collecting revenue for a function that is missing from the core application.  It is the software equivalent of division of labor: each piece of software performs functions relating to specific disciplines and where there are gaps these are filled by niche solutions or Excel.  What this approach does not do is meet the requirements of the PMO control office, since it perpetuates application “swim lanes,” with the multidisciplinary requirements of project management relegated to manual interfaces and application data reconciliation.  It also pushes–and therefore magnifies–risk at the senior level of the project management team, effectively defeating organizational fail safes designed to reduce risk through, among other methods, delegation of responsibility to technical teams, and project planning and execution constructed around short duration/work-focused activities.  It also reduces productivity, information credibility, and unnecessarily increases cost–the exact opposite of the rationale used for investing in software technology.

It is time for this practice to end.  Technologies exist today to remove application “swim lanes” and address the multidisciplinary needs of successful project management.  Excel isn’t the answer; cross-application data access, proper data integration, and data processing into user-directed intelligence, properly aggregated and distributed based on role and optimum need to know, is.

Frame by Frame: Framing Assumptions and Project Success or Failure

When we wake up in the morning we enter the day with a set of assumptions about ourselves, our environment, and the world around us.  So too when we undertake projects.  I’ve just returned from the latest NDIA IPMD meeting in Washington, D.C. and the most intriguing presentation at the meeting was given by Irv Blickstein regarding a RAND root cause analysis of major program breaches.  In short, a major breach in the cost of a program is defined by the Nunn-McCurdy amendment that was first passed in 1982, in which a major defense program breaches its projected baseline cost by more than 15%.

The issue of what constitutes programmatic success and failure has generated a fair amount of discussion among the readers of this blog.  The report, which is linked above, is full of useful information regarding Major Defense Acquisition Program (also known as MDAP) breaches under Nunn-McCurdy, but for purposes of this post readers should turn to page 83.  In setting up a project (or program), project/program managers must make a set of assumptions regarding the “uncertain elements of program execution” centered around cost, technical performance, and schedule.  These assumptions are what are referred to as “framing assumptions.”

A framing assumption is one in which there are signposts along the way to determine if an assumption regarding the project/program has changed over time.  Thus, according to the authors, the precise definition of a framing assumption is “any explicit or implicit assumption that is central in shaping cost, schedule, or performance expectations.”  An interesting aspect of their perspective and study is that the three-legged stool of program performance relegates risk to serving as a method that informs the three key elements of program execution, not as one of the three elements.  I have engaged in several conversations over the last two weeks regarding this issue.  Oftentimes the question goes: can’t we incorporate technical performance as an element of risk?  Short answer:  No, you can’t (or shouldn’t).  Long answer: risk is a set of methods for overcoming the implicit invalidity of single point estimates found in too many systems being used (like estimates-at-complete, estimates-to-complete, and the various indices found in earned value management, as well as a means of incorporating qualitative environmental factors not otherwise categorizable), not an element essential to defining the end item application being developed and produced.  Looked at another way, if you are writing a performance specification, then performance is a key determinate of program success.

Additional criteria for a framing assumption are also provided in the RAND study.  These are that the assumptions must be determinative, that is, the consequences of the assumption being wrong significantly affects the program in an essential way.  They must also be unmitigable, that is, the consequences of the assumption being wrong are unavoidable.  They must be uncertain, that is, the outcome or certainty of it being right or wrong cannot be determined in advance.  They must be independent and not dependent on another event or series of events.  Finally, they must be distinctive, in setting the program apart from other efforts.

RAND then applied the framing assumption methodology to a number of programs.  The latest NDIA meeting was an opportunity to provide an update of conclusions based on the work first done in 2013.  What the researchers found was that framing assumptions which are kept at a high level, be developed early in a program’s life cycle, and should be reviewed on a regular basis to determine validity.  They also found that a program breached the threshold when a framing assumption became invalid.  Project and program managers, and requirements personnel have at least intuitively known this for quite some time.  Over the years, this is the reason given for requirements changes and contract modifications over the course of development that result in cost, performance, and schedule impacts.

What is different about the RAND study is that they have outlined a practical process for making these determinations early enough for a project/program to be adjusted with changing circumstances.  For example, the numbers of framing assumptions of all MDAPs in the study could be boiled down to four or five, which are easily tested against reality during the milestone and other reviews held over the course of a program.  This is particularly important given the lengthened time-frames of major acquisitions from development to production.

Looking at these results, my own observation is that this is a useful tool for identifying course corrections that are needed before they manifest into cost and schedule impacts, particularly given that leadership at PARCA has been stressing agile acquisition strategies.  The goal here, it seems, is to allow for course corrections before the inertia of the effort leads to failure or–more likely–the development and deployment of an end item that does not entirely meet the needs of the Defense Department.  (That such “disappointments” often far outstrip the capabilities of our adversaries is a topic for a different post).

I think the court is still out on whether course corrections, given the inertia of work and effort already expended at the point that a framing assumption would be tested as invalid, can ever truly be offsetting to the point of avoiding a breach, unless we then rebrand the existing effort as a new program once it has modified its structure to account for new framing assumptions.  Study after study has shown that project performance is pretty well baked in at the 20% mark.  For MDAPs, much of the front-loaded efforts in technology selection and application have been made.  After all, systems require inputs and to change a system requires more inputs, not less, to overcome the inertia of all of the previous effort, not to mention work in progress.   This is basic physics whether we are dealing with physical systems or complex adaptive (economic) systems.

Certainly, more efficient technology that affects the units of measurement within program performance can result in cost savings or avoidance, but that is usually not the case.  There is a bit of magical thinking here: that commercial technologies will provide a breakthrough to allow for such a positive effect.  This is an ideological idea not borne out by reality.  The fact is that most of the significant technological breakthroughs we have seen over the last 70 years–from the microchip to the internet and now to drones–have resulted from public investments, sometimes in public-private ventures, sometimes in seeded technologies that are then released into the public domain.  The purpose of most developmental programs is to invest in R&D to organically develop technologies (utilizing the talents of the quasi-private A&D industry) or provide economic incentives to incorporate technologies that do not currently exist.

Regardless, the RAND study has identified an important concept in determining the root causes of overruns.  It seems to me that a formalized process of identifying framing assumptions should be applied and done at the inception of the program.  The majority of the assessments to test the framing assumptions should then need to be made prior to the 20% mark as measured by program schedule and effort.  It is easier and more realistic to overcome the bow-wave of effort at that point than further down the line.

Note: I have modified the post to clarify my analysis of the “three-legged stool” of program performance in regard to where risk resides.

Synchroncity — What is proper schedule and cost integration?

Much has been said about the achievement of schedule and cost integration (or lack thereof) in the project management community.  Much of it consists of hand waving and magic asterisks that hide the significant reconciliation that goes on behind the scenes.  From an intellectually honest approach that does not use the topic as a means of promoting a proprietary solution is that authored by Rasdorf and Abudayyeah back in 1991 entitled, “Cost and Schedule Control Integration: Issues and Needs.”

It is worthwhile revisiting this paper, I think, because it was authored in a world not yet fully automated, and so is immune to the software tool-specific promotion that oftentimes dominates the discussion.  In their paper they outlined several approaches to breaking down cost and work in project management in order to provide control and track performance.  One of the most promising methods that they identified at the time was the unified approach that had originated in aerospace, in which a work breakdown structure (WBS) is constructed based on discrete work packages in which budget and schedule are unified at a particular level of detail to allow for full control and traceability.

The concept of the WBS and its interrelationship to the organizational breakdown structure (OBS) has become much more sophisticated over the years, but there has been a barrier that has caused this ideal to be fully achieved.  Ironically it is the introduction of technology that is the culprit.

During the first phase of digitalization that occurred in the project management industry not too long after Radsdorf and Abudayyeah published their paper, there was a boom in dot coms.  For business and organizations the practice was to find a specialty or niche and fill it with an automated solution to take over the laborious tasks of calculation previously achieved by human intervention.  (I still have both my slide rule and first scientific calculator hidden away somewhere, though I have thankfully wiped square root tables from my memory).

For those of us who worked in project and acquisition management, our lives were built around the 20th century concept of division of labor.  In PM this meant we had cost analysts, schedule analysts, risk analysts, financial analysts and specialists, systems analysts, engineers broken down by subspecialties (electrical, mechanical, systems, aviation) and sub-subspecialties (Naval engineers, aviation, electronics and avionics, specific airframes, software, etc.).  As a result, the first phase of digitization followed the pathway of the existing specialties, finding niches in which to inhabit, which provided a good steady and secure living to software companies and developers.

For project controls, much of this infrastructure remains in place.  There are entire organizations today that will construct a schedule for a project using one set of specialists and the performance management baseline (PMB) in another, and then reconciling the two, not just in the initial phase of the project, but across the entire life of the project.  From the standard of the integrated structure that brings together cost and schedule this makes no practical sense.  From a business efficiency perspective this is an unnecessary cost.

As much as it is cited by many authors and speakers, the Coopers & Lybrand with TASC, Inc. paper entitled “The DoD Regulatory Cost Premium” is impossible to find on-line.  Despite its widespread citation the study demonstrated that by the time one got down to the third “cost” driver due to regulatory requirements that the projected “savings” was a fraction of 1% of the total contract cost.  The interesting issue not faced by the study is, were the tables turned, how much would such contracts be reduced if all management controls in the company were reduced or eliminated since they contribute as elements to overhead and G&A?  More to the point here, if the processes applied by industry were optimized what would the be the cost savings involved?

A study conduct by RAND Corporation in 2006 accurately points out that a number of studies had been conducted since 1986, all of which promised significant impacts in terms of cost savings by focusing on what were perceived as drivers for unnecessary costs.  The Department of Defense and the military services in particular took the Coopers & Lybrand study very seriously because of its methodology, but achieved minimal savings against those promised.  Of course, the various studies do not clearly articulate the cost risk associated with removing the marginal cost of oversight and regulation. Given our renewed experience with lack of regulation in the mortgage and financial management sectors of the economy that brought about the worst economic and financial collapse since 1929, one my look at these various studies in a new light.

The RAND study outlines the difficulties in the methodologies and conclusions of the studies undertaken, especially the acquisition reforms initiated by DoD and the military services as a result of the Coopers & Lybrand study.  But, how, you may ask does this relate to cost and schedule integration?

The present means that industry uses in many places takes a sub-optimized approach to project management, particularly when it applies to cost and schedule integration, which really consists of physical cost and schedule reconciliation.  A system is split into two separate entities, though they are clearly one entity, constructed separately, and then adjusted using manual intervention which defeats the purpose of automation.  This may be common practice but it is not best practice.

Government policy, which has pushed compliance to the contractor, oftentimes rewards this sub-optimization and provides little incentive to change the status quo.  Software industry manufacturers who are embedded with old technologies are all too willing to promote the status quo–appropriating the term “integration” while, in reality, offering interfaces and workarounds after the fact.  Those personnel residing in line and staff positions defined by the mid-20th century approach of division of labor are all too happy to continue operating using outmoded methods and tools.  Paradoxically these are personnel in industry that would never advocate using outmoded airframes, jet engines, avionics, or ship types.

So it is time to stop rewarding sub-optimization.  The first step in doing this is through the normalization of data from these niche proprietary applications and “rewiring” them at the proper level of integration so that the systemic faults can be viewed by all stakeholders in the oversight and regulatory chain.  Nothing seems to be more effective in correcting a hidden defect than some sunshine and a fresh set of eyes.

If industry and government are truly serious about reforming acquisition and project management in order to achieve significant cost savings in the face of tight budgets and increasing commitments due to geopolitical instability, then systemic reforms from the bottom up are the means to achieve the goal; not the elimination of controls.  As John Kennedy once said in paraphrasing Chesterton, “Don’t take down a fence unless you know why it was put up.”  The key is not to undermine the strength and integrity of the WBS-based approach to project control and performance measurement (or to eliminate it), but to streamline it so that it achieves its ideal as closely as our inherently faulty tools and methods will allow.

 

Better Knock-Knock-Knock on Wood — The Essential Need for Better Schedule-Cost Integration

Back in early to mid-1990s, when NSFNET was making the transition to the modern internet, I was just finishing up my second assignment as an IT project manager and transitioning to a full-blown Program Executive Office (PEO) Business Manager and CIO at a major Naval Systems Command.  The expanded potential of a more open internet was on everyone’s mind and, on the positive side, on how barriers to previously stove-piped data could be broken down in order to drive optimization of the use of that data (after processing it into useable intelligence).  The next step was then to use that information, which was opened to a larger audience that previously was excluded from it, and to juxtapose and integrate it with other essential data (processed into intelligence) to provide insights not previously realized.

Here we are almost 20 years later and I am disappointed to see in practice that the old barriers to information optimization still exist in many places where technology should have long ago broken this mindset.  Recently I have discussed cases at conferences and among PM professionals where the Performance Management Baseline (PMB), that is, the plan that is used to measure financial value of the work performed, is constructed separately from and without reference to the Integrated Master Schedule (IMS) until well after the fact.  This is a challenge to common sense.

Project management is based on the translation of a contract specification into a plan to build something.  The basic steps after many years of professional development are so tried and true that it should be rote at this point:  Integrated Master Plan (IMP) –> Integrated Master Schedule (IMS) with Schedule Risk Assessment (SRA) –> Resource assignments with negotiated rates –> Develop work packages, link to financials, and roll-up of WBS –> Performance Management Baseline (PMB).  The arrows represent the relationships between the elements.  Feel free to adjust semantics and add additional items to the process such as a technical performance baseline, testing and evaluation plans, systems descriptions to ensure traceability, milestone tracking, etc.  But the basic elements of project planning and execution pretty much remain the same–that’s all there is folks.  The complexity and time spent to go through the steps varies based on the complexity of the scope being undertaken.  For a long-term project involving billions or millions of dollars the interrelationships and supporting documentation is quite involved, for short-term efforts the process may be in mental process of the person doing the job.  But in the end, regardless of terminology, these are the basic elements of PM.

When one breaks this cycle and decides to build each of the elements independently from the other it is akin to building a bridge in sections without using an overarching plan.  Result:  it’s not going to meet in the center.  One can argue that it is perfectly fine to build the PMB concurrent with the IMS if the former is informed by the latter.  But in practice I find that this is rarely the case.  So what we have, then, is a case where a bridge is imperfectly matched when the two sections meet in the middle requiring constant readjustment and realignment.  Furthermore, the manner in which the schedule activities are aligned with the budget vary from project to project, even within the same organization.  So not only do we not use a common plan in building our notional bridge, we decide to avoid standardization of bolts and connectors too, just to make it that more interesting.

The last defense in this sub-optimized environment is: well, if we are adjusting it every month through the project team what difference does it make?  Isn’t this integration nonetheless?  Response #1:  No.  Response #2:  THIS-IS-THE-CHALLENGE-THAT-DIGITAL-SYSTEMS-ARE-DESIGNED-TO-OVERCOME.  The reason why this is not integration is because it simultaneously ignores the lessons learned in the SRA and prevents insights gained through optimization.  If our planning documents are contingent on a month-to-month basis then the performance measured against them is of little value and always open to question, and not just on the margins.  Furthermore, utilization of valuable project management personnel on performing what is essentially clerical work in today’s environment is indefensible.  If there are economic incentives for doing this it is time for project stakeholders and policymakers to end them.

It is time to break down the artificial barriers that define cost and schedule analysts.  Either you know project and program management or you don’t.  There is no magic wall between the two disciplines, given that one cannot exist without the other.  Furthermore, more standardization, not less, is called for.  For anyone who has tried to decipher schedules where smiley-faces, and non-standard and multiple structures are in use in the same schedule, which defy reference to a cost control account, it is clear that both the consulting and project management communities are failing to instill professionalism.

Otherwise, as in my title, it’s like knocking on wood.