I’ve received some nice feedback from my post “Guvmint Stuff.” One of the e-mails I received was from my colleague, Dan Zosh, with whom I worked previously at a very successful software company that was absorbed by a much larger company. Dan makes a good point regarding the usefulness of EVM and why it is viewed by so much of the Govcon project management community as a reporting as opposed to project management tool: the instability of program requirements and the PMB.
Years ago when I worked on Navy staff in D.C. my immediate military boss would lecture the requirements types on the fact that we live in a world of finite resources and that they couldn’t always get what they wanted–and certainly that the wrong time to start rethinking things was when the program was nearing the end of technical development and had entered its final Preliminary Design Review (PDR).
I had applied this same principle when a PM of software projects on two separate occasions. Had I allowed each review of the internal target customer base to adjust what “done” meant we would have never produced a solution to address a particular set of needs. “We are pushing for the baseline functionality in version 1.0 and revisions will be made in 1.1.” was my response. That was because I was aware of the fact that a very high percentage of software development projects in both private industry and government fail. In fact, one group looking at the history of 50,000 commercial and government software projects has found that of 3,555 projects between 2003 and 2012, only 6.4% were successful, with 52% challenged–meaning that they either didn’t meet expectations or ran over budget and/or schedule, and 41.4% completely failed. This is in line with what we had known to be true 20 years ago.
The reason for this is that we must know where we are going and what “done” means. My colleague Glen Alleman has been waging his own dialogue with the #noestimates crowd and the cultists from the far edges of the Agile side who say that you do not have to assess progress because you don’t have an estimate, what with two week sprints and all that. But my own experience with Agile proves that there is an overall plan and, yes, even if you don’t like it your customer doesn’t have unlimited resources and really expects that you come in within budget and the allotted time.
There is much more in common in IT and A&D than both disciplines would like to admit. After all, A&D is quite heavy in software development and deployment, which is essential to most modern systems. No doubt there are good reasons why requirements change, but the recent trend has been the shifting baseline, where variances are lost as the project is reset to some other definition of “done,” each revision contributing to cost growth.
In that environment EVM then really does only become a reporting tool viewed as an inconvenient historical document that records management by contingency. Is this, then, a project management problem or a contract management problem?
I would posit that this is a failure of control mostly on the contract management side of the ledger. Mr. Alleman’s discipline can be applied perfectly but as long as the underlying project space definition is under constant revision, his measurements will continue to shift. Some on the Agile side seem to have that very agenda in mind–change the goal posts and one never has to be accountable.
While it makes eminent sense to trade off new technologies for old when development proves that there are more economical pathways to achieving the technical goals of the project, this is rarely the case. Contracting officers routinely allow buy-ins to contracts knowing that the performance specification may be weak, and so down the line the contract will be modified significantly after award. Under R&D cost-plus contracts this is somewhat baked into the nature of the contract type and the efforts being undertaken–which is the reason for the periodic reviews–but this also leads to abuses such as zeroing variances as tradeoffs between control accounts, rubber baselines, and harvesting contracts for funding. These practices would not occur without the approval of the contracting officer, but it is a fine line between allowing internal adjustments within scope and changing the definition of the scope.
The issue, then, is a systemic one. The response, then, should be a multilateral, multidisciplinary approach to contract and project management. This includes stronger discipline in financial management that ensures that designated funds are being used for the scope intended, realistic estimates, well written performance specifications, and a contracting corps that understands that variances are never zeroed, contracts never harvested since the funds belong to the budget holders but, more importantly, understanding that the less expensive alternative proposal in which the technical approach to the performance specification is not fully fleshed out and is significantly below the most realistic, risk-adjusted estimate, is out of the competitive range.
Life, Project Management, and Everything 