schedule analysis | Life, Project Management, and Everything

When You’re a Jet You’re a Jet all the Way — Software as a Change Agent for Professional Development

Earlier in the week Dave Gordon at his blog responded to my post on data normalization and rightly introduced the need for data rationalization. I had omitted the last concept in my own post, but strongly implied that the two were closely aligned in my broad definition of normalization beyond the boundaries of eliminating redundancies. In the end in thinking about this, I prefer Dave’s dichotomy because it more clearly defines what we are doing.

Later in the week I found myself elaborating on these issues in discussions with customers and other professionals in the project management discipline. In the projects in which I am involved, what I have found is that the process of normalizing and rationalizing data, even historical data which, contrary to Dave’s assertion can be maintained–at least in my business–acts as a change agent in defining the agnostic characteristics what defines the type of data being normalized and rationalized.

What I mean here is that, for instance, a time-phased CPM schedule that eventually becomes an integrated master schedule has an analogue. For years we have been told, mostly by marketing types working for software manufacturers, that there is a secret sauce that they provide that cannot be reconciled against their competitors. As a result, entire professional organizations, conferences, white papers, and presentations have been given to prove this assertion. When looking at the data, however, the assertion is invalid.

The key differentiator between CPM scheduling applications is the optimization engine. That is the secret sauce and the black box where the valuable IP lies. It is the algorithms in the optimization that identifies for us those schedule activities that are on the critical and near critical paths. But when you run these engines side by side on the same schedule, their results are well within one standard deviation of one another.

Keep in mind that I’m talking about differences in data related to normalization and rationalization and whether these differences can be reconciled. There are other differences in features between the applications that do make a difference in their use and functionality: whether they can lock down a baseline, manage multiple baselines, prevent future work from being planned and executed in the past (yes, this happens), handle hammocks, scale properly, etc. Because of these functional differences the same data may have been given a different value in the table or the file. As Dave Gordon rightly points out, reconciling what on the surface are irreconcilable values requires specialized knowledge. Well, if you have that specialized knowledge then you can achieve what otherwise seems impossible. Once you achieve this “impossible” feat, it quickly becomes apparent that the features and functions involved are based on a very limited number of values that are common across CPM scheduling applications.

This should not be surprising for those of you out there that have been doing this a long time. Back in the 1980s we would use visual display boards to map out short segments of the schedule. We would manually construct schedules in very rudimentary (by today’s standards) mainframe computers and get very long dot matrix representations of the schedule to tape to the “War Room” walls. The resources, risks, etc. had to be drawn on the schedule. This manual process required an understanding of CPM schedule construction similar to someone still using long division today. There was actually a time when people had to memorize their log and square root tables. it was not very efficient, but the deep understanding of the analogue schedule has since been lost with the introduction of new technology. This came to mind when I saw on LinkedIn a question of the types of questions that should be asked of a master scheduler in an interview.

As a result of new technology, schedulers aligned themselves into camps based on the application they selected or was selected for them in their job. Over time I have seen brand loyalty turn into partisanship. Once again, this should not be surprising. If you spent ten years of your career on a very popular scheduling application, anything that may undermine one’s investment in that choice–and which makes employment possible–will be deemed as a threat.

I first came upon this behavior years ago when I was serving as CIO for a project management organization. Some PMOs could not share information–not even e-mail and documents–because most were using PCs and some were using Macs. Problem was that the key PMO was using Mac. This was before Microsoft and Apple got together and solved this for us. Needless to say this undermined organizational effectiveness. My attempt to get everyone on the same page in terms of operating system compatibility sparked a significant backlash. Luckily for me Microsoft soon introduced its first solution to address this issue. So, in the end, the “Macintites”, as we good naturedly called them, could use their Macs for business common to other parts of the organization.

This almost cultish behavior finds itself in new places today: in the iPhone and Droid wars, in the use of Agile, and among CPM scheduling application partisans. It is true that those of us in the software industry certainly want to see brand loyalty. It is one of the key measures of success in proving the product’s value and effectiveness. But it need not undermine the fact that a scheduler is a key specialist in the project management discipline. If you are a Jet, you don’t need to be a Jet all the way.

Since creating generic analogues of schedules from submitted third-party data, I have found that insights into project performance can be noted that previously were not available. The power of digitization along with normalization and rationalization allows the data to be effectively integrated at the proper point of intersection with other dimensions of project performance such as cost performance and risk. Freed from the shackles of having to learn the specific idiosyncrasies of particular applications, the deep understanding of scheduling is being reintroduced. This is a long time in coming.

Margin Call — Schedule Margin and Schedule Risk

A discussion at the LinkedIn site for the NDIA IPMD regarding schedule margin has raised some good insight and recommendations for this aspect of project planning and execution. Current guidance from the U.S. Department of Defense for those engaged in the level of intense project management that characterizes the industry has been somewhat vague and open to interpretation. Some of this, I think, is due to the competing proprietary lexicon from software manufacturers that have been dominant in the industry.

But mostly the change in defining this term is due to positive developments. That is, the change is due to the convergence garnered from long experience among the various PM disciplines that allow us to more clearly define and distinguish between schedule margin, schedule buffer, schedule contingency, and schedule reserve. It is also due to the ability of more powerful software generations to actually apply the concept in real planning without it being a thumb in the air-type exercise.

Concerning this topic, Yancy Qualls of Bell Helicopter gave an excellent presentation at the January NDIA IPMD meeting in Tucson. His proposal makes a great deal of sense and, I think, is a good first step toward true integration and a more elegant conceptual solution. In his proposal, Mr. Qualls clearly defines the scheduling terminology by drawing analogies to similar concepts on the cost side. This construction certainly overcomes a lot of misconceptions about the purpose and meaning of these terms. But, I think, his analogies also imply something more significant and it is this: that there is a common linkage between establishing management reserve and schedule reserve, and there are cost/schedule equivalencies that also apply to margin, buffer, and contingency.

After all, resources must be time-phased and these are dollarized. But usually the relationship stops there and is distinguished by that characteristic being measured: measures of value or measures of timing; that is, the value of the work accomplished against the Performance Management Baseline (PMB) is different from the various measures of progress recorded against the Integrated Master Schedule (IMS). This is why we look at both cost and schedule variances on the value of work performed from a cost perspective, and physical accomplishment against time. These are fundamental concepts.

To date, the most significant proposal advanced to reconcile the two different measures was put forth by Walt Lipke of Oklahoma City Air Logistics Center in the method known as earned schedule. But the method hasn’t been entirely embraced. Studies have shown that it has its own limitations, but that it is a good supplement those measures currently in use, not a substitute for them.

Thus, we are still left with the need of making a strong, logical, and cohesive connection between cost and schedule in our planning. The baseline plans constructed for both the IMS and PMB do not stand apart or, at least, should not. They are instead the end result of a continuum in the construction of our project systems. As such, there should be a tie between cost and schedule that allows us to determine the proper amount of margin, buffer, and contingency in a manner that is consistent across both sub-system artifacts.

This is where risk comes in and the correct assessment of risk at the appropriate level of measurement, given that our measures of performance are being measured against different denominators. For schedule margin, in Mr. Qualls’ presentation, it is the Schedule Risk Analysis (SRA). But this then leads us to look at how that would be done.

Fortuitously, during this same meeting, Andrew Uhlig of Raytheon Missile Systems gave an interesting presentation on historical SRA results, building models from such results, and using them to inform current projects. What I was most impressed with in this presentation was that his research finds that the actual results from schedule performance do not conform to any of the usual distribution curves found in the standard models. Instead of normal, triangle, or pert distributions, what he found is a spike, in which a large percentage of the completions fell exactly on the planned duration. Thus, distribution was skewed around the spike, with the late durations–the right tail–much longer than the left.

What is essential about the work of Mr. Uhlig is that, rather than using small samples with their biases, he using empirical data to inform his analysis. This is a pervasive problem in project management. Mr. Qualls makes this same point in his own presentation, using the example of the Jordan-era Chicago Bulls as an example, where each subsequent win–combined with probabilities that show that the team could win all 82 games–does not mean that they will actually perform the feat. In actuality (and in reality) the probability of this occurring is quite small. Glen Alleman at his Herding Cats blog covers this same issue, emphasizing the need for empirical data.

The results of the Uhlig presentation are interesting, not only because they throw into question the results using the three common distributions used in schedule risk analysis under simulated Monte Carlo, but also because they may suggest, in my opinion, an observation or reporting bias. Discrete distribution methods, as Mr. Uhlig proposes, will properly model the distribution for such cases using our parametric analysis. But they will not reflect the quality of the data collected.

Short duration activities are designed to overcome subjectivity through their structure. The shorter the duration, the more discrete the work being measured, the less likely occurrence of “gaming” the system. But if we find, as Mr. Uhlig does, that 29% of 20 day activities report exactly 20 days, then there is a need to test the validity of the spike itself. It is not that it is necessarily wrong. Perhaps the structure of the short duration combined with the discrete nature of the linkage to work has done its job. One would expect a short tail to the left and a long tail to the right of the spike. But there is also a possibility that variation around the target duration is being judged as “close enough” to warrant a report of completion at day 20.

So does this pass the “So What?” test? Yes, if only because we know that the combined inertia of all of the work performed at any one time on the project will eventually be realized in the form of a larger amount of risk in proportion to the remaining work. If the reported results are pushing risk to the right because the reported performance is optimistic against the actual performance, then we will get false positives. If the actual performance is pessimistic against actual performance–a less likely scenario in my opinion–then we will get false negatives.

But regardless of these further inquiries that I think need to be made regarding the linkage between cost and schedule, and the validity of results from SRAs, we now have two positive steps in the right direction in clarifying areas that in the past have perplexed project managers. Properly identifying schedule reserve, margin, buffer, and contingency, combined with properly conducting SRAs using discrete distributions based on actual historical results will go quite far in allowing us to introduce better predictive measures in project management.

Go With the Flow — What is a Better Indicator: Earned Value or Cash Flow?

A lot of ink has been devoted to what constitutes “best practices” in PM but oftentimes these discussions tend to get diverted into overtly commercial activities that promote a set of products or trademarked services that in actuality are well-trod project management techniques given a fancy name or acronym. We see this often with “road shows” and “seminars” that are blatant marketing events. This tends to undermine the desire of PM professionals to find out what really gives us good information by both getting in the way of new synergies and by tying “best practices” to proprietary solutions. All too often “common practice” and “proprietary limitations” pass for “best practice.”

Recently I have been involved in discussions and the formulation of guides on indicators that tell us something important regarding the condition of the project throughout its life cycle. All too often the conversation settles on earned value with the proposition that all indicators lead back to it. But this is an error since it is but one method for determining performance, which looks solely at one dimension of the project.

There are, after all, other obvious processes and plans that measure different dimensions of project performance. The first such example is schedule performance. A few years ago there was an attempt to more closely tie schedule and cost as an earned value metric, which was and is called “earned schedule.” In particular, it had many strengths against what was posited as its alternative–schedule variance as calculated by earned value. But both are a misnomer, even when earned schedule is offered as an alternative to earned value while at the same time adhering to its methods. Neither measures schedule, that is, time-based performance against a plan consisting of activities. The two artifacts can never be reconciled and reduced to one metric because they measure different things. The statistical measure that would result would have no basis in reality, adding an unnecessary statistical layer that obfuscates instead of clarifying the underlying condition. So what do we look at you may ask? Well–the schedule. The schedule itself contains many opportunities to measure its dimension in order to develop useful metrics and indicators.

For example, a number of these indicators have been in place for quite some time: Baseline Execution Index (BEI), Critical Path Length Index (CPLI), early start/late start, early finish/late finish, bow-wave analysis, hit-miss indices, etc. These all can be found in the literature, such as here and here and here.

Typically, then, the first step toward integration is tying these different metrics and indicators of the schedule and EVM dimensions at an appropriate level through the WBS or other structures. The juxtaposition of these differing dimensions, particularly in a grid or GANTT, gives us the ability to determine if there is a correlation between the various indicators. We can then determine–over time–the strength and consistency of the various correlations. Further, we can take this one step further to conclude which ones lead us to causation. Only then do we get to “best practice.” This hard work to get to best practice is still in its infancy.

But this is only the first step toward “integrated” performance measurement. There are other areas of integration that are needed to give us a multidimensional view of what is happening in terms of project performance. Risk is certainly one additional area–and a commonly heard one–but I want to take this a step further.

For among my various jobs in the past included business management within a project management organization. This usually translated into financial management, but not traditional financial management that focuses on the needs of the enterprise. Instead, I am referring to project financial management, which is a horse of a different color, since it is focused at the micro-programmatic level on both schedule and resource management, given that planned activities and the resources assigned to them must be funded.

Thus, having the funding in place to execute the work is the antecedent and, I would argue, the overriding factor to project success. Outside of construction project management, where the focus on cash-flow is a truism, we see this play out in publicly funded project management through the budget hearing process. Even when we are dealing with multiyear R&D funding the project goes through this same process. During each review, financial risk is assessed to ensure that work is being performed and budget (program) is being executed. Earned value will determine the variance between the financial plan and the value of the execution, but the level of funding–or cash flow–will determine what gets done during any particular period of time. The burn rate (expenditure) is the proof that things are getting done, even if the value may be less than what is actually expended.

In public funding of projects, especially in A&D, the proper “color” of money (R&D, Operations & Maintenance, etc.) available at the right time oftentimes is a better predictor of project success than the metrics and indicators which assume that the planned budget, schedule, and resources will be provided to support the baseline. But things change, including the appropriation and release of funds. As a result, any “best practice” that confines itself to only one or two of the dimensions of project assessment fail to meet the definition.

In the words of Gus Grissom in The Right Stuff, “No bucks, no Buck Rogers.”

Doin’ It Right (On Scheduling)

Been reading quite a bit about assessments and analysis lately–14 Point Assessment, project health assessments, etc. Assessment and analysis is a necessary role of oversight, particularly in contracting efforts involving public funds on the part of agencies tasked with that role. From a project management perspective, however, the only beneficiaries seem to be one-off best-of-breed software tool manufacturers and some consultants who specialize in this sort of thing.

Assessment of the quality of our antecedent project artifacts is a necessary evil only because the defects in those plans undermine our ability to determine what is really happening in the project. It is but a band-aid–a temporary patch for a more systemic problem, for we must ask ourselves: how was a schedule that breached several elements of the 14-Point Assessment constructed in the first place? This is, of course, a rhetorical question and one well known by most, if not all, of my colleagues.

That many of our systems are designed to catch relatively basic defects after the fact and to construct lists to correct them–time and resources that are rarely planned for by project teams–is, in fact, a quantitative indicator of a systemic problem. There is no doubt, as Glen Alleman said in his most recent post on Herding Cats, that we need to establish intermediate closed systems to assess performance in each of the discrete segments of our plan. This is Systems Analysis 101. But these feedback loops are rarely budgeted. When they are budgeted, as in EVM, it is usually viewed as a regulatory cost that requires documentation that proves the overriding elucidation of benefits. These benefits would be more generally accepted if the indicators were more clearly tied to cause-and-effect and provided in a timely enough manner for course correction. But a great deal of effort is still expended in fixing the underlying artifacts on which our analysis depends, well after the fact. This makes project performance analysis that much harder.

Making corrections to course based on your taking fixes when entering port is an acceptable practice. Using a wrong or erroneous chart is incompetence. Thus, there is an alternative way to view this problem and that is to accept no defects in the construction of governing project management planning documents.

In discussing this issue, I have been reminded by colleagues that doing this very thing was the stated purpose of the Integrated Baseline Review when it was first deployed by the Navy in the 1990s. I served as a member of that first IBR Steering Group when the process was being tested and deployed. In fact, the initial recommendations of the group was that the IBR was not to be treated or imposed as an external inspection–which is much as it is being applied today–but rather an opportunity to identify risks and opportunities within the program team to ensure that the essential project artifacts: the master plan (IMP), integrated master schedule (IMS), performance management baseline (PMB), risk analysis systems, technical performance plan and milestones, etc., which would eventually inform both the Systems Description (SD) and CAM notebooks were properly constructed. In addition, the IBR was intended to be reconvened as necessary over the life of a project or program when changes necessitated adjustments to the processes that affected program performance and expectations.

So what is the solution? I would posit that it involves several changes.

First, is that the artificial dichotomy of the cost and schedule analyst disciplines needs to end, both across the industry and through the professional organizations that support them. That there is both a College of Performance Management and a multiplicity of schedule-focused organizations–separate and, in many cases, in competition with one another. It made a great deal of sense to create specialties when these disciplines were still evolving and involved specialized knowledge that caused very high barriers to entry. But the advancement of information systems have not only broken down these barriers to understanding and utilizing the methods of these specialties, the cross-fertilization of disciplines have provided us insights into the systems we are tasked to manage in ways that seemed impossible just five or six years ago: two to three full software generations over that time.

Second, is that we have allowed well entrenched technology to constrain our view of the possible for too long. We obviously know that we have come a long way from physically posting time-phased plans on a magnetic board. But we have also come a long way from being constrained by software technology that limits us to hard-coded applications that do only one thing–whether that one thing be EVM, schedule analysis, technical performance, or based on fixing errors (to finish the analogy) well after we have decided to sail. All too often the last condition puts us in the shoals.