Out of Winter Woodshedding — Thinking about Project Risk and passing the “So What?” test

“Woodshedding” is a slang term in music, particularly in relation to jazz, in which the musician practices on an instrument usually outside of public performance, the purpose of which is to explore new musical insights without critical judgment.  This can be done with or without the participation of other musicians.  For example, much attention recently has been given to Bob Dylan’s Basement Tapes release.  Usually it is unusual to bother recording such music, given the purpose of improvisation and exploration, and so few additional examples of “basement tapes” exist from other notable artists.

So for me the holiday is a sort of opportunity to do some woodshedding.  The next step is to vet such thoughts on informal media, such as this blog, where the high standards involved in white and professional papers do not allow for informal dialogue and exchange of information, and thoughts are not yet fully formed and defensible.  My latest mental romps have been inspired by the movie about Alan Turing–The Imitation Game–and the British series The Bletchley Circle.  Thinking about one of the fathers of modern computing reminded me that the first use of the term “computer” referred to people.

As a matter of fact, though the terminology now refers to the digital devices that have insinuated themselves into every part of our lives, people continue to act as computers.  Despite fantastical fears surrounding AI taking our jobs and taking over the world, we are far from the singularity.  Our digital devices can only be programmed to go so far.  The so-called heuristics in computing today are still hard-wired functions, similar to replicating the methods used by a good con artist in “reading” the audience or the mark.  With the new technology in dealing with big data we have the ability to many of the methods originated by the people in the real life Bletchley Park of the Second World War.  Still, even with refinements and advances in the math, they provide great external information regarding the patterns and probable actions of the objects of the data, but very little insight into the internal cause-and-effect that creates the data, which still requires human intervention, computation, empathy, and insight.

Thus, my latest woodshedding has involved thinking about project risk.  The reason for this is the emphasis recently on the use of simulated Monte Carlo analysis in project management, usually focused on the time-phased schedule.  Cost is also sometimes included in this discussion as a function of resources assigned to the time-phased plan, though the fatal error in this approach is to fail to understand that technical achievement and financial value analysis are separate functions that require a bit more computation.

It is useful to understand the original purpose of simulated Monte Carlo analysis.  Nobel physicist Murray Gell-Mann, while working at RAND Corporation (Research and No Development) came up with the method with a team of other physicists (Jess Marcum and Keith Breuckner) to determine the probability of a number coming up from a set of seemingly random numbers.  For a full rendering of the theory and its proof Gell-Mann provides a good overview in his book The Quark and the Jaguar.  The insight derived from the insight of Monte Carlo computation has been to show that systems in the universe often organize themselves into patterns.  Instead of some event being probable by chance, we find that, given all of the events that have occurred to date, that there is some determinism which will yield regularities that can be tracked and predicted.  Thus, the use of simulated Monte Carlo analysis in our nether world of project management, which inhabits that void between microeconomics and business economics, provides us with some transient predictive probabilities given the information stream at that particular time, of the risks that have manifested and are influencing the project.

What the use of Monte Carlo and other such methods in identifying regularities do not do is to determine cause-and-effect.  We attempt to bridge this deficiency with qualitative risk in which we articulate risk factors to handle that are then tied to cost and schedule artifacts.  This is good as far as it goes.  But it seems that we have some of this backward.  Oftentimes, despite the application of these systems to project management, we still fail to overcome the risks inherent in the project, which then require a redefinition of project goals.  We often attribute these failures to personnel systems and there are no amount of consultants all too willing to sell the latest secret answer to project success.  Yet, despite years of such consulting methods applied to many of the same organizations, there is still a fairly consistent rate of failure in properly identifying cause-and-effect.

Cause-and-effect is the purpose of all of our metrics.  Only by properly “computing” cause-and-effect will we pass the “So What?” test.  Our first forays into this area involve modeling.  Given enough data we can model our systems and, when the real-time results of our in-time experiments play out to approximate what actually happens then we know that our models are true.  Both economists and physicists (well, the best ones) use the modeling method.  This allows us to get the answer even if not entirely understanding the question of the internal workings that lead to the final result.  As in Douglas Adams’ answer to the secret of life, the universe, and everything where the answer is “42,” we can at least work backwards.  And oftentimes this is what we are left, which explains the high rate of failure in time.

While I was pondering this reality I came across this article in Quanta magazine outlining the new important work of the MIT physicist Jeremy England entitled “A New Physics Theory of Life.”  From the perspective of evolutionary biology, this pretty much shows that not only does the Second Law of Thermodynamics support the existence and evolution of life (which we’ve known as far back as Schrodinger), but probably makes life inevitable under a host of conditions.  In relation to project management and risk, it was this passage that struck me most forcefully:

“Chris Jarzynski, now at the University of Maryland, and Gavin Crooks, now at Lawrence Berkeley National Laboratory. Jarzynski and Crooks showed that the entropy produced by a thermodynamic process, such as the cooling of a cup of coffee, corresponds to a simple ratio: the probability that the atoms will undergo that process divided by their probability of undergoing the reverse process (that is, spontaneously interacting in such a way that the coffee warms up). As entropy production increases, so does this ratio: A system’s behavior becomes more and more “irreversible.” The simple yet rigorous formula could in principle be applied to any thermodynamic process, no matter how fast or far from equilibrium. “Our understanding of far-from-equilibrium statistical mechanics greatly improved,” Grosberg said. England, who is trained in both biochemistry and physics, started his own lab at MIT two years ago and decided to apply the new knowledge of statistical physics to biology.”

No project is a closed system (just as the earth is not on a larger level).  The level of entropy in the system will vary by the external inputs that will change it:  effort, resources, and technical expertise.  As I have written previously (and somewhat controversially), there is both chaos and determinism in our systems.  An individual or a system of individuals can adapt to the conditions in which they are placed but only to a certain level.  It is non-zero that an individual or system of individuals can largely overcome the risks realized to date, but the probability of that occurring is vanishingly small.  The chance that a peasant will be a president is the same.  The idea that it is possible, even if vanishingly so, keeps the class of peasants in line so that those born with privilege can continue to reassuringly pretend that their success is more than mathematics.

When we measure risk what we are measuring is the amount of entropy in the system that we need to handle, or overcome.  We do this by borrowing energy in the form of resources of some kind from other, external systems.  The conditions in which we operate may be ideal or less than ideal.

What England’s work combined with his predecessors’ seem to suggest is that the Second Law almost makes life inevitable except where it is impossible.  For astrophysics this makes the entire Rare Earth hypothesis a non sequitur.  That is, wherever life can develop it will develop.  The life that does develop is fit for its environment and continues to evolve as changes to the environment occur.  Thus, new forms of organization and structure are found in otherwise chaotic systems as a natural outgrowth of entropy.

Similarly, when we look at more cohesive and less complex systems, such as projects, what we find are systems that adapt and are fit for the environments in which they are conceived.  This insight is not new and has been observed for organizations using more mundane tools, such as Deming’s red bead experiment.  Scientifically, however, we now have insight into the means of determining what the limitations of success are given the risk and entropy that has already been realized, against the needed resources that are needed to bring the project within acceptable ranges of success.  This information goes beyond simply stating the problem, leaving the computing to the person and thus passes the “So What?” test.

Sunday Contemplation — Finding Wisdom — Charles Darwin

Charles Darwin

“Ignorance more frequently begets confidence than does knowledge: it is those who know little, not those who know much, who so positively assert that this or that problem will never be solved by science.”

The human species owes a debt of gratitude to Charles Darwin that can never be adequately paid.  The young Darwin struggled against being categorized in a society and a time that very much needed to categorize everything and everyone.  His early education demonstrated his keen, inquisitive, and initially undisciplined mind, the last aspect of his character that he himself noted and worked to overcome.

The grandson of two prominent British abolitionists, Erasmus Darwin and Josiah Wedgwood, he was born to a outwardly conventional but inwardly nurturing and intellectually stimulating family.  He was an avid amateur naturalist as a boy and studied to follow in his father’s footsteps as a physician.  He attended medical school but his other interests caused him to neglect his studies.  Frustrated with what they viewed as his lack of prospects, his family enrolled him in divinity school to become an Anglican pastor.  Darwin studied little but found his passion in the then craze of beetle collecting and was influenced by the Cambridge naturalists that pursued what was then known as natural theology–the proposition that the best way to know the deity was to understand its creation.  His main studies focused on what we now identify as botany, geology as well as biology.

After receiving his degree Darwin proceeded to take literally the remonstrance of Alexander von Humboldt to travel widely in order to gain new knowledge.  Upon the recommendation of his mentor at Cambridge, John Stevens Henslow, he was taken aboard the HMS Beagle’s South American surveying expedition as a self-financed naturalist.  This voyage was a transforming one for Darwin and it is best to use his own words from his autobiography in order to describe the nature of that transformation.

“…Whilst on board the Beagle I was quite orthodox, and I remember being heartily laughed at by several of the officers… for quoting the Bible as an unanswerable authority on some point of morality… But I had gradually come by this time, i.e., 1836 to 1839, to see that the Old Testament from its manifestly false history of the world, with the Tower of Babel, the rainbow at sign, &c., &c., and from its attributing to God the feelings of a revengeful tyrant, was no more to be trusted than the sacred books of the Hindoos, or the beliefs of any barbarian.

…By further reflecting that the clearest evidence would be requisite to make any sane man believe in the miracles by which Christianity is supported, (and that the more we know of the fixed laws of nature the more incredible do miracles become), that the men at that time were ignorant and credulous to a degree almost uncomprehensible by us, that the Gospels cannot be proved to have been written simultaneously with the events, that they differ in many important details, far too important, as it seemed to me, to be admitted as the usual inaccuracies of eyewitnesses; by such reflections as these, which I give not as having the least novelty or value, but as they influenced me, I gradually came to disbelieve in Christianity as a divine revelation. The fact that many false religions have spread over large portions of the earth like wild-fire had some weight with me. Beautiful as is the morality of the New Testament, it can be hardly denied that its perfection depends in part on the interpretation which we now put on metaphors and allegories.

But I was very unwilling to give up my belief… Thus disbelief crept over me at a very slow rate, but was at last complete. The rate was so slow that I felt no distress, and have never since doubted even for a single second that my conclusion was correct. I can indeed hardly see how anyone ought to wish Christianity to be true; for if so the plain language of the text seems to show that the men who do not believe, and this would include my Father, Brother and almost all of my friends, will be everlastingly punished.

And this is a damnable doctrine.”

Having thrown off his preconceived beliefs it is during the voyage of the Beagle that Charles Darwin became the modern scientist that we recognize today–the author of On the Origin of the Species and The Descent of Man. Much has been made of the theological nature of his origins and how they influenced his thinking, arguing that the construction of his scientific hypotheses and theories are simply an extension of a type of belief–what today is called “scientism.”  But this is ignorance and the term cannot exist except in the minds of those making the assertion.  It is only when Darwin freed himself from the shackles of his mind that he was able to perceive nature as it is, not as human society would have it.

It is obvious to us now as we read his narrative that he had not completely freed himself from the prejudices of his time.  But such is the nature of human advancement.  I was told early on as an historian that I would learn more about the prejudices of 18th century Britain by reading Gibbon’s Decline and Fall of the Roman Empire than I would learn of the Roman Empire–and it turned out that my mentor was correct.

But, unlike Gibbon, Darwin’s influence transcends his time because of the enforced discipline that he imposed on himself and his method.  After that seminal voyage it took years of study and the weight of evidence before Darwin felt confident to publish his findings–and then only under great pressure since other scientists were coming to the same conclusions and threatened to precede him on his life’s work.  His  theory is an elegant one and the weight of its elegance is found in his overview of it in the introduction to the Origin:

“As many more individuals of each species are born than can possibly survive; and as, consequently, there is a frequently recurring struggle for existence, it follows that any being, if it vary however slightly in any manner profitable to itself, under the complex and sometimes varying conditions of life, will have a better chance of surviving, and thus be naturally selected. From the strong principle of inheritance, any selected variety will tend to propagate its new and modified form.”

Darwin’s observations and theory–which is supported by over a century and a half of observation and confirmation–is one of the key insights in our understanding of ourselves and our position in the universe.  This insight is the basis of all other wisdom and in my opinion, without it, there can be no human knowledge that reaches the level of wisdom that means anything.  For all of the knowledge that we have amassed since that time–in geology, astronomy, biology, physics, neuroscience psychology–in virtually every area of learning–is informed by this one core insight into human existence and what we define as life on our planet.  To understand the evolution of species through the agent of natural selection one must understand the age of the universe, of the earth, the dynamics of geology, and the common origins and interconnection of all life.

As such, its implications transcend science in the same way as its implications transcended biology.  In 1995 the cognitive scientist and philosopher Daniel Dennett wrote his famous work summarizing the influence of Darwin’s theory on modern science and society in the late 20th century.  He gave the book the title Darwin’s Dangerous Idea.  As I sit here in the year 2014 it is apparent that this is still the case, not because what he observed was dangerous to know, but because it is an idea that undermines its opposite–the belief that the strong have a right to dominate the weak, that people can be categorized with some intrinsically superior and others inferior, and that economics and its handmaiden philosophy trumps all other insights when it comes to human society and conduct.

Many evolutionary biologists and others in the sciences with whom I have corresponded and discussed their bewilderment and frustration at the resistance, particularly in parts of the United States, to the essential wisdom in Darwinian observation.  It is, I think, because they do not see the historical and societal implications which is explained in their own theory.  It is dangerous not only because of its transcending of theological explanations of the universe and human existence, but also because it challenges the structure of social control and hierarchy upon which so many societies have been built in the modern era.  In understanding our own biology as primates, our instinctual feelings of tribalism, kinship, and hierarchy are still too strong in many areas to fully liberate us from our self-imposed shackles. Darwinian insight challenges the primacy of these feelings.

So dangerous was (and is) Darwin’s idea that Herbert Spencer published an alternative evolutionary theory based on earlier, pre-scientific evolutionary beliefs, known as Lamarckian evolution, which came to be known as Social Statics and has since been misnamed Social Darwinism.  This competing theory, most recently given new clothes by politicians and followers of the writer Ayn Rand, is without scientific merit, socially abhorrent, ethically indefensible, and sociopathically cruel.  So old is this meme that Darwin himself challenged this twisting of evolutionary theory:

“It is not the strongest or the most intelligent who will survive but those who can best manage change.”

Regarding the societal implications of his theory he wrote in his work The Voyage of the Beagle:

“If the misery of the poor be caused not by the laws of nature, but by our institutions, great is our sin.”

Darwin at first avoided addressing the more controversial aspects of his theory and it took him some time to decide to publish The Descent of Man.  From this work his theory of sexual selection alone stirred more than a little backlash.  As such, we see only glimpses of his view that the understanding of the nature of life would be a liberating force, not only in the sciences but in society at large.  But Darwin struggled with the questions of the “ought” as opposed to the “is” and, in the end demurred. It is only now that his descendents in the sciences have broached the topic once again, most significantly in the book The Moral Landscape, by the neuroscientist Sam Harris.

In the end, though, Darwin’s most significant contribution may result in the survival of our species.  The common origins that we all share and the combined threats of Global Warming, nuclear proliferation, and other weapons of mass destruction threaten our very existence, not to mention the extra-planetary threats from asteroids and comets.  The insights of Darwin and his descendents in the sciences may very well prevent our own self-destructive tendencies and ignorance from causing our extinction from this tiny planet.