Category Archives: Ideas

Christian Robert on Black Swans

Christian Robert, a Bayesian statistican at  Université Paris Dauphine comments on Taleb’s book the Black Swan in his blog article Of black swans and bleak prospects:

… I think that the book can be criticised solely from a statistical point of view as mostly missing the point. For instance, the notions of probable/improbable and randomness [that are constantly in use within the book] are always used in a vague sense and they thus mostly loose their meaning. (The distinction between random—that is, driven by a probability distribution—and fortuitous—that is, lacking any kind of reproducibility to be considered as a probability outcome—comes so late within the book as to be rather useless.) The extreme events that are called black swans are never analysed in terms of model shift, although they mostly correspond to cases where the background model had changed but the players were not aware of it. This somehow gives the impression that the author expects there exists a (deterministic) model that should explain even the most extreme phenomena. When considering some examples in the book like 09/11, this sounds ludicrous: the attack on 09/11 has nothing to do with randomness or a probabilistic model! Similarly, there is no discussion of the possible non-homogeneous nature of the time series leading to black swans.

…It is obviously a difficult exercise to write about popular Science without being populist and it must be almost inevitable to oversimplify one’s discourse by emphasizing a few examples over others, but I think the book overdoes it! By a fair margin. Worse, by attacking modelling tools like the Gaussian, models and modelers as a conglomerate of “charlatans”, it contributes to the anti-Scientist discourse that is unfortunately so prevalent today. Being a skeptic is commendable and scientists should never cease questioning their models, but throwing all models to the winds and using only “facts” to drive one’s decisions is not very helpful. As put by George Box (or by someone else before him), “all models are wrong, but some models are useful” and we (as statisticians) can devise tools to assess how wrong and how useful. Encouraging a total mistrust of anything scientific or academic is not helping in solving issues, but most surely pushes people in the arms of charlatans with ready answers.

via Andrew Gelman

Economist on fat tails and finance

A special report on the future of finance in The Economist Fallible mathematical models: In Plato’s cave:

… although the normal distribution closely matches the real world in the middle of the curve, where most of the gains or losses lie, it does not work well at the extreme edges, or “tails”. In markets extreme events are surprisingly common—their tails are “fat”. Benoît Mandelbrot, the mathematician who invented fractal theory, calculated that if the Dow Jones Industrial Average followed a normal distribution, it should have moved by more than 3.4% on 58 days between 1916 and 2003; in fact it did so 1,001 times. It should have moved by more than 4.5% on six days; it did so on 366. It should have moved by more than 7% only once in every 300,000 years; in the 20th century it did so 48 times.

In Mr Mandelbrot’s terms the market should have been “mildly” unstable. Instead it was “wildly” unstable. Financial markets are plagued not by “black swans”—seemingly inconceivable events that come up very occasionally—but by vicious snow-white swans that come along a lot more often than expected.

This puts VAR in a quandary. On the one hand, you cannot observe the tails of the VAR curve by studying extreme events, because extreme events are rare by definition. On the other you cannot deduce very much about the frequency of rare extreme events from the shape of the curve in the middle. Mathematically, the two are almost decoupled.

The drawback of failing to measure the tail beyond 99% is that it could leave out some reasonably common but devastating losses. VAR, in other words, is good at predicting small day-to-day losses in the heart of the distribution, but hopeless at predicting severe losses that are much rarer—arguably those that should worry you most.

When David Viniar, chief financial officer of Goldman Sachs, told the Financial Times in 2007 that the bank had seen “25-standard-deviation moves several days in a row”, he was saying that the markets were at the extreme tail of their distribution. The centre of their models did not begin to predict that the tails would move so violently. He meant to show how unstable the markets were. But he also showed how wrong the models were.

Modern finance may well be making the tails fatter, says Daron Acemoglu, an economist at MIT. When you trade away and invest in stocks, all sorts of specific risk, in foreign exchange, interest rates and so forth, you make your portfolio seem safer. But you are in fact swapping everyday risk for the exceptional risk that the worst will happen and your insurer will fail—as AIG did. Even as the predictable centre of the distribution appears less risky, the unobserved tail risk has grown. Your traders and managers will look as if they are earning good returns on lower risk when part of the true risk is hidden. They will want to be paid for their skill when in fact their risk-weighted returns may have fallen.

Chris Field says rate of climate change faster than estimated

At the AAAS meetings in Chicago Chris Field gave a presentation that argues that the Pace of Climate Change Exceeds Estimates:

“We are basically looking now at a future climate that’s beyond anything we’ve considered seriously in climate model simulations,” Christopher Field, founding director of the Carnegie Institution’s Department of Global Ecology at Stanford University, said at the annual meeting of the American Association for the Advancement of Science.

Field, a member of the United Nations’ Intergovernmental Panel on Climate Change, said emissions from burning fossil fuels since 2000 have largely outpaced the estimates used in the U.N. panel’s 2007 reports. The higher emissions are largely the result of the increased burning of coal in developing countries, he said.

Unexpectedly large amounts of carbon dioxide are being released into the atmosphere as the result of “feedback loops” that are speeding up natural processes. Prominent among these, evidence indicates, is a cycle in which higher temperatures are beginning to melt the arctic permafrost, which could release hundreds of billions of tons of carbon and methane into the atmosphere, said several scientists on a panel at the meeting.

The permafrost holds 1 trillion tons of carbon, and as much as 10 percent of that could be released this century, Field said. Melting permafrost also releases methane, which is 25 times more potent a greenhouse gas than carbon dioxide.

“It’s a vicious cycle of feedback where warming causes the release of carbon from permafrost, which causes more warming, which causes more release from permafrost,” Field said.

Evidence is also accumulating that terrestrial and marine ecosystems cannot remove as much carbon from the atmosphere as earlier estimates suggested, Field said.

While it takes a relatively long time for plants to take carbon out of the atmosphere, that carbon can be released rapidly by wildfires, which contribute about a third as much carbon to the atmosphere as burning fossil fuels, according to a paper Field co-authored.

Fires such as the recent deadly blazes in southern Australia have increased in recent years, and that trend is expected to continue, Field said. Warmer weather, earlier snowmelt, drought and beetle infestations facilitated by warmer climates are all contributing to the rising number of fires linked to climate change. Across large swaths of the United States and Canada, bark beetles have killed many mature trees, making forests more flammable. And tropical rain forests that were not susceptible to forest fires in the past are likely to become drier as temperatures rise, growing more vulnerable.

Preventing deforestation in the tropics is more important than in northern latitudes, the panel agreed, since lush tropical forests sequester more carbon than sparser northern forests. And deforestation in northern areas has benefits, since larger areas end up covered in exposed, heat-reflecting snow.

Many scientists and policymakers are advocating increased incentives for preserving tropical forests, especially in the face of demand for clearing forest to grow biofuel crops such as soy. Promoting biofuels without also creating forest-preservation incentives would be “like weatherizing your house and deliberately keeping your windows open,” said Peter Frumhoff, chief of the Union of Concerned Scientists’ climate program. “It’s just not a smart policy.”

Homer-Dixon on Our Panarchic Future

pnrchywwIn Worldwatch Magazine Thomas Homer-Dixon writes Our Panarchic Future about Buzz Holling‘s thinking, Panarchy and global transformation.  Homer-Dixon writes:

Holling embodies something truly rare: the kind of wisdom that comes when an enormously creative, perceptive, and courageous mind spends a half-century studying a phenomenon and distilling its essential patterns. In a conversation with him not long ago, I encouraged him to expand on many aspects of panarchy theory, filling gaps in my understanding and giving me nuance and perspective that only he could provide. As we came to the end of our conversation, I asked him a question that had been on my mind since our first meeting a year before, when he’d been adamant that humanity is at grave risk.

“Why do you feel the world is verging on some kind of systemic crisis?”

“There are three reasons,” he answered. “First, over the years my understanding of the adaptive cycle has improved, and I’ve also come to better understand how multiple adaptive cycles can be nested together-from small to large-to create a panarchy. I now believe that this theory tells us something quite general about the way complex systems, not just ecological systems, change over time. And collapse is usually part of the story.

“Second, I think rapidly rising connectivity within global systems-both economic and technological-increases the risk of deep collapse. That’s a collapse that cascades across adaptive cycles-a kind of pancaking implosion of the entire system as higher-level adaptive cycles collapse, which causes progressive collapse at lower levels.”

“A bit like the implosion of the World Trade Center towers,” I offered, “where the weight of the upper floors smashed through the lower floors like a pile driver.”

“Yes, but in a highly connected panarchy, the collapse doesn’t have to start at the top. It can be triggered at the microlevel or the macrolevel or somewhere in between. It’s the tight interlinking of the adaptive cycles across the whole system-from the individual right up to the level of the global economy and even Earth’s biosphere-that’s particularly dangerous because it increases the likelihood that many of the cycles will become synchronized and peak together. And if this happens, they’ll reinforce each other’s collapse.”

“The third reason,” he continued, “is the rise of mega-terrorism-the increasing risk of attacks that will kill huge numbers of people and produce major disruptions in world systems. I’m not sure why megaterrorism has become more likely now. I suppose it’s partly a result of technological changes and the rise of particularly virulent kinds of fundamentalism. But I do know that in a tightly connected world where vulnerabilities are aligned, such attacks could trigger deep collapse-and that’s particularly worrisome.

“This is a moment of great volatility and instability in the world system. We need urgently to do what we can to avoid deep collapse. We also need to figure out how to exploit the opportunity provided by crisis and collapse when they occur, because some kind of systemic breakdown is now almost certain.”

We can see the danger of the tectonic stresses in a new light if we think of humankind-including all our interactions with each other and with nature and all the flows of materials, energy, and information through our societies and technologies-as one immense social-ecological system. As this grand system we’ve created and live within moves up the growth phase of its adaptive cycle, it’s accumulating potential in the form of people’s skills and economic wealth. It’s also becoming more connected, regulated, and efficient-and ultimately less resilient. And finally, it’s becoming steadily more complex, which means it’s moving further and further from thermodynamic equilibrium. We need ever-larger inputs of high-quality energy to maintain this complexity. In the meantime, internal tectonic stresses-including worsening scarcity of our best source of high-quality energy, conventional oil-are building slowly but steadily.

So we’re overextending the growth phase of our global adaptive cycle. We’ll reach the top of this cycle when we’re no longer able to regulate or control the stresses building deep inside the global system. Then we’ll get earthquakelike events that will cause the system’s breakdown and simplification as it moves closer to thermodynamic equilibrium.

Panarchy theory also helps us better understand another critically important phenomenon: the denial that prevents us from seeing the dangers we face. Our explanations of the world around us-whether of Earth’s place in the cosmos or of the workings of our economy-move through their own adaptive cycles. When a favorite explanation encounters contradictory evidence, we make an ad hoc adjustment to it to account for this evidence-just like Ptolemy added epicycles to his explanation of the planets’ movements. In the process, our explanation moves through something akin to a growth phase: it becomes progressively more complex, cumbersome, and rigid; it loses resilience; and it’s ripe for collapse should another, better, theory come along.

The growth phase we’re in may seem like a natural and permanent state of affairs-and our world’s rising complexity, connectedness, efficiency, and regulation may seem relentless and unstoppable-but ultimately it isn’t sustainable. Still, we find it impossible to get off this upward escalator because our chronic state of denial about the seriousness of our situation-aided and abetted by powerful special interests that benefit from the status quo-keeps us from really seeing what’s happening or really considering other paths our world might follow. Radically different futures are beyond imagining. So we stay trapped on a path that takes us toward major breakdown.

The longer a system is “locked in” to its growth phase, says Buzz Holling, “the greater its vulnerability and the bigger and more dramatic its collapse will be.” If the growth phase goes on for too long, “deep collapse”-something like synchronous failure-eventually occurs. Collapse in this case is so catastrophic and cascades across so many physical and social boundaries that the system’s ability to regenerate itself is lost. [A] forest-fire shows how this happens: if too much tinder-dry debris has accumulated, the fire becomes too hot, which destroys the seeds that could be the source of the forest’s rebirth.

Holling thinks the world is reaching “a stage of vulnerability that could trigger a rare and major ‘pulse’ of social transformation.” Humankind has experienced only three or four such pulses during its entire evolution, including the transition from hunter-gatherer communities to agricultural settlement, the industrial revolution, and the recent global communications revolution. Today another pulse is about to begin. “The immense destruction that a new pulse signals is both frightening and creative,” he writes. “The only way to approach such a period, in which uncertainty is very large and one cannot predict what the future holds, is not to predict, but to experiment and act inventively and exuberantly via diverse adventures in living.”

Via Oonsie Biggs

Slow variables that shape bushfire resilience

fireausCreating the perfect firestorm on the BBC writes about two of the slow variables that produce a fire situation: climate and fuel accumulation:

Current climate projections point to an increase in fire-weather risk from warmer and drier conditions.

Two simulations used by Australia’s lead scientific agency, CSIRO, and the Australian Bureau of Meteorology point to the number of days with very high and extreme fire danger ratings increasing by some 4-25% by 2020, and 15-70% by 2050.

The agencies’ Climate Change in Australia report cites the example of Canberra which may be looking at an annual average of 26-29 very high or extreme fire danger days by 2020 and 28-38 days by 2050.

Continue reading

Stephen Pyne on the Australian fires

Photo from National Geographic

Photo from National Geographic

American fire historian Stephen Pyne comments in The Australian on the current Australian fires in Bushfire leader becomes laggard:

Australia knows better. It developed many key concepts of fire ecology and models of bushfire behaviour. It pioneered landscape-scale prescribed burning as a method of bushfire management. It devised the protocol for structure protection in the bush, especially the ingenious stratagem of leaving early or staying, preparing and defending. In recent decades it has beefed up active suppression capabilities and emergency services.

… Yet Australia keeps enduring the same Sisyphean cycle of calamitous conflagrations in the same places. It isn’t translating what it knows into its practices. It seems to be abandoning its historic solutions for the kind of telegenic suppression operations and political theatre that have failed elsewhere. Even when controlled burning is accepted in principle, there always seems a reason not to burn in this place or at this time. So the burning gets outsourced to lightning, accident and arson.

It’s too early to identify the particulars behind the latest catastrophe. But it’s likely that investigation will point to the same culprits, perhaps aggravated by climate change and arson. Both are relevant, but both are potential distractions.

Global warming might magnify outbreaks, but it would mean a change in degree, not in kind; and its effects must in any case be absorbed by the combustible cover.

Arson can put fire in the worst place at the worst time, but its power depends on the capacity to spread and to destroy susceptible buildings.

Yet neither is fundamental. With or without global warming or arson, damaging fires will come, spread as the landscape allows and inflict damage as structures permit. And it is there – with how Australians live on the land – that reform must go.

Australia will have fire, and it will recycle the conditions that can leverage small flames into holocausts. The choice is whether skilled people should backburn or leave fire-starting to lightning, clumsies and crazies.

After the 1939 Black Friday conflagration, a royal commission set into motion the modern era of bushfire management. At the time the official ambition of state-sponsored conservation was to eliminate fire as far as possible, and through fire exclusion alter the character of the landscape so it would become less fire prone. Leonard Stretton asked the nation’s forester why he continued to hold this view when it had never succeeded, when bushfires had inevitably wiped out his every repeated effort. Wryly, Stretton mocked the absurdity of those who sought to make sunburnt Australia into green England.

Black SaturdayII will yield another royal commission. Much has changed in 70 years; Australians are more urban, more sensitive to environmental issues, keener to protect unique ecological assets. Yet perhaps they are substituting another, more modern delusion: striving to remake the burning bush into an unburnt Oz, only to find this vision also repeatedly obliterated by remorseless fire.

Good points, but I think he under-states the change in settlement patterns, as increasing number of people live in ex-urban areas that complicate fire management, and also the risk that climate change produces a disequilibrium between vegetation and climate that can result in much larger than fires one would observe in an equilibrium climate.

Stephen Pyne is also on an ABC radio podcast discussing the environmental history of bushfire in Australia.

Australian fires

From ABC News

The deadly Australian bushfires appear to have been so deadly because they were unusual.  The fires caught people before they new it was coming, and unlike many fires they were too strong to fight.  What made these fires unusual?  At first look it seems to be a combination of a lack of social memory, fuel accumulation, and climate change – that together reduced community resilience to fire.

From the New Scientist’s Short sharp science blog Time for a new Australian bushfire policy?

Australia has a national fire-preparedness policy, perhaps uniquely, of encouraging people to either “stay and defend or leave early”. But as the tragedy continues to unfurl there will be increasing pressure to reexamine a policy developed for conditions that existed half a century ago.

The rationale behind the policy is that if you have a fire plan in place – that is, you have a water source, a pump that is not dependent on the power supply, you have ember-proofed your house, and so on – it is safer to stay and let the front pass over, than to leave at the last moment. And historically, it is true that most houses lost in bush fires have burnt because of defendable ember-strikes rather than direct contact with the fire, and most deaths have been due last-minute evacuations.

But conditions have changed. Southern Australia’s epic 12-year drought, higher temperatures due to climate change, and less “prescribed” burning to remove the plant life that acts as fuel, all combine to increase the risk of extreme fire. This year already, we’ve had several days in the mid-40s that have burnt leaves off trees, and squeezed the last drops of moisture out of already tinder-dry bush. One survivor described the ground underfoot prior to the fires going through as “like walking on cornflakes”.

Under those conditions, a fire plan may simply not be enough, as Victoria’s premier John Brumby told the Fairfax Radio Network today: “There is no question that there were people there who did everything right, put in place their fire plan and it wouldn’t matter, their house was just incinerated.” Brumby wants the policy rexamined.

People have changed too. Small towns like devastated Marysville – 90 minutes’ drive northeast of Melbourne – as well as the ever-expanding fire-prone city fringes, are as likely to be home to retirees and “treechangers” (city people who move to the country for a life change) as they are to families with generations of bushfire experience.

One of the commonest reasons for people who intend to stay and defend their properties to change their mind and leave at the last moment is that with no direct experience of fire they are not prepared psychologically.

According to Robert Heath, a psychologist at the University of South Australia in Adelaide, they don’t bank on the overwhelming heat, the lack of contact with the outside world, the darkness, or the noise: loud and like a huge blowtorch, apparently. Perhaps it’s not surprising then, that many people who lost their lives are thought to have done so while fleeing in their cars.

For more details see:

Thanks to Arijit Guha for pointers.

Traps of abstraction

In a recent paper in Science, Nira Liberman and Yaacov Trope review recent pyschological research on how people think about abstract situations.  They argue that there is similarity between how people cope with situations separated from their situation in time, space, social distance or uncertainty.  In their paper The Psychology of Transcending the Here and Now (2008 322 (5905): 1201) they write:

Our experiences of the world are limited to the self, here and now, yet people, events, and situations that are beyond our immediate experience populate our mind. We plan for the future, remember the past, think about remote locations, take others’ perspective, and consider alternatives to reality. In each case, we transcend the present to consider psychologically distant objects. An object is psychologically distant from us to the extent that it is remote in time (future or past) or in space; refers to experiences of others (e.g., relatives, acquaintances, or strangers); and unlikely to occur. But how do we transcend the present, evaluate, and make decisions with respect to psychologically distant objects? And how does increasing distance from objects affect the way we respond to these objects?

Although evolution, history, and child development have different time scales, we propose that the expanding horizons that all of them entail require and are enabled by the human capacity for abstract mental representation. This hypothesis is based on Construal Level Theory (CLT) of psychological distance (3, 4), which links psychological distance from objects to the mental construal of those objects. In the following, we explain what we mean by mental construal and how it relates to traversing psychological distances. We then describe research findings demonstrating that there is considerable commonality in the way people traverse different dimensions of psychological distance; that similar mental construal processes underlie traversing different distance dimensions; and that these construal processes guide the way people predict, evaluate, and plan psychologically near and distant situations.

In summary, a range of studies suggests that people rely on high-level construals to a greater extent when predicting, evaluating, and taking action with respect to more distant situations. Ironically, the increasing reliance on high-level construals for more distant situations often leads people to make more confident predictions, more polarized evaluations, and clearer choices. This result is counterintuitive if one believes that distant situations should afford less certainty and thus reduce confidence and decisiveness.

Whole Earth Catalog web archive

coevolutiongaiaThe entire 35-year archive of Whole Earth Catalogs, along with its Supplements, and descendant magazines – CoEvolution Quarterly, and Whole Earth are now available on the web.  The Whole Earth Catalog,was published in 1968 by Stewart Brand, it and its related  magazines embodied a certain type of Californian environmental thinking.  A key concept was systems – which included thinking about people,  and computers, as well as ecosystems, and when I read first read issues of the magazine as a teenager in the 1980s it was my first exposure to systems theory.

The full archive is in a difficult to navigate scanned form, which is difficult to link to or search, however some of the later articles are available as text.  However the archive includes lots of interesting stuff.  For example, Dana Meadows famous article on where to intervene in a system there.  Other interesting bits include articles by ecologists such as HT Odum, Paul Ehrlich, and Buzz Holling as well as an issue focused on scenario planning.

Continue reading

Terra preta the only way to save our civilization?

James Lovelock appears to be marginally more positive about our civilization’s capacity to avoid collapse, because of terra preta in a New Scientist interview One last chance to save mankind.  He says:

There is one way we could save ourselves and that is through the massive burial of charcoal. It would mean farmers turning all their agricultural waste – which contains carbon that the plants have spent the summer sequestering – into non-biodegradable charcoal, and burying it in the soil. Then you can start shifting really hefty quantities of carbon out of the system and pull the CO2 down quite fast.

Would it make enough of a difference?

Yes. The biosphere pumps out 550 gigatonnes of carbon yearly; we put in only 30 gigatonnes. Ninety-nine per cent of the carbon that is fixed by plants is released back into the atmosphere within a year or so by consumers like bacteria, nematodes and worms. What we can do is cheat those consumers by getting farmers to burn their crop waste at very low oxygen levels to turn it into charcoal, which the farmer then ploughs into the field. A little CO2 is released but the bulk of it gets converted to carbon. You get a few per cent of biofuel as a by-product of the combustion process, which the farmer can sell. This scheme would need no subsidy: the farmer would make a profit. This is the one thing we can do that will make a difference, but I bet they won’t do it.

Do you think we will survive?

I’m an optimistic pessimist. I think it’s wrong to assume we’ll survive 2 °C of warming: there are already too many people on Earth. At 4 °C we could not survive with even one-tenth of our current population. The reason is we would not find enough food, unless we synthesised it. Because of this, the cull during this century is going to be huge, up to 90 per cent. The number of people remaining at the end of the century will probably be a billion or less. It has happened before: between the ice ages there were bottlenecks when there were only 2000 people left. It’s happening again.

I don’t think humans react fast enough or are clever enough to handle what’s coming up. Kyoto was 11 years ago. Virtually nothing’s been done except endless talk and meetings.