Should we measure resilience?

I’ve been reflecting on the idea of measuring resilience since the conference in Montpellier last month where @vgalaz quipped “Resilience metrics is the new black @resilience2014”. Efforts to measure resilience are well underway while at the same time there are concerns about what exactly is being measured and whether this shift in focus misses the point of what resilience thinking has to offer. My own thinking on this is that it depends on what you are trying to achieve but a deeper understanding of both perspectives is likely to benefit both approaches in the long-term.

Approaching the dialogue from two perspectives
The Resilience 2014 conference aimed to facilitate dialogue among researchers and practitioners from the resilience research community and the development community. To date, resilience has been conceptualized and applied in a variety of ways. Research along the lines of Holling, Gunderson, Folke, and Walker as well as many others in the Resilience Alliance network and beyond, has emerged from a complex adaptive systems’ perspective and in particular, a focus on ecosystems and integrated social-ecological systems. By contrast, development communities tend to approach resilience from a more human-centered perspective with a focus on livelihoods, risk reduction, and human well-being. What both communities hold in common is a desire to operationalize resilience by applying theoretical insights to real world problems and changing the way we manage and interact with the environment for more sustainable and equitable outcomes.

The demand side of resilience in development
The rapid uptake of resilience thinking by development agencies and foundations has forced the issue of resilience implementation and challenged the research community to make the leap from theory to practice to metrics. While resilience practice is not entirely new (see Walker & Salt 2012) and case studies have informed theoretical advances over the years the wide-ranging application of resilience thinking to development issues, is a relatively recent phenomenon. Development programs and projects operate within a different realm and have their own established frameworks, protocols, and practices. Notably, development programs require well-defined mechanisms for evaluating interventions and more specifically, metrics for quantifying and judging the success of their actions and investments. Thus the challenge that presents itself is how to measure resilience, if indeed it can or should be measured? This is a nuanced question, and much like the concept it addresses, there are multiple dimensions and no easy answers but it remains a worthy pursuit.

To measure or not to measure?
There is a concern shared by many that resilience may not live up to its promise for a variety of reasons including the potential for narrow interpretations and a selective or limited understanding of what can be a relatively abstract concept, but also because of a what some have identified as a lack of quantifiable metrics for evaluation purposes. In Luca Alinova’s plenary presentation he spoke of the very real threat of resilience being adopted and applied in name only, whereby others capitalize on the current trendiness of the concept while much of the same ineffective practices continue under the guise of a new name. In his words “there is a big risk of labeling some bad habits with a new name”. Any failures of course, will have a handy scapegoat and an enormous opportunity will have been lost. Similarly, there is a real risk that in the rush to measure resilience and develop quantitative metrics for comparative purposes, what is actually measured may represent the same things that have long been monitored and measured but are now being packaged in the language of resilience to meet the demand.

The fact remains however, that resilience will and already is, being measured.

What exactly is being measured?
If resilience must be measured to be meaningful to the development community, then how best to measure it? Luca Alinovi suggests we need to measure resilience at the household level rather than at an individual level because it is the interactions that are important. He also cautioned though that we are still far away from the dynamic analysis that is needed as well as a general approach for different types of systems.

Much of the discussion at Resilience 2014 around the topic of metrics tended to focus on food security and crisis impacts. Alexis Hoskins presented on the progress being made by the Food and Nutrition Security Resilience Measurement Technical Working Group that has produced a framing paper outlining the challenges in measuring resilience. They have also produced a set of resilience measurement principles that echo Alinovi’s call for dynamic analysis and reflect both systems-based requirements (multi-level interactions, rates of change, inherent volatility) as well as human dimensions (e.g., desirability of system states, people’s perceptions, vulnerability connections). The recommendations and next steps that follow from the measurement principles appear promising because they account for the underlying concepts of complex systems dynamics and cross-scale interactions, while recognizing the need for both quantitative and qualitative data to understand causal mechanisms.

Other presenters similarly advocated for a mixed method approach to measuring resilience, combining qualitative and quantitative data, as well as steps for interpreting data and providing the necessary contextualization that metrics alone cannot fully capture. Yet another type of approach offered by Christophe Bene, was a resilience proxy based on the cost of impacts calculated from the sum of anticipation costs + impact costs + recovery costs. Bene’s postulate being “the more resilient an individual the lower the costs it takes to get through a specific shock”. Assigning monetary values as a means of measuring resilience has many parallels in the ecosystem services literature, which increasingly recognizes the need to also consider nonmonetary values.

What is missing?
There is clearly something to be gained by measuring resilience, but any formula attempting to capture a dynamic system property will inevitably involve tradeoffs for simplifying purposes and something will be lost. Understanding exactly what is missing from resilience metrics or what is potentially lost with a shift in focus from understanding the resilience of a system to measuring the resilience of a system remains to be clearly articulated. In resilience assessment, a main objective of the exercise is to re-conceptualize a system, place, or issue from an alternative perspective, i.e., through a resilience lens and focusing on interactions such that new insights emerge and interventions can be better informed. How a system behaves is not a function of the sum of its parts so it follows that measuring component parts cannot capture what is meaningful about resilience.

To date, most metrics being proposed focus on social variables and the human dimensions of resilience, as opposed to taking an integrated social-ecological systems (SES) approach. Conceptualizing humans as part of nature and placing people within ecosystems, instead of keeping them separate, represents an important advance in resilience research and sustainability science more broadly. Metrics for resilience and more generally, the application of the concept in practice also stands to benefit from taking an SES approach.

Some considerations for developing resilience metrics
It has been said before that resilience is an overarching concept that encompasses many other core concepts. Biggs and colleagues (2012) identify seven principles for building resilience of ecosystem services. Assuming a given bundle of ES is desirable (and knowing for whom it matters), these seven facets can be managed to strengthen and enhance the resilience of the system. They include: maintaining diversity and redundancy, managing connectivity, managing slow variables and feedbacks, fostering complex adaptive systems thinking, encouraging learning, broadening participation, and promoting polycentric governance systems. To the extent resilience metrics can effectively address these seven principles, they would provide valuable information to anyone wanting to characterize and monitor the capacity of the system to maintain a desired set of ecosystem services in the face of continued change or disturbance.

A final consideration is that resilience is not always a good thing. As Brian Walker stated in his plenary presentation, part of the understanding required is knowing where we need to build resilience, and where we need to reduce it to enable transformation. A range of different types of traps characterized by rigid social and ecological processes that are tied to environmental degradation and livelihood impoverishment make change a real challenge (Boonstra and de Boer, 2014). Where traps exist, the goal may be to reduce the resilience of the current state of the system and build transformative capacity, which may require monitoring and measuring a different set of variables.

Measuring resilience should be possible but finding suitable indicators and metrics that retain key attributes of the concept will also need to reflect the fact that resilience is a means and not an end.

Biggs et al. 2012. Towards Principles for Enhancing the Resilience of Ecosystem Services. Annu. Rev. Environ. Resour. 37:421-48.

Walker, B. & D. Salt. 2012. Resilience Practice: Building capacity to absorb disturbance and maintain function. Island Press, Washington, D.C.

4 thoughts on “Should we measure resilience?”

  1. Thanks Allyson! This is very thought-provoking and it reminds me of similar debates around other frameworks that have been applied with more or less success to “development” (for lack of a better word). To me the key is what you allude to in asking to consider what we gain (or lose) when we switch from understanding the resilience of a system to measuring the resilience of a system.

    Let me draw one parallel with the development literature: Sen’s capabilities approach, which was adopted by the United Nations to frame their Human Development Report in 1990. According to Sen people’s well-being depends on ‘functionings’, which are the various things that a person may value doing or being, such as attending a live concert or being healthy. Capability refers to a person’s freedom to achieve and enjoy these functionings. From Sen’s point of view, improving human well-being depends on removing the obstacles that stand on the way of expanding people’s freedom to achieve the functionings that they value.

    Taking even a relatively frivolous functioning such as the concert example above, one can see how there are a lot of factors that might interfere with a person’s capability to fulfill it. For instance, having enough disposable income to purchase the ticket is a factor, but things such as having appropriate clothes to wear, feeling confident that they will not be denied entry to the concert hall based on their gender or ethnicity, or feeling safe coming home at night, are also factors that might interfere with a person’s capability to fulfill this functioning. Hence, in this case, improving human well-being could mean anything from creating opportunities to earn income to enacting policies to eradicate violence and bigotry.

    Of course, one of the difficulties with Sen’s capabilities approach is dealing with the fact that there are as many functionings, and as many combinations of functionings, as there are people, making the approach difficult to communicate and to measure. Eventually, the Human Development Index (HDI) was adopted for the Human Development Report. The HDI is a composite index that includes indicators of income, literacy and life expectancy into a single measurement. The idea behind the HDI is that the three indicators represent functionings that are universally shared by all people, that is, people enjoy having a decent standard of living, being knowledgeable, and living long lives.

    Clearly the HDI is a rough approximation that does not begin to capture the complexity of human experience that Sen initially alluded to. On the other hand, HDI is widely popular, easy to grasp (no jargon), and more multi-dimensional than, say, GDP alone. But we can go on measuring HDI without realizing that a highly educated female lawyer that lives to be 85 might deprive herself of going out to those concerts because she ultimately feels unsafe. The same might happen with resilience: we can come up with system metrics but miss a lot of the systems thinking.

    Marta Berbés-Blázquez
    York University
    Faculty of Environmental Studies
    4700 Keele Street, Toronto, M3J 1P3 Canada

  2. People who want to “measure resilience” need to keep in mind the following lessons from a practitioner’s point of view:
    • “Resilience” has meaning only in the context of the crisis – type of disruption (natural disaster, pandemic, major economic contraction), and magnitude of the disruption. In that sense, it is not “resilience” but “resilience to [disruption].”
    • Resilience is a manifestation of strength; vulnerability is not its antipode. Addressing vulnerabilities may not translate into greater resilience.
    • Community resilience can only be understood in terms of a “Whole Community” approach (i.e., community as a system of systems). A community hasn’t really recovered from a shock until all of its systems have recovered. You’ve aptly captured that above!
    • Social capital (both bridging and bonding) is probably the most important single determinant of resilience. However, this needs to be seen in terms of each of the community’s systems. The resilience of the water system will depend on the social capital of those involved in running the water system. Ditto electricity, finance, health care, social services. While I really like the HDI, it’s really a very coarse measure of a community’s resilience.
    • Finally, measures depend on who’s wielding the yardstick. If I’m a community leader, I have very different motivations than an academic researcher. Most “academic” measures are aimed at explaining what has happened; a practitioner would like to be able to predict what might happen, and then prioritize actions to get to the best outcomes she can with the resources at hand.

  3. Poverty has (at least in the past) benefited from some degree of consensus around the way it can be measured/monitored. Even if the concept of income poverty and its Foster-Greer-Thorbecke metric have been continuously criticized for being too simplistic and mono-dimensional, some would certainly argue that this mono-dimensional nature is actually a strength when it comes to measure poverty

  4. Hi and thanks for this extremely well-thought out piece.

    Resilience is a nebulous concept. The Red Cross invested a lot of money post-Tsunami into evaluating the success of their community-based DRR programs, which led to a study published here: – that defines resilience in terms of a set of minimum characteristics.

    I have to ask, however, a follow-up question.

    I know standardization is the norm, but as you’ve pointed out above, each context and situation is different and may require a unique set of tools to measure successful outcomes. Perhaps the problem we have is that we are trying to find one way of looking at a very complex concept. In the meantime, the entire industry would be better served by investing energy into developing tailored solutions to resilience challenges and documenting and sharing the approaches that are successful with one another.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>