Tag Archives: bromford lab

Learning from failure in complex environments

In his second blog post on the Learning from Failure workshop, Dyfrig Williams looks at failure in a complex environment.

It’s now been a few weeks since the Learning from Failure workshop, and my subsequent admission that I haven’t been very good at learning from my own failure. The event took place at the Wales Audit Office, so it was perhaps inevitable that we discussed the role of audit in learning from failure.

Systematic failure

James Reason Swiss Cheese Model. Source: BMJ, 2000 Mar 18:320(7237): 768-770

James Reason Swiss Cheese Model. Source: BMJ, 2000 Mar 18:320(7237): 768-770

Chris Bolton’s presentation was on the James Reason Swiss Cheese Failure Model, which compares human systems to layers of Swiss Cheese. Reason chose Swiss Cheese for a reason (see what I did there), as each layer is a defence against mistakes and errors, and things go badly wrong when the holes line-up. There’s an interesting critique of the model in the comments by Matt Wyatt of Complex Wales.

After a good Twitter conversation on the merits of different types of cheese as defence (I went patriotic and chose Caerphilly – ‘I crumble in the face of failure’), I looked at a model that Matt has developed, called the ‘Timeline of Inevitable Failure.’

Whereas the Swiss Cheese Model is a reflective model (you look back and check out the failure after Timeline of Inevitable Failureit’s occurred), Matt’s model is interesting as it offers opportunities to reflect on failure and its consequences at different stages, which fits in with a systematic approach to failure and chimes with some of the thinking in my last post on examining failure rigorously.

To be able to rectify failures at the early stage of the timeline, we have to be open and frank about failure, or issues will escalate and become bigger problems. By being comfortable with minor instances of failure, we’ll also be better prepared for when things go drastically wrong. As Matt says in another comment, ‘complex living systems will always fail, so instead of trying to make them failsafe, it’s much more useful to make them safe to fail.’ It’s well worth reading Chris’ post on Trojan Mice, which are safe to fail pilots, before delving in to a video of Dave Snowden discussing them as part of the Cynefin Framework.

You can see this approach in action through the work of the Bromford Lab and Dublin City Council’s Beta Projects. In terms of the latter, it’s worth checking out how their painting of traffic signal boxes led to less tagging and graffiti.

What does this mean for audit and audited bodies?

Aside for the recommendation  in the workshop to take your auditor out for lunch to better understand their approach to failure (which I’m completely on board with by the way!), this all relates to the complex environment in which public services are delivered and audited.

In Wales, this environment is about to change fundamentally with the Wellbeing of Future Generations Act. It’ll need a shift in thinking for organisations, as they’ll have to improve people’s wellbeing without compromising the ability of future generations to meet their needs. It’ll also be a challenge for us at the Wales Audit Office – it’s difficult to measure success when you don’t know what the future will look like. There’s a great post on the Wales Audit Office blog that outlines these challenges by Ann Webster, Assistant Auditor-General of New Zealand.

We’ve already shared steps that organisations can take to report effectively, including integrated reporting, at a seminar we held with the Sustainable Futures Commissioner. But in terms of this event, I was struck by some simple steps that organisations can take to evidence improvement. Jonathan Flowers gave a great example of how a Neighbourhood Network Scheme Manager asked for two instances a month of how the service had improved people’s lives. These narratives show that the service is moving in the right direction and can be used at the project evaluation stage.

Where now?

When it comes to evaluating our project, we’ve been gathering examples of how our work has led to organisations adopting good practice. These aren’t often measures in themselves, but complex case studies of how services have changed.

And in terms of our work, it’s important that we continue to have these conversations about failure, so that it’s normalised and people can be honest about it. And if we can do that, we’re in a better place to help organisations take further steps to improve their services.

Dysgu o Fethiant mewn amgylchedd cymhleth

Yn ei ail blogbost am weithdy Dysgu o Fethiant, mae Dyfrig Williams yn edrych ar fethiant mewn amgylchedd cymhleth.

Mae cwpl o wythnosau wedi mynd heibio ers y gweithdy Dysgu o Fethiant, ac mae fe hefyd wedi bod yn sbel ers fy ymateb iddo, ble wnes i gyfaddef fy mod i heb ddysgu lot o fy methiant i. Cynhaliwyd y digwyddiad yn Swyddfa Archwilio Cymru, felly roedd ein trafodaeth amdano rôl archwilio mewn dysgu o fethiant braidd yn anochel.

Methiant systematig

Model Methiant Caws y Swistir gan James Reason

Model Methiant Caws y Swistir gan James Reason. Ffynhonnell: BMJ, 2000 Mar 18:320(7237): 768-770

Roedd cyflwyniad Chris Bolton ar Fodel Methiant Caws y Swistir gan James Reason, sy’n cymharu systemau dynol i haenau o Gaws y Swistir. Dewisodd Reason Caws y Swistir am reswm, gan fod pob haen yn amddiffyniad yn erbyn camgymeriadau a gwallau, ac mae pethau’n mynd o chwith pan mae’r tyllau’n galluogi i’r methiant parhau. Mae ‘na feirniadaeth ddiddorol o’r model yn y sylwadau gan Matt Wyatt o Gymhleth Cymru.

Ar ôl sgwrs dda ar Twitter ar wahanol fathau o gaws fel amddiffyniad (fel gwladgarwr dewisais i gaws Caerffili – ‘Rwy’n crymblo yn wyneb methiant’), edrychais i ar fodel mae Matt wedi datblygu o’r enw’r ‘Llinell Amser o Fethiant Anochel.’

Llinell Amser o Fethiant AnochelMae Model Caws y Swistir yn fodel adlewyrchol (gan eich bod chi’n edrych yn ôl ar y methiant ar ôl iddo gymryd lle), ond mae gen i ddiddordeb mewn model Matt achos bod e’n cynnig cyfleoedd i fyfyrio ar fethiant a’i ganlyniadau ar wahanol gyfnodau. Mae hwn yn cydseinio â’r ymagwedd systematig tuag at fethiant yn fy mlogbost diwethaf ar adlewyrchu ar fethiant yn drylwyr.

Er mwyn gallu unioni’r methiannau yng nghyfnod cynnar y llinell amser, mae’n rhaid i ni fod yn agored a gonest am fethiant, neu bydd materion yn gwaethygu i fod yn broblemau mwy. Os ydym yn gyfforddus â mân achosion o fethiant, rydyn ni hefyd yn allu paratoi’n well ar gyfer pan mae pethau’n mynd yn rili wael. Fel mae Matt yn dweud yn sylw arall, ‘Mae systemau byw cymhleth bob amser yn methu, felly yn hytrach na cheisio dileu methiant, mae’n llawer mwy defnyddiol i’w gwneud yn ddiogel i fethu.’ Mae’n werth darllen blogbost Chris ar ‘Lygod Trojan,’ sef cynlluniau peilot sy’n ddiogel i fethu, cyn gwylio fideo o Dave Snowden sy’n trafod nhw fel rhan o’r Fframwaith Cynefin.

Gallwch weld y dull hwn ar waith drwy waith y Bromford Lab a Phrosiectau Beta Cyngor Dinas Dulyn. O ran Cyngor Dulyn, mae’n werth edrych ar sut mae paentio o flychau goleuadau traffig wedi arwain at lot llai o dagio a graffiti.

Beth mae hwn yn meddwl i archwilwyr a’r cyrff sy’n cael eu harchwilio?

Ar wahân i un awgrym yn y gweithdy i gymryd eich archwilydd allan am ginio i drafod eu hymagwedd tuag at fethiant (ac rydw i’n hollol gytuno â hyn gyda llaw!), mae hyn i gyd yn ymwneud â’r amgylchedd cymhleth mae gwasanaethau cyhoeddus yn cael eu darparu a’u harchwilio ynddo.

Yng Nghymru, mae’r amgylchedd yma ar fin newid yn sylfaenol pan mae Deddf Lles Genedlaethau’r Dyfodol yn cael ei gyflwyno. Bydd rhaid i fudiadau meddwl ychydig yn wahanol am eu gwaith, achos bydd rhaid iddynt wella lles pobl heb beryglu gallu cenedlaethau’r dyfodol i ddiwallu eu hanghenion. Bydd y ddeddf hefyd yn her i ni yn Swyddfa Archwilio Cymru – mae’n anodd mesur llwyddiant pan ddydych chi ddim yn gwybod beth fydd y dyfodol yn edrych fel. Mae blogbost wych ar flog Swyddfa Archwilio Cymru ble mae Ann Webster, Archwilydd Cyffredinol Cynorthwyol Seland Newydd yn amlinellu’r heriau ni’n wynebu.

Rydyn ni eisoes wedi rhannu rhai camau gall mudiadau cymryd i adrodd yn ôl yn effeithiol, gan gynnwys adrodd integredig, mewn seminar wnaethon ni rhedeg gyda’r Comisiynydd Dyfodol Cynaliadwy. Ond o ran y digwyddiad yma, wnes i gael fy nharo gan rai camau syml gall mudiadau cymryd i ddangos gwelliant. Rhoddodd Jonathan Flowers enghraifft wych o sut wnaeth Rheolwr Cynllun Rhwydwaith Cymdogaeth gofyn am ddau achos y mis o sut y mae’r gwasanaeth wedi gwella bywydau pobl. Mae’r naratif yn dangos bod y gwasanaeth yn symud i’r cyfeiriad iawn ac mae’n gallu cael ei ddefnyddio i werthuso’r prosiectau.

Ble nawr?

O ran ein gwerthusiad, rydyn ni wedi bod yn casglu enghreifftiau o sut mae’n gwaith ni wedi arwain at fudiadau’n mabwysiadu arfer da. Dyw’r rhain ddim yn aml yn fesurau ynddo’u hunain, ond maen nhw’n astudiaethau achos cymhleth o sut mae gwasanaethau wedi newid.

Ac o ran ein gwaith, mae’n bwysig bod ni’n parhau i drafod methiant, fel bod e’n cael ei normaleiddio a gall pobl fod yn fwy onest amdano. Ac os gallwn ni wneud hynny, byddwn ni mewn lle gwell i helpu mudiadau i gymryd camau pellach i wella eu gwasanaethau.

Failing to learn from failure

How can public services make use of learning and information that result from failure? Dyfrig Williams blogs on learning from failure.

Last week I attended the Learning from Failure workshop in Cardiff. Before I go any further, I should clarify that these reflections are very much on my own process of working, rather than the work of the Good Practice Exchange.

The event was a bit of an eye-opening session, as it gave us all as participants the scope to look at aspects of our work that are not traditionally discussed. But why not?

In her paper on ‘Strategies for learning from failure,’ Amy C. Edmondson shares the Spectrum of Failure, which shows that blameworthy failure rarely results from the actions of any one individual. So why do we still tend to think that an effective and productive workplace culture is one that shuns failure and casts blame at all costs?

Order, order!

https://twitter.com/PaulBromford/status/628876214623727617/photo/1

In the group exercise, each table designed an enabling environment for innovation, and each one was an environment where failure was accepted. Which makes sense, because we’re not encouraging innovation by cracking down on failure, we’re cracking down on ideas for new ways of working.

I was given the task of feeding back our table’s thoughts, which were based around the point that an enabling environment is complex and messy. What I personally meant by this is that a traditional approach, and the way that I’ve tended to approach innovation, is messy. My own innovation has often been a by-product, rather than an outcome or focus I’ve chased. It’s not been something that’s been identified in my appraisals, and it’s not been something that I’ve had to report back on to measure success.

This messiness is complicated even further by the environment that we’re working in. There is no one-size fits all solution for issues around public service access and delivery. What works in one community may not work in another.

However, there are approaches to innovation in the wider world that are much more rigorous. It was interesting to look at the work of the Bromford Lab, who have got a much more structured approach to innovation by testing ideas. One of their founding principles is ‘to fail fast,’ which means that they uncover lots of information and learning from their tests.

This planned approach makes it much easier to measure the success of innovation. Whilst there is no doubt innovation taking place at Bromford outside the realms of the Lab, taking issues and ideas to the Lab gives scope to evaluate them in a more formal way. This is at its most obvious with their Trello, where evaluation is planned and built in to their testing process.

Making space to evaluate

Making the time to evaluate impact has been the main learning point for me. I’ve tended to treat evaluation as something I do when I get the chance, rather than a process to embed into my work. And if you don’t take the time to properly look into why something’s failed, then there’s little chance to learn from it.

And so my more rigorous process of evaluation begins. The dates are now in the diary – I’ll be taking some time out from my day to day work and getting on with my evaluation. I’ll be leaving the physical space of my desk, which will help me get away from the usual distractions like email and the pile of paper on my desk, and get me in the right headspace to approach a different aspect of our work. Wish me luck!