Category Archives: Uncategorised

Sharing the learning of the Independent Review of the Impact of the Good Practice Exchange of the Wales Audit Office

The Good Practice Exchange has been in existence since late 2010/11. We set out what success would look like in 5 years, and we committed to an independent evaluation of what we were trying to achieve in 2016. So fast forward five years, and up popped the need to undertake an independent review.

So as with all things good practice, we (Bethan Smith and Ena Lloyd) would like to share the learning… 

What we do (fast forward to the next paragraph, if you know this bit)

GPX teamIn case you’re not familiar with our work, we promote improvement across public services in Wales through better knowledge exchange and shared learning. In our first year we were a team of two (Chris Bolton and Ena Lloyd), so we began with a modest programme, learning and reviewing as we went along.  We then expanded to a team of 4, with Dyfrig Williams and Bethan Smith (who took over from the very talented Tanwen) joining the team.  This meant we could now deliver a full programme of 20 events per year. As well as our programme of events, we provide support to various public service organisations in the form of a digital footprint (providing video content, blogging, social media etc. at their events), our Good Practice blog, Twitter, Pinterest and video content.

Independent Evaluation

The evaluation was undertaken by Professor Merali, from the Centre for Systems Studies at Hull University Business School. The review involved collecting data from:

  • Semi-structured interviews and conversations to capture narratives across a range of stakeholders inside and outside the Wales Audit Office
  • Attending Good Practice Exchange events
  • Reviewing the feedback from participants at our events
  • Reviewing responses of Chief Executives who participated in the recent Wales Audit Office Stakeholder Survey about all things related to the Good Practice Exchange

Views were captured from individuals from across public services, as well as a random mix of internal and external colleagues that had either attended, presented at, or shaped our events.

The review focused on three different areas:

  • Stakeholder perceptions about our role and our relationship with more mainstream audit functions
  • The value that we deliver through our information role and our support for learning and innovation
  • The way in which we achieve outcomes, and the implications that this has for sustainability and innovation

So what were the key messages?

Internal perceptions and relationships

blog picSupporting improvement was cited as one of the purposes of audit by all of the Wales Audit Office staff who were interviewed. The Good Practice Exchange was seen as a discrete part of the Wales Audit Office with a role that is related to, but distinct from the mainstream audit function.

Those in mainstream audit function felt that the Good Practice Exchange has a well-established presence in the Wales Audit Office; “…it seems to have always been there”.

While our role is perceived as being primarily an outward facing one, the Good Practice Exchange staff are proactive in developing connections and relationships with colleagues across the audit function.

External perceptions and relationships

The fact that Good Practice Exchange is an arm of the Wales Audit Office was highly valued by all the stakeholders who were interviewed, and there was a unanimous agreement that the Wales Audit Office “brand”;

  • Vested the Good Practice Exchange with authority, vouching for its impartiality and trustworthiness, and
  • “Gave credence” to speakers, information and ideas presented at Good Practice Exchange events.

Our support for learning and innovation is delivered in two ways; the event programme and the cumulative activity of Good Practice Exchange staff before, during and after events. This builds resources and capability to enable individuals to explore and exploit ideas for innovation and improvement.

“.. there is the thunderclap before the event… and a tide swell after each event whereby it builds on itself – information cascades through past and current attendees…”

We have developed a network of collaborators, contributors and “clients”, which has been key to our success. The analysis of stakeholder narratives showed that our success has been derived from our ability to incrementally generate, sustain and leverage networks and social, relational and reputational capital.

“… it is about individuals that are within that team- that drive, that energy, that thinking, that enthusiasm, that kind of passion for change and different thinking…and to be honest nobody is ever coming along and saying ‘this thing you are doing is wrong ‘. They are saying ‘have you seen there are different ways to do this and there are lots of opportunities out there, and you can pick any of them that you like.’ Nobody has ever said ‘that is wrong’…you can read into that and take from it whatever you wish. And I like that.  One size fits all is for me a disaster”

Conclusion

The report concluded that;

  • The Good Practice Exchange works well in its current form as a lean and agile unit of the Wales Audit Office
  • The Wales Audit Office brand is essential for our authority and credentials for impartiality and trustworthiness
  • Our events and activities are well-received and it is recognised as being an effective catalyst for change and improvement
  • Our modest size and our network mode of operation enable us to be agile and responsive
  • Our digital footprint and use of social media is effective in maintaining currency with its followers and collaborators
  • Over our lifetime of activity and engagement with diverse stakeholder constituencies we have accumulated a valuable and extensive network embodying social, relational and reputational capital

Going forward, our capabilities, resource base, reputation and positioning within the Wales Audit Office make the Good Practice Exchange well suited to support the Welsh public sector’s transition to models of service delivery that are aligned with the Wellbeing of Future Generations Act.

Next steps

As a team, we are really pleased with the outcome of the report. We encourage openness and transparency and felt it was important to share the outcome both with internal colleagues and our external networks. The report has helped us to focus on the areas that are working well, and what we need to work harder on to improve.

We are always keen to hear feedback from colleagues, both internal and external, and over the coming months we’ll be working on our evaluation processes to make sure we fully take into account the comments we receive post events. After all, public services in Wales are changing and evolving and we need to ensure we do the same to meet the changing needs.

So watch this space for next phase of the Good Practice Exchange. And most importantly, if you have attended any of our seminars or webinars, thank you for your time, contribution and feedback, good and bad, as that’s what shaped us.

Making savings and planning ahead

In the following blog, the Wales Audit Office’s Local Government Manager Jeremy Blog - JeremyEvans talks about how savings planning plays a vital role in supporting council financial resilience, following the release of the Auditor General for Wales’ report Savings Planning in Councils in Wales.

Effective savings planning is critical for the effective stewardship of public money and the delivery of efficient public services – in other words balancing the books whilst continuing to deliver quality services to the public.

Councils need to have a medium term financial plan, setting out how, at a high level, they will operate within the income that they receive, be that from Welsh Government or other sources such as council tax. This plan needs to look three to five years into the future. We found that all councils have these plans in place.

Making up the shortfalls

Having identified the shortfall in income – the gap between what they have and what they need – councils then need to identify how to bridge that gap over the life of the plan. As you would expect, current year plans will need to be very detailed, whilst those for two or three years away less so.

The better councils are at achieving their savings, the less pressure there is to find one-off funding streams to balance budgets. There is also less pressure on services to continue to drive out unachieved previous year savings at the same time as grappling with making those set for the current year. Not having to use underspends, reserves or other windfalls to balance the budget also means that they can be used in a more thought through way – potentially helping councils to fund initiatives that will bring financial benefits in the future.

What does success look like?

To be successful, savings plans need to be specific, measurable, achievable, realistic and time bound. We found that about half of councils have such plans in place.

Being clear how savings will be made is key to transparency. This way everyone understands what needs to happen and any concerns about the impact of the savings can be raised at an early stage.

Accurate savings value and realistic timescales are important. This ensures there is a clear benchmark against which services can be held to account and against which they can assess their progress, spotting any problems early.

As the financial pressures continue to bite, the ability of councils to just make across the board percentage cuts reduces.  Savings need to come from more fundamental changes to the way services are delivered or the way councils operate.  We found that these types of savings take longer to achieve as they are more complex and potentially higher risk.  With these transformational savings, there is a greater need to get the plan right.

Join us for an event

On 8 August, our Good Practice Exchange Team is holding a webinar on Building Financial Resilience in Public Services. The aim of the webinar is to share approaches to building financial resilience (including examples of good practice) and identifying the key barriers and how to overcome them.

The webinar is aimed at members and officers of public services in the following roles:

  • Heads of Service
  • Service/operational Managers for major operational delivery
  • Budget holders
  • Section 151 Officers and Finance Managers
  • Cabinet Members with budget and planning as part of their portfolio.

The webinar will be recorded and will be available on YouTube around 1-2 weeks after the live webinar on 8 August. This allows us to add English and Welsh subtitles.

You can register for the webinar on our website or by contacting a member of the Good Practice Team by email: good.practice@audit.wales.

Well managed risks: Context is everything

Darllenwch y flogbost yn Gymraeg

The Good Practice Exchange held a pilot seminar on how you manage risks around organisation change, service transformation and innovation. In this post, Chris Bolton looks at what the data tells us about how public services frame failure.

This is the first of a series blog posts that share some of the learning from our pilot seminar about Well Managed Risks. Here are the slides that we used on the day.

The first three blog posts are structured as following:

  • Post 1. Context is everything.
    This is a brief description of what we did in the session and some observations on how people think they would respond to failure in the context of different risk management approaches.
  • Post 2. Is common sense more useful than the rule book?
    This reviews the data we collected around how people use different approaches when they are making decisions about risk, and
  • Post 3. Does service user involvement in decision making lead to better decisions?
    This tested the technology we used and pushed our understanding to the limits.

There are of course huge caveats around the information presented here. It is very much work in progress, based upon an experiment we carried out in a shared learning seminar. We are grateful to everyone who took part for doing so willingly and allowing us to share the data (everything here is from people who ticked the box saying it was ok). This is very much ‘working out loud, and in the open for us’. If we’ve got anything wrong, please let us know. If you could do it in a supportive way, that would be far more helpful to us that a public flogging. Thank you.

Context is important

This session was developed to try and share knowledge around well managed risk taking. The Auditor General has been saying for some time that he wants to see public services taking well managed risks. You can look at this video where he talks about the importance of trying new things and learning from failure if they don’t work.

There is a however a gap between what the Auditor General has been saying and practice across public services. Nobody is suggesting taking un-necessary risks with services provided to vulnerable people or being reckless with public money, but there is probably some scope to move from the status quo.

Changing our approach

In the spirit of well managed risk taking we decided to do something different with this event. Usually we would have arranged something where people share practice and knowledge that others could learn from, as presentations or workshops. Around this structure we would facilitate conversations and introductions where people can develop relationships to continue their peer to peer knowledge exchange.

Whist this approach is effective, we decided to test an approach which was far more immersive and allowed people to think about situations and how they would respond to them. This scenario type work has been used in other situations, but we wanted to extend it by using a process that allowed people to record their thoughts and opinions in a way that could be analysed and fed back to them rapidly. The idea was that they could see how their attitudes to risk and decision making fit with those around them, and the context they are sitting in. This level of understanding might then support different behaviours and attitudes to well managed risk taking.

How the approach works

Very briefly, we did the following:

  1. Explained an approach to risk taking (Framework 1) to the group. These were adapted from existing approaches and chosen to be at opposite ends of what you might be likely to see in public services.
  2. Presented a scenario of a significant challenge facing public services.
  3. Asked people to discuss the scenario then, individually record their responses to a series of questions about; decision making, benefits/impact and attitudes to failure.
  4. This process was repeated for three scenarios and then in the context of the second approach to risk management (Framework 2).
  5. The approach is summarised in the graphic below.
  6. We used SenseMaker as the tool for people to record their thoughts, analyse their responses, and provide some live feedback. We’ve been working with The Cynefin Centre at Bangor University to get a better understanding of how this approach might be useful for our work.

A diagram explaining the structure of the seminar,where we gathered data from scenarios where different risk frameworks were used

An example of what we asked people to do

In response to Frameworks 1 and 2 and each of the 3 scenarios, we posed people the following question: ‘Transparent reporting of any failure will…’

The options in responding were two extremes; people get fired or people get promoted.

They were asked to move a marker along a sliding scale to a point which they thought reflected the position of the organisation, in response to the risk management frame work they had been presented.

A scale which people used to see whether reporting failure would get them fired or promoted

What the data told us

We collected a total of 218 separate responses to the question.

The graphic below presents a roughly normal distribution between the two options, which is what you might expect.

When you analyse this information to look at how people responded in the context of the two different frameworks things look different with two distinct patterns forming. Graphic 2 with responses in the context of Framework 1 closer towards the left hand side (people get fired) and pale blue responses (Framework 2), closer to the right hand side (people get promoted).

Framework 1 was, Failure is Not an Option. An approach that assumes all risks can be fully understood, assessed, categorised, documented and managed.

Framework 2 was, Safe to Fail. This approach rooted in the Complex Domain of the Cynefin Framework which proposed a number of small, time limited, low resource tests / pilots / experiments. Their objective is to probe what is happening and gain a better understanding before any decisions are made about what to do next.

Graphs 1 and 2, which are bell curves on whether people get fired or promoted

Further analysis emphasised this split in the data. Graphic 3 illustrates the distribution in the context of Framework 1 (Failure is not an option), with the mean closer the left hand side. Graphic 4 illustrates a distinct shift towards the right hand side and the ‘people get promoted’ label.

2 bell curve graphs on whether people get fired or promoted

So does this tell us anything?

The data suggests that how something is described or framed will influence how people respond to reporting of failure.

In the context of Framework 1 (Failure is not an option) people are more likely to think that reporting of failure will get people fired.

In the context of Framework 2 (Safe to fail) people are more likely to think that reporting of failure will get people promoted.

This might not be surprising when you sit back and rationally read about it in blog post. However as one of the delegates commented, “This has big implications for how we make decisions on our committees and Public Service Boards. If we talk about decsions in the context of failure is not an option people will be worried about the consequences, so will be less likely to be innovative and take risks. The language we choose to use and how we frame things is important”

What’s next?

This post will be followed by two more that look at:

  • Post 2. Is common sense more useful than the rule book? This reviews the data we collected around how people use different approaches when they are making decisions about risk, and
  • Post 3. Does service user involvement in decision making lead to better decisions? This tested the technology we used and pushed our understanding to the limits.

As mentioned earlier, this is an experiment for us and an example of us ‘working out loud, doing things in the open’. There is still a lot more we would like to do with this data. We are certain that we haven’t got things right and would appreciate any comments and feedback on what we have tried here. If anyone would like to have a look at the dataset and help expand our understanding, please get in touch, we would very much like to talk.

Public Services working in partnership for better health and wellbeing

SGBack in November, Sarah Wills, Gofal, delivered a workshop at our event ‘Public services working in partnership for better health and wellbeing’. In this blog, Sarah provides an oversight of who Gofal are, what they do and the positive developments since our event…

About Gofal

Gofal have a simple vision – ‘Good mental health and wellbeing for all’. Gofal work with:

  • People living with serious and enduring mental illness who face the most significant challenges in achieving and maintaining independent lives within communities
  • People experiencing mild to moderate mental health problems that impact on their ability to achieve and maintain healthy fulfilled lives within communities
  • The public, employers, groups, other charities and the media to improve mental health awareness and promote whole population mental health and wellbeing
  • Politicians, Government officials and health and social care professionals to inform and  improve  legislation, policy and practice

Today Gofal support over two-thousand people a year; our recovery model provides a strong evidence based framework to operate within and we are able to clearly evidence the positive difference our services make to people’s lives. Our regular consultation exercises mean that we know exactly what matters to people who use our services and this drives everything we do.  We work in thirteen Local Authorities and five Health Board areas.

We firmly believe that we will achieve more by working with others. We have worked hard to develop strong and constructive relationships locally, regionally and nationally. We work closely with statutory, third and private sector colleagues in pursuit of Gofal’s vision.

My Role

As Head of Service for the Central Region I have responsibility for overseeing the operational and strategic management of services across Cardiff and the Vale, Rhondda Cynon Taff and Merthyr Tydfil.  Part of this has included overseeing the development and running of the Mental Health Dispersed Supported Housing Scheme in the Vale of Glamorgan.  The service was evaluated by Welsh Government after being highlighted as a model of good practice and for its innovative approach as it is a partnership between Gofal, Newydd Housing Association, Cardiff & Vale University Health Board and the Vale of Glamorgan.

The Event

Following the evaluation we were invited to host a workshop at the WAO’s Good Practice Event on Partnership approaches to service delivery for better outcomes in North and South Wales, and were also asked to take part in the plenary panel discussion on partnership working.  We delivered the workshops in North and South Wales in partnership with the Vale of Glamorgan and Newydd.

What’s Happened Since?

The Vale Dispersed Scheme has since expanded to an additional unit, taking it to 7 properties and is likely to grow by a further unit to meet increasing demand.

Following the event I was contacted by the Regional Development Manager for Supported Housing for Betsi Cadwaldr UHB, they were really impressed with the partnership working and successful outcomes achieved on our Mental Health Dispersed Supported Housing Scheme.  They visited in early February and we established a shared ethos and approach to service delivery.  We discussed ways in which our approach could be adopted in North Wales to better meet service user’s needs but also other ways in which the UHB could work with Gofal.

We were also asked to give the presentation to the Blaenau Gwent Independent Living Strategy Forum, with representatives from Supporting People, Social Services and Health.  They were very interested in learning more about our approach and have continued their discussions in relation to how the model could be replicated there.

We have successfully submitted a bid for a replica Dispersed Scheme in Merthyr on the back of our successes in the Vale of Glamorgan, working in partnership with the Community Mental Health Team, Social Services and Merthyr Tydfil Housing Association.  This is in the process of being developed now and is already generating a lot of excitement; especially about the opportunities it offers for step-down from higher level supported housing, more independence and a better quality of life for individuals as well as cost savings to Health and Social Services budgets.

We have also given a presentation to the Supporting People National Advisory Board on the Vale Dispersed Scheme, which has allowed the project and approach to have further exposure at a national policy level.  There was lots of discussion among the members of the board about how we ensure this type of partnership working is embedded at a national policy level.

Plans for the Future

We’ve begun discussions about arranging a trip to North Wales to meet with Betsi Cadwaldr UHB again alongside some of the Local Authority leads to see what can be developed there.

We’ve continued to receive interest from other Local Authorities about the scheme and our approach, for example, Monmouthshire Supporting People have recently contacted us to arrange a visit to the project which will take place in April.

We are continuing to make progress with establishing the new Dispersed Scheme in Merthyr; the first four individuals have been identified and we are just in the process of carrying out joint needs assessments with them as well as working with Merthyr Tydfil Housing Association to identify suitable properties.

Find out more about Gofal via their website.

The Good Practice Exchange work programme: What’s it all about?

Darllenwch y flogbost yn Gymraeg

Over the past few years the Good Practice Team in the Wales Audit Office have held a series of seminars and webinars to support public service reform. Ena Lloyd and Bethan Smith look at our programme of events for this year.

Ffotograff o Jess Hoare yn cymryd rhan yn nhrafodaeth panel Caerdydd

The plenary session at last year’s Digital Shared Learning Seminar

The Wales Audit Office created the Good Practice Team to bring together ideas and approaches to help public services improve. When we first started out, we used to get some quizzical looks! I guess it’s not something you would naturally associate with an Audit Office. But then, not every Audit Office has an Auditor General who feels so passionate about wanting to help public services improve. Huw Vaughan Thomas is one of a kind. He gives us a ‘safe to fail’ space to research, engage, learn from others and share knowledge, ideas and approaches in a variety of ways. Whether it be a seminar, webinar, blog, videos, twitter, or good old emails!

You can’t help but want to go the extra mile when you are given such trust and space, and why wouldn’t you.

What we have learnt over the past few years, is our Good Practice mantra of:

  • We don’t advocate a one size fits all approach;
  • Equally we don’t believe in re-inventing the wheel;
  • We believe in adapting not adopting; and using our very privileged position in the Wales Audit Office to bring together colleagues from right across the public, third sector and where appropriate the private sector.

How does the programme get pulled together?

We often get asked how we arrive at the topics in our programme. Our ‘starters for ten’ is our Wales Audit Office Strategic Plan in terms of our key priorities. So you won’t be surprised to see such topics as Digital, the Wellbeing of Future Generations Act and Early Closure of Local Government Accounts included as part of the programme. The topics also have to work across public services and where we can, work in partnership with at least one other organisation. The more partners, the better. We then take soundings from our internal colleagues as well as many people who represent, design and deliver public services in Wales. In fact, the list of people who we chat to virtually or face to face gets longer every year! We meet some great speakers and delegates who just totally blow us away in terms of what they are doing. If you have any ideas about topics you’d like to see in our programme, please do get in touch!

What’s this year’s programme about?

In this year’s programme, if there was one theme that underpins the majority of events, it’s the Wellbeing of the Future Generation Act.

Here’s the programme overview. We have used working titles to give you a flavour of what the seminar is about. However, once we have worked with partners to determine what the focus is, the finalised details can be found here.

The bottom line though, our litmus test so to speak, is what public services colleagues think. At the end of every seminar, we ask for 5 minutes of delegate’s time to complete a ‘Call to Action’ form. Over 1000 delegates attend our events over a period of a year, so it’s a rich source of feedback. We always stress how important it is to us for delegates to complete these forms, not only does it provide us with feedback on the event, but what actions delegates will be taking away and what they’d like to see taken forward by us or other organisations. The feedback we receive also helps shape our programme.

Our events are completely free of charge to all public and third sector organisations in Wales. All we ask in return is that you come to our events armed with ideas, solutions and any issues or challenges in relation to the topic of the event. Our events are called ‘shared learning seminars’ which speaks for itself – we really want delegates to share and learn as much as they can, and take away as much useful information from the event as possible.

For those that have been to our events before, I’m sure they’ll say it’s a packed morning, and we make no apology for that. Our events are purposely designed to equip delegates with as much information and contacts as possible, in order for them to continue conversations after the event.

Every year we seem to have an increasing demand on our events which is brilliant, it means public services are really keen to work together and share ideas and approaches. We have a small budget for our events and whilst they’re free to public services, it is so important to let us know if you can no longer attend before the day of the event as we often have a reserve list for events. We understand work pressures take priority but we’d really appreciate advance notice so we can re-allocate your place to someone else.

Other than events, how else do you share information?

We understand the importance of sharing information in a variety of ways. Whilst seminars might work for one person, videos or blogs work better for another person. With that in mind, our various information channels are listed below:

We share details of seminars/webinars on our website– so keep a look out for them – or if you want to us to add your details onto our mailing lists to receive info for all or some events, then please email good.practice@audit.wales.

%d bloggers like this: