Client again Jazz North

We are delighted to be working with Jazz North again. Back in 2018 we worked together evaluating their 'northern line' programme which was for unsigned new jazz artists and groups. At the time we recommended a back to basics focus, teaching bands how to run the business side of jazz. We enjoyed our time together.

Out of the blue, Jazz North called us in March explaining they had a new project and they wanted to understand the impact in a bit more depth. This is Jazz Camp for Girls. Working with young women, aged as young as 8 years old, Jazz North is trying hard to address gender stereotypes and barriers.

We are delighted to be the evaluation partner on the programme, and look forward to understanding how we can stop jazz being a man's world!

You can see more about Jazz North here. Jazz North is an Arts Council funded National Portfolio Organisation and is the jazz development agency for the north of England.

How are we coping with Coronavirus? 21 statistics around our health and wellbeing from the first week of lockdown. Now updated to include week 2 and week 3.

The world is changing. Here in the UK we are being asked to stay at home, to work from home, and to minimise our movement and travel down to the necessities. We are being asked this to ease pressure on the NHS, to stop the spread of the virus and to save lives.

It's a strange new world, and we wanted to see if we could understand how people are coping. Well, as you all know, we measure wellbeing regularly. So we did a small survey in our local area, Barnoldswick and nearby to see how people are doing.

Week 2 includes a few more further flung people, as people have shared the survey to friends and family across the UK. This was updated on Friday 3rd April and is shown in blue.  

On Friday 10th April, we added the Week 3 results in and they are shown in purple. 

On Friday 8th May we carried out Week 7 analysis, this is shown in green. 

Here are the results.

• 16% of everyone have been asked to stay at home for 12 weeks, 84% have not
• Week 2: 12% of everyone have been asked to stay at home for 12 weeks, 88% have not
• Week 3: 21% of everyone have been asked to stay at home for 12 weeks, 79% have not
• Week 7: 10% of everyone have been asked to stay at home for 12 weeks, 90% have not

• 5% work in the NHS, and a further 11% had close friends or family who worked for the NHS
• Week 2: 5% work in the NHS, and a further 10% had close friends or family who worked for the NHS
• Week 3: 5% work in the NHS, and a further 13% had close friends or family who worked for the NHS
• Week 7 : 8% work in the NHS, and a further 10% had close friends or family who worked for the NHS

• 23% were considered key workers and continued to work, and a further 45% had close friends and family members who were continuing to work
• Week 2: 23% were considered key workers and continued to work, and a further 38% had close friends and family members who were continuing to work. This could mean more people are off work ill at the moment. 
• Week 3: 20% were considered key workers and continued to work, and a further 36% had close friends and family members who were continuing to work. This probably does mean more people are off work ill at the moment. 
• Week 7: 33% were considered key workers and continued to work, and a further 37% had close friends and family members who were continuing to work. 

We asked the statement; how anxious do you feel for yourself and your family?

The average score was 6, but there was a huge range of answers here. 20% were very concerned, scoring 10/10 for anxiety.

18 reasons to love Winter; a creative evaluation of wellbeing

72 seasons is a year-long research project, designed by Kirsty Rose Parker (founder & director of The Evaluator), to measure how being more connected to nature makes us feel. We do that through working with a team of volunteer seasonal seekers. We have planned a whole year - 2020 - where the seasons change every 4 or 5 days, originally inspired by the ancient natural calendar in Japan.

Our seasonal seekers agree to complete health and wellbeing research surveys and then they start their adventure. Trying to notice the changes in nature, we are building a community around Pendle Hill who look a little bit closer, a little bit more often, even just from their gardens and windows as the world changes. We had never heard of Coronavirus when this project began on 1st January 2020; but despite the changes in our daily lives and massive upheavals in our wellbeing, nature soldiers on, and so do we.

Here we share the results of the nature we have spotted. 180 people began this journey with us, a few have left and a few have since joined. We are a community that ebbs and flows, and people can choose how involved they get.

In 2020, we changed the season 'Winter' into 18 smaller seasons and asked our seasonal seekers to go out and about as much as they normally would, and see what they spotted.

The beautiful seasonal illustrations are by local artist, Cath Ford. You can check more of her work out here. Cath lives in Blackburn and she knows the nature we know. She is a very talented artist and we feel very lucky to be working with her.

Season; 1 - 4 January: The Earth is Unyielding 

Recommissioned for Positive Progressions

We announce new partnerships here, and share some evaluation examples, but we rarely share the details of the project we complete. It's time to rectify this.

Positive Progressions is one of our earliest clients and we are still working together. Positive Progressions is run by Craven College, who are based in Yorkshire and is all about getting families who are far away from the labour market back into work. They are co-financed by the National Lottery Community Fund and the European Social Fund and are a Building Better Opportunities (BBO) project.

As their evaluation provider, in the beginning we worked with them to understand the work they were doing and what happened day to day in their project. We looked at the data they had to collect for funders, and spoke to the staff.  The project itself is a 1:1 keyworker support model, which is incredibly flexible. Because staff work so intensively with participants, they get to know them really well. We use that, and get the staff to carry out keyworker assessments.

Overall we have created measurement tools which capture changes in; wellbeing, confidence, resilience and ‘work-readiness’ over time. We chose this list, as these are the aims of the project. We regularly analyse all the data and present it to the wider team at partnership meetings. We can show the project is making an impact. We even can predict when people are nearing the end of their time in the project.

Our numerical information has uncovered unknown issues, for example, that people with disabilities are less likely to cross the final hurdle and get a job. This has helped the project team to focus their time and understanding on supporting disabled participants more.

We have provided bespoke quantitative measurement tools including ‘The Journey Back to Work Road Map’ and the ‘Rose Parker Resilience Scale’ which are very quick and easy for the team to implement and require no additional staff time.

Positive Progressions have consistently been identified as one of the leading examples of good practice in evaluation for all BBO projects across the North of England.

We presented the bespoke findings for this project at a high-profile seminar of good evaluation practice at Newcastle Football Club in May 2019. We were the only project selected to share evaluation good practice!

In September 2019 we were recommissioned to continue to provide their evaluation for the next two years. Working together again, we uncovered what they wanted out of a future evaluation. We decided to continue to use the existing methods and monitor the disability situation and to develop a new arm for evaluation to measure longer term impact. We are currently carrying out a ‘six months later’ impact analysis for this project and as part of that work we are starting to financially proxy the lifetime savings of the project in terms of a public purse.

“The Evaluator has been integral; supporting Craven College to really demonstrate  the impact of Positive Progressions to funders both current and future. From the outset they have developed and adapted methods of collecting data and evidencing impact throughout, this has ensured we can really showcase the success of the project but also to help us constantly improve our practice.” Pippa Syers, Community and Projects Manager.

Hopefully, that will have explained a bit more about what an evaluation partnership can look like. As a firm trying to transform evaluation; we are really trying to make sure it enhances our clients work, and helps them to make data-driven-decisions.

Spring in the 72 Seasons

Would you like to come on an adventure in nature with us? One which can all be done from home?

If the answer is yes, please sign up for Spring in our 72 Seasons project, where we measure your changes in health and wellbeing, while exploring the changes in nature.

In this project there is a new season every 4 or 5 days and we ask you to try and notice the new season. You will be asked to complete health and wellbeing surveys at the beginning and end and short reports as to what you have noticed. In total it takes about 30 minutes of your time over the whole season.

Our current Winter seasonal seekers have told us they feel closer to nature and it's made them look more.

The world is chaotic and overwhelming right now, and many people are stuck at home and anxious. We get that. But nature doesn't. Why not see if this project can help you to get through difficult times?

Sign up here http://eepurl.com/gPQTyj

This project is part of the Pendle Hill Landscape Partnership series of projects. You can see further details here. 

Coronavirus; Our Response

All of The Evaluator staff are working from home in response to the Coronavirus. Our team is normally home based, but now we are not taking any face to face appointments or travelling to clients. We are having video calls, and phone calls, and emailing more people.

Some of our clients have had to cancel all their work, some are busy trying to work out if they can amend any of their programmes. Some are asking us to do more work for them. We are 'figuring it out' day by day at the moment and trying to be as flexible and helpful as possible.

We are still considering ways in which we can be of help. People across the world are feeling anxious; about themselves, about health, about food shortages, about money, about work, about life as we know it. Our wellbeing is going to be changing all the time, and maybe we should be measuring that?

None of us know what is coming next, so all we can say is, if you are reading this...

Stay safe & stay sane.

Recommissioned for Stronger Families

We are delighted to announce our policy work with Leeds City Council has been recommissioned.

It's always lovely to know that people like your work and want to work with you again. Here at The Evaluator we really do pride ourselves on working hard for current clients. Did you know we are still working with 3 of our original 4 clients? It would probably be 4 out of 4 if a member of staff had not left. Even so, that 75% is something we are really proud of. We know who helped us get started, and we are loyal to our clients. We want to build relationships, and we want them to be long ones. We are trying to transform evaluation services, and one of those ways, is by being so great to work with, people automatically include evaluation, and automatically ask us what we can do for them.

Leeds City Council is the partnership lead for Stronger Families which is a Building Better Opportunities project. These projects are co-financed by the National Lottery Community Fund and the European Social Fund. One of the requirements of the funding is to produce measure-able policies on Cross-Cutting Themes. In this case the policies need to include numbers and show progress towards agreed targets for both Sustainability and Equality.

The Evaluator will again be monitoring and measuring these policies for Stronger Families. We are excited to work together again.

New Branding Rules!

Did you notice our new branding? It's a ruler! Isn't that just perfect for us, seeing as we spend all day every day measuring things!

We have been enjoying using it in lots of different ways recently.

Updating client feedback...

We do work hard you know. It's so nice that others notice and mention it. 

We even updated our own measurements... 

Yay, very happy with that one! 

Think here is a good time to shout out to Holik Creative who did the logo design, concept and branding for us. 

They were a dream to work with. Local too. Heartily recommended, give them a shout. 

Working with the National Festival of Making 2020

We are delighted to announce we are working with The National Festival of Making again in 2020.

The National Festival of Making is an award-winning festival with national renown. In 2019 The Evaluator joined the project team to provide quick solutions to capture audience experience in a fast moving, multi-activity, street and venue-based festival.

The festival team were delighted with their final evaluation report explaining “we are using the evaluation so much and receiving incredible comments about the robustness of the data! Thank you so much!” Lauren Zawadzki, Director, Deco Publique

Not only are they lovely clients to work with, they wanted to know what was possible and actually asked us to give them our dream evaluation brief for the festival this year!

You can see more about the festival here. 

Update: March 2020: Sadly the festival has been postphoned due to the Coronavirus outbreak. A full statement is available on the website link above.

Thanks to all our 2019 Clients

It's that time of year, when you spend a minute looking back.

What a year we have had! Thanks to all our clients. We can't wait to see who joins the list for 2020. We will probably have to make a two page image next year.

The 'Eagle-Eyed' amongst you may also notice how many clients have stayed with us for another year. We still have our very first two clients! That makes us feel really proud, you absolutely know you are doing a good  job when people keep coming back.

Christmas Opening

Evaluating Wellbeing

Wellbeing is included in almost every evaluation we complete. It is such an important part of life, and is about being comfortable - whether that is with your emotional health, your physical health, your everyday lifestyle, or just the situation or place you are in.

Comfortable is a really important word here. Did you know that it’s in the official definition of Wellbeing?

Evaluation definition, Wellbeing:

 “the feeling of being comfortable, healthy or happy”. 

We use many different evaluation methods for measuring Wellbeing, but the ones we use the most are probably the Warwick-Edinburgh Scale of Wellbeing, or one that relates to the Five Ways to Wellbeing. This was originally a piece of research carried out by the New Economics Foundation (NEF) in 2007. It's very easy to remember and it feels right, which is when you know it is right. 

Making new friends (or just socialising in general), feeling useful (as though you are giving something back to the world or society), noticing more (which could be the beauty of nature, or what you are grateful for), actually being active (regardless of whether that is a gentle walk, some swimming, or a hardcore gym session), and keeping learning (new skills, knowledge or facts) do make all of us feel better in ourselves.

It is worth thinking about how your project or activity is impacting on the five ways or, of course, we could do that for you. Measuring changes in these will often be a great way to demonstrate impact.

We are hiring!

Freelance opportunities to come and work with us...

The Evaluator is a fast-growing evaluation consultancy, founded in 2017 by Kirsty Rose Parker, helping charities and not-for-profit organisations to demonstrate the impact of their work creatively and visually. The Evaluator uses the tagline “We’ll figure it out for you”.

Ultimately, we are often ‘measuring the unmeasurable’ and fixing problems on a day-to-day basis. The tasks included in this work range from data capture, face-to-face consultations, telephone calls, research, analysis, report-writing, visual data creation, presenting information, and creative problem solving.

As the business is growing fast we need more freelance support, and are looking for people who have some of the following skills. Please note, we don’t need you to be able to do everything and we do really value honesty, so if one or more skills is not your ‘bag’, tell us!

We want people who:

• Love numbers and have qualifications in maths, and ideally statistics. Do you have an analytical mind? Do you love a spreadsheet?
• Love language and are a wordsmith. Can you write brilliantly? Concisely? Do you have a great style?
• Are creative. Tell us what you love to do. Do you like colour, and art and design?
• Care about making a difference to people’s lives. Do you volunteer for a charity? Have you worked in the not-for-profit sector before?
• Have amazing people skills. When was the last time a complete stranger told you their life story?
• Communicate brilliantly. Are you great at public speaking? Can you facilitate a conversation? Carry out staff training?
• Are honest, reliable, punctual, willing to learn, and enthusiastic.

In addition, although not essential:

• Have you experience of evaluations at all? Either carrying them out or simply taking part? Maybe you have carried out surveys and research in the past?
• Have you project management skills? Can you plan something out and make sure it is all done in a logical and efficient way?
• Have you research skills? Tell us about them.
• Can you think and write strategically?

In return, The Evaluator can offer an exciting and dynamic work opportunity - every day is different, and we are often making a difference to people’s lives. This is work that means something.

We are looking for freelance individuals who are happy to work from home, able to travel to East Lancashire occasionally for meetings, and happy to travel around as part of the work undertaken. The majority of the work is currently in the North West and Yorkshire regions, but may grow. We want to build a relationship with our freelancers, you will be the face of The Evaluator and will need to be professional at all times. We are happy to negotiate a fixed amount of work if preferred.

Rate of Pay

Rate of Pay: £175 per day, or £25 per hour. You must be responsible for your own tax and insurance, and prove this with a UTR number, have access to a car which is insured for business use and be willing to travel (please note travel costs are not paid extra), and have appropriate professional indemnity and public liability insurance.

How to apply

To apply; please send a two-page summary addressing the skills we are after, and a CV to: kirsty@theevaluator.co.uk by 5pm on Monday 16^th December 2019. 

The Evaluator working with British Gymnastics Foundation

The Evaluator is delighted to announce a new partnership, working with British Gymnastics Foundation on their ground-breaking dementia programme, Love to Move, which is funded by Sport England.

Dementia steals loved ones, lives, and memories - but it doesn’t have to be this way….

The Love to Move programme is an age- and dementia-friendly seated gymnastics programme, which is transforming the lives of people living with dementia.

It's even been featured on BBC Breakfast! You can see the video here. 

The Evaluator is initially working with British Gymnastics Foundation to carry out a Social Return on Investment for the programme. We will be designing new methods and tools to evaluate experiences of those taking part - whether that is those with dementia who are participating, staff in some care homes who are looking after dementia sufferers, or families and unpaid carers who are observing changes in their loved ones.

The chair-based exercise programme makes the two halves of the brain work together in innovative ways, and The Evaluator is excited to use some of the techniques developed here to collect information. Hopefully, The Evaluator's work will also break some ground!

You can find out more about the project here.

Designing evaluations for children and young people

As evaluation consultants, we are always designing new evaluation methods and makings sure our evaluation tools are fit for purpose.

Evaluating the experiences of children and young people is not exactly the same as evaluating adults. Questions for children and young people need to be short and very simple. You have to make the questions five to seven words long.

Let's look at an example. Sometimes we are evaluating physical health, and whether people eat healthily.

An adult question might look like;

In the last two weeks, on how many days did you exercise?
Answer options; None/ 1-3/ 4-6/ 7-9/ 10-13/ Everyday

It's not a complicated question, but this would be quite difficult for children aged 6-9 years old.

We can change this to:

It won't give us as detailed information, but there is no point in collecting information if the children and young people guess an answer or leave it blank.

We use similar questions for healthy eating:

As you can see the language is simple, the answers are simple, and we have used faces to illustrate the answer.

Then, and this is the most important part, test the question on some actual children and young people. We checked these questions with around twelve different children, ranging in ages from 5 to 9 years old. Once we knew all these children understood the questions and could answer them, we then tested it with the project we were actually evaluating.

It worked really well and we are now collecting useful data from lots of different children and young people who take part in a project with one of our clients.

To summarise: keep evaluation simple, keep it short, and make it as visual as you can if you are working with children and young people.

The Evaluator working with Ripon YMCA

The Evaluator is delighted to announce a new partnership with Ripon YMCA, helping them to measure the impact on their young people which comes from having a secure home.

Ripon YMCA has space for young people who have nowhere to live, they offer supported housing to some of the most vulnerable young people in the area; care leavers, those who have suffered family breakdown, people with mental health problems, addictions and anger among other tribulations.

The Evaluator is going to be working closely with the organisation to help them refine their data collection systems; so they accurately can understand their impact and communicate that quickly and clearly to funders and donors.

Measuring the impact of feeling safe, is going to be really important in this project. The Evaluator is going to be running a workshop with the young people who live there too, to make sure their voices are included; it's a participatory project.

You can see more about Ripon YMCA here.

You can't win them all!

We announced a while ago that our director, Kirsty Rose Parker, was a finalist in the EVA awards 2019.

Sadly, she didn't win. The winner in her category was an amazing photographer; with national photography prizes under her belt and a London exhibition of her work, alongside articles in national newspapers.

"She also had some really great photographs of cows and how can you compete with that!" Kirsty explained.

However, Kirsty had a great night.

"Networking with so many inspirational women was fantastic. Just on my table was a lovely woman who had set up a charity, called Spoons, which supports families who have premature babies in hospitals. Kirsten, the founder, had a son who was born at 24 weeks and survived and is healthy and her experience of spending months with a baby in NICU had inspired her to help others who are in the same boat" Kirsty explained.

You can read more about the charity here - and Spoons was lucky enough to receive £1000 worth of donations on the night! How wonderful was that?

Also on Kirsty's table was the winner of the lifetime achievement inspirational woman; Susanna Lawson, who set up OneFile Ltd which is an online portfolio for apprentices, and which will support 1 million learners by 2020. You can read more about OneFile here.

Kirsty says "Susanna really was amazing, a very impressive businesswoman, I can see why she won such a fantastic award"

Overall, a good night was had by all!

Kirsty concludes; "to be a finalist - one of the last 45 women from over 1000 applicants/ nominees is still a pretty amazing feeling."

Evaluation Tips Series;

This is part three in our evaluation tips series, you can see part one here and part two here.

Once you have decided on your demographics, it's time to look at the specifics of what it is you are trying to change or measure.

Are you trying to make people more connected? Less lonely? Then you can keep it simple and ask people how often they go out, repeat this at the end and see if there is an average change – or an individual change. Or you could ask do you have enough friends?

If your project or activity will have an impact on people's emotions and well-being, the Warwick Edinburgh Scale of Wellbeing is a really useful tool. You can see more about this here. 

A number of projects recently have been helping people back to employment. You can simply measure if people do get a job or go back into training or into volunteering. Often, though, this misses the majority of the journey. People who have never worked, might take a long time and need a lot of support to get back to employment. They might not get a job during the majority of the project, but you have made a difference in their live. In order to show this effectively, we have created a journey back to work road-map. It shows all the steps people may take on their journey from not working, to building confidence and skills.

You might want to measure if people had fun, if they moved more or were more physically active. Did they visit the doctor less or use less medication or support services? Did you change their attitudes or did they learn something new? For example, in a recent project about suffragettes we asked people to name any prominent suffragettes at the beginning and then again at the end and showed how many more people could talk about on a graph.

You might want to understand people's motivation and it can be helpful to find out why people wanted to be involved, and what attracted them. If you ask enough people too, what they think might motivate others, or what might make a better project, you can use this information to improve your services. These kind of questions do need to be specific, you can’t just ask, ‘what motivates you to do this?’ It's much better to ask something like ‘Why do you think people volunteer?’, and give five answers to choose from and an ‘other’ option. 

Some good questions to ask include:

• Did you enjoy yourself? 
• Would you recommend this to a friend? 
• How often were you active in the last two weeks? (Ask at the start and again at the end and see what the change is).
• How often did you visit the GP in the last three months? (Ask at the start and again at the end and see what the change is). 
• Are you content with your employment status? 
• Did you make friends? 
• Do you leave the house as much as you want to? 
• Does your health stop you from doing everything you want to? 

Don’t forget the staff! 

Staff, Volunteers, Artists and Providers can all be a useful source of information. You could get staff to assess people at the start and again at the end or you could ask staff what they thought worked well and what would they change?

Example; questions about marketing:

• How did you hear about the project / activity?
• Did you tell a friend?
• Did you mention this on social media?
• How should we share this?
• Where should we send a press release?
• Or ask staff: What marketing method was most successful?

These are just a sample of the kind of questions you might want to ask. These are questions which are mostly used with adults. We will talk about children and young people in another article. 

If you are in doubt; or feeling unsure about what to ask... here are some core questions. Asking just these will give you a really strong basic evaluation.

• What did you enjoy the most? 
• Or Staff: What worked best?
• What did you enjoy the least?
• Or Staff: What worked the least?
• What could we do better?
• Or Staff: What do you think could have gone better?
• Would you recommend to a friend? 
• Or Staff: Would you run this again?

Above all, keep it simple. People are more likely to understand simple questions and that makes them more likely to answer. We will talk about ways to collect information in part four. Impact is often all about ‘what changed’  so don't forget to ask something at the start, repeat it at the end and see what the change is.

The Evaluator working with Stronger Communities

The Evaluator is delighted to announce a new partnership, working with Stronger Communities across parts of North Yorkshire on their new approach to working with rather than for communities; ABCD.

Asset Based Community Development (ABCD) is a way of encouraging people to think about the strengths that they have already got in their lives, their communities and their neighbourhoods and celebrate the ways in which people already create positive change. ABCD is all about connection. Connecting people. Connecting passions. Connecting ideas.

With that firmly in mind, The Evaluator is working on a participatory evaluation, using the skills of the community and working together to explore what is possible in terms of measurement. Initially working on surveys, designed together, to measure hopes and concerns and understanding in the team itself, The Evaluator is really getting to explore the creative side of evaluation in this project.

Ideas currently include drawing maps, asking people what are good questions and designing a workshop to identify what success means to people who are connecting.

There is a lot of common sense in the ABCD approach, which you can find out more about here.

The Evaluator loves a challenge, and this is one. Rethinking every bit of language is just the start!

The Evaluator working with Community First Yorkshire

The Evaluator has just completed a short term piece of work with Community First Yorkshire. Community First Yorkshire provides practical support and rural advocacy to voluntary and community organisations, parish councils and social enterprises. They run networks, training and projects.

The Evaluator recently carried out a series of twenty case studies; from larger more established organisations like the YMCA right through to small volunteer led organisations. Staff from The Evaluator have spent time in the sunshine, travelling around North Yorkshire, interviewing community groups in stunning locations. What was equally impressive was the range of work going on - from new build parish halls, to reinvented churches, to gardens full of volunteers, professional factories employing adults with learning disabilities and projects working to reduce homelessness in young people, to help men open up about their emotions and to empower all the volunteers and trustees who lead the organisations.

Kirsty Rose Parker, our founder and director explains, "listening to the stories of the charities we work with is, without a doubt, one of the most emotional and moving parts of the business. It really helps to remind us of how important their work is, and ultimately how important our work is, making sure they can tell their impact effectively and making sure they can continue to deliver the life changing activity they do".

You can read more about Community First Yorkshire here