Can evaluation be experiential?

This workshop was presented at the ICEL Conference, Sydney (2008).

Evaluation is becoming a core area of skills development and interest across organisations in the public sector, the voluntary NGO sector and the community sectors. Government agencies and funding organisations are keen to learn more clearly than ever about the outcomes and impact that their initiatives, projects and investment have on key policy areas and target groups. Essentially they want to know, quite reasonably: what difference does it make? But there is also a growing recognition in the value of learning from evaluation. This shift takes us more strongly into the softer and qualitative territory of capturing experiences, understanding change and generating learning through evaluation. My desire is to develop approaches to evaluation that validate and recognise experiences in ways that are relevant for individuals, for the organisations and for funders of programmes and initiatives. This is a perspective of evaluation which involves an exploration of the meaning and value of individual and group experience. Where learning, discovery and development are a priority and where the roles of the evaluator and the evaluated become intertwined and mutually recognised for what they add to the evaluation mix.

In this workshop I am interested to explore a notion of evaluation which recognises experience at a number of different levels:

• Experiences that are the focus of evaluation;
• Experiences, skills and knowledge that are developed by participants through the evaluation process, and;
• Experiences, skills and knowledge that the evaluator adds to the evaluation mix.

This workshop is designed as a peer inquiry to explore ideas on an experiential approach to evaluation.
It will cover:

• Case study examples of an experiential evaluation approach in practice:
• The Community Researcher Approach
• The Added Value Approach
• The Co-Creative Approach
• Theories that inform an experiential evaluation approach
• An invitation to participants’ to share their examples of experiential evaluation
• What are the key features of an ‘experiential’ approach to evaluation?
• What questions and issues does this approach raise?

Welcome and introduction

Aims and purposes of workshop: intro proposition from paper, sets the scene
This is a peer inquiry workshop; it will involve sharing experiences; dialogue; analysis and discussion.
Supporting paper – to be added to and developed through the workshop

Lead-in activity – Perspectives on evaluation (Horseshoe line on floor)

Based on a spectrum/dimension of evaluation as either quantitative or qualitative, individuals position themselves on the horseshoe in answer to the following questions: How does the world out there commonly view the meaning of evaluation?
How do you generally view the meaning of evaluation?
Discuss with the person next to you: What was your movement and why?
Short sample feedback: what do you notice? What does this tell us?
In your practice, where do you stand when you do evaluation?
Discuss with someone else near to you: What was your movement this time? Are you there from choice or because of your stakeholder requirements?

Presentation

My proposition in this workshop is the possibility of another dimension which pays attention to different aspects.
Paying attention to the experiences of the researchers, the participants and the organisation
My starting point – examples in practice.

The Community Researcher Approach   The Transformation Team, works alongside local Faith Communities as they work together to make a difference in the economically poorest communities in Glasgow. They support individuals and communities through a range of services including: community audit, training, accessing funding and developing volunteers. They work strategically to influence and connect policy and decision making with practice within communities. The Transformation Team carry out an annual survey and consultation with their stakeholders to get feedback on their services and to learn more about the difference their work makes for the individuals and communities they work with. A challenge for the Transformation Team was to find meaningful ways to engage with their stakeholders. They were also keen to use the evaluation process to increase the skills of the people they work with. The evaluation approach the Transformation Team chose to develop was a Community Researcher Approach where they recruited, trained and supported a small team of volunteers from the individuals they work with. This group became volunteer co-researchers in the Transformation Team stakeholder survey. This involved them learning about: the co-researcher role and its boundaries; developing and practising their interview skills; creating and piloting questions. Together they helped to design the questionnaire that was used, carried out a series of interviews, and contributed themselves within a focus group. The overall questions that they have helped to explore were: - The impact the Transformation Team had on the ability of faith communities to reduce local poverty? And, what’s faith got to do with it? They came back together to review their experiences as co-researchers and to contribute to the analysis of the data that were collected. The outcomes of using the Community Researcher Approach were: The organisation gathered important qualitative stakeholder feedback. The respondents commented that the approach helped them feel more at ease and they were more honest. The community researchers developed skills and confidence in their new role. The organisation was recognised by funders as being a good example of evaluation practice, and gained further funding.

 
What does each example pay attention to that makes them experiential?
Key characteristics or features and links to theory
Add examples and features to the bulls eye target on the blue wall using pre-prepared cards.

Sharing practice examples, characteristics and themes

Work in groups of 3: share examples from your own evaluation experience which reflect an alternative experiential dimension, record examples on cards described through 3 key facts.
Are they similar or different to my examples? What makes them similar and what makes them different?
Add examples to the target in relation to each other and how ‘experiential’ they might be considered.
Invite the group to identify further and /or new features that can be added.
Cluster examples around the features. Open discussion

Group Discussion – critical questions Talking knot or knot talking

What questions and issues does this approach raise?
Why is experiential evaluation important? (or not?)

A final viewpoint

Finally, share with someone else: How are you beginning to think differently and what do you want to explore further?

© Lesley Greenaway 2008 Return to workshops