Before and after - measuring change without numbers

This workshop was presented at the EEEurope Gathering in Slovenia (2011).

I wanted to try out something new and I wanted to introduce colleagues in EEE to ideas about how to evaluate impact and change as a result of a training experience, or any other intervention or experiential learning for that matter.
 
An important element of any training programme is to be able to show and/or measure what difference training makes for individual participants. Measurement of change most often uses scales or numerical indicators to do this, but in an experiential learning environment this is a contradiction to the principles and values that shape experiential learning thinking and theory. For example: empowering participants to be active learners. The following description takes you through the sequence of activities that I used and includes a commentary on what I did, why and key points that emerged. I ran the workshop at the end of a 4-day programme so that participants could use their experience at the event to try out my ideas.

Workshop outline and commentary

Introduction (15 mins)

At the start of the workshop I met participants all together to explain the aims and structure of the workshop. I had already asked them to come with their digital cameras or another mobile photo device. I asked them to work with a partner and to walk and talk their way to the area where I wanted to hold the workshop (about 5-10 mins). During the walk their conversation was to be about their expectations, feelings, and aims before they came to EEE.

Methods for measuring change (30 mins)

When we gathered at the workshop area, I explained that we would be using their conversations later in the workshop. I started with a short presentation about different methods and discussed problems with these methods.

Using scales

This is a very common method used to evaluate individual participant changes as a result of learning. For example: increases in self confidence, specific skills, knowledge etc. Scales are useful for creating a record of change and providing a reference point or benchmark to see the difference from start to finish. There is often an assumption that change will be about improvement or an increase, which may put the learner under pressure to conform, when actually the experience has been negative!

How much has my self confidence changed?

Low self confidence  _______X (before)__________________________ X(after) ______ High self confidence

But there are other problems with this method that need to be considered:
There is a problem with language and meaning: Is it the language of the learner? Do participants all share the same understanding about what self confidence means? Or is this what the trainer language and meaning?
There is a problem with X: Although it looks like a change has occurred, without an explanation, we don’t know what the participant means or why they chose that mark.
There is a problem with before: we have to wonder how honest the before mark is for a participant who may be uncertain and anxious at the start of a programme.

Using pictures and metaphors

Another approach is to provide the learner with some tools to help them express and explain in their own language how they would assess their level of self confidence at the start and end of a course. For example: in an interview I did with young people asking about how they had changed or developed from being involved in youth work. Using the picture postcards they gave insights about how they had felt isolated and confused or 'at a cross roads', but now they felt that they were 'on the right track' and knew their direction.

Personal Reflection Tree is a picture which uses the image of a tree to show growth, change, diversion etc. (see above image) The tree is full of little people illustrating aspects like friendship, support, isolation, falling, climbing etc. Learners can choose which person best reflects themselves at a moment in time. Note: there are many versions of this tree used in different settings. It is not clear who or where it originated from - but thanks for a very versatile tool.

Picture Postcards or magazine images can be used to trigger ideas and help learners to put into words how they are feeling.

These approaches to measuring change start to empower the learner to use language and meaning that is important to them.

Experimenting with your own BEFORE and AFTER images (30 mins)

In this activity, I asked the participants to create their own before and after pictures to evaluate their personal change through the EEE experience by taking 2 digital photos. I suggested that they should start by checking with their partner, based on their original conversation, what would their personal measure be? What meaning do they want to express?

Participants set out to seek out the images that captured their change message, and met up to share these within the group (about 15-20 mins).

Conclusion (15 mins)

Participants shared their images with the group and commented on how this had been a valuable exercise in ‘putting it in their own words’.

TIP: One thing that I was not sure about at the start was how we would share the pictures. Using a mobile phone or laptop would be an option but a bit limiting. In the end, the solution was easy as someone had a photo printer which could print directly through a USB connection. So participants could print out their pictures, share them and take them away.

© Lesley Greenaway 2011
  Return to workshops