Last time we asked, what is assessment and do we really need it?
This time we ask:
'What's the problem with assessment?'
We're going to continue our analogy from post 1 to post 2 and then move onto the hard stuff in post 3
We asked you to think of yourself as a fledgling assessor, a curious kid trying to understand what makes a butterfly the way it is catch up here
Now it's time to quickly switch roles,
last time you were doing the assessment, now the assessment is happen to you.
The assessor becomes the assessee
World of the assessee
Who are you?
Why are you looking at me?
What's going on?
Now picture this, now you are the butterfly in a jar, being looked at and assessed. you're thinking:
"what the hell is this giant pink wobbly monkey thing doing to me? why can I see out, but can't get out?
- I've got some serious butterflying to do - I am a butterfly! I'm meant to fly and flutter about.
Plus, "I've got to pick some of those yellow butter cup flowery things & see once and for all
if I really do like butter or not."
It doesn't seem fair does it?
This is often the fragile relationship of the assessee to the assessor. The person being assessed often doesn't really know what's going on, or why.
But then again, if they did have the foresight and could 'fully' understand they could just assess themselves and of course, this also totally changes the dynamic...
if I was a butterfly who knew he was being looked at, I'd want the assessor to just go away - I'd do my best boring moth impression, or I'd flaunt my butterfly stuff and show off, hoping to get on the cover of 'Butterfly Weekly'- either way,
the assessor would be looking at something false.
How the world affects assessment
Money, money money...
Does it make the world go round?
Nope! - it's a complex combination of axis rotation, gravity and thermodynamic forces that originate from the big bang - also the moon is thought to have slammed into us like a snooker ball on a cosmic table .
Don't believe me - try it yourself - take out a credit card and push it really hard into the ground and hope the world spins backwards
- it's time well spent.
Now back to the butterfly obsessed lepidopterist, your lab needs funding, your boss doesn't have it, in fact your bosses boss doesn't have it. Why? The institution you're in has to fight & fight hard to justify how any of this makes money.
Your research:
as interesting as it may be,
as insightful as the outcomes may be,
as useful to other work as it may be.
There's someone way up the chain of command who needs to prove how this makes money, after all, someone up there is an investor and they need to see a return on the money they've put in - and - that is - that.
So it's time to
Dance monkey... DANCE!
Now you have to spend 80% of your time trying to apply your work to a money making scheme and the other 20% on the streets dancing in a humiliating butterfly costume, trying to drive up 'the butterfly brand'.
Oh wait, never mind, now you've been replaced by a robot, it can quantify data faster than you, in bigger data sets than you and push out detailed reports like a rocket powered Cheeta - boy oh boy - that thing can sure push out some numbers ... with no insight, or perspective of what they mean...
We don't need none of that human lateral thinking, problem solving, creative insight and application here thank you very much!
If investors want numbers, by God we're going to give it to them...
It's simple duh! - We throw numbers at money and money throws numbers at us! - You want numbers - we want numbers - let's swap numbers and meet for lunch.
Politics, Politics, Politics...
Whoops, you blinked and there's a new government in town - they've just won by a landslide on their 'anti-immigrant butterfly policy'. If you want to survive you've either got to turn on your heels and run, or hop on the bandwagon and twist your research so your results state that butterflies are actually 'well dressed vampires' sucking the life out of the economy and taking all our jobs.
But hooray! Now you've got funding! ... but the new policy only funds the design of a big shoe to be used to squish all the butterflies into a fine paste and sold as anti-wrinkle cream. Your work here is done.
Are we finished with the butterfly analogy now?
Yes, thanks for asking...
Now here's the rub
Assessment does not take place within a vacuum, it is entangled within the world.
Assessment is a tool with many, many hands forcing it one way or another.
Assessment is the amorphous middle man, forming and reforming as values change and pressures change.
Assessment shapes and is shaped by
all of these socio-cultural, economic and political forces.
Assessment changes the world
& is changed by the world.
That is why it's so important to understand assessment, we DO need it, in fact we need even more of it,
But -and this is a big but -
we need the right kind of assessment. We need the kind that a four year old is so good at, assessment driven by genuine curiosity and for the purpose of learning from it.
Assessment can not and should not claim absolute 100% certainty.
it is fallible - it's focus, it's methods and it's values shift - just as seasons of the world change -
assessment is forced to change too.
Context, context context...
An assessment can create whatever it claims to measure. Assessment can take an idea or speculation and make it seem, through using measurement and giving names and classifications to it, as though it really exists. In this way we ‘reify’ our ideas - we give them an independent existence“ (Stobart 2008:30).
This is why assessment, out of context
can be dangerous
In trying to assess, we are just trying to understand,
but deep down we are all just poking around in the dark;
Yes, maybe we could poke around a bit better
Yes maybe we can shine a little light in the darkness
But seeing what's there in full clarity? that's impossible.
Dylan Wiliam has a great analogy for this trade off between validity and reliability -
it's all about focus - we can either:
'Spot light' something small in greater detail.
'Flood light' something big, but with low focus and lower detail.
Want more? -check out his article here
In the end its our choice, all of our choices to understand assessment a little better - if we value something, we must assess it
but...
how we assess?
what forces influence that assessment?
who is assessing and why?
we need to know better in order to help us find our way through the dark.
and don't forget to: