If you can't mesure it, it doesn't exist

Maybe you've heard of her many many times, maybe you've watched this talk 4-5 times over and over again like I did, but still I want to share this inspirational talk of Brené Brown, who is a research professor (a.k.a researcher of storyteller) studying on vulnerability, courage, authenticity, and shame.

I am not going to tell about her story or the main topic of this talk but I am just going to focus on one little message she gives.

When Brené Brown was a doctoral student one of her professors told her:

The physicist side of me was easily convinced with this statement because it was obvious to me that scientists were able to exactly measure heat, height, energy, inertia, temperature etc. and prove it through math (at least in the classical physics world :)). 

However, the psychologist/social researcher side of me struggled for a while to give credit to this kind of an assertive idea. In the early stages of my experience on social research, I used the think that social phenomena should be "relative" making it almost impossible to measure correctly. 

Nowadays, I hate this belief of not being able to measure social phenomena because it is so so wrong! As a Human Resources person I find it my most challenging job to make people understand that, by using right tools and research setup, we absolutely can measure the things that we want to, OF COURSE IF THEY REALLY EXIST! 

This should be HR's magic wand to be used to increase the reliability of human resources practices.

Leaving you alone with this amazing Ted talk..


with English subtitles:

with Turkish subtitles:

Assessment Centers - Valid?

 

An assessment center is a structured combination of assessment techniques that are used to provide a wide-ranging, holistic assessment of each participant. However, in practice, each attribute is analyzed out of the whole 

(Graham & Morley, 1998).

It seems like a good idea to use more than one tool to measure different competencies of a candidate. However, you have to make sure for each and every tool that they are reliable and valid.

Below there is Pilbeam & Corbridge's (2006) summary of the design of an effective assessment center and my criticisms about how they are not applied well. 

1) Proper job and competency analysis.

How competencies are defined, chosen is a total mystery to me. There are tens and hundreds of consulting firms around, coming to you and saying that they are creating competency maps for your company following through some systematic steps. I didn't come up with any company showing me that their method really works scientifically. More sadly, I met with consultants of the most famous consulting firms in the world and I didn't hear them speak a word of data. 

2) Choosing and combining assessment techniques.

Through my academic experience in developmental psychology, what I learned was that it is very very hard to find a strong assessment tool for any kind of variable you want to measure. this is because tools very hard to create. You need time to create the tool getting help from theoretical literature, pilot it on a sample, apply it an a representative sample, analyze the data you have, check the reliability, revise your questionnaire, do statistical analysis and find out the validity, have a lot of money and knowledge to do it all.

Thus, it was very surprising for me to find out enormous amounts of assessment tools out there in the market. They are almost impossible to rely on. 

3) Assembling an integrated program

Once you have achieved #1 & #2, it shouldn't be very hard to achieve #3 :) 

4) Choosing and effectively training assessors

Since tools in the assessment centers are not tests but activities that include group exercises, in-tray exercises, role plays, simulations, interviews, personality questionnaires etc., you have to make sure that assessors are also trained well.

In one of the developmental research projects I've been in, we partnered with a consulting firm which would go out to the field and collected data for us.I remember giving them training about child development, how to use each and every tool, going out to the field and practicing and making them videotape themselves so that we can give feedbacks and training them again and again.

It is not easy! How well do you know you assessors?  How are they trained? I bet almost none of them know the statistical background of the tool they are using because they rarely have data analysis knowledge. 

5) Selecting and preparing candidates

Again you have to use standardized methods to select and prepare you candidates for assessment centers. Usually, management teams in the companies are very eager to interfere to the selection and preparation process, which should also be standardized. 

6) Post-event review and evaluation.

Ohhh, this is the amazing part! You and your candidates are the luckiest if your assessment center achieved all of the above. Now it is time to evaluate and make a decision and you will probably see people (managers/HR) around judging the evaluation process, the feedback and ratings of consultants. Probably, you'll also witness that the manager/HR will make their own decisions :)

Maybe they are right to do so 

BECAUSE YOU COULDN'T MAKE THEM BELIEVE IN YOUR ASSESSMENT PROCESS 

by TALKING WITH SOME DATA!