Predicting performance: How can we determine whether someone will be successful?
How can we determine whether someone will be successful as a student, a manager, a policeman, a musician, an artist, a football player, and so on? This is a highly relevant question for psychology researchers, as well as human resources professionals in many organizations. In order to find an answer to this question, most lay persons would prefer to meet that someone, talk to the person, size him or her up, and figure out what that person is like. In general, most people have great confidence in their abilities to make judgments about others’ future performance after such an ‘assessment’. However, the majority of the population, even experts, are not very good at making correct judgments based on such unstandardized information [1].
Determining future success from a scientific point of view
Answering the question “How can we determine whether someone will be successful?” from a scientific point of view is a central topic in my current research project, as well as in the educational MSc. program Talent Development and Creativity. As psychologists we tend to approach this question by (roughly) following three steps. First, we start by trying to find a theory that could help explaining successful performance. Then, we identify specific traits and characteristics that are theoretically related to performance, such as cognitive ability, conscientiousness, or creativity. Next, we search for reliable instruments to measure these traits, and we statistically determine whether the trait measurements indeed predict successful performance. If this is the case, than someone with the “right” level of certain traits is likely to become successful.
The approach mentioned above is known as the ‘signs approach’ to predicting future behavior, and it is the dominant approach for predicting performance in psychological research and practice. It seems a sound approach that is preferable over (expert) judgement, and it usually is [2]. However, should we always go through all of the steps mentioned above? I argue that, in order to know whether someone is good at something, there is a simpler scientific approach that can be more effective as well.
If you want to know if someone will be successful, just find out if the person is good at the task he or she will need to perform.
In a nutshell, if you want to know if someone will be successful, just find out if the person is good at the task he or she will need to perform. This is known as the samples approach [3] to predicting performance, and it is based on the notion that past or current behavior is an excellent, and perhaps even the best predictor of future performance. According to this approach, we should take samples of relevant behavior, rather than distinguish presumably relevant traits. It is a simple idea and often very effective as I will illustrate below.
For example, In the American National Football League (NFL), the annual selection round among college-level players is a sign-based carrousel including physical ability tests such as jumping, sprinting and strength, an intelligence test, and an interview. However, research has shown that performance on most of these tests does not predict performance in the NFL very well. If you want to predict professional football performance, the most effective way is to just look at past football performance [4].
Predicting academic performance of first year psychology students
Similarly, if you want to know if someone will be good at a certain job, work sample tests that simulate the tasks of the job as realistically as possible are the best predictors for future job performance [5]. In our own research we have adopted this approach to predict academic performance by using trial-studying tests that mimic a representative course in the study program. For admission to the psychology program, for example, future students have to study two chapters from the book used in Introduction to psychology, view an online lecture, and take exams on this material. This is very similar to what they have to do as actual students and that is exactly why this approach to predict academic performance is effective [6]! The score on this test was a good predictor of academic performance in the first year. Additional advantages of sample-based tests are that they are often rated as highly face-valid, and offer the possibility of self-selection. That is to say, these tests also provide insights for the assessees (e.g. do I find this interesting, can I handle the level etc.), and not just for the assessors [7]. As a student stated after taking the trial-studying test “It helped me indicate what studying at the University of Groningen would be like and I really liked that.”
In sum, to know if someone will be good at something, we do not always need to distinguish traits and characteristics. Indeed, one could say that when we take samples of relevant behavior, all relevant traits are being measured together (be it implicitly). For instance, a trail-studying test, like the one we used in our own research, may measure both ability (does the student have the necessary skills and capacities), and motivation (does the student take the effort to prepare well). As a result, the first two steps that psychologists usually take, finding a theory and specifying traits, are not necessary to make good predictions. By designing assessments that are comparable to the tasks that performers will actually have to carry out, we may measure all relevant traits.
So, when it comes to predicting future performance, perhaps we should not spend all our time untangling potential relevant underlying traits, but just assess the actual relevant behavior.
Relevant links and publications
NOTE: Image by Travis Wise, licenced under CC BY 2.0
Susan Niessen is a PhD Candidate at the University of Groningen at the department of Psychometrics and Statistics. Her research project is titled Selection in Higher Education, and her research interests are, talent assessment, educational measurement, academic performance, and prediction of behavior. She also teaches the course Talent Assessment in the Master program Talent Development & Creativity, together with Rob Meijer, starting in the academic year 2016-2017.
For an overview of Susan’s publications, please click here.
