Finding and filling gaps in psychological theory: The case of building a career choice model
Did you know that, according to an increasing number of researchers, using group statistics to make claims about individual behaviour is usually fundamentally incorrect (i.e., ‘the ergodicity problem’: Molenaar & Campbell, 2009)? This may sound as a rather provocative, or maybe even worrisome, start of this blogpost. Nevertheless, for many researchers interested in individual change, this is a real and difficult problem. Most psychological theory is based on group-statistics, and if you cannot use these theories to make claims about individual processes, you are suddenly left with very little theory to base your individual-level hypotheses on. But there are ways to deal with this, and I would like to share a method with you that I found particularly interesting and useful.
One way to gain understanding of individual change processes, taking the ergodicity problem into account, is by building a theoretical simulation model. Theoretical simulation models are different from the models that most psychologists are familiar with. Where many psychologists are used to test differences based on empirically collected data, such as the ANOVA’s, multilevel models, and so forth, in simulation models, there is no data at all! These are models based on a very formal and concrete form of theory – theory so concrete, that you can translate it into mathematical equations.
Collaborating with Theoretical Biologists
I became inspired to build a theoretical simulation model by a friend from Theoretical Biology. This field focuses on the study of evolution. In my PhD project I aim to study the individual process of career choice. You may think these are completely different things, and you would be partly right. Evolution is a process on a much larger time-scale―across millennia―, than the processes I intend to study―taking only a few month or years. However, I found that the type of question they try to answer is similar to my own: What mechanisms underlie change processes?
“That would be exactly what my research needed, a firm, formal theory”
Colleagues from Theoretical Biology showed me some very interesting ways to simulate these change processes, based on sound theoretical assumptions. I started to get enthusiastic about the idea of collaborating: Here was an entire field of scientists with similar types of questions and with a systematic way of formalizing theory. My primary concern and question was: Could this be applied to an individual life or to individual behavior? Or, maybe even to my primary interest, an individual’s career choice process? That would be exactly what my research needed, a firm, formal theory. It did not take long for the collaboration to take off. Together with Piet van den Berg, the theoretical biologist I had snared for this collaboration, we started building a theoretical model on career choice as a process within individuals over time (see also our paper: van der Gaag & van den Berg, in press).
Finding theoretical gaps and filling them
When building the model, it became clear that we had to combine theory from different fields to account for processes on a micro level (like minutes, hours) as well as on a macro level (like months, years). On a macro level, theory on identity development claims that there are at least two types of exploration that play a role in important life decisions (Luyckx et al., 2006): Globally investigating multiple options (‘broad’ exploration), and investing time and energy to find out more about seemingly promising options (‘in depth’ exploration). However, when trying to formalize this theory in our simulation model, it became clear that we were missing an important part of the decision making process: how are the options evaluated, and weighed, to come to a decision?
We filled this gap with theory from decision science, where micro level models on information processing are becoming better and better at realistically describing decision making (Oppenheimer & Kelso, 2015). These models assume that individuals are not perfectly rational decision makers, as older models of decision making assume, but that evaluating options is a noisy process, far from perfect. All sorts of environmental factors influence how well you think an option fits you (like a well-written blog post, a boring speaker), and time also plays a role. If you recently read a really good blog post that convinced you that theoretical modelling might just be the future direction that your research needs, and after this you hear a boring talk on modelling, your previous experience may still influence your evaluation; you may still think modelling is a promising future direction for you.
“In psychology I think we usually have a tendency to stick to our own field of study, and build very specific theory to explain very specific phenomena”
I found this way of combining different psychological theories a very appealing and useful aspect of building a theoretical simulation model. In psychology, I think we usually have a tendency to stick to our own field of study, and build very specific theory to explain very specific phenomena. However, building a model, and trying to formalize how a process exactly works and unfolds over time, made me aware of the limitations of the theory I used. Then it became possible to look for other theories that supplemented this, and in this way, gain a unique and more complete picture of the entire process of career decision making. The results of the simulation model have offered us some very concrete, and some unexpected predictions that we can now test in empirical research.
A theoretical simulation model of career choice
The simulation model is a relatively simple computer program, containing mathematical equations that represent the above-mentioned macro- and micro-level theories (Luyckx et al., 2006; Oppenheimer & Kelso, 2015). We ran this program millions of times, using different settings to represent different types of individuals. Each simulation represents one individual’s career choice process―that is, what an individual career choice process could look like if our theoretical assumptions are correct. In this way, we found some interesting results, which would not be predicted by either theory alone. For example, we found that good choices do not always take longer. Sometimes a good choice can be made quite quickly, depending largely on what type of experiences individuals happens to have with an option, and how well they know their own preferences. We also observed conditions where relatively long decision making processes lead to relatively bad decisions: When picky individuals get stuck investigating options that are never going to be good enough for them. This condition emerged naturally from the model, and is actually in line with an ineffective ‘ruminative’ exploration strategy that has also been observed in empirical research on identity development (Luyckx et al., 2008).
“Trying to formalize your thoughts in math, forces you to be extremely concrete”
These are just some of the results that the model has produced in our research project in progress, which need to be further tested empirically. Although I only recently started with theory building based on simulation modelling, I find that even only the attempt at making such a model can progress theory formation. Trying to formalize your thoughts in math, forces you to be extremely concrete. This can make you see gaps in the theory that you had not noticed before, and it can give you some direction in moving beyond your own particular field of study. So, if you are interested in how individuals change over time, and you want to take the ergodicity problem into account, building a theoretical simulation model may be what your research needs.
Luyckx, K., Goossens, L., Soenens, B., & Beyers, W. (2006). Unpacking commitment and exploration: Preliminary validation of an integrative model of late adolescent identity formation. Journal Of Adolescence, 29, 361-378. doi:10.1016/j.adolescence.2005.03.008
Luyckx, K., Schwartz, S. J., Berzonsky, M. D., Soenens, B., Vansteenkiste, M., Smits, I., & Goossens, L. (2008). Capturing ruminative exploration: Extending the four-dimensional model of identity formation in late adolescence. Journal of Research in Personality, 42, 58–82. doi:10.1016/j.jrp.2007.04.004
Molenaar, P. M., & Campbell, C. G. (2009). The new person-specific paradigm in psychology. Current Directions In Psychological Science, 18, 112-117.
Oppenheimer, D. M., & Kelso, E. (2015). Information processing as a paradigm for decision making. Annual Review Of Psychology, 66, 277-294. doi:10.1146/annurev-psych-010814-015148.
Van der Gaag, M.A.E., & Van den Berg, P. (in press). Modelling the individual process of career choice. Proceedings of the 11th conference of the European Social Simulation Association.
Another interesting example of the ergodicity problem. My personal interest is in the treatment of psychological problems, which is often based on large clinical trials with only pre-post measures and therefore no possibility to study how individual patients improve.