D-PAC, a Digital Platform for the Assessment of Competences
In collaboration with Imec and Ghent University, the University of Antwerp has developed D-PAC: A Digital Platform for the Assessment of Competences. This technology, using the principles of comparative judgment, enables stakeholders in educational and professional organizations to make social, objective and efficient group decisions.
Situation before
Coming to a consensus with in a group can bevery difficult and time consuming, especially when it concerns the evaluation of complex competences. People differ enormously in what characteristics of a competence they take into account when forming their overall judgement. Furthermore, there is a risk that some group members overrule the debate, resulting inbiased decisions. This can have a huge impacton high-stakes decisions that are made in both educational and professional organizations. For instance, think about the detrimental effects of a wrong selection of employees for a job, excellent proposals that do not receive funding, or grades for students that do not accurately reflect their performance level. Objective andefficient assessment procedures that improve group decisions and provide all stakeholders with feedback on the quality of their decisions are clearly needed.
Technology
Researchers from the University of Antwerp, Ghent University and Imec have developed state-of-the-art algorithms that automatically allocate assignments, cv’s, videos, audio or any other competence-based products into pairs. These pairs are then randomly distributed over different assessors in the online tool D-PAC. Assessors only have to indicate which product in a pair they think is the better one, which is an easier, faster and more reliable judgement than classic analytic judgements. Based oncomparative judgements of all assessors together, products are ranked from worst to best. This ranking can be used for selection purposes, but also for providing grades or setting performance standards.
Thanks to the intelligence of the algorithms in the D-PAC technology, the ranking reflects the shared consensus of all group members, without the need of individuals to meet in person or to invest in long hours of training and group discussions. This results in efficient and thrustworthy decisions. The technology also offers the possibility to monitor the quality of individual judgements already during the judgemental process, which can be used to make the process even more efficient or for professional development of individual decision makers.
About the researchers
D-PAC is founded in 2014 by Prof. Sven De Maeyer from the University of Antwerp, and is financed by Flanders Innovation & Entrepreneurship (Vlaio). The D-PAC team consists of 9 researchers at the department of Training and Educational Sciences at the University of Antwerp, 2 researchers of the department of Personnel Management, Work,and Organizational Psychology at UGent, and a designer and developer at Imec. Together they have developed D-PAC in close collaboration with stakeholders in both education and HR. Four years of research have been done on the validity, reliability and efficiency of comparative judgements, and on the impact of feedback on the individual and organizational level. D-PAC is already used widely in (educational) organizations, and was awarded with the Price for Examinations 2017 by NVE, the Dutch Society of Exams.
More information
University of Antwerp
Valorisation Office
Middelheimlaan 1
2020 Antwerpen - Belgium