Predictive validity of recruitment into public health specialist training in the UK

Gray, Selena; Pashayan, Nora; mason, Brendan

Abstract

Background: In 2009 an assessment centre, looking at critical thinking (Watson Glaser Critical thinking Appraisal ) and numerical reasoning numeracy (Rust Advanced Numerical Reasoning Appraisal t) was introduced to identify candidates to invite to a selection centre for recruitment into public health speciality training in the UK. In 2010 a situational judgment test (SJT) developed for use in the public health context, was added to the assessment centre. These three tests have been used consistently since then.
Method: A cohort analysis of the 2009-2012 appointees was undertaken to explore the association between performance in the overall recruitment process (ie assessment and selection centre) and progression through the training scheme, as evidenced by success rates in the Part A and Part B UK Faculty of Public Health professional exams, and by the likelihood of obtaining a satisfactory Annual Review of Competence Progression (ARCP) Outcome..
Results: Of the 281 registrars appointed between 2009-12, 223 appointees had sat the Part A exam;134 (60%) had passed, 42 (19%) had failed, and 47 (21%) had banked one or other of the two parts of the exam. Of the 155 who had taken the Part B exam, 140 (90%) had passed and 15 (10%) had failed. Of the 195 who had an ARCP recorded, 172 (88% had a satisfactory outcome 1; 90% of unsatisfactory ARCPs were in registrars who had not passed Part A at the first attempt.
A discrimination analysis using ROC curves found that each of the different elements of the recruitment process independently contributed to the likelihood of passing Part A and Part B. The odds ratio of individuals with a high score in the overall recruitment process compared to those with a low score of passing Part A first time was 2.64 95% CI 1.52-4.58 and Part B was 3.09 95% CI 1.00- 9.52. ARCP seems not to be an independent outcome measure of progress in training.
Main conclusions of the study: The recruitment process for public health specialist training demonstrates good predictive value with progression as defined by passing key professional exams of Part A and Part B in a timely manner. The overall AC score is better prediction than individual test components, and the overall recruitment score is better prediction than AC or SC separately.
However, the sample size is small, particularly for prediction of Part B performance, and further work needs to be done to extend the cohort to confirm these details.

• Presentation Conference Type: Poster
• Conference Name: International Network for Researchers in Selection in Healthcare
• Start Date: Nov 10, 2014
• End Date: Nov 11, 2014
• Publication Date: Nov 10, 2014
• Peer Reviewed: Not Peer Reviewed
• Keywords: Selection and Recruitment; public health training
• Public URL: https://uwe-repository.worktribe.com/output/807891
• Additional Information: Title of Conference or Conference Proceedings : International Network for Researchers into Selection into Health Care (InReSH)