Assessment of Professionalism

Aims

To gather validity evidence for instruments that are used during the residency admissions process to assess the professionalism of applicants.

Persons involved :

UDREM-UNIGE   Nadia Bajwa Dominique Belli Annick Galetto-Lacour Mathieu Nendaz Klara Posfay-Barbe Nu Viet Vu

Collaboration Internationale   University of Illinois at Chicago Yoon Soo Park Rachel Yudkowsky

 

Résultats :

RESEARCH QUESTION 1 :

The purpose of this study was to investigate the validity evidence for and feasibility of incorporating standardized-patient (SP)-based Professionalism-Mini-Evaluation Exercise (P-MEX) assessment scores into the existing pediatric residency admissions process.

The inclusion of an SP-based P-MEX assessment was found to be a reliable, valid, and feasible way of assessing applicant professionalism during a residency admissions process. In particular, the creation of a composite score was found to be a reliable and valid way to rank and select applicants.

Published in Advances in Health Sciences Education Theory and Practice

https://archive-ouverte.unige.ch/unige:84388

RESEARCH QUESTION 2:

In this study, we developed a standardized assessment letter specifically for pediatric residency applicants, the ALPS (Assessment Letter for PediatricS), based on competencies desired in pediatric residents: professional integrity, scientific curiosity, patient management skills, autonomy and organization, teamwork and collaboration, and communication skills.

To our knowledge, this is the first reported standardized letter developed for pediatrics with supporting validity evidence for its use as part of the admissions process. Our findings indicated that the ALPS items were internally consistent and moderately reliable, with ALPS scores correlating moderately with admissions decisions. These results indicate that the ALPS is a promising tool that can be further refined to support admissions decisions.

Published in Teaching and Learning in Medicine

https://archive-ouverte.unige.ch/unige:100616

RESEARCH QUESTION 3:

Using data across 5 longitudinal cohorts of pediatrics residents, this study examines the

predictive validity (relations to other variables) of Professionalism Mini-Evaluation Exercise (P-MEX) scores in relation to rotation evaluations of admitted postgraduate year 1 residents.

Admissions processes have an underlying goal of identifying potentially successful physicians

who meet standards of competence. Integration of a professionalism assessment, such as the SP-based P-MEX assessment, into the residency admissions process provides a snapshot of an applicant’s level of professionalism and may predict performance in the first year of residency.

Published in Academic Medicine

http://archive-ouverte.unige.ch/unige:128083

RESEARCH QUESTION 4:

Does the factorial structure of Professionalism Mini-Evaluation Exercise (P-MEX) assessment scores align with relevant professionalism domains when administered using standardized patients? Does the degree to which the relationship among P-MEX test items and P-MEX test components conform to the professionalism construct on which score interpretations are based?

Hypothesis: The multidimensionality of the P-MEX will be demonstrated by exploratory factor analysis (EFA).  We hypothesize that factors found in the EFA will be correlated with each other due to the relationship to the construct of professionalism. A confirmatory factor analysis (CFA) will demonstrate relationships between P-MEX items and latent domains and demonstrate construct validity for the factors that emerge from the assessment.

Manuscript in progress