Objective: To determine the inter observer variability in OSCE grading of students.
Methodology: This observational study was conducted at Shifa College of Medicine and included students of 3, 4 and 5 year of medicine, gynae, surgery and pediatric clerkships undergoing their end of clerkship OSCE. We enrolled 217 students, 7dropped out and 210 were analyzed. Each department conducted it on one station of each OSCE with four preceptors assessed the same student on that station. Scores were recorded in two sets; set 1 had preceptor 1 with standardized checklist and preceptor 2 without any check list; set 2 had both preceptors given standardized checklists. SPSS 25 was used for data analysis. ANOVA was applied to see variance of grading, figures and tables gathered showing p values.
Results: In this study, 21.9%, 41%, 37.1% students belonged to year 3, 4 and 5, respectively. 22.9%, 37.1%, 11.4 %, 28.6% belonged to gynae, medicine, surgery and pediatric clerkship respectively. 51.4% were male and 48.6% were female. Inter-observer variability in set 1 was statistically significant (p=0.000), and in set 2 was also significant (p=0.000) in all groups regarding subject and year. However, gender of students did not affect the grading variability.
Conclusion: The inter-observer variability among preceptors was high whatever the assessment method they used, concluding the role of some other factors in OSCE grading then the method alone.
Key words: Inter observer variability (IOV), OSCE, methods of assessments.
|
