STUDENT FEEDBACK ON THE OBJECTIVE STRUCTURED COMPONENT OF THE PRACTICAL EXAMINATION IN PHARMACOLOGY

1. Manipal College of Medical Sciences, Deep Heights, Pokhara, Nepal. Address for correspondence : Dr. Ravi Shankar Pathiyil Department of Pharmacology, Manipal College of Medical Sciences P.O. Box: 155, Deep Heights Pokhara, Nepal Email: pathiyilravi@rediffmail.com The study was planned to determine the attitude of undergraduate medical students towards the objective structured practical examination (OSPE) in pharmacology and to investigate any influence of sex, nationality and medium of instruction at school on these attitudes. The second year undergraduate medical students of the Manipal college of medical sciences, Pokhara, Nepal were asked to complete a questionnaire regarding the spotters and communication skills component of the practical examination. Students considered OSPE a good method of practical examination. The male students agreed more strongly that the spotters develops the ability to choose an appropriate drug while the female students were more in favour of OSPE over animal experimentation. The Sri Lankan students were less in favour of the introduction of stations testing skills as part of OSPE and also disagreed strongly with the point that communication skills should be conducted in a language other than English. Based on these observations, changes should be made in the teaching as well as evaluation methodologies to maximize learning.


INTRODUCTION
Objective structured practical examination (OSPE) is being increasingly used not only in the developed world but also in developing countries like India and Nepal due to benefits like objectivity and reliability. 1 OSPE and the closely related objective structured clinical examination (OSCE) are preferred to other assessment tools. The problem to be solved is the same for all the examinees thus eliminating bias due to different problems given to different examinees. OSPE/ OSCE has been shown to be a reliable device with a good capacity for discriminating between different categories of students. 1,2 Application of the textbook knowledge to solve problems related to clinical practice can be better tested by OSPE/ OSCE. OSPE/OSCE is a robust system of examination with little threat to the validity from sources other than the performance being tested. 3 The sub-components of the problem and the grading system are decided beforehand. All these makes OSPE/OSCE an ideal assessment tool in undergraduate medical education.
OSPE in pharmacology requires the student to go through a series of stations (between 10-20), spending three minutes at each station. At the Manipal college of medical sciences, Pokhara, Nepal affiliated to the Kathmandu university OSPE stations account for 25 out of the 50 marks in the practical examination in Pharmacology. The OSPE consists of two parts: a spotter section of 20 marks and a communication skills section of 5 marks. At the spotter section a student identifies a drug among the choices displayed for a particular condition and then answers one or two questions related to the drug. At the communication skills station the student communicates with and prescribes a drug for a simulated patient. The student is assessed by an examiner, ticking a carefully prepared checklist. This form of examination is being used for formative evaluation in pharmacology in our institution for more than 6 years.
In pharmacology it has been our experience that the non-OSPE components of the practical examination (namely prescription writing and problem solving) are showing bias in assessment due to various factors. Questions pertaining to the OSPE component are proposed to be increased and stations testing skills (as detailed afterwards) are proposed to be introduced. This is in accordance with trends in the west where OSPE/OSCE are increasingly replacing the traditional methods of student assessment. 4,5 The Manipal college of medical sciences, Pokhara has an international student body with students from Nepal, India, Sri Lanka and also non-resident Indians. We are actively considering testing the communication skills, history taking abilities and clinical skills of students using the OSCE/OSPE format and simulated patients. The students come from diverse social, cultural, economic and educational backgrounds.
The aims of this study are: 1. To evaluate the attitudes of medical students towards the OSPE component of the pharmacology practical examination. 2. To evaluate the attitude of students towards procedure stations as part of OSPE component. 3. To investigate any gender and nationality differences in these attitudes. 4. To investigate any influence of the medium of instruction in school on these attitudes.

MATERIALS AND METHODS
A survey was undertaken of 96 undergraduates during the second half (IV th semester) of the second year of their MBBS course. Students were asked to complete a questionnaire which asked them to consider the two OSPE components of their practical examination in pharmacology, namely: spotters and communication skills.
The students were given a brief explanation of the aims of the study, but were not told that the results would be analyzed for gender and nationality differences. The students were assessed by means of a standardized Likert type scale containing 10 items. The student was asked to grade each item using the following scoring system: 5-strongly agree, 4-agree, 3-neutral, 2disagree, 1-strongly disagree.
Responses were received from 70 students (72.92%) of whom 41 were male (58.6%) and 29 were female (41.4%). Results were analyzed using the SPSS statistical package, and Mann Whitney test was performed as the test of significance.
the time allotted for the station testing communication skills (3 minutes) was not enough.
The gender analysis of students' attitudes towards OSPE in pharmacology is shown in Table 3. The male students agreed more strongly with the statement that the 'spotters component of OSPE develops the ability to choose a prescribed drug' (p=0.0169). The female students preferred OSPE The questionnaire used for eliciting information is shown in table 1. The items put within the square brackets were used for further analysis of the different items with regard to differences in sex, nationality and medium of instruction. Table 2 shows the mean attitude score of the different items on the questionnaire. Students most strongly agreed with the points that both 'the spotters and the communication skill were a learning experience' and also with the points that 'communication skills trained them to convey drug related information to the patient'. They felt that to animal experimentation in pharmacology (p=0.044). There was also gender differences regarding the points that 'OSPE decreases the element of luck' (p=0.037) and that 'theory questions given to the students along with the spotters are a good idea' (p=0.0049).

RESULTS
The nationality analysis of students' attitudes towards OSPE in pharmacology is shown in table 4. The attitude of the Sri Lankan students towards the point 'theory questions related to the spotter are a good idea' was less favourable than that of the other nationalities (p=0.031). The Sri Lankans also scored less regarding the introduction of stations testing skills as part of OSPE (p=0.007). They also strongly disagreed with the point that 'communication skills should be conducted in a language other than English' (p=0.0002).

MEDIUM OF INSTRUCTION
The English medium students were more favourable towards the point that 'it is confusing to switch from one problem to another within a short time period' (p=0.004). The vernacular medium students agreed more strongly with the point that 'communication skills teaches how to give non-drug information to the patient' (p=0.022). There were borderline evidence of differences between the vernacular and English medium students regarding their attitudes towards the points that 'spotters is a reliable type of  examination' and 'spotters develops the ability to choose a P drug' (p=0.064, p=0.059).

DISCUSSION
The objective structured practical examination (OSPE) is being increasingly used as an objective instrument for the assessment of the knowledge and skills of the medical undergraduates. It has been used for formative evaluation in the preclinical curriculum of the Kathmandu university for more than six years. It was adapted from the objective structured clinical examination (OSCE). The OSPE appears to be a reliable device with a good capacity for discriminating between different categories of students. It is better in these respects than the conventional practical examination. Moreover, it has the scope for being structured in such a way that all the objectives of laboratory teaching can be tested. 1,6 In the west, group objective structured clinical examination (GOSCE) has been tried as a means of assessing medical students. 7 The GOSCE format makes possible the assessment of a large number of students without the time and expense needed to evaluate students individually. Integrating a structured oral examination (SOE) in an objective structured clinical examination has been shown to increase overall reliability and examiner satisfaction. 8 SOE may be included as a component of communication skills assessment. In the revised curriculum of the Kathmandu university stations testing skills in pharmacology are proposed to be introduced and the attitude of students towards this development was generally positive.
Attitudes have a very strong influence on the behaviour of a person. 5 They form the link between knowledge and practice. The importance of students' attitudes towards the training program in undergraduate medical education is being increasingly recognized. The students' attitudes towards medical college in general and pharmacology in particular have improved after the introduction of problem-based learning (PBL). 9,10 PBL also has other positive effects in terms of use of additional learning resources, interdisciplinary learning, team work and making learning fun. 10

MEAN ATTITUDE SCORES
Item wise scores indicate that the attitudes were favourable except towards the points that it is confusing to switch from topic to topic within the category spotters and item numbers 4 and 5 under the category communication skills. The spotters test a wide range of topics in pharmacology in no particular order and the students did not find this particularly confusing. The time allotted for the communication skills station is not enough and should be increased to at least 5 minutes. The students also suggested that the communication skills should be conducted after they finish the spotters component of the OSPE.

GENDER ANALYSIS OF ATTITUDES
It has been observed that there are gender differences in medical students' assessment of their personal skill levels. 11 The attitude of male students towards the point that 'communication skill develops the ability to choose a P drug' was more positive. More female students preferred OSPE over animal experimentation and this could be related to the greater empathy of females towards animals. It could also be due to the greater number of Sri Lankans among the female students and the Buddhist reverence for life is well known.

NATIONALITY ANALYSIS OF ATTITUDES
The observed differences except the point that communication skills should be conducted in a language other than English are difficult to explain. The Sri Lankans strongly felt that if communication skills are conducted in Nepalese then they would be at a serious disadvantage. Language classes in Nepalese could be a possible solution.

MEDIUM OF INSTRUCTION DIFFERENCES
The differences observed were contrary to expectations with the English medium students having a higher score towards the point that 'it is confusing to switch from one problem to another within a short time period'.
The explanation for these differences emerged from a focus group discussion with the students after the analysis of the data. School education in English is a colonial legacy in most parts of our subcontinent. Students from different parts of the subcontinent have been following a basically similar scheme of education. Practical exercises in Biology follow a different pattern in the three countries namely Nepal, India and Sri Lanka from where the majority of our students originate. In Nepal the students carry out dissections of frog, rabbit, cockroach etc. In India due to the increasing pressure of animal rights activists animal dissections are not being carried out in a number of colleges and students are only demonstrated the dissections by their teachers. In Sri Lanka the students do not carry out dissections nor are there any demonstrations. Previously there were dissections but they were withdrawn following opposition from the Buddhist clergy and animal rights activists. The preference of the Sri Lankans for OSPE over animal experimentation might be due to the influence of school education. These differences will need further studies to be elucidated.
In the past after the completion of the OSPE exercise the different deficiencies/weaknesses were noted and communicated to the batch as a whole. The detailed break up of the results was shown in case of the spotters but not in case of the communication skills. After the analysis of the results of the study we show the detailed break up of the results of each student and discuss with individual students the lacunae observed and ways to improve them. The results till now have been gratifying. We are also trying out a system where the evaluation of communication skills is carried out by both peers and faculty and feedback from both is used for improvement. PBL has been shown to improve both the cognition of and motivation towards medical education 12 and the increased emphasis on PBL, communication skills and information technology skills in the revised curriculum of Kathmandu university is a step in the right direction.