Written by Dr Paul Kemp (Head of Phase 1 Assessments) and Professor Mary Morrell (Director of Phase 1).
It is seven months since the COVID-19 virus entered our lives. During this time ICSM has come together to do all we could to support our colleagues working in the NHS and to ensure that our students’ learning was minimally interrupted by the pandemic. For MBBS Early Years students, we managed to teach almost all of the curriculum online. We really appreciate the commitment of our students, the teaching team and all the support staff who made this possible.
How best to deliver the end of year assessments during the pandemic was very difficult to decide. We had discussions with other medical schools and with the Faculty working in the later years of the programme, who had already successfully delivered online assessments. Our aim was to do the best we could for our students during a period of universal unprecedented uncertainty. We are grateful to the Editors of the Gazette for giving us the opportunity to tell you about the decisions we made.
Firstly, the move to time-limited, online assessments in Phase 1a, was made at a College level on 13 March 2020. The policy explaining these regulations was published on 6 April, following consultation with staff and students. Five options were considered and their reasons for choosing time-limited, online exams within the normal exam window are given in this document: [https://www.imperial.ac.uk/media/imperial-college/about/public/Remote-Assessment.pdf]. Within ICSM, the College decision was consistent with what had already taken place in Year 6, where finals had been successfully delivered online.
In addition to the College decision, in a medical school context, students needed to progress through their programme in accordance with the GMC requirements. Therefore, plans we made were to:
- make the exams open book and to keep the pass marks unchanged even though the exam and pass mark were intended for closed book exams;
- not to change the question style or content in the summative exams, ensuring that they would be consistent with the formative exams that the students had already sat;
- provide further formative exams, in the same (online) format as the summative exams. These would be made available for an extended time to ensure all students could access them;
- move the start times of the summative exams to accommodate as many students as possible;
- provide coordinated and ongoing welfare and academic support via our Academic Tutoring programme.
These plans were discussed with Phase 1a via a Zoom meeting (open to all students) that was also recorded. They were also discussed and approved at the ICSM Board and at College level.
What were the alternative options, and why were these not chosen?
As described above, the decision to deliver time-limited exams was made at College level and the reasons are given in the linked document. We know that other medical schools took different routes, some cancelled exams, or changed them to writing extended reflections. Others increased the pass marks for at least some modules, to take into account the ability of students to look-up answers.
Our approach falls in the middle, enabling our students to progress but aiming to minimise stress by retaining the calculated pass marks, even though the exams were open-book. We believe our approach best facilitated our students’ learning for the knowledge, skills and behaviours, which they will need in their future years as a doctor, while being supportive of the difficult circumstances all students have endured during the pandemic.
What specific factors were involved in the decision to administer Phase 1a examinations in the manner and format seen this year?
The primary factor in our decision, was to ensure that our students were able to complete their year and progress to Phase 1b, thereby fulfilling the course requirements of the College and the GMC. We needed to achieve this with minimum additional stress for our students. We were aware that our Phase 1a students were familiar with the software we would use for the online exams. We also hoped that by communicating our decisions (both in writing and via video conferences) and enabling students to ask questions, we could reduce the level of anxiety students were feeling. The FAQs were made available to all students in a document posted on MedLearn. We know that the Phase 1a and Year 2 students were sometimes frustrated with our response times, as we worked to consolidate the vast numbers of queries.
Were adequate measures taken to accommodate for the varying circumstances of students who may be in difficult situations?
Our Academic and Senior Tutors kept in contact as much as possible. We know that the library being shut generated issues, and students in specific circumstances affecting the ability to study were supported in applying for mitigating circumstances. Each student in Phase 1a has an Academic Tutor who provides ongoing coaching and support. Academic Tutors were provided with weekly additional training in the run-up to exams to ensure they remained informed and able to fully support students.
The main issue that we could mitigate was the start time of any exam, though this was restricted by College rules. We moved exam start times to the middle of the day to minimise the effect of different time zones. We also used the expertise of our Faculty, who research sleep disorders, to provide advice to students on how to modify their circadian rhythm.
Were there any safeguarding measures to ensure that students adversely affected by COVID-19 could receive the support they required? With specific respect to exams, was there any additional mitigation process (in addition to the usual policy)?
Each year some students experience extremely difficult circumstances. The impact of COVID increased the caring roles some of our students have, and also increased personal illness or loss for some of our students. Academic tutors proactively engaged with students in relation to COVID issues. In addition, we encouraged applications for mitigating circumstances for COVID-related issues, including time zone and internet access difficulties. The School received hundreds of requests this summer and these were treated sympathetically.
And what did you do when things went wrong?
A major factor that had an impact on our Phase 1a students was the cancellation of the Principles of Medicine exam on 27 April 2020. The cancellation occurred unexpectedly as a result of a technical failure during the exam and not all students received the communication relating to the cancellation. This was the first remote exam our students sat, and the stress that the failure of this exam caused was enormous and something we are aware of and deeply regret.
As we said in our email, sent after the exam failure occurred (28 April 2020) it was due to an issue completely out of our control. The decision to cancel the exam was taken following over 300 emails, and a similar number of phone calls from students who were unable to start the exam, or who had started but who could not continue. The assessment team were unable to access the examination site, so were not in a position to advise on the issue itself or a timeframe for its resolution. At that point, there was no obvious alternative other than to stop the exam. All this happened within 40 minutes of starting the exam.
Having stopped the exam, we needed to ensure that we could progress our students. This issue was reviewed by the Phase 1 Leadership and the Assessment Team, as the cancelled assessments still needed to be taken. Whilst a number of students had completed the papers, it was unclear if all of the data/answers had been correctly stored making it impossible to use. It would also not have been fair to students who had not been able to start the paper — or who stopped mid-exam at our request — to make them do an additional paper, and not require everyone to do the same, given that there would be knock-on effects on revision for subsequent papers. We therefore made the first papers available so that everyone had a chance to see the questions, and rescheduled the exams using the resit paper that had already been approved by the external examiner. We recognise that this decision was unpopular, and we did not take it lightly.
We have provided this additional information to illustrate that our decisions during this time were taken after significant consideration, to enable our students to progress while providing as much support and as possible. We know that there are students who disagree with these decisions and we respect this. At all times, we kept the best interests of our students in mind, given the extreme circumstances within which we were working.
Were you affected by remote exams and assessments? What were your thoughts and experiences? Get in touch at email@example.com or DM us on Instagram @icsmgazette