In response to the increasing community spread of COVID-19 in the first quarter of
2020, the Singapore healthcare system moved to prioritize pandemic management over
other non-essential services. The delivery of medical school curriculum and learning
processes, which heavily involves clinical faculty from various healthcare institutions,
had to be reconfigured and reengineered as healthcare staff reassignment and infection
containment practices are escalated.
A few years earlier, in 2015, Duke-NUS had implemented an eLearning week, where face-to-face
activities were conducted online in order to prepare for unforeseen disruptions to
the curriculum. During the COVID-19 pandemic, we first put these practices to use
when enhanced social distancing restrictions were mandated across the country in February
2020, and university guidelines required that all classes with 50 or more students
be moved online [1]. As the pandemic escalated, Singapore imposed increased movement
restrictions in April 2020 and all educational institutions were instructed to move
to full home-based learning [2].
The transition to online learning may be smoother for a traditional lecture-based
course, as educators can share pre-recorded lectures, or even deliver live lectures
through video-conferencing platforms. Learning resources can be posted on their institution’s
learning management system for their students to access remotely. However, a growing
trend in higher education is towards the use of “flipped classrooms”, where the acquisition
of information is done by the student prior to class, and the faculty use class time
to reinforce the knowledge that students have acquired by providing opportunities
for students to apply the knowledge through application exercises [3].
A disruption in face-to-face teaching is challenging for Duke-NUS, for two reasons.
First, the foundational science concepts covered in the first academic year are revisited
and built upon later during the clinical years. Disrupting the delivery of the foundational
science content would have profound effects on subsequent parts of the curriculum.
Second, Duke-NUS employs a complex learning pedagogy which emphasizes peer-to-peer
learning through faculty-facilitated, student-led discussions [4]. Known as TeamLEAD
(Learn, Engage, Apply and Develop), the primary mode of teaching for the first-year
medical students at Duke-NUS is an adaptation of Team-Based Learning (TBL). The processes
of TeamLEAD and TBL have been described elsewhere [5–7]. In brief, the three defining
features of TeamLEAD and TBL are pre-class preparation, the readiness assurance tests
(RATs), and the team application exercises [8]. In the TeamLEAD and TBL class formats,
students first take the readiness assurance tests individually (individual readiness
assurance test (IRAT)), then as a team (group readiness assurance test (GRAT)). It
is after completion of the GRAT that TeamLEAD diverges from TBL. Known as the modified
TeamLEAD Readiness Assurance Process (mTRAP), teams will now submit any unresolved
queries to the whole class. The facilitator then assigns other teams to work on these
queries, after which these queries are addressed in a faculty-facilitated, class-wide
discussion. Thus, at Duke-NUS, TBL represents a flipped classroom process that requires
face-to-face interaction and is heavily dependent on student attendance and participation.
Here, we detail the experiences learnt from the eLearning week in 2015 and the challenges
faced in moving an entire module to online learning during the COVID-19 outbreak in
2020.
eLearning Week of 2015
In 2015, a week of the Body and Disease module was delivered in an online format to
the class of 2018 during eLearning week. Body and Disease is the final module of the
first-year medical curriculum, where students learn to integrate 5 different knowledge
tracks: microbiology, immunology, pharmacology, pathology and clinical investigations
[9]. The educational leadership designed an online TBL format de novo, adhering to
the core principles of TBL [5]. During the eLearning week, students received 8 hours’
worth of video-recorded lectures as preassigned material. The online class session
started in a similar manner to a face-to-face session, with students attempting the
IRAT by logging into a test-taking platform, but from their homes. For the GRAT, the
students logged onto a virtual chat room (chatzy.com) to carry out their discussions
and keyed their answers into an in-house assessment tool that provided immediate feedback.
The mTRAP process was carried out using a shared Google Document (Google LLC, Menlo
Park, CA, USA). Students would post their questions on the shared document and the
assigned team would submit their responses in the same shared document. A faculty
member, a content expert on the topic, served as a facilitator. Members of the education
administration team provided logistical and technical support for the class. Students
did not use any video-conferencing platform for their discussions in 2015. Here, we
present the impact of eLearning on students’ perception of TBL by analysing students’
feedback from the eLearning week, which occurred during Academic Week 7, and for the
preceding week, Academic Week 6, which was carried out face-to-face. We analysed the
students’ ratings on whether eLearning achieved the same desired TBL outcomes of developing
verbal, written, collaboration skills as a face-to-face session. As a control, we
analysed the standard evaluation feedback, which was administered after every session,
from both academic weeks. Lastly, we compared the standard evaluation feedback from
the eLearning week of AY2014/15 with a corresponding academic week from AY2013/14
within the Body and Disease module.
Lessons Learnt During eLearning Week in 2015
The students from the AY2014/15 cohort felt that eLearning mode of TBL was less effective
in fostering their verbal, collaboration and leadership skills, compared with the
face-to-face mode in Academic Week 6 (Table 1). A comparison of the feedback from
those two academic weeks showed no difference in the perception on how eLearning versus
face-to-face class sessions would affect their written skills (Table 1). This is perhaps
unsurprising, as only a chat room utility and a shared online document were used to
conduct intra- and inter-team communication during the eLearning week. We then compared
the students’ standard evaluation feedback for the eLearning week and Academic Week
6 within AY2014/15 cohort. We found that moving the class sessions online did not
significantly affect the students’ perceptions of their learning experience (Supplementary
Table 1). The different study material assigned across the eLearning week and Academic
Week 6, however, may play a role in how the students perceived the class session.
As such, we compared the standard evaluation feedback from the AY2014/15 cohort’s
eLearning week with the AY2013/14 cohort’s corresponding academic week (Supplementary
Table 2), where students received a similar set of study material. No significant
differences were found between each cohort’s perceptions of their learning experience
for this particular academic week. To ensure that the lack of difference in students’
perception was not due to a cohort effect, we looked at the overall perceptions that
students had for the face-to-face class sessions in the Body and Disease module. No
significant differences were found between the cohorts of AY2014/15 and AY2013/14
either (Supplementary Table 2). Taken together, these findings suggested that the
study material assigned during this academic week and cohort effect did not have an
impact on the students’ perceptions of TBL outcomes.
Table 1
Comparison of student perceptions of desired TBL outcomes from the eLearning week
versus the preceding face-to-face week within the same cohort in AY2014/15
TBL reaction outcomes*
AY2014/15
Overall, the instructional strategies allowed me the opportunity to develop my:
eLearning (Week 7)
Face-to-face (Week 6)
p value
Verbal communication skills
2.56 (0.99)
3.88 (0.73)
< 0.001
Written communication skills
3.34 (0.96)
3.17 (0.92)
0.307
Collaboration skills
3.36 (0.90)
4.17 (0.68)
< 0.001
Leadership skills
3.00 (0.88)
3.63 (0.79)
< 0.001
The TBL reaction outcomes for the eLearning week, which occurred during Academic Week
7 of the Body and Disease module, and for the preceding week that had face-to-face
classes, during Academic Week 6 were compared. Students from the AY2014/15 cohort
felt that the eLearning mode of instruction did not facilitate their development in
the following TBL outcomes: verbal communication, collaboration and leadership skills
n = 50 for the eLearning week, and n = 52 for the face-to-face Week 6 in the cohort
of AY2014/15
Data is presented as mean (standard deviation). All comparisons were analysed using
Wilcoxon rank sums tests
*Evaluation items are scored on the following Likert scale: 1 = strongly disagree;
2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree
Given how moving the class online did not affect students’ perceptions of their learning
experience, this suggests that the eLearning mode reached most of the same objectives
as the face-to-face mode. Students were overall receptive of this innovation. However,
students felt that the online instructional mode adopted during the eLearning week
did not provide the opportunities to develop verbal communication skills, which they
would have had in a face-to-face TeamLEAD session. This perception was likely due
to the communication during the class being limited to text-based exchanges, rather
than other factors. These findings indicated to the educational leadership that a
sustainable online version of this module would need to provide avenues to develop
verbal communication skills.
Transitioning TeamLEAD from Face-to-Face to Online in 2020
When the National University of Singapore mandated that all classes be moved online
in February 2020, the educational leadership deployed the infrastructure for online
learning that was in-place after the 2015 eLearning experience. While the 2015 eLearning
experience was a week-long, in 2020 all classes were moved online indefinitely. To
ensure successful implementation and continuity of TBL online, constant feedback was
sought from students, faculty and the administrators through student feedback surveys
and regular faculty debriefs.
Keeping in mind that students preferred to have verbal communication during online
classes, in 2020, students were instructed to log on to a video-conferencing software
for the entirety of the TeamLEAD session (Zoom.us, San Jose, CA, USA). This proved
to be a useful medium to dispense instructions to the class and for facilitated discussions
during mTRAP. Students chose their preferred platform for communicating with their
teammates during GRAT. This may be an alternate video-conferencing program or instant
messaging platforms on their mobile phone. Although the breakout room function in
Zoom has proven useful for online TBL [10], we found manually assigning 82 students
into 13 teams to be cumbersome. Additionally, using a separate communication platform
allowed teams to be connected even during class-wide discussions. This helped us to
recreate the “side-discussions” amongst teammates that were frequently seen in class.
Similar to the eLearning experience in 2015, students used a university-sanctioned,
password-protected, shared online document to submit their queries during the mTRAP.
Teams assigned to answer those questions responded on the same document. To enhance
communication, the assigned teams are instructed to present their answers through
the video-conferencing tool. After which, the facilitators gave the class time for
an open discussion. Students could take this time to interact with the faculty through
video. This provision was crucial as, according to the social cognitive theory, individual
consciousness can only be formed through communicative interactions [11].
In online TBL, the role of the facilitator took on a different form from face-to-face
TBL [12–14]. In brief, the facilitator ensured that all the participants were muted
throughout the session and that only one person speaks at any one time. Students were
allowed to key question into the text-chat function of the video-conferencing platform.
The facilitator had to be cognizant of these different prompts and channel them accordingly.
The facilitator’s ability to manage the participants and all the key processes in
a time-sensitive manner was key in creating a conducive online learning environment.
The Education Administrative Team Played a Crucial Role in Implementing Online TBL
in 2020
The backbone of the online classes is the administrative team from the Office of Education
at Duke-NUS that helms the logistical demands of TBL. This team of 7 administrators
was pivotal in the transition and ensuring the continuity of the curriculum during
the pandemic. Prior to the classes, the administrative team was responsible for maintaining
the online learning resources and granting access for all participants. They were
also responsible for conducting training sessions for students and faculty on how
to use the various online platforms and helping users to resolve technological problems
in real time [14]. For example, they worked with hospital-based faculty to overcome
the poor Wi-Fi access in their institutions. Three to 5 administrators would attend
each class session to provide technical support. This included monitoring the attendance
of all participants, cueing students on when to begin each phase of the assessment
and communicating the scores and item analysis of the RATs to the faculty. In addition,
the administrative team was also responsible for troubleshooting any technological
issues. They did so by remaining contactable via multiple communication modes during
the class, through email, instant messaging or phone calls. Examples of contingencies
that they had in place include sending soft copies of the resources through email
and giving instructions or acting as a conduit for student-faculty discussions through
the phone.
SWOT Analysis of the Online Learning Experience of 2020
Strengths
The greatest strength of this online TBL is the ability to provide an effective alternative
TBL format to ensure continuity of learning amidst a global pandemic. Students were
still able to engage in an interactive intellectual discourse with peers and faculty
[15].
Additionally, in our online iteration of the mTRAP session, we used a combination
of text and video. The real-time shared online document created a dynamic mTRAP session,
due to the ability to edit and comment on responses during the discussion. This encouraged
students to remain engaged throughout the discussion. Having the mTRAP hosted on a
shared online document is an improvement from face-to-face sessions, as the information
is now stored electronically for the entirety of the module, instead of being written
on a whiteboard and erased at the end of the session. Having an electronic record
available for an extended period also allows faculty to insert their comments retrospectively.
This feature was important as clinical faculty, faced with increasing clinical demand
during the pandemic, were not always available during the virtual class time.
The combined use of a shared online document and a video-conferencing platform for
the facilitated class discussion during the mTRAP is a unique feature of our iteration
of online TBL. Based on the cognitive theory of multimedia learning, the simultaneous
use of text and video enhances the learning experience. This theory posits that the
brain interprets data from multimedia sources in an organized and dynamic manner,
resulting in the production of logical mental constructs [16]. With this, we provided
students with a dynamic and interactive learning environment [17].
Weaknesses
Difficulty in getting access to a reliable internet network or internet-enabled devices
posed a major obstacle for all participants. Perhaps to ensure that students can carry
out online learning, the school should consider supporting their access to devices
that would allow for the various processes of online learning. For the clinical faculty,
they are in locations where internet separation was practised. Due to the Singapore
Health Services being breached by a series of cyber-attacks in 2018 [18], all workstations
in hospitals were disconnected from the internet. To overcome this obstacle, faculty
participating from the hospitals did so from their personal devices.
Technological literacy ability of participants was another challenge that had to be
overcome. Some participants were unfamiliar with the video-conferencing platform or
unsure about how to access the online documents. We overcame this through the efforts
of the administrative team who provided pre- and in-class technological support.
The strong dependence on the administrative team also presents as a weakness with
our iteration of online TBL. Without dedicated staff to manage the learning process,
the faculty would face an uphill task in teaching and running the logistics of the
course. Institutions interested in implementing online TBL should train an administrative
team to ensure the smooth running of the course or ensure that there is a co-host
available to help with the administrative demands of the class.
This format of online learning may suit the teaching of theoretical knowledge well
but may not be ideal for the teaching of practical skills, which is an important component
of medical education. Faculty have also expressed concern about the inability to assess
professionalism through online learning. Avenues to teach practical skills and assess
professionalism online need to be developed to overcome these obstacles.
Opportunity
The potential for online TBL is far-reaching. While initially deployed to facilitate
remote learning in a pandemic, this mode of learning can connect students with faculty
who are not in the same geographical locality. This will be highly useful in teaching
niche subjects where few experts are available.
There is also an opportunity to develop a single online platform that can house all
the different features of online TBL. Currently, students switch between multiple
platforms for each phase of the lesson. An all-encompassing video-conferencing software
which contains features that allow the administration of the IRAT/GRAT, mTRAP and
discussion processes will be ideal. Such features should include the ability to administer
tests securely and store shared files.
Being able to carry out online TBL will give the institution the capability to conduct
remote learning TBL courses. These can be credit-bearing courses which are open for
any student regardless of location. Most major universities offer online courses,
but these courses are predominantly lecture-based modules with limited collaborative
learning capacity. Having this option will increase the repertoire of courses that
reaches the unique learning objectives reached by TBL and appeal to the segment of
learners who prefer TBL to traditional learning.
Threats
While preliminary observations (Table 1 and Supplementary Tables 1 and 2) suggest
that learning can occur through this modified online TBL format, it is unclear whether
this mode of learning is sustainable and/or achieves the same the educational objectives
as face-to-face modules. Taking the tests online and unsupervised presents a tangible
threat to the learning process. Students with integrity issues might not adhere to
the university’s Honor Code. Testing higher-order thinking skills and developing secure
exam-taking software may mitigate this shortcoming.
A worrying trend observed with the increased use of video-conferencing is hacking.
Incidences of “Zoom Bombing” have been reported by multiple institutions globally,
where hackers gained access and interrupted ongoing online classes [19]. If this trend
escalates, confidential data might be compromised. In response to these intrusions,
the university had suggested several guidelines to increase the security during video-conferencing
sessions [20]. Video-conferencing software providers must maintain vigilance and continuously
work to enhance security in order to prevent unauthorized interruptions.
Moving Forward
The data collected from the 2015 eLearning experience was instrumental in our response
to the COVID-19 pandemic in 2020. The concerns raised by the students in 2015 guided
the leadership in modifying the online format for the 2020 roll-out. In 2020, we found
that we were able to reproduce the face-to-face TBL conditions online by utilizing
the video-conferencing tool judiciously. Furthermore, students in 2020 generally appreciated
the online TBL format of learning and found it easy to communicate with their peers.
Given our observations, further studies comparing the impact of online and face-to-face
TBL classes on student’s academic performance will need to be carried out. These studies
would also need to ascertain if taking an online course for a prolonged period has
an effect on student’s well-being and mental health. Overall, the preparedness of
the educational leadership and the dynamic work ethic of the administrative team allowed
for the successful deployment of online TBL.
Electronic Supplementary Material
ESM 1
(DOCX 26 kb)
ESM 2
(DOCX 22 kb)