Print Page   |   Contact Us   |   Report a Problem   |   Sign In   |   Register
Community Search
News & Press: News and Updates

TBLC Global News Volume 10 Special Edition 1

Tuesday, June 23, 2020   (0 Comments)
Posted by: Cassie Chinn
Share |

President's Message

A Message from Michele Clark
Implementing Scratch Offs for the Team Readiness Assurance Tests (T-RAT) in an Online Environment
How to Use Canvas to administer the iRAT and tRAT and file grades into the gradebook
Challenges of Online Application Exercises
Facilitation Skills in Team-Based Learning


President’s message



A Message from Michele Clark

The TBLC heard you when you requested information on how to integrate TBL into an online environment during the crisis of COVID 19. We have already had two webinars on strategies to implement, putting TBL online. 

For our first Special Edition of the newsletter, we have collected articles from our members to share the technology or resources they use to make sure the three phases of TBL are present in online modules. Our European and Asian Pacific members have contributed to our second Special Edition coming out in a week. The second Special Edition presents solutions for typical challenges encountered when using online TBL for the first time. 


*Our European and Asian Pacific members have contributed to our second Special Edition coming out in July.*

It is not difficult to create a readiness assurance Test (i-RAT) in most online Learning Management Systems (LMS). Most of these programs allow you to create quizzes and make them available at a specific time or during a range of times. This flexibility is the same for the ability to shuffle answers, gives a time limit to answer the questions, and how many attempts you provide students to answer the question are a few that can be used. We recommend that students get one attempt to answer the items before they submit the quiz. They will learn how well they did with the questions when they complete the tRAT. In addition, after the tRAT you can make the quiz score and items available to review.

Michaelsen’s outcome goal for the iRAT is to provide students with incentives to tackle the pre-class learning seriously.  That goal does not change in an online environment. However, you will not be able to assure the students are not cheating unless you set up a formal monitoring system. This monitoring can be expensive and time consuming for the student. There are some strategies you can elicit to maintain Michaelsen’s goal to assure students have a good understanding of the content being tested. Limiting the time students can answer the questions as well as randomizing the delivery of the questions will help some. However, consider writing items that apply some of the content to a situation. Even if the student is looking up the facts needed to answer the questions, the item will require them to understand how to use the content. These questions must be simple applications—higher-order thinking of analysis, evaluation, and creating are used in the application exercises.

Finally, the number of points for the quiz should not exceed the tRAT.  The number of points for the iRAT will need to be in alignment with the difficulty of the test. Remember, the goal of the iRAT is to ensure there is enough understanding of the content to be an active and engaged collaborator for the tRAT and application exercises. 



Implementing Scratch Offs for the Team Readiness Assurance Tests (T-RAT) in an Online Environment

Paul Erickson, Michele C. Clark, Sue Armitage
University of Nevada Las Vegas

Team-Based Learning (TBL) offers students active engagement and opportunities to practice higher levels of learning. When using this evidence-based teaching strategy the readiness assurance process is a critical activity to prepare students for the application exercises. However, when incorporating one of TBL's important principles - frequent and immediate feedback in an online platform - one encounters challenges. It is not difficult to create the Individual Readiness Assurance Tests (I-RAT) in most online platforms. However, creating the T-RAT presents unique challenges, especially for an asynchronous online course. The goal for the online program using TBL was to create an online scratch off that would provide students with immediate feedback on their answers for the T-RAT.

To accomplish this, the UNLV School of Nursing and the UNLV Office of Online Education decided to implement the activity in three components:

  1. A user interface that was originally developed in Adobe Flash and has now been converted to mobile friendly HTML5.
  2. A database application to store asynchronous responses.
  3. A BlackBoard component for interactive group discussions.

In short the online T-RAT scratch off works like this:

  • Questions and responses are stored in the database. A link to these questions is available in the Learning Management System (LMS).
  • When a group is ready to participate, members login in to see the questions and discuss them in their group discussion area in the LMS. On behalf of the group, one student is selected to answer the T-RAT questions.
  • When the group members reach consensus on the answers, the selected student returns to the T-RAT to answer the questions. If the group members answer to a question is correct on the first attempt they receive 5 points and move on. If the groups members' answer is incorrect they can stop, go back to the discussion area, then come back when they feel they have the correct answer. For each subsequent incorrect answer, the point value for that question decreases by two. Therefore, the group receives 5, 3, 1 or 0 points depending on when the correct answer is chosen.
  • Throughout the process the group's responses are recorded in the database and the T-RAT can be stopped and returned to at any time. When group members return, the database returns them to their current question with their previous attempt to answer that question already marked. This ensures they always return to the same place where they left off.

Since responses (both incorrect and correct, and in the order chosen) are stored in the database the instructor can review these data in a dashboard to determine which questions the students are having difficulty with, and which responses may be the most misleading.

Once the group has completed all of the questions they can return to the discussion area to decide if the appeal process would be appropriate for any of the questions.

Videos on how to set up RATs (iRAT and tRAT) in an online environment using Qualtrics

Neal Carter from the department of political science at BYU shared two videos on using Qualtrics for setting RATs in an online environment.  Neil thinks the iRAT works better for point spreading in Qualtrics than in forms since you can control the total points spread.  He hope both videos is helpful to everyone.  iRAT point spread


This video is instruction on how to set up a tRAT in qualtrics.

tRAT Qualtrics



How to Use Canvas to administer the iRAT and tRAT and file grades into the gradebook


Before the tests

  1. Create the iRAT as an ungraded survey. Use student submission of the iRAT as an indication as to whether the student prepared ahead of time and also participated in the tRAT. Adjust Canvas’ settings to shuffle answers, allow only one attempt, and do not show the answers upon completion.
  2. Make two copies of the ungraded survey in #1. Assign the first copy to team recorders (the person responsible for filling out the team’s forms). Assign the second copy to all other students. Adjust Canvas’ settings to NOT shuffle answers (important, so that every student sees the same order of questions during the tRAT process). Also adjust the settings to allow three attempts, see questions marked wrong, and get the average of the grades from these three attempts.
  3. Create a “no submission” assignment for all teams. (This is a placeholder for the tRAT score.)

After the tests

  1. Change the form of the test under #1 to a graded quiz. Note the students who did not submit this test and thus are ineligible for a tRAT grade.
  2. Determine the scores from the ungraded surveys submitted by the team recorders.
  3. Manually assign grades for #3, using the grades shown in #5 for all students who participated in the iRAT.

Advantages of this approach

  •  With the exception of step #6, direct involvement by the instructor is minimal.
  •  All grades flow to Campus’ gradebook.
  •  All students see the same order of questions and answers on the tRAT, facilitating team discussion.

Editors’ notes:

If you are conducting synchronous classes where the teams can take tRATs within breakout rooms, #2 could be conducted with only one ungraded survey with a team recorder sharing their screen and entering the answers once a decision has been made. As an alternative to using ungraded surveys, each question in the iRAT vs. tRAT quizzes can be given different point values (e.g. in a 10 question quiz where the iRAT is worth 40% and the tRAT is worth 60%, you can assign each question 0.4 points for iRAT vs. 0.6 points for tRAT).


Challenges of Online Application Exercises


Caroline Wilson*, Ethan Vieira (Zoom videographer)*, Caroline Rinaldi 

*Chapman University, Health Sciences Department, Crean College of Health & Behavioral Sciences. 

UT Southwestern Medical Center, 


As TBLC membership committee members, we have begun thinking about our approaches to TBL online and wanted to share them with you, with the focus here on application exercises.  For many of us, the switch to creating Team-Based Learning (TBL) application exercises online has been necessarily quick and unplanned. Along with our own experiences, we have asked our colleagues for advice as we proceed with these new, exciting, and sometimes daunting experiments. Caroline Wilson, at Chapman University, is finishing up the semester of ½ in-person TBL and ½ online TBL, and she will first talk about her challenges and advice in Part 1. In Part 2, Caroline Rinaldi, from UT Southwestern, will discuss how her colleagues have adapted to some of the challenges online TBL with a large class size.


Part 1:
At Chapman University, our first challenge was to decide whether the students would participate in a synchronous (videoconferencing live during traditional class time) or asynchronous TBL (students work on their own, outside of the traditional class time).  I chose the synchronous approach to teach my courses for two reasons.  First, students worked well together earlier in the semester, and already understood the basics of this active learning, flipped classroom approach. Second, when students were surveyed, a majority voted to retain “live” class sessions. Live applications made for the capability to maintain some of the “normal” TBL flow, but updates were still necessary. 


To reproduce the TBL application process, a video conferencing tool had to be selected that allowed teams to form their own opinions privately (intra-team discussion) followed by simultaneous reporting, discussion across teams (inter-team discussion), and closure. At the beginning of the online course transition, my university did not mandate how we would meet with our students. We had three different video conferencing tools available to us: Canvas Big Blue Button, Zoom, and Microsoft Teams. I initially tried the Big Blue Button to integrate more Canvas tools. However, there were several students who experienced issues (e.g. getting dropped and audio delays). After a survey during spring break to see how to proceed, students voted to use Zoom, probably because other professors were using it. Though I have not tried Teams, allowing students a voice in this process has been one key to success. 


Once the video conferencing tool was determined, I reevaluated the application exercises I designed.  Many of the cases we use for my Applied Human Neurophysiology course are from the National Center for Case Study Teaching in Science and have been adapted for TBL to adhere to the 4 S’s. The 4 S’s include: 1) Significant problem (big picture, not trivial, not easy to answer with a web search); 2) Same problem (students all working on the same question at once); 3) Specific choices (you come up with 4-5 possible answers instead of open-ended questions where students can go off-track); 4) Simultaneous reporting (teams reveal answers to application exercises all at once). The application exercises were largely the same as they would have been in-class except we couldn’t distribute the questions as paper worksheets. Instead, the  application questions were created in a Google document (or any other linkable or downloadable file source) and the link was provided to each team through their LMS (Canvas) so they can look at the item in a separate browser window. The questions were not released until we worked on that application exercise. The questions all had a short, clinical vignette, and at the end, there were 4 or 5 answer choices.


To recreate intra-team discussions, I utilized Zoom Breakout rooms. Students were introduced to a case in the “homeroom” and then I opened the breakout rooms. The value of breakout rooms is that you can create a timed session, so after 10 minutes, the students return to the homeroom to report their answers. In addition, breakout rooms allow you to join different teams to see what the students are discussing, just like you would if you were roaming around a classroom in person. You can also monitor the breakout rooms to guide your facilitation, to identify if all student voices are heard, and to note when the conversation has moved on. As in in-person TBL, you will never hear everything every team says. You may consider emphasizing the role of peer evaluation to encourage all team members to engage in the online TBL environment.  The TBL online white paper gives suggestions for how to create online peer evaluations. 


When the breakout room was closed, students returned to the homeroom to report their answers. How do students adhere to the 4th “S”, Simultaneous reporting? Students can report their answer simultaneously in a couple of ways: 

  1. Utilizing the poll feature in Zoom
  2. A team reporter types in the team choice by typing “A,” “B,” etc. into the chat. 
  3. Holding up their cards with the letters “A,” “B,” etc. in the video window. 
  4. Using a different software where only the professor can see the results (e.g., Poll everywhere)

Facilitation of the inter-team discussion followed, much the same way as it was in live TBL sessions.  Just like a normal facilitation process, there are challenges. One challenge is getting anyone to speak, especially since the default setting is often to be “muted” while the professor speaks. My colleague here at Chapman, Jennifer Totonchy, uses Zoom polls and Yes/No questions to initiate the facilitation process (although integrated polling is anonymous). Another issue is that certain students always answer the questions, so consider asking particular students to provide answers. Also, you can have an ordered list of students so that they know when they may be asked a question or rotate between team leaders who are reassigned each class period. You can also encourage your quieter students to type their answers in the chat if they don’t want to speak (or can’t if their environment is noisy). Maintaining a “poker face” during the team discussions and allowing teams time to come up with the answers before helping, can also be difficult in the online environment. You must also consider whether the application exercises will be graded or ungraded. Mine are ungraded, but if they are graded, you could create the application exercises within the LMS quiz/testing feature.


Following the facilitation of the question, you will want to properly close the discussion. Many of my colleagues have found this step is especially important in the online environment and that reminders, summaries, and conclusions are critical in keeping the students on-task and to emphasizing your learning objectives.


While this description may sound straight-forward, there are challenges to using breakout rooms. If you have a small number of students, you can assign students to the Breakout Rooms when they first log in. To assist in this, suggest students change their Zoom screen name to e.g. “Team 1_Name” so they are easy to sort.  Note if the students log-out of Zoom or are dropped, you can’t assign them to the room until they return, and you will have to reassign them each time they are dropped or leave Zoom.  If you have many students, you can pre-assign the students to a room by uploading their names and emails (.csv file). Still, the student emails must match their Zoom registration for the pre-assignment to work (see also Part 2 of this article). Zoom can randomly assign teams, but if you are trying to retain teams throughout the course, this choice is not ideal. Zoom has more information on its website, including how to pre-assign students, limitations on participant numbers, and differences between licensed and free accounts. 


Another option is to include asynchronous applications.  One possibility is to have a “gallery walk” style answer to an application question. Students can create a drawing, diagram, or concept map answering a question and then post it to LMS discussion boards. Then the professor can create a survey or poll and have the students vote for the best answer, as they do during in-person gallery walks. Students do not have to use Zoom to collaborate in the asynchronous environment. For instance, you can allow them to choose social media platforms like Google Hangouts, Facebook messenger, or Facetime. These platforms also provide a “Plan B” if Zoom breakout rooms are too hard to manage. My colleague also utilizes these non-Zoom tools so the teams can still chat privately during the inter-team discussion. The disadvantage is that you may not be able to observe these discussions if the platform is private. 


Online teaching has been an interesting experiment, and I think (hope?) the students in this course are enjoying it (You can see some examples of the classroom flow in the slides my student, Ethan, helped create). It is A LOT of work to set up a course in this way, but it makes for very fun online sessions that are not just lecture-based. Getting feedback from the students on what works well and what does not is essential. Continue to ask for feedback as the course unfolds. 


Part 2: 

Moving TBL to an online format at UT Southwestern Medical Center (UTSW) in Dallas has been complicated because of our large class size of 240 students. At UTSW, our students are assigned into TBL groups that are maintained throughout all of the preclerkship courses. We have found it particularly difficult to maintain these permanent TBL groups while hosting TBL sessions in a synchronous online format. For synchronous sessions, UTSW has licenses for both Zoom and Microsoft Teams. However, Zoom Meetings, which allows for breakout rooms, limits the total number of students to a maximum of 200. My colleague, Jim Wagner, has addressed this issue by dividing the class into three groups (he leads a particular session three times).


Another challenge in hosting synchronous TBL sessions, even with some of our smaller classes, has been the issue of successfully and reliably pre-assigning the TBL groups to Zoom breakout rooms. As Caroline Wilson mentions above in Part 1, even if you pre-assign your students into Zoom breakout rooms, there are a number of challenges to using this feature of Zoom. For example, if students set up their Zoom account with a different email address than what you used to assign the groups, or if they try to log in with a different account, it will not put them into the breakout room. Therefore, it is good to have a plan in advance for how the host will quickly get them into the proper breakout room.  When students properly log in, they can break out, come back to the large group, and breakout again into their preassigned groups with no problems. However, one disadvantage of Zoom breakout rooms is that students cannot chat among their small groups when brought back into the large group. This makes it difficult for the small groups to participate in the larger group as a team (inter-team discussion). My colleague, Peter Michaley, developed a workaround for this issue by having students simultaneously use Zoom for the large group discussion and Microsoft Teams for their small group session. When using this approach, the students run both apps simultaneously on their devices. Whatever videoconferencing apps you choose, it is a good idea to make sure your institution has a FERPA compliant license for it, and that you are not requiring students to use apps that do not meet privacy policies. At UTSW, we can allow students to meet in small groups with apps other than Zoom and Microsoft Teams (the apps for which we are licensed), but it must be voluntary and agreed upon by all students in the group. Here is an example of an online TBL schedule (our learning management system is Brightspace by D2L):

0:00 - 0:10    iRAT (Examsoft; provides attendance information)

0:10 - 0:30    gRAT; Students enter answers via D2L (Teams)

0:30 - 0:45    Discussion of the RAT (Zoom)

0:45 - 1:25    Students work applications; enter responses via either PollEverywhere or D2L (Teams)

1:25 - 2:00    Discussion of applications (Zoom)


Online Facilitation Skills in Team-Based Learning

The coronavirus has severely impacted millions of people around the world.  With fears developing and social distancing continuing, students have changed the way they are pursuing higher learning.

You can enter that online learning environment.  Join students where they are. 

As you get ready to facilitate any online Team-Based Learning class, there are a few teaching techniques that you can copy from your days in the classroom.

  • Prepare engaging team application exercises that drive critical thinking and foster robust discussion
  • Call on as many teams as you can in every discussion—so that you facilitate conversations among teams and not with you and these teams
  • Pose challenging questions that encourage deeper reflections in the team application exercises
  • Refuse to share your opinions and avoid playing the role of the “sage on the stage”
  • Challenge students to listen, to think, and to explain—and guide them to a more thorough and deeper application of key concepts
  • You already know that the further away you stand from any student speaking for their team, the more likely that individual would connect with other students from different teams. So, use a similar technique and stay away from engaging in the discussions.  Help teams dialogue with each other and not you.  Call on teams that are not fully engaged in the discussions or which are geographically furthest away in time zones! 

You will find that can still rely on some key questions to further deepen the pursuit of learning and understating.  These questions can work in any learning environment:

  • Did any other team agree with this answer?  Why did you agree?
  • Could we hear from another team which may have disagreed?
  • Are there any other options for a correct answer?
  • What might be wrong with the answers you have heard so far?
  • Based on this discussion, what do you think is the best answer?

There are some different techniques you can use that will help you develop as a polished facilitator in an online environment. 

  • Recruit and engage an individual who can monitor the conversations and questions in the chat box, help students with any technical questions, and place you in different breakout rooms as you monitor team conversations.
  • Use the features of the online portal to engage students frequently.  The chat box can allow you to pose a question and get feedback from all students, even those who tend to be reserved.  A poll can be used to take a short mental break or welcome students when they first arrive in the online classroom.    
  • Call students by name or by team name/number—as many students can be seen on the screen with the name they have revealed in their profile.
  • Know that some discussions take a bit longer to develop in an online environment.  Be patient.  Give students time to master the online portal they are using.  Allow teams time to get to know each other.
  • Audit an online class facilitated by someone else proficient in this environment.  Watch how they facilitate rich discussions.  Learn from them.

In a world currently impacted by the coronavirus—and driven more into virtual learning--the time you take to develop yourself as an online facilitator will be so worth the investment!  Take the time now.  And discover the power of effective online facilitation.

more Calendar

The Essentials of Moving to Online Team-Based Learning

Featured Members
Ruth LevineTBLC Consultant
Marty EngTBLC Consultant

Online Surveys
Copyright Team-Based Learning
Membership Management Software Powered by YourMembership  ::  Legal