Quizzes after lectures

Question

I put together elaborate, very carefully planned out lectures, complete with highly distilled exemplars, student teach-backs, and mid-lecture activities to check for understanding. I've had feedback that my students really enjoy these lessons and appreciate how much work I put in to making sure that my examples are really clear.

However, it has come to my attention that, perhaps due to a course structure which gives students lots of chances to re-demonstrate mastery if they initially struggle, a certain number of the students are only half-engaged with my lecture. Since I realized this, I have spoken to a few of them, and they agree that the lessons are fun, but say that they are stressed by other courses, and sometimes this causes them to work on other things.

On one level, I don't mind this. Student stress is real, and I have ample evidence to demonstrate to me (or to others!) that the students ultimately master the material that I teach them.

On another level, however, being lax about this does a disservice to them (as we could progress faster for everyone if kids took the learning opportunity more seriously), and to me, as I have put almost unreasonable amounts of time into distilling rich and complex ideas into explanations that are both beautiful and clear.

I'm further motivated by the fact that I get evaluated by my administrators, and I have now been dinged twice by having some of the kids off-topic for short bursts.

I'm toying with the idea of adding little, basic quizlets to the ends of my lectures for which students would be immediately responsible.

The questions would purposefully not be difficult. They'd be more on the level of, "were you paying attention to what we just did?" Basic recall, simple factoids, etc. I already give a difficult test or quiz at the end of the unit, so I don't need this quiz to do much more than (1) give students a chance to dredge up information from memory¹ and to help motivate students to take fuller advantage of the lesson opportunities I am giving them.

My question, then, is two-fold: (1) are quizlets a good solution to my problem? And (2), if they are a good solution, is there any wisdom from the trenches on how to pull these together so that they are effective without bringing on substantially more grading work? I'd like to make them as useful in the learning process as they can be, but I have to be careful about what I take on.

^{1 - See points 2, 3, and 4 for how this directly enhances learning.}

Answer 1

I have a few thoughts on this topic. One is a couple ideas about how quizzes could be implemented (if you choose to use them), and the other is some more general thoughts on the tactics that I use to solve problems like this.

Quizzes Without Extra Grading

What are your opinions on peer grading or self-grading? I've seen both work nicely for things like this (short, low-pressure quizzes that aren't too hard).

They require a little bit of logistics, since each student needs a normal colored pen and a "grading" pen, and it's easiest if you supply those. Then you require that normal pens are collected before grading starts, and that students do not have their own writing implements out, and it typically goes well.

This would require you to put aside more time during class, since a 5-minute quiz takes at least an additional 5 minutes to grade.

The pro of students grading their own work is that it becomes a way of reflecting and learning from the experience. If they get a question wrong, they get immediate feedback about the correct answer.

Alternatively, you might want to look into giving digital quizzes. They would have to be multiple choice or fill in the blank, but they auto-grade, saving you the time. I know that Google Forms has a method of doing this, and there are a bunch of other platforms too. Your students could take the quizzes on their phones, or computers.

This system tends to be quick and you can integrate it into the entire lesson, not just the end. I had a professor in my undergrad who would lecture for a bit and then pause for us to answer a question or two. Before continuing. It was her way of doing understanding checks. You could easily establish small point values for each question so that they "count."

To Quiz or Not to Quiz

Personally, I've found that increased examination does not tend to increase student attention. I theorize that there are two reasons for this.

First, many quizzes happen "later" - either at the end of class or even several days or weeks later. Most students (especially teenagers, although adult students as well) have trouble remembering potential consequences that seem far off. If the exam is important enough to overcome this problem, then it's probably invoking some level of anxiety or fear, which is a large detractor from student focus and engagement. It may solve your problem of looking like your students are paying attention, but I believe that it's unlikely to actually increase their learning.

My second theory is that people tend to believe that they are better at multitasking than they are. If you gave an anonymous poll to your students and asked if they ever do work for other classes during your lectures and how much attention they pay to your lectures, I would bet quite a bit that they all feel that they are always attentive, even though you know that not to be the case. Thus, saying "Oh, it's very important to pay attention!" seems unlikely to change behavior because they really believe that they are/can pay attention while doing other things.

So, what are the other options? You could do the honest but maybe frowned upon thing of talking to your students and saying "hey guys. I know you're busy. I know you sometimes finish up other homework while in this class. I'm mostly fine with it on occasion; as long as you get your work done and feel like you're learning what you want to learn, I don't really see a problem. But it makes us look bad when a senior teacher/dean/etc. sees you doing that. So we have some options. I can change something about the way the class is structured (quizzes, etc.), or you can keep the distractions to a minimum, especially when we have guests." Whether this will work mostly depends on your students and your relationship with them. It also obviously depends on the institution you teach at.

A Non-Quiz Suggestion

You've mentioned that you're already putting a lot of thought and work into every lesson. So this might not be a solution that seems feasible. But my personal strategy for student engagement is to keep the students active throughout the lesson. I try to talk for 5-10 minute chunks, broken up by having students work through practice problems or explore ideas. Maybe you can take your current activities and spread them out a bit more throughout the lesson? If students are changing tasks frequently, it will be easier for them to stay engaged and harder for them to pull out something else to become involved in.

If you want more concrete information on what this could look like, pick an example lesson and post another question. I'm sure that lots of people will have a wide range of ideas for active learning.

I hope some of these things help. Good luck!

Answer 2

Think about it a bit more generally. If all you want is to force their engagement, then it will probably work to some extent, with most of them.

But if you expand the goal you can do much more.

A quiz immediately after hearing/seeing something only really tests short term memory, not true learning. An exercise, rather than a quiz would force engagement, but do a bit more in aiding reinforcement.

A quiz the next day, based solely on the previous day's material, given at the start of class, where they know it is coming, will give them a chance to let the information settle, especially if there was an exercise overnight that tried to solidify the material. These quizzes an count for a small, but non-trivial part of the grade.

I would seek to integrate any quizzes into a longer term learning strategy for the students, rather than using it for any simple purpose. Students need to take notes, not just listen. They need, however, to take effective notes, not just try to capture everything.

If students always have a few index cards available (a Hipster PDA) you can ask, at the end of any class for the most important ideas of that class, based on what they have written on their cards. You can accept or reject suggestions. You can make sure there is agreement and that the other students "get it". You can call on students to give "big idea of the day" suggestions, rather than asking for volunteers. You can award points for good/great suggestions (and note them on your own note card for the student).

The next day you can review by asking for the big idea of the previous day, or the main theme or big idea of the previous week.

But, get them to write short summaries of the lecture with pen and paper, capturing what is key. This act of writing is a kind of reinforcement.

You can even hint about the big ideas as you go along. This is easiest to manage if you lecture facing them, rather than a whiteboard.

More elaborate schemes are possible, such as asking every student to submit a half page summary of each lecture at the beginning of the next class. You have to scan these, of course, and you have to guard against students copying them from each other (to some extent).

---

Alternative to the quiz/note idea is just to make the entire course more interactive, such as a flipped classroom. This forces engagement as you aren't lecturing, and they are actually doing things that reinforce learning. You can do that instead of quizzes, or you can blend it a bit.

Answer 3

I'll address the second part first: These quizzes are meant to make sure students were listening and paying attention. As such, any partially completed quiz indicates partial listening (given that the quiz questions are along the lines of "name one thing/point/property/etc. I mentioned while discussing <some subject covered during the lesson>"). This means that a student who paid any attention would be able to answer the entire quiz correctly, which leads to this: grading is easy, as it is somewhat binary here.

Also, the quizzes need not affect final grade. This can either be said to the students, or left unsaid. Each option has its pedagogical benefits (personally, I would tell them sometime mid-course, so that by that time they already subconsciously see a need in paying attention, even if it doesn't directly affect their grades).

Returning to the first paragraph of this answer: a condition for quizzes to be a good solution. Questions in the quiz should be answerable by someone with very minimal knowledge of the subject. All the answers to the questions should appear nearly explicitly in the lecture itself.

However, a question shouldn't rely on the students' ability to remember all the details. As an example (the one from the 1^st paragraph) "name one thing/point/property/etc. mentioned while discussing <some subject covered during the lesson>" can be answerable by a student who listened and remembers that you talked about foo while teaching about bar. They might not remember you also mentioned baz, or qux or some other ones. But if they got, say, more than 2 out of the 6, then it's safe to assume that they were listening.

This is just an example for a quiz question, but the idea is that minimal understanding of the material is more than enough to "get full marks".

For building the quiz, you can request help for a member of a different faculty. Or perhaps simply try some quiz, and see how well it is answered by the students that usually do listen (those who sit in the front row, or those who participate). This should give some indication whether your quiz serves its purpose.

Answer 4

Yes it is a good idea to review regularly

Memory decays over time, but each time in is refreshed, it will last longer. A rule of thumb is to revise after 1 hour, less than a day, ½ week, 1 week, 1 month, 3 months, 1 year, 10 years, 30 years, then you will never forget.

see also https://en.wikipedia.org/wiki/Forgetting_curve

but how to do it.

Non-summative

Summative assessment is not good: while formative assessment has a positive affect. Summative assessment has a negative effect, even when combined with formative assessment. Therefore we need to eliminate summative assessment.

If not summative then what?

Summery approach

At https://courses.exa.foundation/ the author, recommends that we get our students to study for them self, and then to summarise what they have learnt on a single sheet of plain paper. Mostly pictures, few words, and fast to assess:

All pages must be numbered, 1 to 40 with one topic per page, per week.

Each homework must meet the following 5 requirements:

1. Write the complete title and date in full eg. Tuesday 12th September 2017 on each page, underlined
2. You should include a minimum of words to summarise the topic. Do not copy the words from the text.
3. Make full use of the page for each topic by scaling your notes & images appropriately to use of all the space.
4. You must include diagrams, sketches or cartoon doodles to visually represent the topic, try to use humour.
5. Highlight key words and phrases, using underline, highlighter pens. Explain technical terms.

Answer 5

It sounds like that you want a quick, simple solution to the issue, and while quizlets might not 'maximize learning' they can be quick and simple. I would suggest a quick warmup (i.e., they come in, and five minutes after the bell rings they turn it in, so maybe four quick multiple choice questions and one short response) done on Google forms [since you're teaching CS, i'm assuming you have tech in the classroom].

Google forms are uber easy to set up and can be set to automatically grade, meaning that all you need to do is skim through the results and check the short answers yourself. I wouldn't grade them, but instead set up a 'must do well on a minimum of four warmups a week or else you talk to me' system [remember, these warmups are meant to be fairly easy if you were paying attention]. I've had a bunch of teachers use the warmup system, and it effectively forces people to recall the previous day, and get them in gear for that day.

If google forms isn't a great option for you, here's an alternate: hand out a warmup sheet to everybody for the week, and at the beginning of the day put up e.g. a powerpoint slide with your five questions. Then, when the five minutes are up, come around with a stamp and stamp the ones that are good. At the end of the week, have them turn it in, and just count number of stamps. That might help reduce the grading effort.

Answer 6

I'm toying with the idea of adding little, basic quizlets to the ends of my lectures for which students would be immediately responsible.
The questions would purposefully not be difficult. They'd be more on the level of, "were you paying attention to what we just did?" Basic recall, simple factoids, etc.

I'd advise against this. While it sounds like it directly addresses the problem, it's liable to backfire and exaerbate the issue.

While this test would create an obstacle for the willful slackers, it's also going to become an obstacle for students you're not intending to target: students having an off day, students struggling with the material, students who doubt their own skills until they feel they've got a grasp on things, ... These students are all going to feel pressure during your lecture knowing that an inevitable quiz is coming at the end of the lecture.

I suspect your response to this may be along the lines of

"I'm only going to ask them to repeat the lecture, not to have them show they fully grasp it."

But then your quiz is going to be about the lecture itself, not the material that the lecture is about. A student who is ahead of the curve and already understands the current topic might be justified in allocating their time and effort towards other courses they are in fact struggling with, but now you're requiring them to mimic your words back to you. Taking it even further, they could even be helping someone next to them, thus missing particular things you've said.

Comparatively, if you did your quiz at the start of the next lecture, then your quiz can focus on the material more than the actual lecture itself. This enables the "expert" students to not have to listen to you explain something they already understand; and the "slower" students have had a week's time to catch up on anything they may have sturggled to understand the first time.

In a way, quizzing someone directly after your lecture is asserting/guaranteeing that you are able to perfectly explain something and have people immediately understand it. Even if that is not the goal of the quiz, your students will perceive this to be the goal, and it is going to create (possibly unspoken) friction for them.

---

(1) are quizlets a good solution to my problem?
And (2), if they are a good solution, is there any wisdom from the trenches on how to pull these together so that they are effective without bringing on substantially more grading work? I'd like to make them as useful in the learning process as they can be, but I have to be careful about what I take on.

The short answer: It would be wrong for you to cater your curriculum so that students can instead focus on other courses. However, countering this by cracking down on student focus is not a good solution and is likely going to exacerbate the issue for the students.

We've all had teachers who felt entitled to continue the class after the bell rang. We've also had teachers who were sticklers for arriving to class on time. Both teachers can argue that their expectation is reasonable. But a student who is forced to stay late in one class and thus be late for the next one, is going to resent both teachers because pleasing one teacher would get them in trouble with the other teacher.

If you double down on student focus, that's essentially what you're doing. To the students, it will come across as a NIMBY-like "my course is more important than the other ones!".

There is an alternate solution here that alleviates the issue for students: optional quizzes that focus on the subject matter (not the lecture) and what you will be testing for during the end of unit quiz. The implicit suggestion is that "when you can answer these questions, you have grasped the essence of the subject matter".

For students who are struggling with other courses, are having trouble grasping the subject matter, or simply those who are succumbing to stress, this create a pressure-free guideline on the important points to take away from your lecture.

For students who already master the subject matter, they can be used as a way to reaffirm that they understood the full subject matter of the lecture and didn't miss a smaller chunk that is in fact new to them.
This enables them to very quickly identify that they don't need to go through this any deeper and thus can focus on other courses where they're not ahead of the curve. This is a major boon to relieving stress, because it cuts out "wasted" time (mind the quotes) that they could've spent better.

For this reason, some lecturers in college (when I was a student) used to start the lecture off with such a quiz (merely presenting it, not actually handing out sheets). Anyone who knew the answer to all questions was free to leave (it was up to the students to decide for themselves), because they wouldn't learn anything new during that lecture and could instead take the time to dedicate effort towards classes where they did need to catch up to the curriculum. This of course assumes you're in an educational context when attendence is not mandatory.

You can, if you want, enable students to hand in these optional quizzes. This mean it's up to the student to decide whether they want/need your feedback. The request for help from these students is going to outweigh the effort of having to grade (I sincerely hope). Other students who are already confident in their grasp of the subject matter won't bother with the optional quizzes and thus won't require you to go through the effort of marking those (trivial to the students) quizzes.

Any student who does not grasp the subject matter, does not ask for help and does not self-test themselves will be caught out by the end of unit test. However, any approach that would catch these students quicker (e.g. at the end of the lecture, as per your goal) would also create an obstacle for other students.
In short, you can't police daydreaming. And even if you could, the rigorous nature of your policing is going to cause more issues than it solves.