6
$\begingroup$

I advise several teams of undergraduate students in their final project. It is an annual softward-development project that they do during their 3rd year of studies. Most of them develop a web application in a topic they choose. The development proceeds in several phases. One of the final phases, after the application itself is mostly ready, is to give the application to end users (e.g. family and friends), watch them use the application, ask for their feedback, and prepare a report on issues such as: ease of use, enjoyment, user interface, unexpected errors, etc. This report is 10% of the final grade.

Most teams do a pretty good job, but some teams submit a report with generic answers, and I have a gut feeling that they made it up (or, at least, did not do much effort to elicit substantial feedback from the users). This is quite understandable, since they do not intend to bring this application to the market - they just want to get the grade and proceed with their other chores.

My question: how can I incentivize students to take this exercise more seriously?

One thing I considered is: after they submit the report, I can test the system by my own. For every bug that I find, deduct 10% of the grade for this phase. This will incentivize them to test the system thouroughly for bugs. But this does not handle other usability issues that are not proper bugs.

What else can I do to incentivize students to test their system with end-users?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ From my experience as one of the students that did care, I took this kind of course as a serious learning opportunity. Testing from real users wasn't even required, but one of my teammates and I took it so far as to get a programming youtuber to review our open source software in a real stream, and we got to discuss not just with the youtuber but also the audience as everyone was trying out our code. It was an amazing learning experience that I'm glad to have been able to taken, and I understand why you would want to incentivize students to do the same. (cont.) $\endgroup$
    – Simply Beautiful Art
    Aug 9, 2022 at 7:06
  • 1
    $\begingroup$ (cont.) However, I think it is worth pointing out that in the process of trying to get people who don't want to do this, you may hinder those that actually do. For example, @Buffy's approach can waste their time if they actually want to test their software with a different audience, and their peers may honestly not be good test subjects for things like the usability issues you mentioned. What I mean to say is that for those that actually want to do this, it may be better to be less restrictive and instead provide options or examples from previous semesters that show the before/after difference $\endgroup$
    – Simply Beautiful Art
    Aug 9, 2022 at 7:12

2 Answers 2

Reset to default
6
$\begingroup$

Rather than just try to motivate them, you can change the structure of the assignment. Testing on family members and friends seems like it is the problem.

Instead, break the assignment into two parts: building and testing. Teams build their own product and, at some point, are matched with another team who's assignment is to test the first team's work. Both parts are graded; the quality of the build and the quality of the test you provide for the other team you were matched with.

So, team A's product is tested by team B. Team B's product is tested by team C (which might be A, or not). Team B gets a grade for its own product and on the quality of the feedback provided to A.

You will have to explain some things about the testing part. The purpose is not to tear down the other team, but to provide them feedback on what needs fixing. This can also provide a way to encourage writing along with development, which is something many students need to practice.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Right. It's the old horse race technique: have them switch horses. $\endgroup$
    – Scott Rowe
    Jul 27, 2022 at 14:41
  • $\begingroup$ Switching teams is a good idea, but I still have the problem of how to verify that the testing team really tests the product, and not just makes up feedback that looks deep? $\endgroup$
    – Erel Segal-Halevi
    Jul 29, 2022 at 12:03
  • $\begingroup$ If I remember correctly from past interactions, you have a very difficult scale to work with. Too many students, too little help. If you make assignments then you need a way to properly evaluate them. It ain't easy, as you know but scale can make it impossible. $\endgroup$
    – Buffy
    Jul 29, 2022 at 12:12
  • $\begingroup$ Even in small scale, I do not see how I can distinguish between genuine feedback, and feedback made up for the exercise? $\endgroup$
    – Erel Segal-Halevi
    Jul 31, 2022 at 14:13
  • $\begingroup$ You have to read what they write and evaluate it. There isn't any magic in it. Though I fear that magic is required at your scale. $\endgroup$
    – Buffy
    Jul 31, 2022 at 14:30
3
$\begingroup$

You're at something of a dead end. The problem is intrinsic to the assignment; they are aware that, once their project is done, very little is likely to come of it. Thus, it isn't worth putting real, hefty effort into obtaining serious feedback -- to what practical end?

You say that most students find the exercise worthwhile enough to engage in, but those who do not are not being irrational actors.

You can try to incentivize this with grades, but I would suggest that all this would do is cause those who already are disinterested to take a bit more trouble to cover up their tracks.

Again, if I were in your position, I wouldn't lean into the grade aspect. That feels like force, and is fighting against what you want to achieve, which is intrinsic motivation.

What you want, of course, is for them to see the value in it and take it seriously. So what, then, is the value proposition for the students? If I were in your position, I would begin there.

Motivation starts from within, so if you want serious work on obtaining feedback, you need to explain why this part is important. You can say that it is particularly important to you, and that will move the needle somewhat (it really will!), but you must also undertake the comparatively harder effort of explaining what you hope the students will get out of this portion if they take it seriously, why you believe it will be valuable to them, and thus the approach that they should take.

When the students see value in it, you will see the results.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I do explain the internal importance of this exercise. However, the students have other simultaneous courses that give them exercises with both internal importance and grades. When students has to choose where to put their time and effort, the grade aspect wins. $\endgroup$
    – Erel Segal-Halevi
    Jul 29, 2022 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.