5
$\begingroup$

In the scope of my PhD work, I am developing a teaching assistant tool. These tools are used in the context of lab classes to gather "snapshots" from the student's code using one plug-in in the IDE of their choice (in my case, Eclipse and all JetBrains IDEs). The code snapshots are analyzed/processed, and several metrics may be used to determine the performance of the student as well as if s/he is doing plagiarism, and many other things.

Is anyone using a similar tool (e.g. Retina, TestMyCode, ...)? If the answer is YES, how is it working for you? If the answer is NO, why not?

It seems that they are quite familiar in Academia, but not so much in the “real world”.

Improve this question
$\endgroup$
4
  • $\begingroup$ Welcome to CSE! For the unfamiliar, could you flesh out a bit of what these are? I'm unsure about what tools like Retina, TestMyCode, etc are, and I also don't know what "training" worksheets are. They may be known to many, but if I don't know about them, then I'm sure there are also other who are unfamiliar. $\endgroup$
    – Ben I. ♦
    Jun 21, 2017 at 13:47
  • $\begingroup$ Thank you! These tools are used in the context of lab classes to gather "snapshots" from the student's code using one plug-in in the IDE of their choice (in my case, Eclipse and all JetBrains IDEs). The code snapshots are analyzed/processed, and several metrics may be used to determine the performance of the student as well as if s/he is doing plagiarism, and many other things. My big question is that it seems that they are familiar in the Academia, but not so much in the “real world”. Maybe the right term is: “practice assignments” and not “training worksheets". $\endgroup$
    – Nuno Gil Fonseca
    Jun 21, 2017 at 14:04
  • $\begingroup$ Thanks! Could you integrate that into the question? Comments are considered "transient" on SE. $\endgroup$
    – Ben I. ♦
    Jun 21, 2017 at 14:08
  • $\begingroup$ Nice fix. One more thing, though! (Don't feel bad - learning how to write SE questions is tricky for everyone) Are your two questions related? The standard is one query per question. If one question really just clarifies the other, then that is typically given a pass. If that's not this, you should get rid of question 2 here and open a second question from the main page for it. (And if these two are super-related, then the question probably needs more clarification anyway ;-) $\endgroup$
    – Ben I. ♦
    Jun 21, 2017 at 14:42

3 Answers 3

Reset to default
3
$\begingroup$

When I was a TA, we used tools to automatically:

  • Collect submissions and run them against set of predefined test cases
  • Give instant feedback to students on how they did based on the test cases they passed/failed
  • Give feedback on the quality of their code
  • Generate plagiarism reports

For plagiarism, we used MOSS. For everything else, we developed a tool called AutoGradr.

Improve this answer
$\endgroup$
2
$\begingroup$

I am a TA at a University and yes we do use code snapshot tools but for confidentiality reasons I cannot tell you which one we use. It works extremely well for checking plagiarism, but that's about it. I don't know what other metrics you obtain, but I think it is pretty hard to trust someone's code without reviewing with them. We also automark their code and provide a grade and use this to track progress, but this is only a portion of their overall mark.

From your comments - My big question is that it seems that they are familiar in the Academia, but not so much in the “real world”.

This is a common issue, don't be alarmed. Continuing from my above point, during code reviews, I find that many students are unable to clearly explain what their code is doing. Part of the blame is that students use trial and error approach until they come up with some code that works. Since they are actually unsure what was wrong to begin with, they cannot explain why these "trials" fixed their problems. Hence, we have code reviews to get a more realistic view of the understanding of students as well as reiterate concepts to students.

EDIT: Weekly assignments are meant to be 2 hour workload and are purely automarked, while bigger assignments spanning 3-5 weeks are the ones we perform code reviews on.

To address your second question, we have weekly assignments that weigh very little for the sole purpose of "practice" assignments. The student submits their code and it is run against an automarker I write for feedback. We do track improvements on every run of the automarker to see roughly how much a student is able to improve their code in 1 run. We usually run only 2 automarkers and the second run is their final mark.

As a final note, I recommend a Code Reviewer as a teaching assistant tool. Students post a segment of their code into the system and either professors, teaching assistants, or even fellow peers can highlight questionable code and write comments. This allows students to learn the "what", "where" and "why" their code has problems and "how" to fix them. My arguement is that codesnap shots are good for checking against plagurism, but other metrics can't really reflect whether or not the students are learning from their mistakes.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Kaneki, do you provide one assignment per week? The thing is that in my country it is very common to have worksheets with 10-30 small assignments for the period of one week. So, the performance is measured as the number of assignments that they were able to solve corretly. The "big" assignments are obviously assessed differently. $\endgroup$
    – Nuno Gil Fonseca
    Jun 21, 2017 at 15:07
  • $\begingroup$ To clarify, weekly assignments are meant to be completed by first year skills within 2 hours. They are not trivial 1 function assignments, but rather implementing several functions. As stated above, they are purely automarked. We also give bigger assignments spanning 3-5 weeks and these are the ones we sit down with the students and review their code, asking questions. $\endgroup$
    – Kaneki
    Jun 21, 2017 at 15:17
1
$\begingroup$

We don't use code snapshot tools at Mills because our class sizes are small enough (usually <25) that professors and TAs know how individual students are doing. If students are having trouble, they're likely to email me their own snapshot with a request for help. :-)

It sounds like such tools would be useful in larger programs.

Improve this answer
$\endgroup$
2
  • $\begingroup$ What students do when they are having trouble completely depends on their personality. I think that the rise of services like Khan Academy have shown us that students like to make mistakes and get explanations without feeling judged (even though they are not). $\endgroup$
    – tusharsoni
    Jun 26, 2017 at 22:22
  • $\begingroup$ @tusharsoni Good point. I hadn't thought of ways a computer TA could be perceived as superior to a human TA. $\endgroup$
    – Ellen Spertus
    Jun 26, 2017 at 22:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.