That seem an important question. Here is my input.
'm using repl.it for a month, created ~60 assignments in 10 classrooms (java,c#).
(Both basics, and also recursion, lined-lists and Queues. )
The idea of web-based-ide & auto-checking & status-report-to-teacher is fantastic !
It gets students very fast to code, and see results, also teacher knows immediately the situation of the class.
Also some students find it hard to concentrate and listen to me, so they can go do tasks.
It is very easy to learn to use repl both for teacher and student.
Teacher can create testbench and that saves a lot of time to students, so they can focus on their tasks, and get better testing to their code. Such testbench can include creating of various trees/arrays/linked lists , so students can focus on the method they need to write, and get a lot done automatically for them.
It is easy to student to figure out what teacher wanted in a task, since she can see the testcases ! ( so it saves teacher communication time!).
We randomly divided a 30 HS-students-class in half, 15 with me and 15 with another teacher, my class using repl.it, had progressed much faster, compared to the others that used traditional ways.
**** repl.it is both great and at the same time, far from great ****
repl.it opened my eyes, now i want more from it.
There are details that make the experience sub-optimal :
-) test-cases need to be entered using many gui clicks, why can't there be one file,with many lines input/output (see VPL-Moodle) ?
-) Small tasks like renaming an assignment, accessing my solution, publishing, take way too many clicks, and between clicks we need to wait for web.
-) no source control, if you by mistake clicked 'delete' for a test case it is gone forever.Same for changes to source code.
-) hard to maintain assignments/classrooms over time, in collaboration with other teachers.
Guess there should be text files (under wiki/github/jenkins etc.), that contains all assignment info (question text, start/test-bench code visible to students, testcases inputs and outputs , solution ) , that can be put to other teachers review and improve, like an open source or a wiki project.
- hard to maintain common files (teacher utils) common to all assignments
I think that there is going to be some serious money in "web-based-ide & auto-checking & status-report-to-teacher" and would like to see some aggressive competition.
Currently the only tool i know is repl.it.