18
$\begingroup$

For an undergraduate course in AI, there has been an assignment that uses deep learning on neural networks to do image classification.

However, many students complained about their (university provided) laptops were overheating when running their implementations and only a few were able to run enough training epochs to get meaningful results. (While those devices have a high-end CPU (for laptops, at least), the GPU is rather low-end).

I doubt that the problem was bad implementations, as even the better students had these problems.

All in all, just letting the students run their own implementation doesn't seem to have been a good idea. Are there better methods to give students the assignment of practically implementing a computationally intensive algorithm? How feasible would it be to provide cloud compute resources to the students so they can run their own implementations (I don't think we have a computing cluster available to students at my school)?

Improve this question
$\endgroup$
8
  • $\begingroup$ Should the question be taken from the perspective of a grading TA? (I ask because you did not mention this in the prompt) $\endgroup$
    – Ben I. ♦
    Apr 8, 2018 at 15:31
  • 1
    $\begingroup$ @BenI. My intention was to ask independent of my personal perspective, as 1) I think this may be an issue some course designers could encounter and hence focusing on my perspective is unhelpful for the general public. 2) I'm aware of how I can use information for course designers to provide suggestions to the course designer. Hence, I don't think any accommodation to my perspective would help. But good that you mention it. $\endgroup$
    – Discrete lizard
    Apr 8, 2018 at 15:34
  • 9
    $\begingroup$ In grad school, I destroyed two perfectly fine laptops by running computationally-intensive tasks on them that they weren't capable of handling. Please be respectful to your students: if they're telling you that their computers are overheating, there is a very real possibility that some of your students will see their laptops become unusable because of the assignments. $\endgroup$
    – Kevin
    Apr 9, 2018 at 15:16
  • 2
    $\begingroup$ @Vorsprung I was the downvoter, because I don't like the idea of offloading the costs to students, and because that product could be discontinued. If you undelete it, I'd be happy to edit+retract my vote. $\endgroup$
    – thesecretmaster ♦
    Apr 9, 2018 at 20:31
  • 1
    $\begingroup$ @Vorsprung for someone who is on the breadline yes, \$80 can be an issue. Worst thing could be that those (no matter how few) for whom \$80 is actual money switch to a different course. $\endgroup$
    – Tobia Tesan
    Apr 9, 2018 at 23:10

4 Answers 4

Reset to default
22
$\begingroup$

How to give assignments that require heavy computational resources?

Don't.

Most computionally intensive problems can be stripped down to something that's just as instructive but runnable on any normal computer. In particular, for inexact algorithms, you can generally trade off accuracy against computational cost, and often actually get still quite good results with only a fraction of the cost of a publication-level run.

I always like to emphasize that stuff like physics simulations (which is what I work on, and have also done assignments for students) or image classification should in principle, mathematically speaking, have infinite computational cost: an image is a function from continuous 2D space to colour space, i.e. the space of images is an infinite-dimensional $L_2([0,1]^2)$ Hilbert space. When doing numerics one quickly (IMO, often too quickly and without proper thought) restricts everything down to a finite-dimensional subspace thereof, but that's to some degree arbitrary – the subspace just needs to be big enough (and suitably constructed) to still represent the features that are important for the use case.

In an image-processing example, you should definitely check how much you can downsample the images that the students should work with and still get somewhat sensible results. If necessary, choose only a smaller set of images with clearer distinguishable features, that the algorithm can pick up quickly.

If the results aren't so great with a stripped-down accuracy, well, that's actually particularly instructive, because this kind of tradeoff is absolutely crucial in real-world applications too. You may have more processing power available in a research of commercial situation, but you'll also have much higher demands in terms of data amount and needed accuracy. So, discuss these points intensively, and make a presentation of how much better the students' results look when run by you in higher resolution on a solidly powered machine.

Improve this answer
$\endgroup$
19
$\begingroup$

I have a couple of orthogonal suggestions.

First, and you may have done this yourself, before you give any assignment you should create a reference implementation yourself and test it in the student's environment. If you did this, then just let it be a warning to others. This is true, actually, for any assignment, not just one that might run up agains limitations.

But for the current situation, simply realize that there are several ways for the students to learn, not just through successfully building software for a problem. While you might be able to change the conditions for a future offering of the course, you need to deal with today's students.

One of the most valuable educational experiences is a Retrospective of a project in which you formally explore what worked well and what needs to be changed (or should have been changed). Instead of a working program per student, a paper per student (or student group) can impart the desired learning. Students could examine the results of other students, for example, and reflect (in writing) on the relationship between the code and the outcomes.

However, if you only want an answer to the stated question (how to provide for computationally intensive projects), then you need to either scale down the exercise to make it reasonable, or you need to provide adequate resources somehow. Upgrading all the laptops is probably not in the works, but you could, perhaps, provide access to a more powerful system (or systems) on which the students all work (individually or in groups).

Scaling down the project might work, actually, at an undergraduate level, since the students aren't expected to build commercial quality software, but only enough so that the appropriate learning occurs. If this were a doctoral program, on the other hand, where the students are involved in serious research, then there doesn't seem to be any alternative to finding the resources - perhaps through grant writing or even begging resources from local organizations.

Note that Retrospectives are a fundamental Agile Software Development practice.

Improve this answer
$\endgroup$
2
  • $\begingroup$ While I understand the advice given in the first paragraphs, I was already aware of this and think that has been one of the problems. This might be a good time to mention that I'm not the one responsible for designing the assignment, but a concerned grading assistant. Of course, this doesn't make your advice less valuable, merely less useful to me in particular. As for 'scaling down', that would be difficult. Deep learning only gives reasonable results on large datasets, as far as I'm aware. I do think that perhaps 'scaling out' might be possible: circumvent the actual computations, somehow. $\endgroup$
    – Discrete lizard
    Apr 8, 2018 at 15:26
  • $\begingroup$ @Discretelizard Sure, deep learning only gives good results on initial training on large datasets. Then the industry found a workaround for that and transfer learning is the main way of making working nets. This is completely doable with standard laptops. $\endgroup$
    – DonFusili
    Apr 9, 2018 at 6:30
16
$\begingroup$

Giving students credits for a cloud service like AWS might be useful in this case. Amazon's pricing is reasonable for a cloud instance:

GPU Instances - Current Generation

  • p2.xlarge
    • vCPUs: 4
    • ECUs: 12
    • Memory (GiB): 61
    • Storage (GB): EBS Only
    • Price: $0.9 per Hour

The cheapest instance (t2.nano) is priced at $0.0058 per Hour, though you'd probably want to look at a more powerful instance to save time.

For $20 per student, you could give them 22 hours of p2.xlarge compute time or 3500 hours of t2.nano time (or something in between this if another option would fit better).

It would be wise to test the time taken for EC2 to run a few epochs yourself first before going and paying for your students to do it; you could then work out what a 'reasonable' amount would be to give them, or if this is feasible at all.

This is unfortunate in some ways, however, as it means that students who work quickly are advantaged more than those who spend hours experimenting. It also means that students that need more time would have to work at their own expense; whether this is permitted by your University's policies or ethical is questionable.

I would anticipate that it'd be easy for students to accidentally waste credit if they weren't familiar with AWS too. For this reason I'd be wary of putting this into practice... but it's better than overloading the students' personal laptops.

Michael0x2a has a good point that some services may have free credits which you might be able to tap into; for example, Azure offer $100 of credit to students; AWS Educate is also pretty generous, with member institutions gaining \$200 per instructor plus \$100 per student (non-member institutions get less).

Improve this answer
$\endgroup$
5
  • 4
    $\begingroup$ The main problem here is of course that the costs may scale badly wrt to the number of students. But this is a good overview of this option, thanks. $\endgroup$
    – Discrete lizard
    Apr 8, 2018 at 15:28
  • $\begingroup$ Kindly add the FloydHub as well - one of my student used it recently and its not as powerful as AWS, but maybe fit somewhere for some assignments. $\endgroup$
    – Failed Scientist
    Apr 9, 2018 at 1:11
  • 3
    $\begingroup$ @TalhaIrfan Please, don't add every single cloud service to this answer. $\endgroup$
    – pipe
    Apr 9, 2018 at 7:48
  • 2
    $\begingroup$ I second this approach. As a current graduate student, last semester I took a parallel computing class that had all CPU parallelization assignments done (via SSH) on a computing cluster at our university, and all GPU parallelization assignments done on a Google Cloud instance using an Nvidia Tesla K80. The GCloud credits were more than enough to cover usage time during the course. I think @Aurora0001 has the most appropriate solution. $\endgroup$
    – rm -rf slash
    Apr 9, 2018 at 15:38
  • $\begingroup$ Google Cloud Platform gives $300 for a year for all new users. It also offers some grants for universities. $\endgroup$
    – Frax
    Apr 14, 2018 at 11:46
0
$\begingroup$

Since AWS was already mentioned here, I feel I also have to mention Azure ;)

There is currently running a 200 USD free trial offer that you can utilize. There is one setback about this offer though. I think it requires a credit card in order to activate it. But the 200$ will last you the 1st month and allow you to spin up some serious hardware during that time.

You can sign up here: https://azure.microsoft.com/en-us/offers/ms-azr-0044p/

Besides that there is an offer available (Monthly free credits) if someone has access to a MSDN subscription or you can get a MSDN subscription as part of the MCT (As Professor) benefits.

Azure also includes several features for machine learning and cognitive services out of the box.

Besides that I agree with other that mention it is hard to give assignments that require resources that are not commonly available.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ The idea of using clould computing (and even Azure specifically) is already covered by Aurora0001's answer. For that reason, I've downvoted. $\endgroup$
    – thesecretmaster ♦
    Apr 9, 2018 at 14:38
  • 4
    $\begingroup$ Furthermore, you shouldn't rely on the students being eligible for a free trial offer, since they may have already used it up. Rather, you, as the instructor, should apply for an instructional grant from Microsoft. $\endgroup$
    – 200_success
    Apr 10, 2018 at 0:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.