7
$\begingroup$

I'm a computer science teacher and our department is thinking about opening a data analysis and machine learning course for upperclassmen at my high school.

What are some topics that could be covered in a year long timespan that could be done by students with a basic understanding of Python and basic statistics (correlation/regression, measures of central tendency, hypothesis testing)?

I'm thinking the total time commitment available to students is 3.5 hours a week of in-class time, and approximately 2 to 3 hours a week of coding, learning, or doing homework.

Some topics that we were thinking might be relevant are image processing and natural language processing.

Are there any projects that really hit the ground running in terms of showcasing a topic in machine learning?

$\endgroup$
  • $\begingroup$ Welcome to CSEducators. I hope you get good answers to your question and come back for more in the future. $\endgroup$ – Buffy Jan 17 '18 at 17:50
  • 1
    $\begingroup$ Can you translate “upperclassmen” for the international audience. I am from England and have no idea what it means. (may be a foot note, with age etc.) $\endgroup$ – ctrl-alt-delor Jan 17 '18 at 19:24
  • 1
    $\begingroup$ You may also be interested in this question. $\endgroup$ – Ben I. Jan 17 '18 at 22:13
  • $\begingroup$ The definition is loose, but upperclassmen are generally 11th and 12th graders (16 - 18 years old), but sometimes include 10th graders. $\endgroup$ – Jonathan Ma Jan 17 '18 at 23:58
6
$\begingroup$

There are all sorts of things you could cover — machine learning is an extremely interesting and growing field, with many different approaches and tools you could explore.

Some of the more popular tasks for machine learning algorithms include classification, regression and clustering. Not every application fits into this grouping (e.g. AlphaGo, the Go playing program which beat Lee Se-dol).

I might suggest starting with Naive Bayes classifiers. Your motivating example here could be spam filtering; Naive Bayes classifiers were used in some of the first 'learning' spam filters, dating back to the 1990s. If your students are familiar with Bayes' theorem, the assumptions of NB classifiers shouldn't be a huge leap (you may find this derivation interesting).

There are also various other interesting options such as random forests (apparently used by Quora to find duplicate questions), support vector machines and so forth which could be explored potentially.

Many of the most 'fashionable' techniques involve neural networks, and I would recommend spending a decent amount of time with the theory. Unlike some of the classifiers and regressors I mentioned earlier, neural networks tend to be a bit more involved — a Naive Bayes classifier essentially just needs to be given some data in an appropriate form and is "plug and play", so to speak.

The (free) online book Neural Networks and Deep Learning gives a reasonably accessible explanation of neural networks in their various forms (starting with perceptrons, leading towards sigmoid neurons, gradient descent, backpropagation, deep learning, etc.). As you can imagine, there is an immense amount of content even in just the field of neural networks.

Likely you won't want to cover all of the book's topics, but you don't need to go too far to reach another interesting motivating example: classifying the MNIST dataset of handwritten digits. That's something which is relatively hard traditionally, but much simpler with the usage of a neural network.


Note that I've discussed a lot without ever actually explaining how you could write any programs. Of course, any machine learning class would be a little incomplete if your students had never actually applied their skills.

Since your students know Python, I can highly recommend scikit-learn. It's widely regarded as an exceptionally good ML library, and the API is very friendly — you can almost treat it as LEGO, and plug together the pieces needed to do your classification/regression/clustering... even without really knowing how each piece works entirely. You can get functional (though not exactly effective) solutions simply by connecting the appropriate pieces in a pipeline.

For example, if you wanted to classify some texts, you'd just:

  • collect some data
  • setup a Pipeline of a CountVectorizer and a classifer such as LinearSVC
  • train it, and then test it.

There's an awful lot of stuff included, but then again, all the batteries you'll ever need are included, too.

For playing with neural networks, Keras might be worthwhile.

That said, your students may derive some value from implementing the algorithms themselves, at first. The key is to be able to find truly motivating problems:

Struggling with a project you care about will teach you far more than working through any number of set problems. Emotional commitment is a key to achieving mastery. (source)

It's not always easy if you're supervising many students, but it is worthwhile advice to consider.

$\endgroup$
  • $\begingroup$ The course will be taught in high school. May I ask what math prerequisites would be needed for those topics you listed in your answer? $\endgroup$ – scaaahu Jan 18 '18 at 9:54
  • $\begingroup$ @scaaahu: It's mainly linear algebra that comes up in machine learning. A familiarity with vectors and working with them (such as the dot product, gradient of a vector) and some calculus (differentiation and partial derivatives) would be helpful. If you didn't want to get bogged down with the details, you could probably explain what you needed as you taught it, but certainly the vectors pop up everywhere so an awareness would be helpful. $\endgroup$ – Aurora0001 Jan 18 '18 at 15:10
2
$\begingroup$

What are some topics (Machine Learning) that could be covered in a year long timespan that could be done by students with a basic understanding of Python and basic statistics (correlation/regression, measures of central tendency, hypothesis testing)?

With regards to Neural Networks the for an introduction, even before specking oneself or doing suggested reading would be to watch the YouTube series of videos Neural Networks Demystified. When I first wanted to learn about Neural Networks I spent a few days combing the Internet and books and once I saw this series I stopped looking. I still get regular YouTube recommendations but this is the best introductory one by far. It is easy to understand, has great visuals, uses Python as an example programming language, is a real world problem, moves seamlessly back and forth between high level concepts and low level details as source code, etc.

The next topic(s) are to use the free online book Neural Networks and Deep Learning which is often recommended even in the other answer here. You can build a few weeks of the course around just this and it would be worth the effort. This book goes more into the math and explains why certain activation functions are used. While I am not actively writing Neural Networks at present I do keep my finger on the pulse and one of the more intriguing ideas on the street is that using Rectified linear unit (ReLU) are just as effective as the Sigmoid function. The beauty of this is that using ReLU doesn't need the use of a floating point processor and so in theory should speed up the processing without sacrificing abilities. So as a set of labs do the book as written, but then try other activation functions and compare the results.

Another topic is to run the code using a CPU and then using a GPU and compare the speed. If done correctly the GPU will be significantly faster then the CPU. As another option compare those against using a cloud service such as AWS

After the concepts of Neural Networks are understood at the Python source code level, abstract that away and use TensorFlow which is what Google uses.

Lastly one topic I am seeing pop up much to often with regards to Neural Networks is for classification how easily they are fooled, e.g. When all the world's a toaster, according to tricked AI After seeing this I wanted to get a sticker and put it on a shirt saying "I am not a toaster" but then forgot that they already have shirts saying I am not a toaster, referring to Cylons.

$\endgroup$
  • $\begingroup$ Very interesting point about adversarial examples; students might also find examples like this paper interesting in which deep neural nets were fooled by one pixel changes. It really does make it clear how neural nets don't really see things the way we do (and perhaps makes driverless cars and such sound far more dangerous, if such small changes can fool them!) $\endgroup$ – Aurora0001 Jan 19 '18 at 14:02
  • 1
    $\begingroup$ @Aurora0001 Thanks for the reference about the single pixel difference. I know about it but could not find it. $\endgroup$ – Guy Coder Jan 19 '18 at 17:07
1
$\begingroup$

I think one of the easiest ML algorithms to understand is K-Nearest Neighbors (KNN). We do a KNN project in my data structures class in the first week of school so that they can refresh their skills from AP CS and demystify ML a little bit.

What I like about KNN is that it's conceptually easy to understand for high school students (I showcased it at Open House to the parents), it works for both classification and regression, and has its advantages and disadvantages. It's one of my favorite projects we do all year.

I'd be happy to chat offline if you want some specifics.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.