19
$\begingroup$

The kinds of things that people with technical training can do have societal impacts, both for good and evil. Programmers are building the future. One future nightmare is Skynet.

Many problems in the world have been caused by scientific hubris as well as the Fallacy of the Last Move, a concept from Game Theory as it applies to Arms Control and other strategic thinking.

Unbound technologists, especially if amoral, can be very dangerous to the earth and its inhabitants.

So, it seems that future Software Developers need some training in ethical concerns. What, to you are the most important issues and where in the curriculum should they be taught?

Note that ethical issues can be simple or complex. Even something as simple a accepting buggy software can have ethical implications and such lessons can be taught in early courses.

At the other end of the scale is the question of how much a person owes to their employer vs to humanity as a whole.

  • What essential ethical practices do students need to be trained in?

  • What are effective ways of integrating ethics into the curriculum?

  • What are the first steps a teacher should take in integrating ethics into the curriculum?

A general discussion of Computer Ethics is: https://plato.stanford.edu/entries/ethics-computer/

Improve this question
$\endgroup$
8
  • 11
    $\begingroup$ I think faulty engineering is a likelier ethical concern than Skynet. $\endgroup$
    – Ellen Spertus
    Aug 2, 2017 at 15:47
  • $\begingroup$ The BCS Code of Conduct might give some insight into this issue; bcs.org/upload/pdf/conduct.pdf $\endgroup$
    – Mikael Patel
    Aug 2, 2017 at 19:52
  • 9
    $\begingroup$ Speaking as a machine learning guy, I wish creating AIs was actually easy enough that a rogue programmer creating Skynet was a realistic concern. $\endgroup$
    – Ray
    Aug 2, 2017 at 21:03
  • 1
    $\begingroup$ I highly recommend "Ethics and Technology: Controversies, Questions, and Strategies for Ethical Computing" by Herman T. Tavani. $\endgroup$
    – Alex
    Aug 2, 2017 at 22:00
  • $\begingroup$ Some good case studies, would be Uber (bbc.co.uk/news/business-36843386), Ashley Madison (The site itself, and the subsequent information disclosure), Encrypted Messaging vs National Security (express.co.uk/news/world/834597/…) $\endgroup$
    – JeffUK
    Aug 3, 2017 at 8:04

10 Answers 10

Reset to default
15
$\begingroup$

I'm going to begin by quoting Ken Thompson's Turing Award Lecture "Reflections on Trusting Trust" (link).

To what extent should one trust a statement that a program is free of Trojan horses? Perhaps it is more important to trust the people who wrote the software.

In my mind the first and most foundational lesson to impart is this: software, computers, technology -- these things are more about people than about code. There is a trust from the end-user, whoever that user may be, that must not be violated. Users are not means to an end; they are the end. The code is the means.

I am reminded of the Second Formulation of Kant's Categorical Imperative:

Act in such a way that you treat humanity, whether in your own person or in the person of any other, never merely as a means to an end, but always at the same time as an end.

In terms of integrating ethics, the way to start in my mind is discussion. Early and often. Share news stories. Discuss implications of new apps students download. The first 12 minutes of this CS50 lecture cover the the Volkswagen/EPA scandal from 2015 and mention Ken Thompson's lecture. It is essential that awareness is built right from the beginning that software is everywhere and there needs to be an ethical code to guide that development. In my mind, while I wouldn't consider myself Kantian, I do think his point about means and ends is essential.

The AP Computer Science A curriculum outlines the following requirements under the major topics of the course:

VI. Computing in Context

An awareness of the ethical and social implications of computing systems is necessary for the study of computer science. These topics need not be covered in detail, but should be considered throughout the course.

A. System reliability

B. Privacy

C. Legal issues and intellectual property

D. Social and ethical ramifications of computer use

Similarly, in AP CS Principles, students, as a part of the Explore Task, have this on their rubric for the assessment:

Analyzing Data and Information: Identifies one storage, privacy, OR security concern. Explains how the concern is related to the computing innovation.

The moral of the story here is that ethics must be discussed with students as soon as possible whether it is a high school AP course or an introductory computer science course at a university.

This idea of trust must be imparted to students, and it must be emphasized that the world of software engineering and computer science in general is not in a moral vacuum. It does no good to address a particular issue, say data privacy, without first contextualizing it within a larger ethical system, one that sees each user as an end, not a means, and as someone who is placing trust in you the developer.

Improve this answer
$\endgroup$
4
  • 3
    $\begingroup$ @nocomprende I don't disagree. I am reminded of the great wisdom of Dr. Ian Malcolm from Jurassic Park: "Your scientists were so preoccupied with whether or not they could that they didn't stop to think if they should." People need to "stop to think." Instilling that "need": "Ay, there's the rub." $\endgroup$
    – Peter
    Aug 2, 2017 at 16:42
  • $\begingroup$ The only person I would trust is a madman who would rather voluntarily live in poverty than allow himself to be pressured. Well, since everyone else by definition can be pressured and bought to sneak in bad stuff in the code which therefore eradicates any trust. $\endgroup$
    – mathreadler
    Aug 3, 2017 at 14:19
  • $\begingroup$ @nocomprende no idea what this "Tao" says or who he is. But there do definitely exist enough ways to pressure most people to stop trusting the concept "trust" altogether. $\endgroup$
    – mathreadler
    Aug 3, 2017 at 15:58
  • $\begingroup$ @mathreadler. Even the Road Warrior world had some trust. You have to get to Lord of the Flies to go without it. But perhaps you only mean "trust but verify". Also, conversations to the classroom, please: chat.stackexchange.com/rooms/59174/the-classroom $\endgroup$
    – Buffy
    Aug 3, 2017 at 20:04
4
$\begingroup$

What are the first steps a teacher should take in integrating ethics into the curriculum?

I would begin with practical, everyday topics which every student can relate to, and which are (should be) absolutely practically relevant in todays world from day #1 for every developer. Maybe based on current newspaper-level discussions.

  • Privacy (being able to stay private when using an application, controlling who can see what you do)
  • Data security (being able to protect your own data, e.g. in the context of a business)
  • Personal data (being in control of data related to yourself as a human being; related but not equal to Privacy)

You should easily be able to find an abundance of concrete examples, let me give you some that come to mind:

  • Privacy: communicating with someone while being sure that nobody can read your chat. May it be your spouse (say when organising an expensive birthday present for them), your system administrator (when executing your function as a manager to discuss your subordinates with your boss, etc.), or the general public (obvious reasons...).
  • Data security: Keeping contract offers away from competitors.
  • Personal data: Keeping preferences, habits, interests etc. from prospective employers or insurance companies.

I would ask the pupils to come up with their own, they should be able to find a lot of different things.

This, then leads to your question: it is ethically important for everybody involved in the creation of software systems to further these things. I.e., not to be laizzes-faire about them, not to prioritize them lowly when budget or time pressure arises, not to actively violate them; on the other hand, to constantly remind/teach the stakeholders, who may not have an understanding of these matters, and so on.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ @nocomprende: absolutely. Not only the Web, but the Internet as a whole. I was active in Fidonet, Usenet etc.; one day DejaNews, then Google, just slurped in the whole Usenet history, for eternity. Before that, Usenet was kind of temporary. That was kind of the eye opener for me; before that nobody gave a d*mn about what they posted (private information, etc.); afterwards not so much. If I check Google Groups now, I easily find all my post history from the early 90's, including real names, mail addresses, phone numbers in the signature and such things...). $\endgroup$
    – AnoE
    Aug 3, 2017 at 14:52
4
$\begingroup$

None.

Yes, a bold statement, literally. So let me elaborate further.

You cannot cover all potentially relvant aspects of "ethical practices" in a computer science course. Assuming that this is a general, basic course, you simply cannot foresee the potential ethical questions that will arise for these students. One of them may work in the IT department of a hospital. He should not sell patients data to a pharma company. One of them may work in a bank. He should not disclose or manipulate the financial status of customers. Another one will design brain-computer interfaces. He should not use his abilities to alter the behavior of the subject in a non-ethical way (whatever this means!). Another one may be responsible for implementing the emergency handling of a self-driving car, and the ethical implications of that are currently the subject of deeply philosophical (and still open) discussions

Related to that is another problem: You can hardly ever foresee the ethical implications that your work may have. Although this example has been used ad nauseam (and may be considered as some sort of a fallacy), I like the comparison of computer science and a sharp knife: You may use it to stab someone, or to perform a life-saving surgery: The data analysis software for weather forecasts may be used to compute questionable credit scorings. The image analysis software that you're implementing to sort holiday photos may be used to identify people in surveillance camera images.

(This has a huge potential for abuse. The fact that people could now argue that this "helps to catch terrorists" shows how difficult this topic is. The ethical discussion itself is beyond the scope of this answer and this site. In fact, the examples here should only show exactly this: You cannot summarize "ethics" on a few PowerPoint slides...)

And finally, assuming that the students are in their 20's: It's too late. You will not convert an (ethically) "bad" person into an (ethically) "good" person by listing things that they could do but should not do. Acting responsibly (in an ethical sense) is nothing that can (or should) be tought alongside the basics of abstract data types, computer architecture or object oriented design...

To summarize, although most of the downvoters will likely not read up to this point: A computer science course is not the place to train people in ethical practices, as stated in the question.

What you may (or even may have to) teach is awareness. People should not act unethically (or just aid others in acting unethically) due to a lack of knowledge. Computer scientists, particularly in their later (practical) roles, for example, as IT administrators, have a lot of power, and thus, a lot of responsibility. They should be aware of the fact that their knowledge or work (or a moment of carelessness) may literally destroy lifes.

This awareness about the implications of their work is unrelated to ethics. Whether or not they use their powers in an ethical or unethical way is nothing that you can "teach" them.

Improve this answer
$\endgroup$
7
  • $\begingroup$ You seem to be contradicting your bold statement (I noticed the pun). If I understood correctly, you are saying that because of the serious dangers that unethical IT workers might pose, they shouldn't be taught not to do those things? Even if one cannot "foresee" the ethical implications work might have, I doubt it will ever be considered ethical to disclose secret information which leads to direct harm to others. That can be taught, and You don't need the gift of foresight to know what sort of ethics would be useful for your students. $\endgroup$
    – ItamarG3
    Aug 3, 2017 at 17:31
  • $\begingroup$ @ItamarG3 The point is that teaching (and thereby to some extent "defining") ethics is not sensibly possible in a CS course. Regarding your example: Disclosing information that may harm others could be "OK" if the "other" person is a terrorist who plans an attack. Whether this is unethical can neither sensibly be discussed here nor in a CS course. On the other hand, what you can teach is, for example: Which measures may prevent unauthorized access to personal data, or which pieces of information can be combined in order to identify someone, from a purely technical point (...cont'd) $\endgroup$
    – Marco13
    Aug 3, 2017 at 18:52
  • $\begingroup$ (also @nocomprende ) : To judge whether something is ethical/unethical, right/wrong, good/evil depends on the context. Although you can define a motto, like Google did, this won't mean anything. Although there certainly is some consensus about what is "right" or "wrong" (regarding WW2, most people will call it "wrong"), teaching this is totally unrelated to the contents of basic CS courses. Or another summary: You should not teach ethics, but the (technical) points of the work that may have implications for ethics. $\endgroup$
    – Marco13
    Aug 3, 2017 at 18:57
  • 2
    $\begingroup$ "... assuming that the students are in their 20's: It's too late. You will not convert an (ethically) "bad" person into an (ethically) "good" person..." This assertion is not supported by the research about ethics and morality. We don't even seriously start developing a cohesive moral perspective until our 20s, and this process continues well into our 30s. The very late teens and the early 20s are prime time in the brain for discussing moral and ethical action. $\endgroup$
    – Ben I. ♦
    Aug 4, 2017 at 4:43
  • 1
    $\begingroup$ @BenI. It's not tooo surprising that the ethics department of a university claims that ethics can be tought ;-) But that's not the point. The point is rather that this ethics department will not teach the students that ethical implications can be modeled with binary logic on NAND gates. One could blatantly say: "That's off-topic". Again: CS students should be aware of their responsibility and how their work may affect the lifes of others. But trying to teach them what is "right" or "wrong" (and for which reason) seems pointless for me, as these are subjective, open-ended discussions. $\endgroup$
    – Marco13
    Aug 4, 2017 at 12:51
3
$\begingroup$

Raise awareness of the issues, discuss historical (Therac 25 is an old but highly interesting case of software bugs directly causing people to get hurt) and current cases (EU/US safe harbor and followups - there is so much potential to go in-depth about what data protection means and what the risks are!), and maybe make the legal situation (especially international law!) part of the curriculum ... but don't try to "train" people towards any ethical viewpoint. The latter is in the end straight manipulation no matter how well intended, and something people should if at all be trained to guard against.

Improve this answer
$\endgroup$
0
3
$\begingroup$

Teach them to be cautious. This may not sound like ethics, but actually it has a significant ethical effect.

People - especially young men - who are very confident try all kinds of things, including unethical ones. Their confidence is often misplaced. Computers will not forgive you because you are young and charming. They'll happily retain the evidence of wrongdoing, and it's very hard to clean it off them completely.

Misuse of computers tends to be discovered. Unlike people, computers do not cover up evidence of deception, because they're incapable of being embarrassed.

Besides, caution is sound engineering practice. You get two useful results from one set of teaching.

Improve this answer
$\endgroup$
2
$\begingroup$

Think about what makes CS just different from other fields. Ask the students to come up with lists and then discuss. I thought of these five things in about 2 minutes:

  1. Automatic equipment can do unexpected things and cause harm. Industrial machines have large signs that say "Might automatically begin moving." We have to consider unintended operation.
  2. Computers are now so widely deployed that they can affect vast numbers of people all over the globe, now and over time. "To err is human, to really foul things up requires a computer." Or several billion...
  3. Computers are very hard to reason about, perhaps impossible in some cases. Any sort of AI will in fact be impossible to understand, and might function very differently from our reasoning. "I'm sorry Dave, I cannot allow you to jeopardize the mission."
  4. Computers allow people to be affected remotely, which was never true in the case of technology before. Locality used to prevent most forms of crime, and most harm, even from natural causes. (A falling tree can only kill so many people.)
  5. Computer technology can cause problems to spread in a way that was not true before. I read that self-driving cars will all immediately take on new information 'learned' by other cars. So, mis-information could quickly result in catastrophe, especially if it was induced maliciously.

This is what I came up with that makes CS different from more mundane fields where ethics could apply. (I misplaced this list and recreated it mentally with no effort, so these concerns are probably the salient ones.) I don't think that any 'ethics' is required here, and because I am concept-averse, I don't see a need to create any new rules or principles. Just think ahead a bit!

Improve this answer
$\endgroup$
1
$\begingroup$

Software craftsmanship

As professionals, we need a set of ethics (ethics is not the same as morality). Medics have ethics, even lawyers have ethics. They can be struck off from their profession if they do not stick to them. If we do not create our own, then they will be imposed on us (see talk by Uncle Bob Martin).

In some ways it does not matter exactly what they are. You could have a class discussion. You could include those proposed by others. There are many points of view. A few contradict, but many do not.

I have included those proposed by Uncle Bob et al.

Agile manifesto

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Software craftsmanship manifesto

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

  • Not only working software, but also well-crafted software
  • Not only responding to change, but also steadily adding value
  • Not only individuals and interactions, but also a community of professionals
  • Not only customer collaboration, but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

Improve this answer
$\endgroup$
1
$\begingroup$

I was recently watching videos of Libra Planet 2017. A lot of their talks covered ethics.

There seems to be a lot of ethical considerations in software. They did not cover everything, but here is a summary of some of the topics:

  • Jobs.
  • Security.
  • Privacy.
  • Equal rights / user Disability.
  • User rights: Right to use, right to study, right to distribute, right to change …
  • Algorithmic bias: e.g. search, selection (for jobs), crime detection.
  • Law: The good, the bad, and the ugly.
  • Politics.
  • Free speech: Software that you can go to jail for talking about (in the USA).
  • Workers(programmers) rights and responsibilities (guilds and trade bodies).

Some things not covered.

  • Safety.
  • Artificial Intelligence.
Improve this answer
$\endgroup$
0
$\begingroup$

The additional dangers of software over other engineering fields.

Who is in charge

All software is a form of artificial intelligence, by this I do not mean artificial consciousness (e.g. skynet, I Robot, ExMachina, robby, and every other robot from the movies).

I think that the probability of artificial consciousness arising soon is very low. However some form of artificial intelligence is in every bit of software. The danger is not the artificial intelligence, but who controls it. A lot of software to day is controlled by big corporations. Users of the software are not free to: choose how they use it; study how it works; make changes to it; share it with their friends.

Therefore all software should be Free Software.

I am not referring to Open Source. Although (Software that is Free Software) ≈ (Software that is Open Source). I am referring to the concept of Free Software. The freedoms, for it is the freedoms that we need. These freedoms are also in the Open Source definition, but are usually over looked.

I am also not saying that all Free Software is ethical. There are many sides to being ethical. There may also be proprietary software that is more ethical than a particular bit of Free Software. Free Software is necessary but insufficient, to being ethical. Just as obeying one of the 10 commandments is necessary but insufficient, to being good (Even obeying all of the law is insufficient).

“All these [laws] I have kept,” the young man said. “What do I still lack?”. He answered, “If you want to be perfect, go, …”

What I am saying is about the ethicalness of the software, not of the user.

Free Software has nothing to say about money (except to say that you must be free to charge money for it).

Artificial consciousness

Just a small paragraph to counter what I said above.

If some one interfaces a biological brain to a computer, and gives it power. That is the ability to do harm, then it probably will. Any disembodied brain will have no empathy.

Improve this answer
$\endgroup$
7
  • 4
    $\begingroup$ So the only ethical software is "free" software, where "free" is defined by Stallman's turgid GPL. I disagree. $\endgroup$
    – President James K. Polk
    Aug 2, 2017 at 19:27
  • 2
    $\begingroup$ @JamesKPolk The Free Software Definition defines what Free Software is. The GPL is but one implementation. You should give them a read, you will find that the language is very simple and short. $\endgroup$
    – ctrl-alt-delor
    Aug 2, 2017 at 19:51
  • 1
    $\begingroup$ -1 if I could downvote on this site. This doesn't seem to have anything to do with ethical software. $\endgroup$
    – David Mulder
    Aug 3, 2017 at 8:15
  • 1
    $\begingroup$ Why would a disembodied brain have no empathy? Are you suggesting empathy comes from the liver or what? It's not a separate system that restricts the brain, it's part of the brain. Not that I'm saying it's a good idea trying to model a true artificial intelligence after the ridiculously flawed machine built by natural selection :) $\endgroup$
    – Luaan
    Aug 3, 2017 at 8:38
  • 1
    $\begingroup$ @Luaan No I was thinking that empathy comes from your interaction with others. But I would leave it as a discussion for the students, to decide if they think I am correct. $\endgroup$
    – ctrl-alt-delor
    Aug 3, 2017 at 9:37
-1
$\begingroup$

There are certain unsolvable problems (I am not referring to NP), that seem solvable, and an apparent solution can be made and deployed. These systems can be used, but do not fulfil there requirements (security, privacy, …).

This differs from the case where the software lacks these features, because of poor workmanship (also an ethical issue), in this case it can not be done. I turn down the work, because it is impossible. The client thinks they need to find someone that is more clever, that can do it. Eventually they find someone that has no idea of the problems, that agrees to do the project. If the new supplier realises that it can not be done, and tells the client, then the process repeats, else a faulty product is delivered.

An example is electronic voting.

It is impossible to create an electronic voting system that meets all of the requirements. If we just have 2 of the requirements for a voting system.

  • Anonymity
  • Trustable

Then we can not produce an electronic system, that can meet just these requirements. There are also other requirements, thus it is even harder (in the higher than infinity sort of way (∞+1 = ∞), therefore we don't need to look at the other requirements, to see how hard it is).

How would you audit this system? You could check its source code, to see that it does the right thing (if you can read it, so every one involved would have to have a good enough level of CS, and know the language). Oh but how do we know that the system is running this code.

Every single solution to this problem, has a counter measure ( a dis-proof ).

This has not stopped these things from being deployed many times, and in countries that are considered to have a well run voting system.

What is needed

A box, that you put all of your votes into, and out comes a count for each candidate.

Possible solutions

How do you know that the output is correct. Ah that is easy, you produce an audit trail. But now we have no anonymity. So we add encryption. But for it to be audit-able, anonymity can still be broken. (If you can check your own vote, then I can check your vote (if I torture you long enough, then you will give up your key).

A solution that works, but does not use a computer.

The many boxes - you walk into a room (box1), and show your ID, your name is removed from the list, and you are given a voting sheet. - you walk into a subroom (box2), and make your mark on the sheet. - you then fold(box3) the sheet. - you leave box2 and walk to a locked box (box4), and place your sheet into it. - at all times observers (from all interested parties) are walking around box1, to check that the rules are being followed. - at the end box 4 is sealed. - latter box 4 is opened and the sheets counted. With lots of people watching. A set of totals is produced and announced. The sheets are put back in box 4, sealed and locked.

I have probably missed some detail, but this solution cannot be implemented electrically, because there is no way of putting people into the electronic boxes to observe.

Some videos on this

The problem

Some solutions I found a problem with all of these solution, as I was watching it (Did not have to sit and think for days).

Other stuff

Improve this answer
$\endgroup$
11
  • $\begingroup$ You make it sound like paper voting somehow has less issues with trustworthiness. Technical solutions could theoretically improve trustworthiness (as you increase the number of people who would all need to be 'in on it'), but it's a very complex issue. Anyway, -1 if I could vote down on this site. $\endgroup$
    – David Mulder
    Aug 3, 2017 at 8:12
  • $\begingroup$ @DavidMulder So if I provide a system, for an election. It checks each voters ID. We could ensure anonymity by never ever ever, connecting it to a network after it has been used. However there is no way for the voters to know. Then it takes your vote, and reduces it to a count for each candidate. How will you as a candidate ensure that it does this correctly ( It could just make up the numbers, in favour of the candidate that it wants to win). $\endgroup$
    – ctrl-alt-delor
    Aug 3, 2017 at 9:42
  • $\begingroup$ @nocomprende it has already been solved for paper based system, but you may have improved it. However I see no need to use dollar bills: once country elects an untrustworthy precedent, the dollar bill would be as trust worthy as raffle tickets. But still an improvement, as it allows each voter to verify there part. The existing system focuses on the candidates and parties verifying the system. Can you see why it is impossible to implement using a computer system? $\endgroup$
    – ctrl-alt-delor
    Aug 3, 2017 at 12:37
  • $\begingroup$ While the answer has some merit in the context of the question (for example as a teaching aid to show layers of complexity for the use case of electronic voting systems), I would be really reluctant to make the absolute claim that it is prinicipally impossible. Cryptography gives us tools similar to the pen&paper, anonymous envelops, tokens, booths...; without having really analyzed it, I could see possible solutions. Obviously, part of it would be that local machines (+ their software) would be assumed to be compromised, and such compromising must be detected by the "crypto" part. $\endgroup$
    – AnoE
    Aug 3, 2017 at 13:09
  • $\begingroup$ @AnoE because you can not trust or audit the computer, you need end to end checking (crypto). Now get this to work with anonymity and inability to show how you voted. The some solutions video talks about these, but I see problems with all of the solutions (and I am to that clever). This is without having to resort to “How does my mum verify a cryptographic hash by hand (because she dose not trust the computer).” $\endgroup$
    – ctrl-alt-delor
    Aug 3, 2017 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.