Philosophy
Flawless Work for a Harmful End
A team of engineers makes an app that keeps its users scrolling. If the product turns out to harm the people using it, what can the team say it has done well?
The Platonic Dialogues: CharmidesThe Charmides distinguishes a craft’s success from the benefit it brings. An imagined engineering team has to work out what that difference means for its responsibilities.
Orientation
At a Glance
- The engineering succeedsIn the example, the app is fast and holds users’ attention as designed. Later evidence gives the team reason to ask whether meeting the goal served its users well.
- The crafts deliver their productsSocrates says medicine would still heal and cobbling would still provide shoes without knowledge of good and bad. He asks what makes the results beneficial.
- Critias asks what the worker knowsThe dialogue tests whether a doctor can do good without knowing it. Critias is reluctant to call someone temperate if the person cannot recognize the good he has done.
- Professional standards ask moreThe engineering code puts public safety, health, and welfare first. The example asks what a specialist owes the public when a product’s purpose becomes a source of harm.
Charmides 163d–e, 164a–d, 174b–d, 174d–175a · National Society of Professional Engineers Code of Ethics for Engineers
Imagine a team at a successful company that designs an app for teenagers. The engineering is superb, with fast loading, elegant design and a feed tuned with remarkable precision to keep each user scrolling as long as possible. The project hits every target, the code is clean, and the engineers are proud of it. Over the following years, evidence accumulates that heavy use of the app is bad for the very teenagers it was built to captivate. The engineers did what they were asked to do, and did it brilliantly, and the question is whether that is enough. Nobody on the team set out to harm anyone, and most of them would be shocked by the suggestion.
The Charmides provides the distinction that sharpens the question. Without the knowledge of good and bad, Socrates says, medicine would make us no less healthy and cobbling no less well shod, so the crafts would achieve their own results in full, but the speakers still have to ask what it takes for those things to be done beneficially. The engineering team’s craft has succeeded completely on its own terms. What has failed is the benefit, and the dialogue insists that the benefit is the work of a different knowledge from the craft that produced the result. That is exactly the shape of the app case, a full success of craft combined with a failure of benefit.
That might seem to let the engineers off the hook. If judging ends belongs to another knowledge, perhaps their responsibility stops at the edge of their craft, and the people who set the goals, the executives, the regulators or the market, are the ones who must answer for them. Plenty of specialists think this way, and there is something to it, since nobody can be expected to judge the whole social effect of every task, and a society where every worker second-guessed every assignment would struggle to function. The division of labor depends on people trusting that others are handling the parts they can’t see.
But an earlier passage pushes the other way, in which Socrates asks Critias whether a doctor must “also know when his treatment is beneficial and when it is not,” and whether each craftsman must know when he is going to profit from his work, and Critias concedes, “Perhaps not.” That concession leads straight to the uncomfortable result that a craftsman could act beneficially without knowing it, and Critias finds that result intolerable. If a temperate person must know whether his work does good, then a specialist who does excellent work without asking whether it benefits anyone has fallen short of something important, even if his craft is flawless. Critias’ refusal makes the link between good work and awareness of its effects worth pursuing. What that awareness would involve is still open.
The National Society of Professional Engineers puts public safety, health, and welfare first in its code of ethics. That standard asks the engineer to look beyond whether a task was done correctly. A team can rely on other people to investigate risks that its members can’t see, and still have reason to ask questions when a risk becomes clear.
The Charmides leaves the allocation of responsibility open. The division of labor gives people good reasons to rely on one another, and the code is one way a profession says where that reliance has limits. The distinction between a craft’s success and its benefit lets the engineers ask what their achievement is doing in the world.
Notice too how hard the dilemma is for the people inside it. The engineers who built the feed may genuinely not have known, at first, what it would do, and the knowledge of whether a product harms its users often arrives slowly and from outside the team. Once it arrives, though, the question changes, since continuing to perfect a product known to harm its users is no longer a matter of craft alone. A question about effects can start with someone on the team who has good reason to look beyond the assigned task. The dialogue’s doctor who doesn’t know whether he helped is in a different position from one who has been told and keeps going.
The engineers have done something difficult, and they can be proud of doing it well. The craft has made the product they were asked to make. When they learn what the product does to the people using it, they have another decision to make, and the quality of the code cannot make it for them.
Test your understanding
Question 1 of 3
What central distinction does the article use the app case to illustrate?
Guided Inquiry
Questions for Socrates
Explore these questions with Socrates, with this Feature as your reading context.
- When, if ever, should excellence within a craft count as good work if the craft’s result is harmful? What would justify your answer?→
- How should responsibility be divided among engineers, executives, regulators, and others when knowledge of a product’s risks is incomplete or arrives gradually?→
- What kinds of evidence or warning should be sufficient to require an engineer to question, resist, or stop an assigned task?→