Philosophy

Who Tests the Tester?

Critias imagines a person who can tell which experts deserve trust. The claim raises an awkward question of its own. How could anyone tell whether that examiner was any good?

A second examiner would make the same claim at a higher level. The doctor’s test and Socrates’ defense suggest two ways to ask for evidence we can actually check.

Orientation

At a Glance

  • An examiner makes a claim tooCritias wants temperance to recognize expertise in others. The person who claims to have that power gives the rest of us another claim to assess.
  • When each tester needs anotherIf only a higher examiner can certify an examiner, the proposed solution keeps recreating the original problem. The chain gives us no first reason for trusting any of its members.
  • A check within the craftDoctors can look at the cases an examiner uses to distinguish good medicine from bad. That check depends on the field’s own standards.
  • The evidence of a conversationIn the Apology, Socrates recounts how he tested the oracle’s words by questioning people. The inquiry and the jury’s verdict are two different ways his work is put before Athens.

Critias’ temperance was supposed to be the expertise of experts, the ability to inspect others and tell who knows and who doesn’t. That raises an awkward question the dialogue invites. How could anyone tell whether a given person really has this expertise? A claim to be the judge of all claims is itself a claim, and a very large one. The Roman satirist Juvenal asks a version of this question in a different setting. Who will guard the guardians themselves? The question works just as well for Critias’ temperate examiner.

There are at least three ways to answer. A second general examiner might test the first, but then that examiner would need testing too. Or people with the relevant subject knowledge might do the work, with doctors checking whether the examiner can tell good doctors from bad ones. A third possibility is to look at the way he reaches his judgments. Those last two possibilities might give us real evidence. They also change what we’d be asking the general examiner to do.

The first answer leads to a regress, a chain in which every link asks for another. If an examiner can be certified only by someone with a still higher knowledge of knowledge, then the certificate tells us little until we have reason to trust the person who signed it. The higher examiner’s authority is exactly what now needs to be shown. Critias’ temperance was supposed to settle the problem of judging experts. In this version, it reproduces the same problem one level up.

The second answer is more promising. Suppose a consultant claims he can spot good engineers. You could ask engineers to look at the people he picked and judge the work they do. A consultant who picks well may save you time. Still, the check works because engineers have standards for good engineering, and those standards are available whether or not the consultant is there. A referee’s calls can be checked against a replay in much the same way. The replay helps because somebody knows the rules well enough to judge what happened.

The third answer points toward Socrates himself. In the Apology, he reports that Chaerephon asked the oracle at Delphi whether anyone was wiser than Socrates and was told that no one was. Socrates takes that as a puzzle and looks for someone whose wisdom would prove the oracle wrong. He questions a man reputed to be wise and finds that the man’s confidence exceeds what he can defend. We can ask whether that exchange gives us good grounds for Socrates’ judgment. The oracle gave him a reason to go looking, and the conversation gives us something to examine for ourselves.

Then there’s the sobering fact of how Athens actually tested its tester. The jury convicted Socrates. In Plato’s account he says he would have been acquitted if thirty votes had gone the other way. He also says he ought to inform and persuade the jurors instead of begging them for a favor. The trial was a judgment on charges, and it had the force to decide his fate. Whether his conversations justified his claims is still a question we can bring to those conversations.

The doctor and the conversation give us two ways to assess an examiner. One is to see whether his conclusions hold up in the fields he claims to judge. The other is to listen to his reasons and look for an inference we have grounds to challenge. The second route can show that an argument has broken down even when it leaves some of the subject matter to the experts. Both make the examiner answerable to something other people can check.

My verdict is that the tester of testers has to earn trust in some way that gives the rest of us evidence. We can check what he says against the field he’s judging or ask to see how his judgment follows from what he’s heard. If he offers us only the authority of a still higher examiner, we’re still waiting for a reason to trust one of them. The man who examined Athens was finally examined by Athens. That reversal makes it worth asking what the test was capable of showing.

Test your understanding

123

Question 1 of 3

Why does certifying an examiner through a higher examiner fail to solve the article’s central problem?

Choose one answer

Guided Inquiry

Questions for Socrates

Explore these questions with Socrates, with this Feature as your reading context.