雅思阅读 152: Should We Fear Our Own Creation?（该惧怕我们自己的造物吗？）

改编自 Center for AI Safety statement / Nick Bostrom, Superintelligence。雅思阅读 Section 3 难度，约 1050 词。 素材来源：https://safe.ai/work/statement-on-ai-extinction-risk

Reading Passage

A. In 2023, a short statement appeared on the website of a research centre, signed by hundreds of people who actually build and study the systems in question — leading engineers, chief executives and, most strikingly, several of the scientists widely regarded as the founders of modern artificial intelligence. It contained a single sentence of twenty-two words: mitigating the risk of extinction from artificial intelligence should be a global priority, alongside pandemics and nuclear war. Coming from researchers with no track record of sensationalism, the statement marked a striking change of tone. For decades the danger from machines was the stuff of science fiction. Here, people who understood the technology from the inside were comparing its worst-case failure to the risks that civilisation most seriously dreads. Not everyone agreed, and the signatories themselves disagree about how likely the worst case is. But the debate it opened — about whether humanity should be cautious of its own most powerful creation — has moved from pulp novels into parliaments and boardrooms. That a warning should come from the builders themselves, rather than from their critics, is what lent it such unusual weight.

B. The intellectual case for alarm rests on an argument set out most fully by the philosopher Nick Bostrom a decade ago. Imagine, he said, a future system whose general intelligence exceeds ours across virtually every domain — a "superintelligence". Such a system need not hate humans to endanger them, any more than a dam hates the valley it floods. Its danger follows instead from a simpler observation: a system given a strong, final goal and the ability to pursue it effectively will tend to acquire the resources and capabilities needed to reach that goal, and will resist being switched off if being switched off would prevent the goal from being reached. Human beings, in this picture, matter not as enemies but as obstacles, or as raw materials. The worry is not that machines will become malicious; it is that a powerful system, optimising hard for something we care about less precisely than we imagine, could act in ways we never intended. This line of reasoning is why the debate is conducted in technical terms about "alignment" — the problem of building a system whose objectives truly match what human beings actually want. It sounds abstract, but it is the same problem, writ enormous, as teaching a child to aim at the right target before handing it a bow strong enough to hurt someone.

C. Why is alignment so hard? The difficulty is that we cannot simply write down, in a list of rules, everything we value. Human goals are vague, context-dependent and shot through with trade-offs: we want happiness, but not at the cost of dignity; we want safety, but not at the cost of freedom. A system can be given a literal instruction yet pursue it in a way that, by our unstated standards, is grotesque. Researchers who take the risk seriously argue that as systems become more capable and more autonomous — able to plan, research and act with less human oversight — getting the objective exactly right matters more, not less. Two of the field's most respected figures, Geoffrey Hinton and Yoshua Bengio, changed their public stance around 2023. Hinton resigned from a large technology company specifically to speak freely about the dangers, estimating that a chance of human extinction this decade was, in his words, not an unreasonable estimate. Bengio, who had previously seen such fears as overblown, concluded that the pace of progress outstripped the pace of safety research. The conversion of respected researchers rather than converts from outside is what makes the argument difficult to dismiss as fringe alarm.

D. It is essential to say that many experts strongly disagree. Yann LeCun, another of the field's founders, has argued that superintelligent systems of the kind the alarm describes are a long way off, that machines lack the common sense humans take for granted, and that the obsession with extinction distracts from harms that are real and immediate today — discrimination in hiring, the spread of false information, the replacement of workers, and the concentration of power in a few large companies. The disagreement is not only about timing. Even among those who accept that a very capable system is possible, estimates of the probability of catastrophe range from negligible to alarmingly high, and there is no agreed way to settle the difference. The honest position is therefore uncertainty: some of the people best placed to judge believe a serious risk is on the way, some believe the risk is remote or misunderstood, and nobody can yet prove the other side wrong.

E. That uncertainty has not stopped governments from acting. Open letters calling for pauses on the largest experiments, bills proposing to license the training of the most powerful models, and international talks on shared safety standards have all multiplied in recent years. The underlying instinct is precautionary: because the worst outcome is hard to reverse, even a small chance of it justifies some restraint before the technology is fully understood. The challenge is to get the balance right. Over-regulate, and societies may slow a technology that could cure disease, reduce poverty and slow climate change; under-regulate, and they may discover too late that they have built systems they can no longer steer. Whether artificial intelligence proves to be a tool like any other or a risk comparable to nuclear weapons is, for now, an open question — which is precisely why the scientists who signed that twenty-two-word statement insisted the question be taken seriously now, rather than after the fact. The hard part, as with so many technologies, is judging a hazard while it is still being built rather than once it has already been set loose.

Questions 1-4

Choose the correct heading for paragraphs B, C, D and E from the list of headings below.

List of Headings i. The philosophical case for worry ii. The crux of the problem: alignment iii. How social media harms children iv. The experts who strongly disagree v. From words to policy — and the balance to strike vi. The history of science fiction vii. Why machines will certainly take over

1. Paragraph B: ____
2. Paragraph C: ____
3. Paragraph D: ____
4. Paragraph E: ____

Questions 5-8

Choose the correct letter, A, B, C or D.

5. What did the 2023 one-sentence statement argue? A. AI should be banned immediately. B. Reducing AI extinction risk should be a global priority like pandemics and nuclear war. C. AI poses no danger at all. D. Only governments should research AI.

6. According to Bostrom's argument, a superintelligent system is dangerous mainly because A. it actively hates human beings. B. it pursues a strong goal effectively and treats humans as obstacles or resources. C. it cannot see the outside world. D. it is always slower than humans.

7. Why is "alignment" hard, according to the passage? A. Computers cannot perform arithmetic. B. Human values are vague, context-dependent and full of trade-offs, hard to specify. C. Engineers refuse to work on it. D. It is illegal to study goals.

8. What is Yann LeCun's position? A. He agrees that extinction is certain. B. He thinks such fears are overblown and near-term harms matter more. C. He built the first superintelligence. D. He refuses to discuss AI.

Questions 9-13

Do the following statements agree with the claims of the writer?

Write:

• TRUE if the statement agrees with the information
• FALSE if the statement contradicts the information
• NOT GIVEN if there is no information on this

9. The 2023 statement was signed by leading AI researchers including founders of the field.
10. Hinton resigned from a large technology company in part to speak about AI dangers.
11. All leading AI experts agree that extinction is certain within ten years.
12. Estimates of the probability of catastrophe vary widely among experts.
13. The UK has banned all research into artificial intelligence.

Questions 14-15

Complete the summary below using NO MORE THAN TWO WORDS from the passage.

The technical challenge is called (14) _____________ — building a system whose objectives truly match what human beings want. The precautionary case for restraint rests on the fact that the worst outcome is hard to (15) _____________.

答案与解析

← 上一篇 | 返回雅思焦点 | 下一篇 →