不用動(dòng)腦動(dòng)手——自動(dòng)化時(shí)代的隱患

Robert Siegel （Host）： And I’m Robert Siegel， on the road for a talk with author Nicholas Carr. Carr’s books are the 1）nagging， 2）tech-wary 3）conscience of the digital age. In The Shallows， he warned that surfing the Internet is destroying our 4）attention span. And now in his new book， The Glass Cage， Nicholas Carr warns us that computers are starting more and more to make decisions for us and that we risk forgetting how to make those decisions ourselves. He writes a lot about cars. So we are going to pick him up in this 5）state-of-the-art， high-tech driving machine—a 2014 Mercedes-Benz S550 4Matic 6）sedan. It’s a 7）loaner， and Michael Minielly of Mercedes-Benz USA is our 8）chaperone. Thank you very much for lending us this car this morning.

Michael Minielly： Absolutely. Happy to be here.

Siegel： And since this car so far out of my league—talking about a hundred-twenty-thousand-dollar car here？ Minielly： Just about that. Yeah.

Siegel： You’re going to have to explain to me all the amazing things this car can do for me without me having to do them.

Minielly： Absolutely.

（Soundbite of car door closing）

Siegel： This is Mercedes’ 9）top-of-the-line， highest tech model car you can drive. It parks itself. It controls the 10）windshield wipers. Driving in the dark？

Minielly： So we have a system called active 11）high-beam assist， which 12）automatically 13）illuminate the road in front of you.

Siegel： And if a car’s approaching in the opposite direction？

Minielly： It will automatically lower them and continuously lower them to the amount where it’s—it will never 14）dazzle the driver.

Siegel： Talk about dazzling the driver—this car has something called Distronic Plus.

Minielly： There’s the 15）stereo camera， and then there’s also short-， medium- and long-range radars that are continuously monitoring the traffic situation， both in front and behind the vehicle.

Siegel： At one point， as I was driving in rush hour traffic to Nicholas Carr’s hotel in Washington D.C.， I tried this Distronic Plus system out. The system kept me in my lane. Had I 16）strayed， it would’ve taken over the steering. And it maintained my distance from the taxi that had just cut in front. The car ahead of me is moving， so...

Minielly： So you don’t have to use your...

Siegel： The car is following it.

Minielly： Yeah.

Siegel： I’m not 17）accelerating right now.

Minielly： That’s correct. And you’re not braking.

Siegel： I’m not braking. And now the car ahead of me is slowing down， so this car’s slowing down.

Minielly： Correct.

Siegel： No hands， no feet—the Mercedes was driving itself. Nick Carr， welcome.

Nicholas Carr： Thank you.

Siegel： I decided to pick you up in the All Things Considered mobile studio.

Carr： （Laughter） It’s very impressive.

Siegel： （Laughter） I have arrived driving Nick Carr’s nightmare. We set out for a glimpse of what’s to come in nearby Arlington， Virginia. And as I drove， we talked about automation and what it has done for us and to us.

（Soundbite of Global Positioning System）

Unidentified Woman： In 700 feet， turn right onto New Hampshire Avenue Northwest.

Siegel： For Nick Carr， automatic 18）navigation demonstrates how technology gives to human beings while also taking away.

Carr： At least you used to have to figure out where you were. And even with a paper map， you’d have to locate yourself somewhere and figure out what the landmarks around you are and kind of get a sense of place. And that’s no longer necessary，when you have， you know， the voice come on and say， in 500 yards， turn left—200 yards， turn right. I do think there’s something lost there.

Siegel： And it’s not just behind the wheel of a vehicle that can drive itself where Nicholas Carr sees a worrying loss of 19）autonomy.

Carr： Well， you see it in a lot of professions. So if you go in these days to have your annual physical， usually now the doctor’s spending a good deal of time looking into a computer to enter information that they used to dictate—notes about the visit—or write down themselves—to kind of go through processes， 20）templates that give them hints on diagnosis and stuff. Can be good—can be not so good， but changes the doctor-patient relationship in very interesting ways. Pilots—the name of the book The Glass Cage refers to what pilots call the glass cockpit—that more and more， they’re flying by looking at banks of computer monitors.

Siegel： But you acknowledge that， overall， the number of air crashes we’ve had over the past couple of decades since we’ve automated is way， way down.

Carr： It is. I mean， the whole history of 21）aviation is a history of flight becoming much， much safer. And some of that is definitely due to autopilots and automation systems. But I think what we’re seeing—and， you know， the Federal Aviation Administration is concerned about this and has sent out alerts—is that pilots just aren’t getting enough practice in 22）manual flying. Almost the entire flight now is done on autopilot. And so on those rare—thankfully， rare occasions when something unexpected and bad happens， pilots often make mistakes because their skills have gotten 23）rusty. So I think the lesson isn’t that automation is bad， but we have to be very wise in knowing how to automate and when to say， no， let’s not take more control away from the human being.

Siegel： I’ve been talking with Nicholas Carr， whose new book， The Glass Cage， is a—is a warning against the potential dangers of some kinds of automation. And so far， we’ve been in the present—at least， a very expensive version of the present—driving this amazing Mercedes-Benz. But now we’re at the Virginia Tech Transportation Institute， at their research center in Arlington， Virginia. And now we’re going to get a glimpse of the future with Ray Resendes. Tell us about the Cadillac we’re about to take a ride in.

Ray Resebdes： So we have a Cadillac SRX. We have added a technology called dedicated short-range communications. Coupled with GPS， the vehicle puts out a basic safety message， which， basically put， is saying， here I am. Here’s the location of a vehicle. Other cars equipped with the same technology can hear that and just like an air traffic control system， can start figuring out who’s in my path. Am I in an 24）imminent crash danger？

Siegel： Virginia Tech and the Virginia Department of Transportation have also placed short-range radio 25）transmitters on selected roads and highways. Ray Resendes’ Cadillac receives their 26）transmissions and displays them on the 27）dashboard navigation screen. We see a recommended speed on an 28）exit ramp. As we pass a school， we get a warning that schoolchildren are nearby. When a second Virginia Tech car outfitted like the Cadillac passes us， we get an alert. Nicholas Carr says he worries about people being bombarded with such automatic alerts in our future cars and everywhere else.

Carr： This is something called alert fatigue. Doctors now—when they use computers， they’ll often be alerted to possible drug 29）interaction problems. These alerts come up so often now that the doctors start just 30）dismissing them automatically because a lot of them are completely useless， it turns out. There is a big problem that the systems become so sensitive and give you so many alerts that you actually become less alert yourself because you’re just dismissing the alerts.

Siegel： And here’s another question that Nicholas Carr writes about. If the car of the future will make decisions for us， how will it decide what to do when a 31）collision is unavoidable， and a computer is in charge of the steering？

Carr： You have to start programming difficult 32）moral 33）ethical decisions into the car. You know， if you are gonna crash into something， what do you crash into？ You know， do you go off the road and crash into a 34）telephone pole rather than hitting a 35）pedestrian？ Or—you know， things that we do—and we might not do them that well all the time， but...

Resebdes： Very interesting answer ’cause if you look at crashes， the best crash to get into is to rear-end somebody else ’cause cars are well-designed to deal with that. A lot of times， people will steer to avoid a 36）rear end collision. And then you can run into 37）head-on collision， which is the worst crash， or you can hit a pedestrian. Having to 38）codify these issues into the 39）algorithm on a vehicle is a—is a very serious issue.

Siegel： That’s Ray Resendes of Virginia Tech. Nicholas Carr’s complaint against 40）intrusive automation isn’t just about how well or how poorly computers might make moral decisions for us. It’s about the very 41）erosion of human autonomy.

Carr： Once we start taking our moral thinking and moral decision-making away from us and putting it into the hands not of a machine， really， but of the programmers of that machine， then I think we’re starting to give up something essential to what it means to be a human being.

Siegel： That’s Nicholas Carr， author of The Glass Cage： Automation and Us. By the way， he told me that he drives an Audi A4 with no navigation system.

羅伯特·西格爾（主持人）：我是羅伯特·西格爾，我正在去采訪作家尼古拉斯·卡爾的路上?？柕臅?shū)是有關(guān)這個(gè)令人對(duì)科技產(chǎn)生不安與警覺(jué)的數(shù)字時(shí)代。在《淺薄》一書(shū)中，他提醒我們上網(wǎng)會(huì)影響我們注意力的廣度。如今在他的新書(shū)——《玻璃箱》中，尼古拉斯·卡爾提醒我們計(jì)算機(jī)開(kāi)始為我們作越來(lái)越多的決定，我們承擔(dān)著遺忘如何自己作決定的風(fēng)險(xiǎn)。他寫(xiě)了許多有關(guān)汽車方面的問(wèn)題。所以我們將要用這臺(tái)最先進(jìn)的、高科技的駕駛機(jī)器——2014年出的型號(hào)為S550的四輪驅(qū)動(dòng)奔馳轎車去接他。這輛車是借來(lái)的，美國(guó)奔馳公司的麥克爾·米尼利是我們的陪同人員。非常感謝（貴公司）今早能把車借給我們。

麥克爾·米尼利：應(yīng)該的。非常高興能來(lái)到這里。

西格爾：因?yàn)檫@輛車讓我望塵莫及——它應(yīng)該價(jià)值12萬(wàn)美元吧？

米尼利：差不多吧，嗯。

西格爾：不用我來(lái)說(shuō)，你會(huì)給我解釋這輛車能為我做的所有神奇的事情的。米尼利：當(dāng)然了。

（車門(mén)關(guān)閉的聲音）

西格爾：這是奔馳最高科技的頂級(jí)轎車。它能自行停車、操控雨刮器。如果在夜間行駛呢？

米尼利：我們有一個(gè)叫活躍遠(yuǎn)光燈輔助的系統(tǒng)，它能自動(dòng)為你的前路照明。

西格爾：如果有一輛車從相反的方向駛來(lái)呢？

米尼利：那么這個(gè)系統(tǒng)就會(huì)自動(dòng)把光調(diào)暗、持續(xù)把光調(diào)暗到一定程度——它從不會(huì)讓司機(jī)感到眼花。西格爾：說(shuō)到讓司機(jī)感到眼花——這輛車有一個(gè)增強(qiáng)型限距控制系統(tǒng)。

米尼利：車上裝有立體照相機(jī)，在前后都裝有短程、中程、遠(yuǎn)程的雷達(dá)持續(xù)掌握交通狀況。

西格爾：在某段時(shí)間，我正在高峰期開(kāi)車去尼古拉斯·卡爾在華盛頓下榻的酒店，我試了下這個(gè)增強(qiáng)型限距控制系統(tǒng)。這個(gè)系統(tǒng)讓我能保持在一條車道上。如果我偏離了車道，它會(huì)接手控制方向。它能保持我與剛才前面突然插進(jìn)來(lái)的的士之間的距離。前面的車在移動(dòng)，所以……

米尼利：所以你不需要用你的……

西格爾：這輛車在跟著它。

米尼利：嗯。

西格爾：我現(xiàn)在沒(méi)有加速。

米尼利：沒(méi)錯(cuò)。你也沒(méi)有剎車。

西格爾：我沒(méi)有剎車?，F(xiàn)在我前面的車在逐漸減速，所以這輛車也開(kāi)始逐漸減速。

米尼利：沒(méi)錯(cuò)。

西格爾：沒(méi)有用手，沒(méi)有用腳——這輛奔馳能自動(dòng)駕駛。尼克·卡爾，歡迎你。尼古拉斯·凱爾：謝謝。

西格爾：我決定在《面面俱到》的移動(dòng)演播室里迎接你?？枺海ㄐΓ┻@真讓人印象深刻。

西格爾：（笑）我開(kāi)著尼克·凱爾的“噩夢(mèng)”而來(lái)。我們

出發(fā)去看看弗吉尼亞的阿林頓周邊的情況。我邊開(kāi)車，邊和他討論著自動(dòng)化以及它給我們所帶來(lái)的影響。（全球定位系統(tǒng)的聲音）不知名女士：前方700英尺，向右進(jìn)入新罕布什爾州大道西北段。

西格爾：在尼克·卡爾看來(lái)，自動(dòng)駕駛展現(xiàn)了科技為人們帶來(lái)的利弊。卡爾：至少過(guò)去你需要找出你在哪兒。就算有一張地圖，你也需要了解自己的位置，找出周圍的標(biāo)志性建筑，有一種位置感。（如今）這些都不需要了，當(dāng)你有需要，你懂的，一個(gè)聲音就會(huì)出現(xiàn)：前方500碼，轉(zhuǎn)左；前方200碼，轉(zhuǎn)右。我真的認(rèn)為那樣會(huì)有問(wèn)題。

西格爾：能自動(dòng)駕駛的汽車只是尼古拉斯·卡爾擔(dān)心人們丟失自主能力的一個(gè)方面。

卡爾：好吧，你知道自動(dòng)化在各行各業(yè)都有滲透。因此如果你要去進(jìn)行每年一次的體檢，現(xiàn)在的醫(yī)生通常都會(huì)花很多時(shí)間去查看他們過(guò)去在電腦里記錄或者自己寫(xiě)下來(lái)的信息——比如門(mén)診筆記。反正就是一種走程序的形式，按照模板來(lái)進(jìn)行診斷。這樣做有可能好也有可能壞，但卻非常有趣地改變了醫(yī)生與患者間的關(guān)系。飛行員——那本叫《玻璃箱》的書(shū)說(shuō)的是飛行員們口中的玻璃座艙——飛行員越來(lái)越依賴一排排的計(jì)算機(jī)顯示器來(lái)進(jìn)行飛行。

西格爾：但是你得承認(rèn)，自從我們進(jìn)行自動(dòng)化飛行以后，在過(guò)去的好幾十年里總的來(lái)說(shuō)墜機(jī)事件的數(shù)量在直線下降。

卡爾：確實(shí)是。我的意思是，整個(gè)航空史就是讓飛行越來(lái)越安全的歷史。這其中一部分的原因當(dāng)然是因?yàn)椴捎昧俗詣?dòng)駕駛儀和自動(dòng)化的系統(tǒng)。但是我認(rèn)為我們所看到的是——你知道的，美國(guó)聯(lián)邦航空管理局所擔(dān)心并且發(fā)出警示的是——飛行員沒(méi)有獲得足夠手動(dòng)駕駛飛機(jī)的訓(xùn)練?，F(xiàn)在幾乎所有的航行都由自動(dòng)駕駛儀來(lái)完成。所以在有些時(shí)候，當(dāng)一些意想不到的糟糕情況發(fā)生時(shí)，飛行員通常都會(huì)犯錯(cuò)誤，因?yàn)樗麄兊募寄芤呀?jīng)不夠嫻熟，而慶幸的是這種情況很少發(fā)生。所以我想從中得出的教訓(xùn)不是自動(dòng)化不好，而是我們必須明智地知道什么時(shí)候應(yīng)該使用自動(dòng)化，什么時(shí)候不應(yīng)該使用，我們不要過(guò)多地剝奪了人類的自主控制。

西格爾：我一直在和尼古拉斯·卡爾聊天，他的新書(shū)《玻璃箱》提醒我們一些自動(dòng)化使用的潛在危機(jī)。目前為止，我們還在現(xiàn)實(shí)中，至少是一個(gè)非常昂貴的現(xiàn)實(shí)——開(kāi)著那么棒的奔馳。但是現(xiàn)在，我們身處弗吉尼亞大學(xué)交通學(xué)院所在阿林頓的研究中心，我們要與雷·雷森迪一道放眼未來(lái)。跟我們說(shuō)說(shuō)這臺(tái)即將載我們?nèi)ザ碉L(fēng)的凱迪拉克吧。

雷·雷森迪：那么我們有一臺(tái)型號(hào)為SPX的凱迪拉克?，F(xiàn)在我們?cè)谲噧?nèi)加入了一種叫專用短程通信的科技，與全球定位系統(tǒng)一道，這輛車可以發(fā)出一種基本的安全信息。大致上就是能說(shuō)：“我在這兒，這里是車的所在位置?！蓖瑯訐碛羞@一科技的其他車輛就能接收到信息，就像空中交通控制系統(tǒng)一樣，我就能開(kāi)始知道誰(shuí)跟自己同道。我即將會(huì)有發(fā)生碰撞的危險(xiǎn)嗎？

西格爾：弗吉尼亞理工大學(xué)以及弗吉尼亞的交通部門(mén)也在選擇的路段和公路中裝上短程的無(wú)線電廣播發(fā)射機(jī)。雷·雷森迪的凱迪拉克接收到了他們傳送的信息，并在儀表板導(dǎo)航屏幕上顯示出來(lái)。我們?cè)诮?jīng)過(guò)出口匝道時(shí)看到了建議時(shí)速；當(dāng)我們經(jīng)過(guò)一所學(xué)校時(shí)，我們收到了附近有學(xué)生出行的提醒。當(dāng)另外一輛跟這輛凱迪拉克有同樣配置的弗吉尼亞理工的轎車經(jīng)過(guò)我們時(shí)，我們收到了警示。尼古拉斯·卡爾說(shuō)他擔(dān)心未來(lái)的轎車或其他每個(gè)地方都會(huì)受到自動(dòng)警報(bào)的轟炸。

卡爾：這叫作警惕疲勞。現(xiàn)在的醫(yī)生——當(dāng)他們?cè)谑褂糜?jì)算機(jī)的時(shí)候，會(huì)經(jīng)常被提醒藥物交互作用可能產(chǎn)生的問(wèn)題。這些提醒出現(xiàn)得太頻繁，以至于醫(yī)生都自動(dòng)忽略它們，因?yàn)樵S多警示都是毫無(wú)作用的。這里有一個(gè)很大的問(wèn)題，這個(gè)系統(tǒng)變得如此敏感，它給了你那么多的警示以至于你自己實(shí)際上會(huì)變得沒(méi)那么敏感，因?yàn)槟阋恢痹诤雎阅切┚尽?/p>

西格爾：尼古拉斯·卡爾還寫(xiě)到了另外一個(gè)問(wèn)題。如果未來(lái)的汽車能為我們作決定，當(dāng)計(jì)算機(jī)操控著駕駛時(shí)，那么當(dāng)碰撞不可避免時(shí)它將如何作出決定？

卡爾：這樣你就必須在車內(nèi)編寫(xiě)復(fù)雜的倫理道德決定程序。你懂的，如果你將要撞到某個(gè)東西，該撞什么呢？你懂的，你會(huì)開(kāi)到路肩以外而撞向那個(gè)電線桿而不是撞向一個(gè)路人嗎？或者，你懂的，我們做的事——我們不可能總是能做得那么好，但是……

雷森迪：這個(gè)答案很有趣。因?yàn)槿绻阊芯寇囕v的碰撞問(wèn)題，就會(huì)發(fā)現(xiàn)最理想的碰撞就是追尾，因?yàn)槠嚨脑O(shè)計(jì)能很好地處理這個(gè)問(wèn)題。很多時(shí)候，人們都能避免追尾的碰撞。但是你也有可能遇到正面的碰撞，這就是最可怕的，或者你可能會(huì)撞到一個(gè)行人。把這些問(wèn)題編成計(jì)算程序運(yùn)用到汽車上是一件很值得認(rèn)真思考的事。西格爾：這是弗吉尼亞理工大學(xué)的雷·雷森迪。尼古拉斯·卡爾對(duì)侵入性自動(dòng)化的抱怨不僅僅是關(guān)于計(jì)算機(jī)能給我們作出多好或者多壞的道德決定，而是關(guān)于人類自主權(quán)被侵蝕的問(wèn)題。

卡爾：一旦我們開(kāi)始剝奪我們的道德思維和道德決策，沒(méi)有把它交給機(jī)器，真的，而是交到編程人員的手中時(shí)，那么我想我們就已經(jīng)開(kāi)始放棄對(duì)人類來(lái)說(shuō)至關(guān)重要的東西了。

西格爾：這是尼古拉斯·卡爾，他是《玻璃箱：自動(dòng)化與我們》的作者。順便說(shuō)一句，他告訴我他現(xiàn)在開(kāi)的是一輛沒(méi)有導(dǎo)航系統(tǒng)的奧迪A4。