Blind spot
Goddamn AC in here’s like a child’s breath. First meetings are tough enough without getting this sweat on, and we’re still waiting on that refurb thanks to Homeland jumping the line. I still remember the grumbling in DOT when Homeland got their increase. You’ve never seen a pretzel line so long.
“Thank you all for being part of this historic ethics delegation. As you know, the Safer Cars on American Roads program has been active since January in Washington, Pennsylvania, and California. I think we can say that, minor bumps aside, it’s gone well at the state level.” A few Pittsburgh taxi drivers had thrown themselves under the Uber XC90s, but they’d earned only scant sympathy and six-figure medical bills.
“Before we progress to federal and NHTSA approval Meg has asked us to dialog the outstanding ethical questions. So that’s the role of this group.” Important at meetings like this to be metronomic with eye contact, to feign impartiality. There’s protocol, recordings. “Since we have a few newcomers, let’s start with brief introductions. Brief, please.”
I point my pen at the kid I’ve not met, who must be Eric. Twenty-five, six? He leans back in his chair. “I’m Eric Jo. I’m a staff engineer at Alphabet.” He means software engineer, of course – car people don’t wear those ridiculous watches. His t-shirt has ‘Google SDCP’ printed above that laughable cute fascia. Eric wears the resigned look of a young man left holding the short straw, glasses about five degrees off horizontal.
Next to Eric, Bud Carver, in his yellow Wednesday suspenders. Bud’s a professional fucking saboteur. Spends his days hovering in departmental meetings, complaining the meeting shouldn’t happen. I’ve protested to Meg of course, but the guy’s inner circle: some kind of Tea Party figure before Don threw him the DOT trimmer gig.
Been perhaps three years since I last saw Cecilia – she was at Mercedes forever, then went totally quiet. Like, nothing. Vanished from the mailing lists, wasn’t at the Denver convention, then we finally got word from Cupertino two months ago. Great engineer. We still don’t really know what they’re up to over there: all sorts of stories of cutbacks and prototypes. Not exactly a forthcoming bunch.
Of course, Facebook sent Roosevelt. Ever since the Post alleged their latest prototype brakes more heavily in front of Democrats, Legal stepped in. I tried to discourage lawyers – this is a working group after all – but it’s hard to change Facebook’s mind these days.
Dylan and I go way back. ASU, class of ’96. There were rumors Kalanick himself wanted in on the committee until Uber PR nixed the idea. Dylan wiped the floor with me at Spyglass last month: went round in 77 and won’t let me forget it. He emailed last week to announce his new title – “Director of Strategic Infrastructural Programs Direction” – and asking “to powwow about subsidy opportunities” before the next spending reviews.
“I’m Alexandrine Lang, I head up future projects at VW Group.” Three weeks back, an Audi A6 suddenly occupied Meg’s parking space, GM-Ford were out, and Alexandrine was on the list. The PR optics aren’t good since we let VW wriggle off the emissions hook, but at least they’re eager to please.
Finally, Floren Moïse. Heard he was the only Georgia Tech guy to decline the UPS offer. Meg’d be happy to ditch him, but Floren adds spice: specifically, he drives Bud crazy. I don’t know how exchange programs work in that world, but Atlanta has eroded his accent somewhat. He’s wearing one of those collarless shirts, buttoned to the top. A dapper academic: who’d have thought?
“Thanks everyone. Jaxson sends his apologies – as you know, the Autopilot hearings are ongoing and delicate, so we agreed Tesla should step back for now.”
The coffee pot spits in the corner. A fern wilts silently in the heat.
“Our first order of business is federal border protocol. Our friends at Justice tell me you’ve all implemented their patrol-car stop recommendations; they pass on their thanks. We now have to consider how to handle international crossings.
“Per last week’s memo, Homeland has requested not only the checkpoint dead-zone but remote shutdown capability within a 3-mile radius. This would mostly be an issue on the Canadian border, of course, but while California’s dragging its heels over the Wall, it may be relevant in the south too.”
“Three miles is going to be risky," says Eric. "From an engineering point of view of course there’s no obstacle, but at that range I don’t think we can guarantee safe shutdown in mixed-autonomy traffic, unless we have the specialized lane filters.”
Cecilia speaks precisely over steepled hands. “Our view is that a remote disable sets a worrying precedent. A backdoor is a euphemism for a vulnerability, after all.” She hesitates. “I must add however that Apple has no official stance on autonomous vehicle behavior, and I am unable to confirm or deny my presence at this delegation.”
“Look, the Russians are basically all over our asses at this point, Tom,” says Dylan. “Putin’s got a gifted crew behind him now, and if we give Homeland a backdoor it’s just a matter of time until the Russians jack-knife a semi on I-5. Christ, Tom, it’ll be a mess. Uber… they don’t want to put passenger lives in danger.”
“But the true problem,” interrupts Floren, “is we would be subverting the established capitalist model of ownership, we reframe the relations of subject-object. Are we morally justified in imposing this ludicrous proto-authoritarianism on the world?”
It’s usually harmless to let Floren lap the course a few times, to let him exercise those post-nominals and flap about us focusing on the wrong horizon. By the time he’s expounded upon the distinction between deontology and consequentialism, Bud’s eyelids are twitching.
“Let’s get back on track, folks,” I say. “Frankly, I’m not worried about a little top-down control. Our focus groups tell us that, except for the usual ACLU types, citizens are comfortable with authority intervention, so long as it’s framed as a safety measure. But let’s cut the crap – we all know the deal with the remote shutdown. You just want deniability in case another Snowden crawls out.”
“Well, silence that made whole scenario far more painful than it needed to be,” says Roosevelt. “If DOJ actually allowed us to disclose interventions I’m sure we’d all be warmer to the idea. This is basic First Amendment stuff, Tom, and as you know we’ve been over it a thousand times with any department that will listen. And since the FBI disclosure cases are still in this mysterious holding pattern, we don’t seem to be getting anywhere.”
“Yeah, okay, look – it’s between you and Homeland really. We’re just the messenger on this one,” I say.
“I think the moral angle has to be secondary to the legal case. If Homeland makes a formal request we’ll consider it through the proper channels. Until then, it’s a no.”
Bud, already on his third cookie, is staring at Alexandrine. A small column of saliva pulses between his open lips. She shifts awkwardly in her seat.
“Fine,” I say, “I’ll circle back with Homeland’s counsel. They seemed eager, so my guess is this’ll come back. Anyway, since we’re on legal turf, let’s move into collision liability.” Bud shuffles his papers; everyone else swipes their tablet. “Per the April ISO draft, vendors will assume liability in level four-compliant autonomous modes. So we still have to thrash out what we’re doing about laggard firmware. Uh, Dylan, can you update us on Uber’s stance here?”
“Sure,” Dylan says. “They’re now prepared to disable vehicles once they’re 30 days behind the upgrade curve. But as you know, inventory is Uber’s priority – vehicles out of service really hurt their passenger load factor. So for lesser lags the passenger will see a HUD warning, and if they dismiss it, liability shifts to the passenger or his insurer per EULA.”
“Okay, thanks. Alexandrine, since VW is new to the group, can you fill us in on their thoughts?”
“We’ve carried out surveys to crowdsource our collision heuristics, which we expect to underpin all software versions. So patching shouldn’t be an issue. Actually, the surveys have been very illuminating. They’re based on something called the trolley problem – have you all heard of it?”
An exasperated crunch escapes from Floren’s throat, and Cecilia and I exchange a glance. Eric takes pity first. “Look… the trolley problem is basically useless in real life. If you find yourself crashing it’s because you fucked up three seconds earlier, and the answer is almost always just to smash the brakes. Steering makes you lose more control, and a few mph might make all the difference.”
“Gotta say I’m with Eric here, Alexa,” says Dylan.
“Alexandrine.”
“Oh – Alexandra, sure. Sounds like you should lean on your machine learning guys more. Predictive heuristics are a waste of time. Ship with an adaptive ML system, learn from fleet damage, injury records, y’know? Let the algorithm choose a response based on previous patterns.”
“Nom de bleu,” cries Floren. “You tech people are like this always. Such wretched solutionism! Kill a few people but never mind, just tweak the algorithm! It’s… it’s indecent,” – his chair rocks forward – “the way you wash your hands of moral agency, when it is through your own technology that these futures are mediated.”
Bud waves a disdainful hand. “Now Floren, I’m sure we all read your piece in Armchair Ethicist Weekly, but this is about American industry. The real world, you get it? This isn’t the time for philosophizing.”
“Monsieur Carver, now is precisely the time for philosophizing!”
I have to step in. “Okay, so Alexandrine, I think we need to cover VW’s approach in more detail before the next licensing committee. I’ll ask the team to review with you next week.
“I guess we should move on to the final agenda item, which is HIMs of course. As you know, highly intimate moments in moving vehicles are mostly covered by states’ reckless driving or indecency laws. But, er, certain… media properties are concerned about moral corruption – tinted windscreens, mobile brothels, all that panic. So I need to bring Justice a recommendation that’s going to balance the realities of driverless time with this political aspect.”
Bud sits bolt upright. Crumbs fly from his mouth. “Gimme a break! Classic example of government meddling in the lives of ordinary people. Some guy wants to bust a nut in his car? Big deal. So long as it’s discreet, so long as it’s legal, let him.”
“Obv–”
“But nothing kinky.”
“Obviously our incognito mode disables internal cameras,” says Eric. “Or at least, it disables transmission anyway. But our view is essentially that this is just another connected device, and who are we to dictate use cases?”
“Discretion is important,” says Cecilia. “HIMs are, how should I say, seen as incompatible with our brand. As you know, we’ve disallowed adult content on the App Store for years. But also, people will do what people do. Our primary concern is inadvertent audio triggering by user vocalizations. I’ll have the Siri engineering team provide an update, although I can’t confirm whether such a team exists.”
“Facebook is keen to keep this area as loose as possible, Tom,” says Roosevelt. “As you know, intimacy has proven to be a surprising growth area for Oculus, and we think it would be a shame if overregulation got in the way of these exciting new forms of self-expression. Not to mention the potential constitutional issues.”
***
“Hi Meg. Good weekend?”
“Ah Tom. Yes, thanks. Kids up, so you know: zoo, tantrums.”
“So, you wanted to see me?”
“Yes, sit down. How are things with the ethics panel?”
“Fine. Still working with VW on their heuristics. They’re, well, they’re a long way behind. But Homeland have backed off remote disable for now: something about reassessing terrorism vectors, new strategy soon. We’re meeting again early next month. Tenth maybe.”
“Ok, well I have an update from my end. Seems Bud’s been talking to Dylan and the other vendors since the meeting, and… they’ve tabled a new approach. They’ve asked that we look again at self-regulation…”
“Oh, you’ve got to be—”
“…and DOT’s agreed we’ll suspend the delegation…”
“—shitting me!”
“…to let vendors choose their own approaches. We’ll monitor progress with DOJ, and—“
“They screwed me!” Shot a 77, then shot me in the back.
“Oh, don’t be melodramatic, Tom. Look, word is Don himself had a hand in it: jobs on the line and all that. You know policy priorities take precedence; not a lot we can do from our level.”
I shake my head. “Unbelievable. Wait, all of them? Floren?”
“Oh, Floren.” Meg laughs. “He’s fuming, of course, but Legal waved the NDA and he calmed down.”
“Motherfuckers.”
“Look, Tom, I get that you’re upset but really, I think the ethics thing is overblown. These guys are innovators, remember? Smart people. It’s not good business for Silicon Valley to do unethical things; the market will iron out any misbehavior.” She pauses to sip her coffee. “And besides, if things do start out a little hairy, we still have plenty of budget for signage and public information programs.”
