Book review: Design for Safety
Just sometimes, the responsible tech movement can be frustratingly myopic. Superintelligence and the addiction economy command the op-eds and documentaries while privacy and disinformation, important as they are, often seem captured by the field’s demagogic fringe. But there are other real and immediate threats we’ve overlooked. In Design for Safety, Eva PenzeyMoog pushes for user safety to be more prominent in the ethical tech conversation, pinpointing how technologies are exploited by abusers and how industry carelessness puts vulnerable users at risk.
The present tense is important here. The book’s sharpest observation, and the one that should sting readers the most, is that the damage is already happening. Anticipating potential harms is a large part of ethical tech practice: what could go wrong despite our best intentions? For PenzeyMoog, the issue isn’t conditional; she rightly points out abusers already track and harm victims using technology.
I’m very intentional about discussing that people will abuse our products rather than framing it in terms of what might happen. If abuse is possible, it’s only a matter of time until it happens. There is no might.
With each new technology, a new vector for domestic abuse and violence. We’re already familiar with the smart hauntings of IoT: abusers meddling with Nest thermostats or flicking on Hue lights, scaring and gaslighting victims. But the threat grows for newer forms of connected technology. Smart cars, cameras, and locks are doubly dangerous in the hands of abusers, who can infringe upon victims’ safety and privacy in their homes or even deny them a means to escape abuse.
While ethical tech books often lean closer to philosophy than practice, A Book Apart publishes works with a practical leaning. PenzeyMoog helpfully illustrates specific design tactics to reduce the risk of abuse, from increased friction for high-risk cases (an important tactic across much of responsible design), through offering proof of abuse through audit logging, to better protocols for joint account ownership: who gets custody of the algorithm after a separation?
Tactics like this need air cover. Given the industry’s blindspot for abuse, company leaders won’t sanction this extra work unless they understand its necessity. PenzeyMoog suggests public data is the most persuasive tool we have. It’s hard to argue against the alarming CDC stat that more than 1 in 3 women and more than 1 in 4 men in the US have experienced rape, physical violence, and/or stalking by an intimate partner.
Central to PenzeyMoog’s process is an admission that empathy has limits. While we should certainly try to anticipate how our decisions may cause harm, our efforts will always be limited by our perspectives:
‘We can’t pretend that our empathy is as good as having lived those experiences ourselves. Empathy is not a stand-in for representation.’
The book therefore tackles this gap head-on, describing how to conduct primary research with both advocates and survivors, adding valuable advice on handling this task with sensitivity while managing your own emotional reaction to challenging testimony.
Tech writers and publishers often seem reluctant to call out bad practice in print, but Design for Safety is unafraid to talk about what really matters. One highlight is a heartening, entirely justified excoriation of Ring. Amazon’s smart doorbell is a dream for curtain-twitchers and authoritarians, eroding personal consent and private space. PenzeyMoog argues one of Ring’s biggest defects is that it pushes the legal and ethical burden onto individual users:
‘Most buyers will reasonably assume that if this product is on the market, using it as the advertising suggests is within their legal rights.’
That legal status is itself far from clear: twelve US states require that all parties in a conversation consent to audio recording. But the moral issue is more important. By claiming the law is a suitable moral baseline, Ring pulls a common sleight of hand, but for obvious reasons (countries and states have different laws; morality and law change with time; many unethical acts are legal) this is sheer sophistry. Ring has deep ethical deficiencies: we mustn’t allow this questionable appeal to legality deflect from the product’s issues.
Design for Safety also takes a welcome and brave stance on the conundrum of individual vs. systemic change. It’s popular today to wave away individual action, arguing it can’t make a dent in entrenched systems; climate campaigners are familiar with the whataboutery that decries energy giants while ignoring the consumer demand that precipitates these companies’ (admittedly awful) emissions. Design for Safety makes no such faulty dismissals. PenzeyMoog skilfully ‘yes and’s the argument, agreeing that attack on any one front will always be limited, but contending that we should push tactical product changes while also trying to influence internal and industry-level attitudes and incentives.
‘We don’t need to choose between individual-level and system-level changes; we can do both at once. In fact, we need to do both at once.’
This is precisely the passionate but clear-headed thinking we need from ethical technologists, and it makes Design for Safety an important addition to the responsible design canon. If I have a criticism, it’s the author’s decision to overlook harassment and abuse that originates in technology itself (particularly social media). Instead, PenzeyMoog focuses just on real-world abuse that’s amplified by technology. Having seen Twitter’s woeful inaction over Gamergate from the inside, I know that abuse that emanates from anonymous, hostile users of tech can also damage lives and leave disfiguring scars. The author points out other books on the topic exist – true, but few are written as well and as incisively as this.
Design for Safety is a convincing, actionable, and necessary book that should establish user safety as a frontier of modern design. Technologists are running out of excuses to ignore it.
Ethics statement: I purchased the book through my company NowNext, and received no payment or other incentive for the review. I was previously a paid columnist of A List Apart, the partner publisher of A Book Apart. There are no affiliate links on this post.
