[greybox]
A review of
The Glass Cage:
Automation and Us
By Nicholas Carr
About this book
A good reference for UX Theory, Case Studies
Primary audience: Designers and technical roles with some or significant experience with topic
Writing style: Matter-of-fact, mostly text
Publisher W.W. Norton, 2014, 261 pages, 9 chapters
Learn more about our review guidelines
[/greybox]
The Glass Cage: Automation and Us by Nicholas Carr explores our ever-changing relationship with automation. Although it may not initially seem like a design-related text, so much of this book speaks directly to the practice. Through a series of case studies, Carr explores what happens when the balance of power tips too far in favor of automation. For designers, this book offers a fascinating introduction to the field of human-computer cooperation, and shows why careful design is so important.
At its most basic, Carr’s premise is that we are including ever more automation in our products and that this isn’t always a good thing. The book opens with an assessment of aircraft autopilots and automatic transmissions. These two relatively recent inventions are good examples of how technology can change our fundamental approach to a task. Before their introduction, Carr points out that we took pride in using these tools—pilots and drivers alike were directly in control of the machines they operated, functioning as extensions of their body.
Today, many of the tasks that would once would have required continuous human input have been automated away, and for the most part this has had clear benefits. But in many cases we have reached a saturation point where we are now seeing issues relating to over-automation. For example, Carr describes a problem in the airline industry referred to as “deskilling.” This refers to the slow degradation of manual flying skills due to lack of practice.
“When onboard computer systems fail to work as intended or other unexpected problems arise during a flight, pilots are forced to take manual control of the plane,” he writes. “Thrust abruptly into a now rare role, they too often make mistakes.”
This slow decline in skill, also referred to as “the degradation effect,” is not limited to the aircraft industry. Carr shows how programmers, architects, and business analysts all fall victim to deskilling in one way or another. As more of our life becomes automated, we are moving from the role of active participant to that of passive observer.
Although there were many useful takeaways in this book, two concepts stood out to me as being particularly applicable to design: automation bias and automation complacency. Complacency happens when users miss important information or events because they assume that it’s being handled by the machine. Bias, on the other hand, is when users assume that the machine is able to perform a task better than a human. In many cases, computers are indeed able to outperform us, but not always. Carr even debates the fundamental assumptions held by many in the tech industry. “The conviction that we can build an entirely self-sufficient, entirely reliable automated system is itself a manifestation of automation bias.”
The underlying message here is not that task delegation is bad per se, but that it should be more carefully applied. When we delegate, there is always a trade-off being made, and in some contexts it may be beneficial to avoid it all together. By purposefully choosing to have the user manually perform functions that could otherwise be completed by the machine, the designer can help manage issues like bias, complacency, or deskilling. Although this may run counter to intuition, a little bit of friction in the experience can be helpful to make the user step back and really engage with the material.
On a deeper level, this book also considers how the products we’re building are adopted in society and the philosophical role of technology in the human experience. For example, when discussing our reliance on new systems, Carr writes:
“In its early development, technology is malleable; its form and use can be shaped not only by the desires of its designers but also by the concerns of those who use it and the interests of society as a whole. But once the technology becomes embedded in physical infrastructure, commercial and economic arrangements, and personal and political norms and expectations, changing it becomes enormously difficult.”
As we build better tools, we shape the way life is carried out. This, in turn, changes the role that we play in the world. Our personal agency is affected by the tools we use, and a major change is under way.
I was especially struck by the latter part of the book when Carr examines our entire relationship with automation through the lens of poetry. Through artistic words, we learn about the nature of work and what it means to live a satisfactory life. He argues that it is through labor that we find joy in this world—not through idle riches. It is the deep satisfaction of a job done well, and for its own sake, that makes life worth living. Almost without exception, the products being designed today promise less engagement with our work. We can sit back and relax while the machine does the tedious labor for us. But Carr argues that this is a mistake: “when automation distances us from our work, when it gets between us and the world, it erases the artistry from our lives.”
This book presents so many interesting topics for discussion that directly benefit the design community. On this first read, I feel like I’ve only started to scratch the surface, but in the end, I see the overall message of this book is a positive one. Yes, our technology shapes us in many unexpected ways, but that does not mean that we are helpless. As designers, we play a direct role in the ways automation is used, and we can shape the resulting experience. It’s a good reminder that we all have agency in this world, and that the joys of labor are available to us all.
[bluebox]
World and Screen
[R]esearchers emphasize that more research needs to be done before we’ll know for sure whether long-term use of GPS devices weakens memory and raises the risk of senility. But given all we’ve learned about the close links between navigation, the hippocampus, and memory, it is entirely plausible that avoiding the work of figuring out where we are and where we’re going may have unforeseen and less-than-salubrious consequences. Because memory is what enables us not only to recall past events but to respond intelligently to present events and plan for future ones, any degradation in its functioning would tend to diminish the quality of our lives.
Through hundreds of thousands of years, evolution has fit our bodies and minds to the environment. We’ve been formed by being, to appropriate a couple of lines from the poet Wordsworth,
Rolled round in earth’s diurnal course,
With rocks, and stones, and trees
The automation of wayfinding distances us from the environment that shaped us. It encourages us to observe and manipulate symbols on screens rather than attend to real things in real places. The labors our obliging deities would have us to see as mere drudgery may turn out to be vital to our fitness, happiness, and well-being. So Who cares? probably isn’t the right question. What we should be asking ourselves is, How far from the world do we want to retreat?
[/bluebox]
