Automation complacency: a cuneate facet

Raoul Hausmann, Mechanischer Kopf (Der Geist
Unserer Zeit)

Over the last couple of years I've followed Nick Carr's blog Rough Type fairly closely. Your appraisal of Carr's positions as those of either a thoughtful tech skeptic or an atavistic Luddite might well serve as a litmus test for digital nativism. (I prefer the former term, but then you'd probably file me under the latter.)

A recurring theme in Carr's writing is the overlooked or understated hazards of handing over an increasing volume of (formerly) human tasks to computers; this was the subject of his 2014 book, The Glass Cage: Automation and Us. A year prior to its publication, he wrote a piece for The Atlantic ("All Can Be Lost: The Risk of Putting Our Knowledge in the Hands of Machines") which expounds in some detail what he calls "automation complacency:"

Automation complacency occurs when a computer lulls us into a false sense of security. Confident that the machine will work flawlessly and handle any problem that crops up, we allow our attention to drift. We become disengaged from our work, and our awareness of what’s going on around us fades...

Examples of complacency ... have been well documented in high-risk situations——on flight decks and battlefields, in factory control rooms——but recent studies suggest that the problems can bedevil anyone working with a computer. Many radiologists today use analytical software to highlight suspicious areas on mammograms. Usually, the highlights aid in the discovery of disease. But they can also have the opposite effect. Biased by the software’s suggestions, radiologists may give cursory attention to the areas of an image that haven’t been highlighted, sometimes overlooking an early-stage tumor. Most of us have experienced complacency when at a computer. In using e-mail or word-processing software, we become less proficient proofreaders when we know that a spell-checker is at work....

In a classic 1983 article in the journal Automatica, Lisanne Bainbridge, an engineering psychologist at University College London, described a conundrum of computer automation. Because many system designers assume that human operators are “unreliable and inefficient,” at least when compared with a computer, they strive to give the operators as small a role as possible. People end up functioning as mere monitors, passive watchers of screens. That’s a job that humans, with our notoriously wandering minds, are especially bad at. Research on vigilance, dating back to studies of radar operators during World War II, shows that people have trouble maintaining their attention on a stable display of information for more than half an hour. “This means,” Bainbridge observed, “that it is humanly impossible to carry out the basic function of monitoring for unlikely abnormalities.” And because a person’s skills “deteriorate when they are not used,” even an experienced operator will eventually begin to act like an inexperienced one if restricted to just watching. The lack of awareness and the degradation of know-how raise the odds that when something goes wrong, the operator will react ineptly. The assumption that the human will be the weakest link in the system becomes self-fulfilling.

I often work a cash register. I'm not proud of it; I have a bachelor's in English and bills to pay.

Math was always my poorest subject in primary school; when I took my first retail job at the age of seventeen I was more than happy to entrust all arithmetical responsibility to the register. The operation proceeded splendidly until (for instance) I'd punch "20.00" into the CASH TENDERED field when handed a twenty-dollar bill for a $16.21 payment, only to be handed an additional $1.30 the customer had discovered in his jacket pocket.

"Uh," I'd say, eyes darting from the money in the customer's hand to the pocket calculator under the countertop to the line behind the customer to the computer screen (CHANGE TENDERED: $3.79) to my own hand in the open till to the customer's face. "Sorry. I already rang it in."

Then I'd turn a little red while I trusted the dude's word that I should just give him $5.09 and send him on his way.

At some point—maybe it was at the trendy supermarket in Maryland or the coffee shop in the Caribbean, who knows—I got in the habit of mentally calculating the difference between cost and cash and checking my result against the machine's (probably out of boredom, initially). These days I ring up every cash transaction as exact change and just do the arithmetic myself. It's faster. And if a customer surprises me with some extra coins and bills from their pocket, I can amend my arithmetic simply enough, while my coworkers who rely entirely on the software are often flabbergasted. Even though I've been in the same position myself, I can't help feeling a little embarrassed for them.

I don't wish for any of this to be read as a boast; being able to do basic addition/subtraction without a calculator isn't something I should (or do) take especial pride in. But I'm not saying any of my coworkers were/are stupid, either—they're just not in the habit of performing arithmetic on the fly, and are unprepared and confounded when the need arises.

But where learned helplessness is the norm, basic competence becomes a feat.

The Confidence-Man: His Masquerade

I mentioned a little while back I've been reading and very much enjoying Herman Melville's White-Jacket: or, The World in a Man-of-War (1850). Though I'm still a little ways from finishing it, I may eventually have to concur with early Melville critic/biographer Lewis Mumford's assessment of it as Melville's solidest and most fully-realized novel (aside from Moby-Dick, of course; but need that even be said?).

But White-Jacket isn't the Herman Melville Joint we'll be looking at today. I'd like to draw your attention instead to The Confidence-Man: His Masquerade (1857), Melville's ninth and final novel, and one of the strangest and most baffling damn books I've ever read.

This little review is long overdue. It was a full year ago now, during a short visit to the mainland (I was living in St Thomas then) that I found and snatched up a paperback copy of The Confidence-Man from the shelf of a little Maryland bookstore. I got around to reading it in May, in the wake of The Breakup, during the weeks I slept on a friend's couch at night and toured dismal island apartments by day. There could be no better time, I reckoned, to hitch my intellect to an unfamiliar Herman Melville novel, to get myself back on a somewhat even keel amid the chain-smoking uncertainty and jarring emotional aftershocks of the situation.

It was not the best idea I've ever had. Picking up The Confidence-Man in hopes of gaining some clarity was liking groping around a dim room for a light switch and activating a smoke machine instead.

One Fine Day in South Philly

In lieu of a lot of words, here's a few words and a few pictures [comic].

Click to read!

I draw comics when I get frustrated with writing. Before long I get frustrated with drawing and return to writing with renewed enthusiasm.

Pet causes: Herman Melville on corporal punishment

William H. Myers

Lately I've been reading White-Jacket, Herman Melville's fourth novel, whose publication in 1850 antedates that of his masterpiece Moby-Dick by one year. I'm really digging it so far—Melville's jaunty prose and his focus here are on par with what he brings to Moby-Dick, as is his knack for parabolic and metaphorical flourishes. Readers who enjoyed Moby-Dick's descriptions of the tools, practices, and procedures aboard a nineteenth-century whaling ship—bathed profusely in wry wit, corny jokes, social commentary, and philosophical rumination—you're in luck: that's all of White-Jacket (except here the topic is a US Navy frigate, not a whaling ship).

Obviously this makes White-Jacket a relative featherweight: you've got almost none of the man/nature dialectic, no towering symbol of the unknowable and unconquerable, and no luciferous sea captain's quests for supernatural revenge, which together form Moby-Dick's supermassive barycenter. But you can see White-Jacket as a shuddering chrysalis, ready to tear. (You can also see it as a much brisker read than Moby-Dick.)

One of White-Jacket's claims to fame is its place on record as one of the last arguments in the public debate over corporal punishment in the US Navy. (The novel was published in March of 1850; Congress passed legislation abolishing flogging in the Navy in September of the same year.) Melville had spent more than a year overseas in the Navy around 1843; obviously he was outraged to witness more than once the punitive whipping of a fellow crew-member.

Today I think it would be fun to take a look at White-Jacket's chapters regarding flogging. (The text here is cadged from Project Gutenberg.) There's a lot of interesting stuff on the periphery here. Look for Melville making uncharacteristically overt political pronouncements; notice the passing allusions to the American South and slavery with nary a peep of criticism for that institution (possibly so as to conciliate southerners who might otherwise be on his side in the matter of corporal punishment in the military); observe the points of contact with the setting and context of Billy Budd, Melville's final story; witness how in the climate of 1850, a man as sensitive as the young Melville, who saw nuance like a mantis shrimp sees color, is compelled to strike up his own note in the chorus of Manifest Destiny.

Enjoy!

CHAPTER XXXIII
A FLOGGING

If you begin the day with a laugh, you may, nevertheless, end it with a sob and a sigh.

Infinite Jest and the path of least resistance.

By ElDangerrible. Very cool.

I began reading David Foster Wallace's Infinite Jest sometime in September. Last Wednesday I finally, finally, finally finished it, and just in time for the flurry of INFINITE JEST TWENTY YEARS LATER news feed items commemorating the anniversary of its publication in February 1996.

Before you run away screaming, this is not a review of Infinite Jest. That fucking book. No, it's too titanic, too involute, too revelatory for a few paragraphs of a blurb of a blog post. It's the kind of book that other books are (and should be) written about. But I do aver that yes, it does live up to the hype, even though some of the obloquy directed at it isn't without reason; you won't see me exerting myself to refute epithets like "self-indulgent," "in desperate need of an editor," "mashup of instructional manuals for cheap electronics and obtuse postmodern academic criticism," etc., etc. But for all the frustrations and difficulties that accompany a trip through Infinite Jest, it is indisputably a raiser of the bar, a pusher of the envelope, and an argument for the enduring relevance and extant potential of the novel. (I would even make the totally unqualified claim that it's the best American novel since Moby-Dick, which was also a big hot glorious wet self-indulgent heap of a book.)

The sixty seconds that elapsed immediately after I read Infinite Jest's final sentence ("...and the tide was way out.") were eerily reminiscent of how I remember reacting to the end of Chrono Cross when I was seventeen, i.e., WHAT? WHAT DOES THAT MEAN? WHAT IN HELL JUST HAPPENED? WHAT? WHAT. WHAT?

If nobody has made the joke that Infinite Jest is like the American novel meets Dark Souls, please allow me to be the first. This is a book that does not make it easy on the reader. To call the narrative "nonlinear" is like saying a Kandinsky painting is "colorful." Most chapters are datestamped, but since the book takes place in a North America where calendar years are "subsidized" by consumer brands, you'll spend a good 200 pages trying to figure out whether "Year of the Depend Adult Undergarment" comes before or after "Year of the Whopper." There are enough characters swinging around in an n-body orbital knot to make War and Peace seem rather sparsely populated. The book ends on a flashback, and, on an initial appraisal, with none, none of the major plot threads terminally resolved. There are 1079 pages of text in all, strewn from the very beginning with clues to the story's mysteries, and to the first-time reader they are very well camouflaged.

Here's the thing: the story does not come together. Not on its own. Not unless the reader makes the effort to put it together for him or herself.

(Minor spoilers ahead. You ready?)

The truth will set you free (but are you better off?)

Continuing from where we left off:

During my tenure at the Quaker center, I occasionally measured my own life, my own happiness as an atheist against the pilgrims and seekers sojourning at the place. In retrospect, unless the people under examination had serious mental health problems (which wasn't common, but not exactly rare) I found that the majority of religious visitors, while not without their problems, were buoyant, scrupulous, friendly people of conviction and élan. I recall a man from Kenya named Shem who practically glowed in broad daylight. Kenyan Quakers tend to be of a Christocentric, veritably evangelical strain, and Shem's beliefs had a distinctly pentecostal complexion. The termini of his emotional gradient were "cheerful" and "ecstatic." As he roved the campus cum arboretum, his faith-steeped gaze found, as Shakespeare wrote, "tongues in trees, books in the running brooks / Sermons in stones, and good in everything." The world was a parable for Shem; nothing he encountered was not somehow representative of the Trinity, of grace and divine reason, or of man's progress from the womb to the world to the loving embrace of the Creator.

I could not (and cannot) believe in the things that Shem believed, and I did not (and do not) wish to. But I can only speculate as to what it must be like to exist in a world where everything from the commonplace to the celestial stands inextricably braided with the transcendent, and these relations are not only patently obvious but explicable. Listening to Shem speak I imagined he must experience his whole waking life the way the most of us most of us momentarily feel while reading a really good Rumi poem (if we're lucky).

I have to make the assumption that Shem's joie de vivre was the flowering of his faith, and I do acknowledge and emphasize that this is an assumption. Even though a wealth of studies (some of which we'll look at shortly) have suggested a correlation between religion and happiness, they are to be taken with a grain of salt. Secular-minded critics of these reports point out, often correctly, that many of the people and institutions conducting these investigations follow agendas that stand to benefit from promoting a link between happiness and religious belief. (But many of the atheists who impugn the credibility of all such studies are no less guilty of arguing in bad faith, as it were).

Tolstoy on Free Will & Necessity

Count Lev Nikolayevich Tolstoy

Germane to our last confab.

I abandoned my belief in the feasibility of the "free will" proposition some years before I picked up B.F. Skinner and found in radical behaviorism a developed and articulate theory delineating the general situation. Skinner helped to clarify and codify the conviction, but the metanoia that begat the conviction was an earlier occurrence.

When precisely did I stop believing in free will?

I guess it's one of those things that curious or reasonable people wonder about for a period, but either commit to noncommittal wishiwashiness on the issue or eventually affirm their belief in free will. Our culture actively encourages such an outcome: the rhetoric of Western society extols "freedom" as a crowning virtue; our historical/mythical/entertainment narratives are frequently constructed so as to be deconstructed as "freedom versus tyranny" allegories. Most conceptions of "freedom" become meaningless unless free will is axiomatic. Free will can't be called into question without shifting the sands on which the foundations of most of our cultural "truths" have been erected. It's much more comfortable to leave it alone.

The Back to the Future films (which I so loved as a child) conclude with Doc Brown's epiphanic declaration that the future is whatever we make it (implying a human agency uninfluenced by incident and circumstance). That did stick with me for a while. As a brooding teenager I found a compellingly delivered counterpoint in the Soul Reaver games (which, let's face it, were Back to the Future with vampires), whose in-world metaphysics were saliently deterministic. I must have gave the matter some further thought while trying to sort out the antinomies of Paradise Lost, but those efforts were more an examination of whether the poem actually succeeds in pinning a QED ribbon to Milton's thesis. (I guess I had neither the curiosity nor the savvy to look to philosophy or science for answers back then.)

After many years of neither fully believing in nor abjuring the existence of free will, what finally and irrevocably titled the scales was Tolstoy's War and Peace (1869), particularly the culminating synthesis of the epilogue. I think it's very cool that Leo would write a 1300-page story pullulating with characters and subplots through its fifteen-year arc—and then cap it off with a little epilogue wherein he closes the narrative curtain, comes onstage alone, and says a few words about what's been on his mind lately, explains what he's been trying to get at with this thing he's made. (I wonder if the new BBC miniseries will address the epilogue in some way; concomitantly I have to ask if transferring War and Peace to any other medium is really worth it.)

The second part of War and Peace's epilogue (of course there are two epilogues) is an essay on history; it is as much a repudiation of the "great man" theory of history (we can adduce the veneration of modern hero-figures like Steve Jobs to demonstrate its historiographical persistence) as a disproof of free will. (In the context of the novel and of Tolstoy's argument, these angles are complementary.)

I'd like to share with you some words from the final pages of War and Peace, taken from the 2005 translation by Anthony Briggs (who neglects the Oxford comma and is very wrong to do so). By transcribing it here I hope to better internalize its points; I always wish I could summon this stuff verbatim when I find myself speaking to an incensed believer in unconditional human agency. (We usually end up talking in circles until he or she is too angry to continue.)