Engelbart’s proposal for “Augmenting Human Intellect” envisioned innovation in personal workstations, organizational management, and social organization in order to tackle society’s most intractable problems. Recent thinking about animal cognition and in the philosophy of the mind leads to intriguing speculation about the experience of augmentation. A large body of literature, seldom cited in this connection, explores the tensions between the Augmented and the Basic remnant.

1

From the opening of his original 1962 proposal for “Augmenting Human Intellect: A Conceptual Framework,” Doug Engelbart envisioned hypertext workstations as a central component of his plan to expand human capacity to solve complex problems [28]. This goal of augmentation, encompassing improvement to social systems as well as development of tools, remained a prominent ambition throughout Engelbart’s career and it continues to inspire his followers [7,8]. Aspects of the original vision, including the personal workstation, the mouse, and networked computers, are innovations that remain fresh long after the collapse of Engelbart’s laboratory. In Engelbart’s writing, the focus of augmentation is always on the goal: the solution of urgent but intractable problems by a team of men1collaborating through networked workstations.

Augmentation aims to facilitate group performance to surpass what a talented individual could accomplish. The then-recent precedents of war cabinets and the Manhattan Project [49] are clearly implied. One explicit aim of augmentation was the establishment of a virtuous cycle in which better tools and practices could be turned on augmentation itself to give rise to still greater improvement in human capability. The dynamics of this process and the prominent place of social as well as technical considerations were treated in detail, but the experience of the augmented participant was seldom discussed.

This stands in contrast to the work of Engelbart’s contemporary, Ted Nelson. For Nelson, the focus is invariably on personal experience and on the ways that computers could make people better [58]. The reason “you can and must understand computers now” — the subtitle of Nelson’sComputer Lib— is that understanding computers will lead to personal growth, understanding and enlightenment. No longer need children (or anyone) adhere slavishly to a contrived curricular regimen: the networked library (Xanadu, conceiving the universal library as a pleasure dome) lets the reader learn whatever she wants to know. Nelson is always writing about what it will be like to use “literary machines,” while Engelbart is chiefly concerned with iterative refinement of tools.

This note proposes to redress this imbalance and to ask, “What would it feel like to be among the augmented?” The best answer would be to build Engelbart’s toolkit and its accompanying social systems and then to ask people about the experience, but this has (thus far) eluded us. We might begin, instead, by reviewing current thought about the experience of cognition. Can we learn anything useful, for example, about what it is like to be a bat or an octopus? We will then touch briefly on current thinking about intellectual prosthetics and “the extended mind”; can a poet or a painter be said to “think” with the aid of a notebook? If one’s sensorium orumweltis an essential part of embodied cognition, as is often argued, might prosthetic improvements in vision or hearing change our thought? A good deal of the reaction to Engelbart’s vision appeared in the form of speculative fiction, starting with a NASA report written by Vernor Vinge [57], and we turn next to glance at that literature. Finally, we will touch upon a topic that Engelbart avoided in his writing: what might it feel like to lack augmentation in a world where augmentation is possible?

2

Do animals think? If so, what do they think of us? Animal minds (if minds they be) are the only other naturally occurring minds available to us at present, and careful examination of animal experience might provide useful insight into the experience of Engelbart’s augmented people. What can we know about the experience of minds that are not like ours?

We know that we experience things and that we think about them:cogito ergo sum. While it is possible to maintain the solipsistic position that only you experience things and that everyone else is merely pretending, no one believes this [76]. To know the experience of animals, however, is a challenge, because some animals sometimes act as if they have minds, but animals never act or speak as people do [13]. The study of animal cognition has made considerable progress in this century, driven in part by better understanding of genetics and of neurological structures, and also by more precise thinking about thought.

We might start with Nagel’s ground-breaking essay, “What is it like to be a bat?”, and its central observation of the difficulty of thinking about other minds.

“It will not help to try to imagine that one has webbing on one’s arms, which enables one to fly around at dusk and dawn catching insects in one’s mouth; that one has very poor vision, and perceives the surrounding world by a system of reflected high-frequency sound signals; and that one spends the day hanging upside down by one’s feet in an attic. In so far as I can imagine this (which is not very far), it tells me only what it would be like for me to behave as a bat behaves. But that is not the question. I want to know what it is like for a bat to be a bat. Yet if I try to imagine this, I am restricted to the resources of my own mind, and those resources are inadequate to the task.” [56, p. 439]

The experience of augmentation, as Engelbart envisioned it, is therefore not merely imagining that one would be more clever, or have a better office. We should not imagine animals as being human minds (or childlike minds) embedded in different bodies. Nor is mindfulness a simple linear scale: different animals are mindful in different ways [12]. For example, some animals (and most people), having learned something by observing it with their left eye alone, can immediately apply that learning when using their right eye. Others cannot do this at all, and still others (octopuses for one) require a day or so for what the left eye understands to be available to observations made with the right eye. Some animals, seeing themselves in a mirror, recognize that they are looking at a reflection of their body; others, apparently, cannot work this out.

Early 20th-century positivism (of which Computer Science itself is an offshoot [10]) held that we can, in the end, know only what we can perceive (such as the taste of coffee) and what we can deduce (such as first-order logic) [69]. There is no Authority or metaphysical rule that instructs us that (say) monkeys, cats, dogs, and pigs are conscious and bad to eat, but that cows, goats, and trout are acceptable food. We have to work it out for ourselves.

The sensorium orumweltis often seen as crucial to sentience, because by sensing the external world we distinguish it from ourselves. Of course, different animals have very different sensoria: bats have sonar, birds see ultra-violet light, and the arms of octopuses taste, smell, and see [35, ch.4]. Nor is the sensorium determinative, since people who are deaf or blind are still people. Indeed, the purpose of sentience is far from clear; what evolutionary advantage do we gain from our ability to do number theory or discuss psychiatry [34]?

Recent studies of cephalopod (octopuses and cuttlefish) cognition provide an interesting example. Humans are closely related to primates and we have a lot in common with other mammals. From birds we are more remote, but our most recent common ancestor was some sort of amphibian and so comparatively recent and complex. Octopuses, on the other hand, are very remote from us, and our most recent common ancestor was a tiny pre-Cambrian worm. The experience of being an octopus must be quite unlike our own: a large portion of the octopus’s neurons are found in its arms rather than its brains, and it may be the case that the octopus brain sends requests to an arm but then must to keep an eye out to see what the arm decides to do [51]. Yet octopuses act in ways that certainly suggest forms of experience; they recognize different people, squirt water at people they dislike, and can short-circuit the lighting in the rooms where their tanks are kept if the lights annoy them [35, ch. 4]. Some octopuses seem to regard human observers as beings like themselves, escorting them on tours of their home.

Both we and they may perceive the presence of a mind in a peculiar body. Our efforts to test their problem solving are sometimes confounded by octopus personality. We may test octopus learning, but an octopus may never attain a goal of (say) 80% correct answers because individuals become bored, or decide to double-check that the “wrong” choice is in fact permanently wrong[51]. Mice will repeatedly run a maze for another bit of cheese, but it seems that octopuses sometimes decide that they’ve had sufficient snacks and that they’d rather see what new delights might be hidden elsewhere.

We should not, then, expect Engelbart’s augmented people to be just like everyone else, save for doing better on tests.

3

One important line of inquiry has been the extended mind hypothesis [20]. Clark and Chalmers argue that

if, as we confront some task, a part of the world functions as a process which, were it done in the head, we would have no hesitation in recognizing as part of the cognitive process, then that part of the world is (so we claim) part of the cognitive process.

If we count a flock of sheep mentally, that is certainly a cognitive task. If we count on our fingers, we ourselves are still doing the thinking. Clark and Chalmers hold that, if we always carry a small notebook and tally the sheep on its pages, the notebook becomes part of our thinking — an extension of the mind.

Developed in the 13th century, paper notebooks for use with ink pens facilitated trade over space and time by providing a record of contractual obligations that was resisted forgery [2]. The laboratory notebook became both a tool and a symbol of empirical science [41], a contemporaneous record of what the investigator did and saw. Though the Memex (“Memory Extender”) is what we remember from Bush’s “As We May Think” [16], much of that essay concerns the use of inexpensive and unobtrusive photography to help scientists record data more thoroughly than even the laboratory notebook allowed.

Clark and Chalmers propose to extend the sphere of cognition to include devices that are “reliably accessible.” In antiquity, this included slaves who, like Cicero’s secretary Tiro, were in constant attendance on their owners, kept track of appointments, handled tenants, and maintained records [54]. Roman law and custom assumed “the prosthetic nature of the slave, his or her function as an extension of masterly agency. To strike a slave is to strike his master; to tell a free man to dig a ditch is in fact to tell him to have his slaves dig a ditch.” [37]

If notebooks can be regarded as an extended mind, then smartphones must also receive consideration. Indeed, with some 7 billion smart phones in use, a majority of the human population might be regarded as augmented. These devices are not only useful for jotting notes or checking facts, but also extend our senses in interesting ways. For example, one can walk through a park in an unfamiliar city in the confident knowledge that a moment’s glance will reveal your current location and the direction of your hotel. In the past, a few people possessed unusual skill for navigating cities; now, just about everyone does.

Finally, we may mention literal prostheses. My hearing is very poor, but with the assistance of hearing aids I am frequently unaware of the deficit. Cochlear implants allow people with worse hearing loss, or with no hearing at all, to hear. Experimental work on prosthetic limbs includes interfaces that afford proprioception as well as motor control, allowing people to control devices by thought [24].

4

Though the research literature on the experience of augmentation is thin, the topic has been of considerable interest tofantastika2. If taking fiction seriously appears to be a leap, consider that a substantial portion of Engelbart’s 1962 proposal was itself couched as an imaginary presentation, with imagined objections and discussion [28].

A number of writers have wrestled with the intractable3problem of the experience of the augmented. Even in antiquity, writers recognized that a superhuman intelligence might have little to say to, and scant reason to care about, mere humans, and that their concerns might well strike us as arbitrary, inscrutable, or trivial [52,62]. It may be difficult for an augmented intelligence to recognize merely human intelligence [43,61]. Conversely, because we are accustomed to being human, it might be difficult for us to anticipate or even to comprehend what the augmented want [50]. Frankenstein’s monster, for example, apprehends that, being constructed rather than created, he is necessarily imperfect, but since Romantic Love insists that the beloved embody perfection, the monster concludes that he can never be loved [67].

Augmentation means leaving behind a former state, one in which humanness can seem Edenic. A number of profiles of the augmented, notably “Flowers for Algernon”[46], anticipate that augmentation will be transient and that we can (or should) return to innocence (against which see [32] and perhaps [18]). A countervailing opinion holds that, since medicine makes cyborgs of us all, we might as well embrace our posthumanity [38,80,82].

An interesting variant of augmentation considers massive artificial intelligences that, for some reason or other, take a particular interest in the protagonist and extend his or her abilities. This interest may be benign [39] but often involves domination and control [18,48], inviting comparison to1984. Conversely, in “The Lifecycle of Software Objects”, the augmented are normal human supervisors of virtually-embodied, childlike software constructs who are threatened as much by their superiors’ flagging interest as by malign intent [19].

An important strand in recent fantastika begins with Vernor Vinge’s critique of the Engelbart project [57]. Where Engelbart foresaw a virtuous cycle of improvement, Vinge predicted that augmentation would pursue ever-faster iterations culminating in asingularitywhere everything changes and in which humanity will be superseded [79]. This specter of transcendence and/or extinction was taken up by Stross [74,75] and Gibson[33], and has come to be understood as implicit in any consideration of augmentation [81].

5

Gödel’s Incompleteness Theorem [15] is portentous for the augmented. First, it is provably impossible to anticipate every consequence of augmentation; there are more true statements than proofs. Sufficiently augmented people may learn things they would rather not know [23] and their concerns may not be our own, just as our own concerns — psychoanalysis, say, or the Cubs — might be hard for an augmented dog to comprehend [14,57]. People everywhere enjoy music, but music might be difficult to explain this to an alien, even if that alien could also perceive sound [65]. Augmentation may lead to social difficulty, straining established relationships while making it difficult to find interesting people with whom to talk [6]. Augmentation might also bring boredom and loss as old cultural practices lose meaning after long-sought answers become obvious [1,27]. The mental health challenges of exceptional mathematicians (including Kurt Gödel) have been much noted [15,47,49].

Engelbart did anticipate that augmentation in an organization would give rise to shocks, that

the radical change in the scale of Tool System capability - in speed, function, capacity, presentation quality, transmission, etc. of emergent digital technology - greatly transcends any other perturbation in system parameters that our organizations have ever needed to adapt to in so short a time as a few decades. [30]

Such disruption is particularly threatening to large institutions, not only because they have succeeded in the environment of thestatus quo, but because the size of the organization itself creates a powerful opposing weight. The augmented leadership may be correct in their proposals, but those proposals will surely upset a lot of apple carts [39,70]. This might lead the augmented to spend undesirable amounts of time and attention on political defense4, much as society decided to spend the productivity gains incurred by the adoption of desktop publishing on better typography in management reports.

Finally, we must remember that, for Engelbart, augmentation is a fundamentally social process carried out by a work group, and that part of that which is to be augmented lies in the social realm of working relationships. Improvement to communication within the group might make it hard to communicate outside the group, or even to recognize that such communication is possible or worthwhile [50]. Engelbart seems to have made little provision for disagreement among the augmented, perhaps assuming that the facts would always prevail. This proved disastrous for Engelbart’s laboratory. Engelbart wanted to build on the existing code base in order to maximize bootstrapping and minimize delay, but most of his staff were convinced that a deep rewrite to take advantage of new machines and new architectures was an urgent necessity [8,78]. That the augmented might disagree among themselves ought to have been anticipated [26].

6

For convenience, I will refer to people outside the circle of bootstrapped augmentation as “basic humans” orBasics. Of course, some Basics may benefit from some degree of access to the technologies and/or social developments adopted by the augmented. They might use networked personal workstation, perhaps less effective than those used by the Augmented yet still useful.

The resentment of self-identified Basics was a prominent feature of the American, French, and Russian Revolutions [63]. Analogies to our current plight will doubtless occur to the reader. “Do Androids Dream Of Electric Sheep” describes a last-ditch effort to suppressreplicantswho are stronger, more intelligent, and more virtuous than wildtype humans [25]. The vulnerability of even augmented people to the expectation that their technologies will always save them was a common theme of early SF [31].

Young people are often horrid to persons they believe to suffer from moderate deficits [60,73]. Children are not always accommodating to peers of different abilities [36], maturity [72], or social role [45]. Small differences in age can provoke violent antagonism [17,44]. The economic shock of augmentation might well leave those without augmentation in the metaphoric or literal dust[30,81]. Moreover, the fact of augmentation itself would create social and political divisions between the Augmented and the Basic. They could no longer play chess together, or speculate on commodities, or argue about Manchester United: the Augmented simply know too much.

The hostility of jealous Basics might lead the Augmented to seek safety by hiding [5]. It might prove difficult to identify the Augmented, for the same reason that the curiosity and contrariness of octopuses makes it hard to measure their abilities. Of course, the accusation that a group is augmented, or that it considers itself to be augmented, was a powerful force in early Fascism [4,22,83]. Far from appreciating the Augmented as working tirelessly to solve society’s most complex problems, Basics might perceive them as a threatening cabal.

7

Perhaps we are already augmented, using technology to increase our intellectual abilities. If so, we know the answer to the question this paper poses: what it feels like to be augmented is simply what it feels like to be ourselves. Admittedly, the rapid cycle of technological and social improvement that Engelbart predicted has not eventuated, but perhaps that is still to come, or perhaps Engelbart’s theory of bootstrapping was flawed [8] and augmentation had different effects than he expected.

Engelbart’s proposal noted that our culture might have evolved very differently if the sort of routine symbol manipulation he performed with pencil and paper had been substantially more difficult. If pencils necessarily weighed 2kg rather than 6g, working out a bit of trigonometry would take a lot more paper and muscular effort; to confirm this, Engelbart’s staff tried writing with pencils attached to bricks [30, Fig. 2]. If octopuses or whales wanted notebooks to record observations or memorialize transactions, they would have to accommodate the viscosity and wetness of water; differences of scale and environment could have outsize impact on the development of such facilities.

We know a surprising amount about the development of writing in the city of Uruk around 3200 BCE5, and about the difficulty of symbol manipulation performed by inexperienced hands in the intractable medium of damp clay [64,66]. Early students of the art of writing complained bitterly of the difficulty of learning a vast number of arcane symbols denoting necessary things like commodities, professions, omens, and amounts; a thousand years later, students would still be copying the same lists and complaining of the difficulty of getting the clay to do what they wished [3,77]. It wasn’t easy. The development of suitable representations of abstract concepts took time and effort. The adoption of decimal arithmetic for accounting, replacing a Babylonian system based on units of 12 and 20, was only accomplished in the U.K. in 1971. School children today can perform arithmetic manipulations that would have defied the computational experts of Mesopotamia because they have pencils, paper, and Arabic numerals.

So, perhaps we are all already augmented. Still, a note of caution is appropriate. We have more facility with reading than did the ancients, and we have more to read [55], but they were not fools: McCuthcheon [53, p. 21], for example, has a fascinating discussion about the hypertextuality of scrolls. Scrollingpasta section could prepare the reader for the section they wished to consult. John of Damascus defended holy icons because “it is impossible for us to think without using physical images.” [40, p. 105] Byzantine scholars, too, felt that the extended mind had augmented them, and they were no more certain than we that it was entirely good.

I would like to thank the Widener and Frances Loeb Libraries of Harvard University. I have profited from many discussions on these and related topics, particularly with Sam Beshers, Blake Hannaford, Megan Barnet and Myles Thornton, Paul D. Kahn. and Jack Park.

[^fn1]: ¹In a 1985 retrospective [29], Engelbart listed a “cast of characters” who contributed to NLS/Augment. Of the 42 persons on this list, I believe three were women. This ratio was (regrettably) typical for scientific and engineering projects in this era[68], though far from universal [42,71].

[^fn2]: ²Fantastika [21] is a stream of fiction, beginning in the 18th century, responding to growing awareness of the planet’s finiteness and fragility. Augmentation has been of greatest interest to science fiction and to horror, for which see [11]. For the widespread misunderstanding of genre fiction, see [9].ACM citation style is disadvantageous to arguments that depend on dialogue with the sources. It is difficult to know what “[67]” is trying to express, even if you are perfectly familiar withFrankensteinand might even puzzle out [Shelley 1818]. Requirements being as they are, I urge the reader to print out the reference and keep them at hand.

[^fn3]: ³The Science Fiction Encyclopedia summarizes the problem as one of finding a “form of language appropriate to describing an experience which by its very nature cannot be fully comprehended by either the reader – or indeed the writer.[59]

[^fn4]: ⁴The political background ofStar Wars: A New Hopeincludes a popular revolt against a government led by ambitious, augmented men.

[^fn5]: ⁵Writing developed independently at least two other times, in China and in South America. Interestingly, allalphabeticscripts appear to have a single ancestor.