Building Time Machines:
thinking about the future of
interpersonal communication

Davis Foulger
Visiting Associate Professor
Oswego State University

May 7, 2002

An earlier version of this presentation

Media are Invented and Evolved in five spheres of invention

An Invention Quintet:

Mediators, when combined in a particular way, have characteristics

Characteristics enable uses

but use inevitably creates problems

that are resolved in practices

which effect, in turn, both

uses (cycle of genre), and

mediators (cycle of media)

While there are linear dependencies

Activity proceeds in parallel ...

... as users play in all the spheres

Hence invention is not a solitary act

but a social one

Everyone who uses a medium plays a role in its continuing evolution

The Idea of a Time Machine

Goes back over 100 years

A Christmas Carol (Charles Dickens; 1843)

A Connecticut Yankee in King Arthur's Court (Mark Twain; 1889)

The Time Machine (H. G. Wells; 1895)

Einstein's thought experiments (relativity effects)

Time machines and time travel are a recurring feature in science fiction

A major recurring them in Star Trek episodes

The most recent incantation: The temporal cold war on Enterprise

Perhaps done best in successors to Ender's Game (Orson Scott Card)

Relativity effects a given for near speed of light travelers

A slowing of time

One can travel forward in time, but never back

Instantaneous information transfer through space

Bit transmission is space and time independent

Innisian effects on power structures and priesthoods

Instantaneous information transfer used for social control

The Science Fiction notion of a time machine seems far fetched to many

But we have been using media for time travel for over a million years

Indeed, the original time travel novels are themselves

instances of media time machines

It is convenient to think of media as Time, Space, and Scale Machines

Historically, we conquered time first, space second, and scale last

But we continue to improve in all three domains

Media Space: The domain of time, space, and scale machines

A system that enables messages

Hundreds of media

Eight+ kinds:

Interpersonal

Presentational

Static Art

Dynamic Art

Correspondence

Publishing

Telephonic

Broadcast

Three+ dimensions

Over 190 defined characteristics

4 power mediators:

human memory

paper analogs

manufacture

computers

A conquest of time, space, and scale

A Brief History of Time-Manipulation in Media

Apologies to Steven Hawkings :-)

We, and our ancestor species, have been talking for a long time

At least 2 million years and probably much longer

Evidence of memory-based homo-X media goes back a million years

Direct evidence of human communication media over 30,000 years

Evidence of symbol use over 10,000 years

In every case, the relevant media manipulate time

Not all media enable manipulation of time or space

Ordinary face to face interaction manipulates neither

Ordinary telephone interaction only manipulates space

But many do manipulate time in one way or another

A toolmaker manipulates time when teaching an apprentice the trade

The toolmakers art lives on, and with it, the toolmaker

A cave painter manipulates time when creating cave art

Both the sculpter and the model defeat time in a Venus figurine

The anscestors of Abraham live on in the world's oldest continuously used book

Manipulation of time with media has accelerated since, and especially of late

The keys to time manipulation in media are:

Record

Playback

Recall

(Rewind, Fast Forward, Forward, Back, Link, Index, Bookmark, etc)

There are other ways to manipulate time in media (systems of communication

Ways of Bending Time in Media

Time Relationship	Pre-Internet Media	Internet Media
Delinearization	Correspondence, Newspapers	On-line Databases, Listservs, Newsgroups
Relinearization	Drama, Shopping Lists	Agent relinearized Listservs and Newsgroups
Desequencing		Chat, Instant Messenger
Resequencing		E-Mail, Computer Conferencings, Autothreaded Messaging, Voice into Text Concurrent Interaction
Hyperlinearity		Web Sites, Task-Oriented UI
Metalinearity		Wiki Collaborative Composition, Structured Conferencing
Synchrony	Face to Face Interaction and Telephone	Shared Whiteboards, Shared Desktops, Internet Telephone and Videophone
Asynchrony	Letters, Art, Recorded Music, Film, Books	E-Mail, Computer Conferencing
Near-Synchrony		Instant Messenger, Asynchronous Telephone
Supersynchony		Supersynchronous Interaction, Synchronous Media Use Ecologies
Serial Creation	Television, Music, Theatre, and Film	Groupware, Managed Workflows
Indexing	Reference Books, Text Books	On-Line Databases, E-Mail, Computer Conferencing
Reference Pointers	Bird Books	E-Mail, Computer Conferencing, Hypermedia Documents, Relational Databases

Interpersonal internet media can warp or obliterate time linearity

Time and thread linearity can be broken altogether

Commonplace in Wiki Collaborative Composition and on-line databases

The desired linearity can be the subject of discussion (meta-linearity)

New parallel referential linearities can be created at will (hyperlinearity)

New variations on synchrony become possible

Near-synchronous interaction: Chat, Instant Messenger

E-Mail and Computer Conferencing when participation is synchrnonous

Supersynchrony: receiver control of level of synchrony with parallel interactions

Synchronous Media Use Ecologies

The forward edge of time bending entails hypermedia and agentry

In roughly these relationships:

Variation on Time

User Interface Element

Action

Effect

Synchrony

Near Synchrony

Supersynchrony

Wait

Playfast

Skip

Asynchrony

Record

Serial Creation

Index

Fragment

Delinearize

Desequence

Reference

Hyperlink

Playback

Surf

Playlist

Restructure

Relinearize

Resequence

Computer Decision Making and Intelligent Agents

A variety of names have been given to this manipulation over the years

Artificial Intelligence, Intelligent Heuristics, Expert Systems, Knowbots

The terminology doesn't really change what you are trying to do

Build software to make reasonable decisions based on available information

Apply that decision making in places where it can make a difference

Make people more productive

Remove drudgery by automating routine (e.g. uninteresting) decision making

Assist human decision making

Change the structure or flow of information

Intelligent agent is a great name for this kind of thing

suggests interaction

between lots of limited scope decision makers

that can be created incrementally

pretty much exactly how brains are organized

suggests "acting for" or in the interests of a person or group

suggests the notion of change (change agent, cleaning agent, etc)

We can use intelligent agents to manipulate time

or at least our perception of time

Using agents to manipulate time in interpersonal communication

Agent structure interaction for purposes of:

Enforcing specialized rules

Brainstorming

Distributed Decision Rooms

Structured Meeting Environments

Synchronize Interaction

Manage Interrupts During Teleconferences

Autothread interactive content

Agent identify interaction related patterns

Create task-optimized collaborative environment

Turn voice recognition into a mainstream interface technology

take advantage of its order of magnitude entry speed differential

we can talk an order of magnitude faster than we can type

we can read at least twice as fast as people talk

do auto-transcription and indexing

Use mediator correlates of interaction to

auto-identify speakers

identify beginnings and ends of interacts

Let's take a quick look at three variations

autothreaded messaging

voice into text concurrent interaction

supersynchronous teleconferencing

Autothreaded Messaging

Instant Messenger and Chat systems allow two or more users to talk at the same time

Interaction speed can be fast or slow

Fast probably is a non-verbal indication of involvement

and you thought non-verbal was absent from Computer Media :-)

Slow could mean lots of things, including

involvement in other tasks

parallel instant messenger sessions

synchronous interpersonal media use (commonplace)

Fast often entails similtaneous typing, often producing

out of sequence replies

intermingled topics

Simple time tracking would allow either:

Resequencing of interaction (e.g. moving reply under its subject message)

Marking thread relationships (with font, background color, or other means)

These systems already track submission time

Adding relative "composition time" would

allow the reply to be tied to subject message

enable either resequencing or primitive thread marking

An interaction trajectory tracking agent could do better thread marking

Effects:

Resequencing could be more problematic than out of sequence, as it breaks the reading flow.

From this perspective, thread marking is probably the better idea

Some replies will reply to multiple threads, which may compromise thread marking

Voice into Text Concurrent Interaction

Voice recognition is generally underutilized in computer media

Translation quality remains an issue, especially for continuous speech

Audible punctuation is a difficult thing to do in any case

And we rarely speak with the smoothness which we associate with writing

It remains, however, that most people can

talk (150 WPM) an order of magnitude faster than they can type (20 WPM)

read (300-1500 WPM) at least twice as fast as people typically talk

process information at least 5 times faster (800 WPM) than people typically talk

And it is already apparent that people are capable of maintaining parallel conversations

students routinely have eight or more instant messenger sessions going in parallel

I had several sessions going while writing this page

Anyone can use voice recognition to drive instant messenger now

and it is almost certainly the case that some people do

thus allowing them to interact more quickly in multiple IM sessions

But translation quality remains an issue

accuracy

loss of non-verbal paralanguage

A better voice into text messaging system would preserve the original voice stream

for optional playback

Messaging would proceed normally from the message receivers perspective

but if they want to hear exactly what was said,

and how it was said

they can

While this has been cast strictly in terms of Instant Messaging,

it could be applied in any Internet medium for which Voice into Text was used

Its simply a matter of design,

and existing hypermedia protocols suggest several alternatives

Supersynchronous Teleconferencing

Teleconferencing is often quite painful

Floor control is problematic

Knowing who is and isn't there is difficult

Sharing information often requires advance preparation

Display cannot be controlled

The local environment of each participant provides local distractions

that sometimes have broad implications for the call

but which are certainly distracting for the affected participant

Imagine a teleconferencing system in which

a meeting display environment, in one's web browser

coordinated the call

provided presence information

provided a synchronized display of meeting information

provided various collaboration tools, including:

collaborative composition

whiteboard

voting tools

with voice tracking or recognition capabilities such that:

w/ voice recognition and transcription, you could read instead of listening

with audio play on demand

and the ability to look back in the transcript while the meeting was going

w/ speaker marking (identity and content mark), you could

skip content

playback content

catch up (double speed) when you need to

save and annotate interacts

even thread replies in advance, as one might in instant messenger

such that you could handle an interrupt by a colleague or a child

and still be able to keep up (or catch up) with the meeting

This level of control might be referred to as supersynchrony

allowing you to vary your interaction level

from fully asynchronous to fully synchronoous

with near-synchronous providing a comfortable normal mode

and properly applied, it suggests other possibilities

participation in parallel teleconferences

continuous asynchronous meetings

with periodic distributed synchronous discussion

(e.g. structured voice computer conferencing)

higher speed meetings where multiple parties can talk at once

and everyone can keep up

would require sophisticated auto-threading capabilities

A Note on Synchronous Media Use Ecologies

A recurring theme in the above is parallelism in use of interpersonal media

multiple parallel instant messenger sessions

parallel meetings

etc

Kids are already living in this world of synchronous media use, and at several levels.

cell phone ecologies

The famous cell phone scene from the beginning of Clueless is reality

And more: Kids can be seen walking in groups, talking to each other

and talking on cell phones

Seamless continuity between group and extended group on phone

instant messenger ecologies

3, 5, 8 or more concurrent session

while on the phone

with people wandering into room

e-mail messages being written

files and web-site pointers being shared

and each medium complementing the other

seamless transitions from instant messenger to phone and back

setting up calls ("Lets go live")

setting up get-togethers

ending calls and get-togethers agreeing to chat

Nobody needs us to build the wilder stuff I talked about

with synchronous use of intersecting media

they are building it anyway

The Process of Media Invention and Evolution

That's how media happen for the most part

the invention and positioning of a set of mediators is just the start

users decide what characteristics of media matter

what uses a medium will have

what effects matter

what constitutes effective behavior

and even the ultimate shape of the medium

And in some cases, they can combine media themselves to create something else

There's really no question, in my mind, that stuff like I've described will happen

its just a question of when, how, and who will profit from it

And at least for what I've described, it changes our the relationship with time

autothreaded messaging:

resequence desequenced interaction

voice into text concurrent interaction:

leverage speaking and reading speeds

speed up interaction

increase concurrency

supersynchronous interaction:

end user control of synchrony

ability to break linearity, or at least skip stuff

and fix linearity by catching up from interrupts

And theres more ...

regressive resequencing of Wiki collaboration

asynchronous telephone:

near-synchronous

desequences mobile audio interaction

structured conferencing:

automatically enforce sequence

provide user summons

task oriented UI's:

brings media into our task environments

in truth, enhances productivity be creating integrated task environments

its just a matter of time, and imagination