90-05/NASA.2
From: good@baviki.enet.dec.com
Subject: McGreevy on James Gibson (long)
Date: 10 May 90 12:52:50 GMT
Organization: Digital Equipment Corporation



Howard Rheingold suggested that I post the following excerpt from
a trip report on the ACM SIGGRAPH Symposium on User Interface
Software in Banff, Alberta that I wrote up on November 4, 1988. 
This excerpt describes Michael McGreevy's talk at that conference
on October 19, 1988.  That talk was my main motivator for starting
research in this area - and I think it had a similar effect on at
least a couple of others who were present.

The focus of this excerpt is to show how James Gibson's theories
of ecological optics applies to virtual worlds work.  It's
basically a paraphrase of McGreevy's interpretation of Gibson,
but I'm not aware of anything that Michael has published directly
on this topic, so it seems this re-interpretation may be useful.

Michael Good

Good@Baviki.Enet.Dec.Com

-----------------------------------------------------------------

Copyright ) 1988 Digital Equipment Corporation

1.3  3-D focus - Mike McGreevy, NASA-Ames Research Center

But now let me describe the undisputed star of the show: Mike McGreevy
from the NASA-Ames Research Center.  He gave an invited address on
"Personal Simulators and Divergence from Realism."  I wish this talk
had been videotaped - it was one of the most inspirational talks I've
seen.  My notes capture only a small part of the total effect.

Calvin, from the Calvin and Hobbes comic, is the mascot of McGreevy's
group.  As readers of the strip know, Calvin has a wonderful
imagination.  McGreevy showed examples from the strip, where Calvin
imagines what happens if gravity is arbitrary (he floats away on an
adventure) or size is arbitrary (he grows to King Kong size and adopts
King Kong behavior).  Personal simulators are built into every child. 
Imagination is always an experimental process.  Personal simulators
are a tool for imagination in childhood, science, and art.

McGreevy's talk then discussed:
   - varieties of realism,
   - natural vision: beyond the eyes and brain,
   - a test of utilitarian realism,
   - remapping the physical world, and
   - illustrations of work at NASA-Ames.

1.3.1  Varieties of Realism

Most graphic systems today view the user as a pinhole-camera; or, if
they are stereoscopic systems, as 2 pinhole-cameras.  This is not an
very enlightened view for providing realistic graphics.  In most
systems, the image is not related to the user.  Images are constructed
as viewed from a single point; if the user is not at that point, the
image lacks realism.  A more realistic interface would also stimulate
peripheral vision, not just focal vision - it would be wide-angle, not
just 400 of angle.  There are spatial and dynamic links to the user
ignored in current systems.

1.3.2  Natural Vision

McGreevy's discussion of natural vision was based on Gibson's
"Ecological Approach to Visual Perception."  This theory was new to me
(as are most things in visual perception), but seems well worth
exploring as it appears to come from a similar perspective to our own
work, and perhaps to Maturana's work in vision as well.  In this
approach, natural vision is viewed as part of our body, not as a
separate phenomenon.

A comprehensive visual interface must enable the user to directly and
naturally observe and manipulate objects and explore environments.  To
perceive the world is to co-perceive one's self.  We perceive meaning
and utility of the environment and objects within it by observing our
own capacity for visual, manipulative, and locomotor interaction with
the environments and objects.  Environments provide a context; objects
provide operations like grasping, manipulation, and much more.

Current graphical interfaces fall far short of these goals.  Current
interfaces are sterile: unlike physical environments, they have no
indigenous creatures, no objects with a history that reflects
experience, and provide poor representations of other people within
the environment.  For manipulation, a mouse for examples severely
attenuates what people can gesture to a system.

1.3.3  A Test of Utilitarian Realism

This discussion introduced Gibson's concept of affordances. 
Affordances are reflected in the following test of utilitarian realism
for artifical realities:

   Are the objects and environments as real as I am, and do they
   afford me the opportunities for use, interaction, manipulation,
   and exploration that they appear to offer.

For example, many graphics system use wine glasses to illustrate their
capabilities.  Well, does the virtual wine glass hold virtual wine? 
Can I drink the virtual wine?  Can I throw it in the fireplace and
have it shatter?  In another example, consider an architectural 3-D
system. Do walls block my progress, or do I just fly through them?

1.3.4  Remapping the Physical World

The realism of affordances serves scientific visualization. Remapping
the inaccessible components of the physical world into virtual objects
and enviroments supports exploration, experimentation, and
imagination.

Earlier examples from history show other types of visualization.  For
example, medieval and renaissance art was often a form of religious
visualization.  McGreevy showed an example of a church ceiling, which
when viewed from the proper perspective on the floor, makes it appear
that the church is opening up into the heavens (for example, columns
in the church are lined up with columns in the ceiling painting from
this perspective).  Perspective drawings like this are distorted,
sometimes very severely, when not viewed from the same point from
which they were drawn.  In this case the distortion causes the
religious visualization to break down.  But this type of distortion
happens very frequently in current graphic systems which treat the
user like a pinhole camera.

1.3.5  Work at NASA-Ames

McGreevy then gave an overview of three generations of head-mounted
displays developed at NASA-Ames.  These displays use technologies such
as wide-angle optics, a pair of LCD displays with LED arrays, and head
trackers.  Though higher-resolution LCD's are coming, a pair with
100x100 resolution worked amazingly well in early 1985, since you
could move around and see things from different perspectives.  Now,
sound has been added with headphones and a microphone, and more work
has been done on the affordances of the interface.  It's important to
perceive the floor at your feet.  3-D escalators should not have their
steps hit you in the chest.

The first version of the head-mounted display used a red, white, and
blue motorcycle helmet painted with a NASA logo.  This version is
still often used in NASA photos for its photogenic quality.  However,
later versions are less claustrophobic.  One reason is that you can
get a sea-sickness effect with this display, I imagine especially in
people who are prone to motion sickness.  Not having a helmet helps
alleviate the problem somewhat, and works better in cases where the
user does get sick.

They have used visual environments to explore physical environments,
in the context of planetary exploration.  McGreevy showed two
videotapes of work done with JPL.  The first was a visual environment
of Los Angeles.  Using techniques developed in that project, they then
developed a visual environment for Miranda, one of the moons of Uranus
photographed by Voyager 2, and we saw "Miranda: The Movie".

The head-mounted display work is described in the NASA Tech Briefs of
July/August 1988 (Vol. 12, No. 7).  NASA is transferring this
technology to anyone interested in it...

The question-and-answer session had some interesting moments.  Bill
Buxton asked about doing work in real artificiality as well as
artifical reality.  McGreevy agreed, saying that's what Calvin does in
the comic strip.  One person asked about artists working in this area.
McGreevy pointed to the work of Myron Kreuger of Videoplace fame.
Someone asked about using holograms, referring to Steve Benton's work
at the MIT Media Lab.  McGreevy's reply was that with head-tracker
technology working now, there was to need to wait for technology like
holograms.  Another person asked about incorporating tactile feedback. 
McGreevy replied that they've done a little bit of work there and see
a movement to the use of exo-skeletons.

In conclusion, McGreevy reiterated his call to go beyond a pinhole
model of the eyes, and called for work on personal simulators to
provide another step along the ascent of humanity.

