Virtual Worlds Research at Columbia University Contact: Prof. Steven Feiner Department of Computer Science Columbia University 500 W. 120th St. New York, NY 10027 feiner@cs.columbia.edu 212-939-7083 212-666-0140 (fax) Description: Our research on virtual worlds is centered about the development of new metaphors for visualizing and interacting effectively with rich information spaces. This work is performed within the context of Columbia's Computer Graphics and User Interfaces Group, whose research spans a wide gamut, including knowledge-based graphics, animation, rendering, visualization, visual languages, and hypermedia. Our virtual worlds research facilities include HP workstations with high-performance 3D graphics accelerators, a VPL DataGlove, several 3D tracking systems (including a four-receiver extended range Ascension Flock of Birds and three Logitech ultrasonic trackers), Stereographics CrystalEyes 3D stereo eyewear, a custom-built see-through head-mounted display (based on a 720 x 280 Reflection Technology Private Eye, mirror beam splitter, and Logitech tracker), and a Crystal River Beachtron 3D sound processor. Our n-Vision visualization testbed [3,4] allows users to explore abstract virtual worlds populated by objects representing functions of large numbers of variables. A ``3D window system'' partitions the physical space in which users interact -- a volume containing objects that are viewed in stereo and manipulated using the DataGlove. One current application is an example of ``financial visualization'' in which users can determine the effect of market variables on the value of financial instruments. n-Vision incorporates a novel approach to visualizing higher-dimensional data that uses nested heterogeneous coordinate systems. We are currently developing a knowledge-based ``world-design'' component that will select appropriate interaction and presentation techniques from n-Vision's repertoire [1]. We have also been experimenting with an approach to user-interface design that embeds the physically small flat panel display of a portable workstation within a virtually large information surround. The information surround is presented on our see-through, head-mounted display. A mirror beam splitter merges the user's view of the regular screen with that of the surrounding virtual world. We refer to this approach as a ``hybrid user interface'' because it attempts to combine the strengths of heterogeneous user interface technologies. Our current prototype is an X11 window manager that allows the user to move windows between the flat panel and the surround [7]. This project is being carried out in conjunction with a larger multifaculty collaboration in which we are building the software infrastructure for a 2Mbit/sec wireless mobile computing network [2]. KARMA (Knowledge-based Augmented Reality for Maintenance Assistance) [5] designs virtual worlds that explain how to operate, maintain, and repair equipment. By creating such material automatically using AI techniques, we address the tremendous human effort that currently underlies the design of ``hand-crafted'' hypermedia and multimedia presentations. This work builds on our research on the knowledge-based generation of 3D graphics [8,9] and on a multifaculty collaboration in the coordinated generation of text and graphics [6]. KARMA uses our see-through head-mounted display to create an ``augmented reality'' in which synthesized material overlays the user's view of the physical world. Our experimental domain is simple end-user maintenance for a laser printer. We attached several 3D trackers to key components of the printer, allowing the system to monitor their position and orientation, so that the physical and virtual worlds can be registered. A modified version of the IBIS rule-based illustration generation system [9] interactively designs overlaid graphics and simple textual callouts that fulfill a set of goals that are input to the system. Program: Opportunities to participate in our research projects are available as part of Columbia's undergraduate, M.S., and Ph.D. programs in Computer Science. References 1. Beshers, C. and Feiner, S. Automated design of virtual worlds for visualizing multivariate relations. Proc. Visualization '92, Boston, MA, October 19-23, 1992, 283-290. 2. Duchamp, D., Feiner, S., and Maguire, G. Software technology for wireless mobile computing. In IEEE Network, 5(6), November 1991, 12-18. 3. Feiner, S. and Beshers, C. Visualizing n-dimensional virtual worlds with n-Vision. Computer Graphics, 24(2), March 1990 (Proc. 1990 Symp. on Interactive 3D Graphics, Snowbird, UT, March 25-28, 1990), 37-38. 4. Feiner, S. and Beshers, C. Worlds within worlds: Metaphors for exploring n-dimensional virtual worlds. Proc. UIST '90 (ACM Symp. on User Interface Software and Technology), Snowbird, UT, October 3-5, 1990, 76-83. 5. Feiner, S., MacIntyre, B., and Seligmann, D. Annotating the real world with knowledge-based graphics on a ``see-through'' head-mounted display. Proc. Graphics Interface '92, Vancouver, Canada, May 11-15, 1992, 78-85. 6. Feiner, S. and McKeown, K. Automating the generation of coordinated multimedia explanations. In IEEE Computer, 24(10), October 1991, 33-41. 7. Feiner, S. and Shamash, A. Hybrid user interfaces: Breeding virtually bigger interfaces for physically smaller computers. Proc. UIST '91 (ACM Symp. on User Interface Software and Technology), Hilton Head, SC, November 11-13, 1991, 9-17. 8. Karp, P. and Feiner, S. Issues in the automated generation of animated presentations. Proc. Graphics Interface '90, Halifax, Canada, May 14-18, 1990, 39-48. 9. Seligmann, D. and Feiner, S. Automated generation of intent-based 3D illustrations. Computer Graphics, 25(4), July 1991 (Proc. ACM SIGGRAPH '91, Las Vegas, NV, July 28-August 2, 1991), 123-132.