2002 SBIR Phase I Technical Proposal

BrahmsVE: Proof of Concept for Human/Agent Intelligent Augmentation in the topic

H2.02 Human Centered Computing


TABLE OF CONTENTS


PART              

DESCRIPTION

PAGE

1

Table of Contents

3

2

Identification and Significance of the Innovation

4

3

Technical Objectives

7

4

Work Plan

15

5

Related R/R&D

17

6

Key Personnel and Bibliography of Directly Related Work

19

7

Relationship with Phase II or Future R/R&D

20

8

Company Information and Facilities

21

9

Subcontracts and Consultants

22

10

Commercial Applications Potential

22

11

Similar Proposals and Awards

23

Part 2 - Identification and Significance of the Innovation

2.1   Identifying the Problem: Complex human and machine systems can overwhelm mission support

On August 16, 2002, the following news item appeared in the wires of the Associated Press [See bibliographic reference 1 in Part 5]:

Whitson and the space station's veteran commander, Valery Korzun, got off to a late start installing the Russian cosmic-debris shields. They evidently forgot to open an oxygen valve in their spacesuits while getting dressed, and the air lock had to be repressurized so they could open their suits and fix the problem.

By the time the spacewalkers finally opened the hatch, 250 miles (400 kilometers) above the South Atlantic, almost two hours had been wasted…

Because of the late start, Russian flight controllers cut the spacewalk short at 4 1/2 hours. The retrieval of a collection tray for measuring jet residue was put off, as was wiping the area for signs of contamination.

As NASA sends more human explorers into the solar system the vehicles being constructed to carry them and assist in their science missions become more complex. The above story illustrates an event in day in the life aboard the most complex of these vehicles to date, the ISS. In this case, the mistake was easily resolved without danger to the mission. However a similar error aboard a human mission in transit to Mars might prove fatal, especially if a failed micro-meteor shield needs to be replaced in an emergency. 

A generation ago, NASA used specially equipped, operational spacecraft doubles as simulators to allow mission control to plan activities or assist the crew in problem solving. The ground-based doubles of the Apollo XIII CSM and LEM were instrumental for mission control to test power, life support and other system survival strategies and bring the crew safely back to Earth.

Today NASA is building craft that are too large or complex to be able to have operational physical doubles on Earth. In addition, efficient and safe day to day operation of longer duration missions requires a deep understanding of human work practice, psychology and group dynamics. Without a means to clearly visualize the status and ongoing dynamics of increasingly complex system and human interactions, the cognitive capacity of CapComs any anyone in mission operations can often become overrun. Therefore, there is an urgent need for NASA to acquire tools to aid mission training, planning and operations overcome this complexity overload.

2.2   Virtual Environments: A long development now yielding solutions

Virtual environments created in 3D VR technology on computer workstations or projected onto immersive environments such as CAVEs, head-mounted-displays have been used to good effect in mission training and planning for over two decades. Projects in this tradition include: early Ames tele-operations training with the Canadarm, the Virtual Shuttle, the modeling of the Mars Pathfinder surface environment, and the virtual training environment used by the crew of the 1993 Hubble Telescope repair mission [2].

We claim that no simulation environment thus far has represented more than a fraction of true underlying mission complexity. For example, an ordinary EVA aboard the ISS not only has hundreds of individual variables including checklist items and utilized equipment, but every human mission controller, PI and engineering contractor back on Earth play a role in that EVA activity, working from geographically separated areas through a thin and often unreliable and distorting pipe of electronic communications.

2.3   The Innovation: BrahmsVE

We believe that any comprehensive simulation of a mission or task within a mission must account for “humans in the loop” as well as being able to instance agents to represent hundreds or thousands of different systems and processes. In addition, where appropriate, such a simulation must be able to provide highly realistic and accurate spatial visualization of the actual vehicles, people and spaces, driven by the simulation in real time.

For several years, teams at NYNEX, the Institute for Research on Learning and now, at Agent iSolutions working with USRA/RIACS have been developing an intelligent multi-agent environment used for modeling, simulating and analyzing work practice. The environment is called Brahms. Brahms is a data driven (forward chaining) discrete event environment usable for simulation purposes as well as for agent-based software solutions requiring the use of intelligent agents. Brahms is described in detail in bibliographic references [12, 13]. From the Agent iSolutions web site [3]:

Brahms allows us to model the work activities of each type of role, and each individual (or artifact) playing that role in an organization. The focus of a Brahms model is on the context of work, meaning, how does the work really happen. One of the essential requirements for Brahms is that we can model collaboration and coordination between people working on one task, as well as that people can work on more than one task at a time, and are interrupted and able to resume their activities where they left off.

Two years ago Digital Space Commons was fortunate to team up with the Brahms efforts at Agent iSolutions via a STTR program funding our collaboration with USRA/RIACS [4]. Through that collaboration we brought our multi user 3D virtual world framework (called OWorld) to Brahms to execute a batch mode reenactment of a scenario from the FMARS/Haughton-Mars Project. In 2002, continuing with direct support from USRA we have evolved an integrated architecture we call BrahmsVE for Brahms/Virtual Environment. See the BrahmsVE project pages at [5].

BrahmsVE enables highly realistic recreations of environments involving people interacting with systems. Figures 1 and 2 below illustrate some of the human figure, gesture representation and scenario reenactment being completed for the 2002 phase of BrahmsVE development. These are figures modeling the crew aboard the FMARS/Haughton-Mars 2001 season.


Figure 1: Human figure recreation and gesture in BrahmsVE

Figure 2: Scenario of the Bill Clancey agent astronaut enacting a sequence inside the virtual FMARS habitat

Current work involves the creation of a comprehensive event activity protocol that will drive three long scenarios of a “day in the life” of the FMARS habitat. By the end of the 2002 Brahms VE should be executing two way communications with server-side Brahms models and reporting events from the virtual environment such as collisions occurring in the scene physics.

2.4  Components of the innovation

We believe that BrahmsVE is a uniquely powerful new tool that will offer human-centered computing unique opportunities for advancement. BrahmsVE has the following unique properties:

·Brahms Java-based PersonalAgent with compiler, virtual machine, builder, IDE and AgentViewer all of which are currently utilized in a number of NASA projects including mobile agents.

·Brahms Virtual Environment runs on industry standard consumer grade computing platforms over oridinary internet connections with no special hardware required.

·BrahmsVE employs industry standard languages and protocols such as Java, JavaScript, SOAP and XML.

·BrahmsVE’s virtual environment uses industry leading technology in Adobe Atmosphere such as Havok physics, Viewpoint models and inverse kinematic animations all running in an open framework utilizing an open source community server.

We believe that the properties of BrahmsVE will enable us to meet the technical objectives in part 3 below.


Part 3 - Technical Objectives

The following statement from the topic for this SBIR call for proposals to produce the following tools:

Visualization tools combining "virtual reality" projection with actual objects in the environment, conveying information about object identity, part relationships, and assembly or operational procedures. "Cognitive prostheses" that qualitatively change the capabilities of human perception, pattern analysis, scientific domain modeling, reasoning, and collaborative activity. Such tools could incorporate any of a variety of modeling techniques such as knowledge-based systems and neural networks, and fit tool operations to ongoing human physical interaction, judgment, and collaborative activity.

This statement is a broad-brush vision challenging the entire field of visualization, human-centered computing and cognitive science. While this modest SBIR Phase I proposal cannot hope to encompass such a broad scope we nevertheless commit to produce a proof of concept that will be a good first step along the road to this total vision. Our objective is to produce a simple cognitive prosthesis in a virtual reality environment that will implement a canonical scenario in the study of human-centered computing: a semi-autonomous mobile agent interacting with a human subject. We call this project the Personal Bot Assistant, or PBA.

3.1   The Objective


Figure 3: Canonical example of a semi-autonomous agent interacting with a human astronaut

The objective of this project is to produce, in a web-based 3D virtual environment, the canonical example of human/machine augmentation, that of a semi-autonomous agent assistant interacting with a human inhabitant of a space station or surface habitat. As the above figure illustrates, that agent should be able to execute a minimum set of activities interacting both with its environment (the geometry and physical properties of a virtual space station or habitat) and with an astronaut agent. The astronaut agent in this example could be driven by a simulation engine or directly by a user at a workstation as an “avatar”. The virtual astronaut would in turn have a limited repertoire of commands, which can be directed to the PBA.

This project benefits from two years of prior work on the BrahmsVE platform as supported by a year 2000 STTR contract [4] and under continuing USRA funding in 2002. For this SBIR Phase I, unique new elements will be constructed in the BrahmsVE platform to enable a sophisticated, agent-based multi-user and visually realistic interactive environment to test concepts of human/machine augmentation.

3.2   Inspiration and Background for Project

This project is inspired by a number of efforts, notably the FMARS/Haughton-Mars Project, and the Personal Satellite Assistant (PSA).


Figure 4: FMARS habitat on Devon Island, Canada.

The FMARS/Haughton-Mars Project [8] has over several summer field seasons yielded data on work practice, group dynamics and design factors to consider for a human mission to Mars.

The research effort involving Bill Clancey et al has suggested several areas where autonomous agent assistants could be employed to reduce the complexity of life aboard FMARS and its sister habitat, the MDRS. One example is the operation to fill the habitat water tank which requires three crew members in continuous communications (one on EVA next to the well and pump, one monitoring tank levels in the hab and one serving as a go-between). An autonomous agent could be requested to monitor the tank water level and directly report to the astronaut on EVA, making this operation possible with one crew. In addition, predictive behavior by a autonomous agent, such as a rover, could save time and reduce task complexity. For example, if a rover on EVA with an astronaut to collect rock samples was intelligent enough to turn on geological processing systems (such as a sample oven) in the habitat once the sample was sensed in the rover’s receptacle, this would save time and augment sample collection activities.


Figure 5: The Personal Satellite Assistant (PSA)

The PSA, now under development at Ames for use aboard the ISS, faces a number of design challenges including the need to produce a viable test environment on Earth prior to flying. To this end a test fixture is being constructed at Ames [9]. The cost and difficulty of recreating a micro-gravity environment, noise levels, airflow, astronaut movement, and the precise geometry and physical environment simulating the interior of the ISS begs an effort to attempt this recreation utilizing a virtual environment. While the modest effort proposed here would fall far short of the sophistication needed to truly test the current PSA design, we hope it will provide a compelling enough demonstration to garner support for future work.

3.3   The Physical Model Concept


Figure 6: Physical Model of the PBA Proof of Concept

The physical model of the concept is shown in Figure 6 above. The PBA would exist in the physical analogue of a simplified space station or surface habitat and use various simulated devices to interact with a virtualized astronaut agent. For the purposes of this Phase I proof of concept, the simplified physical environment would consist of geometric space with named “anchor points” (destinations which can be navigated to, ie: “base of ladder”) as well as spatial geometry and active physics, permitting collisions with that geometry. Within that Environment would be two types of locales which would have meaning to the PBA, one or more Power Points (locations to which the PBA must automatically take itself when its internal power state reaches a certain threshold), and a Stationkeeping Point, a locale where the PBA must take itself when there is no activity assigned to it.

A set of simulated sensor devices would enable some complex interactions between the PBA and its environment, primarily the astronaut agent. A Proximity Sensor would use ray casting to determine closeness to an object and be used to follow the astronaut agent. A Conversational Interface would permit the Brahms engine or a user intervening at a client workstation to send the PBA textual commands. A Tactile Interface would be represented by clickable surfaces on the 3D model of the PBA itself, and permit a user to interact with the PBA directly using a mouse. The virtual Clock would permit Brahms to set a start time and duration for any activity. Lastly, a Camera would be an affordance allowing Brahms or the user to place a point of view either outside the PBA in “third person” perspective or in “first person” perspective to allow viewing of the scene from the PBA’s point of view.

3.4   The Technological Components

Figure 7: Technology Components to be employed in PBA Proof of Concept

The current implementation schedule for BrahmsVE will give us the technology components shown in Figure 7, by the end of 2002, making these components available for this Phase I SBIR in the first quarter of 2003. Currently the only component whose implementation is still in the process of design is the real time interface between the client-side virtual environment (Oworld/Atmosphere) and the server-side Brahms multi-agent system. We expect to implement this interface in the Fall of 2002 using Simple Object Access Protocol (SOAP) and tie this layer into BRAHMS utilizing a file-based interface, KaOS, CORBA or SOAP directly. Each of the other components is in a state of readiness for this effort.

3.5     The Logical Model and Development Specification


The following logical model serves as a specification for the implementation of the new PBA semi-autonomous agent functionality within BrahmsVE. The majority of the development will occur within the virtual environment’s OWorld JavaScript engine but there will be a need for a new type of Brahms activity event. Thereafter, a simple Brahms model will be developed to instance the astronaut agent, the PBA and control execution of the simulation. 3D modeling of all three of these components will use current model archives and newly created models.


Figure 8: Logical Model to be employed in PBA Proof of Concept

Figure 8 above illustrates the logical model we will be employing for the PBA proof of concept, as based on the existing BrahmsVE architecture. BrahmsVE employs message queues for every action. Initially these queues are filled by Brahms “Activity Events” which are exported from Brahms and then move up into the functional modules via a Message Ingestor/Emitter.

a) Brahms server-side modules and activity events

Activity events are structured messages passed from BRAHMS to the virtual environment (see specification development in [6] and [7]). These messages are exported in the format:

activityevent ::=

activity|type|starttime|endtime|who|activityname|arguments

        type ::=

primitive|communicate|createobject|createagent|createarea|

                   put|get|move|gesture

Brahms exports several types of activity events, listed in Table 1 below. To support the semi-autonomous intelligent operation of the PBA agent, new activity event exports will need to be supported by Brahms, also listed in Table 1. However, the entire PBA environment can also be developed and tested independent of Brahms using the web-based interface module to initiate the agents and run the scenarios.

Table 1: Brahms Activity Types

1. Primitive

2. Broadcast/Communicate, to be extended for support of the PBA.

3. Create Object

4. Create Agent, which would be extended for instancing semi-autonomous PBA agent.

5. Create Area

6. Put, also used to start the PBA in a locale.

7. Get, will be extended to query the PBA location.

8. Move, used to move the PBA.

9. Gesture, used to activate gestures in the PBA.

NEW TYPE: 10. Create binding with agent (allows Brahms to receive communications back from the PBA) by linking to exposed methods in the agent’s JavaScript. This new type of interactive message would be handled through the Brahms Event Watcher in Figure 8. This activity type would also permit Brahms to interrupt the simulation and resume using different actions. This is also the interface used to message the PBA to return to a Stationkeeping Point.

The Message Ingestor/Emitter parses these messages, extracts numeric and literal values and then inteprets them into messages in the internal Queue for Oworld. As messages often have a start time and duration assigned they are scheduled for execution by the Event Timer. Related to the message queuing and event timing is the BRAHMS Event Watcher, a module which will be in place in Fall 2002 to watch for events from BRAHMS that will cause the execution of a running simulation to be halted or altered. This module will allow the BRAHMS model to have decision making power over the course of the scenario in the virtual environment using the binding with the PBA agent’s exposed methods.

b) New modules to be created within the virtual environment’s OWorld JavaScript engine

Several new modules will be created internally to the Oworld JavaScript engine in the virtual environment. As much of the focus on this proof of concept is to create local intelligence, the coding efforts will focus on the VE and have little impact on Brahms per se beyond the addition of several new activity event outputs (see Table 1).

Internal Physics Processor: this module will represent and affect the physical properties of the simulated PBA’s virtual vehicle, representing drivetrain torque, current forward and rotational velocity (pitch and yaw), coefficients of friction on external surfaces, and the power level (battery life). The design of the PBA is discussed in section 3.6 below.

Situational State Machine: this new module will contain a state machine, implemented as tables in JavaScript. This module will be the core functionality responsible for enabling the PBA to act autonomously in the virtual environment. The State Machine will be a micro-scale analogue to BRAHMS itself (representing a single, not multiple agents). For the PBA proof of concept this state machine will be very simple, performing a series of continuous loop checks for the following conditions, and executing actions:

  1. Checking proximity of a target object (usually an astronaut agent) and commanding the Internal Physics Processor to move toward that object (maintaining a minimum following distance).
  2. Checking power level and sending a message to the report module and/or BRAHMS if battery charge drops below a certain level. If permitted, executing the Path Planner to return to the nearest Power Point or base station.
  3. Checking the External Physics Processor for messages signaling a collision or force vector (which could indicate a blower fan) which significantly modifies the position of the PBA from its current heading. If this occurs, the State Machine will process a message to the reporting module and BRAHMS.
  4. Testing the Sensor Watcher, Message Queue and BRAHMS Event Watcher for any direct commands to the PBA.
  5. Watching the virtual world message queue for events that affect the simulation, such as a direct click on a tactile interface on the PBA model.
  6. Watching the Web-based Interface Module for the processing a conversational interface command.
3.6     The design of the PBA agent in two environments

The PBA agent has the following design components, assuming two renditions will be produced, one designed as a wheeled vehicle moving along the floor of a lander habitat (FMARS) and one analogous to the PSA aboard the ISS. The internal algorithms need not be unique to a surface versus free-floating rendition, although initial parameters will be set to determine behavior supporting each environment. PBA components include:

  1. Logical Interfaces: exposed methods in JavaScript to which Brahms communicates via activity events or the user can communicate through the web-based interface.
  2. Internal State Machine: the generic control structure tying events, locomotion, and dialogue to create the behavior of the agent. This State Machine also embodies the clock used by the agent, testing against tasks that need to be started or completed by a certain time. The State Machine will also count down power level (batter charge) and indicate when the PBA is required to seek a power point.
  3. Physical Interfaces: these algorithms will embody locomotion, proximity sensing,
  4. 3D Model Representation: for the surface version, a wheeled vehicle will be designed and built in Viewpoint to execute in the existing FMARS model in Atmosphere. For the free floating version, a ball-like model will be built using some of the exterior look and feel of the PSA.
  5. 3D Model Tactile Interfaces: these interfaces are user-clickable surfaces on the 3D model for the PBA which will directly drive the state machine or physical interfaces. These interfaces will allow testing of the PBA independent from having to code a predictive Brahms model. Each of these interfaces will be made accessible on the web-based button interface as well.
3.7     A virtual test fixture for the PBA agent and its interaction with an astronaut agent

In direct analogy to the PSA test fixture [9], we will set up a virtual test fixture inside both the existing surface habitat for FMARS and we will construct a 3D model of the interior of the ISS for another series of tests. In the virtual FMARS, gravity will be in effect while in the virtual ISS the PBA and agent astronaut will be “floating” (although no attempt will be made to model their relative orbital dynamics). Into these two scenegraphs we will place one PSA agent along with one or more astronaut agents. The test fixture will be operable with the options in Table 2:

Table 2: Test fixture operation matrix

 

Agent astronaut driven by user

Agent astronaut driven by Brahms

User Operation of PBA

A

B

PBA driven by Brahms

C

D

The effectiveness of the BrahmsVE PBA simulation for the study of human/machine interaction will be derived from two classes of feedback:

  1. Empirical feedback derived from the studying the user experience, especially if the user is driving the agent-astronaut directly while Brahms is initiating the PBA to autonomously follow the user in the virtual environment (option C). Of course the user may choose to drive the PBA directly (B) and also drive the agent astronaut (A). This last option is possible through a combined use of navigation of the astronaut (as an avatar) by mouse in the virtual world and operation of the PBA through the web based interface. Option D is the best option for testing the Brahms model as this is a “hands off” mode. Note that during any test option the user can decide where to place their point a view: on a camera attached to the PBA, the agent astronaut, or fixed at a point in the scene.
  2. Statistical reporting in the report module, which the PBA will be messaging. Statistics to be reported by the PBA will include:
    1. Collisions occurring with the geometry of the virtual environment including the astronaut agent.
    2. Failure or success of the PBA to stay with proximity of the astronaut agent by use of a ray-casting function to determine distance.
    3. Failure or success of the PBA to maintain a powered up state through its ability to find a power point and to report its lower power state to Brahms or the end user and seek permission to engage the power point.

Interviews and possibly ethnographic techniques could be employed on the empirical data as has been applied to FMARS field seasons [10]. Tools for filtering, interpreting, and visualizing the data reported from the simulation could be subjected to analysis, as per the Bayesian  decision-making systems referenced in [11].

3.8     Measuring the virtual test fixture against the stated goals of the solicitation

It is our goal that this virtual test fixture will be capable of beginning to address some of the key goals set forth by the solicitation (see Table 3 below):

Table 3: Goals from Solicitation and how the PBA test fixture might address them

Goals set forth in solicitation

How initial PBA simulation space might address these goals

On-board systems must aid people in diagnosis and repair and enhance safety

Drive PBA to location to hold camera or obtain a readout from an instrument or sense the environment saving agent astronaut exposure to risk involved.

Interface design of mobile or built-in agent-assistants, human perceptual-motor coordination, cognitive operations, and group dynamics

Train users to navigate the astronaut agent while communicating with the PBA. Permit several users to enter the scene simultaneously as “avatar astronauts” and interact with a single PBA, testing group dynamics in a kind of “game” environment.

Permit the testing of procedures where machine systems complement human activities

Move PBA in tandem with astronaut-agent and provide a camera sightlines to a second astronaut-agent (the viewer).

Enable just in time training

Train users about the following behavior of the PBA and how to avoid collisions with it.

Understanding appropriate task separation between human and machine systems

Study where a PBA-like mobile agent could play a role and take on human tasks where appropriate. Examine error rates of the PBA in complex path navigation suggesting limits of a mobile agent.

Develop robust control systems and exploration tools that can be understood by people, easily learned, maintained, and directed

The use of the “tactile interfaces” on the 3D model of the PBA as well as web-based interfaces (akin to a control panel) will permit experimentation with learning and operating a mobile agent.

Part 4 - Work Plan

In order to achieve the technical objectives described in Part 3 of this proposal, we have divided the project into eleven major tasks.

4.1       Project Work Plan by task

The following table provides our projected allocation of hours by labor category by task and corresponds to our proposed budget.=

Table 4: Work Plan

TASK

DESCRIPTION

PI

PM

TE

SE

CD

TG

1

Expert interviews to establish parameters for FMARS and ISS 3D modeling

10

5

20

0

0

0

2

Design and architecture phase for the new logical modules

20

5

20

0

0

0

3

Implementation of new logical modules in OWorld engine

10

0

10

50

0

0

4

Creation of 3D models for interior spaces, for PBA and agent-astronaut in FMARS and ISS simulated environments

10

0

10

0

20


0

5

Implementation of external web-based interfaces and reporting module

5

0

10

10

0

0

6

Implementation of new activity events from Brahms

10

0

10

10

0

0

7

Integration of total environment with 3D content

5

0

10

10

0

0

8

Initial testing of environment (user driven)

5

0

0

5

0

10

9

Testing of environment via Brahms model

5

0

0

5

0

10

10

Testing via user and Brahms combined

5

0

0

10

0

5

11

Project report generation

15

5

10

0

0

0

Total

All tasks:

100

15

100

100

20

25

Where the roles are defined as:

PI         =          Principal Investigator

PM       =          Program Manager

TE        =          Technical Experts

SE        =          Software Engineer

CD       =          Content Developer

TG       =          Test Group

4.2       Project Reference Website

The Project Reference Website will be a center for ongoing progress and resources surrounding the project, from the interview phase to the prototype test fixture evaluation. The site will consist of the following components:

·         Project goals, timeline, team biographies and contact information

·         Documentary results of each phase of the project, from the posting of interviews to architectural and specification documents to codebases and 3D models

·         Listserver for project participants with log of message traffic

·         Executable releases of test fixture environments and related tools and libraries

·         Links to related team resources, other NASA sites and industry sites

4.3   Project Work Schedule

This section describes the work schedule for the Phase I effort (see Table 5 below).  Digital Space Commons work is to be coordinated from its corporate offices located near Santa Cruz California. Digital Space Commons design and testing teams are located at several places around the United States and internationally. This schedule assumes a five-month project duration.

Table 5: Work Schedule

Month and Phases

1

2

3

4

5

Expert Interviews

¦

¦

     

Creation of architectural plan, specifications

 

¦

   

Commencement of software and content development

 

d

d

d

d

Mid Point Project Review

   

¦

¦+

 

Completion of development, commence testing

     

dt

t

Testing and iterative development

       

td

Final testing

       

t

Final Phase I Report

       

*

 

 

 

 

Where:

¦ = Specification and or Design and Documentation

d = Software and Content Development

t = Software Testing

+ = Status Report

* = Final Report

Part 5 - Related R/R&D

Digital Space Commons has been engaged in the development of virtual world platforms for seven years and has successfully completed a number of major projects (see Part 8.1 below). The PI and members of Digital Space Commons have contributed numerous publications to the medium [14]. In addition, work with USRA/RIACS and NASA collaborators began in 1999 continuing through an STTR Phase I project [4] and following on in 2002 under support from USRA [5]. We are working closely with the Brahms team at Agent iSolutions [3, 6, 7, 12, 13] as well as supporting the vision of the NASA team involved with FMARS, Mobile Agents and other projects [8, 10].

5.1 Team Collaboration

The Brahms team at USRA/RIACS, and the Agent iSolutions group will be involved in a key supporting role in this Phase I deliverable (as identified in the Technical Objectives above).

5.2 External Cooperation

We will be cooperating with Dr. Don Brutzman and his group at the Naval Postgraduate School/MOVES Institute on the Extensible Modeling and Simulation Framework (XMSF), an XML-based cross platform framework for simulation. Benefits to this cooperation will be the possible use of an XMSF subset for the SOAP-based XML dialogue layer which will be implemented within BrahmsVE and drive the PBA. This Phase I SBIR work will in turn enable the XMSF working group to have an active intelligent agent case study in which to test XMSF concepts. XMSF is a broadly based standards effort that will bring interest and compatibility with simulation efforts at the DOD and at other government agencies and contractors which will strengthen the BrahmsVE offering and make possible broader SBIR Phase II development and Phase III commercialization.

5.3 Bibliography of Directly Related Work

[1] Associated Press News Report dated 08/16/2002, on the web at: http://www.cnn.com/2002/TECH/space/08/16/station.spacewalk.ap/index.html

[2] Loftin, R.B., and Kenney, P.J., "Training the Hubble Space Telescope Flight Team," IEEE Computer Graphics and Applications, vol. 15, no. 5, pp. 31-37, Sep, 1995.

[3] Brahms is described on the web at http://www.agentisolutions.com

[4] STTR Phase I Brahms VE: A Collaborative Virtual Environment for Mission Operations, Planning and Scheduling Contract #NAS2-01019, Bruce Damer, PI. Final Report on the web at: http://www.digitalspace.com/reports/sttr-techreport-final2.htm

[5] BrahmsVE/FMARS Project Home Page on the web at: http://www.digitalspace.com/projects/fmars

[6] TM00-0024 Brahms/OWorld XML DTD Specification, Version 1.0 For Review, 28 November 2000, Ron van Hoof et al, NASA Ames Research Center.

[7] TM00-0025 BRAHMS OWorld Event Specification Version 1.0 Draft August 14, 2002, Boris Brodsky et al, NASA Ames Research Center.

[8] FMARS/Haughton-Mars Project Home Page on the web at: http://www.marssociety.org/arctic/index.asp and http://www.arctic-mars.org

[9] Personal Satellite Assistant (PSA) Test Fixture (Greg Dorais, Yuri Gawdiak, Daniel Andrews, Brian Koss, Mike McIntyre) described on the web at: http://ficworkproducts.arc.nasa.gov/psa_test_fixture/psa_test_fixture.html

[10] Clancey, W. J. 2001. Field science ethnography: Methods for systematic observation on an Arctic expedition. Field Methods, 13(3):223-243, August.

[11] David C. Wilkins, Patricia M. Jones, Roger Bargar, Caroline C. Hayes, Oleksandr Chernychenko: Collaborative Decision Making and Intelligent Reasoning in Judge Advisor Systems. HICSS 1999

[12] Clancey, W. J., Sachs, P., Sierhuis, M., and van Hoof, R.1998. Brahms: Simulating Practice for Work Systems Design. International Journal of Human-Computer Studies, 49, 831-865.

[13] Sierhuis, M. 2001. Modeling and Simulating Work Practice; Brahms: A multiagent modeling and simulationlanguage for work system analysis and design. Ph.D. thesis, Social Science and Informatics (SWI), University of Amsterdam, SIKS Dissertation Series No. 2001-10, Amsterdam, The Netherlands, ISBN 90-6464-849-2.

[14] Find an archive of Digital Space Commons publications on the web at: http://www.digitalspace.com/papers

Part 6: Key Personnel and Bibliography of Directly Related Work

6.1   Management and technical staff members

The following brief resumes are the proposed management/technical staff members for the proposed project.  Digital Space Commons certifies that Bruce Damer, the Principal Investigator, has his primary employment at Digital Space Commons at the time of award and during the conduct of the project.

Name:   
Bruce Damer
Years of Experience: 
21
 Position:     
CEO
 Education:
Bachelor of Science in Computer Science (University of Victoria, Canada, 1984); MSEE (University of Southern California, 1986)
Assignment: 
Mr. Damer will be the Principal Investigator and also manage the SBIR Phase I effort.  He will coordinate all interaction between Digital Space Commons and its collaborators and NASA, be responsible for all staffing, technical design, reporting and documentation.   Mr. Damer will devote a minimum of 100 hours per month of his time to the NASA SBIR project.
Experience:
Mr. Damer is the world's recognized expert on avatars and shared online graphical virtual spaces having created much of the early literature, conferences and awareness of the medium. Mr. Damer is a visiting scholar at the University of Washington Human Interface Technology Lab and a member of the staff at the San Francisco State Multimedia Studies Program. See http://www.digitalspace.com/papers for a complete bibliography of Mr. Damer's work.

Name:   
Stuart Gold
Years of Experience: 
27
 Position:     
President
 Education:
Royal Institute of British Architects
Assignment: 
Stuart Gold will evaluate the technology components and architecture for the BrahmsVE/PBAl as well as coordinating the 3D modeling teams and provide any database and real-time community tools infrastructural support on the project and the SOAP based interfaces with Brahms.
Experience:
Mr. Gold is a pioneer of online systems, starting with his work on transaction processing for Prestel in the 1970s and concluding most recently with his leadership in the design and delivery of online virtual worlds including: TheU Virtual University Architecture Competition, International Health Insurance Virtual Headquarters, and Avatars98-2001 online events. Mr. Gold also is the chief architect of the Digital Space Commons communities platform, implementing SOAP and JS based community tools for use by all Digital Space Commons projects. See http://www.digitalspace.com/papers for his recent writings.

Name:   
Roel Bomers
Years of Experience: 
4
 Position:     
Lead programmer, Oworld/Atmosphere
Assignment: 
JavaScript programming, interface with Brahms and 3D contnet, testing
Name:   
Galen Brandt
Years of Experience: 
24
 Position:     
New business development, Digital Space Commons
Assignment: 
Market development for Phase II and III
Name:   
Dave Rasmussen
Experience: 
7 years experience in virtual world design, skills: 3DS Max, Java, Active Worlds, Adobe Atmosphere, PHP/MySQL database development
 Position:     
Member of the 3D Design Studio, Digital Space Commons
Assignment: 
Directing team performing 3D modeling and animation, testing
Name:   
Merryn Neilson
Experience: 
7 years experience in virtual world design, skills: 3DS Max, Java, Active Worlds, Adobe Atmosphere
 Position:     
Member of the 3D Design Studio, Digital Space Commons
Assignment: 
Web design on project, 3D worlds, avatar design, testing
Name:   
Ryan Norkus
 Position:     
Graphic artist, 3d modeler and animator, 3D Design Studio, Digital Space Commons
Assignment: 
3D Modeling of PDA, environment and astronaut agent
Name:   
Peter Newman
 Position:     
Developer in C++, JS, PHP, HTML, 3D Design Studio, Digital Space Commons
Assignment: 
Focusing on the automation of animated sequences

Part 7: Relationship with Phase II or other Future R/R&D

Digital Space Commons has made a multi-year commitment to the development of the vision we share with the NASA and USRA/RIACS team members who have made this effort possible. Briefly stated, our joint mission is to create the world’s most comprehensive, graphically realistic, collaborative work practice, mission planning and operations development environment. Successful completion of this proposed SBIR Phase I will set the stage for full production packaging of BrahmsVE. We expect to engage multiple NASA and outside customers for BrahmsVE within a Phase II SBIR to qualify the platform for full Phase III commercialization. At the start of Phase III, Digital Space Commons plans to either finance its initial operation with customer revenues or venture capital, or if no venture capital is obtained, the principals are committed to self-finance the venture during Phase III.

Part 8: Company Information and Facilities

Digital Space Commons was incorporated in the state of California on August 24, 1995.  Digital Space Commons is a company organized to innovate and commercialize in the multi-user virtual worlds and virtual communities market.  We are located near Santa Cruz California and currently lease office space in a 2 story building at 221 Ancient Oaks Way, Boulder Creek, California 95006. Additional Digital Space Commons member-owners have offices in Phoenix Arizona, Seattle Washington, Amsterdam, Holland, and Victoria, Australia. The company’s business is based on the following concepts:

•      That the Internet and especially 3D virtual worlds, voice and text environments, can be effective meeting places and enable communication, learning and team based projects. Digital Space Commons uses these spaces daily in a proof of concept that a company can base its entire operations on them;

•      as the need for teleworking, distance learning, virtual communities of interest and Visualization grows, so will grow the capabilities of ordinary consumer personal computers to deliver real-time 3D multi-user experiences. It is the convergence between these needs and the capabilities of consumer computing hardware that will create a large industry producing and hosting virtual worlds and communities in the future.

On these two premises, Digital Space Commons has provided solutions for dozens of clients to produce both demonstration and fully functional virtual spaces since 1995. See our web site at http://www.digitalspace.com for a portfolio of projects and clients. We will feature a number of them here for their relevance to the SBIR proposal:


8.1 Portfolio of recent projects (separate from BrahmsVE work)

8.2 Equipment used

All Digital Space Commons member-owners have at least one personal computer connected on the Internet (most have IBM PCs, Pentium class, while others have Macintoshes or both in their local offices as well as other equipment).  

Part 9: Subcontracts and Consultants

Digital Space Commons will not use outside consultants on this project. Consultants used for BrahmsVE work in the past will become member-owners of the company by the end of 2002.

Part 10: Commercial Applications Potential

There are several classes of customer for a BrahmsVE environment having a local, autonomous agent capability.

10.1 NASA Internal Customers

Numerous projects within NASA ranging from the PSA at Ames to the Mobile Agents project of interest to several NASA centers could benefit from the addition of “intelligent” agents within the virtual environment. Putting more complexity in the agent will offload the modeling requirements at the Brahms level and permit better data on agent performance in simulated physical spaces. Rapid, iterative design of a mobile agent in a virtual world can be made possible by changing both the Brahms model, where appropriate, and the script locally controlling the agent in the 3D world, where appropriate, as the physical and control system behavior is observed in the virtual space. Local algorithms in the virtual agent could then be downloaded into the physical robot, as is done today with Brahms.

10.2 K-12 and College, Education and Entertainment

The growing popularity of “robot games” and “robot wars” for  K-12, college, and adult audiences suggests that BrahmsVE and the PBA could be a basis for the creation of a successful multiplayer online game both as a learning tool and as a pay-per-play tournament environment. Digital Space Commons has engaged several potential partners in an exercise to gauge the market for such a game, both as a learning tool and as a pay-per-play tournament environment. Of course, agent-based virtual environments can be of great value to museums and science learning centers such as the Exploratorium in San Francisco, where the robot games are held each year.

10.3 Industrial design applications

From factory automation to security systems, complex environments where humans work in tandem with mobile agents or other autonomous machine systems all need a comprehensive model-based environment with high fidelity 3D re-creation during both design and operations phases.

10.4 Industrial design applications

The military will be using semi and fully autonomous agents working closely to support troops and command in surveillance and combat missions throughout the 21st Century. Therefore we expect a great deal of interest surrounding a product in this space. We are already in contact with the Naval Postgraduate School MOVES Institute about cooperation on a new XML based standard in simulation communications.

10.5 Consumer markets

The emerging era of wireless, wearable personal assistants is picking up momentum with ever more sophisticated cell phones and other handheld devices. In a real sense, each of these devices represents the pairing of humans with machines, as the BrahmsVE PBA project will study. In 2003 we will be partnering with a Swedish university in a book and conference on the theme of “Interrupt Culture” which will explore the relationship between portable devices and their human hosts. We expect that BrahmsVE/PBA would be of great interest to the sponsors and researchers at this event.

Digital Space Commons plans to make use of its business plan developed as part of Phase I to obtain private venture support for a number of opportunities, including the robot games application. Such a venture will be approached in tandem with the Brahms/Agent iSolutions group. In addition, Digital Space Commons’s member-owners are committed to self-financing if no venture capital is obtained.

Part 11: Similar Proposals and Awards

Digital Space Commons has received past support for this work from a 2000 Phase I STTR Contract #NAS2-01019 [see final report in reference 4 above] and independent support from USRA/RIACS in the years 2001-2002 [see ongoing progress in reference 5 above. Digital Space Commons is submitting no other proposals similar to this SBIR Phase I at this time.


(c) Copyright 2002 Digital Space Commons, All Rights Reserved
Return to Digital Space Proposals