DigitalSpace
Technical Report
BI-MONTHLY TECHNICAL PROGRESS REPORT NUMBER 001

For Contract #NAS2-03134 SBIR 2002

Proposal #: H2.02-8957: “BrahmsVE: Proof of Concept for Human/Agent Intelligent Augmentation”.

Reporting Period: January 14, 2003 – March 14, 2003
PI/Contact: Bruce Damer, DigitalSpace Corporation, (831) 338 9400

NASA Program Recipients
NASA Ames Research Center
Accounting Operations Branch, M/S 203-18
Moffett Field, CA 94035-1000
and
Natalie LeMar MS 241-1
COTR MS 269-3
SBIR Specialist MS 202A-3

SBIR Team Recipients:
M Sierhuis (RIACS), R Van Hoof (RIACS), D Rassmussen (DigitalSpace), B Damer (DigitalSpace)
Report

(1) A quantitative description of work performed during the period

The DigitalSpace team initiated work on this Phase I SBIR by constructing the first model of an International Space Station (ISS) interior, that of the US habitation module. Subsequently we created a 3D representation of a “Personal Bot Assistant” modeled on a prototype design of the NASA Ames Personal Satellite Assistant (PSA). This PSA model was then placed within the virtual ISS interior and given a path following algorithm that would have it pilot itself around the space to four predefined areas. Cameras were then set up and a user interface built to permit multiple points of view of the simulation (including a camera on the virtual PSA itself). This running model can be seen depicted in Figure 1 below.


Fig 1: Running ISS/PSA simulation with new BrahmsVE 1.1 “Skin” showing startup sequence for ISS/PSA demonstration


Figure 2 below shows how the camera viewpoint settings work as well as the simulation clock, which was an important feature during our 2001-2002 work on BrahmsVE, is now built into the interface.



Fig 2: BrahmsVE interface showing changed camera view showing PSA making a turn, note simulation clock time has included in this new interface.

The running example of this model is online at:

http://www.digitalspace.com/content/atmosphere/2003/nasa/brahms/index.html

The second phase of our work during this period was to create a prototype 2-way communications mechanism that can communicate events inside the virtual world (such as collisions or arrivals of the virtual PSA in a certain area) back to Brahms.

Extensive research revealed that the use of XML/HTTP, a facility built into the web browser, could be used to send events, via the JavaScript “spigot” in the Adobe Atmosphere client to JavaScript running on the web browser page and from there to a back end server listening for dialogue on port 80 (HTTP). Figure 2 below shows the console output from the first test of this system. Appendix A explains how this facility works.



Fig 3: BrahmsVE 1.1 Console showing parsing of incoming and transmitting of outgoing Brahms Actions

The log of this communication dialogue is included below.

Thu Mar 13 14:52:47 PST 2003

** Outgoing **

time=0&event=Idle&area=projects.issvre.Point0&agent=projects.issvre.PSA

** Incoming **

activity|move|0|10|projects.issvre.PSA|Walk||projects.issvre.Point0|projects.issvre.Point3

alert|10|projects.issvre.PSA|Idle||projects.issvre.Point3


** Outgoing **

time=10&event=Idle&area=projects.issvre.Point3&agent=projects.issvre.PSA

** Incoming **

activity|move|10|20|projects.issvre.PSA|Walk||projects.issvre.Point3|projects.issvre.Point1

alert|20|projects.issvre.PSA|Idle||projects.issvre.Point1


** Outgoing **

time=20&event=Idle&area=projects.issvre.Point1&agent=projects.issvre.PSA

** Incoming **

activity|move|20|30|projects.issvre.PSA|Walk||projects.issvre.Point1|projects.issvre.Point2

alert|30|projects.issvre.PSA|Idle||projects.issvre.Point2


(2)  An indication of any current problems, which may impede performance or impact program schedule or cost, and proposed corrective action

Current problems that may occur are possible issues with the physics and collision callbacks now that have been shipped on March 14, 2003 to Digital Space by Adobe. These callbacks need to be tested for any issues but we fully expect they will be ready for use within the time frame for completion of Phase I work.

Another issue is identifying whether the Brahms team can create a facility within Brahms to catch the event messages on port 80 (via HTTP). We are awaiting confirmation of this and hope that if the Brahms team can accomplish this we can complete the full two way interaction between Brahms and the Virtual Environment.


(3) A discussion of the work to be performed during the next reporting period

During the next period we expect complete the following:
  • The current alert event system was created here to fulfill the need for this example. However, there may already be a specification for a similar event in the Brahms format, or there may be a better way of implementing such an event, while keeping in the style of the other commands. In the current implementation, the Oworld system only requests additional data when an event occurs, in response to an alert command. A possible future expansion is that the system will periodically create a request to the server, just to check for any additional commands. Creation of Jserv Servlets for the reading of Brahms simulation action statements and distribution to running version of Oworld for representation in virtual FMARS (the Manifest Manager).
  • Defining the protocol (XML) for the communications with Brahms as well as implementing the new BrahmsVE Action commands as specified in the proposal.
  • Building a scenario, 3D models and objects and scripting for a test scenario with a virtual astronaut and PSA working in concert in the ISS virtual
(4) Estimated percentage of physical completion of the contract

We estimate we are approximately thirty three percent toward the completion of the contract.

For ongoing project tracking please refer to the web resources references


1) The BrahmsVE project homepage is at

http://www.digitalspace.com/projects/fmars


2) The full SBIR 2002 phase I proposal may be viewed on the web at:

http://www.digitalspace.com/proposals/sbir-2002-phase1/


3) The Brahms specification and engine is described at:

http://www.agentisolutions.com/home.htm


4) ISS/PSA homepage is at

http://ic.arc.nasa.gov/projects/psa/


end of report.


Appendix A: Explanation of the HTTP driven ISS PSA


This system uses an implementation of the Oworld library, implementing the PSA as an "agent", the same way that the NASA agents were used in the other uses of this library. The code also sets up some basic information about the ISS environment, such as four "area’s”, again in the same way used in previous implementations.

The main expansion of this library is the addition of an "alert" event. This is an addition to the Brahms defined constants for command strings. This event causes the relevant agent to execute an Alert action. This action uses the spigot to call a function embedded in the containing web page, passing this function the data specified in the "alert" command line. This function uses the XMLHTTP ActiveX interface to make a request of a HTTP supporting server, providing the alert data in the request.

Depending on what the server actually is, it will handle this information differently, however the result will be the server returning data (using the normal HTTP response to a request). This data will be one or more command strings.

On receiving this data, the web-page functions parse the data, breaking it into separate strings and then using the spigot to pass these strings into the Atmosphere plugin, to the Oworld libraries. If the strings are correctly formatted Brahms command strings, they will be processed, if not, discarded.


Specifics


In the current example, the PSA does not create "alert" events on its own. Instead, alert commands are passed to it, through the spigot-server-spigot interface, as the final command in a sequence. Below is an example sequence of commands:


activity|move|0|10|projects.issvre.PSA|Walk||projects.issvre.Point0|projects.issvre.Point1
alert|10|projects.issvre.PSA|Idle||projects.issvre.Point1


In this example, there are two command strings (if you ignore the line wrapping). The first is for an activity, move, from time 0 to time 10, performed by the PSA (projects.issvre.PSA), using the Walk action. Additional data is provided, being the beginning and end location for the move command. As stated earlier, Point0, Point1, Point2 and Point3 are pre-defined locations, currently defined within the Atmosphere Javascript.


The second command string is the alert command, to be performed at (or after) time 10, by the PSA (projects.issvre.PSA), for the reason of "Idle". Additional data provided is the agents (PSAs) current (expected) location.


The net result of this is to cause the PSA to move from Point0 to Point1, then request additional instructions. If the recieved instructions include an alert command, more instructions will be requested at that point, thus continuing the process. In the existing example, the server document is a PHP script, however it could be anything, as long as it can be accessed through HTTP. This includes a web server running ASP, or CGI, or PHP, or a server-side application specifically designed to provide HTTP responses. A likely example of this would the Brahms server, which is apparently Java coded.


Design Decisions


In this design, it was decided that "alert" commands should be provided by the server, rather then the PSA creating them "on-the-fly". While it is possible for the PSA to create alert events when it goes idle (has no remaining commands), it was simply easier at this point to have them as part of the provided commands.


Similarly, it was suggested that the PSA should be able to create alert events when it detects a collision, or potential collision. However, from looking at the current design of Brahms commands, as well as their current implementation, it appears that the Brahms server will be responsible for preventing collisions, by not issuing commands that would cause one.


Also, in terms of moving from one location to another, the libraries allow the creating of "paths", which are used if available. An example of this is in the FMars simulations, the agents walked around the table, using the defined path.


Another problem with this is the Brahms system seems to define locations by name, and collisions would generally happen at unnamed locations. So the system would be unable to tell the Brahms server where it was when the collision happened, and thus, the Brahms engine would be unable to tell the system where to move from, in moving to a new location. (For example, in the example move command shown earlier, it contained both source and destination information.)


Another feature implemented in this design is the display of all outgoing and incoming information, between the client (web-browser) and the server. This is provided by the transfer functions embedded in the web-page (not by the Oworld library), and is used just for clarity.


Also it probably be noted that the "events" initiated by the OWowrld library (inside Atmosphere). A suggested course was having script on the web-page polling a status of a variable stored within Atmosphere, however in implementation that system would be unwieldy. The explanation behind this is, information cannot be read through the spigot in one command. Rather, the web-page code would have to request the state of the variable (using the spigot, web-page into Atmosphere) then Atmosphere would have to respond, passing the value out (using the spigot, Atmosphere to web-page). This would result in two distinct events, when it is more simple to simply have the second type of event (Atmosphere to web-page) when there is significant data to be provided. In terms of alteration to the Oworld system, they are both of approximately the same impact.


It is also worth noting that the current system allows for event queuing. For example, if there are multiple agents operating in the environment, and the second one requests information in response to an alert event while the first one is still waiting for a response to its request for information, the second request will be queued, and executed when the first is complete.


Future Directions


The current alert event system was created here to fulfill the need for this example. However, there may already be a specification for a similar event in the Brahms format, or there may be a better way of implementing such an event, while keeping in the style of the other commands. In the current implementation, the Oworld system only requests additional data when an event occurs, in response to an alert command. A possible future expansion is that the system will periodically create a request to the server, just to check for any additional commands.


Creation of Jserv Servlets for the reading of Brahms simulation action statements and distribution to running version of Oworld for representation in virtual FMARS (the Manifest Manager).


This report can be found online at:

http://www.digitalspace.com/content/atmosphere/2003/nasa/brahms/help.html