Phase II Brahms VE (FY02)b
DRAFT Statement of Work for
Documentation and Training in 3D Model Development
and
Creating a Prototype BrahmsVE Communications Layer
Draft version 1.0: July 25, 2002
Detailed Statement of Work
Contents
A) Executive Summary of Projects
1. Documentation for the BrahmsVE modeling process.
2. Training in BrahmsVE.
3. Creation of an initial prototype two-way connection mechanism with Brahms.
B) Proposed Schedule for all three projects
C) Proposed Budget
Appendix
A: Team comments leading up to the SOW
A) Executive Summary of Projects
The following points give a brief overview of the proposed project.
1. Documentation for the BrahmsVE modeling process
Creation of comprehensive documents to allow Brahms team, or other third party 3D modelers, to create gestures, anchors (areas), and map to the current JS API. We would fully document our JS API. This would be the beginning of a developer environment for BrahmsVE.
Time: 10 person days
Staff Assigned: Merryn Nielsen, Rolu Boemers, Ryan @ DM3DS, Bruce Damer
Deliverables: Web-based interfaces to explain the following aspects of creating gesture sets for BrahmsVE 3D worlds:
· Model creation: Tutotial in the use of Max3D, Poser and other tools to generate 3D models (astronauts or objects) and export to Viewpoint (MTX) format and its handling within the Atmosphere (AER) format and scenegraph. Subsection on the embedding of gestures within Viewpoint models.
· Guide to Modeling tools and steps: Max3d is used for the initial modeling of the human figures or other animated objects. They are then exported to Poser where a skeleton is applied and it is skinned along with textures and then the animation steps are applied. A good basic understanding of Viewpoint models will be conveyed as well as a working knowledge of the Adobe Atmosphere Builder.
· Tutorial in the generation of JavaScript: gestures are exposed and other animations from Viewpoint to the Atmosphere Scenegraph are wired in.
· Documentation on Rolu’s Oworld JavaScript library: used within Atmosphere’s JS engine to sequence and execute gesture (multi threaded operation)
· A guide to creating web-based JS and controls: to talk through the Atmosphere “spigot” and address the Oworld library.
2. Training in BrahmsVE
Training of one or more Brahms team members in the total DS/Oworld process such that you could add to the gestural set from your side. This would require you to become proficient in 3D Studio MAX (Max3d) and Poser. On the other hand, we could focus on proficiency in JavaScripting pre-existing modeled objects.
Time: 5 person days
Staff Assigned: Merryn Nielsen, Rolu Boemers, Ryan @ DM3DS, Bruce Damer
Deliverables: training for Alessandro, Boris or any other Brahms team members during a period of time at the end of September and assignment and evaluation of daily exercises in creating BrahmsVE 3D content. The tutorial exercises and live interaction would be given online inside a virtual space with results evaluated online. Exercises could concentrate on modeling selected prototype elements of the ISS including the PSA for example. Training support could continue on into the fall.
3. Creation of an initial prototype two-way connection mechanism with Brahms
|
|
|
|
Fig 1: Sequence of the Bill Clancey Agent on a walk through and action in the FMARS hab |
Fig 2: Generated JavaScript status from the sequence |
We propose to develop a JavaScript mechanism to initiate SOAP messages from the BrahmsVE client to a Java-based server or servlet which will generate an XML/HTML based report output that will be available for viewing or parsing by Brahms. This will allow us to begin the process of reporting BrahmsVE activity over the network prior to the selection of the ultimate method to connect to Brahms in real time (using CORBA, KAoS or a direct SOAP implementation). Figures 1 and 2 above illustrate the current output from BrahmsVE as a scenario is running. Records (loosely, facts) to be reported would include specific actions, start and finish times, locations reached, messages coded from action frames, and (see caveat below) collision events.
For more detail, please see:
Appendix
A.2: Bruce Damer on BrahmsVE communications architecture choices and tradeoffs
and the other team inputs in Appendix
A
Time: 40 person days
Staff Assigned: Bruce Campbell, Rolu Boemers, Bruce Damer, Bryon Hibbetts, Stuart Gold
Deliverables: Functional SOAP-based reporting mechanism operational for all three completed scenarios (planning meeting, water tank filling, and EVA)
Caveats: Adobe has scheduled implementation of a physics API to allow us to report collisions through the communications channel. If this API is delivered in August we will be able to report these events. In the first phases, all collisions will be reported. In future we must determine which collisions matter to the simulation, as “grazing collisions” as an astronaut agent moves along a floor are probably less important than “frontal, intersecting collisions” where an agent makes significant contact with a surface.
B) Proposed Schedule for all three projects
Project Start: August 1
August 31st, Delivery of first round Documentation (1)
August 20: first tie-in of XML generation to the server from whole or part of planning meeting scenario.
September 15th: Commencement of Training/Tutorial exercises for (2)
September 20th: Completion of BrahmsVE communications reporting module for all three scenarios (planning meeting, water tank, EVA), fine tuning of results, integration of Adobe Physics methods if available
Formal Project Conclusion: September 30
Continuation of fine-tuning to October 31
C) Proposed Budget
Budget to be proposed later.
end of SOW.
Appendix A:
Team
comments leading up to the SOW
A.1 Ron Van Hoof on Connecting Brahms to the VE
Hi Maarten, Bruce, Bill,
When integrating Brahms with OWorld to create a Brahms virtual environment I would not create one com agent that handles all the communication between the two environments. This would be a big bottleneck and would not provide us with the desired results.
Each agent in the Brahms VM has a counterpart in the VE, these two would be mapped one on one. For correct behavior each Brahms agent will control and be controlled by it VE counterpart. The two agents have to be tightly integrated using some layer that uses either SOAP, Corba, or KAoS.
A Brahms agent initiates a move and waits for notification of its VE counterpart that it arrived in the desired destination location with notifications during the move about things observed while going to the destination location.
The nice thing about KAoS is that we’ld have a standardized agent interface in place that allows for messaging between the agents regardless of the protocol used. KAoS currently supports Jini but will soon have support for Corba and in the future SOAP. KAoS would be a great candidate for the integration between the two environments. It would simplify the work on the Brahms end.
The current problem with KAoS is that it has no support for SOAP and that there can be issues with going through firewalls. The advantage of SOAP is that it uses a standard web server with http port 80 that is open on all firewalls so it could be used over the internet. I’m not sure how valuable that currently is for Brahms. In most scenarios both the VM and the VE will run on the same machine even for demo purposes and would not require firewall access. I could be mistaken as I don’t know what the future plans are for the Brahms VE.
Note that with the integration of the two there will still be some issues we’ll need to work out such as timing. We run in discrete time, provide a discrete start time with possibly a desired discrete end time. Any events that happen between those times need to be time stamped with a discrete event time by the VE. Not sure how that is supposed to work yet. Just sending me an event is not sufficient in simulation mode. In real-time mode there won’t be any issues, but the VE is mainly used for simulations.
We’ll also need to work out how things get started. Can the Brahms VM initiate the loading and start of a VE set, loading the appropriate agents, initializing them, etc?
Also, I remember that DigitalSpace raised issues with running a number of agents in one scenario together with each doing its own thing in parallel. I wonder whether those issues have been resolved, otherwise that would be a first priority. We should be able to run a dozen agents together in a scene.
So to summarize, I’ld be in favor of using KAoS since we have the infrastructure in place for KAoS. Without it I’ld have to develop wrappers for each Brahms agents to talk to the agents/bots in the VE. A layer we should be able to switch on/off. With KAoS I wouldn’t have to switch things on/off we just won’t get any messages from the VE.
Ron
A.2 Bruce Damer on BrahmsVE communications architecture choices and tradeoffs
From BruceD:
Thanks for the detailed feedback on connection options Ron. This helps me to understand the situation at the server. I am copying Bruce Campbell here on the discussion as he is much more familiar with the Java back end (KAoS) than I. He is also here at SIGGRAPH with me and when I locate him I will sit down and sketch out options. I am also copying Gavin Miller of Adobe who will be working with us to design the collision API.
I agree with Ron's assessment about the utility of SOAP regarding firewalls, we have standardized our community platform on SOAP messages as it allows us complete universal freedom to run on any network topology (firewalls, port settings). Also, the entire BrahmsVE front end is written in JavaScript which means that it is the easiest path for us to embed our JS SOAP implementation to start to send messages, on a timer or on demand, back to any server. SOAP is the logical and least-time-to-implement solution for us at the client end. That said, we obviously havent worked out a server side strategy that makes an easy path for Brahms (more on this below).
Rolu has built a multi threaded system in JS that executes multiple visual agents in Atmosphere, and even has a message Queue structure so that we can accommodate any process delays in the virtual world execution. I would expect that we would build a simple router that would send a SOAP message with an identifier telling Brahms the ID of the "visual agent" (agent embodiment in the VE) the message is generated from. Thus at the server side, soap messages with their identifiers would be streaming in or be generated to go out to the VE, and these messages identify targeted agents on either end. Rolu is copied here and can comment on the ability of the BrahmsVE/Oworld javascript to instance multiple agents to correspond with the agents instanced in Brahms, to allow Brahms direct control over individual "visual agents".
Now this does not address the issues of routing messages to Brahms' KAoS-based agent system. I will speak with Bruce Campbell and others who know server back end options better so I am perhaps speaking out of ignorance here, but perhaps we could build a simple java server or servlet that could intercept BrahmsVE SOAP and route it to KAoS via an existing KAoS API?
I am assuming it is very important that BrahmsVE be tuned from day one to run on distributed client systems that communicate with BrahmsVE over the net. Maarten and Bill can comment here. Given the distributed nature of our teams and the ultimate vision I have heard discussed for BrahmsVE, it seems we have been making the assumption of distributed net operation all along.
As to timing, these are complex issues that I know have solutions and we should make sure the architecture supports start times/places and durations for the parallel execution of each visual agent. I would agree with Ron that Brahms should "set the scene" and then operate the visual agents like puppets on a string. There can be little intelligence and autonomy in the 3D world as all the brain cells are in Brahms. This goes back to earlier discussions. Local smarts would come in for things very much related to the local understanding of the 3D geometry. For example, the adobe team will be putting in the API for the collisions for us soon, and local decisions in the VE can come from these events. For example: "if you collide, stop (this agent, or entire scene) and report", is a relevant implementation of decision making at the level of the VE. This allows Brahms to get the fact, and then switch execution to a different set of action frames.
RECOMMENDATION
I would recommend that a viable and desirable goal for Oct 1st, would be to create a starting point which allows us to then work out the bigger issues. I therefore propose we build a simple mechanism to report all "batch" VE activity, perhaps generating a file on disk on the server running Brahms (or any other server accessible vial HTTP). This file could be used as a "bread board" to allow us to design the future full two way (non-batch) Brahms-VE architecture. Thus we would build a SOAP layer that would report back a stream of facts regarding each visual agent and a servlet would take this SOAP stream and create a report. This report could be accessible via any web browser as we would store the file as HTML (could be XML). The report could even be parsed and displayed inside the Brahms agentviewer.
In the next fiscal year we would be in a great position to create the live, two-way dialogue with the existing 3 scenarios.
I really hope we can engage in this next phase and I am looking forward to dialogue on this such that we could quickly create a self contained SOW and make our proposal.
Bruce
A.3 Maarten Sierhuis on priorities, questions
Bill, Bruce,
Great!
The question I have, before deciding on prioritization is:
- How long would 1 take and what would be documented?
- Would 1 be enough for us to understand how to create the Java scripts for gestures?
- What would 2 entail? Would one afternoon of intense demo not be enough to give us the initial knowledge we need to understand what needs to be done to create new gestures? What my idea was when I suggested this to Bruce is that Alessandro would like to understand how gestures are created, so we can get an idea of what an ISS modeling effort would entail.
I like 3 a lot. We have been talking to Rich Keller's group about a SOAP interface to Brahms for ScienceDesk. This would use such a SOAP Brahms communication agent. Note that we already have done a Corba Brahms agent.
Another idea, which I would like Ron to comment on, is to use KAoS for the Brahms VE. It takes too much to explain KAoS to Bruce in this message, but it suffices to say that KAoS is IHMC's agent communication framework. We use KAoS to create a distributed Brahms environment. Every Brahms agent is also a KAoS agent. If Digitalspace would encapsulate the Brahms VE (do you still call this OWorld?) as a KAoS agent (maybe see every Atmosphere bot/avatar as a KAoS agent), then any Brahms agent would be able to easily communicate to its bot (in the Mobile Agent project we call these proxy agents). Anyway, I think this deserves some thought.
Bruce, if you could work out some more detail on all three, providing a plan that could show Mike Shafto what could be done (and for how much before October 1, i.e. End Of Year '02), then I can ask Mike if they still have some money available to be spent. Obviously, I can't guarantee this to be the case, and in case there is no money available we will at least have some more detailed plans for next year.
Btw, Boris showed us the latest and greatest gesture animations yesterday in our Brahms modelers meeting. They are great!!! I told Boris I expect to see a complete batch run of his model in Atmosphere, executed by Brahms VE using the Brahms Oworld file that is generated by the Brahms VM. Bruce, when do you think you guys can provide this?
Doei
... MXS
A.4 Bill Clancey on priorities
Maarten,
I agree with Bruce; my initial prioritization would be 3, 1, 2. We want to be looking towards a real-time connection in the next FY, especially in providing information to the simulation for collision avoidance and whether something is in line of sight.
By the way, Bruce, does your conception of modeling physics include audio? We need a way to model whether a sound generated at point A can be heard at point B (e.g., inside a stateroom, upper vs. lower deck).
Bill
Tuesday 23 July
Bruce,
Yes, distributed net operation is assumed. Besides collaborative
engineering, it is essential for distributed learning (students at
different locations interacting with simulated agents).
Bill
End of Appendices.