Behavior as Proposed for VRML 2.0Behavior in virtual worlds will take a great leap forward in Moving Worlds VRML 2.0. Behavior will be enabled by the communication of events between nodes and the use of behavioral scripting languages such as Java and plug-ins to provide scene regeneration, transformation and other effects.
The key characteristic of this type of behavior is that it is point to point between nodes via the ROUTE facility. In a complex behavioral situation, a greatly tangled web of event traffic will quickly arise. This N to N computing requirement will quickly overload a VRML browser as it struggles to keep up with the event set_ and _changed actions.
One other factor to consider is that programming languages (scripts) are inherently inflexible. Any given script node can only produce a certain combination of behaviors. The effect of realism in a VRML scene could be lost if, say, the digital dog always behaves in a predictable set of ways. If our goal is to make believable, compelling and realistic virtual worlds, then they will have a lot of the properties of the natural world. We posit that the only way to look and act like real biology is to emulate real biology.
How does Nature do it?In nature, sophisticated behaviour tends to be embodied within a complex control structure. Parts of a physical object do not communicate directly with other parts, this is true for animate forms, such as organisms and inanimate forms, such as a mountain. Behaviour is a series of stimuli which are channeled, collected and delivered throughout a structure.
Nature has embodied behavior in biological forms by developing a network of vessels which pump stimulus between pools. Life forms often have several distinct networks: the cardiovascular system, microtubules, xylem and phloem, or the nervous system. All of these systems have common properties.
How to Package Embodied Behavior in VRML 2.0To avoid the N to N problem, serve fewer behavior bits over limited network bandwidth and embody compelling behaviour in VRML, we are proposing an infrastructure called Nerves. Nerves is a simple packaging of a neural network within a VRML node. The Nerves package would be shipped down to client browsers within a VRML scene description, be parsed, and execute on the client system.
How would Nerves work inside the Client Browser?Nerves could serve as an extension of ROUTE. When script or transform nodes need to be tied together in a complex manner, a Nerve bundle inside a Nerve Node could be the wiring system. This tiny neural network can filter incoming event messages, accumulating and transforming them as they pass through. Thus, a Nerve Node can concentrate events, reducing traffic and more intelligently arbitrating behaviour in a scene.
There is another, equally powerful application of Nerves. Instead of serving as a pass-through filter for events, a Nerve Node could embody behavior and enable scripts and other nodes to take their cues from a living model. In this scenario, nodes would extract behavioural information through _changed events as set by the Nerve Node.
An Example: a Tree with Leaves Blowing in the WindAn illustration of this: if we were to simulate a physical tree, that tree tree could _set events into its leaves or the leaves themselves could take their cues from the overall tree structure by watching _changed events. Communication with a Nerve Node would occur through exposedFields (having both the ability to be set_ for eventIn and _changed with eventOut).
Another Example: the Cat and the BumblebeeNature has built many event prefilters, such as the glomerulus in the olfactory system (your sense of smell). Many proximate events can be going on near an organism but don't trigger action until a specific threshold or combination of stimuli has been reached. For example: a cat's optical processing neurons will track a bumblebee for quite a while before the cat is notified and begins to actually watch the bee or jump for it. Nerves could provide some of this peripheral preprocessing.
A Third Example: a Swimming FishThe swimming body of a fish might be simulated in a VRML scene by a set of instanced and cascaded Nerve Nodes simulating the spinal cord and muscle nerve actions. Each of these Nerve Nodes might be setting the exposedFields of Script Node to sense and cause Transforms in the scene, showing the fish moving forward with an undulating gait. The Nerve Nodes could generate the behaviour independent of its description in VRML, allowing script nodes to sense _changed fields and display the fish in different views. We have provided a pseudocode example of how a Nerve Node might be designed for a swimming fish.
Level of Behavior (LOB)Nerves could provide a viable basis for a kind of Level Of Behavior (LOB) analagous to Level Of Detail (LOD). LOB could be implemented by flagging certain neural pathways to be shut down when the level of stimulus (LOS) is low. Filters could be applied to selectively dampen down behavior. Nerve Nodes could be definable directly inside LODs and provide scalable behaviour in a manner familiar to VRML programmers.
Uses of NervesA key use for Nerves would be to enable multi-participant worlds with compelling behavior. Avatars might use Nerves to express realistic movement, gestures and facial expression. Applications of Nerves are described in more detail here.
A few more features of NervesA few more of the features of the Nerves design and engine are:
- Nerves comes in several versions, more full implementations vs. minimim implementations, allowing a simple version to be implemented early and upscaling of features later,
- Nerve Nets can be packaged and invoked, enabling re-use and inheritance,
- Nerve Nets were designed with hardware based acceleration in mind.
Some Final WordsThe use of Java and other scripting languages is a powerful way to extend VRML without locking users into one technology. However, the general solution is not always the most efficient. If VRML is to be competitive in the marketplace with other, more proprietary approaches, it cannot suffer from sluggish behavior. Nerves is designed to extend ROUTE and provide an early reactive network which will respond quickly to a user's actions. A Nerve Node could give an immediate set of primitive responses while the script nodes are preparing to generate more complex behavior. This is similar to the functioning of the autonomic nervous system: you will quickly pull your hand back from a flame without having to first load more compute intensive thought processes.
Thus, Nerves is proposed as a kind of middleware for behavior. We hope it will be accepted into VRML and give it a technical and marketing edge over its competition. We also hope that you will join with us in promoting this approach to the general VRML community.
President and CEO,
May 26, 1996