TABLE OF CONTENTS
|
PART
|
DESCRIPTION
|
PAGE
|
|
|
Proposal
Cover
|
1
|
|
|
Proposal
Summary
|
2
|
|
1
|
Table of
Contents
|
3
|
|
2
|
Identification
and Significance of the Innovation
|
4
|
|
3
|
Technical
Objectives
|
7
|
|
4
|
Work Plan
4.1
Technical Approach
4.2 Task
Descriptions
4.3 Meeting
the Technical Objectives
4.4 Task
Labor Categories and Schedules
|
13
|
|
5
|
Related
Research/ Research and Development
|
15
|
|
6
|
Key
Personnel and Bibliography of Directly Related Work
|
17
|
|
7
|
Relationship
with Phase II or Future R/R&D
|
19
|
|
8
|
Company
Information and Facilities
|
20
|
|
9
|
Subcontracts
and Consultants
|
20
|
|
10
|
Potential
Applications
10.1
Potential NASA Applications
10.1
Potential Non-NASA Commercial Applications
|
21
|
|
11
|
Similar
Proposals and Awards
|
22
|
|
|
Proposal
Budget
|
|
Part 2 -
Identification and Significance of the Innovation
The Solicitation
2.1 Innovation: The declared new
exploration vision and the strategic role of a new Telerobotics Simulator and
Physical Test Fixture
The
solicitation for this topic calls for innovation in the way that people will
interact with both physical and virtual data sets using multi-sensory displays
and interfaces (including force-feedback) to support richly endowed situational
awareness and telerobotics.
The
exploration systems called for in the President’s new vision for space [1] are in
alignment with the goals of this topic. For example, from the recommendations
of the President’s (Aldridge) Commission report [2] it is clear that a return
to the Moon will require an extensive period of imaging and surface telerobotic
operations to select and prepare a site for a human crew on long duration (90
days or longer) stays. The new exploration initiative calls for in-situ
resource utilization (ISRU) of lunar materials. Prior to any extensive mining
and smelting of lunar ores or extraction of fuels, 3He or water, the very first
ISRU application will focus on creating a safe living environment for crews. A
leading health hazard on the lunar surface as in space will be exposure to
radiation levels, especially during peak sunspot cycles or in the event of a radiation
spike from an unpredictable Coronal Mass Ejection (CME). Radiation shielding
solutions for pressurized lunar habitats therefore must have high effectiveness
and be able to be enhanced on short notice. A leading solution for extendible
shielding is the layering of Lunar regolith above a pressurized habitat or the
burying of a habitat at sufficient depth to provide protection. In either case,
there will be a need for an industrial grade excavator using a bucket wheel, a front
end loader or other type of vehicle to be teleoperated on the Lunar surface.
Given
the criticality of reaching a high technology readiness level (TRL) for lunar
base site preparation and shielding early in the development of Lunar
exploration options, we propose the following systems-within-systems pathway approach
to develop this capability:
1. Development
of a telerobotics simulator supporting both a virtual and physical test fixture
which as its first application will allow the design, prototyping and testing
of lunar excavator designs.
2. In
subsequent phases, this test fixture will be employed in the following
development spirals:
a. An
all-virtual fixture with a simulated regolith environment and excavator operating
in test using analog dynamics.
b. The
virtual test fixture driving a small scale physical excavator prototype article
in a “sandbox” laboratory environment.
c. The
virtual test fixture tele-operating a full scale excavator prototype in a
terrestrial desert field site (such as the Nevada Test Site).
d. At
a high TRL, the virtual test fixture would be a fully equipped telerobotic,
latency-managing interface to actual flight hardware on the lunar surface,
perhaps including several generations of excavators.
e. During
lunar surface operations, the test fixture would be employed jointly by Mission
Control Centers (MCC) and lunar crews to operate excavator equipment.
Figures
1 and 2 below illustrate how these development spirals permit one 3D simulation
platform to support a range of TRLs from in-lab prototyping to full scale
vehicles in terrestrial desert tests and finally as a mission operations tool.
|
|
|
|
Figure 1: Spiral
Development pathway for 3D simulation-based mission support
|
Figure 2: Corresponding
levels of development of a Lunar telerobotics development cycle
|
Focus of this Phase I proposal
During
Phase I we will concentrate on the lowest TRLs of this capability with the
development of a virtual test fixture performing a high caliber 3D dynamic
reproduction of an actual lunar bucket wheel excavator prototype developed at
the Colorado School of Mines. This test fixture will support both visual
interfaces and traditional workstation-based interactivity from DigitalSpace
and a haptic force feedback interface integrated from Stanford
University’s Biocomputation
Center. From NASA’s Ames
Research Center
we will employ the Brahms agent technology and SimStation Procedures Module CAD
representation and interaction also developed at ARC in collaboration with NASA
JSC and Raytheon.
The long term vision
To
stay on course at the beginning of a development cycle a long term vision is
necessary. We have developed the following story vision piece to support the
long term goal seeking of our proposal of a Simulation-Based
Lunar Telerobotics Design, Acquisition and Training Platform for Virtual
Exploration:
It is 2019 and late night in Houston but a brilliant day at Mare Smythii on the Moon. Lunar
mission operators at NASA-JSC are easing the first piece of heavy equipment
down the ramp and the haptic operator receives a satisfying force feedback
"bump" as the big excavator encounters its first piece of lunar
surface. The real time topography system flickers to life as the stereoscopic
cameras on board the excavator produce a 360-degree model of the regolith
terrain in all directions, indicating surface and subsurface anomalies
including a boulder the size of a house buried by 3 meters of regolith.
At NASA's Ames Research Center in California, donning a lightweight immersive display
the excavator operator gets into the driver's seat. Training for years to learn
the highly sensitive haptic cab and accommodate the 3 second round trip signal
delay the operator is ready to "sense" the lunar surface herself as
she drives the excavator forward. She has weeks of careful work ahead, piling lunar
soil thickly on top of pressurized hab modules automatically landed at the
chosen site for the base. The human crew is only a year away from entering the
modules and a lot of extra shielding is needed to keep them safe from high sun
spot and coronal mass ejection radiation spikes.
As she shifts her virtual excavator's bucket down and piles up the
first load of regolith, terabytes of data drive the particle system model
presented in her heads up display. That model tells her the likely volume and
makeup of the load based on the stereo camera's real time analysis combined
with input from the radar depth profiling system. Satisfied with the virtual
excavator's performance in the model, she pulls the lever for the actual lunar
excavator and switches to vehicle video to observe it at work on that first
load. By load 99 the operator is in as tune with the excavator, the feel of the
lunar terrain and gravity and the job site as if she had been standing there at
Mare Smythii with a shovel.
Prior Projects by DigitalSpace
supporting this work
DigitalSpace has nine years of experience in
the development and deployment of 3D simulation, training, design and multi
user collaborative systems. Table 1 below details four recent projects which
provide a strong foundation on which the work of the proposal will proceed.
DigitalSpace’s prior closely related work and partnership with many of the
experts listed in this proposal lowers the risk to the project’s successful
completion. Project 1 below enabled us to develop lunar terrain and
vehicle simulations from Boeing studies [3]; project 2 allowed us to
develop experience with Stanford University’s Biocomputation center and their
Spring haptic teleoperations environment [4]; project 3 permitted us to
develop simulations of agent-driven human and robotic operations for the Brahms
team at NASA ARC [5,6,7]; and project 4 permitted us to gain experience
building virtual simulators for NASA ARC, JSC and the Neutral Bouyancy
Laboratory in the telerobotic operation of a major upcoming repair to the ISS
[8,9].
Table 1: DigitalSpace projects
that form the background for this Phase I proposal
|
|
1.
Simulation and animation of concepts from Boeing for Constellation/CEV and long duration lunar facilities.
Supporting prior
DigitalSpace work: Support of Boeing CREATE
workshop and follow-up work on Space Exploration Online (Module 1: Lunar
Base), a Chairman’s Innovation Initiative. (2004)
|
|
|
2.
Simulation template and applications for crew
health and safety on long duration missions for ISS and Constellation/CEV.
Supporting prior
DigitalSpace work: VAST simulation for JSC
and Lockheed Martin space medicine, use of CHeCS rack on ISS by crew
performing emergency medical procedures, for refinement of procedures. (2004)
|
|
|
3.
Simulation template and applications for the design of human-robotic systems and practices to support a long duration
surface facilities on the Moon or Mars.
Supporting prior
DigitalSpace work: BrahmsVE/FMARS, MDRS/Mobile Agents with ARC and JSC. (2000-2004)
|
|
|
4.
Simulation template and applications for rapidly developed, low cost just-in-time virtual training for
in-flight ISS and future Constellation/CEV crews.
Supporting prior
DigitalSpace work: SimStation SimEVA application - STS-114 CMG change-out crew
refresher simulation for Raytheon and JSC/Neutral Buoyancy Laboratory.
(2003-2004)
|
Part 3 - Technical Objectives
3.1 The Objective
This project will carry out the following objective and thereby providing
a proof of concept for a key capability in support of the NASA’s new exploration
mission:
This
project will result in the creation of a simulator of an existing prototype
remotely controlled lunar bucket wheel excavator, the testing of that virtual
robotic vehicle inside a virtual lunar regolith stimulant sand box and the final
application of the combined virtual environment with haptic force feedback
devices and multi-modal immersive
displays.
Having achieved the above objective,
DigitalSpace will be in a strong position to propose a Phase II project which would
tie the Phase I virtual environment and its haptic force feedback devices and
immersive displays to drive the physical bucket wheel excavator creating, a closed loop virtual and physical simulator
of a tele-operated robotic lunar surface materials handling vehicle.
Iterating this design would provide NASA the ability to advance the TRL of
Lunar surface operations vehicles and work practices and meet a cornerstone
objective set by the Office of Exploration Systems (OExS).
Combining three technology elements:
Element 1: prototype Bucket Wheel Excavator
The
Colorado School of Mines, Center for Commercial Applications of Combustion in
Space (CCACS) is one of the leading Lunar ISRU center’s in the world. CCACS built
a prototype small bucket wheel excavator at approximately the scale of the
rovers that are carried to Mars on the Mars Exploration Rover Mission [10].
Testing of this prototype in a physical lunar stimulant sand box permitted the
collection of data on forces exerted and power requirements for excavation and
provided data on which more efficient designs can be based. This rover was able
to excavate approximately one rover mass of material per hour. The rover and
its bucket wheel assembly is pictured in figures 3 and 4 below.
|
|
|
|
Figure 3: CCACS Bucket
Wheel Excavator
|
Figure 4: Detail of Bucket
Wheel (CCACS)
|
Element 2: 3D virtual environments
Figures 5 and 6 below depict real-time computer
graphic renderings from DigitalSpace’s 2003-04 project with the VisOpps team at
NASA ARC and JSC to experiment with realistic reconstructions of a surface
robot vehicle, in this case the Mars Exploration Rover (MER). DigitalSpace
employed its platform SimSpace to model MER with close attention to the rover’s
drive train dynamics. A simulated Martian terrain with analog surface, gravity
and day/night cycles was implemented. The vehicle is “drivable” in real time
and its contact with the synthetic surface was simulated through a
sophisticated physics engine. NASA engineers felt that the model expressed a
reasonable fit with actual MER dynamics on the Martian surface. Through this
project, DigitalSpace has established a high degree of expertise and confidence
in its ability to model surface vehicle operations [11].
|
|
|
|
Figure 5: DigitalSpace’s
virtual Mars Exploration Rover
|
Figure 6: Detail of
MER rover showing physics engine and Rocker-Bogie suspension working on
simulated rocky terrain
|
Element 3: Haptic force feedback interfaces
Low-level
teleoperation permits the user to directly control the motions and contact
forces of a remote manipulator in real time. The most common application of
this technique is in construction equipment such as excavators in which the
operator controls the velocity of the joints of a machine to perform a physical
task. Typical construction equipment does not provide sensory force feedback
directly to the hand which would be needed for high latency, highly efficient
control of tele-operations.. Considerable engineering effort must be applied to
reproduce the sensory feedback information which allows accurate and efficient
control. Both teleoperation and associated virtual environments visualization/training
need this rich and self-consistent sensory feedback.
In
both teleoperation and virtual environment applications of haptics, a loop is
closed between the human operator's motion "inputs" and forces
applied by the haptic device. In teleoperation this loop is closed via a
communication link, robot manipulator, and the environment. In virtual
environments, the loop is closed via a computer simulation [12]. Stanford University’s National Biocomputation Center has developed a series of virtual gloveboxes, and
haptic virtual/physical force feedback applications for tele-surgery [13,14,15].
Based on Biocomputation’s long history working with NASA ARC and JSC and its
expertise and willingness to employ its Spring platform for this Phase I
project, DigitalSpace has invited them to participate in this Phase I prototype
project. Figures 7 and 8 below illustrate Stanford Biocomputation systems in
action.
|
|
|
|
Figure 7: Bimanual laparoscopic
haptics device developed with Immersion and SUMMIT at
Stanford
|
Figure 8: Tele-surgery
hysteroscopy simulator at Stanford Biocomputation
|
New elements to be constructed
To achieve the integration and features described
above we will be building a new series of integrated modules based on
DigitalSpace’s SimSpace platform and Stanford University Biocomputation Center’s Spring platform.
Figure 9: New integration and module builds for this
Phase I proposal
The existing and new components that will be used
and integrated in this project are described next and depicted in figure 9
above.
·
Physical Excavator Model: The documentation, test
results, and expert guidance from CCACS at the Colorado School of Mines will
provide the background for this model, which informs the virtual telerobotic
excavator.
·
SimSpace Virtual
Telerobotic Excavator: Implementation of the virtual excavator as a dynamic 3D model will
employ DigitalSpace’s SimSpace platform. The model will adopt the following
additional component technologies:
o
Virtual Simulant Sandbox: A particle system or
deformable surface model will emulate the sandbox used by CCACS in the original
testing of the physical excavator.
o
Analog Teleoperator
Interface:
DigitalSpace will design a 2D interface that will simulate telerobotic
operations including the multisecond latency that would be experience during
Lunar surface operations.
o
SimStation Procedures
Module:
this module derived from the SimStation project with NASA ARC [8,9] will be employed
to manage the CAD components and behavior of the virtual vehicle.
o
Brahms Agents: The Brahms and BrahmsVE
platform developed with NASA ARC [5,6,7] will be used to embody an agent for
the robotic vehicle and another representing the operator. Brahms is currently
used for the Mobile Agents projects testing human/robotic operations in terrestrial
desert locations so is judged as a low risk, capable platform for this project.
·
Spring Haptic Operator
Model: The
Stanford Biocomputation Center’s Spring system will be
utilized to generate a first step haptic force feedback implementation. Spring
is also able to drive immersive (parabolic) displays which will give an added
dimension to this first Phase project.
The SimSpace and Spring implementations are both informed
by the physical vehicle and by each other. The validation of future physical
vehicle designs by the simulators and their operator interfaces is a major goal
of this project and is represented by the dashed line feedback in figure 9.
This kind of feedback often termed a “closed loop simulation” which provides a
tight coupling between physical vehicle design and performance and the virtual vehicle
realizations and interactivity [16, 17].
Architecture of the components
A more detailed treatment of both the SimSpace
architecture and the Spring platform is depicted in the following two figures.
Figure 10: the SimSpace architecture in 2004
As
figure 10 illustrates, SimSpace combines 3rd party Foundation elements (Brahms, SimStation,
shared models and any other pluggable system) within a common Simulation layer (SimSpace including the
BrahmsVE interface, Oworld Agent Information Broker (OWIB) and OWorld engine)
and drives a flexible Presentation
layer (commercial or open source 3D engines) enabling a much larger range of Applications than would be possible with
a monolithic system. Figure 11 illustrates the Spring platform architecture from the Stanford Biocomputation Center.
Figure 11: the
Spring platform from the Stanford Biocomputation Center
Additional capabilities to be implemented
The
solicitation proposal calls for a number of areas to be addressed by responses
and this project addresses a number of them, including the following:
·
We will implement
a multisecond communications delay mode in the virtual vehicle environment to
heuristically aid in the development of Predictive interfaces for real Lunar
telerobotics missions.
·
We will be
employing 2D (through heads-up display list and dialogue UI modalities) and 3D
(real time vehicle and scene rendering) multimodal displays. The Spring system
version will enable us to create a view on a parabolic display.
·
Typically low
level performance simulation of virtual vehicles generates a very large amount
of data which needs to be post-processed later. Our back end support for the
project will include a high performance SQL database to assemble the scene and
record data output in a highly compressed format. Data from each test will be
presented in real time through the interface and key features data mined on the
fly will include actual kinematics data, modeled power needs of the vehicle and
effective mass of regolith moved.
·
This data will be
captured in the database to be made available for more advanced processing via
professional packages such as VisualDOC® (Vanderplaats R&D) which allows
the generation of a kinematic simulation to engineering CAD/CAM specifications.
·
The physics
engine employed in the project is a high performance physical simulator able to
create both the simulated physics of vehicle-to-regolith contact and the
particle systems-like behavior of actual regolith being affected by the vehicle
actions in the simulator. If the need arises, the project may apply for use of
the 128 processor SGI Origin 3800 shared memory supercomputer at the Biocomputation
center.
·
As the project
will produce a 3D immersive display with 2D elements and involve a first state
implementation of force feedback interface, we will be able to report on issues
brought up by evaluating the situational awareness successes and shortcomings
of the simulation. In addition, the human sensory augmentation can be compared
with the operation and controls of the physical excavator prototype at the
Colorado School of Mines.
Construction of virtual test fixture, test, measure, and report
DigitalSpace will implement the
virtual environment to permit collaborative review by domain experts listed in
part 6.2 below. Issues such as performance evaluation, scaling (between
operator and robot), control (how stable is the control and kinematic effects),
mechanization (stiffness and torsion, control by master and slave), and
kinematics (degrees of freedom use, constraints) [18].
Part 4 - Work
Plan
The project will commence with extensive
consultation with expert advisors at CCACS, Stanford, Caterpillar, Bechtel and
several NASA centers as other contractors and university collaborators. Thereafter
the following staged implementation will be undertaken:
·
Produce
a 3D virtual model of the existing CCACS prototype lunar analog bucket wheel
excavator. Validate the virtual excavator with the CCACS.
·
Create a virtual stimulant “sandbox” to be able
to test the virtual excavator. Endow the virtual sandbox with basic physical
properties. Validate sand box with the CCACS.
·
Build a simple control interface and test
operating the virtual excavator in the sandbox. Match the control interface to
likely haptic force feedback controls.
·
Iterate
the virtual prototype to match the properties observed by CCACS in their
original testing of the physical excavator in its simu
