Human-Guided Simulation for Lunar Rover in Virtual Environment

Lunar rover development involves a large amount of validation works in realistic operational conditions, including its mechanical subsystem and on-board software. The traditional simulation step is to construct model, input parameters, execute a simulation, and visualize the results. However, a more insight and higher productivity can be achieved if these activities are guided by human simultaneously. In this paper, a human-guided simulation method for China lunar rover simulation environment (CLRSE) that affords real time capabilities with high fidelity has been presented. It studied the input of kinematics simulation, analyzed a quick steering method of complex dynamic interactions between wheel and soft moon ground, and presented a way of path choosing by human guided. And the application which runs on PC Cluster and Silicon Graphics is also developed in this environment.


INTRODUCTION
In recent years, there has been a growing interest in exploration of the moon among the nations.China is one of the great powers of aerospace research, and lunar exploration will be the start of its future interplanetary explorations.The development of lunar rover is a complex systemic project that involves many techniques of different areas.A lunar rover is a kind of detector that loaded with scientific detection instruments and transmits the final probe data back to the earth.However, a lunar rover system requires a large number of tests.Compared with the real ground simulation test, to a large extent, computer simulations with the application of virtual reality technology is a high-efficiency, low-cost way that provides a good means of verification for the design and control algorithm optimization.
National Aeronautics and Space Administration (NASA) has been doing a great many researches in lunar rover as well as Mars rover simulation.NASA Jet Propulsion Laboratory (JPL) researched in different types of lunar rover navigation systems [1,2], and developed a simulation system called ROAMS [3].Besides, Carnegie Mellon University (CMU) [4] and France LAAS-CNRS have done a lot of works on autonomous navigation system respectively.Autonomous navigation is a key technology in lunar rover system, many scholars [5][6][7] presented lots of papers.However, The AI agents on lunar rover are still not "clever" enough to perform an exploration, and a total autonomous system is also not robust for such a complicated system on moon.Today tele-operated robot systems are becoming more and more significant in various aspects of controlling lunar rover.It is often required one human-operator to control several properties of a rover.NASA instituted a research program in Tele-robotics to develop and provide the technology for applications of tele-robotics for the United States Space program [8].Many scholars [9] studied tele-operation, most of them focused on technologies like the real-time of data transmit [10], styles of interaction [11][12][13] etc. However few people studied how they effect on robots or rovers when the objects were tele-operated and what parametric should be input.
The aim of the approach is to realize the ideal relationship for the human-oriented collaboration between a human operator and a controlled rover in a human-robot-environment system.Preview/predictive displays are very useful for tele-operation tasks; it provides an effective VR interface with immediate visual prediction/verification to the operator.
By combining the strengths of human and computers, Human Machine Collaborative Decision Making has been shown to generate higher quality solutions in less time than conventional computerized methods.In many cases, it is difficult to model continually changing problems and incorporate human objectives into the solution.Human-guided algorithms (HGAs) make use of both sophisticated algorithms and computers to provide flexibility for the human decision maker to model correctly and dynamically of the problem and steer the algorithm to solutions that match his/her objectives with the given problem.As Neal Lesh etc. [14] said that HGAs are designed to make the power of Operations Research accessible to problem domain experts and decision makers, and incorporate their expert knowledge into every solution.
The fundament of the whole system is rover dynamic model, an accurate contact dynamics model and vision-based navigation.The paper considers input of kinematics of lunar rover, such as steering angle, velocity and angle velocity by human interaction.Also the article gives a method which allows people to operate a computer towards effective dynamic equations iteration based on their knowledge of real-world constraints.Meanwhile system affords an interface that if the navigation part prejudge that there is a un-across obstacle the system will ask for human advice.The feedbacks of the system are stored in a database which can guide the future tele-operation decision support system, as Fig. 1.
Finally, the article is organized as follows.In section 2, an overall simulation modeling of lunar rover is described.Other previous work, autonomous navigation is also proposed.Human-guided simulation is described in details in section 3. Besides, a case study is included in section 4 to verify the usefulness of the developed environment as well.Section 5 concludes and discusses the future work.

II. PREVIOUS WORK
Traditional simulations are focus on autonomous control and AI agent.With vision-based navigation technology lunar rover can identify obstacles and navigate on moon.To take into account the obstacles on moon, rover's kinematic and dynamic characteristics as well as the ability whether the motor can drive through these obstacles, this system integrated not only the vision-based navigation, but also the kinematics and dynamics matter of lunar rover.The major works include simulation modeling and vision-based navigation.

Simulation Modeling of Lunar Rover
The lunar rover consists of 6 wheels using a rocker-bogie design as seen in Fig. 2(a).Two main rockers are hinged to the sides of the body.Each main rocker has a steerable wheel at one end and a smaller rocker at the other end.Two un-steerable wheels are attached to the ends of the smaller rockers.The two main rockers are connected to the body via a differential.
First, we define a set of coordinate frames for the lunar rover.These coordinate frames are illustrated in Fig. 2(b) for the left side of the rover and consists of wheels 1, 3 and 5, and rocker 1.The right side is assigned similar frames and consists of wheels 2, 4 and 6, and rocker 2.
The cycle of simulation will proceed: • Create a simulation scene, and import the geometry model to the scene; • Create a dynamic world, then create dynamic bodies attached to the geometry model; • Create a collision world and collision geometry objects as necessary; • At the beginning of the simulation loop, rigid bodies are in position and moving with velocity.Define a set of equality constraints, and a set of contact constraints.Apply external forces to the bodies as necessary.Calculate the Jacobian matrices and mass matrixes based on the current position, and then solve the forward dynamics LCP to determine the velocity for the next time step; • Pass the position vector to the collision detector.New contact constraints from these contacts will be created according to the feedback of the collision detector.The Jacobians will be updated based on this new position vector; • The simulation loop ends, destroys the dynamics and collision worlds.

Autonomous Navigation based Hybrid Vision
The binocular stereo vision technique is achieved by capture of images, camera calibration, information image preprocessing and feature detection and 3D reconstruction.In the camera calibration part, it is the way between traditional calibration and self-calibration that is adopted.It's based on plane calibration cyclostyle.Then paper reference Lu etc. [15] offline calibration model based least squares method.According the complexity of moon terrain, we presented a hybrid vision-based navigation method [16].Its framework is as Fig. 3.The hybrid method makes use of both monocular and binocular visual navigation in virtual environment.It uses a monocular camera to avoid/get across crater to give a path planning for raw planning, and a binocular stereo camera to avoid obstacles/stones in close scene to make a micro-scene path-planning.

The Framework of Human-guided Simulation
Human-computer interaction serves the critical function in the visual simulation system.Human-in-the-loop simulation is an encompassing process.Computers are used to model both the geographic environment including digital moon terrain, lunar rover models and the kinematics, dynamics characteristics of lunar rover.Decision makers are inserted in the process to make the decisions and judgments as they would in actual tele-operation practice.The human-in-the-loop evaluation enables managers to observe whether the solutions achieved are effective and proper, and can also make people to identify problems and requirements much more readily.
In the tele-operation decision support system, the scheduling system help decision makers obtain an optimal (or a near-optimal) schedule quickly.The initial schedule will be evaluated by human-computer interface with the computational results of multi-disciplinary approaches.The human-computer interaction in the decision support system is shown as Fig. 4, and it includes the functions for evaluating: 1) feasibility and difficulty of task execution; 2) appropriate precedence constraints; 3) difficulty of task execution caused by obstacles avoidance; 4) impact caused by kinematic and dynamic matters (sink, slip etc.).Human can participate in the process of simulation, which is the system will turn to human for advice when it estimates a specious obstacle.Meanwhile, tele-operator can determine different parameters of lunar controls according to different kinematics/dynamics of lunar rover.The final result can store in a database which can be used in real tele-operation.

3.2.1
Joint Constrain The paper gives multi-disciplinary computing which include kinematics and dynamics analysis.
Firstly, we should create the joint constrain of lunar rover.The position of rigid body is represented by a constraint function () , and velocity constraints are given as linear equations of the form () •  = 0 .Thus, every constraint object must implement methods to calculate () and ().From the physical model of our lunar rover, we can see that the main equality constraint type is hinge and the typical inequality constraint is contact.So the implementation of the hinge joint and contact joint will be taken as example.
In RSVE, a 3 × 1 vector represents the position of the body's center of mass: =(  ,   ,   ) , and a 3 × 3rotation matrix R, or a 4 × 1 quaternion vector q represents the body's orientation.Note that R is an orthonormal matrix, so  −1 =   .Every column can be considered to be the body's own local x, y and z axes, which is fixed to the body and changed according to the rover moving and  ̇= � .A hinge joint is a one degree of freedom joint, so the bodies can only rotate relative to each other around the axis of rotation.It can be seen as an extension of the ball joint, just with two extra constraint added to ensure that bodies can only rotate relative to each other around the axis of rotation.If we define ( 1 ′ and  2 ′ ) to be the attachment vectors that is from  1 (and  2 to the center of the ball, relative to body 1 (and 2).Let u and v be two mutually orthogonal vectors that are also orthogonal to the hinge joint's axis of rotation.So the constraint functions are: Take the time derivative of (): Appending the two rotation constraints on to the ball joint constraint (2), gives us the hinge joint constraint equation:

Kinematics of Lunar Rover by Human-guided
The process of driving of the lunar rover is an optimum result of the iterative calculation which would cost a long period of time, which might have a negative impact on real-time rendering.As in some cases that the rover confronts with the situation in slopes or valleys, it's difficult to calculate the convergent result.Under these circumstances, human potential experiences may play an important role to help shorten the duration of iterative calculation.
The human-guided mode is based on a simple communication process in which information about the current kinematic situation is sent to a receiver (the human operator), controlling some behavior of the rover, as shown in Fig. 5. What decision will be made is dependent on the previous accumulated experiences of the operator.The reason we based on a communication process is to avoid the discrete result and long duration of calculation.Indeed, it will become much more troublesome if a discrete answer is coming out of the iterative calculation.In such situations, if it is not feasible to automate the action, a human-guided advice would be more likely to prevent the failure.

3.2.4
Collision Detection Actually, contact between two bodies is a complex thing.Representing contacts by contact points is just an approximation.Contact patches might be more physically accurate, but representing these things in high speed simulation software is a challenge.Having large number of contacts will lead to the LCP solver running slowly.
The collision detection module gives information about the geometry shape of each body.At each time step it figures out which bodies touch each other and passes the resulting contact point information to the user.The user in turn creates contact joints between bodies.In system, collision detector is a black box, so alternative collision detection system can be used as long as it can supply the right contact information which is contact position, normal vector at the contact point and the geometries ID which contact with each other.

3.2.5
Real-time Response of Constrained Dynamics Once we give the input and collision, the response of constrained dynamic should be calculated.According to the stiffness of both lunar rover wheels and the terrain, we assume that wheel is rigid and the moon terrain is soft.
Wheel-terrain interaction plays a critical role in rough terrain mobility.Numerous researchers have developed models of wheel-terrain interaction that rely on physical terrain parameters, Bekker [16] and Wong [17] presented the fundamental works.A free-body diagram of driven rigid wheel of radius r and width b traveling through deformable terrain is shown in Fig. 6.A vertical load W and horizontal drawbar pull force DP are applied to the wheel center which has linear velocity V.The angle from vertical at which the wheel first makes contact with the terrain is denoted θ 1 .The angle from the vertical at which the wheel loses contact with the terrain is denoted θ 2 Fig. 6.Free-body diagram of rigid wheel on deformable terrain [16] For such a model, a formula is known to describe the relationship between the share stress () and the normal stress () of loose soil beneath the wheel: For the tangential force that is called drawbar pull,

()(
) The results are displayed in real-time.Relative parameters of dynamic are linked iteration of simulation and they can be adjusted by keyboard arrow button.

3.3
Navigation and Path Planning by Human-guided In consideration of the unpredictable environment on moon, it is far more insufficient to rely on mere autonomous navigation.Besides, with respect to the avoidance of all the big crater and stones, in some cases we tend to go through the crater forward for more experiences and reduction of time as well as energy, but the rover would avoid it directly under autonomous navigation.This paper presents a human-in-the-loop path planning for lunar rover in virtual environment, as illustrated in Fig. 7.This process can be conducted to reduce the opportunities that lunar rover run into dangerous areas.To begin with, in the destination identification step, the system identifies the destination assigned by the operator.Then in the autonomous navigation step, the rover will move according to the results calculated based on hybrid vision.This may or may not be possible or desirable, depending on the current environment forward.In the third step--obstacle identification, the rover identifies a specious obstacle, and leaves the strategic decision to human.At last, human make the final judgments of a new destination.

Prototype of simulation system
The prototype system is developed on a SGI ONYX 350 super-computer and 16 nodes PC-cluster with three BARCO projectors, Flock of Birds, Data Glove and Stereo CrystalEye, etc. we construct scene by SGI OpenGL Performer.According the datum and scene of screen, the engineers choose the best path and best iteration direction of dynamic and kinematic computing with keyboard input, as Fig. 8. Fig. 8. Human-guided simulation with keyboard input Fig. 9 shows the simulation of rover cruising in lunar surface.Lunar rover can roam in moon surface autonomously by vision based navigation otherwise it is interrupted by human.The kinematics and dynamics is developed in CMLab Vortex and integrate soft soil model with Bekker theory.Many parameters can be displayed in the system which is to validate lunar rover's comprehensive performance which will be stored in a database.

Validation of simulation
The researcher can investigate the wheel's slip and how steady the chassis is.Moreover, lunar rover on board software can be tested in this environment.For example, we can give a test on a motion controlling algorithms which can adjust lunar rover's speed according to the gradient of the terrain, then, besides investigating lunar rover's situation intuitively, the result calculated is an important reference for validating the controlling algorithms through checking whether the amplitude of the simulation result become less than before, as Fig. 10.

Navigation and Path Planning Simulation
The panoramic camera can see 15-20m.The first step, we can get path 1(black line).The path takes stones and craters into consideration and they are avoided.The path is divided into some segments by 2m, thus the lunar rover will pause every 2m to get information from human judgment, as Fig. 11(a).With avoidance camera which views range from 1m to 5m, the path 2 (red line) can be obtained, as Fig. 11(b).It is obvious that path 2 is shorter than path 1 and it occurs two turns.

V. CONCLUSIONS
The paper presents a human-guided simulation method for lunar rover in virtual environment.The paper considered three factors of human-guided simulation.One is the input of kinematics of lunar rover, such as steer angle, velocity and angle velocity by human interaction.The second is a steering method which allows people to operate a computer towards effective dynamic equations iteration based on their knowledge of real-world constraints.The last is an interface that people can choose the most suitable path among some paths calculated by vision-based path planer.The aim of the system is to use the result that guide the future tele-operation decision support system.The article also developed a prototype of system.Some validations of simulation are presented.The follow-up work is expected to generate a digital terrain which is the same with real experimental terrain so we can conduct comparison between real test data and simulation data.Besides, since there is high requirement in computing ability for 3D rendering and image processing, to develop a High-Performance-Computing (HPC) system to support simulation system is another research field.

Fig. 10 .
Fig. 10.Dynamic results of lunar rover along with chosen path