Crowd Rendering Optimization for Virtual Heritage System

Previously, crowd simulation plays small part or probably ignored in virtual heritage. Architectures or artifacts bit more focused to be reconstructed into virtual environment. By inserting crowd into simulation of virtual heritage, it will give more impact and achieve higher realism level to the reconstructed site. Before inserting the crowd into virtual environment, a research needs to be done to manage the complex environment of virtual heritage and the crowd itself. This paper presents a framework with a vision to reduce the computation cost for rendering crowd simulation in virtual heritage environment while maintaining realisms of the scene. We first review the existing acceleration techniques applied on crowd rendering. Then we introduce a framework that will integrate acceleration techniques for crowd simulation in virtual heritage.


I. INTRODUCTION
The wide use of computer graphics in education, entertainment, games, simulation, and virtual heritage applications has led it to become an important area of research.In order to deliver to the user an immersive experience, it is important to create an interactive, complex, and realistic virtual world in any simulation applications.As the size and complexity of the environments in the virtual world increased, it becomes more necessary to populate them with peoples.Nowadays, virtual heritage is one of the latest growing applications in computer graphics industries.One of these paper objectives is to collaborate crowd into virtual heritage system with minimal computational cost for easy delivering on common personal computer.
Virtual heritage commonly describes reconstruct of architecture of the historical site.For more accurate simulation social aspect cannot be missed.Crowd in virtual heritage is one of social aspect can be display in virtual heritage.The importance of visualizing heritage site is to allow us a better understanding of the selected site without being there.Crowd in virtual heritage can strengthen the feeling of immersion by recreate virtual life like human and livestock.
When rendering crowd in complex virtual heritage, there are two things need to be deal with: large-scale static virtual environment and moving crowd and traffic.These two things content thousands of polygons need to be managed to avoid low frame rate and not interactively for user to walkthrough in real-time.The limited number of polygons that can be processed in graphics processing pipeline show that the need of 3D acceleration techniques to manage crowd rendering especially in virtual heritage.Most used techniques in crowd rendering is level-of-detail (LOD) and image-based impostor.

II. RELATED WORK
Virtual Heritage system is normally content of complex 3D object as example building, forest and crowd.Due to complexity and densely of virtual environment (VE) in crowd rendering causes the system usually rendered by high performance computer hardware and almost impossible in lower specification computer system [1].Crowd rendering can be divided on three parts: geometry, motion/animation and behavior [2].For interactive rendering crowd simulation the geometry part needed to be managed.The geometry part of crowd rendering can be managed by 3D graphics acceleration techniques: culling, level-of-detail (LOD) and image-based impostor [3].Example of applied virtual environment is Ancient Malacca Virtual Walkthrough project which used this technique to enhance their virtual environment rendering speed [1].These acceleration techniques also applied to CHARISMA project for Medieval Norwich virtual heritage environment [4].In Virtual Hagia Sophia, optimization techniques needed to lower amount polygons for interactive simulation [5].

Crowd Rendering
The complicated part when dealing with thousands of characters is the quantity of information that needs to be processed.Simple approaches, where virtual humans are processed one after another without specific order will produce high computational cost for both the central processing unit (CPU) and graphics processing unit (GPU).This is the reason why data that flows through the same path need to be grouped for an efficient use of the available computing power.Therefore, for the best simulation result, characters capable of facial and hand animation are simulated in the area near to the camera to improve believability, while for farther area, less expensive representations are used.Concerning efficiency of storage and data management, database must be used to store all the virtual human-related data.In this chapter we will discuss one of the major components in crowd simulation development which is rendering.Figure 1 shows certain problems that arise when rendering crowd [6].For instance, collision avoidance problems for a group of peoples in the same place required different strategies in comparison with the methods used to avoid collision between individuals.Moreover, motion planning used in a group that walks together requires more information compared to individual motion planning.The trajectories computed for agents in the same group that walk together with similar speeds must be different even when they share the same environment and goals.In addition, other levels of behaviors can exist when treating crowd in this hierarchical structure.The group behaviors can be used to specify the way a group moves, behaves, and acts in order to fit different group structures (flocking, following, repulsion, attraction, etc).Individual abilities are also required in order to improve the autonomy and intelligence of crowd.

Crowd Rendering Optimization for Virtual Heritage System
However, to render thousands of individuals, these complex behaviors cannot be provided individually due to the hardware constraints and computational time rates.Another problem relates on how to improve the intelligence and provide autonomy to scalable crowd, in real-time systems.Furthermore, the simulation of large crowd in real time requires many instances of similar characters, that why an algorithms to allow for each individual in the crowd to be unique is needed.

Culling
Culling can be defined as removing portion of scene that do not contributes to final output [7][8].They are collections of techniques that function to discard invisible and not required part of 3D objects to decrease the high computation cost of real-time computer graphic application.For complex virtual heritage, culling will accelerate rendering process by quickly culling unwanted 3D objects before rendering process [1] [7].There two type culling suitable to be applied in crowd rendering: view frustum culling and occlusion culling.

View Frustum Culling
There two approaches of view frustum culling (VFC): transform the bounding volume into perspective view [9] and check the bounding volume against view frustum volume by it six planes [10].Both of them used when object place outside of the view frustum [1].In crowd simulation, crowd lie outside of view frustum will remove before rendering process.The problem of view frustum is when rendering large data set which content many polygons inside frustum to be processed.This view frustum usually helped by a spatial data structure.There are two types of spatial data structure, spatial partitioning and bounding volume hierarchies (BVHs) [11].Hierarchical grids, portal, octrees, KD-trees are the most popular methods, and KD trees are usually considered the option of choice for massive models.Spatial data structure used attempted to pre-compute visibility relationships within a complex virtual scene [12][13] [14].Figure 2 shows the number of object drawn in the screen with and without Range Detection Technique for view frustum culling to be culled [1].

Occlusion Culling
Occlusion culling removes 3D objects hidden by groups of other objects [1].In rendering crowd simulation, this technique is not suitable to be applied.Occlusion culling needs distinct organization of the whole geometry database.The reason why occlusion culling is not good for individual, deforming and animating objects is because the scene need to be divided in smaller units or cells.Dynamic culling introduced as new occlusion culling based on binary tree merging with KD-tree [15].This method used discretization and properties of urban scene for fast cull away invisible static object and also avatar.Figure 3 shows dynamic culling introduced to improve occlusion culling for crowd rendering [15].

Level of Detail
Level of detail (LOD) involves reducing the complexity of a 3D object representation as it moves away from the 3D viewer in VE for faster rendering process [16].This technique improves the efficiency of rendering by reducing the data structures on graphics pipeline stages, usually vertex transformations.LOD used to improve system performance and quality of graphic system [17].LOD helps culling technique in reducing the polygon to be rendered.The criterion always be used in LOD is the size and quality of object relative to distant of the viewer.Figure 4 shows LOD used for crowd rendering, as the object became far, number of polygons reduced [3].LOD is common techniques applied to crowd simulation to speed up rendering process.Aubel et al. review that LOD is one of acceleration techniques that traditionally used in crowd rendering and automatically generate multiple LOD.Adaptive Level-of Detail for Human Animation (ALOHA) introduced to be applied LOD in behavior apart than geometry and motion [2].There are three rendering techniques applied with LOD [18].Geometry, adapted layered imposter and flat imposter contain sub LOD to create a continuous hierarchy of LOD.LOD applied in KD-tree to improve existing crowd simulation [19].Two components required in LOD: a method of approximating the desired result and a metric for determining how much approximation is allowed.

Image-based Impostors
These approach is different by consider the world as a collection of 2D images, one for each position, orientation and possibly time.The main target of image-based rendering (IBR) is to generate images by directly recreate such an image collection given the view parameters [20], without the need of a full three-dimensional reconstruction.IBR has the theoretical advantage of reducing rendering complexity from (geometric) scene complexity.Fully IBR approach is typically impractical to create full dense for complex model as a set of image.Impostor is the idea of changing geometry-based approach for close object to image representation for far object.Figure 5 shows compressed image used for impostor to reduce texture memory usage [21].Rendering crowd using lower detailed representation of impostor is largely used [21] [22].Impostor inspired from original skeletal animation is a sequence of keyframes [23].The advantage using impostor in crowd rendering is only take two triangles per one virtual human to be appear on the screen.The main drawback is high storage of impostor animation from the total number of textures to be saved in graphic processing unit (GPU).This problem can be reduced by compression technique to lower the amount of texture memory [21].Dobbyn et.al. introduced switching between geometry and impostor representation method known as "pixel to texel" ratio.From the results proved that image-based impostors are an excellent replacement for geometry to efficient rendering, and also in terms of visual realism in crowd rendering.

III. ACCIENT MALLACA VIRTUAL WAKLTHROUH SYSTEM
The Ancient Malacca Virtual Walkthrough [1] is a project that focuses on the modelling and visualization of Malacca city in 15th century.It is based on local and foreign sources, such as the Sejarah Melayu and the descriptions by Portugese writer; Tome Pires described the city and the empire as an opulent and prosperous centre of maritime Malay civilizations.As a maritime empire, trading and commercial activities, both local and foreign, became the mainstay and the backbone of her economy.The focus area of visualization is Central Business and Administrative District of the Malacca Empire.The project is visualized in real-time rendering mode using SGI Onyx 3800 with 16 CPU, 32GB RAM and three Infinite Reality3 graphics pipes.
Figure 6 show a screen capture of Ancient Malacca Virtual Walkthrough.Currently, there is no crowd simulation done to this walkthrough project.The challenge of this project is to bring the visualization of crowd simulation to the project with low computational cost.IV.FRAMEWORK Propose of this research project is to integrate crowd simulation into the Ancient Malacca Virtual Walkthrough.Ulicny et al. [24] has proposed Crowdbrush as an approach to create complex scenes involving thousands of animated individuals in a simple and intuitive way.By employing a brush metaphor, analogous to the tools used in image manipulation programs, they can distribute, modify and control crowd members in real-time with immediate visual feedback.They define concepts of operators and instance properties that allow creating and managing variety in populations of virtual humans.An efficient technique allowing rendering up to several thousands of fully three-dimensional polygonal characters with keyframed animations at interactive framerates is presented.Millan et al. [25] present another technique suitable to render large crowds of characters that takes advantage of existing programmable graphics hardware.Impostors are used for low-detail representation, while pseudo-instancing is used for higher detail.Their technique also used a LOD map to select between both representations, based on a customizable threshold.Figure 7 shows the overview of our framework for crowd rendering in Ancient Malacca Virtual Walkthrough [26].In the beginning, user will run the walkthrough using user interface provided.Database will store all the models and objects, either static or dynamic in database manager.Before rendering start, view frustum culling will be used to cull any unneeded objects from the walkthrough system.View frustum culling that will be used here is a new improved technique that is developed to reduce the computational cost of virtual heritage application.This is named as Range Detection Technique (RDT) [1].It is based on the View Frustum Culling (VFC) method.The conventional VFC method tested the intersection of six planes using the plane equation to determine the visibility scope.Unlike conventional VFC technique, RDT is based on camera referential points and test the 3D objects whether they are in the viewing range or not.RDT execute the testing with the combination of bounding volume.
The database manager will be called to render all objects after crowd rendering starts.A fragment program will read the character pixel buffer as a texture, and then will update the attributes for characters.The new position and heading will be then copied to a new character pixel buffer.Both character pixels buffers can be swapped after the rendering procedure has finished.This pass can also include the update of the animation frame for characters.This framework will utilized particle system technique for simulating crowd behavior in crowd rendering part.Particle system [27] will calculate and recalculate each character destination during the simulation.From this calculation, the crowd can move randomly throughout the Ancient Malacca terrain within the boundaries that has been set earlier in the particle system.Using collision detection, each character that collides with other character will turn around randomly into another direction and continue walking to a new destination that has been calculated in particle system.Collision detection and height-map are added into the crowd rendering to add realism of the crowd movement.Using height-map, the crowd can move ups and downs along the terrain provided in the Ancient Malacca Virtual Walkthrough.Lastly, the system will update the simulation for user visualization.

V. CONCLUSION
In this paper we have presented some of the related work in virtual heritage system, some of crowd rendering issues, a few types of rendering optimization methods.We also introduce our virtual heritage system named Ancient Malacca Virtual Walkthrough that was used as main platform in this research.Later, we give an overview of the framework that used for developing crowd simulations in Ancient Malacca Virtual Walkthrough project.For the future work, there are rooms of improvement to the framework developed.As the framework will be using CPU for the entire processing task, we can also make use the power of nowadays GPU to take part in the processing task.This will make the application not only rely with the CPU but also distribute the task to GPU.As conclusion, we hope that this research is useful for the virtual walkthrough system developer.Consequently, it can also benefit the virtual heritage community.

Mohd
Shahrizal Sunar, Mohamed `Adi Mohamed Azahar, Mohd Khalid Mokhtar and Daut Daman Department of Computer Graphics and Multimedia, Faculty of Computer Science and Information Systems, Universiti Teknologi Malaysia.

Fig. 2 .
Fig. 2. The difference virtual walkthrough executed between with (a) and without (b) Range Detection Technique to improve view frustum culling [1].