A Skinning Method in Real-time Skeletal Character Animation

With regard to skeletal character animation, the question on how to ensure the quality of transition between key frames is of crucial importance. The lack of properly defined motion ranges based on movements would leave the animator with no choice but intervene the result based on camera perspective afterwards, thus creating a lot more work for the animator to modify or clean up the animation curves. Although a number of methods have been raised in these years, such as the Linear Blending Skinning (LBS), they may still have shortcomings in some specific cases, one of which is the obvious unnatural deformation around the joint areas. The primary investigation in this paper is directed to address the problem and improve the framework of rendering in real-time environment with satisfactory skinning effect to the aforementioned scenario, with assistance of GPU computation.

Abstract-With regard to skeletal character animation, the question on how to ensure the quality of transition between key frames is of crucial importance.The lack of properly defined motion ranges based on movements would leave the animator with no choice but intervene the result based on camera perspective afterwards, thus creating a lot more work for the animator to modify or clean up the animation curves.Although a number of methods have been raised in these years, such as the Linear Blending Skinning (LBS), they may still have shortcomings in some specific cases, one of which is the obvious unnatural deformation around the joint areas.The primary investigation in this paper is directed to address the problem and improve the framework of rendering in real-time environment with satisfactory skinning effect to the aforementioned scenario, with assistance of GPU computation.

I. INTRODUCTION & RELATED WORK
The field of skeletal animation is gaining momentum as it provides more physical plausibility than key-frame animation.Instead of manipulating every single vertex directly, "bone" as a conceptual structure helps the whole animation model to be controlled in a more predictable manner, while consuming less memory.The characteristic of separating animation data and model mesh brings the animator with more flexibility: the same set of animation data can be applied to many models, while a model is allowed to load different sets of animation data.
In the traditional key frame animation [1] [2], we usually use liner interpolations to handle the parallel movements of vertices.However, when it comes to rotation, this approach will generate weird outcomes.Because in the process of rotation, the movement is not smooth, as it often involves accelerations and decelerations.Moreover, the trace of a vertex is an arc rather than a straight line, which means even the direction, is ever-changing.Under this circumstance, linear interpolations certainly cannot be satisfying.
By far, the best solution to this problem is using the approach of quaternion [3].The ease of interpolating along a spherical surface will eventually generate a smoother and thus more natural outcome.
Manuscript Received on June 28, 2011 E-mail: x.universe.stars@gmail.comMost research works mainly focus on two topics: physically based modeling and geometric methods.For physical model simulation, it is to simulate the internal structure and make motion analysis.Not only bones, but also muscles and other physical factors are considered.These methods may be employed to generate relatively realistic results.However the methods are in general restricted to off-line simulations or small domain real-time simulations.For geometry methods, there are five directions:

1) Vertex based blending:
The method uses direct interpolation between two sampled meshes in key frame [1] [2].
2) Multi-weight Enveloping [5]: Since direct interpolation makes some errors in process of motion.By using multi-weight enveloping, the transformation of vertex is influenced by more than one matrix to make a relatively smooth result.
3) Skeletal Animation: Vertices are bound to bones, and the influencing weight of each bone must be specified.4) Linear Blending Skinning [6]: Blending the matrices of each bone directly with weight bound.
In our paper, a set of handling process is presented to the animated character with complex motions, in taking account the consumption for skinning.Besides, it performs high efficiency of rendering by means of dual-quaternion and new characteristics of the Geometry Shading Program (GS), making our algorithm quite suitable for middle scale real-time environment.
The structure of our paper is as follows: after briefly summarizing the recent works on the topics related to our research interest in section 1, we show an overview and the main contribution of our method, with descriptions of the overall algorithms given in section 2. Some key solutions for implementation and the testing results of our method will be presented in section 3. Finally, we conclude in section 4 with some discussion to the future work.

II. PRINCIPLES & METHODS
Our overall algorithms consist of the following stages as shown in Fig. 2: • Data Import (Bones, Hierarchy, and Meshes); Reflection, Gridding, etc.); • Based on the previous steps, producing a frame buffer for the whole character, via skinning and pixel shading.

Weight Allocation
In the traditional workflow, after modeling, artist needs to bind the vertices to the bones, and allocate the weight for each vertex manually.For different type of lives, the weight allocation varies, for instance: • For snakes, fish bodies, and other mollusks, the blending area is relatively large, shown as in Fig. 3.For this case, the weight allocation principle may be expressed by where B 0 , and B 1 are the center position of bone, J is the joint, p is the position of the vertex.We solve w as the unknowns for weight in such case, and the related work will be reported elsewhere.
• For limbs of human beings, vertices are almost allocated to one bone, except the joint area.Thus, the blending area is small as shown in Fig. 4. We use (2) to determine the allocation.
( ) where r is the influence area radius.The result of w is clamped to [0, 1].Our work here mainly focuses on the second type of characters, i.e., human-shape characters.

Hierarchical Frames
Normally, a model is represented in two parts: a collection of interconnected bones, and a surface representation for the whole skeleton.
First, a hierarchy of bones is constructed and developed based on a bone called "the root", and the rest of bones are   either its siblings or its children.When we do the indexing on the skeleton, normally we would look up for the root first, and then index from it until the specific bone is targeted.
Second, we do the transformation on those bones, so as to generate different movements.The transformations traverse from root.For each frame, we process it in the following steps: • Transform the vertex from modeling space to the local space of the bone with the inverse of bind-pose transformation.
• Transform the vertex with local transformation of the bone.
• Transform the vertex from bone space back to world space.
• Transform the vertex with parent transformation.
After each transformation, we need to update the whole skeleton so that each new movement is recorded.

Skinning
After transformations are computed for each bone along the hierarchical frames, we send the matrices to GPU constant buffer as matrix palette.Since the index of bone is bound to each vertex, in vertex shading program, the corresponding matrix can be taken by the indices in the vertex structure from matrix palette.For the vertex with n indices bound, the simplest blending of matrices is Linear Blending Skinning (LBS) [6] (3).
where w is the weight of influencing bone matrix, M is the transformation of the bone.However, this blending method may cause wrong twisting when the two blending bones twist by rotating with different directions.Fig. 6 shows an example.To solve this problem, L. Kavan et al. proposed the method of Dual Quaternion Linear Blending (DLB) [9].Rotation and translation can be denoted as dual quaternion (DQ).The blending is calculated by where w is the weight of influencing bone, q is the dual quaternion, and q (r) is the first row of the dual quaternion.

Quaternion
1) Unit Quaternion: Rotation can be represented by unit Quaternion: ( ) In ( 5) and ( 6), vector u denotes the rotation axis and rotation angle is 2φ.
2) Splitting Matrix: Since the modeling software commonly export matrix to represent skeletal animation data, to transform with dual quaternion, we must split the matrix into scaling matrix, rotation matrix and translation matrix.
[ ] ( ) ( ) 3) Matrix to Quaternion: A group of dual quaternion is denoted as (8) w z y x q q q q q q q q q q q q q q dq |   When the rotation matrix is split, we generate a quaternion for the first row of DQ.The translation matrix is transformed into a quaternion form using following calculation: 4) Quaternion to Matrix: After blending the dual quaternion, the transformation quaternion should be transformed back to matrix form.
For unit quaternion, we generate a rotation matrix using (11): q q q q q q q q q q q q q q q q q q q q q q q q q q q M (11 Translation matrix is generated by

Illumination
We apply Phong Illumination Model [10] to light up the character.

III. IMPLEMENTATION
Our system of skeletal character animation basically has two parts: preprocessing and frame rendering.For the preprocessing part, we use CPU to process the work, and for the frame rendering part, we place the main computation on GPU.
Preprocessing is processed with data initialization, and all the relatively intensive computations should be placed here as much as possible.
1) Data Import: Mesh data is exported from 3D modeling software, which includes vertex data, index data, bone information and hierarchical frames.
2) Hierarchy Construction: We construct hierarchy to hold the index of bones and record the parent-child relation according to the hierarchical frame from the imported data.Matrix container is also created within the hierarchy structure to load the transformation matrices of each frame.
3) Weight Allocation: We calculate the center position of each bones and joint position according to the bind-pose matrix which transforms data from bone space into world space.Then, by solving the equations ( 2), we acquire the weight value to correct the weight binding of joints from modeling software.The result of weight is written into vertex buffer.
Frame rendering runs through the whole lifetime of the program frame by frame.The efficiency of real-time algorithm is highly demanded.

1) Hierarchy Updating:
The coressponding matrices of the current frame is loaded into the matrix container, and the final matrix for each bone is calculated through the hierarchy.
2) Dual Quaternion Construction: Since most of 3D modeling sofwares export animation data in matrix form, while in our implementation, we use dual quaternion.The dual quaternion with scaling data is constructed using (7) ( 9) and (10).After the dual quaternion palette for current frame is constructed, we send the data in float3x4 to GPU.
3) GPU Vertex Shading for Skinning: This is the most important stage for skeletal animation, where skinning computation runs.First we blend the quaternions, and then transform the quaternion into matrix form or other proper forms of transformation.Finally we output the position in modeling coordinates.
4) GPU Geometry Shading for Feedback: This stage is to accelerate multi-pass rendering workflow for preparation.We output the transformed vertex data back into memory buffer.

5) GPU Vertex Shading:
In this stage, we regard the skinned input vertex data as static mesh like other common meshes.The output is transformed into screen coordinates.
6) GPU Pixel (Fragment) Shading: This is the stage to present the illumination effect to final framebuffer or middle data to texture buffer.For illumination, color texturing is sampled combining with the decoding data from normal map, we calculate the final color with Phong Illumination Model.

Transformations with Scaling
Commonly skeletal animation rarely contains scaling bone transformation.Due to the fact that some exporter does export the animation with scaling factor (e.g.FBX format), we must consider the scaling transformation.For weight blending, we use the dual quaternion, but for final transformation, matrix form is still a direct alternative.However transform quaternion to matrix is still expensive.We use ( 14) to simplify the calculation in GPU [11].Since there is no necessity to use rotation matrix and translation matrix finally, we just directly multiply the vertex position by scaling factor instead of matrix calculation, because normal-inverse-matrix generation is still an expensive issue.In details, we handle the transformation and weight blending using following steps in order: • Rotation quaternion is stored in the first row of DQ.
• Translation quaternion is stored in the second row.
• Scaling factor is stored in the third row.
• Transform vertex position using scaling factor S.

Multi-pass Geometry Rendering
In order to avoid the overhead of repeat rendering, we enforce multi-pass geometry rendering using the function Transformed Feedback.Transformed Feedback, also called Stream-Out (SO) is enabled since OpenGL 3.0 and DirectX 10 were developed.With geometry shading, a transformed vertex stream can be stored from GPU to a certain buffer for the use of the next passes.
Due to the expensive consumption of skinning, especially using DLB, we try to make the skinning algorithm run as efficient as possible.Using SO, we can first skin the character only once, and then stream out the skinned vertices to a buffer.For other passes such as shadow map generation, deferred lighting, reflection, we just directly read the skinned vertex buffer as an instance for each rendering pass.

IV. RESULTS
In this section, we show the results acquired by our approach.We show a pair of results to compare the skinning blending methods and an integrated scene with complex models.Our testing machine is equipped with Graphic Card Nvidia R Geforce GT240M and Intel R CoreTM Duo CPU P7450.

Dual Quaternion Linear Blending with Scaling VS Linear Blending Skinning
The main advantage of Dual Quaternion Linear Blending (DLB) with scaling transformation is to eliminate the error morphing of joints when twisting two bones with opposite directions.
For our DLB (Fig. 8) with scaling transformation, the result is obviously improved from the one using LBS (Fig. 9), especially in the shoulder part of the character, since humans frequently raise and swing their arm, leading the shoulder twist fiercely.Besides, the bone transformations we used in our demonstration concern scaling transformation, thus only using the dual quaternion is not enough, due to the omission of scaling.In this case, our method also presents the solution with scaling transformation on bones.

Integrated Example
Here we demonstrate a full system including the figure with complex skeletal animation example (Sword Playing, in Fig. 10, 11 and our attached video) and other 3D models in a complex scaled scene.The example also includes many techniques commonly used in current game.
In our integrated example, the whole scene including trees and bamboos are all modeled in 3D space, with a relatively complex environment.The model statistics is as given in the Table 1.Testing with such environment shows that the algorithm and method proposed is feasible for a general gaming environment in real time operations.V. CONCLUSION In order to provide skinning method for organic models with satisfactory outcomes, we propose a detailed skinning procedure based on dual quaternion while providing strong mathematical validations.Our method can be easily incorporated into many existing real-time animation systems, yielding the benefits of both efficient rendering and highly-realistic results.With this regard, we combine the DLB method with some novel and fast algorithms and models, such as IK solution, motion capture, motion effect with complex character mesh, etc., to enhance the efficiency of computation and improve the interactive effects for real-time character animations.
With the benefits of transformation feedback, skinning computation runs only once, which greatly speeds up the rendering, at the same time avoiding the overhead in the multi-pass rendering workflows.For some pre/post-passes such as shadow mapping, deferred lighting, and so on, the skinned data can be instantiated by those passes.
For future applications, our work has space to improve in the aspects of automatic weight allocation.Since the types of character structures vary, manual weight allocation is still a tough work.To speed up character rigging, we need to enhance our weight allocating algorithm for broad range of applications.To improve the quality of motion, morphing, and emotion of the character, a real-time muscle simulation method with low consumption is also expected.

TABLE . 1
. ORIGINAL MODEL STATISTICS FOR THE INTEGRATED SCENE.