Automatic Modeling of Frequent Behaviors of Avatars and Players in a On Line Game

Millions of people now participate in on line games, placing tremendous and often unpredictable maintenance burdens on their operators. Thus, understanding the dynamic behaviors of a player is critical for the systems, network, and designers. To the best of our knowledge, little work builds character interaction model based on the data stream mining. This work improves our understanding the behaviors of avatar/player in a on line game by collecting the behavior data, extracting frequent behavior patterns, learning the hidden hints and making good prediction on responses to the unexpected impacts. Besides, we develop two efficient approaches for mining the behavior data to find the interesting behavior pattern for future prediction on responses of opponents. Our novel findings include the following: One, due to the constraints of limited resources of time, memory, and sample size, MSS-MB was proposed to meet these conditions; the other, due to the constraints of real-time and on-line, there may have some errors occurred in the processing period, MSS-BE was proposed to control the errors as needed. Finally, based on the experimental results, we can predict the responses of opponents efficiently in the on line game.


INTRODUCTION
Character interaction model has become prevalent in the computer graphics industry, the general adaptation and re-use of the data is limited by the available techniques for editing and modifying data in controllable ways.In many games and movies, when impacts are detected, motion-capture driven characters are turned over to a dynamics engine to accept the impulse [1,14].The problem with this method is that their models and approaches are based on vector models [1,2,3,14] or final state machines [2,3,14].
To gain a better understanding of the patterns of player interaction (or NPC: non-Player Character) and their implications for game design, we analyze "how and what players interact".Analyzing user behavior based on database-level traces is particularly useful for our purpose, since the inferred user behavioral patterns would naturally connect to interactions factors, such as styles of attacking (and defense) between participating parties.Based on an empirical analysis of player interaction inferred from network traces, this research work aims to put forward architectural design recommendations for online games.
Also the nature of adaptive behavior needs to be decided.As Mozer [5] states in his criticism of smart houses, it is extremely hard to infer the state of mind of the user from observing her behavior, so a more subtle approach needs to be considered, similar to that of recommender systems [6], where the information provided is filtered, according to patterns of similar behavior of other users.Therefore, our User Model (UM) constitutes a basic component of every adaptive game system.The UM represents user characteristics that enable the system distinguish among different players.The UM is built using data that are requested either directly by the players or obtained by logging player interaction.There are different types of player behavior that can be used for building the UM, for example: • Player's characteristics (such as age, gender, location, etc.); • Player's preferences and interests; • Player's knowledge and skills; • Player's behavioral patterns.
On the other side, a data stream is an ordered sequence of items that arrives in timely order [6,7,8,3].Different from data in traditional static databases, data streams are continuous, unbounded, usually come with high speed and have a data distribution that often changes with time [6,14].Under careful observations, our high speed game play data (e.g.continuous sequential player's styles) coincide to have the same characteristics.Besides, there are some inherent challenges for data stream association rules mining [3,12].First, owing to the unlimited amount of stream data and limited system resources, such as memory space, a mining mechanism that adapts itself to available resources is needed.Second, due to the characteristics of data streams, there is a huge amount of data in both offline and online data streams, and thus, there is not enough time to rescan the whole database or perform a multi-scan as in traditional data mining algorithms whenever an update occurs.
Because of these constraints, traditional data mining methods developed for static databases cannot be applied for mining data streams.Here, in our action MMORPG genre, the style data is generated as a stream, yet we would like to find sequential patterns from the players for analysis.However, there has little work on finding effective methods for prediction of actions of opponents in MMORPG.In this paper, we adopt the algorithms from L. Mendes .et. al [8] and make necessary modifications for our work.We propose two effective approaches for mining sequential patterns from continuous player's style data: MSS-BE (Motion-based Stream Sequence miner using Bounded Error) and MSS-MB (Motion-based Stream Sequence miner using Memory Bounds).
The remainder of this paper is organized as follows.Related works are discussed in Section 2. Section 3, describes the proposed problem definition.Section 4 presents our system framework and 5 explain the recommended data streaming mining algorithms with illustrative examples.Section 6 then presents the experiment results.Conclusions are finally, drawn in Section 7, along with recommendations for future research.

Character Interaction Models
Many virtual humans are used in computer games like RPGs, sports games, and action games.In these games, the reactions of virtual humans are created by connecting motions in a motion database such as Motion Graph [1,2,3,9,14,15,16].Therefore, variations among the reactions are limited, and there is a large cost to creating a motion database.Using a database approach, Lee and Lee [10] created realistic motions for boxers; but their method did not create dynamic reaction for contacts.Advances in human interface and virtual reality technologies make possible a wide variety of user inputs.Hence virtual humans are required to perform a wide variety of reactions.Dynamic control and simulation methods use controllers to compute joint torques based on current states and desired actions.These methods create dynamically correct motions from specified motions [1,14], and state machines [2,14].Those methods generate appropriate motions for variations of the input.State machines are used to create the motions and actions of virtual characters that correspond to environments Lattner.et.al. [3] present an approach which applies unsupervised symbolic learning off-line to a qualitative abstraction in order to predict future situation using frequent patterns.However, these researches did not support on-line and real-time direct character interactions.

Data Streaming Mining
Several novel algorithms have been proposed in the literature to tackle this problem.There are generally two approaches: counter-based methods, and sketch-based methods.Counter-based algorithms maintain counters for a fixed number of elements of the stream, and only this limited number of elements is monitored.If an item arrives in the stream that is monitored, the associated counter is incremented, else the algorithm decides whether to discard the item or reassign an existing counter to this item.The prominent counter-based algorithms include Sticky Sampling and Lossy Counting (LC) [8], Frequent (Freq) [6,8], and Space-Saving (SS) [8].
The other approach is to maintain a sketch of the data stream, using techniques such as hashing, to map items to a reduced set of counters.Sketch-based techniques maintain approximate frequency counts of all elements in the stream, and they can also support deletions.As such, these algorithms are much more flexible than the counter-based methods.The prominent sketch-based algorithms include CountSketch1 (CCFC) [6,7], GroupTest (CGT) [6,7], Count-Min Sketch (CM) [8], and hCount (hC) [8].
denote a set of literals, called items.A sequence s contains an ordered set of items X from I denoted by . A data stream of sequences is an arbitrarily large list of sequences.A sequence s contains another sequence s′ if s′ is a subsequence of s.The count of a sequence s, denoted by count(s), is defined as the number of sequences that contains.The support of a sequence s, denoted by supp(s), is defined as count(s) divided by the total number of sequences contained.If supp(s) ≥ σ, where σ is a user -supplied minimum support threshold, then we say that s is a frequent sequence, or a sequential pattern.

■ Style Definition:
In this work, we define six different styles for distinguishing responses from opponents and NPC.The styles are shown in Table 1.IV. SYSTEM FRAMEWORK Our system is divided into four phases including input data stream sequence, data batching mechanism, data stream mining and style prediction mechanism, and NPC animation control.In the beginning, player's on-line data are logged and broken into fixed-sized batches.Next, the data stream mining algorithms are utilized to build the lexicographic tree T0.Once the tree is built, we use the on-line play's style input to query the tree and make predictions.Finally, the returned next possible player's style and its probability of appearance are used to play the NPC's animation.Please note that, before the tree is built, we have no idea about what player's next style is and just play the default corresponding attacked animations.The framework is illustrated in Fig. 1.In the following section, the detail algorithms are described.

V. STREAM SEQUENTIAL PATTERN MINING ALGORITHMS
Inspired by concepts of [8,12], we make necessary modifications and propose two novel streaming mining algorithms, namely SS-BE and SS-MB.These algorithms can break up the stream into fixed-sized batches and perform sequential pattern mining on each batch.They use a lexicographic tree T0 to store the subsequences seen in the data stream.

The Motion-based SS-BE (MSS-BE) Algorithm
Now, we will explain our modified mining algorithms.For the modified method, each node below the root in the tree T0 will have four attributes: Style: denotes a player's style represented by this node; Count: denotes the count of the sequence corresponding to the path from the root to this node; PID: denotes the last pruning batch number before this node was inserted in the tree; batchCount: indicates how many batches have contributed to the count of this node since the last time it was introduced to the tree The MSS-BE algorithm will given below and demonstration example will.

Motion-based SS-BE (MSS-BE) Algorithm
// Tree T 0 is set initially to have the root node with its count that denotes the number of sequences occurred set to 0. Our modifications are in underline and using the node's PID value to compute the B's value instead of using the timestamp approach.This improves the efficiency of tree pruning.
Whenever the set of sequential patterns are needed, we can output all sequences corresponding to nodes in the tree having count ≥ (σε)*N, where N is the number of sequences occurred.Example 1 (MSS-BE).Suppose the batch length L is 4, the minimum support threshold σ is 0.7, the significance threshold ε is 0.5, and the batch support threshold α is 0.4, and the pruning period δ is 2. We assume that player's styles arrive in the following data stream shown in Fig. 2. We then moves on to the next batch, applying PrefixSpan [12] to B2, again with support 0.4.The frequent sequences found, followed by their support counts, are: <a>:2, <b>:3, <c>:3, and <b,c>:2.This causes the nodes corresponding to sequences <a>, <b>, and <c> to have their counts incremented by 2, 3, and 3, respectively.In addition, each of these nodes has its batchCount variable incremented to 2.
A new node is created corresponding to the sequence <b,c>, having count of 2 and batchCount of 1.The state of the tree at this point is shown in Fig. 3(b).S3:Because the pruning period is 2, we must now prune the tree.
For each node, B is the number of batches elapsed since the last pruning before that node was inserted in the tree.In this case, the last pruning can be thought of as occurring at time 0. Thus, B = 2 for all nodes.For each node, B' = B − batchCount.In this case, some nodes have B' = 0, whereas others have B' = 1.According to the algorithm, we prune from the tree all nodes satisfying: In this case, that reduces to: count + B' ≤ 4. The state of the tree after pruning is shown in Fig. 4. If the user requests the set of sequential patterns now, the algorithm outputs all sequences corresponding to nodes having count at least (σε)*N = (0.7−0.5) * 8 = 1.6.The output sequences and counts are: <a>:6, <b>:5 and <c>:5.

The Motion-based SS-MB Algorithm
Style: denotes an player's style represented by this node.Over_estimation: denotes an upper bound on the over-estimation of the count of this node in the tree compared to the true count of the sequence corresponding to this node.Count: denotes the count of the sequence corresponding to the path from the root to this node; Each node below the root in the tree T0 will contain only three attributes.Another algorithm-MSS-MB algorithm and their pseudo codes are as follows.

Motion-based SS-MB (MSS-MB) Algorithm
// Tree T 0 is set initially to have the root node with its count that denotes the number of sequences occurred set to 0.

Input:
D: a incoming stream of sequences S1, S2, . ..; σ: the minimum support threshold; ε: the significance threshold where 0 ≤ ε < σ; L: the length of batch; m: the maximum number of nodes in the tree.Output: Tree T 0 Begin 1. break the data stream into fixed-sized batches, where each batch contains L sequences; 2. min = 0; 3.While (each batch B k that arrives) do 4. Begin 5. Apply PrefixSpan [1] to it, using support ε; Our modifications are in underline and we found that it is necessary for correctness of the algorithm according to the proof.Whenever the set of sequential patterns are needed, we can output all sequences corresponding to nodes in the tree having count > (σε)*N, where N is the number of sequences occurred.
Example 2 (MSS-MB).Suppose the batch length L is 4, the minimum support threshold σ is 0.7, the significance threshold ε is 0.5, and the maximum number of nodes allowed in the tree after processing any given batch is 6.We assume that player's styles arrive as the Fig. 5 shown.S1 Break the data stream into fixed-sized batches, where each batch contains L sequences.Let batch B 1 be consisted of sequences <a,b>,<c,a>,<a,b> and <c,a>.Let batch B 2 be consisted of sequences <a,b>,<b,c>,<c,a> and <b,c>.S2.The algorithm begins by applying PrefixSpan to the first batch with minimum support ε = 0.5.The frequent sequences found, followed by their support counts, are: <a>:4, <b>:2, <c>:2, <a,b>:2, and <c,a>:2.Nodes corresponding to each of these sequences are inserted into the tree T 0 with the respective counts and an over_estimation of 0. The state of the tree after processing this batch is shown in Fig. 6(a).We then moves on to the next batch, applying PrefixSpan to B 2 , again with support 0.5.The frequent sequences found, followed by their support counts, are: <a>:2, <b>:3, <c>:3, and <b,c>:2.So the nodes corresponding to sequences <a>, <b>, and <c> to have their counts incremented by 2, 3, and 3, respectively.A new node is created corresponding to the sequence <b,c>, having count as 2 + min = 2 and over_estimation as min = 0. S3:Because there are now 7 nodes in the tree and the maximun is 6, we must remove the sequence having minimun count from the tree.Breaking ties arbitrarily, the node corresponding to the sequence <c, a> is removed.The variable min is set to 2 where this sequence's count before being deleted.The current state of the tree is shown in Fig. 6(b).
If the user requests the set of sequential patterns now, the algorithm outputs all sequences corresponding to nodes having count > (σε)N.

Action Prediction and Response Algorithm
Once the frequent pattern tree T0 is created and dynamically maintained by the data stream algorithms.With a input of the player's styles in continuous game play, we can predict his next possible style by traversing the tree and compute the probability of its appearance.The following are our algorithms.

Our Style Prediction Algorithm Style Prediction(SP) Algorithm
// T 0 is the lexicographic tree to store the subsequences seen in the data stream.S i is a player's style seen in the data stream.S j denotes the player's possible style that follows S i and prob(S j ) is its probability of appearance.Input: Tree T 0 and some style S i .Output: Predicted style S j and its probability Prob(S j ) Begin 1.If there exists first level node T k that its style match S i Then 2. Look for the child node of T k with largest count and the sequence corresponding to this node is frequent; 3. IF found Then 4. Return the style value S j and its appearance probability Prob(S j ) = count / N where N is the total sequences seen; 5. EndIf 6. EndIf 7. Return "Unknown" and probability value 0; 8. End // end of SP algorithm To speed up the prediction and lower the errors, the tree structure was implemented and shown below.Play the corresponding defensive animation; 8. Else 9.
Play the corresponding attacked animation; 10.EndIf 11.EndWhile End; // End of NPC response algorithm Intuitively, this algorithm will mimic a human opponent's behavior during a hand-to-hand combat situation.

VI. EXPERIMENTAL RESULTS AND DISCUSSION
In this section, the effectiveness of the proposed stream mining algorithm is investigated.All algorithms were implemented in C++ in Torque Game Engine Advanced [14] and The award wining COA Online Game environment.The experiments were run on a PC with a AMD Athlon II X3 2.8GHz CPU and 4 GB main memory, running Microsoft Windows 7 64bit.
The player's and NPC's attacking and defensive style animations are recorded using Moven motion-capture system [15].All the styles are listed in Table 1 and can apply to both player and non-player characters.
The simulation model we used and the experimental results are provided in Section 6.1 and Section 6.2, respectively.

Test data and Simulation Model
First, our on line game model, named as Caotic Age Online, and its link is as http://coaonline.mes.stut.edu.tw/coa_imagepage/imagepage_index.php.Therefore, we can implement these algorithms in this on line game, and observe all possible results and make necessary improvements.To ensure our algorithms feasible, we use the following data stream sequence as a sample player's style data shown in Fig. 8.As a result, they can be broken into total 3 batches and each batch is composed of 4 sequences.The first 16 styles and the order are the same as listed in Example 1 where each style is illustrated in Table 1.For the MSS-BE algorithm, we assume that the batch length L is 4, the minimum support threshold σ is 0.7, the significance threshold ε is 0.5, the batch support threshold α is 0.4, and the pruning period δ is 2. For the MSS-MB method, the maximum number of nodes allowed in the tree after processing any given batch is 7 and the other parameters' values are same as above.

Experimental Results
In this subsection, we report our experimental results both on the MSS-BE and the MSS-MB algorithms.Both algorithms can achieve real-time response running in a separate working thread.
By observing both Fig. 9 and 10, we can easily realize that, in the first 16 styles that player performs NPC responses in just the same behavior for both methods.The main difference shows afterward in the third batch coming up.Because all the second level nodes of MSS-BE tree are pruned after batch 2, therefore we are not able to predict the next possible style during the third batch period.
In summary, the final comparisons are listed in Table 2. Besides, these parameters emphasize the significance and can derive different results in our implementation.In addition, the MSS-MB algorithm is better overall for our style prediction algorithm by exploiting all of the available system memory.The interesting relationship between the minimum support threshold σ and the significance threshold ε is yet to be determined, and will be an interesting topic.

VII. CONCLUSIONS AND FUTURE WORK
In this paper, we propose two efficient tree-based algorithms, called MSS-BE, for mining the set of frequent stream patterns over data streams with bounded-error; the other is called MSS-MB, for mining the set of frequent stream patterns over data streams with memory-bounded.The differences between them lie on the purpose.MSS-BE focus on the number of errors under the user control; on the other side, MSS-MB pay attentions on the upper bound of used main memory.Based on these algorithms, we develop an on-line and real-time model for predicting response actions of opponents in on line game.Experiments show that the proposed algorithms not only attain highly accurate mining results, but also run significant faster and consume less memory than do existing algorithms, such as vector-based on line game.
In the future work, we still investigate correlations among the parameters suggested in our approaches.Besides, more complex situations between NPC and opponents will be our next goals and researches.

α
Input: D: an incoming stream of sequences S 1 , S 2 , . ..; σ: the minimum support threshold; ε: the significance threshold where 0 ≤ ε < σ; L: the length of batch; α: the batch support threshold where 0 ≤ α ≤ ε; δ: the pruning period.Output: tree T 0 Begin 1. Break the data stream into fixed-sized batches, where each batch contains L sequences; 2. While (each batch B k that arrives) do 3. Begin 4. Apply PrefixSpan [1] to it, using support α; 5.For each frequent sequence s i that is found, having count c i do 6.If a path corresponding to this sequence does not exist in the tree Then 7. Create one new path, setting the batchCount and count values of the new nodes to 0 and the PID values to the last pruning batch number; 7. Else 8. Increment nodes' count by c i and their batchCount by 1; 9. EndIf 10.EndFor // For 11.If the number of batches seen is a multiple of the pruning period, δ Then 12.For each node in the tree do 13.Let B equals total number of batches elapsed minus the node's PID value; 14.Let B' = B -batchCount; 15.If count + B'( -1) ≤ εBL Then 16. delete the entire sub-tree rooted at that node from the tree; 17.EndIf 18. EndFor 19.Update the last pruning batch number; 20.EndIf 21. return T 0 ; 22. EndWhile End // procedure MSS-BE ends

Fig. 2 .
Fig. 2. Overview of incoming stream sequence for MSS-BE.S1.Break the data stream into fixed-sized batches, where each batch contains L sequences.Let batch B 1 be consisted of sequences <a,b>,<c,a>,<a,b> and <c,a>.Let batch B 2 be consisted of sequences <a,b>,<b,c>,<c,a> and <b,c>.S2.The algorithm begins by applying PrefixSpan [12] to the first batch with minimum support 0.4.The frequent sequences found, followed by their support counts, are: <a>:4, <b>:2, <c>:2, <a,b>:2, and <c,a>:2.Nodes corresponding to each of these sequences are inserted into the tree T0 with the respective counts , a batchCount of 1 and a PID of 0. The state of the tree at this point is shown in Fig. 3(a).We then moves on to the next batch, applying PrefixSpan [12] to B2, again with support 0.4.The frequent sequences found, followed by their support counts, are: <a>:2, <b>:3, <c>:3, and <b,c>:2.This causes the nodes corresponding to sequences <a>, <b>, and <c> to have their counts incremented by 2, 3, and 3, respectively.In addition, each of these nodes has its batchCount variable incremented to 2.A new node is created corresponding to the sequence <b,c>, having count of 2 and batchCount of 1.The state of the tree at this point is shown in Fig.3(b).S3:Because the pruning period is 2, we must now prune the tree.For each node, B is the number of batches elapsed since the last pruning before that node was inserted in the tree.In this case, the last pruning can be thought of as occurring at time 0. Thus, B = 2 for all nodes.For each node, B' = B − batchCount.In this case, some nodes have B' = 0, whereas others have B' = 1.According to the algorithm, we prune from the tree all nodes satisfying:

Fig. 7 .
Fig. 7. Tree traversal of our SP algorithm 5.3.2NPC Response Algorithm // A streaming sequence of player's style such as D = S 1 , S 2 ,.. (assumed that this is infinite) Input: D. Output: NPC animations results Begin 1.While (each S k that arrives) do 2. Begin 3. Predict player's possible next style and its appearance probability P using SP algorithm; 4. If P ≥ 0.5 Then

Fig. 8 .
Fig. 8. Overview of incoming stream sequence for NPC response algorithm.

TABLE 1： DIFFERENT
STYLES IN OUR CHARACTER INTERACTION MODEL.

6 .
For each frequent sequence s i that is found, having count c i If the number of nodes in the tree exceeds m Then 14. Remove from the tree the node of minimum count ; 15.Set min to equal the count of the last node removed.
End // end of MSS-BE algorithm