Abstract

Here we present the conception, implementation and application of a virtual environment simulator dedicated to the assessment and training of slap shot performance in hockey. The simulator is based on human-avatar interaction, namely a real shooter and a virtual goalkeeper whose behavior is dependent on that of the shooter. The synthesis of the virtual goalkeeper relied on a high-quality model and realistic, motion-captured movements. A regression model based on Kriging was used to predict in real-time the shooter's behavior in order to trigger the blocking moves of the virtual goalkeeper at the right time.


Our model provided accurate predicted values as well as an estimation of the reliability of these values, which allowed us to optimize the behavioral animation of the virtual goalkeeper. We then ran a validation experiment testing the effectiveness of our simulator. The simulator proved very useful both to assess the initial performance of the players and to train and improve this performance. In particular, training as little as 3 hours with our simulator gave rise to substantial and significant improvements (up to 22 percent) of the redirection threshold, i.e., the minimum time required to successfully redirect a shot during movement execution when the outcome is imperiled. Importantly, in comparison with ‘classical’ training methods, our simulator better triggers (precisely and timely) the movements of the goalkeeper based on the movements of the shooter.

Introduction

The slap shot is the most powerful shot in hockey, with puck speed reaching 175-180 km/h (Lomond et al., 2007). However, as compared to a wrist shot, one downside of the slap shot is its predictability. In particular, because it requires more preparation, a slap shot gives out more information to the goalkeeper who can then better anticipate. The performance of a slap shot depends on three factors: the quickness of the backswing, the timeliness of the shot (in order to catch the goalkeeper by surprise) and its precision. In practice, slap shots have become less decisive due to the improvement of goalkeepers' performances and better anticipation from defenders, so that many slap shots are actually blocked. Therefore, the shooter's ability to anticipate his opponent's intentions within a very short time becomes all the more important. This issue is common to many actions in interactive sports.

In line with this, the ability of shooters to redirect their shot when the goalkeeper anticipates the shot and blocks their shooting angle plays a key role. Studies on the penalty kick in soccer have shed light on how shooters select their target area. In 75% of the cases, the goalkeeper makes the first move and then the shooter aims at the open side of the goal (Kuhn, 1988). Hockey players face a similar issue and need to improve their ability to react and alter the direction of their shot at the last moment. In particular, for close spatio-temporal interactions between shooter and goalkeeper (and / or opponent), a few ms delay on the part of one or the other can change the outcome of the action. Offering a personalized and synchronized learning process between two real protagonists is therefore unrealistic. Virtual Reality (VR) and Virtual Environnent (VE), on the other hand, makes it possible to have complete control over the movements (i.e., type of movement and timing) of one of the protagonists. VR also entails clear benefits in this case: (1) it reduces training times (Wright, 2014), (2) it facilitates the transfer of learning to the real situation (Covaci et al., 2014), and, (3) it grants the possibility to repeat sequences, each at various speeds, which cannot be done during a real training session (Metoyer & Hodgins, 2000). In this study, we used a human-avatar interaction paradigm in VR to assess and improve the ability of hockey players to alter the direction of the slap shot during movement execution (i.e., online) when required.

The virtual simulator used in this study consists of a virtual goalkeeper who interacts in real time with the players and forces them to adjust and redirect their shot at the “last moment”. First, the simulator assesses the players' reaction ability during an action, i.e., the rate of successful redirections and the redirection threshold. For each player, it yields (i) a time threshold value, (ii) an effectiveness value, and, (iii) data on the redirection preferences. In addition, the simulator provides a training tool to improve the players' sensorimotor skills. Specifically, a training protocol is tailored to each player in order to optimally improve his/her ability to redirect the shot during movement execution when the situation requires it.

In the following section, we review the existing literature on human interactions with a virtual opponent (VO) in sports, with a focus on analysis and learning. The next section addresses issues related to online control for complex movements like the slap shot. In section 4, we present the simulator, from the creation of the 3D animation of the goalkeeper to the implementation of a model of interaction capable of calculating the remaining time before the stick hits the puck. In this section, we also detail the controllers (goalkeeper AI) we used to test and collect the players’ reaction times. Lastly, in section 5, we present an experiment validating the use of our simulator to assess and train the sensorimotor skills of hockey players on the slap shot.

Human-virtual opponent interaction for assessing and training skills in virtual environment

Human-VO Simulators

Virtual opponents have been used in several entertainment-oriented projects. We call them entertainment-oriented because their purpose was not a systematic improvement of performance. For instance, Rusdorf and collaborators (Rusdorf et al., 2007) implemented a table tennis simulator. The animation and strategic choices of the VO resulted in a spatially and temporally consistent interaction with the player. The VO's strategy consisted in following spatial rules that would challenge its opponent. More recently, Moazzen and Ahmadi (2017) implemented a tennis simulator with a VO whose behavior was controlled by a multilayered perceptron neural network. Like in the previous study, the behavior of the VO was not time-locked to the behavior of the user. In another study, Zhang et al. (Zhang et al., 2018) created a karateka simulator that interacts with a human karateka. The VO's strategy was predefined by a procedure set by the designers. There were no specified reaction times for the VO, but its actions were coordinated in a plausible way, i.e. it would engage as soon as an attack was possible. These studies on training provide no analysis of the trainees' behavior, nor are they designed to improve sensorimotor skills.

Skills assessment and training

Several studies used VE technology to improve ball sports performance (see (Miles et al., 2012) for a review). For instance, Bolte & al (Bolte et al., 2010) designed a system to assess goalkeepers' skills in handball. Their system consisted in displaying videos of attackers who randomly aimed at the corners of the goal. Score analysis was done by comparing the targets aimed at by the attackers and the reactions of the goalkeepers. Another study on baseball analyzed the judgment and interception skills of participants who were asked to intercept virtual balls in a CAVE (Zaal and Michaels, 2003). Specifically, they analyzed the behavior of players to catch balls in different virtual situations and confirmed that the catching behavior was consistent with Chapman's strategy (strategy frequently used by catchers). They found that in a CAVE, judgment and interception are performed in the same manner as in natural environments. They also highlighted the importance of giving feedback for improving judgment. Regarding decision-making, Brault and colleagues (Brault et al., 2012) analyzed the skills of expert vs novice rugby players. They quantified the participants' ability to predict the change of direction of a virtual opponent performing a deceptive movement. Some (few) other studies put a stronger emphasis on training. Bideau and colleagues (Bideau et al., 2004), who studied the behaviors of handball goalkeepers interacting with virtual throwers, have shown that VR can be used as an alternative to real situations in a training context. Vignais and collaborators (Vignais et al., 2009) designed a simulator to train handball goalkeepers facing virtual shooters aiming at 3 different zones of the goal. They compared response accuracy in a judgment task (manually predicting where the ball will go) and in a motor task (reacting in the same way as in reality). They observed a higher accuracy in the motor task. Using the same simulator, Bideau and colleagues showed that handball goalkeepers are better at anticipating the target aimed at by the virtual shooter when they observe the trajectory of the ball rather than the throwing action (Bideau et al., 2010).

Contribution

In our study, participants had to produce 'spatially' adequate actions, i.e., shoot the puck at the free space. But in addition, our simulator and training protocol also had a strong focus on the response time. Specifically, we assessed and aimed at improving their sensorimotor skills in terms of response time to detected perturbation. Along that line, our contribution goes beyond previous studies at two levels: at a technical level, we designed a new model of interaction based on Kriging regression in order to control the behavior of the VO based on the movements of the trainee; at an experimental level, we addressed a new issue, namely the ability to modify online an ongoing complex movement taking into account the movements of the opponent.

Sensorimotor control of simple vs complex movements

Figure 1: The different key times during the slap shot action. t2: beginning of the slap shot for the player and idle position for the goalkeeper. t1 is the goalkeeper trigger time fine-tuned according to each player’s sensorimotor skills. t0 corresponds to the contact between the stick and the puck. t-1 corresponds to the end of the virtual goalkeeper’s movement.

The sensorimotor loops underlying the control of simple movements have been extensively investigated (see (Sarlegna and Mutha, 2014) for a recent review). Most of these studies are based on reaching paradigms (Prablanc, 1992; Sarlegna et al., 2003). Specifically, the participants are instructed to reach for visual targets with their arm, and perturbations are introduced during movement execution. For visual perturbations, online modifications of the arm trajectory are usually observed between 280 and 350 milliseconds after the perturbation, whereas the first electromyographic responses take place about 110 ms after the perturbation (Reichenbach et al., 2009). For more complex movements, such as the penalty kick in soccer, response times are usually longer (Mallo et al., 2012; Morya et al., 2003; Van der Kamp and Masters, 2008; Van der Kamp, 2006; Wood and Wilson, 2010). For instance, Morya et al. (2003) implemented a computer program which measured the reaction times of kickers whose task was to decide during the run-up if they would kick the ball to the right or left side of the goal. The challenge here resided in the fact that the goalkeeper could make a move during the run-up to the ball. When the goalkeeper moved more than 400 ms before foot-ball contact, the kicker's response was correct in 100 % of cases. On the other hand, success was down to chance when the goalkeeper moved less than 150 ms before contact. Closer to real conditions, Van der Kamp (2006) conducted a similar experiment on a soccer field. The reaction of the goalkeeper was represented by lights that went on to the right or to the left side of the goal. The percentage of success reached a minimum of 75 % when the visual target changed around 600 ms before foot-ball contact and remained under 50 % when the target changed around 400 ms before contact. To this day, the ability to redirect a complex and very fast movement as the slap shot in hockey has never been investigated.

Application to our context

As mentioned above, redirecting a movement during its execution (i.e., online) in response to a visual perturbation takes about 300 ms for a simple reaching movement performed with the arm (Bingham et al., 1999; Schmidt and Lee, 2005), at least 400 ms for a more complex movement performed with the leg (Van der Kamp, 2006), and more than 700 ms for a whole-body movement, such as a dive for a soccer goalkeeper (Kerwin and Bray, 2006). The redirection of the lower body during a penalty kick action is usually observed between 400 and 600 ms before foot-ball contact. The slap shot movement involves a joint chain going from the shoulder to the wrist. We assumed that the redirection of such a movement would require less time than the redirection of a penalty kick. In addition, depending on the speed of the slap shot, we calculated that the puck would reach the goal (8-10 m away) in 180 to 320 ms (for respective speeds of 160 km/h to 8 m and 90 km/h to 8 m”). Because the goalkeeper needs at least 280 ms to react and about 600 ms to perform his blocking movement (from ~500 to ~700 ms in our captures), he must start moving before stick-puck impact to stand a chance to block the puck. Thus, depending on the situation, the goalkeeper should start moving between 180 (worst case for the shooter: the goalkeeper is fast [500ms] and the shot slow [320ms]) and 520 ms (best case : the goalkeeper is slow [700ms] and the shot fast [180ms]) before stick-puck contact. Our problem statement therefore applies to part of the slap shots that are shot between 8 and 10 m, and applies more generally to those shot under 8 m.

The action proceedings are illustrated in Figure 1. t2 marks the beginning of the slap shot for the player and the idle position for the goalkeeper. t0 corresponds to the contact between the stick and the puck. t-1 marks the end of the virtual goalkeeper’s movement. The "last moment" mentioned earlier, represented by t1, is the time left before stick-puck contact from the kicker’s point of view. It also corresponds to the time to trigger the goalkeeper. We will see in the coming section that t1 is fine-tuned according to each player’s sensorimotor skills. These adjustments are made by using two algorithms. One assesses the remaining time before stick-puck contact, while the other analyzes the player’s reaction times and sets t1 at different times before puck contact for each slap shot iteration.

The simulator

Technology used

Many different technologies can be used in VR, both for motion capture (inertial, ultrasonic, optical, etc.) and scene / simulation display (glasses, screen, impact screen) (Miles et al., 2012). Regarding motion capture, we used an optical system which captures at high frequency and provides accurate data. The virtual scene was displayed on an impact screen, which allowed the player to perform real slap shots and thereby 'physically' interact with the virtual scene. More details about the simulator are provided in section 5.

Purpose

Several studies have shown that the conditions of practice must be as close to reality as possible to optimize transfer to real performance (e.g. (Miles et al., 2012)) (P.1). These authors also suggested that to ensure good learning outcomes with a virtual environment (VE), several constraints must be complied with: (P.2) the movements performed must be the same as in reality, (P.3) the learner should be free to perform his movements under various initial conditions, which means that the participant can position and orient himself as he wishes. (P.4) the learner should get feedback on the consequences of his movements, (P.5) if the action allows it, the learner should be provided with sensory feedback about the performance of his movements (Schmidt and Lee, 2005).

Our goal was to design a simulator meeting these criteria. Note however that for the slap shot, sensory feedback about movement performance is quite limited, namely to the impact of the stick on the floor and on the puck. We also wanted our simulator to be as immersive as possible. For that, our simulator needed to be realistic, fast, and the 'behavioral' interactions with the user / trainee needed to be relevant / consistent. By "realistic" we mean that the VE should reflect the player’s usual environment and the goalkeeper’s moves should look natural. It has to be "fast" because the virtual goalkeeper needs to interact in real time with the participant. By "relevant / consistent", we mean that the goalkeeper needs to move in the right direction at the right time.

Conception

The conception of the simulator relied on three main 'features': (i) the creation of the virtual scene which comprises the environment and the virtual goalkeeper (Virtual scene subsection), (ii) the implementation of a learning algorithm capable of computing in real time the time before stick-puck contact at any moment (Interaction goalkeeper learning algorithm subsection) and (iii) the ability to control the application through psychophysical algorithms (Controller algorithms subsection).

Figure 2: Overview of the different steps performed to implement our simulator.

Virtual scene

We start by explaining how we designed and implemented the virtual scene to comply with P.1. This phase is akin to the work required to create a video game. The animation of the goalkeeper can be divided in three steps:

Goalkeeper capture. We asked a professional goalkeeper who plays in the Swiss first league to perform the movements required for our experiment, namely idle phase, top-left, top-right, bottom-left and bottom-right saves. Performing the motion capture was challenging because the goalkeeper’s equipment is loose-fitting and the markers, if stuck on it, are too far from the joints’ centers to perform quality captures. Therefore, we chose to capture the goalkeeper’s movement without his equipment. Note that this choice generated other problems in the subsequent step of the pipeline. The capture of the participant was performed with the Optitrack system and 12 infrared cameras. The goalkeeper was equipped with 49 markers. The computation of the skeleton was performed by Motive (NaturalPoint Inc.). This computation consisted of 1) the creation of a skeleton consistent with the morphology of the goalkeeper, and 2) the updating of the skeleton rotations driven by the 3D displacement of the markers over time. The captures were done at a frequency of 120Hz.

3D mesh creation. The 3D mesh was created in two steps. First, the mesh of the body was designed with Fuse (Adobe) according to the morphological dimensions of the subject (28000 triangles). These dimensions have been collected on the participant himself as well as on the skeleton computed by Motive. The equipment was designed by an infographic designer (46000 triangles).    

Motion retargeting and editing. The retargeting process between the captured skeleton and the mesh skeleton was straightforward thanks to the work that had been done during the previous phase (same morphology, same length of segments, etc.). However, we needed to edit the movement to overcome the inter-penetration conflicts due to the voluminous equipment of the avatar. To compute this task, we used an inverse kinematic algorithm driven by constraints preserving the spatio-temporal relationships of the motion. The method is detailed in the work of Le Naour (Le Naour et al., 2013).

Regarding the virtual environment, the stadium was created by a graphic designer and is illustrated in Figure 2.

Real-time computation of the time to stick-puck contact

A controller algorithm was dedicated to the choice of the stimulus (i.e., initial target corner for the participant and blocking move of the goalkeeper) and of the key time to trigger the goalkeeper. This controller needed to estimate at any time how much time was left before stick-puck contact. We start below with a short description of a typical slap shot sequence before explaining how we computed the key time prediction.

The slap shot can be divided into six distinct phases. The first three phases are set before stick-puck contact (backswing, downswing, preloading) whereas the other three are set after contact (loading, release, and follow-through) (Hoerner, 1989). For the issue addressed in our study, only the first three phases are important. We first asked two professional hockey players (Swiss first league) to perform a set of slap shots (short to long ones) and inferred that the duration of a slap shot ranges between 550 and 1500 ms. The average duration of the downswing is of about 330 ms. As mentioned in section (4.3), the minimum time needed to redirect a movement ranges from about 280 ms for simple movements to 700 ms for complex movements. Thus, to predict the time remaining before stick-puck contact (t1), we needed to consider both the backswing and downswing phases.

At this stage, we wished to comply with various criteria specified above: (1) the participant must be free to position himself and move as he wishes (P.2 and P.3); (2) the simulator must be quickly operational (the calibration phase must be short). Accordingly, we chose to capture only the trajectory of the extremity of the stick over time. The position of the puck was fixed. The trajectory of the stick is defined by the time series pk = pki where pki ∈ R3, i ∈ [0, n] is the index of the current frame. 0 is the first capture of the backswing slap shot and n corresponds to the last stick/puck contact capture. k is the index of the slap shot (here k ∈ [0,4]). Determining t1 from pk is a prediction problem.

Let us state the problem clearly. We denote by x = {x1, …, xn}T the n-dimensional vector which represents an observation of n input parameters. For each observation x corresponds a time y at m distinct locations (m = p slap shots * mk observations). Our training data is composed of the matrix of observation X = [x01, …, x0i, …, xp1, …., xpmk]T and the matrix of times Y = [y(x01), …, y(xpmk)]T. xki represents the observation at the frame i of the slap shot k. Each observation x is mapped to a time value y(x). We want to estimate ŷ(x*) given a new x*. As explained in the next paragraph, by using derivatives of the positions, this problem is non-linear. Dynamic time warping procedures or Hidden Markov Models are usually used to solve time series problems (Fu, 2011). However, in our case, the computation needed to be performed in real time and during movement execution. These methods are not suited to this kind of problem. Recent methods using deep neural network make it possible to compute good prediction in real time, but they do not estimate the error of prediction of the computed value.

Universal Kriging (UK) as regressor

We chose to use Universal Kriging to predict ŷ(x*). Kriging is traditionally used in the field of geostatistics to compute spatial prediction of meteorologic phenomena (Cressie, 1988). More recently, UK has been applied in other fields, as for instance to interpolate motions in computer animation (Mukai, 2005). UK extends Gaussian Process by including a trend as a determinist component in its formalization, for problems that are not stationary:

Y(x) = 𝜇 (x) + Z(x)

Where x ∈ Rn and Z(x) ~ N(0, k) is the Gaussian process. k is a covariance kernel. The scalar value y(x) is considered as a realization of the stochastic process Y(x). More precisely, y(x) = Y(x, 𝜔) where 𝜔 is the probability of the realization. UK is based on a training consisting of two steps. In the first step, (represented here by polynomials) is computed. In the second step, the Gaussian process Z(x) based on the residuals of is computed. Finally, the predictive value is given by ŷ(x*) = 𝜇(x*) + Ẑ (x*).

The choice of UK was motivated by several factors: (1) UK predicts values with estimations on error of the prediction. This feature of UK was useful to determine whether to trigger the goalkeeper or not; (2) The model can be trained with a small number of samples. We propose to use 4 slap shots here ~400 to ~600 values from a capture rated at 240 Hz). (3) The prediction interpolates the observations. (4) After a training of the model, prediction times are short. (5) Our captures were quite noisy. There was an uncertainty of ~0.5 mm between each time step capture. With a high frame rate, this uncertainty influences the acceleration and velocity derivatives. By using a white kernel (included into the Gaussian process), UK is efficient for processing noisy data (Cressie, 1988).

Choice of x

The time series pk provided several parameters at each time step: the height H, the distance to the puck D, the velocity V, the acceleration A as well as the vertical direction S. We chose not to use the 3D positions directly since they represent information that is too specific to the occurrence of the slap shot (positioning and orientation of the participant). Table 1 shows the errors of prediction using different set of parameters. The last line gives the average computation time for a prediction. According to the measures illustrated in the Table 1, we decided to combine all these parameters at 3 time-steps: the current time, 160 ms before and 320 ms before (i.e., respectively 20 frames and 40 frames before at 120Hz). Figure 3 illustrates the error and the variability as a function of time before stick-puck contact. As illustrated by the Figure and the Table, the error of prediction in the interval of interest (350 ms to 500 ms) is of ~15 ms with a maximum of 44 ms. This time is not perfect but thanks to the estimation of prediction error provided by UK, we can decide if we use this prediction or not. The average time to compute a prediction is 8 ms (125 fps).

Table 1: Time computation and comparison of errors of prediction times with different sets of parameters (ms). The learning step is computed using 4 slap shots (540 samples). The mean and median errors correspond to the error of prediction given from a set of 2850 samples ranging from -1200 ms to 0 ms (21 slap shots).

Figure 3: Evolution of the prediction time error (red line) related to the time before stick/puck contact. The blue line represents the stick height related to the time.

Finally, the predicted time ŷ(x*) is estimated by:

ŷp(x*) = ŷ(x*) – (c0 + cn + cc)

where c0 = 4.5 ms corresponds to the motive system latency (reception of camera data and processing in motive), cn = 5ms corresponds to the network latency and = 8 ms corresponds to the computation time of ŷ(x*).

Time control algorithms

We used two algorithms to control the time of the visual stimulus at each shot, i.e., the time at which the virtual goalkeeper was triggered. The first algorithm was dedicated to the analysis of sensorimotor skills through the staircase algorithm. The staircase procedure is commonly used in psychophysics to quantify the relationship between a stimulus (here its time) and its impact on human behavior (the redirection). This procedure is iterative and converges towards the threshold time corresponding to the point in time from which the player can no longer redirect his shot. The second algorithm controlled the learning feature through a scan of a range of values randomly distributed around the threshold time that had been collected by the first algorithm.

Figure 4: Overview of our simulator.

General conception

To summarize, our simulator can be subdivided into 4 components illustrated in Figure 4:

a real-life set-up centered on the immersed user / trainee;

a step that collects the data (data part). This step is performed by Motive / PC1;

the time control algorithms are used at the beginning of each slap shot and the time predicting algorithm triggers the virtual goalkeeper in adequacy with the previously collected data. This step is performed in C++ (homemade application) by PC2 (control part);

and lastly, a display of the virtual scene (View part). PC2.

Experiment

Methodology

Participants

A study conducted by Wimshurst (Wimshurst et al., 2012) showed that it is possible to train high level players regardless of their age and level of expertise. 12 participants aged 20 to 32 participated in our experiment (mean age: 25.2, standard deviation: ± 3.8), each having a good level of expertise in the slap shot movement (~19 years of experience in ice-hockey). All participants were left-handed players (i.e., holding the stick on the left side of the body). Eleven of them played ice-hockey five times a week in the same amateur 1st league team (the third highest league in Switzerland). The twelfth participant practiced 9 times a week in a professional National League A team (the highest league in Switzerland). Participants wore their skates and had their own stick.

Apparatus

The experiment took place in a 6x9 m room equipped with a 4 m wide x 2.7 m high impact screen. We used the same motion capture system (Optitrack) with the same settings as for the capture of the goalkeeper and for building the learning database: 12 cameras / 240 frames/sec + motive software on PC1. The frame rate was of 80 frames per second. As explained previously, a second computer (i.e., PC2), is dedicated to the control, animation of the goalkeeper and the rendering of the scene (see Figure 5). The PC1 and PC2 were equipped with an i7 3.40 Ghz core i7 processor (16GB of Ram) and a NVidia GForce GTX 770 graphics card.

Figure 5: Top: the idle position of the goalkeeper with the 4 target corners. Bottom: the 4 possible goalkeeper saves.

Procedure

The shooter had four shooting options, namely the four target corners, and four goalkeeper actions were possible, for a total of 16 scenarios. Four of these scenarios (Conflict) required the shooter to modify his shooting movement during movement execution because the goalkeeper action blocked the corner corresponding to the target corner. Because the aim of the experiment was to assess and train the ability to modify the slap shot movement online, the main part of data analysis focused on these four 'Conflict' scenarios. The other 12 scenarios were used to make sure that each participant correctly applied the procedure (we could check if their shot was consistent with the target corner indicated by the program). Figure 5 illustrates the movements that could be performed by the goalkeeper.

For each shooting iteration, the participant was assigned one target corner, i.e., he was told at which corner to shoot. This target was indicated by displaying a green rectangle in one of the four corners of the goal. For Conflict scenarios, i.e., when the target corner of the shot was the one blocked by the goalkeeper’s move, the shooter had to try to redirect his shot in order to score. He remained free to position and orient himself as he wished to.

Each participant followed the same test procedure consisting of three shooting sessions with a one-week interval between two successive sessions. The duration of each session was approximately one hour and included exactly 200 slap shots with a minimum of 80 shots in Conflict scenarios. The procedure consisted of 3 steps: (1) Pre-test controlled by the staircase algorithm (week 1); (2) Range training with thresholds based on the results of the pre-test (week 2); (3) Post-test controlled by the staircase algorithm (week 3).

The four staircases were configured with a 2up-1down procedure, an initial value of 500 ms, a maximum value of 600 ms, a minimum value of 150 ms, and a maximum of 5 reversals. The four staircases were randomly chosen. A hockey expert familiar with psychophysical tools validated or invalidated the trial at each step of the staircase procedure. For each Conflict scenario, the prediction algorithm detailed earlier estimated the prediction error. We decided arbitrarily that beyond 20 ms of estimated error, the shot would be postponed to the next iteration. In addition, after each Conflict scenario, only the shots with less than 20 ms of error compared to the predicted t1 were kept. For the training session, the choice of the corner was randomized for each trial.

Results

Data analysis focused on values measured in conflict scenarios, i.e., when the goalkeeper blocked the target corner, forcing the shooter to redirect his shot. Redirection performance was assessed using two dependent variables, namely the percentage of successful redirection, and the redirection threshold, i.e., the minimum time before stick-puck contact required to successfully redirect a shot. The Figure 6 gathers the means and p-values of successful redirection and redirection threshold for the four target corners before (pre) and after (post) training.

Because our two dependent variables were likely to be related and to share some variance, we first ran a 2*4 MANOVA for repeated measures (2 sessions [pre, post] * 4 target corners [bottom left, bottom right, top left, top right]). The Pillai’s trace test indicated a significant effect of both main factors, namely the session (i.e., pre vs post, V=0.74, F(2, 10) = 14.137, p < 0.01) and the target corner (i.e., initial target corner, V = 0.43, F(6, 66) = 3.01, p < 0.05) on the percentage of successful redirection and the redirection threshold. The interaction between the two main factors was not significant.

Figure 6: Rate of successful redirection (top panels) and redirection threshold (bottom panels) for the four target corners before (pre) and after (post) training with our simulator. The results of the different statistical comparisons are also indicated.

As a second step, we ran two separates 2*4 repeated measures ANOVAs, i.e., one on each of the two dependent variables. For both ANOVAs, normality was assessed running the Shapiro-Wilk test on the residuals, and p-values were Huynh Feldt corrected for sphericity when required. Post-hoc tests were Bonferroni-corrected for multiple comparisons.

Regarding the percentage of successful redirection, the ANOVA indicated a significant main effect of the session (F(1,11) = 8,96, p< 0.05, n2g = 0.12), i.e., 12 percent of the total variance was explained by the factor), the percentage of successful redirection being significantly higher in the post than in the pre session (see Figure 7, top panel). The ANOVA also indicated a main effect of the target corner (F(3,33) = 3.99, p< 0.05, n2g = 0.07). On average, the percentage of redirection was significantly lower when the initial target corner was the bottom-right corner than when it was the top-left or the top-right corner. The interaction between the two main factors was not significant.

A very similar pattern of result was observed for the redirection threshold. Specifically, there was a significant main effect of the session (F(1,11) = 19.54, p< 0.01, n2g = 0.13), redirection threshold being significantly lower in the post than in the pre session (see Figure 8, bottom panel). There was also a main effect of the target corner (F(3,33) = 4.42, p< 0.05, n2g = 0.14), the redirection threshold being significantly higher when the initial target corner was the bottom-right corner than when it was either of the other three corners. Here again, the interaction between the two main factors was not significant.

Figure 7: Overview of the rate of successful redirection (top panels) and redirection threshold (bottom panels) for the four target corners.

To assess the initial performance of the players, and notably to determine whether their ability to redirect their shot depended on the initial target corner before training, we compared the percentage of successful redirection measured for the 4 goal corners in the pre session. The one-way repeated measures ANOVA indicated that the percentage of successful redirection was similar for the four corners. The same analysis was performed on the redirection thresholds, and here again, there was no significant difference between the four corners.

We then more specifically assessed the effectiveness of our training. As mentioned above, the 2*4 repeated measures MANOVA as well as the two separate 2*4 repeated measures ANOVAs all indicated a significant difference between the pre and the post session, redirection performance being always better in the post session (i.e., higher percentage of successful redirection and lower redirection threshold). To have a more 'fine-grained' idea of the improvement, for each target corner, we compared the percentage of successful redirection performed in the pre and post sessions. For each corner, we first computed the post-pre difference and assessed the normality of this difference vector using the Shapiro-Wilk test. Because the vector of difference was normally distributed in all four cases, pre-post comparisons were performed using paired t-tests. For all four corners, the percentage of successful redirection was higher after than before training (see Figure 8, top panel). However, this difference was significant only for the bottom-left corner (t(11) = 2.56, p < 0.05) and the top-right corner (t(11) = 2.75, p < 0.05). It also barely failed to reach significance for the top-left corner (p=0.06). For each corner, we also compared the redirection threshold measured in the pre and post sessions. Here again, all pre-post comparisons were performed using standard paired t-tests because the difference vectors were normally distributed. For three of the four corners, namely bottom-right (t(11) = -3.48, p < 0.01), top-left (t(11) = -2.99, p < 0.05) and top-right (t(11) = -3.55, p < 0.01), the redirection threshold was significantly lower after than before training (see Figure 8, bottom panel). In other words, after training, the participants needed less time to be able to redirect their shot. However, this difference was marginal and non-significant when the target was the bottom-left corner.

Figure 8: Redirection choice (in percent) for the pre and post sessions. The arrow values correspond to the proportion of percentage of slap shots redirected into a state.

Figure 8 gives an overview of the corners 'selected' to redirect the slap shot (in percent), both before and after the training. Because the observed improvement after training was not the same for the four corners, both the percentage of successful redirection and the redirection threshold were entered in a one-way repeated measures ANOVA with which we compared the performance measured for the four corners in the post session. As before, normality was assessed running the Shapiro-Wilk test on the residuals, and p-values were Huynh Feldt corrected for sphericity when required. Regarding the percentage of successful redirection, there was a main effect of the target corner (F(3,33) = 6.8, p< 0.01, n2g = 0.12), redirection being significantly more successful when the initial target was the top-left or the top-right corner than when it was the bottom-right corner. Very similar results were observed for the redirection threshold. Specifically, there was a main effect of the target corner (F(3,33) = 8.85, p< 0.001, n2g = 0.28), redirection threshold being significantly lower when the initial target was the top-left corner than when it was the bottom-left or the bottom-right corner. The threshold was also significantly lower for the top-right corner than for the bottom-right corner.

Discussion and conclusion

We designed a new system based on human-avatar interaction in Virtual Environment to assess and train slap shot performance in hockey. The slap shot is a fast and complex movement, and during its execution, the shooter can be 'constrained' to modify his shot online if the initial conditions change, as for instance if the goalkeeper moves to block the intended shot area. In line with this, our system was designed to assess and improve the sensorimotor skills required to modify the shot during its execution. Our system works in real-time and relies on a learning algorithm able to compute the remaining time before stick-puck contact at each step of the slap shot movement, so that for each shot, the blocking moves of the virtual goalkeeper are triggered based on this computed time. Our system is sufficiently accurate to be combined with psychophysical algorithms, so that the reactions of the virtual goalkeeper are adapted from one shot to the other via a staircase procedure. From a graphical point of view, our system consists of a virtual environment similar to the real hockey environment, e.g., rendering and animation of the goalkeeper, stadium, dim light, light projectors. Note that the rendering and animation of the goalkeeper required to adapt the movements of an avatar wearing loose-fitting equipment. We then validated the effectiveness of our system with two main objectives: 1. to assess the initial performance of the players, notably to identify whether their ability to redirect a shot depended on the target corner of the goal, and 2. to quantify the improvement resulting from the training. Our results confirmed the effectiveness of our system at both levels.

Figure 9: Redirection threshold for the four target corners before (top panels) and after training with our simulator (bottom panels) for five participants. The figure illustrates individual differences in redirection abilities.

Regarding performance detection and assessment, our system allowed us to identify the initial (i.e., pre training) rate of successful redirection of the hockey players who participated in the study. It also allowed us to measure the minimum time they required to modify their slap shot movement during its execution in reaction to the blocking moves of the virtual goalkeeper. On average, our results indicate that the rate of successful redirection and the redirection threshold are similar for the four target corners. In other words, before training, the average redirection performance of the players was not affected by the corner of the goal they intended to shoot at. However, as indicated in Figure 9, not all players had the same performance, and different patterns could be observed for the different players. In addition, our analysis of players' preferential redirections for each target corner (Figure 9, left panel) indicates that, in general, participants tended to redirect along a vertical axis (more particularly to the bottom) rather than along a horizontal axis. We can also observe that shooters tended to redirect their shots towards the bottom rather than the top corners of the goal. We believe that these results have real applications for hockey coaches and players. In particular, our system would allow coaches to identify the sensorimotor abilities of each player, and notably to identify his strengths and weaknesses. It would also give the coaches the possibility to identify the 'easiest' redirection patterns, e.g., which target corner constitutes the best initial choice because it provides the best possibility to redirect the shot afterwards.

Regarding training and performance improvement, our results clearly indicate that our system allowed the players to improve the sensorimotor skills required to redirect a slap shot under a strong time constraint. Specifically, training with our system gave rise to a significant improvement of both the rate of successful redirection and the redirection threshold, i.e., it decreased the minimum time required to react to a perturbation and modify the shot during its execution. This is a very important result, especially considering that the training period was relatively short, namely 3 weeks and a total of 3 hours of training. In addition, we should mention that such an improvement would be very hard to reach using 'classical' training. In particular, without a system such as ours, it would be impossible to precisely and timely trigger the movements of the goalkeeper based on the movements of the shooter.

When analyzing separately the results obtained for the four corners, one can see that the percentage of successful redirection improved for all corners, even though the improvement was somehow less marked (and non-significant) for the bottom-right corner. As for the redirection threshold it improved for all corners but the bottom-left corner. In this latter case, it is important to note that the redirection threshold was already relatively low before training, which might have contributed to limit training-evoked improvement. However, the average redirection threshold before training for the top-left corner was even lower, which did not prevent players to significantly improve their performance with training. More generally, both the rate of successful redirection and the redirection threshold tended to improve more for the two top corners than for the two bottom corners (see the results of the one-way ANOVAs performed on values measured in the post training session). In other words, training was more beneficial to redirection performance when the initial target was one the two top corners of the goal. Interestingly, training altered the pattern of redirection choice (see the difference between the left and right panel of Figure 8). In particular, after training, the rate of redirection towards the bottom-left corner increased, whereas redirections towards the two top corners decreased slightly. It should also be noted that in some cases, players significantly modified their redirection strategy by eliminating difficult redirections as diagonals, and by favoring other transitions, notably to the bottom left corner. Taken together, these results indicate that top-to-bottom redirection is easier than bottom-to-top redirection, and that for left-handed shooters, right-to-left redirection is easier than left-to-right redirection. In future studies, it would obviously be very interesting to test the performance of right-handed shooters. One could expect similar results regarding top-bottom redirection, and probably an inverse pattern regarding left-right redirection. Based on our results, we could imagine tailored training sessions based on the requirements / weaknesses of each player (and their handedness?), these training sessions being based on successive iterations of assessment-training-assessment-training...

From a more fundamental point of view, our study allowed us to identify the minimum time required to modify online (i.e., during movement execution) a complex and fast movement as the slap shot. The ability to modify a movement during its execution has been extensively studied in the motor control literature (see (Prablanc, 1992; Sarlegna and Mutha, 2014) for reviews on the topic). However, almost all studies that have been performed in this domain focused on 'simple' reaching movements. These studies have notably shown that automatic (Day and Lyon, 2000; Pisella et al., 2000) and fast modifications of the hand trajectory usually take place as soon as 280 ms after the perturbation (Day and Brown, 2001; Prablanc, 1992; Reichenbach et al., 2009; Soechting and Lacquaniti, 1983). Much less is known regarding humans' ability to modify online more complex movements. Few studies investigated the ability to redirect a penalty kick (Mallo et al., 2012; Morya et al., 2003; Van der Kamp and Masters, 2008; Van der Kamp, 2006; Wood and Wilson, 2010), and these authors found redirection thresholds in the 400-600 ms range. Using a staircase procedure, here we found redirection thresholds ranging from 364 to 466 ms after the perturbation, i.e., after the virtual goalkeeper initiated his blocking move. As expected, this threshold is higher than those observed for simple reaching movements, which indicates that more time is required to be able to successfully redirect a slap shot than to modify the trajectory of a reaching movement. However, the thresholds measured in the current study are lower than those reported for redirecting a penalty kick. This difference likely results from the fact that a slap shot mostly involves the upper body and fewer joints than the penalty kick. Note however that the nature of the stimulus used in our study, namely a realistic avatar performing human-like movements, is very different from that used in the studies on the penalty kick, in which the stimuli consisted of lights sources. This difference might also have affected the response times to the visual stimulus, as several studies have evidenced the 'specificity' of the processing of biological motion in the human brain (see (Thompson and Parasuraman, 2012)).

Another important contribution of our work to the field of motor control is that to our knowledge, our study is the first one to show that the sensorimotor 'skills' required to modify a movement during its execution can be trained and improved. By skills, we mean here the sensorimotor loops that underlie the rapid modifications of the motor output based on the processing of visual information. On average, the redirection threshold was 68 ms (397 ms – 329 ms) lower after than before training. And for some target corners, the difference reached 90 ms, which constitutes a large improvement. Indeed, if one subtracts the 80 ms of 'pure' visual (about 60-70 ms, (Bullier, 2001; Odom et al., 2004)) and motor processing (about 10-15 ms, (Bawa et al., 2004; Di Lazzaro et al., 2004)), the threshold measured after training represents a 21 percent improvement relative to the threshold measured before training (i.e., 68 ms retative to 329 ms), which is quite substantial. Of course, such an improvement is very unlikely to generalize to much simpler and more highly automatic movements such as reaching movements. Additional studies focusing on the attention/perception aspects as well as on movement production might be useful to identify the sources of the improvements observed here. Such studies might notably rely on eye tracking technology and specific kinematic analyses.

An upgrade of our system would consist in removing the intention phase to completely adapt the AI to the participant. The goalkeeper would then detect the player's intention in real time through an artificial intelligence and then try to block his shot. From a more general point of view, considering the complexity of the movement involved and the results obtained here, we believe that our model could be adapted to study other similar actions (other attacker vs. opponent situations such as found in handball or soccer) or to new 'anticipation' situations with other stimuli, and thereby constitute a very useful training tool. Finally, it would be relevant in future work to conduct study on user experience to better understand the motivational aspects and immersion perception.

NOTES

References

  • Bawa P., Hamm J.D., Dhillon P., Gross P.A., 2004. Bilateral responses of upper limb muscles to transcranial magnetic stimulation in human subjects. Experimental Brain Research, 158(3), 385–390. doi:10.1007/s00221-004-2031-x
  • Bideau B., Kulpa R., Vignais N., Brault S., Multon F., Craig C., 2010. Using virtual reality to analyze sports performance. IEEE Comput Graph Appl, 30(2), 14–21. doi:10.1109/MCG.2009.134
  • Bideau B., Multon F., Kulpa R., Fradet L., Arnaldi B., Delamarche P., 2004. Using virtual reality to analyze links between handball thrower kinematics and goalkeeper’s reactions. Neuroscience Letters, 372(1–2), 119–122. doi:10.1016/j.neulet.2004.09.023
  • Bingham G.P., Schmidt R.C., Zaal F.T.J.M., 1999. Visual perception of the relative phasing of human limb movements. Perception and Psychophysics, 61(2), 246–258. doi:10.3758/BF03206886
  • Bolte B., Zeidler F., Bruder G., Steinicke F., Hinrichs K.H., Fischer L., Schorer J., 2010. A Virtual Reality Handball Goalkeeper Analysis System. Proceedings of the Joint Virtual Reality Conference of EuroVR - EGVE - VEC, 1–2. http://viscg.uni-muenster.de/publications/2010/BZBSHFS10/bolte_jvrc10.pdf
  • Brault S., Bideau B., Kulpa R., Craig C.M., 2012. Detecting deception in movement: The case of the side-step in rugby. PLoS ONE, 7(6), 1–13. doi:10.1371/journal.pone.0037494
  • Bullier J., 2001. Integrated model of visual processing. Brain Research Reviews, 36(2–3), 96–107. doi:10.1016/S0165-0173(01)00085-6
  • Covaci A., Olivier A.H., Multon F., 2014. Third person view and guidance for more natural motor behaviour in immersive basketball playing. Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology - VRST ’14, 55–64. doi:10.1145/2671015.2671023
  • Cressie N., 1988. Spatial prediction and ordinary kriging. Mathematical Geology, 20(4), 405–421.
  • Day B.L. and Lyon I.N., 2000. Voluntary modification of automatic arm movements evoked by motion of a visual target. Experimental Brain Research, 130(2), 159–168. http://www.ncbi.nlm.nih.gov/pubmed/10672469
  • Day B.L. and Brown P., 2001. Evidence for subcortical involvement in the visual control of human reaching. Brain, 124(9), 1832–1840. doi:10.1093/brain/124.9.1832
  • Di Lazzaro V., Oliviero A., Pilato F., Saturno E., Dileone M., Meglio M., Cioni B., Papacci F., Tonali P.A., Rothwell J.C., 2004. Comparison of descending volleys evoked by transcranial and epidural motor cortex stimulation in a conscious patient with bulbar pain. Clinical Neurophysiology, 115(4), 834–838. doi:10.1016/j.clinph.2003.11.026
  • Fu T., 2011. Engineering Applications of Artificial Intelligence A review on time series data mining. Engineering Applications of Artificial Intelligence, 24(1), 164–181. doi:10.1016/j.engappai.2010.09.007
  • Hoerner E.F., 1989. The dynamic role played by the ice hockey stick. In Safety in ice hockey. ASTM International.
  • Kerwin D.G. and Bray K., 2006. Measuring and modelling the goalkeeper’s diving envelope in a penalty kick. In The Engineering of sport 6 (pp. 321–326). Springer.
  • Kuhn W., 1988. Penalty-kick strategies for shooters and goalkeepers. Science and Football, 489–492.
  • Le Naour T., Courty N., Gibet S., 2013. Spatiotemporal coupling with the 3D+t motion Laplacian. Computer Animation and Virtual Worlds, 24(3–4), 419–428. doi:10.1002/cav.1518
  • Lomond K.V., Turcotte, R.A., Pearsall, D.J., 2007. Three-dimensional analysis of blade contact in an ice hockey slap shot, in relation to player skill. Sports Engineering, 10(2), 87–100. doi:10.1007/BF02844206
  • Mallo J., Frutos P.G., Juárez D., Navarro E., 2012. Effect of positioning on the accuracy of decision making of association football top-class referees and assistant referees during competitive matches. Journal of Sports Sciences, 30(13), 1437–1445. doi:10.1080/02640414.2012.711485
  • Metoyer R.A and Hodgins J.K., 2000. Animating Athletic Motion Planning By Example. Proceedings of Graphics Interface, 61–68.
  • Miles H.C., Pop S.R., Watt S.J., Lawrence G.P., John N.W., 2012. A review of virtual environments for training in ball sports. Computers and Graphics (Pergamon), 36(6), 714–726. doi:10.1016/j.cag.2012.04.007
  • Moazzen S.M.A. and Ahmadi A., 2017. 3D simulation of tennis game using learning algorithms by application of virtual reality. 2017 25th Iranian Conference on Electrical Engineering, ICEE 2017, 1527–1531. doi:10.1109/IranianCEE.2017.7985285
  • Morya E., Ranvaud R., Pinheiro W.M., 2003. Dynamics of visual feedback in a laboratory simulation of a penalty kick. 87–95. doi:10.1080/0264041031000070840
  • Odom J.V., Bach M., Barber C., Brigell M., Marmor M.F., Tormene A.P., Holder G.E., Vaegan A., 2004. Visual evoked potentials standard (2004). Documenta Ophthalmologica, 108(2), 115–123. doi:10.1023/B:DOOP.0000036790.67234.22
  • Pisella L., Gréa H., Tilikete C., Vighetto A., Desmurget M., Rode G., Boisson D., Rossetti Y., 2000. An ‘automatic pilot’ for the hand in human posterior parietal cortex: toward reinterpreting optic ataxia. Nature Neuroscience, 3, 729. doi:10.1038/76694
  • Prablanc M., 1992. Automatic Control During Hand Reaching at Undetected Two-Dimensional Target Displacements. 67(2).
  • Reichenbach A., Thielscher A., Peer A., Heinrich H.B., 2009. Seeing the hand while reaching speeds up on-line responses to a sudden change in target position. 19, 4605–4616. doi:10.1113/jphysiol.2009.176362
  • Rusdorf S., Brunnett G., Lorenz M., Winkler T., 2007. Real-time interaction with a humanoid avatar in an immersive table tennis simulation. IEEE Transactions on Visualization and Computer Graphics, 13(1), 15–25. doi:10.1109/TVCG.2007.18
  • Sarlegna F., Blouin J., Vercher C.B.J., Gauthier G.M., 2003. Target and hand position information in the online control of goal-directed arm movements. 524–535. doi:10.1007/s00221-003-1504-7
  • Sarlegna F.R. and Mutha P.K., 2014. The influence of visual target information on the online control of movements. Vision Research, 1–11. doi:10.1016/j.visres.2014.07.001
  • Schmidt R.A. and Lee T.D., 2005. Motor control and learning: A behavioral emphasis, 4th ed. In Motor control and learning: A behavioral emphasis, 4th ed. Human Kinetics.
  • Soechting J.F. and Lacquaniti F., 1983. Modification of trajectory of a pointing movement in response to a change in target location. Journal of Neurophysiology, 49(2), 548–564. doi:10.1152/jn.1983.49.2.548
  • Thompson J. and Parasuraman R., 2012. Attention, biological motion, and action recognition. NeuroImage, 59(1), 4–13. doi:doi:10.1016/j.neuroimage.2011.05.044
  • Van der Kamp J. and Masters R.S., 2008. The human Muller-Lyer illusion in goalkeeping. 37, 951–955. doi:10.1068/p6010
  • Van der Kamp J., 2006. A field simulation study of the effectiveness of penalty kick strategies in soccer: Late alterations of kick direction increase errors and reduce accuracy. Journal of Sports Sciences, 24(5), 467–477. doi:10.1080/02640410500190841
  • Vignais N., Bideau B., Craig C., Brault S., Multon F., Delamarche P., Kulpa R., 2009. Does the level of graphical detail of a virtual handball thrower influence a goalkeeper’s motor response? Journal of Sports Science and Medicine, 8(4), 501–508.
  • Wimshurst Z.L., Sowden P.T., Cardinale M., 2012. Visual skills and playing positions of Olympic hockey players. Perceptual and Motor Skills, 44(0).
  • Wood G. and Wilson M.R., 2010. A moving goalkeeper distracts penalty takers and impairs shooting accuracy. Journal of Sports Sciences, 28(9), 937–946.
  • Wright W.G., 2014. Using virtual reality to augment perception, enhance sensorimotor adaptation, and change our minds. Frontiers in Systems Neuroscience, 8(April), 1–6. doi:10.3389/fnsys.2014.00056
  • Zaal F.T.J.M. and Michaels C.F., 2003. The Information for Catching Fly Balls: Judging and Intercepting Virtual Balls in a CAVE. Journal of Experimental Psychology: Human Perception and Performance, 29(3), 537–555. doi:10.1037/0096-1523.29.3.537
  • Zhang L., Brunnett G., Petri K., Danneberg M., Masik S., Bandow N., Witte K., 2018. KaraKter: An autonomously interacting Karate Kumite character for VR-based training and research. Computers and Graphics (Pergamon), 72, 59–69. doi:10.1016/j.cag.2018.01.008

Authors


Thibaut Le Naour

Affiliation : Department of Neuroscience and Movement Science, University of Fribourg
Country : Germany

thibaut.le-naour@univ-ubs.fr

Ludovic Hayoz

Affiliation : Department of Neuroscience and Movement Science, University of Fribourg
Country : Germany


Jean-Pierre Bresciani

Affiliation : Department of Neuroscience and Movement Science, University of Fribourg
Country : Germany

Attachments

No supporting information for this article

Article statistics

Views: 172

Downloads

PDF: 36

XML: 1

Citations