Abstract

In large-scale maritime maneuvering simulators, the multi-channel systems are usually used to tile multiple projectors on a cylindrical display to create a large virtual environment, which brings more realistic immersion to the users. In the non-planar multi-projector display system, to finish the work of geometric and photometric corrections, there exist the problems of difficult projector calibration, more manual intervention and poor adaptability. In order to solve these problems, a multi-projector calibration based on virtual viewing space is proposed in this paper. First, a virtual environment of multiple projectors is created, and by using the structured light and re-projection model on a cylindrical wall, the calibration of internal and external parameters of a projector are realized separately. Finally, the geometric correction is completed by the mapping relationship between the target images and the calibrated re-projected images. This method is free of limitation by the angle of view of a single camera and can calibrate any of the projectors in the multi-channel systems. The example results show that the projector’s parameters can be estimated more accurately, and the efficiency and accuracy of geometric correction can be improved. Good results are achieved when applying this method in the maritime maneuvering simulator system.

Introduction

High-resolution and wide-field displays create an immersive visual experience that facilitates applications such as remote collaboration, scientific visualization, and human-computer interaction (Yicheng and Yong, 2013; Yiyu et al., 2013). Multi-projector display walls are an economical and effective way to build large seamless displays today. In the maritime maneuvering simulator, the multi-projector display wall often adopts a 180-degree or even 360-degree cylindrically non-planar display, and the multi-channel visual simulation system is used to project a completely virtual environment image onto the curved display wall, which gives the user a more realistic immersion (Yicheng and Yong, 2013). In this system, geometric and photometric corrections are the key technologies, and geometric correction is the premise and foundation of the display system, which directly affects the display results.

Images projected directly from the projector to the curved surface are distorted. In order to construct a multi-channel visual simulation system to display an undistorted image on a curved display wall, it is necessary to pre-deform the projected image, which is called a geometric correction. The reason why the multi-channel system needs geometric correction is due to the difference between the parameters of the projector and the view frustum of a virtual camera. The key of geometric correction is to establish an accurate correspondence between the projector image and the display wall (Chao et al., 2013; Fang, 2013).

This paper proposes a multi-projector calibration based on virtual viewing space (VVS) for the cylindrical display wall in a maritime simulator. The characteristics are as follows:

(1) The projection and re-projection of the actual projector and virtual camera are modeled, and the mapping relationship between VVS and the actual viewing space (AVS) is realized.

(2) In the calibration process, the actual projector is modeled by the pinhole model, the internal parameters of the projector are calibrated by the structured light method, and the external parameters are calibrated by the re-projected image on VVS.

(3) The geometric correction is completed by the mapping relationship between the calibrated re-projected image of an actual projector and the target image of a virtual camera in the multi-channel visual simulation system.

Related word

At present, geometric correction technology often uses a camera as an auxiliary device and cooperates with the projector to perform it. Raskar et al. (1999) proposed a planar geometric correction system using a single camera to achieve geometric correction of a multi-channel system. However, it used a single camera to obtain a limited range of projected images, which limits, to some extent, the scalability of this system. Bhasker et al. (2007) achieved planar projection distortion correction (i.e. radial and tangential distortions), using rational Bezier patches. Chen et al. (2002) used multiple cameras on planar displays to achieve a homography-tree based registration across multiple projectors, but this increased the complexity of the system and generated cumulative geometric correction errors.

In non-planar displays, geometric correction is relatively more complicated, and the correction process is more complicated. Brown and Seales (2002) used a single camera to acquire equidistant feature points projected by each projector to establish a one-to-one correspondence between projector and camera spatial feature points, organized these feature points into a triangular mesh, and then used texture deformation to achieve geometric correction of arbitrarily curved displays. Raskar et al. (2002, 2004, 2005) used the curved displays as a parametric model of quadric surfaces (such as domes and cylinders) to achieve geometric correction of them. Harville et al. (2006) and Sun et al. (2008) pasted artificial marker points on the upper and lower edges of the curved display and used the two-dimensional linear mapping to establish the relationship between the projector and the display. However, since there is no spatial three-dimensional information, it cannot be used for viewpoint-related applications. Using a single camera, Sajadi et al. (2009) calibrated the internal and external parameters of the projector by extracting the upper and lower edges of the curved display to complete the non-planar geometric correction. The limitation is that it can only be used on smooth vertical surfaces, and the curve display cannot have sharply changing edges and corners. Later, Sajadi et al. (2010, 2011) improved its algorithm to extend it to all vertical extension surfaces. Xiao Chao et al. (2013) used the structured light to calculate the mapping relationship between each projector and the display wall to achieve nonlinear geometric correction. Fang (2013) calibrated the projector in an interactive way, and then combines the channel view frustum to realize the multi-channel geometric correction. Dong et al. (2015) used structured light to realize the automatic splicing of the cylindrical projection image, and corrected the errors caused by manufacturing and arranging of the display. However, limited by the angle of view of a single camera, it can only calibrate 4 to 6 projectors in a projection display.

The above methods show that the mapping relationship between the projectors and the display is the key of the geometric correction in the multi-channel systems. Compared with the actual application, the following problems still exist in the existing work:

(1) To the lack of projection matrix of the projector, the geometric correction is simply performed by using rational Bezier patches to require a large amount of manual intervention (commercial software generally adopts this method).

(2) In the automatic calibration of the projector, the reference point is often placed on the display wall to increase the complexity.

(3) When calibrating multi-projection, the number of calibration projectors is often limited by the single-camera field of view.

The new algorithm proposed in this paper constructs the VVS and moves the geometric correction process of AVS into VVS. It can calibrate the projectors separately and calculate the mapping relationship between the viewing image and the target image to complete the multi-channel geometric correction. It can improve the efficiency and application range of multi-channel geometry correction.

Virtual viewing space

In a multi-channel visual simulation system, it is often necessary to display a large viewing angle scene on a large screen to provide an immersive environment. This environment is mainly composed of a physical screen, the projectors, a frame buffer image, and a re-projection image. Among them, a frame buffer image refers to the image formed by a virtual camera in the virtual environment, and a re-projected image refers to the image formed by re-projecting a frame buffer image onto the screen.

Figure 1: A multi-channel visual simulation system

As shown in Figure1, the cylinder simulation system maps the multi-channel frame buffer images centered on a certain viewpoint in the virtual scene on the curved wall, so that people can view the continuous virtual pictures. However, since the viewpoint reference center is different from the projection centers of actual projectors, eventually, the viewing images in the channels and the overlapping area of the channels will be geometrically deformed.

This paper establishes a virtual viewing space that is consistent with the actual one, including establishing projection and re-projection models for the actual projector and virtual camera, and realizes the mapping of VVS to AVS for geometric correction in multi-channel visual simulation systems.

Projection model

According to computer vision, the camera can be modeled by the pinhole. The world coordinate system is converted into the camera coordinate, and the following transformations are to be completed (Bradski and Kaebler, 2009).

xcyczc=R T xwywzw1(1)
zcuv1=fuλu00fvv0001Kxcyczc (2)

R is 3x3 rotation matrix, and T is 3 x 1 translation vector. fu=f/dx, fv=f/dy, λ=λ'f, f is the focal length of the camera, while dx and dy are the physical size of the horizontal and vertical pixels, with λ as the tilt factor. [R, T] is an external parameter matrix, which shows the transformation of a world coordinate system with the origin at the camera center, and K is the internal parameter matrix, while u0 , v0is the coordinates of the main point. A camera projection matrix is the KR, T.

In VVS, to simplify the cropping operation of the scene,the internal parameter matrix of the virtual camera adopts a normalized projection matrix KC, that is, the tilt factor λ'=0, and u0 , v0 is (0 , 0). The projection model of an actual projector can be described as the inverse projection of the camera, and be modeled using a pinhole, which also is composed of an internal and external parameter matrix (Moreno and Taubin, 2012).

As shown in Figure 2, the projection model of the virtual camera refers to the view frustum of the vertical field of view 2α with the viewing center OC. The frame buffer image plane of ACBCCCDC is obtained by the transformation of perspective projection. Similarly, a projector can be described as the inverse of a camera. In VVS, the projection matrix of the projector is calculated according to the internal and external parameters of the actual one, and then the frame buffer image is projected into the three-dimensional space through its center, and the projection model of the actual projector refers to the view frustum formed at this time.

Figure2: Projection and re-projection models of a virtual camera and an actual projector

Re-projection model

In perspective projection, it is called a re-projection transformation, as the original image is projected onto another surface. Geometric correction of the projected image is required for the re-projection. This paper will continue to establish re-projection models based on the display in VVS, including the virtual camera and the actual projector re-projection models.

As shown in Figure 2, the virtual camera re-projection model means that the frame buffer image ACBCCCDC by virtual camera re-projects onto the screen EFGH with the view frustum of the viewing center OC, and finally forms the target image ECFCGCHC. Similarly, the re-projection model of projector means that the frame buffer image ACBCCCDC by actual projector re-projects onto the screen EFGH with the view frustum of the projection center, and finally forms the viewing image EPFPGPHP.

In multi-channel systems, the re-projection image of a virtual camera is the target image on the display formed by ideal viewing point and projection parameters, which is the result that people want to view from the screen. Due to the different parameters of the internal and external projection matrix, the re-projected image of an actual projector (the viewing image) is obviously deviated from the target image and required geometric correction. In Figure 2, the two re-projected images will form an intersection area. The purpose of geometric correction in multi-channel systems is that the viewing image shows the corresponding content of the target image in the intersection area but does not include that outside the intersection area. The geometric correction is performed by the re-projection model to make the projector projecting the image that people wants.

Virtual viewing space calibration

VVS is a mapping of AVS. It can correctly establish the above projection and re-projection models of the actual projector and virtual camera for geometric correction in multi-channel systems.

Projection screen construction

In actual space, projection screens tend to have regular geometric shapes and are located at corresponding locations in actual space. According to the geometry of the screen, the virtual scene viewing point is used as the center point to construct the shape and position of the screen in VVS.

In large-scale maritime simulators, the projection screen is often cylindrical. As shown in Figure1, through actual measurements, the width (w), height (h) and depth (d) of the screen can be obtained. By equations (3) and (4), the cylinder radius (R) and the angle (θ) of view can be calculated. And the screen can be constructed in the virtual viewing space with the three parameters R, θ and h.

R=4d2+w2/8d   3        θ=2arcsin(4wd/4d2+w2)  (4)

Virtual camera calibration

The internal and external parameters of the virtual camera can be obtained from the virtual camera position, pose and perspective matrix, which is known in the multi-channel visual simulation system. Through the mapping relationship between VVS and AVS, the geometric center of VVS is used as the virtual camera viewing point, and this point is also the re-projection center point of the virtual camera.

Actual projector calibration

In VVS, the key is to calibrate the actual projector, establish the mapping between the projector and the wall, and the mapping between a viewing image and a target image. There are fewer methods for projector calibration, which has different shortcomings of lack of accuracy, poor anti-interference ability, or difficulty in calibration (Gao et al., 2008; Martynov et al., 2011; Drar´eni et al., 2012). In this paper, VVS is used for multi-projector calibration and the projector is modeled according to Section 3.1, including internal parameters and external parameters.

Internal Parameter Calibration

Figure 3: A projector and a camera pixel matching based on a structured light

The projector is modeled using a pinhole and can be calibrated using a camera calibration method (Zhang, 2000). However, since the projector can't take photos, we use the projector to project the structured light to the checkerboard at different positions and calibrate the projector through the structured light images obtained by the camera (Moreno and Taubin, 2012). As shown in Figure 3, by decomposing a photograph with a structured light sequence, the correspondence between the points in the camera photo and in the plane of the projector is found. As shown in equation (5), a local homography H^ is calculated for each inner corner of the checkerboard in the photo. Then use equation (7) to convert the inner corner v(x, y, 0) of the checkerboard from p¯ in the camera imaging plane to q¯ in the projector imaging plane.

H^=argminHpq-Hp2   5


H  R3x3, p=u, v, 1T, q=i, j, 1T  (6)


q-=H^.p-  (7)

Therefore, the correspondence between the inner corner points in the checkerboard and in the plane of the projector is obtained. Using the formulas (1) and (2), the internal parameter matrix KP in the calibration projector can be obtained. Then, we get the rotation matrix R and translation vector T of the projector, which is relative to the checkerboard coordinate system. We need to further determine the external parameter matrix of the calibration projector, which is relative to the projection screen.

External parameter calibration

Figure 4: Calibrating the external parameters of projector

We obtained the parameter matrix KP in the projector, including the projector's center point (u0, v0), fu and fv. Since the width (w) and height (v) of the projected image are known, we can calculate the vertical field of view by equation (8). As shown in Figure 4, we can decide the view frustum of the projector defined by its center and five planes (top, bottom, left, right and the image plane).

cotVfov2=(fu+fv)/2h/2
(8)

After obtaining the view frustum of the projector, in VVS, the external parameters of the projector are calculated by using the re-projected image on the screen.

In Figure4, first, the re-projected grid image EPFPGPHP of the projector taken by the camera in AVS is mapped to the corresponding position on the virtual curved display wall in VVS. Then, in VVS, we select the 3D curve EPFP and GPHP of the re-projected grid image and fit a plane to the samples of EPFP and GPHP in a linear least squares sense to estimate T and B plane. Then we find the intersection of T and B to find the line L0, and the center of the projector is on this line. Finally, the center of the projector model in VVS moves on the line L0, so that the image formed by the view frustum of the projector model on the virtual curved display wall coincides with the re-projected image EPFPGPHP. At this time, the pose of the projector in VVS is consistent with the pose of the projector in AVS. From the pose, the external parameters of the actual projector are obtained.

It can be seen that the use of the virtual viewing space for projector calibration is not limited by the camera's field of view angle, and can realize the calibration of any projector in the actual space individually, and has the characteristics of practicability and versatility.

Geometric Correction Steps

To sum up, the geometric correction based on VVS is divided into the following steps:

(1) By measuring the dimensions of the cylinder, the position and shape of the cylinder are modeled in VVS.

(2) The camera is used to obtain the checkerboard image with structured light, and the mapping between the pixels of the actual projector and the corner points of the checkerboard is established. The internal parameters of the calibration projector are determined, and the view frustum of the projector is constructed in VVS.

(3) The re-projected grid image of the calibration projector in AVS is mapped to the corresponding position of the display wall in VVS.

(4) Comparing the re-projection images between in AVS and in VVS, the external parameter matrix of the calibration projector is determined when the two are matched.

(5) Comparing the re-projection image of the projector on the wall and the target image of the virtual camera, the mapping between the two images is established, and the geometric correction of the multi-channel system is realized by the mapping.

In addition, we manually use rational Bezier patches for further adjustment to eliminate the errors between AVS and VVS include screen model, projection and re-projection model, and achieve more precise geometric correction.

Results and applications

We use the maritime simulator with 5 channels to do the experiment. The projection screen is a cylinder wall with a radius (R) of 8.42 meters, a height (h) of 3.0 meters, the horizontal angle (θ) of 180 degrees.

This experiment comprises five SECO-APD700L projectors and a Canon EOS 6D camera. Projectors’ resolution is 1024*768, and the camera’s resolution is 2736*1824. The internal parameters matrix of the projector is calibrated by using Moreno’s method (Moreno and Taubin, 2012).

Figure 5: The experimental diagram for calibrating internal parameters of projector

(a) a completely illuminated image, (b) a projected gray code, (c) the pixels to projector column, (d) the pixels to the projector row

Single channel calibration

We use the structured light to calibrate the internal parameters of a projector separately. The following is an example of calibrating the intermediate channel projector of this system. As shown in Figure 5 (a), the intermediate channel projector projects a checkerboard illuminated by white light, and the obtained image is used for calibration of the camera. In Figure 5 (b), the intermediate channel projector sequentially projects the code of the gray code, and the corresponding checkerboard picture is obtained by the camera with the structure of the gray code. It needs to generate 210=1024 different codes to meet the projector resolution requirements. According to the literature [19], a total of 40 pictures are required. These pictures are then decoded to obtain the distribution of the pixels of the projected image. As shown in Figure 5(c) and (d), respectively, the pixel vertical and horizontal coordinates of the projected image are distributed, and the pixel coordinates of the same color are equal. At the same time, use formula (5) to convert all the corner points in the checkerboard into projector coordinates according to Zhang’s method (Zhang, 2000).

αx=tan-1(Ox-Ixf) αy=tan-1(Oy-Iyf)  (9)

Table 1 shows the results of internal parameters for the projector, which are the focal length (fu, fv) and the optical center (OX, OY). In this paper, (fu, fv)/2 is taken as the focal length fof the projector to calculate the parameters of its view frustum, where its vertical field angle is calculated according to formula (8), and its optical center shift angle is calculate as shown in equation (9). (IX, IY) is the center of the projected image.

Table 1 Results of calibration of internal parameters


Name

Intermediate channel projector

Calibration result

Focal length (pixel)

(761.65,769.15)

optical center (pixel)

(523.63,477.41)

Conversion result

Vertical field of view (degree)

53.29

Optical center shift (degree)

(0.87,6.93)

Figure 6: the re-projection image of projector in AVS mapped to the virtual display wall in VVS

(a) a cross checkered pattern, (b) Re-projected image, (c) Mapping process, (d) Mapping results

When calibrating the external parameters of the projector, in order to facilitate the matching of the re-projection images in VVS and AVS, as shown in Figure 6(a), the projected image adopts a cross checkered pattern. This pattern can also serve as the remedy for the situation when the projected image cannot be fully projected onto the cylinder display wall, which leads to the failure of calibration (Sajadi and Majumder, 2010). We use the camera to obtain a re-projected image of the intermediate channel projector, as shown in Figure 6(b). Figure 6(c) shows the parameterization of the captured picture and maps the red frame area of Figure 6(b) to the corresponding position of the virtual screen. As shown in Figure 6(d), it completes the viewing image mapping in the two spaces, which provides a reference image for calibrating the external parameters of the actual projector.

After determining the reference image in VVS, as shown in Figure 7(a), first, the view frustum of the projector is determined by the internal parameters that have been calibrated. In Figure 7(b), the upper and lower parallel lines provided by the cross checkered pattern on the screen are used to find the line where the center of the projector is located. At the same time, the projector model simulates actual projector to project a cross checkered pattern onto a virtual screen. As shown in Figure 7(c), if the re-projected image of the projector deviates from the reference image, the position of the projector is manually fine-tuned to match the two. As shown in Figure 7(d), the final result with the re-projected image and the reference image matched determines the external parameters to realize the whole process of projector calibration. Table 2 shows the results of the calibration of the external parameters of the intermediate channel projector in the system.

Figure 7: Calibrate external parameters of actual projector

(a) the view frustum, (b) initial position, (c) the two images deviates, (d) the two images matched

Table 2 Results of calibration of external parameters


Name

Intermediate channel projector

Calibration result

Translation vector (m)

(-0.0482,1.0907,-3.6416)

Rotation vector (degree)

(0.0,0.0,-0.4)

Multi-Channel calibration

Figure 8: Five-channel projector calibration

(a) Five-channel re-projected image with cross checkered pattern (b) Re-projection image mapping to virtual display wall (c) The result of five-channel projector calibration (d) Grid display on the wall before calibration (e) Grid display on the wall after calibration (f) Grid display on the wall adjusting by rational Bezier patches

According to the calibration of the internal and external parameters, we have calibrated the five channel projectors separately. Figure 8 (a) is a five-channel viewing image with a cross checkered pattern taken by a camera. Figure 8 (b) is a mapping result of the five-channel viewing image from AVS to VVS, which corresponds to Figure 8(a). Figure 8(c) shows the results of the re-projection image of the five-channel projector in agreement with the reference image, thereby realizing the calibration of the internal and external parameters of the five-channel projector. Figure 8(d) shows the results of the five-channel projection grid to the cylinder display wall without calibration. It can be seen that the grids on the projection wall are obviously not aligned, and the grids are not completely displayed on the screen. Figure 8(e) shows the results of the five-channel projection grids on the screen after calibration. It can be seen that the grids on the projection screen are basically aligned, and the entire projection screen is fully utilized with the full grids. In Figure 8(e), the grids of the channel 1 and channel 2 overlapping area are almost perfectly aligned, but these of the channel 2 and channel 3 overlapping area are not well aligned. It is caused by the projection distortion of the projector, the errors of the parametric projection screen and the mapping re-projected image. These errors can lead to inaccurate geometric corrections. Further manual adjustments can be made by using rational Bezier patches for more precise geometric correction. As shown in Figure 8(f), after adjusting by rational Bezier patches, the overlapping area grids are completely aligned.

Figure 9: Five-channel geometric correction

(a) virtual camera re-projection images (b) the viewing image and target image mapping (c) Overall rendering of the five-channel

Figure 9(a) is the re-projected images of the five virtual cameras on the projection screen, referred to as a target image. Figure 9(b) shows the viewing images of the calibrated projectors and the target images of the virtual cameras and establishes the mapping between the cameras and projectors. The viewing image only displays the corresponding content of the target image in the intersection area, without displaying that outside the intersection area. The effect of forming a complete virtual scene is achieved, and finally, the geometric correction of the multi-channel visual simulation system is realized.

After the geometric correction, photometric correction is required to obtain a uniform brightness projection effect. This paper focuses on geometric correction and uses the method of literature (Yang et al., 2001) for photometric correction. Figure 9(c) is a whole rendering of the five-channel maritime simulator through geometry and photometric correction.

Applications

The visual simulation system with 5 channels is implemented by our method and applied to a maritime maneuvering simulator. It can project a completely virtual environment image onto the cylinder wall, which gives the user a more realistic immersion. In Figure 10(a), the system carries out ship maneuvering and navigation training through the rudder, the electronic chart, and the navigation radar in the virtual bridge.

In addition, we perform geometric correction based on VVS, and it can establish a 3D mapping between the projector and the display wall, and render the virtual scene according to the actual wall. There is a one-to-one correspondence between VVS and AVS. In Figure 10(b), Using this correspondence, we can place a bearing repeater in the center of the display wall, and directly measure the bearing of the virtual object for navigation training.

Figure10: Applications for maritime maneuvering simulator

(a) Ship training in the virtual bridge (b) A bearing repeater for navigation training

EVALUATION

We use the VVS method to perform geometric correction on the 5-channel the cylinder wall, and then analyze and discuss the accuracy and consumption time of the geometric correction.

Projector Calibration Accuracy

The accuracy of projector calibration is the key to geometric correction, directly affecting its accuracy and efficiency. In our system, the height of the projector's optical center is convenient for an actual measurement. The actual measured height value is compared with the projector calibration value, which can reflect the accuracy of our method for projector calibration. The results are shown in Table 3. The errors are below 2%.

Table 3 the heights of comparison results


Projector number

1

2

3

4

5

Measuring height (m)

1.059

1.068

1.072

1.053

1.046

Calibration height (m)

1.047

1.057

1.091

1.043

1.029

error

0.012

0.011

-0.019

0.010

0.017

ratio

1.1%

1.0%

-1.7%

0.9%

1.6%

Geometric Correction Efficiency

In the whole geometric correction process, it is divided into four main steps including internal parameter calibration of the projector, the construction of VVS, external parameter calibration of the projector, and rational Bezier patches fine-tuning. For this 5-channel geometric correction, each step takes approximately as shown in Table 4.

In Table 4, the 5-channel geometry correction takes about 57 minutes, which is mostly done automatically by the computer, and the manual time-consuming only accounts for about 26 % of the entire process.

Table 4 5-channel geometry correction time-consuming results


Step

Way

Time (minutes)

percent

internal parameter calibration

automatic

7*5(channel)=35

62%

construction of VVS,

automatic

5

8%

external parameter calibration

automatic + manual

7

12%

rational Bezier patches fine-tuning

manual

10

18%

Discussion

We use VVS for geometric correction. Compared with the real space, we can quickly construct the display wall and the projector model in VVS, which is convenient to establish a 3D mapping between the projector and the screen, and improve the efficiency of geometric correction and the scope of application.

In the mapping, we turn the wall of AVS into VVS, which makes the construction and control of the wall more convenient. It avoids marking points on the physical wall and reduces the complexity of the wall compared with the literature (Chao et al., 2013; Fang, 2013; Harville et al., 2006; Sun et al., 2008).

In the multi-projector calibration, the internal and external parameters of the projector are corrected in two steps, and the calibration of each projector is performed independently in the VVS. Compared with the literature (Sajadi et al., 2009; Li Dong et al., 2015), the number of calibration projectors is not limited by the angle of view of a single camera, and it is convenient to realize the calibration of any projector in a large viewing angle (including 360 degrees) scene. Compared with the literature (Chen et al., 2002), the method of this paper does not bring the cumulative errors of the projector calibration.

In operation mode, as shown in Table 3 and 4, the high calibration accuracy of projector can be obtained by the automatic, and the calibrated projection grids are basically aligned on the wall. Only a small amount of local further manual adjustments by rational Bezier patches are required for more precise geometric correction. Compared with the literature (Raskar et al., 2002; Raskar et al., 2004; Raskar et al., 2005), it has better adaptability to the projector and display wall nonlinearity.

In the applications, the VVS method establishes a 3D mapping between the projector and the screen. As shown in Figure10 (a) and (b), using this correspondence, it can be applied in the virtual environment related to viewpoints such as virtual driving and target positioning. Compared with the literature (Bhasker et al., 2007; Harville et al., 2006; Sun et al., 2008), it has a wider range of applications.

Summary

We propose multi-projector correction in VVS and apply to a 5-channel navigation simulator. This method more conveniently makes a 3D mapping between the calibrated projector and the display wall, which can bring many advantages. First, the calibration of each projector is performed independently, and the increase of the projectors does not bring additional correction complexity. Second, the number of projectors is not limited by the angle of view of a single camera. Finally, in terms of time consumption, the calibration process is basically automatic, and a linear superposition in the increase of the calibrated ones.

In addition, the algorithm proposed in this paper can be widely applied. As long as the display wall can be parameterized, it can be used for geometric correction, such as the planes, the domes, and so on.

However, the method also brings the errors when constructing VVS, including the parametric display error, the projector model error, the mapping re-projected image error, etc., which can lead to inaccurate geometric correction. How to reduce or avoid these errors is worthy of further study. In addition, after geometric correction, the second important challenge in a multi-channel system is a photometric correction, and how to achieve a better smooth transition of the brightness of the overlap zone will also be one of our future works.

Acknowledgements

This research is sponsored by large-scale surface warship integrated maneuvering simulation training system of Chinese Navy Staff.

Notes

References

  • Bhasker E., Juang R., Majumder A., 2007. Registration techniques for using imperfect and partially calibrated devices in planar multi-projector displays. IEEE Transactions on Visualization and Computer Graphics, 13(6) :1368-1375.
  • Bradski G. and Kaebler A., 2009. Learning OpenCV. Beijing: Tsinghua University Press. (in Chinese)
  • Brown M.S. and Seales W.B., 2002. A practical and flexible tiled display system. Proceedings of the 10th Pacific Conference on Computer Graphics and Applications. Los Alamitos: IEEE Computer Society Press.
  • Cai Y., Chia N.K.H., Thalmann D., Kee N.K., Zheng J., Thalmann N.M., 2013. Design and Development of a Virtual Dolphinarium for Children With Autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21(2):208-217.
  • Chao X., Hongyu Y., Haijun L., Yulong J., Xinsheng L., 2013. Geometric Calibration for Multi-projector Display System Based on Structured Light. Journal of Computer Aided Design & Computer Graphics, 25(6): 802-808. (in Chinese)
  • Chen H., Sukthankar R., Wallace G., 2002. Scalable Alignment of Large-Format Multi-Projector Displays Using Camera Homography Trees. IEEE Visualization Conference, Boston, United states.
  • Drar´eni J., Roy S., Sturm P., 2012. Methods for geometrical video projector calibration. Machine Vision and Applications, 23: 79–89.
  • Fang S., 2013. An Interactive Warping Method for Multi-channel Visaul Simulation System. Journal of Computer Aided Design & Computer Graphics, 25(9): 1318-1324. (in Chinese)
  • Gao W., Wang L., Hu Z.Y., 2008. Flexible calibration of a portable structured light system through surface plane. Acta Automatica Sinica, 34(11): 1358–1362.
  • Harville M., Culbertson B., Sobel I., Gelb D., Fitzhugh A., Tanguay D., 2006. Practical methods for geometric and photometric correction of tiled projector. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06). Workshops, New York, USA.
  • Li D., Xie J., Zhao L., Zhou L., Weng, D., 2015. Multi-projector auto-calibration for non-planar surface. Computer Engineering and Application, 51(7):194-203. (in Chinese)
  • Martynov I., Kamarainen J.K., Lensu L., 2011, Projector calibration by inverse camera calibration. in Proceedings of the 17th Scandinavian conference on Image analysis, Heidelberg, Berlin.
  • Moreno D. and Taubin G., 2012. Simple, Accurate, and Robust Projector-Camera Calibration. Second Joint 3DIM/3DPVT Conference: 3D Imaging, Modeling, Processing, Visualization & Transmission. Los Alamitos, CA : IEEE Computer Society.
  • Raskar R., Brown M.S., Yang R., 1999. Multi-Projector Displays Using Camera-Based Registration. IEEE Visualization Conference, Los Alamitos, United States.
  • Raskar R., van Baar J., Chai J., 2002. A Low Cost Projector Mosaic with Fast Registration. Proc. 5th Int. Conf. Computer Vision. Melbourne, Australia.
  • Raskar R., Baar J., Willwacher T., (2004). Quadric Transfer for Immersive Curved Display. Computer Graphics Forum. 2004, 23(3): 1-10.
  • Raskar R. and van Baar J., 2005. Low-Cost Multi-Projector Curved Screen Displays. Digest of Technical Papers-SID Int. Symposium, 36(1): 884-887.
  • Sajadi B. and Majumder A., 2011. Autocalibrating tiled projectors on piecewise smooth vertically extruded surfaces. Visualization and Computer Graphics, 17(9): 1209-1222.
  • Sajadi B. and Majumder A., 2010. Auto-calibration of cylindrical multiprojector systems. Proceedings of Virtual Reality Conference , Boston, USA.
  • Sajadi B. and Majumder A., 2009. Markerless view independent registration of multiple distorted projectors on extruded surfaces using an uncalibrated camera. IEEE Transactions on Visualization and Computer Graphics, 15(6): 1307-1316.
  • Sun W., Sobel I., Culbertson B., Gelb D., Robinson I., 2008. Calibrating multi-projector cylindrically curved displays for wallpaper projection. Proceedings of the 5th ACM/IEEE International Workshop on Projector Camera Systems, Marina del ray, USA.
  • Yang R., Gotz D., Hensley J, Towles H., Brown M.S., 2001. Pixelflex: A reconfigurable multi-projector display system. Proceedings of Visualization’01, IEEE Computer Society.
  • Yicheng J. and Yong Y., 2013. Maritime Simulator. Beijing: Science Press. (in Chinese)
  • Zhang Z., 2000. A flexible new technique for camera calibration. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(11): 1330–1334.

Authors


Liu Xizuo

Affiliation : School of Naval Architecture, Dalian University of Technology / Department of Surface Ship Command, Dalian Naval Academy, Dalian China
Country : China

xizuoliu@163.com

Lin Yan

Affiliation : School of Naval Architecture, Dalian University of Technology
Country : China


Xu Xiaogang

Affiliation : School of Computer and Information Engineering, Zhejiang Gongshang University, Hangzhou China
Country : China


Ma Liang

Affiliation : Department of Surface Ship Command, Dalian Naval Academy, Dalian, China
Country : China

Attachments

No supporting information for this article

Article statistics

Views: 212

Downloads

PDF: 34

Citations