Computer-Generated Holography for Dynamic Display of 3 D Objects with Full Parallax

In this paper a new holographic three-dimensional (3D) display system based on computer-generated hologram (CGH) is developed for the reconstruction of 3D objects with full parallax. A new algorithm is also developed to reduce the hologram computation time and memory usage. The dynamic 3D objects are successfully reconstructed at video rates in both real and virtual spaces.

such as lenticular lenses and parallax barriers, which direct different images to specific viewing zone in space.Two eyes of the observer in front of the display screen will receive different images and create 3D effects.Usually, this display system only allows us to observe horizontal parallax of 3D objects through optics or head tracking by moving from left to right, and vice versa.It will be more complicated if vertical parallax is introduced.Another disadvantage of this technology is that the number of viewing zones is limited.It is not convenient for many people to observe 3D objects at the same time.Furthermore, due to fixed focus length, our eyes cannot change the focus length like that when viewing real objects, and it may also cause eye fatigue.
Volumetric display technology [2], [3] constructs 3D images in real 3D space by using many methods such as volumetric scanning, screen rotation and light projection techniques.One can observe this 3D image from almost any angle without eye fatigue.This display technology can be used in the fields such as medical imaging, mechanical computer-aided design and military visualization.However it cannot cause occlusion and essentially has fixed 3D volume.
Holographic display is a true 3D display technology [4], providing all four eye mechanisms (depth cues): binocular disparity, motion parallax, accommodation, and convergence.One can view 3D objects with full parallax displayed with holographic technology without wearing any special glasses and no visual fatigue will be caused to human eyes.The occlusion can be introduced and the reconstructed 3D images can be scaled to a desired size.Holographic 3D display products might be launched into the market with target applications like 3D photos, 3D games, and scientific visualization in the next 5-10 years.From long-term point of view, there is a potential to develop a new 3D TV system based on digital holographic display technology.
Recently, a joint project between DSI and I2R of A*STAR of Singapore has been initiated to develop a new holographic display system for the reconstruction of 3D objects.In this paper, we introduce our research progress in developing the holographic 3D display system with computer-generated holography (CGH).We also present a new algorithm that has been developed to reduce the hologram computation time and memory usage.The 3D objects have been dynamically displayed at video rates in both real and virtual spaces with our system currently developed.

II. RELATED WORK
Holography can be used to record and reproduce the amplitude (luminance), wavelength (chroma) and phase differences of light waves via the interference and diffraction of coherent light.It has been successfully applied to the holographic video display systems [5], [6].Holographic 3D display technology is based on computer-generated hologram (CGH) and spatial light modulator (SLM).A CGH is a digital hologram generated by computing the interference pattern between an imaginary object wave and a reference wave.The CGH is launched onto the SLM device.An expanded laser beam is then used to illuminate the SLM and to reconstruct the 3D image through light diffraction.Many people can view different perspectives of the reproduced 3D object at the same time in different angles without glasses.However, the practical application is hindered by the lack of algorithms that can generate high-resolution holograms fast enough.Therefore, it is still necessary to develop new algorithms that can greatly reduce the hologram computation time.
Various holographic 3D display systems have been developed by different R&D groups in the past years [4], [7], [8].Hilaire et al. [7] at MIT developed the holographic display system using diffraction-specific (DS) algorithm and multichannel acousto-optic modulator as an SLM.The computation time was much reduced when they implemented the DS algorithm on PC's graphics hardware.Maeno et al. [8] used liquid crystal devices (LCDs) as SLMs and discarded the vertical parallax.The performance of their system was limited by the LCD specifications such as pixel pitch and pixel number.Slinger et al. [4] at QinetiQ developed the first commercial level holographic display system using active tiling method in which they replicated EASLM (electronically addressed spatial light modulator) frames onto OASLM (optically addressed spatial light modulator).They implemented active tiling by introducing shutter system synchronized with EASLM frame rate to tile sub-holograms onto OASLM in correct sequence.The work done by Masuda et al. [9] at Chiba University was initially related to the development of new hardware for the fast computation of CGH using coherent ray tracing (CRT) method.Later they implemented CRT on PC's graphics hardware to improve CGH computation time.In our implementation, we used a digital micro-mirror device (DMD) as an SLM for 3D object reconstruction and performed CGH computation with a new algorithm.

III. SYSTEM OVERVIEW AND DESCRIPTION
We have developed a holographic 3D display system that allows us to view the reproduced 3D objects either through a 2D display screen in virtual space or via a 3D display medium in real space.A system block diagram is schematically shown in Fig. 1.A new algorithm has also been developed to reduce the computation time and memory usage for the CGHs of 3D objects.A detailed explanation of this algorithm will be given in section IV.Our 3D display system is based on ultra-high frame rate DMD (up to 8 kHz) for rendering holograms in real-time.The pixel size of DMD is 13.86 µm.Other off-the-shelf optical components and a 50 mW red laser diode at 655 nm are used to realize the reconstruction of 3D objects in virtual space as well as in real space.Two sets of optics are used.One is before DMD for cleaning, expanding and collimating laser beam as well as illuminating DMD with plane wave at required angle.The other is after DMD for guiding two types of reconstruction and filtering out the unwanted spatial frequency components in reconstructions.The reconstruction in virtual space is viewed by directly looking into the DMD.The reconstruction of 3D objects in real space is viewed by looking at 3D display medium such as gel tank or 2D paper on which the filtered 3D objects can be imaged at different depth locations in real space by free space propagation.
The system interface for control and display is developed by using LabView software.The simultaneous reconstruction of multiple 3D objects at different locations and the dynamic 3D object reconstruction at video rate are realized just by controlling the DMD frame rates with time-sequencing different 3D object holograms.This is enabled by using the DMD with an ultra-high frame rate up to 8 kHz and a large on-board memory (16 GB).All other information regarding shape, size, and orientation of the 3D objects is pre-encoded into the holograms computed.
With our holographic 3D display system currently developed, we have reproduced dynamic 3D objects at video rates in both real and virtual spaces.Fig. 2 shows different perspectives of a 3D cuboid with the size of 1 cm × 1 cm × 2 cm reconstructed in real space, rotating along the vertical axis at different angles.Intensity distribution used in original CRT can be described as [9], [11]: One commonly used method to reduce computation time is to off-line pre-compute all possible values for the cosine function in (1), and store all results in a big table for further in-line computation.This algorithm is known as CRT with look-up table.
As in most of the cases, the object-hologram distance is much bigger than the wavelength (i.e.z j >> λ), and also z j >> (x-x j ), (y-y j ), (1) can be approximated as (2) using Fresnel approximations: In (3), θ only depends on x, x j and z j , while φ only depends on y, y j and z j .For a single object point, the resulted hologram is ( ) a real e e j j ϕ θ = * * .
It is the product of three factors, object point intensity a j , (x j ,z j ) dependent factor e iθ and (y j ,z j ) dependent factor e iφ .Physically, e iθ modulates the beam in horizontal (x) direction, and e iφ modulates the beam in vertical (y) direction.
Thus, let horizontal light modulation factor ( ) Object points on the same vertical line share the same horizontal light modulation factor H(x).According to (3) and ( 5), the hologram resulted from these object points can be written as: ( ) where n is the number of object points falling on the same vertical line, a j the intensities of these n object points, I'(x,y) the intensity in the hologram plane for these n object points.
The part ( ) 6) is independent of x and x j , thus ( 6) can be broken down into two steps: Step 1: ( ) ( ) 1 Step 2: ( ) where S(y) is the sum of contribution in vertical direction from those n object points to the hologram.
Assuming the width and height of hologram are X and Y, computation complexity of ( 7) and ( 8) is in O(nY) and O(XY) respectively, where O( ) is the big O notation.Thus, for these n object points in the same vertical line, CGH computation complexity is in O(nY+XY) using ( 7) and ( 8), while in O(nXY) using ( 1) or (3).
As number of possible H(x) and V(y) values are limited, they can be off-line pre-computed and stored in memory for in-line CGH computation.This can avoid using square and square root operations in in-line computation.
Detailed comparison of the number of operations for n object points is listed in Table 1.Both CRT with look-up table and the new algorithm show large reduction in computation load.When more than one object points fall on the same vertical line, the new algorithm has advantage over the one using look-up table.
The final hologram I(x,y) in ( 3) for all N object points can be computed by summing up holograms I'(x,y) computed by (8) for all vertical lines which have object point(s) on them.Due to DMD properties, holograms have been binarized before they are launched onto DMD.The binarization is done in such a way that number of bright pixels and number of dark pixels are as close to equal as possible in resulted CGH.
Memory usages of these algorithms are listed in Table 2, assuming the whole object space is sampled into m depth layers along z axis.A typical binary CGH of a cuboid (1 cm × 1 cm × 30 cm) computed with the above new algorithm is shown in Fig. 6.This CGH is 2D distribution of binary fringes decided by the object points in 3D space.As the CGH is to be displayed on DMD, its size is fixed to 1024 × 768, which is the same as DMD's resolution.Looking directly at this CGH does not provide any visual information unless it is illuminated with laser light with wavelength specified during computation.
Even though for virtual reconstruction the 3D object size is limited by SLM active window size (0.7 inch diagonal), for real reconstruction the object size is limited by projection distance only, i.e. real reconstructed object size is bigger than the object size used in CGH computation, when projected at far distance.The computed hologram has inherent lens function to create sharp 3D image at pre-decided distance.
The 3D object reconstructed from this CGH in virtual space with our holographic 3D display system is shown in Fig. 7(a), 7(b) and 7(c), which are corresponding to the perspectives viewed from right, center and left, respectively.We can see that the 3D cuboid has been reconstructed with full parallax.
Implementation and optimization of this new algorithm has also been done on graphic processing unit (GPU) and the computation speed has been increased by around 10 times as compared to that on CPU.Fig. 8 shows a typical 3D teapot reconstructed from its binarized holograms (CGHs) computed with our new algorithm on GPU.Dynamic display at video rate of such a 3D teapot rotating along its vertical axis has been realized with our holographic 3D display system.Due to its less computational complexity and lower memory requirement, a few frames per second running speed on normal PC with GPUs and a few MBs of memory usage per CGH with thousands of object points are realized.It has shown the potential of this algorithm in producing full parallax and high quality 3D objects reconstructed at video rates.It may make holographic 3D display more suitable for handheld devices and home entertainment applications in the near future.

V. CONCLUSION
A holographic 3D display system is developed by using a DMD as an SLM, which allows us to view 3D objects either through a 2D display screen in virtual space or via a 3D display medium in real space.The dynamic display at video rate of 3D objects at different locations is realized with this system.A new algorithm is developed, which significantly reduces the time and memory usage for CGH computation.
x,y) is the intensity in the hologram plane at z = 0, N the number of object points and λ the wavelength.(x j ,y j ,z j ) is the object point coordinates and a j the intensities.
computation time and memory usage among algorithms are shown in Fig.4and Fig.5, respectively.The new algorithm increases computation speed by 30 times as compared with original CRT and around 3 times as compared to CRT with look-up table.It also reduces memory usage to 1/438 as compared to CRT algorithm with look-up table for the hologram with the size of 1024 × 768.The new algorithm requires very low memory usage and hence almost overlaps with the line of the original CRT algorithm in Fig.5.This set of data is based on a PC using Intel QX9650 CPU.
of depth layers, m

TABLE 1 :
COMPARISON OF THE NUMBER OF OPERATIONS AMONG ORIGINAL CRT, CRT WITH LOOK-UPTABLE AND THE NEW ALGORITHM.

TABLE 2 :
COMPARISON OF MEMORY REQUIREMENT AMONG ORIGINAL CRT, CRT WITH LOOK-UP TABLE AND THE NEW ALGORITHM