A Pen-Based Interactive Projection System Based on the Infrared Scanning Projector

Nowadays, how to make a human-computer interaction system which can take into account large-area display, high precision tracking and multi-user operation at the same time is a hot research topic. This paper presents a novel low cost pen-based projective interaction system that can write or draw like writing brush. The system is composed of infrared scanning projector, digital pen, projector, screen, wireless receiver module and computer. The infrared scanning projector is designed to achieve fast projection, which is to track the pen’s two dimensional location accurately along with the digital pen. The system has the advantage of multi-user supported, high tracking precision, and has the function of identity recognition and writing calligraphy. Key Words—Gray code, Indentify identification, Infrared scanning projection, Pen-based interaction.


INTRODUCTION
With the great development of computer and multimedia technology, large-area human-computer interaction display system has been greatly improved and widely used in many fields, such as education, commerce, multimedia and games, etc.The method of position measurement is an important part of a human-computer interaction system, such as ultrasonic, laser, pressure, resistance, capacitance and electromagnetic method, etc.But most of above mentioned methods cost a lot or have some defects on the application of large-area, which greatly limit their industrial development.
As we all know, pen is mostly used by people as the traditional writing or drawing tool, so the operator would like to use the digital pen to write or draw on the large screen instead of fingers.Currently the research of interactive pen becomes hotter and hotter.Hongfeng Guo et al. [1] used PSD (position sensitive detector) as the detector to track the light of the LED on the tail of the pen.This method can solve the problem that our body or hand would block the light wherever the photo detector is, but the PSD is expensive and the algorithm determines the capability of the system.Junxing Zhang et al. [2] used a surface position method based on two-dimension encoded recognition.In this system, CMOS image sensor was adopted in the Pen-type coding detection system, and the structure of the pen is very complex.Moreover, the lens in the pen may reduce the stability of the system.Florian Block et al.
[3] designed a digital pen, which writes with real ink on the interface palettes in order to bind either physical or sketched components to a function, the user can annotate a new control with a handwritten label.
In this paper, we present a novel low cost pen-based projective interaction system based on infrared scanning projector.The infrared scanning projector along with a digital pen is designed to implement infrared optical tracking.The system enables multiple users to operate simultaneously.Each digital pen has a unique ID which is capable of keeping track of each individual's drawing.The digital pen can be used in Chinese calligraphy.The system has the advantages of fast response, high accuracy and low-cost, etc.

Review Stage
The pen-based projective interaction system is composed of infrared scanning projector, projector, screen, digital pen, wireless receiver module and computer.Users can interact with the projected image using the digital pen, which is designed for using in Chinese ideograms, especially in calligraphy.The system supports multiple users to operate simultaneously, which could discern the users' identifications.Different users can write or draw independently without influence by each other.The block diagram of the system is showed in Fig. 1.
The infrared LED is used as the projection light source, in the infrared scanning projector, which projects a order of binary Gray code patterns onto the screen in a certain frequency under the control of the microcontroller; The digital pen detects the infrared light by the infrared detector located at the nip of the digital pen, then the detected infrared light is transformed to the digital signal by the information process module located in the mid of the digital pen, and the wireless transmitter sends digital signals to the wireless receiver connected with thecomputer; The information process and control computer calculates the two-dimensional screen location of the digital pen.With the real-time tracking of the digital pen, users can control the content on the screen such as the pictures or the videos projected by projector; the digital pen is a digital writing brush, users also can do calligraphy or draw Chinese painting using it.The structure of the system is shown in Fig. 2. III.TRACKING PRINCIPLE

Binary Code Pattern and Binary Gray Code Pattern
The location tracking of the digital pen is based on temporal coding.The basic principle involves projecting a sequence of patterns constructed from alternating black and white lines that are half the width of those in the previous pattern, thus creating a natural encoding hierarchy.In the system we use the binary Gray code pattern as the encoded pattern.Gray code (Grey Code, also known as Greer code, binary cyclic code) is invented in 1880 by Jean-Maurice-Emlle Baudot who was the French engineer, which is a kind of absolute encoded code.Gray code structured light has been widely used in 3D measurement [4].With binary Gray code, m patterns are necessary to encode m 2 stripes.Binary Gray code has been widely used in distance measurement system [5,6].
The Gray code is superior to a binary code projection in structured light analysis.Successive numbers of the Gray code vary exactly in one bit.Thus, wrong decoding which is most likely to occur at locations where one bit switches, introduces only a misplacement of at most one resolution unit.Then the width of bright and dark lines in the pattern with finest resolution is twice as wide compared to the binary code.This is useful for analysis especially at steep object surfaces where the code appears to be compressed.Since we make use of a per pixel varying threshold, the Gray code solution is very robust and makes the system have more accurate tracking accuracy, as shown in Fig. 3.

Infrared Scanning Projector
The infrared scanning projector is composed of several grating projection modules (the number depends on the resolution of the location tracking of the infrared scanning projector).A grating projection module consists of an infrared LED as light source, grating encoded film and lens.The grating encoded film is located at the focal plane of the lens.The role of the grating projection module is projecting encoded pattern onto the screen, and the projection of the encoded pattern can't be seen by the human eyes because of the infrared light source.The projection content of the visible projector will not be affected by the infrared scanning projector.
When the infrared LED is on or off in turn as a certain frequency, the Gray code pattern is projected on the screen as the certain frequency.The processing is shown in Fig. 4 The first pattern divides the screen into two, when the digital pen is in the bright area, the detector of the digital pen can detect the infrared light and convert it into digital signal 1.On the contrary, when the digital pen is in the dark area, the detector converts the signal into 0. The dividing line between light and dark is special, which can't be determined.Then we judge the position through the following pattern.Thereafter, the computer can receive a unique binary string after one scanning.A unique binary string could be derived from the 1D location coordinate.

IV. IMPLEMENTATION
To make it more authentic for users to use the digital pen, the digital pen is equipped with three pressure sensors, which make the digital pen write calligraphy like the writing brush.The pressure sensors could detect the direction and the force when user is writing on the screen.
The digital pen is not just for location tracking.Moreover, the wireless transmitter is independent that gives each of digital pens a unique ID, as shown in Fig. 5 In multi-points interactive system, how to distinguish every point is vital.Some multi-touch screen uses digital image processing method to recognize the touch points, it is hard to distinguish the identity of each point.So our method makes it easy to distinguish different users' identity.The display interface is designed by XNA engine.XNA is a 3D game development environment based on Direct X, which can help develop 3D games more quickly and better.The system uses XNA as the development platform.The interface is designed with the characteristic of Chinese calligraphy, as shown in Fig. 6.When the distance which is from the infrared scanning projector to the screen is 2 m, the screen is 40 centimeter long, 30 centimeter width in the experiment.We tested 100 points on the screen for measuring the error, and calculated the measurement error.The statistical result is shown in Fig. 7. CONCLUSION AND FUTURE WORK This paper presents a novel low cost pen-based projective interaction system based on infrared scanning projector.The structure of the system and the principle of location tracking are discussed.The accuracy of the system is decided by the resolution of the infrared scanning projector.In this system, the infrared scanning projector has five patterns, so the screen can be divided into 5  2 parts at most.Our screen is 40 centimeter

Num (points)
Error (cm) long, 30 centimeter width in the experiment.The resolution is 1.25 centimeter in principle.The resolution of the screen is 1.4-1.5 centimeter according to our measurement.The delay of the system is not obvious when user is writing or drawing.The infrared scanning projector can be expanded easily, and the size of the screen can be changed in need.So the accurate of the system has much room for improvement.If three infrared scanning projectors are used in the system, three dimensional location tracking could be achieved.Therefore, the system is promising to be applied in industry because of its attractive performance and low cost.

Fig. 3 .Fig. 4 .
Fig. 3.The boundary dot in binary code pattern and in the binary Gray code pattern

Fig. 5 .
Fig. 5. Wireless transmission and the structure of the digital pen

Fig. 7 .
Fig. 7.The measurement error of the location of the points on the screen