Museum augmented interface for historical scale models: towards a new way for cultural heritage promotion

In this paper, we describe an interactive museum application dedicated to historical scale models. This comes from a joint work between multidisciplinary teams: industrial engineering researchers, historians, museum curators and interactive interface designer. We present here the result of the project, based on scientific methodology. Results include system architecture, hardware and software, some use cases and user evaluation figures. This paper also underlines some methodologies issues that illustrate future possibilities.


INTRODUCTION
The increasing availability and performance of digital technologies is making museum's collection promotion more interactive.Cultural heritage benefits a lot from both acquisition techniques and promotion applications.But accurate interactive experience of history does not only rely on innovative technologies but also on innovative uses of low-cost digital material.
One of the most crucial issues in cultural heritage promotion is the process of curation.From heterogeneous data acquisition to appropriate visitor experience, a complex methodology is involved to achieve scientific purposes and cultural promotion mission.
Methodological aspects of this work have already been published in details.In [2], we explained the process of historical scale model acquisition and the combination with semantic information.We explained in [4] how knowledge modeling is important to avoid one-shot application.Finally, in [3], we introduced the advanced visualization possibilities to explore cultural heritage history thanks to virtual reality.
In this paper, we focus on an innovative use of digital technologies to promote cultural heritage.The museum context is related to a 100-year old scale model designed in 1899 to promote Nantes (French city) harbor activity.

Fig. 1: Nantes harbor scale model in Museum of history
Due to its huge dimensions (~18m²), this scale model is a precious witness to historians.Its scale is 1:500.It is currently exposed in the permanent exhibition of Nantes history museum in the "Château des ducs de Bretagne" and only little information is displayed to visitors: a 10-lines label and 3 archive movies.Moreover, nobody can touch the object, for obvious conservation purposes.
We propose an augmented visualization system to allow visitors to virtually manipulate the scale model.This system has been designed in a reproducible way, so that the approach can be used to promote other scale models.Application source code, database schema and historical material will be available for reuse purpose.
Cultural heritage comprehension and promotion would benefit from formal methodology.In this paper we describe a process to promote one kind of CH object (scale models), but the methodology could be applied to the whole museum's collection.And it can enhance the museum discourse as far as object's histories are connected.

RELATED WORKS
Virtual 3D mock-up is a widely used artifact for cultural heritage promotion and understanding.It can be used to explore archaeological sites, or for hypothesis analysis.Some interesting projects combine 3D visualization to scientific historical purposes [5].Such applications provide great tools for researchers to explore physical space in its historical context.Virtual reality technologies such as panoscope 3D [9] can also be used to provide immersive experience to visitor.Interactivity such as immersive experiences is considered useful when dealing with cultural heritage comprehension [6] [7] [8].
As far as scale models are concerned, many projects already exist.But most of them concern blank model on which it is possible to make video mapping and other visual effects.Very few deal with historical "plan-reliefs".Some have emerged worldwide.The following table summarizes the different projects characteristics.These figures illustrate a real need for museum when dealing with such objects.Besides the design of interactive application for visitors, there are great challenges to face with.
Here, we focus on the design process of museum visitors' application based on physical object and information acquisition.First, we need to define the scope of the discourse in space and time.The space delimitation is quite easy as it is delimitated by the object itself.The time period can be more complicated as we can broaden the historical study.We chose to limit the research to object's contemporary sources but provided to the system the ability to handle any kind of elements in time.On a historical point a view, this is a great advantage as it allows future research to be included and to be compared with existing material.
Considering the promotion challenge for visitors, we did not find any existing satisfying application for such objects.Historical scale models are almost never integrated in an interactive application for museum.Those who actually designed visualization facilities focus on one-shot application without content evolution possibility.
Our approach is mainly based on semantic historical content.We then connect any available source (graphical, textual, audio, video), geographical information and virtual 3D data.3D data comes from the scanning of the object [13].Finally, we design an augmented system based on multitouch interface and video projection, both connected to specific software and to a dedicated open database.

PHYSICAL SYSTEM ARCHITECTURE
The proposed "augmented scale model" system is based on 5 main components: - The physical scale model - The virtual 3D model -One or several multitouch screens (depending on the size of the object).We choose 30 inches screens for accessibility issues and ergonomic purposes.-One or several video projectors (depending on the side of the object) -Web accessible database -User graphic interface

Fig. 2: Proof-of-concept proposal for "augmented scale model"
Human machine interface relies on the connection between these 6 elements.Visitor can virtually explore the city through the multitouch screen.The main interface is composed of two highdefinition pictures (8000px width): human-like view (using image stacking) and orthophoto.The two pictures are done thanks to semi-automated photo shooting.

Interactive software description
The multitouch designed application is written in Python, with the free software Kivy 1 framework.Kivy is widely used to design multitouch applications and is portable on every platform.
It allows gestures with several fingers: selecting, scaling, panning, and rotating.
Our software connects to the database and retrieves metadata and values in JSON format.It can display a window with 3D virtual model thanks to Panda3D open-source library 2 .
Our application uses virtual 3D model to automatically detect scale model part that need to be illuminated using Polygon python library 3 .Moreover, it also uses the virtual 3D model to cut in the main interface picture a newly inserted area.
For research purposes, we designed a proof of concept (Fig. 2) of the proposed system.It is based on a smaller scale model, from the same author and the same epoch, that also belongs to the Nantes history museum.

DESIGN PROCESS OVERVIEW
An analogy with an iceberg easily gives an overview of the underlying process.Our approach is mainly user-centered.But our global system users are not only museum visitors.They also are museum professionals and other experts such as historians, and archaeologists.
Therefore, we do insist on knowledge organization system.This is crucial to design proper final application [1].Indeed, we did not want to design one-shot device but rather imagine a global information system for museum.This is important to understand our process described on Fig. 3.

Project resources
Resources involved, and especially human resources, in such project are significant.Here is a non-exhaustive taxonomy of involved skills: -Curators who define the scope of the project.

Physical object acquisition: 3D scanning
Our methodology has been published in [2] in relation with University of Liège geomatics unit that work on similar project.Global process consists of merging semantic and physical information into a virtual artifact.In the case of urban city models, it is called 3D virtual city model (3DVCM).
Scale model 3D digitization (Creaform HandyScan 3D) provides a STL file composed of almost 100 million of points.Many topological issues still have to be treated at this step: holes, files merges, missing parts and so on.Post-treatment (using Geomagic software) to solve these issues is described in [11].
Once the virtual 3D model is properly treated, it can be used for several purposes -Virtual world generation could lead to virtual immersive experience in old Nantes city harbor.This implies a texturing step that is not covered in the scope of this paper.-Semi-automatic geo-location is interesting to retrieve geographical information.This could lead to geolocated mobile applications for example.In our case, this is not fully implemented because of the poor quality of the extracted 2D map.The main other issue is the incorrectness of the scale model regarding to actual city.-Connection between touchscreen, software application and video-projection on the physical object is the main use case for instance.When an area is select on the main user interface, the corresponding virtual area is detected in the virtual model.Then, the system retrieves the corresponding area of interest in the database to highlight it on the scale model in front of the visitor.

Physical object acquisition: photogrammetry
In addition to the 3D digitization, photo acquisition is done to reproduce the view a visitor would have of the object in front of him.This step is semi-automatic and requires home-made traveling system to move the camera.2 pictures are made for the application purpose: a 45° oriented picture using image stacking technique and an orthophoto using photogrammetry.
For both points of view, the camera is programed to cover every part of the scale model.25 photos are done for each camera position with a different focal length each time.20 000 photos are necessary to provide the 8000 pixels width final picture.The final picture resolution guarantees that the visitor will be able to zoom in the picture.
It also allows us to make the link between geographical parts and semantic data in the database.Every city parts (building, place, street) on the physical object is stored as a polygon in the scale model coordinate system.
Polygons can be drawn manually on our dedicated web interface, on top of the picture, or can be retrieved semiautomatically for the 2D map extracted from the 3D model (this has not been fully implemented yet).Polygons coordinates are stored in WKT (well-known text) to PostGIS software.
A mapping of objects ids allows the system to identify what part of the museum scale model is linked to which database historical object.
For the future, we work on the possibility to semi-automatically derive geo-located data from these local coordinate system's polygons.

Semantic data acquisition
Historians have made research in archives for about 500 points of interest.They can be related to a company, a building (shipyard), a person (an employee) or even a thematic ("Shipment industry at the beginning of the twentieth century").
Each one of these subjects is documented thanks to historical sources that can be either texts and/or graphics.In addition, they are also linked together thanks to metadata and especially keywords.
For every subject, we have a dedicated sheet containing metadata (title, description, address, keywords), paradata (historian information, annotation) for scientific methodology, associated sources and spatio-temporal information.For example, we can specify that the "Chantiers Dubigeon" company was at place "A" from date 1 to date 2. We can also mention the precision for time information.Space precision is not handled for instance.

Data modeling
Conceptual model is based on the DHRM (Digital Heritage Reference Model), a meta-model for industrial heritage analysis [4] [12].
In our context, we deal with semantic information (items of interest and relationships between them), geographic information with time specification (location of physical parts on the scale model and possibly geo-location information), and other assets such as digitized paintings, postcards, photos, etc.For each class, we store dedicated metadata.Graphical assets metadata come from the existing museum collection database.Semantic information metadata (title, address, description) are adapted from the museum collection database.For design purpose, we add some metadata (summary, formatted keywords, coordinates in the physical object coordinate system) and relations based on keywords.
Object class attributes basic metadata such as title, description, address, and keywords.This class could be extended with reference models such as CIDOC-CRM [14] to specify the typology of objects involved.
Relationship class stores the links between objects based on keywords.Some metadata can be added to specify the relationship properties: annotation, date of the relation's beginning, date of its end.Once again, the model could be extended to take metadata from external schema (Dublin Core, CIDOC-CRM) into account.
Geographic information is time dependent.This allows the system to possibly be scalable: current data is approximately situated between 1880 and 1930, but we could imagine extending the time span.An object can have several geographical positions over time.
We implemented the model using free software Postgres.Geographic information is stored in PostGIS extension.For the semantic data, tests have also been done using Neo4j graph database with full success.But this is still experimental for future research purposes (identification of historical paths of interest).

Web interface
A web interface, written in standard web languages (HTML, CSS PHP, Javascript) with CodeIgniter framework, allows researchers to upload their data and museum teams to edit and access the data.
A moderation and validation process has been decided to guarantee the quality of content.Once the data is validated, it is available online to everyone.Registered users can also propose new content such as postcards, photos or give information on new element depending on their access level.For example, basic user can only upload assets whereas researcher can add new information about a building.Main behavior is based on two steps.The scale model has been divided into three main neighborhoods.Thus, any visitor's action first redirects him to one of these neighborhoods.Then, the sub-parts of the neighborhood are highlighted and can be selected.
Once he selects an area, the system displays related content available in a carousel.Graphical resources can be zoomed and text can be resized depending on user abilities.Some related points of interest are also underlined to give some navigation clues for the visitor.Non-geographical elements are displayed at the bottom of the screen.They allow visitors who select a particular bridge or street to see all the bridges/streets on the physical scale model.This leads to serendipitous navigation based on our semantic network.

Fig. 6: Area selection and lighting feedback
We do not force any scenario; everyone can make research on any available element.

Fig. 4 :
Fig. 4: "Dubigeon" shipyards instanciation example All these concepts have been modeled in a relational database.For each class, we store dedicated metadata.Graphical assets metadata come from the existing museum collection database.Semantic information metadata (title, address, description) are adapted from the museum collection database.For design purpose, we add some metadata (summary, formatted keywords, coordinates in the physical object coordinate system) and relations based on keywords.