Semantic Modeling Approach of 3 D City Models and Applications in Visual Exploration

In recent years, the necessity of the incorporation of semantic information into three-dimensional city models (3DCMs) has become a consensus in 3D GIS field. In order to provide practical support for visual applications concerned with semantics, this paper firstly presents an extended semantic model based on the CityGML standard, which was worked out for the general storage and representation of semantics. In this model, concepts like Room, Corridor and Stair are all derived from concept Space which corresponds to the concept of Room in CityGML. This extension will benefit the indoor structure representation. Geological feature is also supported by the model for the underground analysis. Next, for the promotion of semantic modeling by this model, a semi-automatic process of semantic enrichment is implemented in a data integration tool. It provides an adaptive way to link semantics with pure geometry. Finally, two typical cases of visual exploration are illustrated to prove the model’s practicability in a national 3D GIS project of China. One is indoor routing, which adopts this model to extract the geometric path and thus enrich traditional semantic-enhanced navigation routine; another case is unified profiler, where semantics are intergrated in order to fill up the cross section correctly and ensure the topological and semantic consistency.


INTRODUCTION
As the success of Google Earth and Bing Map 3D brings a new time of 3D Geo-information, the accessibility of 3D city models has appeared to grow unprecedentedly [1].A number of 3D city modeling projects have also been proposed within ten years like 3D Berlin project.Meanwhile, increasing professional applications put more emphasis on conceptual meanings beyond geometry since the pure geometrical features mainly focus on the visual representation but ignore a full comprehension of the data in terms of human cognition [2].For example, applications like urban planning, facility management, and personal navigation urgently require additional information about the city objects given in a standardized representation [3].In fact, this supplementary information expresses the semantics of city models.However, the existing method of Geo-information management just by means of semantic metadata can't reflect clearly the relationship of semantic information [4].Therefore, it is desirable to incorporate the semantics of the 3DCMs through a normalized and intergrated semantic model.
As one of the pioneers, the 3D Berlin project firstly integrates semantics into the 3DCMs produced from traditional photogrammetry procedures [5].Semantic information is then used to facilitate the information querying and spatial analysis.Some vendors of commercial database management system like Oracle provide object data type to represent 3D geometry and link it with the traditional attributes [6].In parallel with practice, related topics such as the consistency of geometry and topology as well as the coherence of geometry and semantics were extensively studied [3,7], which laid the basis of CityGML.
CityGML is an international standard for the representation and exchange of 3D city and landscape models accepted by the Open Geospatial Consortium (OGC), which not only represents the shape and graphical appearance of city models but also specifically addresses their semantics, thematic properties, taxonomies and aggregations [8].It is known that LandXplore in 3D Berlin project has become a standard package to visually interact with CityGML datasets [5].Recently, several extensions of CityGML are also proposed, such as the integration of both above and underground features and the support of geological data [9,10], and an lane-based road model and a semantic house property model which would be interpreted as an enrichment of the ADE in CityGML [11,12].
However, the current version of CityGML's framework does not include underground features like geological body and underground infrastructures [9].This shortcoming would obviously become an obstacle to the underground analysis.Moreover, specified concept of Space such as Stair and Corridor can be further derived from the Room of CityGML's building semantics in order to better support indoor applications.Therefore, this paper presents an extended semantic model based on CityGML and illustrates it in two typical examples of visual exploration.The rest of the paper is organized as follows: Section II introduces an intergrated semantic model and its extensions.A flexible structure for storage of geometry-semantic coherence is discussed in Section III, followed by an introduction of the process of semantic enrichment for semantic modeling.The validity of this semantic model is illustrated in the visual exploration of indoor routing in Section IV and unified profiler in Section V. Finally, conclusions are drawn in Section VI.

Semantic Classification
It is an objective demand of city management and representation to classify ground features on semantic level.The semantic classification is also required by various applications which focus on certain feature's importance.Fig. 1 shows the basic classification by different thematic properties according to the Technical Specification for Three Dimensional City Modeling of China [13], such as the Building here is specialized into residential building, public building, etc., which will make the semantic enriching process more standardized and adaptive.Similar to the proposed approach in [9], an extended class Geology is added to the model to support underground analysis.In the following, the extension of the geological semantics as well as the building semantics, will be discussed in more detail to illustrate what is different from the definition in CityGML.

Semantic Hierarchy
The semantic classification partly reflects human's conceptual knowledge and direct cognition of different types of objects and their relationships, which also lays the foundation for the establishment of inherent semantic hierarchy.In fact, the detailed structure of features' semantic hierarchy is a further improvement on features' semantic classification so as to meet the needs of superior applications like semantic interoperability.Take emergency response for example.The pure semantic classification can help querying objects themselves and the best path between any of them.However, it fails to make advantage of semantic components inside the feature and thus couldn't In order to provide practical support for visual applications concerned with semantics, the paper gives detailed design of semantic hierarchy structure of building and geology features based on CityGML.
The UML diagram of Building is shown in Fig. 2. Via this hierarchy, the semantic model is more convenient and precise for constructing the geometrical route network automatically.
In the paper, a building is represented by optional attributes and constituents.The former contains function, usage, class, measured height, numbers and so on, while the latter refers to the components in its semantic hierarchy.Class Building is aggregated by two classes: IntBuilding and ExtBuilding, which could facilitate the extracting of LOD3.Especially, the term Space from IFC is used to represent the meaning of concept Room in CityGML in the paper [14], and subclasses of Room, Stair, ElevatorRoom, and Corridor are all derived from it to define specified space region with two or more Entrances (Entrance here is similar to ClosureSurface in CityGML but also derived from Opening).This extension based on CityGML is more appropriate for indoor application because it gets rid of the ambiguity when automatically extracting geometrical path directly from the topological accessibility graph based on Room and Opening of CityGML.For instance, the path extracted from the Stair in CityGML would be probably a vertical line segment instead of a line between a start entrance and an end entrance as illustrated in [15], which is obviously not suitable for elaborate applications such as indoor routing and navigation.
In the following, detailed definitions of the derived concepts are shown: (

2) Stair
A Stair is a specified Space which contains a stair and is bounded by WallSurfaces with at least two Entrances, as shown in Fig. 3.A Stair contains a label to record the bounded WallSurfaces as well as Entrances, by which the adjacent Space is connected.In the topological network, a Stair is represented by a connection.

(3) Corridor
A Corridor is a specified Space which contains a passageway and is bounded by WallSurfaces with at least two Openings, as shown in Fig. 4. A Corridor contains a label to record the Openings which are adjacent to it.In the topological network, a Corridor is represented by a connection.
The structure of semantic hierarchy of Geology is expressed as the UML diagram in Fig. 5.The underground features can be interpreted as the composition of Underground Body and Geology Object, separately representing discrete and continuous semantic information under the ground.Besides, the former derives Underground Buildings, Underground Pipelines and Underground Tunnels, while Geological Structure, Geological Stratum, Geological Body and Geological Ore Body are derived from the latter.This structure helps implement the classified storage and management of city underground features, and provide comprehensive support for underground analysis and visual exploration no matter whether the information is discrete or continuous.In the context of spatial data infrastructures, the needed data from distributed sources are often thematically and spatially fragmented.And semantic information can help to reduce the ambiguities for geometric integration, if it is coherently structured with respect to geometry [16].Therefore, the paper presents a geometry-semantic coherence structure to solve the problem of the separation of geometry and semantics.In addition, to seamlessly integrate heterogeneous data in the terms of the semantics, it is necessary to obtain the unified semantic description of the models and implement semantic enrichment by an intergration tool.

Geometry-Semantic coherence structure Semantic enrichment
A flexible and extendable structure is introduced for the storage of the geometry-semantic coherence, as illustrated in Fig. 6.Firstly, abstract thematic module is defined, from which hierarchies of semantics as well as topology could be derived.Then, semantic information is mapped onto its corresponding geometrical objects like a solid or a surface, based on which the geometry and appearance structure could be compactly designed for high performance visualization and different domains of semantics could be mapped to the geometry model and be utilized in different applications.

Semi-automatic process of semantic enrichment
As discussed in section 2.1, the unified semantic description of the 3DCMs were already defined.For it is hard to extract semantics from pure geometrical models automatically, a semi-automate tool is developed here for semantic enrichment.The incorporation of semantic information and the establishment of semantic hierarchy turn out to be simply choosing the geometrical components from top-down (from exterior to interior) and mapping the corresponding semantic concept onto them.To improve the efficiency of this semi-automatic process, an adaptive technique is employed which could automatically refine the selection of semantic concepts according to the semantic model.After all the components are mapped, the hierarchy is built at the same time.
The interface of the integration tool is illustrated in Fig. 7 (a).Firstly, the entire model is mapped to its abstract conceptual model of the semantic tree, for example, Fig. 7 (b) shows the building is mapped to the commercial building.
Then, in the step termed component editing, the geometry-semantic coherence is built up by grouping geometry surfaces like wall and door etc. and mapping it to the semantic concepts derived from the abstract building, the process is shown in Fig. 7(c).
Finally, in the step of component aggregation, the mapped components are aggregated based on the semantic hierarchy, such as a closed Space bounded by Wallsurface would be aggregated into a Room or a Corridor, as illustrated in Fig. 7(d).
After the semantic enrichment, a semantic model is acquired, based on which various applications like conceptual querying and emergency path searching can be implemented.

Overview
Indoor routing is widely used in almost all of the city emergency response scenarios.However, it is always a labor-intensive work to manually build the topological accessibility network inside a building.Fortunately, Lee proposed a method to extract the indoor network based on the connectivity graph of the building parts [17].This method is also fit for the semantic model presented in the paper, and can improve the results by using pre-discussed Stair and Corridor concepts.

Route deducing: Automatic extraction of path network
The automatic extraction of path network can be divided into two steps: topological network construction and geometric network construction.Topological network is a pure graph that represents the adjacency, connectivity and hierarchical relationships among the semantic nodes.Based on the definition of the semantic model (particularly the Opening), the topological network can be constructed automatically [8].For example, a connection will be built between two Spaces if they can be connected by an Opening.
In order to implement network-based analysis such as shortest path algorithms, the topological network needs to be complemented by geometric properties, which accurately represent the cost of connection.Lee presented a well-developed method, "Straight-MAT", to identify line segments from a simple polygon based on combinatorial data model (CDM) which describes the topological relationships between 3D geographic entities [17], as shown in Fig. 8.However, Stairs in their method are represented as vertical lines between the center points, which might lead to unconvincing results of routing.Moreover, the existing method can hardly deal with a near-square Corridor because the direction of the path line could not be uniquely defined.To partly solve this problem, the paper employs different strategies to extract path lines from different Spaces.For example, "Straight-MAT" method is used to extract path in a Room while the path of a Stair or a Corridor is directly extracted by connecting Entrances, as discussed in Section 2.2.Results are shown in Fig. 9 by green lines.

Enhanced route navigation
Unlike the ordinary navigation application, indoor navigation should inform user the key components and meaningful features along the path besides the path itself [18].With the help of semantic model, we could extract information such as the Room number and the name of the building part passing by.Selective representation routines are adopted to highlight the important features such as doors, Stairs and fade out the unrelated features such as other Room and Furniture.
Some of the results are show as follows.Fig. 10

Overview
The unified 3D profiler, which could help users make good sense of urban architecture, underground infrastructure, internal structure and interrelationship of stratum, is an effective tool in providing visual hints and revealing the context.However, since features in 3D city scene vary largely depending on geometry, topology and semantics, the topological and semantic consistency has a strong impact on the validity of profiling explorations and analysis results.

Archive consistency for topology and semantics
Standard languages for 3DCMs such as CityGML provide "solid" as the most important type to represent buildings, rooms, public infrastructure or other volume object in geometrical modeling."Solid" here is described mathematically by rigid body, which is a bounded, regular and semi-analytical subset of R 3   An example of profiling without considering topological and semantic consistency is given in [7].In boundary representation schemes, which are widely used in CAD modeling, solids are represented by their bounding surfaces, or topologically, a single closed 2-manifold.During the process of profiling, each solid should be rebuilt in accordance with topology consistency.Otherwise, the unstructured cross section will hinder not only the successive spatial analysis, but also the clear cognition of the objects.
On the other hand, for 3DCMs, marking, identification and affiliation descriptions of object type should be implemented with semantic consistency, which is the essential difference that distinguishes 3D city models from general geometrical models.In profiling, the meaning of cross sections cannot be told in a reconstruction with inconsistent semantics.And it is also difficult to ensure query accuracy and descriptive validity of relationships among objects.Therefore, it requires the implementation of creating topology-accurate profiles, which contains their semantic information derived from original semantic property and avoid topological mistakes such as degeneration and punctuation, to ensure the consistency of topology and semantics.During profiling, whether to fill up cross-sections or just extracting cross section lines should be judged according to original semantic and topological information.Then, the generated surfaces can be mapped to semantic concepts in semantic model.

Profiling with semantics
Semantics has two functions in the profiling, one is that feature classes and LODs can be told apart with the help of semantics; the other is that semantics could be used to help ensure the topology consistency.
In the profiling analysis, objects in a city scene can be divided into three categories according to their topological descriptions, namely, models represented by open surfaces, models by strictly closed surfaces and models by a mixture of both open surfaces and closed surfaces.The first category contains terrain surface included in the LOD0 model in CityGML, which is expressed in the form of an open surface.The second category consists of stratum in geology and models from LOD1 to LOD3 of CityGML as well as architecture components in BIM models, which are described in rigid bodies.The third category is mainly formed of LOD4 models in CityGML, of which building components such as walls, floors are expressed by open surfaces and rooms, IntBuildingInstallations and Furniture are expressed by solids, a good illustration of the difference between models of LOD4 and BIM models can be found in [19].
During the processing, cross sections should be treated separately, models belong to the first category only produce cross section lines, for which filling up is needless.For models belongs to the second category, cross section should be filled by surfaces which could be directly determined according to the topology of the original model.However, in cope with Space in some of the BIM models, inner Space should be revealed, the generated surface should therefore be shown in translucent mode.For models belong to the third category, which is the most complicated, different classes of surfaces should be distinguished based on the semantics so that we could fill the cross section of a Space with translucent surfaces which contain the semantic information associated to the Space, while treat other cross sections as WallSurfaces which should be filled by solid surfaces in order to close the gap between Spaces.The IntBuildingInstallation as well as Furniture can be deal with employing the same method used for the second category.
The main steps of the profiling are shown as following: Firstly, objects in the scene are classified into different categories according to the criteria discussed above.
Then, intersection lines are extracted between cross section surface and intersected surfaces.Delaunay triangulation routine is adopted to re-mesh the cross section surface.
Thirdly, a BSP tree is built and the semantic information, especially the Space, is employed to partition and classify the surfaces.
At last, the cross section is filled up according to semantics and mapped to the semantic hierarchy which ensures that new elements inheriting semantic information correctly from the original model.

Results
Fig. 12 shows the setting of a profiler and Fig. 13 shows a built cross section surface.Different types of cross section surface including planes and curved surfaces are supported.And what is the most important, the objective of the profiling application, such as which kind of feature should be profiled, could also be defined which represents the semantics of the task.
Examples of profiling with typical kinds of models in city scene are given in Fig. 14.The top shows a LOD3 model, with geometry and texture profiled correspondingly.The restored cross section is displayed transparently so as to reveal the inner Space.The middle shows a fine architecture model with no Space concept inside.All the components are filled up by solid suface with the same material after profiling.For example, the gray part on the first floor shows a cross section of a Buddha statue, which was filled up by the right texture.The bottom shows a geology model represented as a layer-cake.The profiling is executed in each stratum, which is properly filled up.An example of user-defined profiler based on semantics is illustrated in Fig. 15, in which Furniture "sofa" is set to be include in the analysis, while floor and Furniture "table" are excluded.In this exploration case, it is demonstrated that the unified profiler based on the topological and semantic consistency could provide a dimensional reduction method, which would probably be one of the key analyzing technologies in 3D GIS.

VI. CONCLUSION
This paper presents a semantic model based on CityGML, which is extended to support geological model and specified Spaces such as Stair and Corridor for indoor navigation.An integration tool is also proposed for enriching existing geometric models with semantics.At last, in a national 3D GIS project of China, two typical cases of visual exploration, i.e., the semantics-enhanced indoor routing and unified profiling of the complex city scene, are achieved by means of this semantic model.
Currently the ongoing project is developed to support most of the commonly used data type and the storage of massive city dataset in the database.Moreover, dynamic features like fluid and clouds are expected to be integrated in the future.

Fig. 1 .
Fig. 1.The tree structure of semantic classification

( 1 )
RoomA Room is a specified Space which should has functions other than passing through, such as resting, working or entertaining etc.It should be bounded by WallSurfaces (Similar to BoundarySurface in CityGML) like interior wall surfaces, ground surfaces, roof surfaces, Openings and Furniture as well as other IntBuildingInstallations.A Room contains a label to record the WallSurfaces, by which the adjacent Space is connected through Openings.In the topological network, a Room is represented as a node.

Fig. 8 .
Fig. 8. Lee's "Straight-MAT" method (a) and (b) show the calculated shortest path, represented, from outside and inside the building.In (b), the important features such as Stairs, Corridors and Doors are highlighted while other rooms are faded out.Fig. 10 (c) and (d) show the semantic tag for notifying features passed by during the navigation, which would enhance the impression of the route.

Fig. 10 .
Fig. 10.a) The calculated path observing from outside; b) The calculated path observing from inside; c) The semantic tag shows the corridor on the 2nd floor; d) The semantic tag shows the Room 204

Fig. 11 .
Fig. 11.The intricate and complicated intersecting-lines are difficult to be handled in other applications.

Fig. 11 .
Fig. 11.The extracted intersecting-line set of a geometrical model

Fig. 12 .
Fig. 12.The settings of a unified profiler

Fig. 15 .
Fig. 15.An example of user-defined profiler based on semantics.
Weiping Xu, Qing Zhu and Yeting Zhang State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, China II. AN INTEGRATED SEMANTIC MODEL