Improving Virtual Reality Applications in CAD through Semantics

—In this article we propose an architecture for the inclusion and exploitation of semantic aspects in a CAD environment. Our schema focuses on the enhancement and improvement of a commercial (general purpose) CAD system on its Virtual Reality (VR) capabilities. There is a need in combining CAD and semantics in the new generation of computer assisted engineering systems. Usually semantic aspects in traditional CAD applications relay on the user’s tacit knowledge, a fact that slows the learning curve while the user is getting trained with a new tool, no matter if he is an expert in others. Moreover, current CAD programs offer a myriad of options to the designer, but their limited integration of the non geometric information (e.g. PDM systems), and in general the semantics of the design process, can still be improved for the current needs of the industry. We argue that the use of semantics could improve the workflow and capabilities of the software and would benefit a new user (non experienced one) in order to produce better results in shorter times. In this paper our main focus is the enhancement of the visualization and VR capabilities in generic CAD applications. Our semantic CAD framework uses ontology modeling as well as engineering standards in order to conceptualize and exploit the information contained in a CAD model. To show some benefits of our proposed schema, we present two cases in different engineering domains (plant design and Steel detailing design), where the inclusion of semantics empowers a VR experience.


I. INTRODUCTION
In most engineering companies, the information related to a product design usually starts growing right from the designer's computer. There are several computer oriented applications that play a big role in design stages. Between such applications, CAD (Computer Aided Design) programs are a common tool in almost every design workshop. From its original approaches as 2D pseudo sketch applications to the actual 3D versatile high-end suites, capable of stress calculations and simulation, CAD has been evolving along with the new technologies appearing in the market. General purpose CAD suites (e.g. AutoCAD, MicroStation, etc) do not consider the domain ([13， 14]), although special add-ons do exist for specific needs. These packages are focused in the geometries and topologies of the elements in the design, providing a myriad of tools to the user to handle operations such creation, modification and basic Manuscript Received on September 15, 2006. This work has been partially financed by Basque Government through the MiroView Project (INTEK-4140/2004) and the European Network of Excellence AIM@SHAPE, IST project 506766 .The exchange of researchers during this work was supported in part by the German Academic Exchange Service (DAAD) and the Spanish Ministry of Education. querying. However, there exists information in the model (e.g. operational parameters, vicinity, etc), whose nature is not entirely geometric. That information can be modeled, implied, queried and handled using ontologies. The consideration of semantic brings new possibilities in traditional computer applications brings new possibilities as it increases on the one side accurate knowledge sharing, and on the other side the reliability and performance of such systems.
The main advantages of using embedded semantics are: (i) the improved information management, (ii) searching and sharing enhancements, (iii) the empowerment of the intrinsic knowledge embedded in the elements being described, and (iv) empowerment of the user. In this article we propose an architecture for the inclusion and exploitation of semantic aspects in a CAD environment [19]. Our schema focuses on the enhancement and improvement of a commercial (general purpose) CAD system on its Virtual Reality (VR) capabilities. We argue that the use of semantics improves the user's workflow allowing him to focus in designing task, lowering labor cost and enhancing the efficiency of the design stage. Our semantic CAD framework uses ontology modeling as well as engineering standards in order to conceptualize and exploit the information contained in a CAD model. To show some benefits of our proposed schema, we present test cases in different engineering domains (Plant Design and Steel detailing design), where the inclusion and use of semantics represent a clear benefit for real world industries. This paper is organized as follows: In section 2, a review of related works and basic background is presented. In section 3, we present our architecture for the inclusion and exploitation of semantic aspects in a CAD environment. Our schema focuses on the enhancement and improvement of commercial (general purpose) CAD systems on its VR capabilities, mentioning some highlights about the implementation process. In section 4, we present some test cases in different engineering domains that have been benefited by the use of semantics, all developed within our proposed schema. Finally in section 5, we present our conclusions and future work.

II. STATE OF THE ART AND BACKGROUND
CAD systems can be classified in generic (general purpose) and specialized systems. Some widespread systems have a reasonable price; but their lack of functionality when compared to high end systems is notorious. It is true that high-end CAD systems have already excellent visualization tools, but several CAD systems still depend on converting the model to a VR format, such as VRML [20]. When the models are not very big, these tools of direct conversion usually work acceptably well.
However, this conversion process is not addressed with enough detail for large models in normal working

Improving Virtual Reality Applications in CAD through Semantics
Carlos Toro, Jorge Posada, Stefan Wundrak and André Stork environments. As a result, in many cases, the interactive visualization of such a Large model in VR fails, since it takes to the limit the resources of the computer (memory, processing power, etc.), making it unusable in normal circumstances. A problem that usually appears is the loss of information during the conversion process to VR model. CAD models store a big amount of information, including (but not only) the geometrical and visual representation. Actually, in some cases, a relevant part of the dataset is invested in complementary information, not directly visible but very useful, such as the organization of the graphic elements in a level-tree or the information associated to the characterization of the different parts. In some CAD systems, after the conversion, a complete model becomes a mere graphical scene in which it is only possible to access the visual geometric information; the access to the rest of the information is only possible in the CAD format, so the VR model is not as useful as it could be. In some other cases ,the information associated to such fabrication processes, design tasks and element relations is missing in the VR environment (typical case of generic CAD Systems).

A. Virtual Reality in Generic CAD systems
There is a clear established evolution in CAD systems, which is a good basis for their VR representation. An interesting extension is the development of visualization tools that allow the visual representation of the information in a way as close as possible to the reality. In this sense, VR is a quite logical solution, since it allows a more "real" perception of the model, bringing important aspects to the visualization such as immersion, advanced interaction, and three-dimensional representation. Several researchers have addressed the visualization of large CAD models in VR, mostly in specialized computers, but sometimes even in PC environments. To mention three efforts in this direction, Aliaga and Manocha ( [2,12]) have presented a framework for the visualization of massive CAD models that combines different techniques, proved in the cases of submarines and process plants. Hubbold [7] presents also a technique for the visualization of large CAD models in a normal PC by means of introduction of landmarks to support navigation. For an extensive review of similar systems and the techniques used, the interested reader is referred to the work by Manocha [12].
In fact, some systems already allow the integration of CAD models into VR environment, but they present some practical problems, particularly connected to the necessary resources for the right visualization of large models. It is important to remember that CAD systems are used in a wide range of companies (including many small and medium industries) and most of them do not have access to very specific set-ups specialized in visualization, such as a CAVE, ImmersaDesk, Iwalls, etc. [12] or high-end graphic machines with advanced interaction devices. Usually the degree of sophistication they can afford in a normal workplace is a personal computer for general purpose with just decent graphic capabilities or a small network of them (usually not fully exploited for parallel rendering or advanced visualization). Also, although some CAD systems provide very advanced visualization capabilities based on proprietary information, several commonly used CAD systems do not give these possibilities. Thus, visualization of very large models is still not feasible in the normal workplace for many users.

B. Large model Visualization of Industrial Plants in VR
There are four families of techniques used commonly in interactive walkthroughs of large databases [12]: i) rendering acceleration techniques, (ii) database management, (iii) interactive collision detection, and (iv) system integration. As seen in [12] the main acceleration techniques used are basically visibility culling, object simplification and image-based representations. Geometric simplification techniques e.g. Levels of Detail (LOD), Hierarchical Levels of Detail (HLOD) [13] give good results in handling massive data sets; the integration of LOD and good occlusion culling techniques are usually the key factors to achieve interactive rates in walkthrough systems [12]. On the other hand, there are emerging commercial applications (e.g. NavisWorks, Mantra4D [14]) that incorporate the latest graphics hardware accelerations as well as many of the classical culling and simplification techniques with good results. In a direction similar to the first part of our work, Shikhare [17] describes an algorithm for automatic discovery of repeating geometric features. The collected information is then used for compressing the model by removing the redundancies on its representation, although it doesn't take into account the semantic significance of the repeating structures found. In general, we found that in most approaches the semantic implicit in the geometric representation is not considered for the model, but assumed, no matter what the user's motivation or background is.

C. Semantics Aspects in Engineering (a short overview)
Different approaches related to the application of semantic aspects in different engineering domains have been presented by the R&D community in forums and conferences. Bedell [3] presented a system whose main component was an interactive design for the support creation and edition of 3D hierarchical detail graphic objects. This early work used databases through the data model compiler (DMC) which is a project of the University of Connecticut. DMC allows the development of database management systems corresponding to a data model specified by a data model designer. The work of Bedell being an interesting approach is nevertheless based on a taxonomy point of view while our approach is ontology based allowing us a better use of the semantic load embedded in the CAD model. Ketabchi [10] developed ODM (object-oriented data model), which attempts to increase the productivity of computer-aided design operations by integrating functional data models, semantic data models and actor models. As many of these early works, Ketabchi´s approach was strongly based on a CAD database model with a disadvantage for the user in the sense that every relation between the elements must be declared explicitly in the database model. In recent approaches, Razdan et al [16] used ideas extracted from the multimedia retrieval and storage domain, where 3D digital libraries are used as components of a model. Adding semantic information to raw 3D data (an element of the future library) is done by the segmentation of the 3D object into a set of bounded regions or features based on curvature. These regions are targeted with a discipline specific nomenclature, adding in this stage the semantic information to, later on and via a visual query system, retrieve, visualize and analyze in 3D the information. The storage model uses an XML schema to keep the semantic content. Rappoport [15] presents a very general outline called the Universal Product Representation (UPR) architecture, whose main idea is to provide support for all data levels employed by today's CAD systems. In his approach, parametric feature based design is focused as the main goal. The engineering standards (STEP and IGES) where used by Rappoport extensively to allow the interchange of features between different CAD applications in a sort of storage model. Bidarra [4] presented a work where the semantics of features are defined and maintained during all modelling operations through a specification of feature classes, the structure and functionality of the feature model and the model validity maintenance schema are described in his work as the corner stone of his feature model approach. In the work by Posada [14], semantics are used as a basis for the consideration of user profile, intention and available resources in the Large model Visualization of massive CAD models representing industrial facilities. In the European project SpacemantiX [18] a series of semantic based tools focused in different domains (mechanical, automotive, interior design and architecture) where presented.
By using semantics as well as spatial information, the system defines which components can be placed and their location and interactions and how the overall design should be configured. The outcome is an easiness of the design process for expert and non-expert users alike. In a similar direction, our approach handles the semantic information using ontologies. A recent European network of excellence called Aim@Shape [1] has [as should be deleted ] its main goal to advance research in the direction of semantic-based shape representations and semantic-oriented tools to acquire, build, transmit and process shapes with their associated knowledge. Their focus foresees a new generation of shapes in which knowledge is explicitly represented and, therefore, retrieved, processed, shared, and exploited to construct new knowledge. In the next section, we will shortly review some background related to ontology modelling and the Ontology Web Language approach.

D. Knowledge Representation Using Ontologies and the Ontology Web Language (OWL)
An ontology is the explicit specification of a conceptualization [6]. In simple terms, it is a description of the concepts and relations in a domain for the purpose of enabling knowledge sharing and reusing. A body of formally represented knowledge is based on a conceptualization: the objects, concepts and other entities that are assumed to exist in some area of interest and the relations that hold among them [4]. The existence of tools for ontology definition, querying and reasoning gives interesting possibilities to several application domains, including the engineering domain. A widely used ontology language is OWL, which is a semantic mark-up language for publishing and sharing ontologies on the World Wide Web [11]. OWL is developed as a vocabulary extension of RDF (the Resource Description Framework) and it is derived from the DAML+OIL Web Ontology Language allowing more expressiveness than RDF depending on the OWL flavour used [9]. There exist several OWL API's >APIs to handle computationally an ontology, between the most known ones, we can mention the Protégé OWL API [14], KAON2 [9] and the WonderWeb OWL API [21].
In the next section we will use these new possibilities offered by the knowledge representation techniques to improve the VR environments in some engineering applications.

III. AN ARCHITECTURE FOR THE ENHANCEMENT OF VR EXPERIENCES IN A GENERIC CAD ENVIRONMENT
In the general purpose CAD packages, the use of explicit semantics is almost non-existent. Our approach allows exploit semantics in CAD typical scenarios following a simple-to-implement architecture. This architecture is used as a guideline for developing extension to the original CAD system via its API in order to extend the VR capabilities of the software.

A. An Architecture for the Semantic Enhancement of VR in Generic CAD Applications
The architecture is divided in five layers (Fig. 1) in order to show a clear differentiation between the different conceptual levels involved, prototype implementation was implemented and its results can be reviewed in section 4. The purpose of this architecture is the seamless integration between the different components needed (CAD API, Ontologies, User Interaction, etc) for the semantic enhancement of the VR experience. The User Layer is in charge of the bidirectional interaction between the user and different application modules of the system, this layer handles the user-system interactions. This layer can be implemented as an external application (stand alone) that is able to share data and communicate between the VR environment and the CAD kernel.

1) Modules description
The second layer is called the Application layer and contains two modules, being the first one the 3D CAD system itself (with its API) and the knowledge-based VR tools module, that contains a knowledge model of the VR tools available (e.g. stereo capabilities, specialized hardware handling, etc).
In the third layer called the Reasoning layer, the architecture includes two modules that are in charge of the semantic reasoning. At one side the rendering strategy module that contains a reasoning engine whose main inputs are the user type, the computational available resources, processor speed, available RAM, max Ktris per second. The second module in this layer is called the CAD reasoner, whose main activity is to perform geometric reasoning about the elements contained in the actual CAD model. Among others, some of the geometric reasoning activities include geometric similarity between elements, handling of cells or groups of elements and parts handling at a parametric level.
The next layer is called the Semantic abstraction layer. Its main purpose is to store the actual ontologies that will be instantiated with the information from the model. This layer is divided in two modules: the first one is called the Rendering Techniques ontology which contains an ontology model of the different (CG and /or semantic oriented) techniques available to produce the VR output. The second module within this layer is called the Model ontology. It contains the standard based compliant conceptualization of the model domain (ISO STEP, CIS/2). The last layer is called the instances layer and contains three modules; the tessellated model module, which is the triangulated model, generated using the chosen rendering techniques suggested by the reasoning layer. The Model Instances module contains the categorized list of elements from the model according to the geometric definition given in the model ontology. That is, a collection of elements whose characteristics are similar (e.g. in the context of Industrial Plants, groups of elements of the ISO STEP 10303 AP227). The third module contains the CAD instances, in other words, the actual elements that belong to the CAD model.
It is important to mention that an application that follows this architecture can use some of the modules and not necessarily all of them.

2) A short note about ontology modeling based on international standards
A 3D complex model in most cases has representations of pre-defined engineering parts. These elements are described using standards. ISO STEP-10303 ([5, 8]) is the international standard for the computer interpretable representation and exchange of product data. The objective of STEP is to provide a neutral mechanism capable of describing product data throughout the life cycle independently from any particular system. The nature of this description makes it suitable not only for neutral file exchange, but also as a basis for implementing and sharing product databases and archiving. We have modelled our Process and Model ontologies based on the ISO-STEP standard because our ultimate objective is to have a system where the concepts and relations of the domain can be handled and queried using semantic criteria, beyond the mere data modelling structures of the norm. This improves the generality of the presented approach, as the elements and their characteristics are not described following a widespread agreement between a well recognized group of users, engineers and industries. Of course, some properties can be completed/extended, as we found out that certain elements described by the standard are viewed from a mere functional point of view (e.g. valve element in ISO STEP 10303 AP227).

3) Interfacing a CAD applications and ontologies
Most of today's CAD programs offer an API in C++ to extend the system capabilities (ARX in the case of AutoCAD, MDL in the case of MicroStation, etc). On the other hand ontology programming is mostly done through a Java interface due to historical reasons. Interfacing C++ with Java is a technical problem that could be partially solved using extensions like JNI (Java Native Interface) that allows Java code to be called from a C++ based application. Other possibilities include CORBA (Common Object Resource Broker Architecture), Sockets, and more recently Web services technology.
In order to communicate with an API for ontology handling (e.g. protégé OWL API), we developed an interface that allows the ontology instantiation of elements from the CAD program, and the querying of the model from both environments. The interface uses socket technology in order to send and receive tokens that represent function calls and returns. Using this technique, we are able to handle OWL API manipulations programmatically right from the CAD coding environment.

IV. APPLICATION CASES
In this section, we present two application cases based [in>on] our proposed architecture. These test cases where implemented during two research projects held in cooperation with leading industries and research centres in Spain and Germany. In the first project, the main idea was to develop a Large model Visualization system for the real time navigation of industrial plant facilities using user, model and resources information following a semantic approach. The second project's intention was the development of a series of semantic based tools to support the structural design (Steel Detailing) process and one of the tools was specifically the development of an advanced VR viewer to extend a generic CAD. Within the development held during the mentioned projects, we present different application schemas of the CAD enhancement approach introduced in this paper.
In the next table the relation of tools used in both projects is presented: A. Case 1 -Using Semantics for visualization purposes of large industrial plant models As mentioned before, in Large model Visualization, the basic idea is to navigate smoothly through a CAD model containing a large data set of 3D elements (most of them repetitive). Several techniques can be applied (e.g. Occlusion culling, Drop culling, Levels of Detail, etc) [7]. The LMV problem is common to ship design, Graphical Information Systems (GIS) and Plant Design.

1) Semiautomatic reconstruction of conceptual components /parts (Catalogue Reconstruction)
The geometric objects in CAD models of Plant Design are produced mainly by occur in the application domain (in this case, Plant Design). This tool is used to store the information about instances of elements in the model and to set up a relation between repeating elements and their possible matching concept in the ontology. An example of partial catalogue can be seen on Fig. 2. The catalogue has to be semi-automatically filled using structural information of the CAD model and the knowledge of the user, who interacts in the process [13].  The model ontology in this case was modelled using the STEP Application protocol 227 that describes the elements in an industrial plant facility, the CAD reasoner performed geometric similarity operations between the modelled parts and the CAD instances and the 3D CAD used was MicroStation extended via the provided API (MDL). In this case, the process ontology was modelled to reflect design review needs.specialized software that specifies how standard components/parts are located and defined in the actual model. However, the final CAD model itself does not keep this information in an explicit manner. Based on the model characteristics, we look for conceptual parts/components that Left to right: elbow, T-joint, Clamp, Valve.

2) Selective part simplification on a Per-Part Basis
Probably the most effective technique of the current implementation is the ability to define rules for applying different optimization techniques on a per-part basis. Using the information of the reconstructed catalog, the export system has control over the export process on a much finer granularity; instead of applying simplifications techniques globally it can now be decided on a per-part basis which technique with which parameters to apply. The decisions are made considering information available through the semantic-triangle [13]: information about the model, the user and its intention, and the resources. The logic for exploiting this decision is encoded in a set of semantic rules. Two schematic examples of rules (not following any particular notation) are: IF part = part.piping.valve.* AND user = engineer THEN REPLACE part WITH valve-symbol IF part = part.piping.clamp.* AND user = manager THEN DROP part The model instances in this tool are the reconstructed instances of the model. Following a similar implementation, the model ontology was STEP 10303-227, the CAD reasoner performed geometric similarity operations between the modelled parts and the CAD instances and the 3D CAD used was MicroStation extended via the provided API (MDL). The Process ontology was modelled to reflect design review needs.

3) Conceptual 3D Symbols
Once the catalogue reconstruction is done, semantic information about parts of the model is recovered. In other words, the available implicit information has been made explicit. A technique that exploits this fact is the replacement of complex parts with conceptual 3D symbols that are much faster to render and nevertheless hold the semantic information associated by the user. The conceptual symbols are, so to speak, synonyms for parts. Fig. 3 shows a part of the model containing an element from the class elbow. In the upper image, the elbow is drawn with full detail (as it was a normal VR output), in the lower left image, the number of segments is reduced keeping the semantics (for a user is still an elbow) but reducing visual quality in order to fasten the visualization. In the lower right image, the same elbow is rendered for a non engineer user; the clamps (the pieces that grab the elbow to another sub-structure) are dropped from the model.
The semantics are still kept as the element is still recognizable as an elbow, the sole fact that we are not drawing the clamps saves us around 300 triangles per clamp and taking into account that[ the>should be deleted ] these clamps doesn't add any information to a non technical user, the overall result is a real triangle saving technique as in an average model the number of clamps could easily exceed one thousand.
For this Tool, the ontology reasoner plays a decisive role as it is charge of the logics related to the correct choosing of the semantic symbols. The Model instances and CAD instances are strongly related as the actual CAD model changes upon the ontologies recommendation. The concept of the semantic triangle (user-resources-model) and its implications is explained in [13].
In Fig. 4, can be seen the set of modules present in the implementation of the tools in both test cases for the Plant design test case. In this application case, the main idea was to develop a series of semantic based tools to support the structural design process. The main tool developed was a Design Review Module. In this application we implemented not only the traditional LMV techniques presented in the literature, but also the semantic compression module presented in this article. The visualization tool is a stand-alone application and includes tools for pan, zoom, and navigation in real time. The collection of cells in a tree hierarchically structured appears at the left side of the scene, allowing the selection of a given cell or group.

B. Case 2 -VR enhancement of a generic CAD for Structural design using semantics
Other possibilities present in the viewer are the per-part identification; the seek function and the possibility to manipulate the parts (move spatial position, scale, etc). For the Ontology modeling we used a standardization effort called CIS/2 which is based on deliverables of the Eureka EU130 CIM Steel Project and is an extension to the general STEP model for the specific case of the Steel Detailing industry [5]. CIS/2 is an extended release of the CIMSteel Integration Standards (CIS), a set of formal computing specifications that allow software vendors to make their engineering applications mutually compatible. The CIS/2 documentation specifies what information may be transferred between software applications, and how that information must be structured in a repository or data exchange file. CIS/2 substantially extends the engineering scope of CIS/1, and introduces advanced data management capabilities to enable data sharing. At its simplest, the CIS provides specifications and guidelines for the development and implementation of translators that enable the users of engineering software to export data from one application into another. However, CIS/2 also allows software vendors to support concurrent engineering via more direct mechanisms for information sharing and management. Thus, CIS/2 also provides specifications and guidelines for the development and implementation of Database Management Systems (DBMSs) built around the CIS and its related technology. Such a DBMS is known as a Product Model Repository (PMR). Fig. 5. Implemented modules following our architecture for the second test case -(1) Model simplification and camera dependent synchronization tool 1) Model simplification and camera dependent synchronization. In this test case, the biggest enhancement is the object recognition and matching with the elements contained in the CIS/2 ontology (domain ontology) that allows the simplification of the VR output and a synchronization between the cameras present in the walkthrough and the CAD itself in a seamless approach to handle the movements in the VR extension and obtain the same view in the CAD once the user reaches a desired point of view. In Fig. 5, an exemplary tool developed following our architecture is shown in the shaded area.
This tool scans the 3D model and classifies the CAD elements into CIS/2 compliant classes and then simplifies the geometry representation by choosing the appropriate rendering techniques (according to semantic criteria) in order to produce a tessellated model suitable for a enhanced VR output. There are several complex structures that are used repeatedly throughout CIS/2. Most of these structures come from the integrated resources that are common to all ISO-STEP product models. In Fig. 6, a screenshot of the VR structure can be seen. Finally, for both application domains (cases), the VR output was ported to a setup with passive stereo and back-screen projection, using a SpaceMouse as input device (Fig. 7).

V. CONCLUSION
In this work, we presented an architecture that allows the enhancement of generic purpose CAD VR capabilities through semantics. We presented also two implementations of the mentioned architecture in different domains with their application domains and model ontologies based on international standards. Our architecture implementation allows the semi automatic recognition, and semantic simplification of the elements contained in the model. We introduced semantics into the visualization of VR large models by applying semantic criteria to traditional optimization techniques. This new approach could lead to an improvement in quality and performance of most current techniques in visualization. As future work, we will explore the collaborative design possibilities of our architecture and continue the exploration of new tools and applications in different domains where CAD is applied (e.g. GIS Systems, Architecture and urban planning, etc).