An Easy Approach to Integration for IWMS, BIM, GIS and Database - Part 1
October 17, 2011
Have you ever seen this place?
|
It is called Bachalplsee and it is a lake area in the Swiss Alps. The lake carries the name, but the lake is just one part of the place. There are mountains and land, flowers and sky. This is really not just a lake; it is an environment, a very relaxing looking environment. Just to be clear (for all of you scholars out there) an environment has been defined as: 1. The surroundings or conditions in which a person, animal or plant lives or operates. 2. The setting or conditions in which a particular activity is carried on. 3. The overall structure within which a user, computer, or program operates. While doing part-time work for a federal agency (that shall remain nameless to protect the innocent [and the guilty]) as a subcontractor, I was asked to work on an initiative that, until now has been hidden from the outside world. The initiative has been called SLiiC (pronounced ‘slick’). S L i i C stands for Structure for Linking information via integrated Components. SLiiC is an environment, similar to Bachalplsee, but built for a specific purpose. In accord with the second definition for environment SLiiC is not a solitary piece of software, it is not a spreadsheet, it is not integration but an environment that allows the activity of integration easily. SLiiC provides the structure for various programs to access standards, establish relationships between elements of business data, and exploit these to accomplish the aforementioned act of integration with greater ease. The various major parts of the SLiiC environment will be examined in future posts as well a couple of examples of its use. THE PROBLEM SLiiC ADDRESSES In 2005 two persons named Bertrand Diard and Fabrice Bonan released the first commercial open source data integration software to the industry. One would think that the initiative was doomed in that there were several commercial packages already on the market. However, this group was immensely successful, in fact in the 6 years since then, the company has become an international organization with a paying customer base of 2,000, an estimated 12 million downloads, and an open source user base of about 550,000. Additionally, funding from venture capitalists for Talend amounted to approximately $42 million. What accounts for the ‘success’? Talend identified a need that exists in most organizations. That need is affordable integration. These reports from the industry confirm an integration issue persists: 1. UMass Memorial Health Care: “We’re taking five years and spending more than $100M to overhaul our clinical and financial systems, and migrating selected data from our legacy applications to the new systems. . . . we are able to fully analyze all the data to be migrated, apply rules to ensure the quality of the data we do choose to move, and then move large volumes of data, reliably and efficiently, which was not possible before. . .” 3/17/2010 2. Mike Riley, Program Manager, Air Force Knowledge Services: “Before Air Force Knowledge Services [an integration entity in the Air Force], there wasn’t visibility on a timely basis. It might have taken weeks or months to extract that information manually out of all the stovepiped transaction systems.” (brackets ours) There are plenty of other examples, however the above suffice demonstrate that this issue is receiving more than a little attention. Additionally, the first two above citations involve extremely costly initiatives (in the millions). If Talend solves the integration issue, what need is there for SLiiC? Well, integration is not just an issue that affects applications that use imageless business data, spatial data (including IWMS [CAD], BIM and GIS) applications are involved. In fact, in recent years the attention toward integrating these environments has been heightened. Talend, while extremely powerful, does not integrate spatial environments completely out of the box. In addition, the tool may be on the complex side for some business users. SLiiC is made to be easy and is made not to link only database but also vector information. Some in the industry believe that ignoring the integration issue is the only logical recourse while accepting the level of integration possible today. While such a strategy may be the only option that some have, this is not the case industry wide. The SLiiC initiative has been started by some who feel that the issue can and should be addressed now. In order for this to be possible though, we found that a good structure needed to be in place. THE WAY SLiiC WORKS Consider the lake environment image above. The earth (land and rock) in the scene provides structure for both the water to reside and the plants to grow. Additionally, the earth (the planet) itself keeps the atmosphere close by its gravity. The structure for the SLiiC environment is defined (for the most part) in Microsoft Excel. Keywords are used in Excel to enable software components to understand just how information is linked to elements in the software component environment, or to databases, as they access the SLiiC Structure. A linking utility is used to allow the user to define the way information should flow from one environment to the other using simple drag and drop. Confusing? Then consider it this way: In a simple game of catch what are the requirements for the game to be played? There must be a ball and at least two people. Is that all? Let’s make it a little more like communication in the industry today. We want to play catch but one player has hands and the other (for whatever reason) has a couple of sticks. How easily is the ball going to be caught by the second player? In the same way the industry is passing information from one environment (for instance BIM) to another environment (an FM database). On one side, BIM is using bare hands (raw data and reports), on the other the FM needs to be told to put the sticks down, and use your hands to grab specific information in this specific way. SLiiC creates a “hands-on” game, ensuring each of the players use their hands. The SLiiC Linking Tool is used to tell the players not just to catch, but what direction the ball is coming from, how big it is, how fast, and what to do with it once it arrives. Game on! Let’s talk a little bit more about the structure in SLiiC. SLiiC STUCTURE As mentioned, SLiiC uses Microsoft Excel to structure the interchange of information. As mentioned before, this structure will be considered in detail in a future post. Great integration initiatives in the past have endeavored to use Microsoft Excel as a least common denominator for integrating facilities data. The SLiiC team learned from the COBie initiative, for instance, how powerful this underestimated resource can be for integration, as almost anyone can use Excel and feel comfortable. However, SLiiC’s structure exists to enable the absorption and enforcement of standards (including COBie). Unlike some rules analysis tools, SLiiC is made to bring standards inside of the editing environments to enable small scale analysis while editing, not after the work is done. This can be really helpful for BIM. SLiiC looks for worksheets to be named specifically, defining the constitution, use and relationships between data. Sheets are required to specify the (1) Database (2) Pick Lists (3) Link Tables (4) Relationships. This structure provides a few things: 1. Database sheet(s) provide information on the table(s) and fields of interest. These can refer to information in a database or information separated by various sheets of a workbook. This is done by naming the table(s) and the subsequent fields that are to be integrated. 2. Any “pick-lists” (or drop down lists) have their values defined in a sheet called LimitLists. 3. Any relationships between values in multiple pick-lists can be stored in a sheet called LinkTables. These relationships can be “one-to-one” or “one-to-many” (if you understand database speak). 4. Finally, all of the relationships between various data elements are defined in a sheet called Relationships. This is the SLiiC structure in a nutshell. Although deceptively simple, SLiiC allowed us to do much more than we expected. We were able to specify and require the use of standards as lengthy as OmniClass tables in the pick-lists and as complicated as the spatial data validation that GSA seems to endorse. We were able to absorb all of the COBie tables in a CAD environment and turn around and do the same in BIM, once COBie was defined the SLiiC way, without any additional work. With this simple structure we have created the foundation for an environment where standards can grow and change and we are endeavoring to enable integration in a consistent way across vector and database environments. SLiiC LINKING Where is the water in the Bachalplsee lake image? “In the lake?” Well yes. . . but there is a more complete answer. The water in the lake is just part of the liquid story there. The lake water is saturating the earth around it, even the rocks, and as it does it brings vital nutrients to all living things around it, all of the components of this environment. In essence the water links almost everything. Just as the Bachalplsee environment has the lake, SLiiC has a linking tool as well. While it is not as seamless (or beautiful) as water is, it is a very powerful tool for bringing data to the appropriate places for it. This linking tool is designed to allow business or technical integrators an easy medium to specify all features that will be saved into the SLiiC structure. A person can easily just drag and drop the information that links, delete relationships that are not relevant, create templates for report output, and specify a host of other operations without opening Excel. All of the information about the integration or integration metadata is saved in Excel though. (As with SLiiC structure, this linking tool is going to be examined in a future post dedicated to it.) The output from this linking tool is digested by the components of SLiiC in various applications. However, this linking portion of the environment is independent from any and all components so that these can be developed at different rates by different teams. SLiiC COMPONENTS Please take note of the beautiful flowers adorning the edge of the lake. These depend on both the lake and the structure of ground and rock for survival. In the SLiiC environment there are components that read from the SLiiC structure and the Excel product of the SLiiC Linking tool to allow users specific integration activities in various applications. These applications include: 1. BIM Environments (Currently a Revit component is in development, IFC is planned using an IFC viewer and Excel) 2. CAD/GIS (Currently an Autodesk Map component is in development, ArcGIS Explorer is planned) 3. Databases (Microsoft Excel is in development and Word is in planning) One benefit of this approach is that we are able to use a consistent way of designing these components (where possible) so linking, reporting, complying with standards specified in Excel always have the same look and feel no matter which environment they are in. We are also attempting to maintain consistency with the design of forms and windows between SLiiC Components. Another benefit is that as components read data from the SLiiC structure, they can define the best way to create objects and data in their native environments (using the native APIs) and redefine these as necessary to be integrated with the greatest accuracy. Other components in SLiiC can create the placeholders for such data in environments that they have been developed in, and ensure that such data be transported into the appropriate places. (NOTE: SLiiC is currently focused on non-vector relationship mapping. There is a goal to focus SLiiC on vector integration as well, as resources become available.) The benefits mentioned above will further simplify the activity of creating and using data-to-objects and data-to-data relationships. (Existing and Planned components of SLiiC and their inner-workings will be considered in a future post.) As SLiiC continues to get developed, it is the hope of the SLiiC team that components will be developed independently of my employer, perhaps by the industry. It is also hoped that interest will arise from government agencies, universities and the private sector to contribute their insight to the further development of the structure SLiiC is employing. CONCLUSION |
While SLiiC is an environment, it is obviously not a recreational area. Although it will provide some relief in integration, significant work still needs to be done (and is being done) on the integration front both with SLiiC and across the industry. Those seeking to participate with the development of this novel approach to integration can contact me here through email. If there is significant interest manifested, the agency may consider opening the initiative up to several other agencies or large corporations, perhaps as an opensource initiative. Stay tuned please for more information on the initiative and thanks for your consideration.
-JK
Posted by Horatio McDowney. Posted In : Integration Concepts