Navigating the inter-application  zone

As e-business becomes the norm for many industries, the days of the stand-alone computer application may be coming to an end. What transpires between systems is often as important as what goes on in the applications themselves. Adjusting to this new reality will require an unprecedented level of cooperation between business stakeholders and IT.

By Jacob Stoller

We sleep at night with the confidence that if a disaster occurs, the right people will be phoned at the right time. In Alberta, that critical call could depend on a middleware link that joins two computer applications: an emergency management system located in Edmonton, and an automated telephone centre based in Burnaby, B.C.

The ability of this critical integration link to relay to a remote system exactly whom to call, and when to call, is an example of how sophisticated application integration, and the middleware that supports it, have become. This particular system, owned by Edmonton-based TELUS Geomatics, provides emergency notification services for the Alberta Government and a number of clients in the energy sector.

Pointing out the need for reliability here is an understatement. As the intelligent “go between,” the link has to be able to check for errors and missed calls, prevent security breaches, and provide audit trails and critical reporting on call-out incidents. To do this, it has to “understand” much of what happens in the system that determines who gets called, and what happens in the system that places the automated calls.

The Edmonton-based core system uses a combination of geographical and meteorological information to assess disasters and threats such as forest fires, floods, and environmental threats such as toxic equipment fires. The assessment identifies affected areas, and automatically initiates calls. Type and severity of the incident, location of sensitive infrastructure such as phone lines, or wind direction can all figure into automated decision processes. “If the wind is blowing east, you only phone the people on the east side of the fire,” says Jim Huff, TELUS Geomatics’ general manager.

The correct interpretation of commands depends on the establishment of precise business rules affecting the interaction of the systems. These are codified, pre-programmed, and tested for accuracy through an interactive process between people who understand the business processes and IT people.

The link itself was built using Microsoft Biztalk, an integration platform widely used in Canada due to its suitability for the smaller-sized companies typical of the Canadian business landscape. Implemented by Calgary solution provider Five by Five Software Ventures, the integration platform is giving TELUS Geomatics the flexibility to grow the system, and to support a number of new clients with a variety of needs.

Managing conduits of information between applications is becoming a core competency in what could be called a new IT era. Most organizations have a long way to go to transition from traditional siloed information systems to highly interactive e-business systems in which information is shared globally, around the clock. This challenge is demanding new skills from IT, and an increasing level of involvement from business stakeholders.

Integration drivers

Not all integration situations involve life and death situations, of course. More typically, it’s used to achieve greater efficiency. Montreal-based aircraft manufacturer Bombardier Aerospace was able to streamline processes in their plant by integrating an older legacy warehouse management system with a more recently implemented SAP system. According to Khalil Nasrallah, Bombardier’s manager of emerging technology, the problem was that workers on the shop floor had to re-enter barcodes when a part passed from one system to the other. “We didn’t want the worker in the plant to have to use two systems,” says Nasrallah. By integrating the systems with IBM’s Websphere Business Integrator, problems of this sort were eliminated, along with the accompanying risk of error.

A product like Websphere Business Integrator is a substantial investment that requires more than a simple interface to justify. Nasrallah had to convince senior management of the long-term benefits of creating a sophisticated integration platform. There were several key factors. From an IT standpoint, the platform is easy to maintain, helping to keep IT’s long term integration costs under control. Secondly, it offers reporting capabilities that help Bombardier comply with audit requirements, such as those imposed by Sarbanes-Oxley. Finally, having a robust integration platform puts Bombardier in “ready mode,” able to implement integrations quickly when business conditions dictate. True to Nasrallah’s predictions, Bombardier has moved on to leverage the capabilities of the Websphere Business Integrator implementation in projects involving their HR and financial applications, and has facilitated tighter integration with business partners.

Like Bombardier, many companies see the need to eliminate manual data entry as the first step in setting up an integration platform. Re-keying data from one system to another is astonishingly prevalent; according to a brochure published by the Business Application Software Developers Association, “95% of documents that are used for data entry are computer gener-ated.” As organizations strive to do more with less, this kind of largely redundant manual work is a huge and obvious target for cost savings. Because repetitive manual work is not only expensive but error-prone, integration can also result in noticeable quality improvements.

Another common business driver for integration is the interactive nature of many IT projects. Enterprise applications such as customer relationship management (CRM), supply chain management (SCM), and enterprise resource planning (ERP) are by their very nature dependent on sharing data and business processes with other applications, both inside and outside of the company’s walls. To take supply chain management as an example, being able to view the supply chain in “dashboard” fashion means having live connections to procurement, warehousing, transportation, and suppliers’ IT systems. As a result, often half or more of the implementation budgets for projects of this sort go into integration.

The decision to implement an integration project may not be internal; the issue could be forced by a major customer or partner. Nasrallah cites several examples at Bombardier. In one case, a major airline announced that they would no longer place parts orders by phone, but needed to place them automatically via a Web interface. Another source of pressure to integrate is an outsourcing arrangement with U.S. firm CAT Logistics. CAT does all the picking, packing, and updating of the parts distribution systems, meaning that real-time links have to be negotiated between Bombardier’s and CAT Logistics’ IT systems. In one instance, Bombardier had a system for sending faxes about certain information updates that had to be upgraded to an automated real-time link to satisfy CAT Logistics’ requirements.

Small steps

Integration isn’t just about cutting costs and complying with the demands of customers and business partners. Some companies are leveraging interoperability in highly proactive endeavours, such as expanding their reach in the marketplace, or the augmentation of their capabilities through multiple partnerships. BC-based Mark Anthony Group, the manufacturer of Mike’s Hard Lemonade and a number of other beverages, is a prime example of this kind of strategy. Mark Anthony’s core competencies are the management of their brands and their supply chain. They own no facilities to manufacture, bottle, ship, or distribute the product; this is all done through partners. Furthermore, they leverage a relatively small organization to distribute their product in markets throughout Canada and the U.S.

Mark Anthony is building an integration platform based on Microsoft Biztalk, implemented by BC-based integrator Sunaptic Solutions. This gives Mark Anthony the tools they need to tightly monitor parameters such as inventory levels, distribution patterns, and product formulas through direct communication with a wide variety of IT systems. It also means the company can keep critical information flowing 24/7, so it’s more responsive to retailers in multiple time zones.

While integration is a no-brainer for companies like Mark Anthony, it is no panacea. As Sunaptic President Mike Hilton points out, developing an integration platform is a huge commitment that may not be justified. If the integration is just a “one-off,” such as a link to allow e-filing of tax returns or audit information, there are cheaper routes available. For example, at a fraction of the cost of an integration platform, a company could use what is known as the hand-coded approach. This involves engaging a programmer to create a single interface using standard protocols and tools. Hilton advises looking at the complexity of the processes involved, and the volume. Sometimes it’s even best to stick with a manual process. “We were approached by a customer whose transaction volume only turned out to be seven invoices per month,” says Hilton. “In this case, why automate?”

With Mark Anthony and other customers, Sunaptic takes a conservative, modular approach, implementing successful sub-projects, and then looking for the next win. “We use a divide and conquer approach,” says Hilton. “We look for situations where we can measure the business value of the project, and see change by the end of a quarter.” A current example at Mark Anthony is the use of the integrated system to distribute product formula updates to the production plants. This is expected to result in measurable benefits due to the elimination of a manual, fax-based system that required a considerable degree of babysitting.

This practical, step-by-step approach to integration typifies the business-driven orientation of most projects today. This is a far cry from the large scale Enterprise Application Integration (EAI) strategies that were advocated, and sometimes attempted, during the so-called dot com era. The fallout from the earlier approach has given application integration a bad name, according to Five by Five Software’s vice-president of sales and marketing, Neil Kenyon. As a case in point, Kenyon reports that a customer once said to him, “I don’t want integration, but if I could get my applications to talk to each other, that would be great.”

From a practical standpoint, the business driven approach involves the identification and codification of business processes that will be automated once two or more applications are integrated. The processes in question may be simple, such as a credit verification on a new customer, or an inventory level check for specific part numbers. Or they could involve a complicated procurement process based on a number of complex business rules.

Risks and exceptions

Mapping out the processes involves substantial input from the business process owners, and typically involves having a business analyst sitting side by side with end users. “You logically go through every step of the end user’s requirement,” says Kenyon. Care is taken to make sure no details are missed. “Where most projects go awry,” says Kenyon, “is in the requirements and the scope — what we call scope creep - where somebody says ‘we forgot about this’, or ‘let’s add that’”.

Codification can be a tricky process. As Kenyon points out, “with integrations, exceptions are always there.” For example, expense reports over $20,000 might have to be approved through a special process. If no allowance is made for the exception when the process is automated, a $21,000 expense report “could sit in a loop, waiting for parameters to be met.” Black holes like this are common in hand-coded integration scenarios in which there are no checks and balances. “This is why middleware was invented,” says Kenyon.

Another important caveat is that some processes can’t be automated without sacrificing value. Mike Hilton of Sunaptic is careful to be on the lookout for these. For example, in purchasing, you get a number of situations where a good relationship and quick judgment can save the day. As Hilton relates, “a large retailer might say, ‘By the way, we just got a shipment of X; they’re damaged, but might suit your needs.’ This may be just the thing the buyer needed, but no system could ever cover this scenario.” In some situations, Hilton says that hybrid solutions — processes that include both automated and manual components — are the best fit.

Large consulting firms, like IBM Global Services and Montreal-based CGI, do a lot of work at the field level as well. “Businesses don’t know what their detailed business processes are,” says John Donaldson, IBM’s Websphere Sales Executive. One hallmark of the approach of both companies is that integration is recognized as an ongoing quest. Therefore, deliverables are structured in a way that maximizes their usability in future integration projects. These typically include reusable integration components, standard integration architectures, and integration competency centres.

CGI’s Hugh Fletcher, an executive consultant, describes the firm’s three-tiered approach to working through the integration process with customers. The first stage is to examine high-level patterns and processes, resulting in the creation of flow charts that are typically presented on a graphic program like Visio. Once these processes are established, detailed requirements documents are created for each process to be automated. Consistent with the practice of smaller firms like Five by Five and Sunaptic, this involves one-on-one sessions with end users, and also, with managers and other experts. The final step is the incorporation of these requirements into an integration architecture that reflects both short and long term goals.

Selection of integration middleware is part of this process, and has some pitfalls of its own. Most middleware technology is proprietary, so there are often alignments and potential incompatibilities that have to be considered. There are also many sub-types of middleware. And middleware is often aligned with specific applications. As a result of these factors, middleware environments are not as simple and transparent as they were meant to be.

There is hope, however, that emerging standards will simplify the integration playing field. The much-anticipated arrival of Web services promises to bring in a new era of standardized interoperability between applications. By defining universal protocols for the structuring and labeling of application components or services, standardization will make modular integration solutions easier to implement, regardless of platform considerations. This will also reduce the dependency on middleware. Major players like CGI and IBM are solidly behind this trend, which is one of the reasons they strongly advocate the common component strategies for integration described above.

The typical corporation has a great deal of information that could be shared through a centralized service approach. Credit card verifications, inventory checks, employee authentication, or the display of a corporate events calendar could all be defined as universally available Web services. The benefits are the elimination of redundancy and all of the manual intervention that this creates.

The tough part about common services is that people have to agree on standards. For the credit card check to work universally, all stakeholders in the organization have to agree on what the parameters of this check should be. If division A wants to initiate an extra security check based on a yellow flag, and division B would prefer to ignore that flag, the differences have to be worked out. There also has to be agreement on how to handle confidential information, how to comply with various regulations, and the list goes on. There’s sure to be lots of wrangling as business stakeholders sit down over flowcharts and try to agree on what their business is all about. But perhaps this is something that should be happening in all businesses anyway.

Jacob Stoller is principal of StollerStrategies, a Toronto-based consultancy focused on technology issues.

Top