CIO Government Review –
Service-oriented architecture (SOA) can demolish the status quo. Decades of siloed system design have left most government organizations with antique, rickety systems that don't play well with others. By putting new SOA wrappers on old proprietary applications, modular interfaces can be built, shared, linked, reused and recombined as needed, to create an infinitely interoperable IT utopia.
No need to rip and replace old systems; instead, they can be refurbished and extended internally and even externally via the Web. This is where SOA shows promise well beyond rejuvenating legacy enterprise systems, says Bill St. Arnaud, senior director of advanced networks at Ottawa-based CANARIE Inc.
"SOA is now seen as a key component in a broad range of fields beyond enterprise IT: chemistry, biology, everything," he says. "Whether it's a traditional payroll application or radio telescope research, it makes sharing, mapping and transferring data, and creating new mash-ups, simple."
SOA can also have a profound impact on business processes. Many complex processes that require human back-and-forth can be automated as SOA-based Web services, which in turn can invoke other Web services, and then others, throughout the service chain. "If GM orders a phone line from Bell Canada [for example], it has to be validated, checked, tested, delivered and invoiced by many people," says St. Arnaud. Instead, all the specialized steps in the transactions can be itemized, agreed in a contract, and automated as interlinking Web services between both companies.
Take-up of SOA is stronger in more competitive markets, he says. In the U.S., about 70 percent of companies say they plan to invest in it over the next two years, according to IDC Canada research. In sluggish Canada, the figure is 40 percent, with the public sector lagging still further behind the private sector.
Building this SOA utopia won't be easy. There are many impediments, ranging from making the business case to fix systems that aren't entirely broken to governance and liability issues to standards wars, notes St. Arnaud. Nevertheless, SOA is slowly but surely creeping into many areas of Canadian government.
The way SOA works
In 2005, the Alberta Ministry of Justice deployed a SOA-based enhancement to its maintenance enforcement program (MEP), driven by new legislation enacted the year before. Dubbed "Deadbeat Dad" legislation, the Ministry was concerned with finding ways to track and enforce adherence to court orders by withholding access to government services, explains Stuart Charlton, enterprise architect at BEA Systems Inc. in Toronto. "So if someone doesn't pay child support, they might suspend their driver's licence."
As in other provinces, Alberta's ministries and government departments are far from integrated.
Tracking thousands of these MEP cases over time, sometimes more than 10 years, and building in the triggers for enforcement actions based on patterns of misbehavior was a major system undertaking within the Justice Ministry. But developing processes to ensure actions are executed by a multitude of external ministries would have been a monumental inter-departmental undertaking.
Instead of labor-intensive back-and-forth between multiple departments -- phoning, faxing, exchanging forms and information -- the Ministry of Justice used Web services to automate the workflow. "The new approach is to agree on a shared contract that meets the needs of the consumers and producers of services and information," says Charlton. "The biggest challenges in SOA are coming to those contractual agreements between parties, and the processes and change management around that."
But once these issues are settled, the technology itself is relatively straightforward. A producer posts the services it has made available for common use -- for example, an application that identifies an Albertan as an MEP case -- through an electronic interface based on SOA standards, which can be used by any authorized consumer using the same Web-based technology. All the various conditions for an exchange between departments, including exceptions that require human judgement, are agreed and scripted in advance.
"So this is a conversation between the consumer's and producer's systems. The consumer says, I would like to request this service, and will validate the information I provide," says Charlton. "It will then grab and use the service automatically if there is no exceptional condition. If there is, it'll provide a worklist to another interface for human review. But all this communication is happening behind the scenes."
Like a high-tech version of an Amish barn-raising, all the departments touched by the MEP program share and link the applications, processes and data that relate to it, instead of building a specific system for it.
While the quest for competitive advantage may be a clear driver for SOA uptake in the private sector, the scenario is more complicated in the public sector. Without a profit motive, SOA is typically a hard sell that must be tackled indirectly and incrementally.
"There's no imperative to adopt it. Instead, it comes up in IT projects after budgets are approved to renew applications or to build new ones. SOA is almost incidental," says Michael Turner, an e-government strategy consultant and former ADM with Public Works.
The Treasury Board's CIO branch has provided direction on SOA, he says. "It has done some ground-breaking work, providing overarching policies and standards," says Turner. "But there's no requirement for departments to comply or demonstrate they're doing it to get budgets approved."
Turner acknowledges that without a compelling need, it would be virtually impossible to obtain funding to rearchitect a system with SOA. "That implies jacking up the house to redo the foundation. People don't do that. Instead they say, the next time we develop this application, we'll build a different kind of foundation with more flexible components that can be re-used."
But the consequences of putting off repairs to faltering IT infrastructure are higher costs in the long run, he warns. "The operational costs of application management are significantly higher than they need to be. And it's more complex to make modifications or add-ons to non-SOA systems. We're paying for the fact we're not getting on board."
Even a trivial change like adding a new field to a legacy system can cost millions, agrees Tom Metzger, director of solutions engineering at Vancouver-based Make Technologies Inc., which specializes in modernizing legacy systems. "Future maintainability of systems is a big driver in the public sector. And many government organizations are on the hook to respond to legislative changes."
Some crown corporations have been early adopters of SOA, says BEA's Charlton. "We've seen some aggressive adoption at Farm Credit and Canada Post." He points out that Treasury Board is actively pursuing a strategy for IT transformation across the public sector. The challenge is overcoming the inertia to get started by focusing on agencies with a pressing need to undergo change.
"Agencies tasked with counter-terrorism are great examples," says Charlton. "The RCMP and DND are starting to adopt SOA. They're already doing it in front-line applications and they're modeling their approach after the U.S. Department of Defense, which is based on a SOA foundation."
The lack of a clear vision to sell to politicians and constituents contributes to funding issues, suggests Michael Kuhbock, founder and CEO of the Integration Consortium, an international industry association based in Calgary.
While creating citizen-centered services may be a worthy cause, the public doesn't perceive any urgency. "They say: Everything seems to work okay, so why spend millions on SOA when there are other needs?" says Kuhbock.
Kuhbock believes an issue that fires passions and unifies efforts, such as environmental stewardship, may play that role in the future. Every service the government provides, and every supplier the government uses, has an environmental impact. "Right now, the data around that is not tracked," he says, pointing out green issues are gaining traction with the public. "In the future, the public may say: We won't stand for this anymore. We want to ensure we know what every service organization in government is doing to the environment."
SOA is still a work in progress, and its capabilities are ill-defined and confusing. SOA facilitates system integration, but it doesn't actually do the integration. "It's an architecture, nothing more," says Kuhbock. "In the construction industry, you engage an architect to design the building so it has a sustainable infrastructure and everything works together - but then you engage construction crews to build it."
SOA design is counter-intuitive and difficult for IT staff trained in classic waterfall systems design, adds Turner. "The traditional way is to look at application suites as a whole, design in the most integrated way possible, internally within the system, and use programming logic that produces tightly-bound functions with minimal duplication."
While it may be an elegant system initially, the costs to make any modifications or additions later are high, since the system isn't designed to change or grow over time, he says. "Then SOA comes along, intentionally breaking things down into blocks and duplicating functions so they can work as loosely coupled modules. It's less elegant, but in the end, they're cheaper to put together."
Governance and accountability issues are also stumbling points when sharing SOA infrastructure and interlinking modules -- a problem that's more acute in the public sector versus the private, he says. The horizontal shared services movement in government butts directly against the brick wall of vertical organizational structures.
"The cabinet system we have in Canada is built on the intentional design of silos," says Turner. "Each set of programs is enabled by legislation and statutes, and each organization supports a minister who's accountable to Parliament for that set of programs. Natural silos are designed into the system, so when you try to apply common technology across departments, it goes against the grain. It almost comes down to, do you want efficiency or accountability?"
The tendency to treat SOA as an IT project rather than a strategic architecture is a consequence of these silos, says Kuhbock. At a recent presentation he made to government IT officials, many of them said they had the same central issue: "They can do SOA in their own compartmentalized area, but they can't get it broadly driven; so they're just stuck in their silos again. It's like architecting a town, district by district, without thinking about how the whole town will interact, or mapping out the whole community."
Vendors and standards wars
Beyond the public sector, there are some industry-wide SOA issues that need to be settled. "SOA is on a hype cycle," says Kuhbock. "There's confusion about it in the marketplace across all industries. There's no universally accepted definition, so it's different from one industry to the next."
IT vendors feed this confusion by treating SOA as a selling point in their marketing strategies, suggests Kuhbock, with many branding their particular product or flavor around their own definitions.
"It's all based on what they're selling," he says. "Everybody has different reasons for being active in this space, be they producers or consumers of solutions. All have different objectives, so it's hard to get consensus, and this bleeds into every area of its use."
As a consequence, the number of SOA standards is rapidly proliferating. "At last count, there were over 110 SOA standards," says Kuhbock.
St. Arnaud agrees this lack of common standards is a major issue. "There are standards wars going on. Web services have exploded into a multitude of different variations and special sector types. Now suddenly, this great nirvana solution that could make everything compatible has itself become incompatible with different versions."
But various government entities -- the federal Treasury Board, the Ontario Government, Canada Health Infoway Inc. -- have done extensive work in defining SOA standards and providing technical guidance for their constituent areas. So any lack of standard definitions should in theory be less of an issue in government relative to other sectors that don't have top-down leadership.
Not so, says Kuhbock. "SOA has been muddled by the vendor community," he says, pointing out public sector IT people still need commercial software tools to re-architect their systems. "If the government is dealing with vendors A, B and C, and each of those has a set of standards around their SOA-based tools, then the government is back into analysis paralysis sorting it out. No one's really come out with best practices, methodologies and so on, and the vendors keep redefining the standards around their tools."