However, most SOA stack vendors do not address many of the key requirements of SOA, including solution patterns around runtime SOA goverance. Considering this limitation, it’s important to address these issues with the proper technology, leveraged in the proper way. Thus, the purpose of this paper. Download File

23-slide PowerPoint in pdf format.Download File

What’s core to the problem is that we have a tendency to consider Services as simple building blocks, something that will always be there, always using a good design, relevant patterns, and always in good operational condition. That’s just not always the case. Indeed, Services are the building blocks of SOA, and like building blocks of a house or a building, the quality will define the value of the finished product, in this case, the SOA implementation itself. Thus, spending time on what Services do, how to define them, how to design them, and how to build them is a good investment in time, and something that’s missing within many architectures.

Clarify these Services issues at the outset of a SOA project to build better blocks:

First, Services don’t need to be Web Services. ZapThink has been very clear about this point. However, this is a confusing statement for those of you who have been absorbing the hype. The fact is, you can build a SOA without Web Services, opting for more traditional approaches such as transactions, distributed objects, or custom software systems. Indeed, when considering “special needs architectures,” such as those requiring high performance, the use of Web Services are clearly contraindicated.

Keep in mind as well that the Service contract language is more than just WSDL. So such Services (or more precisely, Service interfaces) are similar to Web Services, but have more information in their contracts than WSDL can provide.

Second, Services produce behavior and data, not just data. Most people who design, create, and/or expose Services think of them as data providers, and indeed they are in most instances. You invoke the Service, and it produces data in the context of a structure, and consumed into another system. However, while many Services are very data-oriented, Services are able to provide behavior as well, or, the ability to do something around the containment of the data, or perhaps provide behavior without data at all.

Third, Services are not applications, and you shouldn’t design them like applications. As you’ll see below, Services have their own specific design orientation. The way you define and design a Service is very different than what many consider traditional application design. You’re building a much smaller system that exists within many systems, and thus you must pay special attention to interoperability, granularity, core purpose, and testing approaches.

Services are not complete applications or systems. They are a small part of an application. Nor are they subsystems; they are small parts of subsystems as well. Indeed, Services are more analogous to traditional application functions in terms of design and how you leverage them to form solutions, fine- or course-grained. Knowing that, we have a better basis of understanding when approaching the Service design problem.

Finally, each Service has a specific purpose, and they are not complex or naturally dependent upon other Services. Thus, they are easily abstracted into composite applications, in essence, leveraging these Services as if they are functions local to the composite. This is where exposed Services have a tendency to fall down. Since they were not designed, but abstracted, they typically have far too many dependencies to be as useful as Services that were designed correctly from scratch. That’s the tradeoff. Services should exist with a high degree of autonomy. They should execute without dependencies, if at all possible. This allows you to leverage the Service by itself, and design the Service with this in mind no matter how course- or fine-grained the Service is.

It’s in the Design
So, how do you design a Service? It’s important to follow a few basic principles. While following these principles does not insure success, it will clearly send you down the right path. First and foremost, you should often design Services for reuse. Services become a part of any number of other applications, and thus must be designed to provide behavior and information, but not be application specific. This approach is difficult for many developers since custom one-off software is what they’ve been doing for most of their careers. Thus, the patterns must be applicable to more than a single problem domain or application, meaning you must have use for your reusable Service. Else, the exercise is in vain. However, not all Service can or should be reusable, thus the criteria of reuse should be selective.

In addition, Services have to be designed for heterogeneity. Build Services so that there are no calls to native interfaces or platforms. Remember that a Service, say, one built on Linux, may be leveraged by applications on Windows, Macs, and even mainframes. Those who leverage your Service should do so without regard for how it was created, and should be completely platform independent.

Among other benefits, abstraction allows access to Services from multiple, simultaneous consumers; hiding technology details from the Service developer. The use of abstraction is required to get around the many protocols, data access layers, and even security mechanisms that may be in place, thus hiding these very different technologies behind a layer that can emulate a single layer of abstraction. Also, when we build or design Services we need to account for aggregation. Many Services will become parts of other Services. Thus they become composite Services that are leveraged by an application, and you must consider that in their design. For instance, a customer validation Service may be part of a customer processing Service, which is part of the inventory control systems. Aggregations are clusters of Services bound together to create a solution.

Services are not applications and should have limited scope, as we discussed above. In other words, they do simple things such as check inventory or calculate reorder points. If your needs are more complex, you simply write more Services and not overload a single Service with too much functionality. Services with too much functionality are considered heavy, and are difficult to reuse since you may deploy a Service where you’re only leveraging ten percent or less of its functions.

So, now that we understand the common design patterns we must follow, the question is: How do you design a Service? Also, what tools are available? There are certain steps architects and developers can follow. Here are some suggestions, assuming new Service design.

1. You need to define the purpose of the Service. What will the Service do, and who are the intended user; human, application, and / or other Services?
2. You need to determine the information to be bound to the Service, including schemas and other metadata. You must understand how the Service leverages information, and what functions require what data.
3. You need to determine the functions (methods) encapsulated inside the Service; in other words, the behaviors you would like to expose. It’s also at this step where we define each function, including how the function breaks down using a traditional functional decomposition chart.
4. You need to define any interfaces into the Service, both machine and human. This means we need to determine how the Service will interact with the calling applications, and through what mechanisms.
5. You need to define how to test the Service, using the suggestions above. Testing is an important but often neglected step where you define how those who leverage the Service will test the Service within the context of their usage pattern. You need to define test information, Service invocation, and validity of results.

What’s core to the success of SOA is a clear approach to Service design, development, and testing. At the end of the day, it’s good old-fashioned discipline that comes into play here, more so than new technology, tools, and programming tricks. That’s not what people want to hear in the context of the deafening hype, but that’s the reality.

The ZapThink Take
The truth of the matter is that Services are a new challenge for developers, and they bring their own sets of requirements. In many ways it’s as much of a shift in thinking as was the movement from structured analysis, design, and development to object-oriented analysis, design, and development. We all know how long that took, and in many respects it’s still going on today.

What we’ll see in the short term is what you expect to see with any new approach — poorly designed, developed, and tested Services that bring high cost and lost productivity. Thus, the failures will lead to rethinking, relearning, and retooling to get Service design right. Count on large expenditures on training, tools, and testing infrastructure over the next 3 to 6 years. Also, count on some pushback on the whole SOA concept as people understand that they are dependent on the underlying Services, and thus the architecture will suffer as well as developers move along this learning curve.

You must also pay attention to the process of exposing existing legacy systems as collections of Services. While it appears that you’ll have to take the interfaces as they are deployed, now exposed as Services, there is actually a lot the developers can do to design abstracted Services that better serve the architecture. While many people believe that tools and technologies that turn APIs or transactions directly into Services are the way to go, most will find that Services built using those principles provide very little value in the long run.

SOA is indeed architecture, but it’s based on the proper design, development, and testing of Services. These Services provide what’s core to SOA, and the discipline and process that software engineers put around this effort should be significant. Changes to SOA Services design, development and testing need to be made and understood right now.Download File

First, be wary if consulting organizations point out their experience in the world of SOA by putting up past projects as proof of their experience. Most, if not all, of these past projects are really JBOWS (just a bunch of Web Services) and have no underlying mechanisms to provide agility, which is a core benefit of SOA. The fact is that most end users out there don’t know the difference. Always keep in mind that Service-enabling existing systems is useless without a mechanism for turning those Services into solutions.

The problem is that many of people looking to hire SOA consultants don’t understand the difference between JBOWS and true SOA, and accept JBOWS as “experience.” In reality, it’s an indication that the consultants don’t understand the core value of SOA, and thus could send you off in all sorts of dangerous and costly directions. So, make sure to hire consultants who understand that SOA is really about architecture, agility, and changeability, and not just about Service enablement. It’s easy to expose Services; turning those Services into solutions is another level of sophistication.

Second, many consultants are a bit too chummy with vendors. Thus, you’ll find that they implement the same vendors and technology each and every time. This situation should present a huge red flag since SOA problem domains are all very different, and technology solutions that work best for the solution are never, ever, from the same vendor, over and over again. However, when you sell hammers, everything looks like a nail. The path of least resistance is what you know, not what you should do.

People responsible for managing SOA consultants need to state clearly that you are looking for the right solution, not the one they know, or what may be part of an existing partnership. Indeed, you should quickly overlook consulting organizations with many preexisting vendor partnerships. You need folks in there who are agnostic when it comes to technology, and are willing to leverage the best-of-breed to address your issues.

You’ve heard it all before: technology should add value to a concept like SOA, not control it. Lately ZapThink has run into SOA efforts that are not playing by that rule. Indeed, the focus is on ESBs and governance layers, and not on getting SOA right from an architecture standpoint. I thought SOA was architecture, right?

At issue are the multi-billion dollar marketing budgets that vendors bring to the SOA world, and thus lead through sheer volume of sales. People like ZapThink who promote the architectural value of SOA, are not being heard above the noise and flash of the SOA-in-a-box movement that’s occurring all around us. The end state is invariably a failed project, due to the misuse of technology, and project managers who do not understand their own issues before pulling out credit cards and buying technology. Thus, too much focus on technology is hurting SOA.

Don’t get me wrong, I’m not attacking the vendors here. That’s just too easy. What I am doing is pointing out the fact that technology, while being very powerful and necessary within the world of architecture, needs to occur as an outcome of the architecture, instead of driving it. While most of you will see that as a logical piece of information, it’s still a de facto practice to take an “ESB-oriented” approach to SOA, no matter what the issues are at hand, or, perhaps an “App Service-oriented” approach, or a “Governance-oriented” approach, or… Okay, I’ll stop.

The result, in many cases, is not something that does not work; it’s something that does not work as well as the organization requires. Thus, many companies settle for sub-optimal solutions that they must rework every few years to keep up with business demands, versus solving the problem once and for all, maturing the architecture over the years and not having to keep replacing it.

The correct approach is simple. You start with your core requirements, understanding all aspects of the business, existing Services, existing data, processes, etc., and then create a vision of the end state architecture and a complete, iterative map for getting there. You understand patterns here, not instances of technology, and only then back the correct technology into the problem domain, making sure to meet the core requirements.

Third, there must be a predefined process. While many SOA consultants try to use older software development lifecycle (SDLC) and enterprise architecture processes, SOA requires a specific approach that addresses the unique nature of its architectural patterns. For instance, there is a traditional focus on data, but the data must be properly bound to Services. Moreover, those Services must be properly bound to processes. Plus you need to keep track of all the interdependencies, and how all of this stuff links to SOA governance, SOA security, and event management and monitoring.

Many consultants attempt to oversimplify the process, rapidly moving through or even foregoing the planning steps. Their main focus is the selection of the technology, or, in some cases, they attempt to force fit a problem with a predetermined technology solution. This approach never has a positive outcome.

The fact of the matter is that SOA implementations are complex distributed systems, and thus complex to plan, design, build, and test. The time spent in planning will later pay huge dividends. You should define a rigorous process/methodology that, at a minimum, provides you with a semantic-level, Service-level, and process-level understanding of the problem domain, not to mention the governance model and security strategies. Trust me, you can get SOA right the first time, but with more planning and sweat than you expect.

Finally, there is the lack of understanding about ongoing SOA operations, and links with traditional enterprise architecture. You just can’t implement SOA, you have to live with SOA on an ongoing basis, which means understanding how the solution exists currently, and how to align it with business as it changes. If your consultant has done his or her job, you should have an architecture where the volatility has been abstracted into a configurable domain. As a result, it’s a matter of changing things at the configuration layer to adjust to the changing nature of the business. Therein lies the value of SOA.

If your architecture does provide that degree of agility, than you’re going to have to loop back to the beginning to see where you went wrong. Typically, per my previous point, you’re planning was insufficient and perhaps you employed the wrong technology solution. In any event, you need to bite the bullet, spend the money and fix it, or you’ll find that your SOA actually takes you backward.

The ZapThink Take
Bad intentions are not the problem here. I think everyone is honestly looking to do their best, but it’s a lack of understanding and perhaps talent in some instances. Consultants who succeed with their SOA initiatives have a wide range of skills, a good understanding of architecture and the value of SOA, and all of the good work that needs to occur to make it work for the client. However, I don’t see many out there who fit that bill, and the amount of bad advice is becoming a huge issue. Unfortunately, many of their clients won’t figure this out until it’s too late.

When doing SOA, or another other complex distributed computing problem for that matter, it pays to obtain honest and objective advice. Even if you only need a consultant for validation of key artifacts, approaches, and technology. That’s just cheap insurance. Unfortunately, the number of organizations out there calling themselves “SOA consultants” is huge, while the number of consultants that actually know what they are doing is relatively quite small. Thus, most of their clients are finding that the selling around the consulting services was dazzling, but the Services they delivered, and end state solution is typically suboptimal. Indeed, a common trick is to have the “SOA experts” come in a sell the deal and you’re left with the kiddies to actually do the work.

The most troubling aspect of all these issues is that most organizations won’t understand the impact of leveraging less-than-stellar SOA consultants until it’s too late. Indeed, the full impact of the architecture is years, not months away, and selecting the wrong technology, or getting the wrong strategic advice, won’t be obvious until it’s too late. At least until the checks for the huge fees have cleared the banks, and the consultants are on to their next project.Download File

Guest Expert Jake Sorofman, VP of Marketing, JustSystems.

Listen to this Podcast and you will:

• Learn about the role of documents in SOA
• Get an idea of the role XML has in the creation of content
• Understand how dynamic documents that display application functionality can help implement document-centric business processes in the context of SOA

Setting the Stage: ZapThink Analysts

ZapThink analysts Jason Bloomberg and David Linthicum set the stage by discussing how documents are important players in the world of Service consumers as organizations pass the “Services tipping point” in their SOA implementations.

Next, Jake Sorofman explains the role of documents in SOA, and distinguishes between unstructured and structured information. He then explains why XML matters in the content creation process, and the relevance of DITA to such efforts. He then details JustSystems’ approach to dynamic documents offers application functionality and how such documents support agile business processes in the context of SOA implemetations.

Featured Guest Expert: Jake Sorofman

VP of Marketing, JustSystems.

Jake Sorofman is a seasoned software marketing executive with a strong product strategy and communications background. Previously, he was VP of product marketing with Mercury Interactive (now part of HP Software), where he was responsible for the Systinet product line. He joined Mercury though Mercury’s $105 million acquisition of Systinet Corporation. Before Mercury, Jake led marketing for two WebSphere products at IBM Software Group, which he joined through the 12/04 acquisition of Venetica. Prior to Venetica, Jake was director of product marketing with Documentum, Inc. (now part of EMC), which he joined through the acquisition of eRoom Technology. Jake also has extensive experience in public and analyst relations, having led corporate communications for Ironside Technologies (now part of Infor). Jake has a BA in english and political science from University of New Hampshire and an MBA from the McCallum Graduate School of Business at Bentley College, where he was an American Marketing Association George Hay Brown Scholar.

This ZapForum Podcast is presented as a 26.3 MB mp3 file. Running time: 28:48.Download File

Guest Expert Kyle Gabhart, Director, SOA Technology, Web Age Solutions.

Listen to this Podcast and you will:

• Learn how a “Selective SOA” approach can reduce the risk of a SOA initiative
• Understand how a limited scope SOA project should handle infrastructure and governance issues
• Learn about how to train SOA architects as well as details on the ZapThink/Web Age Solutions partnership for delivering Licensed ZapThink Architect credentialing sessions

Setting the Stage: ZapThink Analysts

ZapThink analysts Jason Bloomberg and Dave Linthicum set the stage by discussing the challenges of implementing limited scope SOA projects.

Next, Kyle Gabhart introduces the “Selective SOA” methodology that lowers the risk of SOA initiatives by rolling them out on an incremental, combined bottom up/top down basis. Kyle also discusses the importance of calculating the ROI for any SOA initiative, as well as the role that governance plays in such initiatives. Finally, Kyle discusses the training of architects, and Kyle, Dave, and Jason review the Licensed ZapThink Architect training and credentialing program that ZapThink and Web Age Solutions are jointly offering.

Featured Guest Expert: Kyle Gabhart

Director, SOA Technology, Web Age Solutions.

Kyle Gabhart is a subject matter expert specializing in Service-Oriented technologies enabled by Java, .NET, and XML. He has served in a variety of technical and business-focused roles across a broad range of industries, including Defense, E-commerce, Finance, Semiconductor, and Telecom, to name a few. Kyle’s current post is as the SOA Strategic Lead for Web Age Solutions, a premier provider of technology education and mentoring.

Kyle is a popular public speaker recognized for his enthusiasm and dynamic analysis and presentation of emerging technologies. He has written articles for The Web Services Developer’s Journal, served as the Java Pro for DevX.com for two years, and published close to two dozen articles on IBM developerWorks, including the J2EE Pathfinder series. In addition to writing over 50 technical articles both online and in print, he has contributed to two books, Professional Java and XML and Professional EJB Development, both by Wrox Press.

Kyle is the original contributor of the kXML-RPC library (the first J2ME client implementation of the XML-RPC protocol), the founder of the Web Services Java Users Group based in Dallas, TX, and a Founding Member of the Worldwide Institute of Software Architects where he has served as the Subject Chair for Architectural Patterns.

This ZapForum Podcast is presented as a 13.0 MB mp3 file. Running time: 28:28.Download File