February ArcReady – The Microsoft Services Engine

Raul Camacho, part of the Microsoft SOA solutions team, presented on Service Lifecycle Management at Monday’s ArcReady event held in the Mason Microsoft offices. Along with discussing the obvious and everyday challenges of architecting, implementing, and governing services in an enterprise architecture, Raul discussed and demonstrated service virtualization in a managed services architecture using the Microsoft Services Engine.

Before we dig in, I want to thank you for all the great marketing intern candidates that I’ve been looking for. These folks will get a solid head start on their personal marketing efforts with the skills they’ll learn. Again, thanks for directing folks my way.

Here’s the overview of ArcReady: Most presentations contain at least one line that gets a chuckle from the audience. Not this one. Okay, you can go home now.

Many of the usual suspects attended the event including Mike Wood, Kishore Subramanyam, Joe Wirtley, Joe Mirus, Leon Gersing, Nino Benvenuti, and Matt Brewer. I had an opportunity to meet Resurgent Capital’s Jeff Brown and Suganthi Giridharan. LUCRUM’s Paul Stephens, Suzanne Lorch, and Tom Rudd accompanied me.

Mike announced the spin up of an architect group scheduled for Tuesday evenings. I’ll get it on the calendar when I understand the details.

Service Lifecycle Management. SOA, although still somewhat immature as a platform or strategy, has reached a stage that it is more than vendors trying to sell a software package with a different spin. Organizations begin to find themselves realizing some of the returns of a more agile architecture based on SOA. Still, no customers today realize the full value of SOA. And, as an aside, if the reuse promises of OO serve as an example, the governance issues of managing a SOA environment may keep the full value of SOA out of reach for many. Perhaps that is why in the most successful implementations, SOA modeling and governance happen at a collaboration level between IT and the business. Perhaps with business having a stake in services success, a reuse, or more appropriately an agile, architecture will have the enterprise governance aspects in place to realize a greater success than the IT managed OO implementations of an earlier era.

When Raul asked the room who was implementing an SOA, about 7 hands of the 30 or so in attendance were raised.

Based on Raul’s experiences, the common scenarios in today’s SOA implementations include:

  • Abstracting back-end systems, where about 85% of customers currently stand in their SOA maturity as they work to bring existing systems into the SOA world.
  • Automating mission-critical processes
  • Enabling new channels and business models, for instance passenger check-in with an airline. Until recently, an airline passenger had to walk up to a ticket counter in order to check-in for a flight. SOA enables multiple new channels to handle check-in functions, including web and mobile phone check-in. These new paradigms were driven by a business need and the entire conversation becomes more agile with an SOA backbone.
  • Providing visibility and governance across disparate heterogeneous systems. Applying governance and policies to an SOA in its infancy becomes difficult to as an organization does not have a baseline to work from or any meaningful use cases to apply to their organization. Take the SLA for instance. IT needs to balance performance against the infrastructure tax to meet those SLAs. Without some implementation as a benchmark for the business to use to determine and drive its SLA requirements, requiring specific SLAs up front becomes disadvantageous as out-of-the-box solutions don’t really exist in the enterprise architecture space. All enterprises are different.

So what are the challenges of SOA? Hmmm… You talk to a new customer and they tell you, “We have 100 services!” Okay. You ask, “How many are used by customers?” You continue digging. Finally you get to the root of the matter and find out 10 versions of 10 services *really* exist. One service returns all customers. A variation returns customers by region, and so on. Then you have the discussion that Web Services don’t an SOA make. Tightly coupled implementations are difficult to change. Limitations of web services implementations reveal themselves with increasing numbers of services and consumers. A bunch of web services are not an SOA.

SOA vs. Web services. S does not stand for web service. A service architecture provides agility, especially across the enterprise, where IT can deliver functions and workflows to an organization. Functions don’t necessarily translate into web services. When contrasted with a service architecture, web services on their own quite simply provide interoperability.

WSDL Boundaries. WSDL is too inflexible and requires custom coding. Let’s take a look at an example of a service. Take the Starbucks paradigm for instance. The end-to-end Starbucks’ service consists of walking into a store, ordering a cup of coffee, paying for the drink, receiving it, and finally, leaving the store. That is the basic Starbucks’ service. Over time the storefront around me, the menu, the experience may change, but the service remains the same. WSDL is the same, once I define the way to consume the service, everything else can change as long as I adhere to the definition.

You’ve probably faced the constraints of tightly coupled web services implementations. In this scenario a developer creates, say, a three-tier application with nicely defined separation of the business layer. The developer decides to expose this layer through web services. Another developer discovers the services and starts using it. The first developer is reassigned and no longer supports the web services, or better still, the application gets end-of-life’d and simply goes away. Without a maintenance and governance structure in place, and without a growth and use plan, services are left to wither on the vine creating future maintenance nightmares as systems couple themselves to one-another without regard for business plans or IT strategy.

Given SOA layers in the enterprise – a process layer, a services layer that delivers end-to-end workflows, and an integration layer – what if the services layer is a virtual layer that you can easily spin off at run-time? A developer will only get burned once in the tightly coupled model. In the tightly coupled model, lack of visibility, control, and coordination make it plain that services are not first class citizens in the enterprise architecture. They are a convenience until they are an inconvenience. To eliminate barriers that the tightly coupled model creates, services governance requires creating a service lifecycle management process. The governing body, comprised of the business as well as IT, integrates services into a lifecycle management process. In this model, features, as a consumer of a service, is important to the enterprise and not a convenient afterthought.

How is this model created? How does an organization plan so that the outcome of an SOA demonstrates enterprise value? The questions themselves point to the answer that you’ll find in the value of an enterprise SOA Roadmap. The organization planning for SOA will need to ask itself some questions: Why are you going after SOA? What does agility buy you? What do you need it for? Without a plan, an enterprise might adopt SOA without any goals and the inability to understand if they head in the right direction or have even reached their goal.

The Microsoft assessment of the SOA Maturity Model (SOAMM) is split into levels: basic, standardized, advanced, and dynamic that you can find on slides 17, 19, and 20 of this PDF (this PDF is NOT the slide deck used in today’s presentation). The more advanced the maturity the higher the value of the SOA implementation to an enterprise. You’ll notice that the matrix is partitioned into the perspectives of administration, consumption, and implementation. SOAMM then becomes a matrix of capability levels across perspectives.

Business and IT collaborate to create the value unlocked in an SOA. The business identifies WHAT services to provide. IT determines HOW to coordinate services into the infrastructure. BAs, the role that bridges IT and the business, might ask, “does it take too long to troubleshoot services”, “can you determine if a service is paying for itself?”, etc. Given the answers to these questions, the enterprise can determine functionality and SLAs for services.

Now that an organization has determined a services direction, it can successfully plan for change. From this point, use the SOAMM as a living model to map where your organization’s current state in terms of its goal. How does an organization manage the accompanying change? The roadmap describes how IT responds and works with the business, and following the roadmap will smooth out, or perhaps straighten out the road to change.

Implementation of a services architecture wrapped with service lifecycle management will strain an enterprise’s culture. These are large changes. Identifying appropriate boundaries and roles up front will provide communication channels to deal with this strain. Business owners will push requirements. A strong business process (BP) architect who can talk the business language and who can, given some limited technical ability, create flow charts and diagrams will strengthen the success proposition. The BP architect then talks to the service architect to mold the nouns and verbs into the required service contracts. The implemented solution then becomes the responsibility of the solutions architect. An enterprise architecture team oversees the integration of a solution into the enterprise infrastructure.

One obvious organizational challenge and enterprise will face is the fact that business ownership simply doesn’t promote decisions for consistent design and reuse. They just want their app. As the architecture team overcomes technical barriers, political barriers may become more pronounced. As an architecture and its accompanying processes mature, businesses put in the pipeline they may not feel they are getting the attention they need to deliver business value that impacts the bottom line. Finally, over time, many people and groups will interact with the system both uncovering weaknesses and defining points of change.

IT tends to focus on the architecture and infrastructure. Services in large enterprises may be portfolio-based, including products such as ERP, CRM, and other systems. Business comes along with some request, perhaps a requirement for a system that allows controls on document exchange with partners and vendors outside the corporate firewall. IT usually develops a passable solution, or a view of a solution that is a hodgepodge of products. Solutions become a combination of packages and custom code ultimately winding up in a silo. How do you know? You’ll find your customer ID living in multiple systems because of IT partitioning.

Models and Tools. Easing governance and design overhead requires business modeling and technology modeling all brought together in a service model. The service model is ultimately implemented by a technology model that supports the service contract where the business orchestrates business capabilities focusing on service offerings. Boundaries between business and IT softens as both work together to building out the services.

If business alone has to express the service the bar will probably be too high as the business is not expected to understand the technology. Models, then, allow the business to architect a service on their terms. Using tools, these models can be stored as workflows, flow charts, business processes, etc. The BA serves as a bridge between the business and IT and can help translate and interpret models for both audiences, and if modeling tools are used, collaboration between business and IT may be facilitated. Resulting artifacts might eventually live as, say, .NET assemblies in repositories. And that becomes the end-to-end view in terms of service lifecycle management. An enterprise develops an ecosystem that allows all the players to fulfill their role in the lifecycle. When an organization reaches this point it may realize a 10X increase in value in terms of productivity, agility, performance, etc.

To support the service lifecycle, Microsoft is developing the Oslo platform, a set of model-driven and service-enabled applications. Olso includes the next gen releases of VisualStudio, BizTalk, and other business process modeling and services tools with a view towards an end-to-end service lifecycle management solution. The goal is to break services out of their silos. This next generation of tools will be built around a modeling language focused on the analyst – a textual language describing both data and behavior. BizTalk Process Server will consume and execute models using the XLAS language, and a service bus will facilitate communication between process servers as processes can’t live in isolation. The end goal, being all things Microsoft, is discovery of consumable services in a cloud of services living in a Microsoft hosted environment available to the BA, architect, developer, and IT Pro.

Taking loose coupling one step further, can organizations realize the equivalent savings with a virtual service layer that virtualization of hardware creates today? What building blocks emerge in this case? Done correctly, a repository can store actionable metadata that allows the spawning of necessary services and behavior at run-time.

To this end, Microsoft is working on their Managed Services Architecture (MSE), a service catalog supported by a service runtime engine built on metadata and runtime messaging. The runtime includes a messenger, broker, and dispatcher. A client sends a message, the messenger normalizes the message and hands the process to a broker. The broker translates the information between the virtual world and the real world, finally handing off to a dispatcher. The dispatcher materializes a client with the actual expected implementation.

Communications between the runtime are, themselves, services, so that they, too, can live in a scaled environment without hard-wired connections. Since everything is metadata, flexibility allows changing the transport without affecting service essentials. Issues inherent with scaling web services are handled as consumers and services are added.

The MSE allows total service interface abstraction, true service versioning support, and extensible/pluggable service control through a broker.

For more information on the MSE see:

MSE Availability: repository, runtime engine, MMC SnapIn, Service Test Client, Technical Guide, and Walkthrough Guide. The solution was built in the field with customer feedback by MCS. The next generation of tools is being shaped with customer feedback. Please feel free to provide feedback.



arcready.com will have the actual slide deck at some point.



~ by Andy on March 5, 2008.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: