What is application orchestration?
Application or service orchestration is the process of integrating two or more applications and/or services together to automate a process, or synchronize data in real-time. Often, point-to-point integration may be used as the path of least resistance. However, point-to-point integration always leads to a complex tangle of application dependencies (often referred to as "spaghetti code") that is very hard to manage, monitor and maintain. Application orchestration provides a) an approach to integration that decouples applications from each other, b) capabilities for message routing, security, transformation and reliability, and c) most importantly, a way to manage and monitor your integrations centrally.
Implementation
Let's examine how a simple bank loan processing application can be implemented using orchestration. In this example, we'll utilize a canonical messaging format and the Mule product to handle communication and transformation between services. While a canonical message format isn't required it does provide consistency and makes it easier to transition to an ESB architecture later if needed. Our bank loan processing application is a simple SOAP-based Web service and orchestrates three backend fine-grained services, namely:
Credit Status Check REST - Web service
A simple service that returns the credit status of a loan applicant.
Retrieve Interest Rate JMS - Legacy service
Another simple service that retrieves the correct interest rate to be applied to a loan application based upon a "daily interest" source.
Store Application to Data Warehouse JDBC - Database
Each and every application received by the bank must be cached into the data warehouse database to comply with regulatory requirements. This component performs a simple insert into an existing database.
Integration core principles
An ESB implementation use case typically involves the following:
| Orchestration | Composing several existing fine-grained components into a single higher order composite service. This can be done to achieve appropriate "granularity" of services and promote reuse and manageability of the underlying components. In our loan processing example this includes leveraging existing credit check, interest rate and data warehouse applications to create a new composite service. |
|---|---|
| Data transformation | Transformation between canonical data formats and specific data formats required by each application or service being orchestrated. An example of this would be transforming between CSV, Cobol copybook or EDI formats to either SOAP/XML or JSON. Canoncial data formats can greatly simplify the transformation requirements where there are many consumers and providers, each with their own data formats and definitions. |
| Transport protocol negotiation | Transport protocol negotiation between multiple formats (such as HTTP, JMS, JDBC). Note: Mule treats databases like another "service" by making JDBC just another transport (or endpoint) where data can be accessed. |
| Mediation | Providing support for multiple interfaces for the purpose of a) supporting multiple versions of a service for backwards compatibility or alternatively, b) to allow for multiple channels to the same underlying component implementation. This second requirement may involve providing multiple interfaces to the same component, one legacy interface (flat file) and one standards compliant (SOAP/XML) interface. |
| Non-functional consistency | For application orchestration, this can include consistency around the way security and monitoring policies are applied and implemented. Additionally the goals of scalability and availability can be achieved by using HA clustering. |
Mule integration components
Bank Loan Application flow
The Bank Loan Application flow illustrated here is a "Service Orchestration" pattern comprised of the following steps:
- An HTTP inbound endpoint & SOAP component receive SOAP/XML "Bank Loan Application" requests,
- A Credit Status Check (REST) service is invoked to retrieve credit status details. This is implemented using an HTTP outbound endpoint and a message enricher pattern to "augment" this information to the original message payload. A request transformer is also used here (not shown) to convert the data format from its canonical XML representation into JSON.
- An Interest Rate (JMS) service is invoked to retrieve the current interest rate for this category of loan application. As in the previous step, a message enrichment pattern is used here as well.
- A JDBC endpoint is used to cache the entire message to the Data Warehouse component. This component enables a transactional "INSERT/UPDATE or DELETE" operation to be invoked on an external database. If required, stored procedures and bulk updates can also be utilized.
- Finally a simple transformer component, Calculate Response, re-formats the message payload to prepare the response message, which is returned to the original consumer.
Summary
Most software development efforts today require some sort of application orchestration. Being able to quickly integrate with different applications, manipulate data and mediate message flow is hugely important as organizations are required to do more with less. Application orchestration is a better approach than point-to-point integration because the integtation logic is decoupled from the applications themselves and instead is managed in a container that provides security, transaction management, reliability patterns and crucial monitoring and tracking capabilities. Application orchestration with Mule makes it easy to start a project and add to it over time. It also allows you to migrate to an Enterprise Service Bus (ESB) architecture later as your needs grow.
