#854 SOA Suite to OIC - how easy is it? Part 2

In this post, I look at the similarities/differences between SOA Suite and OIC, both from a design time and runtime perspective. I then look at an approach to implementing SOA integrations in OIC.

SOA Suite and OIC  

Now let's go a bit deeper and compare the components of each. 

I like to tell customers OIC is a toolkit. The tools include Integration - to connect apps, Process - to extend apps or implement custom business processes, Visual Builder - to create net new apps on top of your App apis, Integration Insight - giving you business user facing Dashboards and Alerts on top of your integrations and processes. Complement these with a B2B component and an sftp conform File Server then you have the toolkit that is OIC. But there is more...

OIC runs on OCI, along with many other PaaS services from Oracle. Many of these services could be interesting for you from an integration perspective as well e.g. OCI API Gateway to protect your OIC apis, OCI Streaming Service (Kafka), OCI Functions (Serverless Functions for your more complex business logic), OCI Eventing and OCI Logging - to name but a few. OIC provides adapters for many of these services - net, net OCI services can provide a compelling business value-add to OIC.

Let's now look at the components of SOA Suite - 

 

To the above we can add Service Bus, BAM (Business Activity Monitoring), MFT (Managed File Transfer) and B2B components. Naturally both come with a rich set of adapters - SOA Suite has 20+ adapters, OIC has more than 70 and growing.

So how do these SOA Suite components relate to the OIC toolkit?

Naturally, the OIC Integration component or tool is not a superset of BPEL, Service Bus and Mediator. However, one can implement the standard VETO / VETRO patterns (Validate, Enrich, Transform, Route, Operate).  

With OIC you can implement the various technical patterns - synchronous request/response, asynchronous, fire and forget (One Way Process). One can also Subscribe to Events - application and technical e.g. new purchase order created in Fusion ERP, or new row created in the Customers database table. The OIC scheduled orchestration pattern in ideal for regular bulk data import/export to your apps, data lakes etc.

OIC Scheduled integrations can also be called from other integrations via the OIC REST api -

   

Net, net the same patterns that you have in SOA Suite, albeit with a much richer set of adapters to many Oracle and non-Oracle cloud and on-premise apps as well as to many Oracle and non-Oracle technologies.

From my experience, the Human Task component is not heavily used by SOA Suite customers. On-premise Human centric processes are usually implemented in tools such as BPM Suite, which complements SOA Suite. The Process component of OIC allows one to create structured and unstructured business processes (Dynamic Processes). BPM Suite has an implementation of the latter - ACM - Adaptive Case Management, however, the OIC offering is much more compelling. OIC Process also includes the Decisions Service - a rules engine that can be invoked from OIC Integration and indeed any 3rd party tools.

     

OIC B2B supports processing of EDIFACT X12 and UN documents over AS2 and FTP. It also supports trading partner management, just like SOA Suite B2B. However, the B2B component includes more automation compared to its SOA Suite counterpart. From my experience, only a small subset of SOA Suite customers use B2B, however, those that do use it, do so extensively.

Let's take a deep dive into the integration building blocks in OIC and SOA Suite BPEL 2.0 -

The question is how many of the SOA Suite BPEL activities are actually used? For example, SDOs and EJBs are no longer bleeding edge technologies. Consider Compensation - this occurs when the BPEL process service component cannot complete a series of operations after some have completed, and the BPEL process service component must backtrack and undo the previously completed transactions. For example, if a BPEL process service component is designed to book a rental car, a hotel, and a flight, it may book the car and the hotel and then be unable to book a flight for the right day. In this case, the BPEL flow performs compensation by going back and unbooking the car and the hotel. BPEL provides us with 2 constructs that make it easy for us the undo what has been done. These compensation activities are not available in OIC, but we can essentially accomplish the same using a pattern such as SAGA.

Some customers have embedded code in their BPEL processes, either leveraging Java Embedding or Javascript. OIC supports Javascript embedding. Regarding Java - one could externalise that to the OCI Functions service or have the code exposed as a REST service, running literally anywhere - on OCI, Azure, on-premise etc. 

Other activities such as Phase - this activity creates Oracle Mediator and business rules service components for integration with a BPEL process - are seldom used. 

One BPEL activity used by some is Dehydration. This activity compresses the process state and stores it in the underlying engine database; essentially saving the current process state. BPEL itself dos this automatically in order to maintain the state of long-running asynchronous processes, while waiting for asynchronous callbacks. The BPEL worker threads are thus freed up to do other more urgent tasks. Explicit use of dehydration can be useful for mitigating system/network instability. Here again, the developer is required to have knowledge of the underlying engine and how it works. 

SOA Suite on-premise does not live in an hermetically sealed world, however, the developer has control of everything; just consider all the BPEL engine tuning knobs you have in SOA Suite - syncMaxWait etc. Often the apps/technologies to be integrated are within the organization's control, making global transactions possible. But with control comes responsibility - responsibility for managing the SOA Suite infrastructure - Weblogic, Database and SOA apps - and all that entails. OIC frees you of that responsibility, while also providing you with a platform that will adhere to your SLAs, in other words, giving you control of your business processes. If you need a platform that will support the processing of thousands of payments per hour and also allow for periodic peaks, where payments rise to the tens of thousands - then OIC is the platform for you. Think of OIC as a more SaaS like service - you define what you need, you use and you pay for what you use. Add to that - improved lifecycle management and you have a very compelling offering.

Net,net - one should not make decisions purely based on an activity to activity comparison. 

I have worked with many SOA Suite customers who have adopted OIC. Their approach was to look at what needed to be done - not - this is how I do it in BPEL, I want to do it the same way in OIC. The simple Order Process example from the previous post was essentially a 1:1 implementation of what I had in BPEL, however, that will probably not apply to more complex examples. Successful SOA to OIC customers need to appreciate the differences, especially when the starting point is a SOA composite the essence of which needs to be implemented in OIC. Such customers benefit from appreciating the extra business and technical value-add OIC brings to the table. I already mentioned the plethora of OCI services available, but there are other Oracle Cloud services out there, such as ATP, which may be relevant.

Another area to look at is Exception Handling. Many SOA Suite customers leverage the declarative policies of the Error Hospital - this may require custom work in OIC. Again, the mantra is what needs to be done, not this is how we did in SOA Suite.

A fool with a tool is still a fool, albeit a more dangerous one. OIC does free us from having to have a deep knowledge of the underlying engine in order to design and performantly manage integrations. But like any other design time, one can implement anti-patterns that may impinge on performance etc. This is where it really does pay dividends to read the Using Integrations in Oracle Integration guide, available here. 

    

The chapter begins with a discussion of the various integration patterns OIC supports - here are the main ones 

This will be one of the major decisions you will make, when re-designing your SOA based integrations for OIC - which pattern to use?

Service Bus - App Driven Orchestration

Synchronous - App Driven Orchestration

Async/Fire and Forget  - App Driven Orchestration -  with no callback, just an End activity.

MFT - File Transfer  

Scheduled/Batch processing - Scheduled Orchestration

The guide also contains a discussion on common anti-patterns and how to avoid them. A typical example would be "chatty" a integration that processes a large amount of customer records from SFDC, retrieving each individually, instead of leveraging the batch processing support (e.g. 200 customers at a time) offered by the OIC SFDC adapter.

Another thing to avoid is very large synchronous integrations - here one can look at implementing such as fire and forget, or maybe as async response. Here is an example of the latter - in an order processing integration, return an acknowledgement of order receipt to the calling client and then proceed with the complex processing.

Net, net - well worth reading the docs, before starting.

Another aspect to consider is Event Driven Integrations - e.g. Purchase Order creation in Fusion ERP triggers an integration flow in OIC. 

Developing integrations with OIC is very simple compared to SOA composite development in JDeveloper. There are 2 main areas of concern in the former - your connections and your integrations. I need to implement an opportunity to order integration between SFDC and Netsuite? Then step 1 is to create the 2 connections - SFDC and Netsuite. Then I can leverage those connections in my integration.

  

As already mentioned, OIC comes with over 70 adapters you can use for connection creation. Many of these are bidirectional - supporting Trigger and Invoke. Trigger (Inbound) - the app triggers an integration flow, as in my Fusion ERP Purchase Order example above. Invoke (Outbound) - OIC integration invoking the creation of a Purchase Order in Fusion ERP. 

The guide also lists the adapters supporting Trigger and Invoke. Naturally, this is also something you can consider when deciding on how to implement your SOA based integrations in OIC.

Implementing SOA integrations in OIC

Discovery / Design / Consolidate / Re-Engineer

Pre-requisite is a knowledge of OIC, it's components and the technical/business value add they provide.

One needs to start off with a discovery and design phase. Discover which integrations are still being used, you may find there are some that are no longer being used, for whatever reason. Discover reusable components -

• wsdls (if you need to stay with SOAP interfaces)
• xsds
• xslts

Naturally, re-platforming to OIC allows re-engineering, maybe replacing SOAP interfaces with REST based apis. Also the fact that OIC includes much more adapters compared to SOA Suite, may allow you to leverage such going forward. Integration consolidation is also a possibility, considering one could be dealing with a SOA real estate that has grown over a period of many years.

Once you have decided on which integrations you will need, select the appropriate OIC product and pattern for each of them. The previous section discussed the various patterns - sync, one-way /async, scheduled, file transfer, event driven etc.

There may be a need to include data staging/caching, here's where ATP can play a decisive role. 

Also one may need to consider an archiving strategy for storing logs etc, maybe for compliance purposes. One should not forget the format of the data here. Going thru Activity Stream data may not be the way to go for compliance officers; for example, if you need to prove order 123 was processed on May 2nd 2021.

Error Handling is another important area. OIC does not have Fault Policies ala SOA Suite. However, OIC does have a robust error handling framework - the following actions are available at design time -

  

This is complemented by a Global Fault Handler at integration level -

as well as Scope-level Fault Handlers - 

Again, as in SOA Suite, one requires clarity regarding what should happen when specific business and technical errors occur.

Integration Insight

As I mentioned earlier, OIC is a toolkit, with a set of tools that allow one to - connect apps - extend apps - implement customer human centric business processes - create net new apps, web and native mobile - create business user facing dashboards/ alerts on top of one's integrations and processes. 

The latter component - Integration Insight - should always be considered as part of your SOA to OIC move. It is non-intrusive, meaning it is extremely easy to map points in your integrations/processes to Insight Milestones. It gives your line of business owners real time insight into the integrations that affect them. This insight can take the form of rich visual dashboards, which can be embedded into any app - that means the HCM LOB folks can see the dashboards pertaining to the Employee Onboarding process directly in Fusion HCM. Alerts will inform business stakeholders when interesting events occur or do not occur - e.g. customer X just placed an extremely large order or the products ordered by customer Y were not shipped within the standard 8 hour period.

Finally, OIC comes with a plethora of Integration Recipes and Business Accelerators; these provide best practice integrations for common business and technical use cases, thus kickstarting your development with OIC.

  

One of the technical accelerators that may be especially interesting for SOA Suite developers is the Re-sequencer, allowing one to re-sequence messages for processing, e.g. I receive POs in the order 5,3,1,2,4 and need to process in the order 1,2,3,4,5. From what I remember, this is something done by the Mediator component in SOA Suite. Please look at this post for more details.

 

Conclusion 

Integration Insight is just one example of the business/technical value-add of moving to OIC. 

There are many more -  

- more adapters 

- more functionality including seamless access to all OCI services

- more productivity - easy and intuitive design time

- more insight for LOB folks

- more releases -ZDT functional releases every quarter, give you more functionality quicker.

- less management overhead - and by this I mean minimal management overhead, as this is a platform managed by Oracle.

Finally, SOA Suite is not going away - you can of course lift and shift to SOA Suite on OCI, you can remain on premise and leverage OIC for your new more cloud centric integration requirements.

Whatever you decide - Enjoy the ride!