#899 The OIC ecosystem - Integration & OCI Services - an Overview

Customers buy solutions not products - OIC is the solution for connecting your apps, extending your apps, gaining business insight into your business processes, creating net new web and native mobile apps on top of your app apis. But what if OIC is not enough for some use cases? Then remember OIC is a cloud native OCI service, one of many others - these "others" can be easily leveraged from OIC to provide the business solutions customers require.

 

The above graphic shows the OIC capabilities and the OCI Services discussed in this post. Seems a lot doesn't it? Especially when you consider that most customers will start of with a simpler set of requirements. But let's start with our EBS customer, who is transitioning to Fusion ERP. 

Yes, the graphic below is from me - apologies for it's basic nature - but I like keeping things simple -

The above details the basic requirements from a customer perspective. Note the rest of the customer's ecosystem is not shown here. There will probably be many other COTS and custom applications involved e.g. PeopleSoft, Siebel etc.

This post will not discuss what the customer plans to do with such - e.g. leave PSFT on-premise or maybe run it on IaaS or even transition to Fusion HCM. I also do not discuss the customer's current on-premise integration architecture. Maybe she is using SOA Suite and will need to run many of those integrations for the near future. In other words, the transitioning from EBS to Fusion ERP means the customer has 2 ERPs for a certain length of time; the same goes for integration solutions. There is bi-directional connectivity between OIC and SOA Suite, which makes this transition easier. So now, after all of those disclaimers, let's start...     

Data Integration may be interesting for the initial seeding of EBS data in Fusion ERP. It may also be useful for high volume database replication use cases - real time data lakes / operational reporting - look at Oracle GoldenGate here. Think of data integration for large amounts of data at rest that need moving somewhere (bulk data movement / preparation) i.e. after the business processes, that put the data there, have ended. Application integration is more focused on integrating (near) real time data between the apps in your ecosystem.  

I have already alluded to the fact that Application Integration is a must for ensuring communication between your ERP and its ecosystem - consider all your stovepipe apps and the horizontal business processes that traverse them. 

For example, that ecosystem will also contain your CRM, where you manage your customer relationship - contacts, opportunities etc. Application Integration is there to help keep those contacts in sync as well as implement the orchestration required for your Opportunity to Order business process, amongst a multitude of other integration use cases.

You are extremely happy with your ERP of choice, and sure why wouldn't you be? But what about that extra approval workflow that is specific to your organization? This is where App Extensions come into play. You need the ability to easily create such human centric workflows that provide that extra functionality you need. Those supplementary workflows also need to be embedded  in your ERP, so your ERP end users have a streamlined experience.

Now with all of these application integrations and extensions up and running, don't your line of business owners require insight into what's going on? How long does it take to process an order? Which products are selling really well at the moment? Where are those orders coming from? Can I drill down from aggregate information to individual orders? All this can be garnered from the integrations and processes you are running and easily visualised for line of business owners. These visualisations can also be easily embedded in the user's app of choice, be it ERP or a company portal.

Now to leveraging the power of ERP, outside of the app itself. What about native mobile apps, providing mashups of CRM and ERP data for sales reps? What about leveraging innovations such  as Conversational UI? OIC opens the door to Digital Innovation.

Net, net - the world of integration is much more than simply connecting App A to App B.

Oracle Integration (OIC) + Oracle Data Integration + OCI API Gateway + Oracle Digital Assistant (ODA) provide a compelling answer to the customer transitioning from EBS to ERP. But is that the whole story in respect of customer requirements?

Chapter 1 is a quick refresher on the functionality offered by OIC itself. too many folks concentrate purely on the application integration components. You will see that OIC covers most of the needs highlighted in the section above.   

OIC - Integration

Of course, that is where the initial customer focus will be - implementing the must have business use cases, such as transforming those won opportunities into orders as quickly as possible. Let's look at the different integration patterns supported by OIC  -

Transactional - e.g. retrieve customer's current orders for display in a sales rep's mobile app. Typical real-time request/response paradigm.

Batch/Scheduled - e.g. retrieve that humungous file of new orders every day at 5pm from each of our regional ftp servers, then do the magic on those orders so they can be imported into Fusion ERP.

File Transfer - Moving files from A to B. 

Event Driven - e.g. push an Opportunity to ERP (morphing it into an order), as soon as it is marked as won in CRM. Immediately react to the ERP business event - purchase order created, by pushing the order data to downstream systems. Many apps support such business events, the magic here is the ability of your Application Integration engine to hook into such events.

Event Driven Architecture (EDA) has been around for a long time, the difference now is applying this effectively to a cloud or multi-cloud environment(s). Oracle Integration (OIC) comes with a rich set of adapters, including messaging adapters - Kafka, AQ, JMS etc. But this is only half the message. What about OIC's Pub/Sub capabilities? Essentially we support EDA at the following levels in OIC /OCI -

1. Event Driven / Pub-Sub between OCI Services

2. Event Driven / Pub-Sub within OIC

3. Event Driven - SaaS Business Events, which OIC can subscribe to via its rich set of SaaS adapters.

More about these later. Let's continue looking at the business value-add OIC provides.

OIC - Process

Process can be used for those human centric app extensions. Naturally, it can also be used for any of your business processing in need of automation. From the replacement of the excel sheet based vacation request to complex approvals of orders and resolution of insurance claim cases - Process can handle the lot. It supports structured and unstructured processes, enabling you to cope with uncertainties.      

Business Processes can involve documents, think of an order and its attendant documentation e.g. export license etc. These documents can be stored inline in OIC - Process, however, one could also leverage Content and Experience for this. More on Content Management later. 

OIC - Insight

Insight provides you with those business user dashboards on top of your integrations and processes - 

  

These dashboards can be viewed within OIC or else embedded in the user's app of choice. In this case, we could embed in Fusion ERP. 

OIC - Visual Builder 

VB is a low code component that can be used to build native mobile and web apps. It is tightly integrated with OIC - Integration and OIC - Process. Fr example, if you do not like the default Tasklist that ships with Process, then create your own in VB.  

      

So that's our application integration, app extensions and biz insights boxes ticked. OIC provides us with comprehensive support in this area. What about the customer's other requirements?

OCI - OIC's Extended Toolbox

OIC - Integration takes care of the plumbing between your apps / technologies. In respect of OCI, we have some dedicated adapters for specific OCI Services -

We just leverage the rich OCI REST api for the other services -

The secret sauce here is the OCI Signature Version 1 security policy. Here we have a generic way to connect to the plethora of OCI Services. 

Talking about services, let's begin with API Management and the OCI API Gateway -

OCI API Gateway

Nearly every OIC customer will want to expose at least a subset of their integration apis, maybe to their own developers or those of partners. Think of the mobile app developer needing to integrate with enterprise systems of record - e.g. get outstanding customer orders from ERP. Wouldn't it be great just to point them to the REST based api of the relevant OIC integration that does the business; allowing the developer to simply invoke the functionality without having to concern herself about the actual implementation?

OIC can easily leverage your OCI API Gateway - I have 2 Gateways, as you can see. I will use the testGW in this example -

First step is to configure OIC to point to my API Gateway component - configuration in OIC is simple -

Now to publish an integration api - 

I select the testGW gateway and click Deploy, naturally, I could have changed the path prefix to make it more user friendly. I then check out the gateway deployments, and there is my OIC api  - 

Now I can apply whatever policies are required and my mobile app developer has access to the customer data she requires! The api gateway may be all the customer requires in respect of api management/security. But imagine our Fusion ERP customer needs to expose apis to 3rd parties. We may need to protect the customer from malicious and unwanted internet traffic. Here we can leverage OCI WAF (Web Application Firewall) in conjunction with OCI API Gateway and OIC.

OCI WAF

Oracle Cloud Infrastructure Web Application Firewall (WAF) is a regional-based and edge enforcement service that is attached to an enforcement point, such as a load balancer or a web application domain name. WAF protects applications from malicious and unwanted internet traffic. WAF can protect any internet facing endpoint, providing consistent rule enforcement across a customer's applications.

the rest in a couple of bullet points - 

• PCI-compliant
• Cloud Native 
• Integration and Automation through APIs, CLI. 
• Global security service
• Distributed PoP 
• Protect any Internet-facing endpoints
• Cross-Site Scripting
• SQL Injections
• Known vulnerabilities
• Bot Management
• Layer 7 DDoS protection
• Threat Intelligence

Also note, WAF is very interesting for OIC customers when hitting the max number of entries allowed when configuring OIC Allowlist.

Thanks to my colleagues Antonio B., Valeria C. and Baptiste S. for the WAF content! 

So which other OCI services should we be highlighting to our customer? This is totally dependent on what she needs to do today, her plans for tomorrow and further down the road. So think of tactical and visionary - OIC + OCI supports both!

ATP

Oracle offers many database options on OCI. Here I will focus on ATP. Our Fusion ERP customer could be looking at ATP for the following reasons -

1. Transform multiple Data sources into the Oracle FDBI Templates using ATP - e.g. our ERP customer has inbound files of between 4 - 10 different source systems, which need to be transformed into the relevant import format for GL Journals, AR Invoices, AP Invoices.

2. Data Caching - OIC does not currently provide explicit caching functionality. There are many integration scenarios where data is being fetched from various sources. The data doesn't change much, so caching would make sense. Think of an HCM use case - customer has purchased Oracle HCM cloud but also has SAP on-premise. The latter is still the single source of truth when it comes to base employee data. Caching this data in ATP would appreciably accelerate integration throughput.     

OIC can, via the ATP adapter, easily leverage this "cache" in its integrations and processes.

3. Superfast file loading to ATP - OIC supports super fast bulk data import to ATP. Check out my post of this topic here

4. ATP for complex data centric processing logic -  e.g. processing a large file where, for each line,  I need to check if there is a duplicate in the file. This is very easily done in PL/SQL, once the data has been loaded into a table. Also useful for requirements around complex grouping of data.

Again, super easy for OIC to execute PL/SQL via the ATP adapter. This leads to faster processing and appreciably less complexity in your integrations.

5. ATP for OIC Audit Logging - Some customers also leverage ATP for offloading audit data from their integration flows. The data could be compliance related, so having it in a DB makes it easily retrievable and available to other tools.

6. ATP as the back end to use in near-real time analytics - 
(Data_Sources_Silos-->OIC-->ATP-->OAC).

OAC = Oracle Analytics Cloud. One can extract metrics/KPIs from the Integration flows and Process workflows. Very useful if the customer needs more B/I style "flexibility", above and beyond what OIC Insight offers.

7. ATP for throttling - Downstream throttling (Parking Lot Pattern) - Check out the following A-Team post for details.

The reasons so far have been mainly tactical, they make sense for specific use cases or a combination of them, but ATP can also be seen in the context of your Digital Integration Hub.

Functions

Oracle Cloud Functions is a serverless platform that lets developers create, run, and scale applications without managing any infrastructure. Functions integrate with Oracle Cloud Infrastructure, platform services and SaaS applications..

So what does that mean to our Fusion ERP customer?

Think extensibility, supplementary complex processing logic etc.- value add that can be implemented in a variety of languages - Java, Python, Go, Node etc.

With OIC, you can easily invoke OCI Functions from your integrations and processes. The following blog post introduces you to the subject of invoking OCI functions from OIC. Check it out here.

The follow up post here covers creation / invoke of python based functions from OIC.

Streaming

The Oracle Cloud Infrastructure Streaming service provides a fully managed, scalable, and durable solution for ingesting and consuming high-volume data streams in real-time. Use Streaming for any use case in which data is produced and processed continually and sequentially in a publish-subscribe messaging model.

The Pub-Sub pattern has a lot going for it - loosely coupled systems etc. The OIC UI still seems to support Pub/Sub - but like, Basic Routing, the other pub/sub patterns have been deprecated -

 

As you've probably guessed, I have a couple of blog posts that tell you how to exactly do this - implement pub/sub with OIC and OCI Streaming. A trilogy of posts is available -

Part 1, Part 2, Part 3

I have always taken the Gaelic saying - Bi Ullamh! - Be Prepared - seriously!

There are, of course, other use cases for OCI Streaming in respect of OIC and our ERP customer. Imagine our customer has a website and a very popular one at that. She wants to get user behaviour data - who looked at which product when - into her CX Unity environment. 

Use Streaming for capturing activity from websites or mobile apps, such as page views, searches, or other user actions. You can use this information for real-time monitoring and analytics, and in data warehousing systems for offline processing and reporting.

Website -> OCI Streaming -> OCI Streaming Adapter -> OIC -> CX Unity Adapter -> CX Unity

Object Storage

OCI Object Storage can be used as a store for your OIC artifacts -

First step is to create an OCI Storage bucket and then register it with OIC - 

Note the format of the Swift URL. Also the format of the User Name. The password is NOT your usual OCI password. It is an Auth Token you generate at OCI User level - 

 

Now this connection can be used in exports/imports - 

 

 

Let's look at the export - 

The content of the .zip is as follows - 

You can export from one OIC instance, import to another, or just OCI Object Storage as your OIC backup store.

This service can, of course, be used for storing whatever you want, for as long as you want. You also saw in the ATP section how Object Storage is used to enable bulk data load to ATP, via OIC. 

OCI Object Storage is one of those unsung heroes - maybe we need to sing its praises more!

OCI Logging / OCI Logging Analytics / Service Metrics

OIC does provide comprehensive monitoring functionality; however, this is complemented at OCI level. 

I begin with OCI Service Metrics, which provides us with salient dashboards and the ability to create our own queries - 

Here is a simple integration I will use - AA-CreateContactsNS-SFDC - 

 

It reads the new contacts file from FTP and then inserts those new contacts into SFDC and Netsuite - 

Here is the Service Metrics view - 

Now I want to check how long the SFDC invokes took - 

Excellent stuff! Now to OCI Logging...

OCI Logging Service

It is simple to push the OIC Activity Stream logs to OCI Logging. Just enable this for your OIC instance.

From here, the logs could be pushed to OCI Object Storage, if our ERP customers needs to retain them for compliance purposes. Here we use Logging --> Service Connectors -

 It is a simple matter of specifying source and target. Service Connectors support the following targets - 

This is a great segue into OCI Logging Analytics, one of our targets.

OCI Logging Analytics

Our focus here is on providing a dashboard to allow monitoring of multiple OIC instances, using data from Service Metrics and OIC Activity Stream logs. This allows me to include technical and business metrics in the dashboard. e.g. Average time taken for SFDC invokes, the most popular product from an order perspective 

Again, I have a plethora of blog posts about how you can leverage OCI Logging and OCI Logging Analytics in respect of OIC. Check them out here

OCI Events / Notifications

Here we are looking at pub/sub between OCI Services.

So what's happening here? 

• Event fired on file upload to a particular bucket in OCI Object Storage
• Event Service Rule picks up this event and publishes to OCI Notification Service
• Topic can have 1 or more subscribers

One of these could be  custom http -> invoke api gtwy -> OIC. Let's look at the configuration required to set this up - 

Step 1 - configure Object Storage bucket to emit events - 

Step 2 - configure Notifications Service Topic - 

Step 3 - configure Event Service Rule - 

As you can see, it picks up the object storage event and posts to the OICTopic we just created.

Step 4 - Add a subscriber to the Topic - 

I have added two subscribers - function and email.

Now to testing this out - I upload a file to the bucket - 

I check for the email - 

btw. here's the jpeg - the magnificent skyline of Baile Atha Cliath.

Endless possibilities here.

OCI AI Language

Language provides you with the artificial intelligence and machine learning capabilities to detect the language in your unstructured text. Also, it provides other tools to help you gain further insights about your text e.g. sentiment analysis. Think of the following situation - we are processing orders and one of the fields - comments - allows the customer to enter free text. Wouldn't it be great to get the sentiment out of those comments? OIC could then push this info downstream to our ERP customer's CRM.

Again, I have some dedicated posts on this - check them out here.

OIC in a Multi-Cloud World

OCI Fast Connect / OIC Connectivity Agent / OCI VPN

In our example, our Fusion ERP customer has adopted Oracle Cloud Infrastructure (OCI) and Google as cloud providers. Some integration use cases will involve integrating Fusion ERP and apps running on Google Cloud, such as Big Query. Latency may be a concern here, maybe not in respect of integration here, but for other near real-time use cases. OCI Fast Connect can play a crucial role here. 

FastConnect allows customers to connect directly to their Oracle Cloud Infrastructure (OCI) virtual cloud network via dedicated, private, high-bandwidth connections. Then, based on the amount of data, customers simply choose an appropriate port speed and pay a consistent, low price each month. Other providers charge for the amount of data moved, which can be expensive and unpredictable. 

Net, net OCI FastConnect will give our Fusion ERP customer a Fast, Secure and Dedicated communication channel - excellent for her very demanding multi-cloud workloads.

OIC itself provides a lightweight connectivity agent which works on the call home principle - it can be installed on the Fusion ERP customer's on-premise data center to enable secure communication between EBS and OIC. The connectivity agent can also be installed on IaaS in other clouds. This is the method of choice for most OIC customers.   

OCI VPN allows you to securely connect your existing infrastructure to the cloud by using industry-standard encryption algorithms. so this is also an option for securely connecting to workloads in other providers’ cloud services. VPN can provide a redundant connection to Oracle Cloud Infrastructure, for those already using FastConnect. 

Conclusion

Business value add is the best reason for adopting OIC and it's ancillary OCI Services. I hope this post whetted your appetite and apologies up front for not including Oracle Analytics Cloud (OAC), Machine Learning and Oracle Digital Assistant (ODA); expect separate posts on those services soon. 

My parting advice - take the helicopter view of your ecosystem and what you need to do today. It may be more than you think. Also consider what you could do today to get business advantage. Don't be afraid of being a digital warrior, make innovation your strength! 

Remember, integration is more than connecting A and B.

Thanks to my colleague Ravi P. for his support in producing this post.

Note: text in italics is from the OCI docs.