#1057 - OIC - Tracing/Logging

Introduction 

In this post I will bring clarity to what is traced when in OIC. I will also discuss the broader area of OIC logging and give some recommendations. I will use the following integration for demoing what is traced. This, of course, depends on the trace level you select -

• Production
• Audit
• Debug  

Here is my simple integration - 

First test is with Debug -

Debug is not recommended for production, but it is very useful for your initial functional testing. 

So let's look at what's logged with Debug -

Debug Tracing

We'll go throw the activity stream, step by step - 

First log message contains the "wire" message sent to OIC.

The next message shows the internal, xml based, representation of the request - 

The next message is logging the start of the mapper action. It shows an empty <OrdersCollection>, the structure we will be passing to the database -

The next message shows the result of the map action - 

Next, we have the database invoke - 

Let's look at the first message, this is our request payload for the database insert - 

The next message contains the wire message sent to the DB adapter - this is in xml format - essentially the same as the previous screenshot - 

So think of these 2 as follows - the first is what OIC intends to send to the target. It is in xml format, which is the lingua franca of OIC. The second is the actual message sent - in this case it's still in xml format, but for other adapters this could be REST etc.

The final 2 actions with the invoke have, in this particular case of DB Insert, no payloads -

The next step is to map the response -

Here we just see the empty response structure.

However, the next log message shows the result of the response mapping action - 

Next we see the response OIC has prepared, again in the vernacular xml -

Finally, we see the REST response OIC has marshalled and returned - 

So very simple - let's add a couple of actions to the integration.

I run again in debug mode - 

Here are the new log messages - 

Net. net - debug tracing logs
everything, including payloads.

Audit Tracing

Now we run the same integration, but in audit mode - 

Here we see that the request and response wire messages are logged. The internal representation (xml) of request is not shown. We do not see what has been assigned to the variable in the Assign step. However, we do see the Logger message - 

Now to the DB invoke, Invoke insertOrder,  here we see the wire message sent to the database adapter -

We don't see any response, as there is none.

Finally we see the wire message returned by OIC - 

Net, net, with audit we get the inbound and outbound wire messages logged. We also see the output of any Logger and Notification actions. 

The audit level may suffice, if you need to log inbound and outbound payloads, for compliance purposes.

Production Tracing

So what's logged in production mode?

Firstly, the Logger action - 

Also the Notification action -

We see the steps executed for the DB invoke, but no payload data - 

We also do not see the request and response wire messages for the integration.

Net, net, Production is fine if you just need a log of the actions executed. You can, of course, use the Logger action to log any business data.  

Data Retention

The default data retention period is for 32 days. Net, net, just over a month of log data can be retained.

Debug data is retained just for 24 hours, the integration then reverts to production level trace.

Audit data is retained for 8 days, after 8 days the payload data is deleted and you're left with production style logging for up to 32 days.

Production trace data is kept for 32 days.

Again, this 32 days is the default, however, you can increase this - 

There will be a cost applied to this, going forward. This cost will not be applied to the OIC for Healthcare edition.

Leveraging OCI Logging and Logging Analytics

You can easily "push" a subset of the OIC Activity Stream to OCI Logging. From there it can easily be ingested into OCI Logging Analytics.

The big value adds here -

• you can retain the data for as long as you want
• you can create compelling dashboards in OCI Logging
• you can leverage a rich set of OIC specific dashboards in OCI Logging Analytics. You can also create your own OIC centric dashboards as well
• you can push OIC logs easily to OCI Object Storage for long term data retention.

It all starts here, go to your OCI page for your OIC instance and then click on the Logs link - you can create a target log group in OCI Logging, from here, and off your logs go!

Now to a simple example - I've activated the createOrder integration again, in "audit" mode. I run the integration with orderNr set to 310397.

I now open the OCI console for Logging - 

Select logs for search - default is root, but I just want the activity stream log for this particular OIC instance.

Here you see the order with the orderNr 310397 - 

Note the format of the activity stream log messages in OCI Logging. You will appreciate this is a subset of the OIC activity stream logs -  

The message field contains the incoming payload -

Summa Summarum

We have rich tracing/logging within OIC. We have a plethora of other tools available in OCI for further analysis and data retention. Check out my numerous posts on OCI Logging and Logging Analytics, to see how these services can augment the comprehensive Oberservability &  Monitoring you get OOTB with OIC.

And with that said, adieu! 
 

  

  

 




  

#1056 - OIC3 File Server Factory API

Introduction

You may be an OIC customer leveraging the File Server. You may also have use cases that require using the File Server REST API. If you're such a customer, read on!

Firstly, the File Server API docs are available here.

Always good to start there - now there are some subtle differences between usage of the OIC Integration Factory api and that for File Server. To begin with, you generate the OAuth token differently.

Generate a Token

So let's look at generating the token - here I'm using Postman - 

The url is - 

https://yourIDCS/oauth2/v1/token

As you can see, I use x-www-form-encoded as the format.

grant_type is set to password

scope is set to the 2 OIC scopes you use with the Integration factory api.

username and password is that of my OIC user.

Execute the POST request and you should get the token. If you get an unauthorized back, then check the IDCS app you are using - here is mine -

Validate IDCS App

Note, I've also checked Resource owner.

As you can see, I've added this app to the ServiceAdministrator role. There is a dependency between OIC role and what you can do with File Server. This is discussed in a upcoming section.

Net, net - you should be able to generate your token.

Using the token in a File Server API Request 

First api I invoke is - 

I realise the documentation format is somewhat different to the OIC REST API docs for integration etc. But don't fret - the protocol, host and base path are analogue to the integration api.

https://design.integration.yourRegion.ocp.oraclecloud.com/ic/api/fileserver/v1/services/sftp/configuration?integrationInstance=yourOICinstance

I copy and paste in the token and execute the request -

So that's the sanity test done. 

Now to creating a new folder...

Create a New Folder via the File Server API

Here's the Postman configuration - 

Request payload is - 

{"createdDate": "01-01-2025",

"name": "myFolder",

"parent": "{{fsPath}}"}

The {{fsPath}} refers to an environment variable I created -

 

Response - 

{

    "guid": "a75885b1e5dd473c8b9834b82bcd19d9",

    "path": "/home/users/niall.commiskey@oracle.com/myFolder",

    "createDate": null,

    "directories": [],

    "files": [],

    "links": [

        {

            "rel": "resource",

            "href": "/ic/api/fileserver/v1/filesystem/root/home/users/niall.commiskey@oracle.com/myFolder"

        },

        {

            "rel": "permission",

            "href": "/ic/api/fileserver/v1/filesystem/directories/a75885b1e5dd473c8b9834b82bcd19d9/permissions"

        }

    ]

}

I validate the creation of the new folder, by checking the File Server UI in OIC - 

OIC Roles & the File Server API

There is a dependency between OIC Roles and what you can do with the File Server REST api. The details are available here. 

Net, net - 

• ServiceAdministrator role for the managing file server via the REST api.
• ServiceDeveloper - read/write files via FTP Connection or StageFile action in OIC integrations.
• ServiceUser - can access the File Server via an ftp client, such as FileZilla.

So your user needs to have the ServiceAdministrator role assigned to use these APIs.

Summa Summarum

File Server provides an embedded SFTP server within Oracle Integration, enabling organizations to focus on building automated business processes, without needing to host and maintain a separate SFTP server. This is a huge value-add for OIC customers; the factory api makes it even easier to adopt and manage this component.

 

 

#1055 OIC3 New OCI Consumption Dashboards

Just in case you've notice the new consumption charts in the OCI Integration instance page.

These are new metrics which are surfacing and nothing like a simple example to explain them.

I run a test in SOAPUI, executing a simple sync integration 6500 times. I check out the Successful Messages chart - 

1086 messages. I then check the other data point.

+ 5414 = 6500 messages

I could also look at the table view - 

I could also change the interval to 1 hour - 

The Consumed Messages chart is next - 

also 6500 messages.

I've only allocated 1 message pack to this instance, so the Consumed Message Packs from Overage chart is as follows - 

The above chart shows 1 extra message pack, makes sense, as we've gone over the 5k limit of the 1 message pack assigned to the OIC instance.

The Total consumed message packs chart is as follows - 

So 1 assigned message pack + 1 "overage" message pack = 2 message packs.

 

#1054 - OIC3 unique outbound ip address with Private Endpoint

Introduction

You may have multiple OIC3 instances in the same region/datacenter. Check out the About box in OIC designtime and you will see - 

Now imagine you need to have a unique outbound ip. How can you have such?

Here is a simple example of using Private Endpoints to ensure your OIC instance has a unique outbound ip address.

Simple Example

I've obfuscated the outbound ips, but, believe me, they are the same for both OIC instances. Again, this is not tenancy specific, rather region or datacenter specific.

Again, the requirement here is a unique outbound ip address for each OIC instance.

The architecture is very simple -

The pre-requisite for setting up a Private Endpoint is a VCN - so let's set up one specifically for this.

Open the OCI Console go to Networking -> Virtual Cloud Networks -

I just accept all the defaults here - 

Now we can create the Private Endpoint, which will leverage the VCN we just created. 

Go to the OCI Console --> Developer Services --> Integrations. Select your OIC instance - 

Now to testing this - I have an integration that calls a service, which returns the outbound ip address.

I run the integration and see the 130.... ip address.

I now edit the connection I use for invoking that service. Here I will specify use of private endpoint - 

I run the integration again - ip has changed to 141...

So where does this 141... ip address come from?

Again go to your OIC console --> Networking --> Virtual Cloud Networks. Select your VCN then navigate to NAT Gateways - 

Check out the public ip address above.

Summa Summarum

Ensuring a unique ip address for your OIC instance(s) is easy to achieve, literally, just a matter of minutes.