#1087 - OIC Scheduled Jobs monitoring

Introduction

Most, if not all, OIC customers use scheduled integrations, as there are many such requirements in the integration space e.g. daily upload of new orders to Fusion ERP etc. FBDI is key to Fusion apps, so scheduled integrations have a large role to play here. This post will look at monitoring scheduled jobs, via the factory api. The reason for doing such may be to get salient data, create mashups from that data or export the data to 3rd party apps for further processing. Typical questions are -

• what is the mean execution time for certain scheduled integrations?
• are my scheduled integrations executing on time?
• are all scheduled runs being executed?

OIC Observability will answer most of these, i.e. within OIC itself you get the answers to many questions. However, you may need to extract key execution data and import it into a 3rd party app, such as Splunk. You could also have other questions that are currently not covered by OIC Observability.

I'll begin with a couple of facts about scheduled integrations and also some best practices. Then we get to our demo project, which contains a couple of scheduled jobs. Finally, on to the factory api to extract salient data.

Facts and Best Practices 

1. Scheduled Integrations are treated as asynchronous integrations, thus adding to the service limit - concurrent asynchronous requests.
2. This limit is dependent on the number of message packs you have assigned to your OIC instance.
3. Schedule integrations execute as singletons, i.e. only one instance of a specific scheduled integration can be active at one time.
4. Use the following pattern, if non-singleton - scheduled integration --> async integrations(s). In other words, offload the bulk of the work from the scheduled integration to an async integration, if possible. This will ensure your scheduled integration run completes quickly, thus making a more efficient use of resources.
5. You can define a schedule for your scheduled integrations; try and spread out the runs over the day, i.e. don't have all starting at 17:00 every evening, if possible from a business perspective.
6. The next run is queued after the previous run completes. This may cause issues, if your schedule is too tight, e.g. run every 10 minutes. For example - Future Runs are -

• 10:00 am
• 10:10 am
• 10:20 am
• etc.

        The 10 am run takes longer than 10 minutes, this will cause the 10:10 am run to be skipped.

Sample Project and using the Monitoring Factory APIs

 

Here is my sample project - 

The integration processInvoicesBatch should run every 2 hours; processOrdersBatch is scheduled to run every hour. Both include a wait, initially set to 300 seconds - 

I check out the future runs page in project observability - 

Retrieve aggregated data for each scheduled integration

Now let's get some high level data via the factory api - 

https://design.integration.us-phoenix-1.ocp.oraclecloud.com/ic/api/integration/v1/monitoring/integrations?offset=0&limit=999&q=%7BprojectCode%3A+%27AA_PROJECT2%27%2Ctimewindow%3A%271d%27%2C+status%3A%27ACTIVATED%27%7D&orderBy=time&return=monitoringui&integrationInstance=yourOICInstance

Here is the response - 

"dataFetchTime": "2025-10-07T08:36:22.204+0000",

    "items": [

        {

            "code": "PROCESSORDERSBATCH",

            "flowStatus": "ACTIVATED",

            "id": "PROCESSORDERSBATCH|01.00.0000",

            "lastUpdated": "2025-10-07T08:32:53.195+0000",

            "lastUpdatedString": "7 Oct 2025 08:32:53 GMT",

            "links"...

            "mepType": "MEP07",

            "name": "processOrdersBatch",

            "nextRunDateString": "No Data",

            "noOfAborted": 0,

            "noOfErrors": 0,

            "noOfMsgsProcessed": 0,

            "noOfMsgsReceived": 1,

            "noOfSuccess": 0,

            "optimizedVersion": "1.0",

            "projectCode": "AA_PROJECT2",

            "scheduleApplicable": true,

            "scheduleDefined": true,

            "scheduleStatus": "ICS_SCHEDULE_ACTIVE",

            "successRate": 0,

            "version": "01.00.0000"

        }

    ],

    "links"...

    "timeWindow": "24",

    "totalResults": 1

}

The first run is scheduled for 10:42:52 - this timestamp doesn't appear in the initial response.

I wait until the first run starts and execute the api call again - 

"dataFetchTime": "2025-10-07T08:43:34.020+0000",

    "items": [

        {

            "code": "PROCESSORDERSBATCH",

            "flowStatus": "ACTIVATED",

            "id": "PROCESSORDERSBATCH|01.00.0000",

            "lastUpdated": "2025-10-07T08:42:52.476+0000",

            "lastUpdatedString": "7 Oct 2025 08:42:52 GMT",

            "links":...

            "mepType": "MEP07",

            "name": "processOrdersBatch",

            "nextRunDateString": "No Data",

            "noOfAborted": 0,

            "noOfErrors": 0,

            "noOfMsgsProcessed": 0,

            "noOfMsgsReceived": 1,

            "noOfSuccess": 0,

            "optimizedVersion": "1.0",

            "projectCode": "AA_PROJECT2",

            "scheduleApplicable": true,

            "scheduleDefined": true,

            "scheduleStatus": "ICS_SCHEDULE_ACTIVE",

            "successRate": 0,

            "version": "01.00.0000"

        }

    ],

    "links":...

    "timeWindow": "24",

    "totalResults": 1

}

Now the first run has completed and the second run is queued - 

back to the api - 

{

    "dataFetchTime": "2025-10-07T08:52:05.217+0000",

    "items": [

        {

            "code": "PROCESSORDERSBATCH",

            "flowStatus": "ACTIVATED",

            "id": "PROCESSORDERSBATCH|01.00.0000",

            "lastUpdated": "2025-10-07T08:47:52.732+0000",

            "lastUpdatedString": "7 Oct 2025 08:47:52 GMT",

            "links":...

            "mepType": "MEP07",

            "name": "processOrdersBatch",

            "nextRunDateString": "No Data",

            "noOfAborted": 0,

            "noOfErrors": 0,

            "noOfMsgsProcessed": 1,

            "noOfMsgsReceived": 2,

            "noOfSuccess": 1,

            "optimizedVersion": "1.0",

            "projectCode": "AA_PROJECT2",

            "scheduleApplicable": true,

            "scheduleDefined": true,

            "scheduleStatus": "ICS_SCHEDULE_ACTIVE",

            "successRate": 100,

            "version": "01.00.0000"

        }

    ],

    "links":...

    "timeWindow": "24",

    "totalResults": 1

}

the lastUpdated timestamp is the completion timestamp. I see 1 message successfully processed and 2 messages received.

Now the second run has completed and the 3rd is queued - 

{

    "dataFetchTime": "2025-10-07T09:03:09.877+0000",

    "items": [

        {

            "code": "PROCESSORDERSBATCH",

            "flowStatus": "ACTIVATED",

            "id": "PROCESSORDERSBATCH|01.00.0000",

            "lastUpdated": "2025-10-07T09:02:52.654+0000",

            "lastUpdatedString": "7 Oct 2025 09:02:52 GMT",

            "links":...

            "mepType": "MEP07",

            "name": "processOrdersBatch",

            "nextRunDateString": "No Data",

            "noOfAborted": 0,

            "noOfErrors": 0,

            "noOfMsgsProcessed": 2,

            "noOfMsgsReceived": 3,

            "noOfSuccess": 2,

            "optimizedVersion": "1.0",

            "projectCode": "AA_PROJECT2",

            "scheduleApplicable": true,

            "scheduleDefined": true,

            "scheduleStatus": "ICS_SCHEDULE_ACTIVE",

            "successRate": 100,

            "version": "01.00.0000"

        }

    ],

    "links": ...

    "timeWindow": "24",

    "totalResults": 1

}

You get the idea. 

Retrieve instance data for each scheduled integration flow

Now to another api - retrieving each instance

https://design.integration.us-phoenix-1.ocp.oraclecloud.com/ic/api/integration/v1/monitoring/instances?offset=0&limit=50&q=%7Btimewindow%3A%271h%27%2C+includePurged%3A%27no%27%2CprojectCode%3A%27AA_PROJECT2%27%7D&orderBy=lastupdateddate&fields=detail&integrationInstance=yourOICInstance

Here is the current status from project observability -

The api response is as follows - I get 4 results - here is the data for a completed flow - 

{

    "dataFetchTime": "2025-10-07T09:11:40.990+0000",

    "hasMore": false,

    "id": "instances",

    "items": [

        {

            "creationDate": "2025-10-07T08:57:53.453+0000",

            "date": "2025-10-07T09:07:52.363+0000",

            "duration": 299788,

            "flowType": "SCHEDULED",

            "hasRecoverableFaults": false,

            "id": "tbZNIaNbEfCM5rOh3FVr_w",

            "instanceId": "tbZNIaNbEfCM5rOh3FVr_w",

            "instanceReportingLevel": "Production",

            "integration": "PROCESSORDERSBATCH|01.00.0000|AA_PROJECT2",

            "integrationId": "PROCESSORDERSBATCH",

            "integrationName": "processOrdersBatch",

            "integrationVersion": "01.00.0000",

            "invokedBy": "niall.commiskey@oracle.com",

            "isDataAccurate": true,

            "isPurged": false,

       

            "mepType": "SCHEDULED",

            "nonScheduleAsync": false,

            "opcRequestId": "oci-B3DDF17AC79D1BA-202510070832/F8099A9604244C7B99F259BF18E138A6/EAC9569AD2204496B18B4E0248CFA0DC",

            "outboundQueueNames": [],

            "processingEndDate": "2025-10-07T09:07:52.363+0000",

            "projectCode": "AA_PROJECT2",

            "projectFound": true,

            "projectName": "AA-Project2",

            "receivedDate": "2025-10-07T09:02:52.575+0000",

            "replayable": false,

            "replayed": false,

            "status": "COMPLETED",

            "trackings": [

                {

                    "name": "param_wait",

                    "primary": true,

                    "value": "300"

                }

            ]

        },

Here is the data for a scheduled flow -  

        {

            "creationDate": "2025-10-07T09:07:52.340+0000",

            "date": "2025-10-07T09:07:52.340+0000",

            "fifo": false,

            "flowType": "SCHEDULED",

            "hasRecoverableFaults": false,

            "id": "Gq0NFKNdEfCM5rOh3FVr_w",

            "instanceId": "Gq0NFKNdEfCM5rOh3FVr_w",

            "instanceReportingLevel": "Production",

            "integration": "PROCESSORDERSBATCH|01.00.0000|AA_PROJECT2",

            "integrationId": "PROCESSORDERSBATCH",

            "integrationName": "processOrdersBatch",

            "integrationVersion": "01.00.0000",

            "invokedBy": "niall.commiskey@oracle.com",

            "isDataAccurate": true,

            "isLitmusFlow": false,

            "isLitmusSupported": false,

            "isPurged": false,

            "lastTrackedTime": "2025-10-07T09:07:52.340+0000",

            "links": ...

            "litmusResultStatus": "",

            "mepType": "SCHEDULED",

            "nonScheduleAsync": false,

            "opcRequestId": "oci-B3DDF17AC79D1BA-202510070832/B2065CFB86E94F549B34CD5489FC5B32/3569DF4A925A44D486AE282C1EC2F70C",

            "outboundQueueNames": [],

            "projectCode": "AA_PROJECT2",

            "projectFound": true,

            "projectName": "AA-Project2",

            "replayable": false,

            "replayed": false,

            "scheduledTime": "2025-10-07T09:12:52.000+0000",

            "status": "SCHEDULE_WAITING",

            "trackings": [

                {

                    "name": "Primary",

                    "primary": true,

                    "value": "undefined"

                }

            ]

        },

scheduledTime is the time this should start running, I check in the activity stream -   

It started executing at 11:12:52:493

The 2 hour difference is because of UTC vs CET.

The scheduledTime value is only returned when the status is SCHEDULE_WAITING.

So let's just recap, before we continue -The first api gives us an aggregated overview of the schedule integration, the second, one record per instance.  

Retrieve the future runs schedule

The next api is used to retrieve the future runs schedule - 

https://design.integration.us-phoenix-1.ocp.oraclecloud.com/ic/api/integration/v1/monitoring/futureruns?q=%7Bstartdate%3A%272025-10-05+22%3A00%3A00%27%2Cenddate%3A%272025-10-12+21%3A59%3A59%27%2CprojectCode%3A%27AA_PROJECT2%27%7D&returnAll=true&timezone=Europe%2FBerlin&integrationInstance=yourOICInstance

The result - 

{

    "items": [

        {

            "code": "PROCESSORDERSBATCH",

            "futureRuns": [

                {

                    "runTime": "2025-10-07T11:42:52.971+0200"

                },

                {

                    "runTime": "2025-10-07T11:52:52.971+0200"

                },

                {

                    "runTime": "2025-10-07T12:02:52.971+0200"

                },

                {

                    "runTime": "2025-10-07T12:12:52.971+0200"

                },

                {

                    "runTime": "2025-10-07T12:22:52.971+0200"

                },

                {

                    "runTime": "2025-10-07T12:32:52.971+0200"

                },

                {

                    "runTime": "2025-10-07T12:42:52.971+0200"

               

We can now bring all these together - Let's invoke the 3 apis from an integration -

I begin with the Aggregated stats - that the following api - 

https://design.integration.us-phoenix-1.ocp.oraclecloud.com/ic/api/integration/v1/monitoring/integrations?offset=0&limit=999&q=%7BprojectCode%3A+%27AA_PROJECT2%27%2Ctimewindow%3A%271d%27%2C+status%3A%27ACTIVATED%27%7D&orderBy=time&integrationInstance=yourOICInstance

My integration switches on integration type, as I'm only interested in scheduled jobs -

Next step is to get salient data on executed runs - 

I log the following data - 

Integration: PROCESSORDERSBATCH Instance Id: E2wGsKNrEfCM5rOh3FVr_w StartTime: 2025-10-07T10:52:52.317+0000 EndTime: 2025-10-07T10:57:52.642+0000 Duration (msecs): 300325 Status: COMPLETED

Duration in msecs is 300325 which translates to 5 minutes 325 msecs.


Here is the activity stream view for this instance - 

Next step is getting the timestamps for the future runs - 

Summa Summarum

Granted this is a very basic implementation, however, it does give us the salient information we require for monitoring scheduled integrations.

The high level data tells us how many instances have executed and what the outcomes were ('COMPLETED' etc.)

The next invoke gives us intel on what has already executed - when the run started, how long it took, and whether it was successful or not.

The final invoke gives us intel on the future runs schedule. With all this data, we could work out min/max/mean execution times. We can also see whether the schedule is being adhered to, i.e. whether runs are executing on time.

#1086 - OIC Observability - View Slowest Activities

Introduction

This feature has been available for some time now; it's availability doesn't jump out in your face, so let's look at it in some detail.

View Slowest Activities 

This feature is available in instance tracking. The goal here is to give you insight into where most of the flow execution time is being spent in respect of the actions you have defined in your integration.

Just to make sure there is no ambiguity, we can look at the execution time as being divided into 2 areas -

1. The time taken to execute the actions you have defined within your integration. In the following case that would be - Map(writeFile), File server(writeFile) and Map(saveFile). The trigger is an exception here. Let's classify this as infrastructure. 
2. infrastructure - triggers, transitions between actions etc.

As you see above, the integration has been run with the trace level set to debug. Let's check out the slowest activities.

Here we see that writing the file to OIC File Server, via the file server native action, consumed almost half the execution time.

I run the same integration again, this time with trace set to audit - 

As you can see, the Map actions are not included. 

Production level trace is the same - 

Summa Summarum

This feature is extremely useful, especially in the development phase, where you are testing in debug mode. It is also very useful in production, for seeing where processing time is being spent, in respect of invokes. You can always flip between production / debug tracing to get more data - 

If you need data over a longer time period, then look at OCI Service Metrics, e.g. Outbound Request Invocation Time. Here you can investigate the time taken by your invokes over whatever period of time you want. You also get the rich variety of stats -

 

 

 

 

   

#1085 Giving OIC Monitors access to OIC

Introduction

When I'm playing around with my OIC instance, I'm usually doing so as a user with the Service Administrator role. However, customers will need to apply more fine grained security, for example, if I have folks that need to monitor OIC and only that, then that is all they should be able to do. This post will detail how to do this.

We will be looking at monitoring within OIC and without. Without refers to OCI. Here OIC users can leverage OCI Logging, OCI Service Metrics and OCI Log Analytics as well as OCI Alarms etc. 

So let's begin with our OIC monitor and let's create a group for such -

Only one user is currently assigned to this group - 

This group is assigned to the Service Monitor role - 

Here I can assign the group I just created - 

I now login to the OIC instance - 

I click on OIC Observability - 

I now navigate to Projects - I can see projects, but, naturally, cannot access them -

I can give the monitor user/group access to specific projects, for monitoring purposes - 

Now my monitor can access observability for the AA-HCM-ONLY-Project -

Now let's move to OCI. Here we will enable our monitors to check out the OCI Service Metrics for OIC -

Let's look at the 2 policy statements -

I begin with the service metrics -

allow group oci-oic-monitors to read metrics in compartment yourCompartment

Now our monitor can login to OCI and navigate as follows -

As you can see, the relevant metric namespace is oci_integration.

The next policy statement gives access to OCI Logging - 

allow group oci-oic-monitors to use log-content in compartment yourCompartment

You can enable OIC to push it's activity stream logs to OCI Logging. Check out my other posts on this topic to see what is actually forwarded to OCI Logging and what you can do with the data once it is available there.

Naturally the monitor can see all activity stream data from all projects.

So that is something you have to consider, when granting access.

Now on to OCI Log Analytics - by default, our monitor user does not have access -

We need to add the following policy statements -

Now my monitor user can access the OOTB OIC dashboards in OCI Log Analytics -

Now to OCI Alarms - 

per default, our monitoring user does not have permissions to create an Alarm -

This can be rectified by adding the relevant policy - verb depending on what you want to allow.

The verbs are - inspect, read, manage. I want to allow the monitors to create alarms, so I add the following statement to my policy - 

Allow group oci-oic-monitors to manage alarms in compartment yourCompartment

We also need the permission to create a topic, because we want an email sent, when the alarm fires.

allow group oci-oic-monitors to manage ons-topic in compartment yourCompartment

Let's try and create that alarm again -

 

#1084 OIC3: Monitoring performance of target endpoints

Introduction 

How can we monitor specific endpoints in OIC? Here I'm referring to outbound invokes to 3rd party services. 

For example, I have an integration that retrieves SR data from Fusion. This integration is being invoked by a call centre UI, so I need to monitor performance. Here's my simple SR retrieval integration - 

Here's the SR in Fusion - 

Monitoring using OCI Service Metrics 

The metric we need is - 

Check out the Dimensions - 

adapterIdentifier - 

As you've probably guessed, erp refers to the Fusion ERP adapter.

Here we see the GetServiceRequests endpoint.

You can choose from a variety of time intervals and statistics - 

Before looking at the graph, let's check the times in OIC Observability -

The first flow took ca. 17 secs, the second, just over 1 second.

Here is the Fusion ERP invoke stats for the first flow -

It took 16s 986ms for the getSR request to complete.

Now back to OCI Service Metrics; I've clicked Update Chart, so let's view it - 

I run the integration 10 times and now choose the Mean statistic. This will probably be the most popular, however, you may be interested in Max etc. 

Here the mean is down to ca 300 msecs.

Just to recap what I've covered - the metric Outbound Request Invocation Time can be used to give us an insight into overall erp adapter performance. This we get by adding the dimension adapterIdentifier and selecting erp.

We can also drill deeper, by adding the dimension outboundInvocationEndpointInformation and selecting our endpoint, in my case, the get service request.

We can select different stats - mean, max, sum etc.

Finally, we can create OCI Alarms, based on the query - 

This alarm can be set to fire on a condition of your choice. For example, in our call centre use case, the integration team have an internal SLA for retrieving SR data. This is set to 500 msecs. 

We have the choice of creating the alarm on the integration itself and/or the invoke of Fusion.

Let's carry on with our current query - 

check out the query code editor - here you can amend the query -

OutboundRequestInvocationTime[1m]{resourceId = "yourOIC OCID", adapterIdentifier = "erp", outboundInvocationEndpointInformation = "Get ServiceRequests in ERP Cloud"}.grouping().max() > 500

 

I specify a destination, this will result in OIC admins receiving emails, when the alarm fires.

Back in OCI Service Metrics - there is still one dimension I haven't covered.

The values shown are the statuses received in the query time frame. 

So you could also create alarms, based on the response status from Fusion.

Some folks note that OCI Service Metrics charts are very basic, one very good reason to look at OCI Log Analytics.

Creating Log Analytics Dashboard Widgets

This we will now do, based on our query. The best place to copy the query from is the alarm definition -

I edit an existing dashboard and add a query based widget.

Summa Summarum

OCI Service Metrics for OIC are very useful for ad hoc or programmatic monitoring of your integration flows. The value add of OCI Alarms & Log Analytics just makes it easier for you to get the insight you need, when you need it!

And speaking of alarms, mine has just fired; here's the mail I received -

Alarm status is also set to firing - 

Later, I receive an email, informing me the alarm has been deactivated - I wish my house alarm worked the same way! 

Now to adding more data to the email - what do we require here? Naturally, how long the invoke took (in msecs) and when the violation ocurred.

This can be done by adding the following to the alarm body - 

I test again, to trigger the violation. Here's the email - 

The invoke took 3839 msecs.