#1158 - OIC 26.07 - New Service Limit for Async Queue Depth

Introduction 

In Oracle Integration Cloud (OIC), asynchronous requests decouple the client from the backend process using a fire-and-forget message-queuing mechanism. This pattern maximizes throughput, protects downstream systems from spikes in traffic, and makes efficient use of integration resources.

Great to know! However, we did introduce a service limit; on this with the 26.07 release this is based on the number of message packs assigned to the OIC instance.

We also recently introduced a new service metric - Async Queue Depth -

This will be your tool for monitoring.

Now back to the new service limit - the actual formula is built up as follows -

Take the number of messages per hour and multiply that by 10. That, essentially is the limit on the potential size of your async queue.

A simple example - I have 1 message pack assigned to my OIC instance, so this equates to 5000 * 10. That gives us a 50k limit on the queue size. This is like saying I can cope with a spike of 10 times the usual number of requests, or in other words, I have a 10 hour backlog.

We do add a buffer on top of this; in the case of a 1 message pack OIC instance, this insures the 50001th message will not be rejected. However, you should focus on the limit for the number of message packs you have assigned, and not rely on the buffer.

As you can see, the upper limit is 600k, however, this can be increased, if necessary. For such, you would need to contact Oracle, open an SR etc.

Hitting the Limit

I will run a test on an OIC 26.07 instance, but, before doing that, let's check the Async Queue Depth service metric for that instance -

As you can see, nothing in the Queue.

I will run the following async integration that, in turn, invokes a sync integration. The sync integration includes a WAIT, set to 120 seconds.

That's how I build up the queue.

I test from SOAPUI as follows - 

Test run 1 creates 20k flows.

Here we see the concurrency limit for async being enforced.

I check the service metrics and see the queue building up - the service metric query I use is as follows - AsyncInboundRequestsDepth[1m]{resourceId = "yourOCID"}.grouping().max()

20000 - 615 = 19385.

I run another 20k test - 

40000 - 1269 = 38731

Now for a 3rd SOAPUI test - 

and then a 4th run - 

Here the queue levels out, i.e. it does not increase. Note the buffer I've been given is ca. 10%, but, as already mentioned, please do not rely on this.

I now run the same async integration from within OIC - 

Error message - 

429 Too Many Requests

{"type":"429","title":"Too busy right now. Please retry later.","detail":"The service is temporarily busy. Please retry later."}

Alarm on 80% of Queue Limit 

Let's create an OCI Alarm to ensure we get informed when the queue reaches 80% of its limit. 

In your OIC instance Service Metrics page, select the Async queue metric - 

As you can see, I am using a pre-existing topic.

I run another load test - 

The email body - 

80% is probably a very high limit, from a business perspective. I discussed this with a customer, who said she would set the limit at 10%, i.e. 1 hour backlog. Net, net, this is up to you to decide on what works best for your business.  

Summa Summarum

Please be aware of this new limit, especially if you are prone to huge spikes in traffic. Again, this limit can be raised Please contact your friends at Oracle Support if you need > 600k. 

Cerevisia aestiva

As it's summer, the next set of posts will contain an extra section on the beers I'm drinking this summer. 

The Latin word for beer is cerevisia, which surfaces in the Spanish, cerveza. Aestiva is the Latin for summer.

The first beer I'll review is from Dorn Bräu, a brewery based in the Franconian town of Bruckberg, about 20 km away from me. It's a family owned brewery, founded in 1685, so these guys know what they're doing.  

This beer is what the Germans call "süffig". There is no direct English translation. The word I got was quaffable, but sure who uses that word today? 

Net, net - this is a "helles" - that means golden in colour, smooth malty taste and low in bitterness. At 5%, it's the usual strength of German beer. A great everyday beer!

#1157 - Offloading OIC Logs to Object Storage

Introduction 

We have a plethora of OIC monitoring possibilities both within OIC itself and also via various OCI services - Logging, Log Analytics, Alarms etc. But what if you have standardised on a different log analysis tool, such as Dynatrace etc?

We need to get the activity stream logs there, and an easy method is as follows - 

OIC -> OCI Logging -> Service Connector -> Object Storage

Then an OIC scheduled integration that leverages the Object Storage native action to read the exported logs and push them to Dynatrace or whatever other tool via a REST api.

Step 1 - surface OIC Activity Stream in OCI Logging 

This is easy, simple enable such here -

Note - you can have multiple OIC instances using the same log group.

Then do a sanity test - run an integration.  Before that though, remember, the quantity/detail of log messages generated depends on the trace level you select -

This run fails, due to the following - 

Now to OCI Logging, to check we have received the log messages - 

I start at Log Group level - 

I click on my activity stream log -

There are lots of logs being written, as there is also a load test running on this OIC instance. Net, net - we have the logs surfacing in OCI Log Analytics.

Step 2 - create a target bucket in OCI Object Storage

Here is my Bucket - 

It already has content, including some OIC log exports - 

I have 2 flavours here - 1. push logs to a sub folder exp/ or, 2. push logs to the top level.

Note the file sizes, ca. 1MB.

Here's the same at the top level - 

You get the idea!

Note, bucket and OIC instance are in the same compartment.

So, I've created my bucket, now to the next step.

Step 3 - create the required Policy

Allow any-user to manage objects in compartment id yourCompartmentOCID
where all {
  request.principal.type='serviceconnector',
  target.bucket.name='niallc-demoBucket',
  request.principal.compartment.id='yourCompartmentId'}

Now to the next step, creating the service connector - 

Step 4 - create the Service Connector

My Object Storage connector has been configured as follows - 

Let's go through the creation steps in detail - 

Note: I also enable the connector logs, creating a new log group for these.

Back to the connection creation wizard - 

Note the ability to add more logs, also the ability to add filters; more about filters later.

Target configuration - 

Then select the bucket - 

Note the ability to add a prefix; this is how the exp/ folder got created. Here I just entered - exp. 

Now to the batch options - this is important, considering the REST api you use, to push the logs to a 3rd party, may have size limitations.

Let's set these manually - 

Again, set these to suit your actual requirements.

Note the Create policy button - this will automatically create the required policy.

and that's it!

Step 5 - check out the exports arriving in Object Storage

Download one of the zips - 

What first hits you is the data{} entries from the log - 

However, you get exactly what you see in OCI Logging, e.g. 

{"data":{"actionType":"Mapper","eventId":"uZ-AyIvcEfG8IAFF_hcuYw","executedTime":"2026-07-30T06:05:54.850Z","instanceId":"MH3ptYvcEfG9ee_6FVinEA","integrationFlowIdentifier":"ASYNC_CREATE_ORDER!01.00.0000","message":"Data Mapping completed with Message <nstrgmpr:POST xmlns:ns20=\"http://xmlns.oracle.com/cloud/adapter/REST/createOrder/types\" xmlns:ns24=\"http://www.oracle.rest.converters.autogeneratedns.com/collocatedics/localCreate_REQUEST\" xmlns:xml=\"http://www.w3.org/XML/1998/namespace\" xmlns:nstrgmpr=\"http://xmlns.oracle.com/cloud/adapter/collocatedics/localCreate_REQUEST/types\"><nstrgmpr:components.schemas.request-wrapper><ns24:orderNr>123</ns24:orderNr></nstrgmpr:components.schemas.request-wrapper></nstrgmpr:POST>","opcRequestId":"WXEMOWRXL94JEMBZDTRLMFO8GTFHON8Q/PZ3GZUPC46685QIHK8LBKCXZLJJTEU2Z/A9RQU74YK6JJHZN91N2LBZSXAFDER5YT","parentEventId":"uZ9Zt4vcEfG8IAFF_hcuYw","projectCode":"AA_2604_NEW_FEATURES","userId":"nnn"},"id":"nnn","oracle":{"compartmentid":"yourCompartmentId","ingestedtime":"2026-07-30T06:05:59.412Z","loggroupid":"yourLogGroupId","logid":"yourLogId","tenantid":"yourTenancyId"},"source":"yourOICId","specversion":"1.0","time":"2026-07-30T06:05:54.850Z","type":"com.oraclecloud.integration.integrationinstance.activitystream"}

Step 6 - Applying Filters

The filter dropdown ostensibly lists all the fields on which you can filter - 

I'm only interested in errors, hence my filter above.

I check out the query syntax, by switching to advanced mode - 

The query searches compartment / log group / log file for a specific entry -  (data.instanceCompletionState='ProcessCompletedWithFaults')

You can try this query out in OCI Logging - 

This is what I see in the export to Object Storage - 

{"data":{"actionName":"syncUpdateOrder","actionType":"Receive","endPointConnectionId":"REST_TRIGGER","endPointName":"syncUpdateOrder","endPointType":"rest","errorId":"mu1VJowLEfGSfZ-fcacITg","eventId":"mu1VJYwLEfGSfZ-fcacITg","executedTime":"2026-07-30T11:41:29.697Z","executionTimeInMillis":"67","instanceCompletionState":"ProcessCompletedWithFaults","instanceId":"muMb8owLEfGSfZ-fcacITg","integrationFlowIdentifier":"SYNC_UPDATE_ORDER!01.00.0000","message":"Error in reply to syncUpdateOrder","opcRequestId":"CRX9S17GJ9CAR8E6XG72SBX5OMXWIXAA/BPUGJJMZI677ZKAAZJMVV31MRZUW2TEG/2F4VKYOCIZ24B4TZEOGWQ6GIQDFM88JC","parentEventId":"muySgowLEfG8IAFF_hcuYw","projectCode":"AA_ERRORS","userId":"niall.commiskey@oracle.com"},...}

times 4. 

Summa Summarum

This is a simple way to export activity stream logs, there are other options, which I can cover later, if anyone is interested. Just ping me.