#1157 - Offloading OIC Logs to Object Storage

Introduction 

We have a plethora of OIC monitoring possibilities both within OIC itself and also via various OCI services - Logging, Log Analytics, Alarms etc. But what if you have standardised on a different log analysis tool, such as Dynatrace etc?

We need to get the activity stream logs there, and an easy method is as follows - 

OIC -> OCI Logging -> Service Connector -> Object Storage

Then an OIC scheduled integration that leverages the Object Storage native action to read the exported logs and push them to Dynatrace or whatever other tool via a REST api.

Step 1 - surface OIC Activity Stream in OCI Logging 

This is easy, simple enable such here -

Note - you can have multiple OIC instances using the same log group.

Then do a sanity test - run an integration.  Before that though, remember, the quantity/detail of log messages generated depends on the trace level you select -

This run fails, due to the following - 

Now to OCI Logging, to check we have received the log messages - 

I start at Log Group level - 

I click on my activity stream log -

There are lots of logs being written, as there is also a load test running on this OIC instance. Net, net - we have the logs surfacing in OCI Log Analytics.

Step 2 - create a target bucket in OCI Object Storage

Here is my Bucket - 

It already has content, including some OIC log exports - 

I have 2 flavours here - 1. push logs to a sub folder exp/ or, 2. push logs to the top level.

Note the file sizes, ca. 1MB.

Here's the same at the top level - 

You get the idea!

Note, bucket and OIC instance are in the same compartment.

So, I've created my bucket, now to the next step.

Step 3 - create the required Policy

Allow any-user to manage objects in compartment id yourCompartmentOCID
where all {
  request.principal.type='serviceconnector',
  target.bucket.name='niallc-demoBucket',
  request.principal.compartment.id='yourCompartmentId'}

Now to the next step, creating the service connector - 

Step 4 - create the Service Connector

My Object Storage connector has been configured as follows - 

Let's go through the creation steps in detail - 

Note: I also enable the connector logs, creating a new log group for these.

Back to the connection creation wizard - 

Note the ability to add more logs, also the ability to add filters; more about filters later.

Target configuration - 

Then select the bucket - 

Note the ability to add a prefix; this is how the exp/ folder got created. Here I just entered - exp. 

Now to the batch options - this is important, considering the REST api you use, to push the logs to a 3rd party, may have size limitations.

Let's set these manually - 

Again, set these to suit your actual requirements.

Note the Create policy button - this will automatically create the required policy.

and that's it!

Step 5 - check out the exports arriving in Object Storage

Download one of the zips - 

What first hits you is the data{} entries from the log - 

However, you get exactly what you see in OCI Logging, e.g. 

{"data":{"actionType":"Mapper","eventId":"uZ-AyIvcEfG8IAFF_hcuYw","executedTime":"2026-07-30T06:05:54.850Z","instanceId":"MH3ptYvcEfG9ee_6FVinEA","integrationFlowIdentifier":"ASYNC_CREATE_ORDER!01.00.0000","message":"Data Mapping completed with Message <nstrgmpr:POST xmlns:ns20=\"http://xmlns.oracle.com/cloud/adapter/REST/createOrder/types\" xmlns:ns24=\"http://www.oracle.rest.converters.autogeneratedns.com/collocatedics/localCreate_REQUEST\" xmlns:xml=\"http://www.w3.org/XML/1998/namespace\" xmlns:nstrgmpr=\"http://xmlns.oracle.com/cloud/adapter/collocatedics/localCreate_REQUEST/types\"><nstrgmpr:components.schemas.request-wrapper><ns24:orderNr>123</ns24:orderNr></nstrgmpr:components.schemas.request-wrapper></nstrgmpr:POST>","opcRequestId":"WXEMOWRXL94JEMBZDTRLMFO8GTFHON8Q/PZ3GZUPC46685QIHK8LBKCXZLJJTEU2Z/A9RQU74YK6JJHZN91N2LBZSXAFDER5YT","parentEventId":"uZ9Zt4vcEfG8IAFF_hcuYw","projectCode":"AA_2604_NEW_FEATURES","userId":"nnn"},"id":"nnn","oracle":{"compartmentid":"yourCompartmentId","ingestedtime":"2026-07-30T06:05:59.412Z","loggroupid":"yourLogGroupId","logid":"yourLogId","tenantid":"yourTenancyId"},"source":"yourOICId","specversion":"1.0","time":"2026-07-30T06:05:54.850Z","type":"com.oraclecloud.integration.integrationinstance.activitystream"}

Step 6 - Applying Filters

The filter dropdown ostensibly lists all the fields on which you can filter - 

I'm only interested in errors, hence my filter above.

I check out the query syntax, by switching to advanced mode - 

The query searches compartment / log group / log file for a specific entry -  (data.instanceCompletionState='ProcessCompletedWithFaults')

You can try this query out in OCI Logging - 

This is what I see in the export to Object Storage - 

{"data":{"actionName":"syncUpdateOrder","actionType":"Receive","endPointConnectionId":"REST_TRIGGER","endPointName":"syncUpdateOrder","endPointType":"rest","errorId":"mu1VJowLEfGSfZ-fcacITg","eventId":"mu1VJYwLEfGSfZ-fcacITg","executedTime":"2026-07-30T11:41:29.697Z","executionTimeInMillis":"67","instanceCompletionState":"ProcessCompletedWithFaults","instanceId":"muMb8owLEfGSfZ-fcacITg","integrationFlowIdentifier":"SYNC_UPDATE_ORDER!01.00.0000","message":"Error in reply to syncUpdateOrder","opcRequestId":"CRX9S17GJ9CAR8E6XG72SBX5OMXWIXAA/BPUGJJMZI677ZKAAZJMVV31MRZUW2TEG/2F4VKYOCIZ24B4TZEOGWQ6GIQDFM88JC","parentEventId":"muySgowLEfG8IAFF_hcuYw","projectCode":"AA_ERRORS","userId":"niall.commiskey@oracle.com"},...}

times 4. 

Summa Summarum

This is a simple way to export activity stream logs, there are other options, which I can cover later, if anyone is interested. Just ping me.