#1026 - OIC REST API Log file download - How To for OIC3

 

This OIC Gen2 api lets you download -

• ICS Diagnostic Log
• ICS flow log
• Audit Log

The request format is as follows - 

https://oicpm-oicpm-px.integration.ocp.oraclecloud.com/ic/api/integration/v1/monitoring/logs/log

The valid log values are -

• icsdiagnosticlog
• icsflowlog
• icsauditlog

Download the Diagnostic Log

I now download the diagnostic log and see the following, when I unzip it -

 

Note the structure -

The following files are weblogic specific -

• access.log
• oic_server1.log
• oic_server1.out
• oic_server1-nodemanager.log

As you can see, this is essentially of little use to an OIC admin.

Now to the ics-flow.log - 

It does contain OIC relevant data e.g. 

 

ActionID:y0][ActionName:InvoicesToATP][ActionType:Return]: Response received from InvoicesToATP

[2024-09-09T16:45:09.088+00:00] [oic_server1] [NOTIFICATION] [] [oracle.ics.trace.soa.bpel] [tid: 102] [userId: niall.commiskey@oracle.com] [ecid: d505d8d9-9d4e-48cb-9f6f-ee1c4a2edd04-002e7192,0] [APP: soa-infra] [partition-name: DOMAIN] [tenant-name: GLOBAL] [FlowId: 0000P7NfC8v5uXdLxe4EyW1anhva0002yu] [oci.instanceName: OICPM] [oci.identityDomain: identityDomainName]  [ICS Activity Stream Logging]: [Code:INVOICESTOATP][Version:01.00.0000][Instance:62405859][Operation:execute][ActionID:y0][ActionName:InvoicesToATP][ActionType:Return]: Integration execution has completed

However, it's not that readable.

Same applies to the oic_servern_diagnostic.log -

I'll skip the nodemanager log as that is pure weblogic.

Net, net, these logs are useful for checking out engine errors, but are not useful from a compliance perspective, e.g. these files don't help you prove that orderNr 123 was processed on a certain date.

Download the ICS Flow Log

The Structure of the downloaded zip - 

I executed the following simple integration, before running the REST request -

Now I activate the integration with trace enabled - 

I download the log file again, and now I see the payload in the log -

Net, net - the ICS flow log is useful, when you enable tracing / include payload.

Download the Audit Log

This log details who did what in the designtime -

e.g. here my activation of the integration has been audited - 

[2024-09-25 13:29:17.577] [userId: niall.commiskey@oracle.com] [niall.commiskey@oracle.com,ACTIVATE,ICS_ProjectV2,AA_SIMPLE_SYNC_WITHCHIL,AA-Simple-Sync-WithChild,01.00.0000] User niall.commiskey@oracle.com activated Integration AA-Simple-Sync-WithChild 01.00.0000

[2024-09-25 13:29:17.361] [userId: niall.commiskey@oracle.com] 

So now we have looked at what's available from OIC gen2.

Let's turn to OIC3.

OIC3 Retrieve Audit log

Note the structured format of the response, easier to read and process, compared to OIC Gen2.

The following OIC3 api can be used to retrieve instance flow details, similar, but not the same as the OIC gen2 ICS Flow log. - 

The response is as follows - 

As you can see, even in debug mode, the request payload is not returned.

What we do see, however, is the tracking fields and their values.

So how can we retrieve the integration flow request payloads?

First, let's take a step back - 

Customers usually use the OIC gen2 log api on a daily basis, for example, run the job to download the ICS flow log on a daily basis and push the response to a monitoring / analysis tool such as Splunk.

So, from an OIC3 factory api perspective, I could do the following - 

1. Retrieve integration flow instances for the time period e.g. for the last day.

2. For each of those instances, retrieve the request payload.

Let's look at 1 - 

note the id returned - 

that from the screenshot is as follows - 

na1TEHvfEe-BH8NDEl59ew

I will now use this to retrieve the payload via the activity stream, this is task 2 -

 

 

Final step is to automate this with OIC. Here I create a scheduled integration that will invoke both apis. In my simple demo, I will write the instances + payloads to a file.

#1020 - Gen2 to OIC3 Upgrade - Factory REST API changes

Some customers are using the OIC Gen2 Factory api to download the log files -

left side - OIC Gen2

right side - OIC3

Download a log file - 

/ic/api/integration/v1/monitoring/logs/icsflowlog

This is what I get, when I save to file - 

Let's run a simple integration - 

I can find the error in the downloaded log - 

But, naturally, I see this also in the activity stream - 

I can download the logs, which provide much more salient data -

So the question is, what is the business impact of this api not being available in OIC3? 

To recap, the OIC Gen2 api can be used to download log files, such as icsflowlog or icsauditlog. The latter is the designtime audit - which developer did what in the designtime and when.

In OIC3, we can download the audit log from the Design time audit page - 

Summa, Summarum - If you are using this api in OIC gen2, then please consider why you are using it. Which data do you need to retain? Remember, the production trace level in OIC3 ensures the log data is available for 32 days. If you need to hive off data for compliance purposes, then please look and OCI Logging Analytics and Object Storage. 

Net, net - the fact that this api is not available in OIC3 may have little impact on your ability to upgrade.

#1019 - OIC Gen2 to OIC3 Upgrade - Instance Id

In OIC runtime, integration flows are identified by a unique id, the instance id. This is defined in the OIC Gen2 and OIC3 api as a string -

In OIC Gen2 - this string contains a numeric value -

In OIC3 this instance id string is alphanumeric - 

Storing Instance Id in a DB

Some OIC Gen2 customers have stored the instance id in a DB table, sometimes for compliance purposes, also to enable lookups of instance state by other integrations e.g. Integration B checks on the state of Integration A via the OIC Factory apis.

Check out my simple DB table below -many customers have defined the column holding the instance id as NUMBER, when they really should have been using VARCHAR.   

This could be fed by a common integration, such as the following - 

I use the Oracle DB adapter - 

So, in our scenario, the OIC instance has been upgraded to OIC3, where the instanceId value is now alphanumeric.

This means I will need to change the DB column type - 

This is a major change, moving from NUMBER to VARCHAR2, so, as you can see, I need to delete the rows. 

So, in this case, let's do a backup of the table - 

Now I delete the rows from the original table - 

I drop the table and re-create it, with instanceid column set to varchar2(30).

However, when the table is empty, you could also use the ALTER Table command to change the column datatype, e.g. ALTER Table oic_instance_flows modify (instanceid varchar2(30)); 

Remember the instance id value in OIC3 is 22 characters.

Next step is to restore the data from the backup table - 

SQL> insert into oic_instance_flows (instanceid, integrationname, rundatetime) select instanceid, integrationname,rundatetime from oic_instance_flows_backup;

I now execute the integration the writes to the DB table - 

This error requires me to edit the integration. First thing I do is change the REST trigger request from -

{

  "instanceId" : 123456,

  "integrationName" : "ABC"

}

to 

{

  "instanceId" : "ABC123456",

  "integrationName" : "ABC"

}

Now to the DB invoke - the quickest method is to delete this and recreate it.

Here we import the new definition of the table i.e. with instanceid as VARCHAR2.

 

Here's an example based on a stored procedure - 

Now I am upgrade to OIC3, so I need to do the following -

1. change the DB column to VARCHAR2

2. Edit the PLSQL Stored Procedure - 

3. change the REST trigger request, setting instanceid as string.

{

  "instanceId" : "ABC123456",

  "integrationName" : "ABC"

}

4. Re-create the DB invoke of the Stored Procedure - 

 

Summa summarum, OIC developers leveraging such will probably have just 1 integration that inserts the instance id into the database. This common integration will be invoked by the other integrations. So the actual amount of work involved in refactoring this should be minimal.

Using OIC getFlowId Function

The instance id could also be retrieved via the following and used in the mapper - 

The function could also be used in an Assign action -

you will need to validate the usage of these in your integrations to see what you are doing with these.

Invoking OIC Factory REST APIs with Instance Id   

Many of the OIC Factory apis, especially those concerned with monitoring, use the instance id - 

as you can see, the instance id, from a factory api perspective, is a string. So there should be little change required here.