#1114 - OCI API Gateway and OIC Monitoring

Introduction

We have many Oracle Integration customers who front OIC with OCI API Gateway. Ergo incoming requests to OIC, initially pass through OCI API Gateway, before being routed to OIC.

Wouldn't it be great to be able to monitor those requests as they traverse both OCI Services? This is possible because of a correlation id which is shared between both services. This id is based on the opc-request-id. The latter is a unique, Oracle-assigned identifier for a specific API request, used for tracing, auditing, and troubleshooting.

This post will detail the mechanics of how to actually implement this monitoring.

But firstly, a big thanks to Sreeji from the OCI Log Analytics team for his support here.

Backend OIC Integration

In this scenario, the OIC Integration invoked is called processOrders. It calls another integration, validateOrder, via local invoke.

API Gateway Setup

Here is my API Gateway Deployment setup -

Ensure logging is enabled - 

I execute the request to the gateway -

Check out the logs in OCI Logging

Now, in OCI Logging, I see the api gateway and oic logs. Note the first log entry shown is from api gateway, routing the request to OIC. Note the last entry is from api gateway, processing the OIC response.

 

The api gateway log has the following format - 

Note the opcRequestId entry - 

"/14A3FC5552D04597A8EDF81EECFB635D/7D3278EF2D014AC4B359D70ADCA03A09"

Now to the OIC activity stream entry - 

Note the opcRequestId entry - 

"5TYRCGIW24W2CNSPKVG7JC5FAE84P1XU/14A3FC5552D04597A8EDF81EECFB635D/O6TIS1HTURTM4M3HMR232FLC5W9DYWOA"

As you can see, the first part of the api  gateway opcRequestId surfaces as the second part of the OIC opcRequestId. 

We need to create Correlation IDs that reflect this relationship in OCI Log Analytics.

OCI Log Analytics

Getting API Gateway logs into Log Analytics

I create a new Log Group for api gateway execution log in OCI Log Analytics -

I then create a Connector in OCI Logging to push the api gateway log data to OCI Log Analytics - 

Creating the Correlation ID field for the API Gateway log

The example field entry -

/14A3FC5552D04597A8EDF81EECFB635D/7D3278EF2D014AC4B359D70ADCA03A09

The regex - \/{Correlation ID:.*}\/

Creating the Correlation ID field for the OIC Activity Stream Log

The example field entry -

5TYRCGIW24W2CNSPKVG7JC5FAE84P1XU/14A3FC5552D04597A8EDF81EECFB635D/O6TIS1HTURTM4M3HMR232FLC5W9DYWOA

The regex - .+/{Correlation ID:.+}/

Now to creating the Log Explorer Query.

Log Analytics - Log Explorer Query

Query - 'Correlation ID' != null | link includenulls = true 'Log Source', 'Correlation ID', 'OPC Request ID', Identifier | stats unique(Instance) as 'Instance ID', avg(Duration) as 'Avg. Duration' | eval 'Avg. Duration' = unit('Avg. Duration', second) | rename 'Group Duration' as 'Time Taken' | sort 'Correlation ID', 'Start Time' | fields -'OPC Request ID'

Let's look at these columns in detail - 

The Log Explorer query can be saved and added to a dashboard. I did this - 

Note, I don't see the tabular data. For this I need to punchout to Log Explorer - 

The times shown can be easily verified in OIC Observability, let's execute another request from Postman and check out the dashboard.

I zoom in on the times - 

I check out the activity stream for the first instance id in OIC Observability - 

Message Received at 08:57:34.108

Reply sent at 08:57:34:680

Duration = 572 msecs.

The second instance id - 

Duration = 119 msecs.

We see the duration for OCI API Gateway logs is 705 msecs. There are 2 log messages here, one when the request is initially processed and routed to OIC, and the second, logging the response from OIC.

The duration here is 705msecs, which gives us an CI API Gateway overhead from - 

705 - (572 + 119) = 14 msecs.

Summa Summarum

This simple demo shows the power of OCI Log Analytics for monitoring requests as they traverse different OCI services. I hope this provides you with the wherewithal to create related dashboards in this space.