One of the compelling advantages of OIC, apart from being the best EiPaaS ever, is it's membership of the OCI family. Today I am looking at OCI Logging Analytics and the value add it brings to OIC flow monitoring and log analysis. OIC itself does provide extensive monitoring capabilities and this is complemented by the analytical capabilities of OCI Logging Analytics, which I will often refer to as OCI LA.
Firstly, a big thank you to my colleague, Valeria C. for her ideas and support here!
In respect of OIC, OCI LA has 2 main sources data souces -
- OCI Service Metrics
- OIC Activity Stream logs.
This post will cover how you can create rich dashboards for your OIC instances in OCI LA.
Before we begin - a quick review of OIC Logging levels - this is important as this determines what gets sent to OCI LA.
OIC Logging Levels
Production - All activities outside loops and invoke/logger activity inside loops (up to a 1000 iterations) are shown in the activity stream and this data is retained for a period of 32 days.
Audit - Additionally wire payloads of trigger/invoke(s) are shown in the activity stream and this data is retained for a period of 8 days.
Debug - Additionally all payloads and all activities inside loops (up to a 1000 iterations) are shown in the activity stream and this data is retained for a period of 24 hours. It may slowdown your system and also pose a security risk as sensitive information from payload will be logged.
Here's a simple example of Audit output in OIC - this simple integration creates a customer in Netsuite.
Note, I see the payload passed to Netsuite -
Now I activate the integration with Production level logging, I then run the same test - As you can see, only the activities executed are logged - no payloads etc.
Let's switch to OCI Logging and see what data we're getting there. Note, you can enable your OIC instance to auto-push the logs to OCI Logging.
OIC Logs in OCI logging
Here's the log for 1 of the createCustomer Netsuite flows, with trace set to production -
Let's check out the message structure - the salient content is in red.
"message": "Message started processing",
Let's look at the trace set to Audit and run -
Note, the payload is now available in the Message field -
This is important, as this is also the log format in OCI Logging Analytics (OCI LA). OCI LA allows us to parse the logs so we could extract payload data from this field and use it in dashboards, e.g. top products ordered etc.
You create a service connector in OCI Logging to push the OIC logs to OCI LA.