#1059 - OCI Logging Analytics Dashboards for OIC

 

Starting off with a screenshot that is indecipherable may seem strange, but please humour me. 

The screenshot shows some of the widgets from an OCI Logging Analytics Dashboard I created specifically for one of my OIC instances.

My goal was to give OIC admins the salient data they need in one dashboard. You can decide whether my "salient" applies to you, if not, you can certainly extrapolate from what I did.

I'll go through the widgets one by one, starting off with Billed Messages -

As you can see, I don't have many messages here. This is because I'm using one of my test instances. But, anyway, I see my exact billed message consumption.

Let's look into what's powering this widget -

This widget is based on the OCI Service metric for OIC - BilledMessageCount.

Let's create this in a new dashboard - 

I give it a name - 

then I create a metric based widget - 

I don't see any data, so I choose a longer time window -

Now I see some data -

But let's go for a bar chart - 

Let's give it a title - 

Now we change the statistic on the source data, as we want the message count -

I then check for the last hour and see much fewer messages -

Now back to editing the data source - I can also add more dimensions - 

integrationFlowIdentifier - allows me to get the stats for a specific integration.

adapterIdentifier - allows me to get the stats for integrations triggered by a specific adapter. All my demo app driven integrations is this instance have REST triggers, so I only see rest -

resourceId - the value shown here is the oic ocids. I only see 1, as I have only one OIC instance actively processing requests during this time window. I'll add this dimension, as this dashboard is for a specific OIC instance. 

I then click apply and see my widget in the dashboard -

Billed Messages are on an hourly basis so I change the time window to the last 24 hours - 

The next widget will break down these numbers by integration. This tells me where the money is being spent, from an integration perspective.

We edit the dashboard -

and create a query based widget -

Essentially, what I want is a pie chart showing the billed message breakdown, grouped by integration.

So where will get this query from?

The easiest way is via the OCI Service Metrics, shown on your OIC instances' page in OCI -

I go to the consumed messages section - 

Here's the base query- 

Edit the query to add the group by integration - 

then click Create Alarm -

this will open a dialogue to create an alarm based on the query. However, we will only copy the query - 

Mine is as follows - BilledMessageCount[60m]{resourceId = "myOIC-OCID"}.grouping().sum()

Exit the Create Alarm screen and return to our dashboard - here we click Create query based widget - 

Overwrite the query offered - 

with the following - BilledMessageCount[60m]{resourceId = "myOIC-OCID"}.grouping(integrationFlowIdentifier).sum()

Change the Visualisation to Pie Chart - 

Save the widget, to return to the dashboard -

I now increase the time window to 7 days. during this time I did some large load tests, using an integration, ProcessOrder.

The dashboard tells me that, in the last 7 days, the ProcessOrder integration consumed 99.9% of the billable messages.

I change the time window to 3 days - 

The next 2 widgets I add are related to technical messages and their breakdown by integration. 

There is a subtle difference between technical and billable messages e.g. an app driven integration that has a request payload of 80kb, counts as 2 billable messages, as it is over the 50kb limit.

These 2 new widgets will essentially be a variation on what we have already done for billed messages.

The next widget with be query based and here's the query - 

MessagesReceivedCount[60m]{resourceId = "your-OIC-OCID"}.grouping(flowCode).sum()


Note the group by here is using flowCode and not integrationFlowIdentifier, however, both refer to the integration.

The next pair of widgets highlight adapter errors -

Both of these are query based widgets - 

Adapter Errors - I check here for outbound invokes that have a response status of http 500.

NumberOfOutboundRequests[5m]{resourceId = "ocid1.integrationinstance.oc1.phx.myOIC-OCID", outboundInvocationResponseStatus = "500"}.sum()

Adapter Errors by Integration - this uses the same query, but has the following configuration - 

I can hover over the segments of the adapter errors widget to get more details - in this this is a DB adapter error on a SELECT.

The FTP adapter error is on a LIST files.

One can also create more generic error widgets - 

Failed Messages - is a widget based on the service metric, MessagesFailedCount. 

Failed Messages by Integration - is a query based widget. The query is as follows - 

MessagesFailedCount[5m]{resourceId = "ocid1.integrationinstance.oc1.phx.myOIC-OCID"}.sum()

Configuration is as follows - 

Finally let's consider the situation, where I've literally 100s of integrations running in production. The pie charts I've used my not be all that appropriate. However, I can amend the queries to just show the top 5 or 10 integrations etc.

Just amend the queries as follows - 

BilledMessageCount[1m].grouping(integrationFlowIdentifier).sum().top_n(5)

Note that only 4 integrations are listed above.

Now to the Technical Messages - Top 5 Integrations widget - here the query is - MessagesReceivedCount[1m].grouping(flowCode).sum().top_n(5)

Note, I see 5 integrations here, the extra one is a scheduled job that just reads a very small file, < 50kb, hence not chargeable.

Quo Vadis? Gen2 Usage Metrics?

Some customers want a monthly overview of billed messages. This is common for OIC for SaaS customers who have a monthly commit of a number of packs of 1 million messages.

This data can be easily extracted from the first widget I created - 

Here I set the dashboard time window to Jan 1st - Jan 31st.

I refresh the dashboard, then -

This is one of my test OIC instances, one I don't use much - 

All I need to do is add a SUM function at the bottom -

Billing data via OCI Monitoring API

I covers the OCI Monitoring api in a previous post - now to leveraging that api to get the billing metrics.

The url is as follows - 

https://telemetry.us-phoenix-1.oraclecloud.com/20180401/metrics/actions/summarizeMetricsData?compartmentId={{compartmentId}}

The request payload - 

{

      "namespace": "oci_integration",

    "name": "BilledMessageCount",

    "query": "BilledMessageCount[5d]{resourceId = \"ocid1.integrationinstance.oc1.phx.yourOIC-OCID\"}.grouping(integrationFlowIdentifier).sum()"

}

The response structure is as follows - 

As you can see, invokes of ProcessOrder generated 7 billed messages in the last 5 days.

Summa Summarum

OCI Logging Analytics allows us to create compelling dashboards for OIC. The widgets I have created are all based on OCI Service Metrics for OIC and can be reused in other dashboards. We can go a step further and push our OIC activity stream data to OCI Logging Analytics, allowing us to create more dashboards based on activity stream data. 

Net, net, OCI Logging Analytics is a great addition to your OIC management & monitoring toolkit. 

So get dashboarding, my friends!    

  

 

 

  

  

 

#1058 - OCI Monitoring API for OIC Service Metrics

Introduction

OCI provides a rich set of Service Metrics for OIC. You can check them out on the OCI page for your OIC instance -

This post doesn't discuss the individual metrics. Check out another post of mine for details on each of these.

A second, more recent post, discusses the new consumption metrics. You can check it out here.

What this post does cover is using the OCI Monitoring api to retrieve OIC Service Metrics. This api is documented here.

Essentially, there are 2 endpoints for OCI Monitoring - my region is phoenix so here are my 2 - 

https://telemetry-ingestion.us-phoenix-1.oraclecloud.com
and
https://telemetry.us-phoenix-1.oraclecloud.com

The latter is the one I use for retrieving data, the former for ingesting data.

Oracle make it very easy to check out these apis, all I needed to do was download the oci collection and environment from postman - 

The environment needs to be updated with your values for - 

• tenancy ocid
• compartment ocid
• user ocid
• fingerprint
• private key
• region

Note: regarding private key, you can generate this in OCI and store locally, then open in an editor and, from there, copy paste into the environment variable field. Don't worry about any carriage returns etc.

Note: regarding authentication - this is taken care of by a pre-request script - 

First api I try is -

Request payload - 

{
  "namespace": "oci_integration"
  }

This request returns a list of the oci_service metrics - 

I have run the following integrations - 

Note the entries in the REST API response - 

Some of these may be for integrations from another OIC instance. 

So how can I restrict the REST api call to my specific OIC instance?

Simple, I just need to add a dimension to the request payload.

I can also restrict to only 1 metric - e.g. MessagesSuccessfulCount.

{
    "dimensionFilters": {"resourceId": "ocid1.integrationinstance.oc1.phx.myOIC"},
    "namespace": "oci_integration",
    "name": "MessagesSuccessfulCount"
}

Here is a subset of what I see in the response payload -

{        "name": "MessagesSuccessfulCount",        "namespace": "oci_integration",
        "resourceGroup": null,        "compartmentId": "ocid1.compartment.oc1..myCompartment",
        "dimensions": { "flowCode": "PROCESSORDER",
            "flowVersion": "01.00.0000",
            "resourceId": "ocid1.integrationinstance.oc1.phx.myOIC"
        }
    },

Honestly, this is not really useful information, so let's look at the summarize api -

Before testing the api, I run my processOrder integration, which invokes validateOrder -

Now to invoking the summarize api -

Note the value = 2.

The summarize api requires a query to be added to the request payload. 

You may be a genius when it comes to the query language, I certainly am not, so I just copied the MQL from the OCI Service Metrics dashboard - 

Then I can extrapolate from here.

The MQL I copied was as follows - 

MessagesSuccessfulCount[5m]{resourceId = "ocid1.integrationinstance.oc1.phx.myOIC-OCID"}.grouping().sum()

I added this to the query as follows - note the 2 forward slashes I added to escape the double quotes.

"query": "MessagesSuccessfulCount[5m]{resourceId = \"ocid1.integrationinstance.oc1.phx.myOIC-OCID\"}.grouping().sum()"

[5m] specifies a time window of the last 5 minutes. Say I need the data for a full day (24 hours), then I replace with [1440m]

Say I want to specify one particular integration - 

"query": "MessagesSuccessfulCount[5m]{resourceId = \"ocid1.integrationinstance.oc1.phx.myOIC-OCID\", flowCode =\"VALIDATEORDER\"}.grouping().sum()"

The result - 

Retrieving Consumption Data

I executed my processOrder integration, that invokes the validateOrder integration. I see the following in the Successful Messages metric - 

As you can see above, 2 "technical" messages, one per integration.

I now look at the consumed Messages metric - 

Here I see 1 "billable" message, as the local invoke of validateOrder is not charged.

Let's retrieve this data via the api - 

BilledMessageCount[60m]{resourceId = \"ocid1.integrationinstance.oc1.phx.myOIC-OCID\"}.grouping().sum()

Note: for billing the period is 1 hour i.e. 60m.

Here is the response -

 

Naturally, we can add a dimension for a specific integration - Let's try this out in the UI first -

We can extrapolate from this to construct the query - "BilledMessageCount[60m]{resourceId = \"ocid1.integrationinstance.oc1.phx.myOIC-OCID\", integrationFlowIdentifier = \"PROCESSORDER!01.00.0000\"}.grouping().grouping().sum()"

Summa Summarum

The OCI Monitoring api enables the easy retrieval of OIC Service Metrics. It augments the functionality already provided in the OCI Dashboards, by giving us access to the underlying data.