#1156 - OIC - Monitoring Costs/Billing

Introduction 

Controlling costs is important, knowing where your universal credits are being spent is important. 

Realising that overage is now charged is important. The OCI Billing Service Metrics for OIC make it easy for us to get a handle on all of this.

Just to remind you, we have the following Billing related service metrics for OIC - 

• Configured Messages - based on Message Packs e.g. 1 Message Pack = 5k Messages

• Total Consumed Message Packs - actual Message Pack consumption, from a billing perspective i.e. Configured + any Overage

• Total Consumed Messages - consumed by all OIC components

• Consumed Messages for Integration - consumed by OIC Integration

• Consumed Message Packs for Disaster Recovery

• Consumed messages for Process Automation and Human in the Loop

• Consumed messages for Decisions

• Consumed messages for Robotic Process Automation

• Consumed messages from data retention
  

You may not be using all of the OIC components, so you can ignore some of the above.

Imagine if you are using OIC just for integrations, then the following billing metrics are relevant to you.

 

• Configured Messages - based on Message Packs e.g. 1 Message Pack = 5k Messages

• Total Consumed Message Packs - actual Message Pack consumption, from a billing perspective i.e. Configured + any Overage

• Total Consumed Messages - consumed by all OIC components

• Consumed Messages for Integration - consumed by OIC Integration

You may also be interested in the top nn integrations, from a billing perspective.

Here is the link to the Oracle documentation for OIC3 Billing, for those who want to recap on the charging rules.

Now let's begin by looking at the Service Metrics in more detail, as these are the basis for all the queries and dashboards I will create in this post.

OCI Billing Service Metrics for OIC

The 4 metrics above can be found, along with the others, in the service instance page in OCI Console. The service metrics for OIC can be divided into 2 broad types - Billing related and technical. 

All metrics have filters - the time interval and the statistic. The filters applied by default differ between the metrics. For the Billing filters, the default time interval is one hour; this makes sense as billing is done on an hourly basis.

I just ran an integration 4 times in my Dev instance, so let's check the billing metric for integrations - 

Here we see the internal metric name and the full query - 

BilledMessageCount[60m]{resourceId = "yourOIC-OCID"}.grouping().grouping().sum()

 I change the time interval to 1m -

Now I see the billed message count - 

Back to Edit Query mode - 

Dimensions allow me to slice and dice the data. Each service metric will have a set of Dimensions one can use. These may differ between metrics.

For BilledMessageCount we have the following - 

Plus, of course, the instance ocid, referred to as resource id.

Net, net, we can breakdown the billed messages by integration.

I run a different integration 3 times, so that we have a choice here.

Back to the Dimensions - 

Now I only see the 4, I'm sure you get the idea.

You can create OCI Alarms, based on these metrics - 

Here you see the updated query and can add a condition, such as - 

BilledMessageCount[1m]{resourceId = "yourOIC-OCID", integrationFlowIdentifier = "CREATE_ORDER_NETSUITE!01.00.0000"}.grouping().sum() > 100

The Alarm can trigger various channels, via OCI Notifications / Subscriptions - 

Net, net - your OIC admin will receive an email, if the sum of billed messages for the integration, CREATE_ORDER_NETSUITE!01.00.0000, exceeds 100.

Monitoring Billing via a Dashboard 

We have a couple of options here - begin by entering Dashboard in the OCI Search box - 

OCI Dashboards

Click New Dashboard -

Build from scratch - 

Note the options here - 

I select Monitoring - 

Let's Configure - 

Let's not add any Dimensions yet - 

Note how we get the breakdown by integration OOTB - 

I now add 2 more widgets, based on the BilledMessagesCount metric. One for my OIC Dev instance, the other for Prod. I add the resourceId dimension to both, and set accordingly.

Net, net - here we have billed messages over all my OIC instances, as well as the sum for each instance.

Here's the data in tabular format - 

The green line shows the stats from my Prod instance.

 The Configured Messages metric gives us the number of message packs assigned to a service instance. I add 2 widgets based on such, one for Dev and one for Prod -

I now run a load test from SOAPUI, which will exceed the 5k messages limit - 

Let's add a widget, based on the Total Billed Message Count - 

I also add a widget, based on Billed Message Pack Count - 

Wait a wee while for the widgets to get updated - 

Over 2k messages overage, so we will be charged 2 message packs for this hour.

We can also see the "offending" integration - 

Remember, we could also have created an alarm on any of these so that the OIC admin is informed immediately.

Back to the dashboard we just created, the URL can be easily shared - 

OCI Management Dashboards

Here we are notching it up a level - 

Oracle Cloud Infrastructure (OCI) Management Dashboards are toolsets that display real-time and historical technical data through visual widgets. They focus on application health, resource performance, and deep log analytics.

These Dashboards can be seen as part of OCI Log Analytics, but the latter can do much more e.g. enable deep analysis of OIC Activity Stream logs. But we'll stick to the former here and create a similar billing dashboard.

You have 2 options for creating the dashboard, with or without filters. Log Analytics delivers a couple of OIC related dashboards OOTB. These dashboards can be viewed here, simply query on oracle integration.

The ones with Created by set to Oracle are the OOTB dashboards. The others were created by me. I did so by simply duplicating one of the OOTB dashboards.

Open one of the Oracle dashboards -

Note the filters available - 

These are - 

Log Group Compartment, OIC Environment (allows filtering on a specific OIC Instance), Integration Identifier (allows filtering on a specific integration), Integration Instance Id (allows filtering on a specific flow). With these filters we can start at fleet level, then drill down into a specific OIC service instance, from there to a specific integration etc.

I duplicate the Health Overview dashboard - 

I then open the new dashboard and delete the widgets - 

I can now add new Billing widgets- 

As you can see, this is a similar interface to the Dashboards interface - 

You will see the query defaulting to - 

We can overwrite this with - BilledMessageCount[60m]{resourceId = $(params.selectedOCID)}.grouping(resourceId).sum() - 

Note the use of - resourceId = $(params.selectedOCID). This is allowing me to leverage the filter. Here's where I link selectedOCID to the actual filter OIC instance filter on the page - 

Net, net - selectedOCID is just a variable I created. It is now linked to the OCID shown in the filter.

Let's clean this up; firstly, look at the entries with -, this means no billable messages for that hour. We can get rid of these, easily - add "preventFillGaps": true to the json source.

Note how you can add, delete columns - 

I delete the name and dimensions.resourceid.

Now lets address the issue of the column names; I'll make them more user friendly. We need to edit the JSON source again - 

I add a new widget, to display the configured messages metric - 

The underlying query is - ConfiguredMessages[60m]{resourceId = $(params.selectedOCID)}.grouping().sum()

Now for a widget that displays the total consumed message packs -

BilledMessagePackCount[60m]{resourceId = $(params.selectedOCID)}.grouping().sum() 

Here we see overage billed for 2 hours in the afternoon of the 24th of July.

Here's a widget that displays the total of consumed messages - 

TotalBilledMessageCount[60m]{resourceId = $(params.selectedOCID)}.grouping().sum()

I arrange the widgets as follows, so that it makes more sense - 

We can see 5k messages configured (1 message pack), 7.192K messages for the 2 hours ending 04:30pm on July 25th, 1 extra message pack charged for each of those hours.

Next question to which we need an answer - which are the top integrations from a billing perspective? 

The query for this is - BilledMessageCount[1h]{resourceId = $(params.selectedOCID)}.groupBy(integrationFlowIdentifier).sum().top_n(10)

Last week I ran a couple of load tests for SYNC_UPDATE_ORDER; that's why it looks skewed.

The dashboard now contains 5 widgets -

Regarding the 5th widget, it's debatable whether we need it, but let's hang on to it for the present.

Here is where we can add more widgets, dependent on what other OIC components we are using.

I will add the following for OPA/HITL - 

The underlying query - OPABilledProcessMessageCount[60m]{resourceId =$(params.selectedOCID)}.grouping().sum()

You could, of course, add widgets for RPA, Decisions and Disaster Recovery, if you are availing of those features.

 

I add these, for completeness sake.

That's it!

This dashboard is available via OCI Management Dashboards and OCI Log Analytics -

As already mentioned, Log Analytics is a key tool for deeper analysis of your OIC activity stream logs; a tool I can very highly recommend. You can check out my other Log Analytics posts here.  

Summa Summarum

I hope this post will be useful, please let me know if you think anything is missing. Net, net, the combination of dashboards and alarms can ensure you effectively monitor your OIC billing.