Tuesday, November 17, 2026

#1191 - File Server Concurrent Connections Metric

Introduction

Many OIC customers use the OIC File Server for file processing scenarios, MFT etc. This sftp server, however, has a concurrency limit - 

So how do I know where I stand in relation to this limit of 50? For that there is an OCI Service Metric - File Server Inbound Connections. 

The default MQL you see in the OIC Monitoring page in OCI is - FileserverInboundConnections[1m]{resourceId = "yourOIC OCID"}.grouping().sum()

More about this later.

I'm going to run a couple of tests to see how this metric surfaces, but, before doing so, please note that metric sampling currently happens every 5 minutes. So this metric may not deliver an exact view of concurrency, but can certainly help you isolate trends etc.

File Server Concurrent Connections Test

I run the test in Windows PowerShell; it will open up 10 connections and keep them open for 12 minutes.

try {
    # Start 10 independent SFTP processes.
    1..10 | ForEach-Object {
        $info = New-Object System.Diagnostics.ProcessStartInfo
        $info.FileName = (Get-Command sftp.exe).Source
        $info.UseShellExecute = $false
        $info.RedirectStandardInput = $true
        $info.Arguments = (
            '-b - -i "{0}" -oBatchMode=yes ' +
            '-oControlMaster=no -oControlPath=none ' +
            '-oConnectTimeout=20 -P {1} "{2}@{3}"'
        ) -f $KeyPath, $OicPort, $OicUser, $OicHost

        $process = [System.Diagnostics.Process]::Start($info)
        $connections += $process
    }

    # Keep every connection active for approximately 12 minutes.
    1..12 | ForEach-Object {
        $minute = $_

        foreach ($process in $connections) {
            if ($process.HasExited) {
                throw "SFTP process $($process.Id) exited unexpectedly."
            }

            $process.StandardInput.WriteLine("ls .")
            $process.StandardInput.Flush()
        }

        Write-Host "Minute ${minute}: sent requests to all 10 sessions."
        Start-Sleep -Seconds 60
    }
}
finally {
    # Close only the processes created by this test.
    foreach ($process in $connections) {
        if (-not $process.HasExited) {
            $process.StandardInput.Close()

            if (-not $process.WaitForExit(3000)) {
                $process.Kill()
            }
        }

        $process.Dispose()
    }

    Write-Host "Test connections closed."
}

I will execute the following in another PowerShell window to check on concurrency - 

$port = [int](Read-Host "Enter your OIC File Server port")

$connections = @(
    Get-NetTCPConnection -State Established |
        Where-Object { $_.RemotePort -eq $port }
)

$connections |
    Format-Table LocalPort, RemoteAddress, RemotePort, OwningProcess

"Established connections to port ${port}: $($connections.Count)"

I begin by outputting current datetime in both windows - 

I run the concurrency check again at ca. 11:36 -

The test has reached minute 6 - 

Check out the Service Metric

Note the time is UTC i.e. CET -1.

Now set the metric filter to 5 minutes / sum

Here we hit the 10.

Test is now at minute 10 -

Test completes at 11:45 CET / 10:45 UTC - 


Here's a 6 hour view of the metrics -
Note, I ran a similar test, a couple of hours before. Also note how we do not see a line down to 0, after the 2nd test ended. 

Again, be aware the stats are emitted  every 5 minutes, giving us a view of connection concurrency at that particular point in time. 

Let's try the following - I will have the same 10 connections test running for 12 minutes in one window. In other I'll have a shorter test for 20 connections for 3 minutes.

30 concurrent connections - 

Still 30 - 
The short test completes - 

so only 10 connections is use - 

Back in OCI for the service metric - 
Here we see a peak at 14, before falling back to 10. Ergo, the peak of 30 concurrent connections is not highlighted.

Summa Summarum

The metric in its current iteration is useful for highlighting trends, however, due to the 5 minute sampling, it is not able to provide an exact view of concurrent connections. Remember to switch the time window to 5 minutes, and you're good to go! 


















 

Wednesday, November 11, 2026

#1190 - Surfacing HITL Logs in Log Analytics

Introduction 


HITL is being used by lots of our customers, in conjunction with AI Agents.

Recently we released a dedicated OCI Log Analytics dashboard for monitoring your HITL driven workflows. We will delve into the dashboard later; let's start with the pre-requisites - how to get HITL logs to OCI Log Analytics. For that we need a log group, as target for the HITL logs. We also need a parser, which will interpret the log data. Some policies are required for this to function, but, once all is set up, we just need to activate the sending of logs. This we do in the OIC console page for your OIC instance. But let's begin with Log Analytics.

Parser in OCI Log Analytics

This is provided OOTB -

The parser requires a "source" log -

This is an example HITL log file, on which the parser is based.

Back in the parser - 

Here is the example log content - 

{
  "processDefinitionId": "UniqueProcessDefinitionID",
  "processDefinitionName": "UTProcess",
  "processDefinitionTitle": "UTProcess",
  "taskTitle": "User task 1",
  "processDefinitionKey": "OIC_PROJECT_1--01.00.0000--UTProcess",
  "startTime": "2026-05-25T10:49:00.188+00:00",
  "endTime": 1779708077902,
  "processInstanceState": "CREATED",
  "processInstanceId": "UniqueProcessInstanceID",
  "processDefinitionVersion": "01.00.0000",
  "processInstanceDuration": 1938110,
  "activityType": "userTask",
  "activityInstanceState": "COMPLETED",
  "activityInstanceSubState": "INITIATED",
  "activityInstanceDuration": 1936318,
  "activityInstanceId": "UniqueActivityInstanceID",
  "assigneeId": "beb0d855cf9040a3a0f16dUniqueID",
  "assigneeName": "beb0d855cf9040a3a0f16dUniqueID",
  "assigneeType": "USER",
  "activityId": "flow_UniqueActivityID",
  "activityName": "User task 1",
  "taskOutcome": "SUBMIT",
  "faultCode": "ABC123",
  "taskSubtype": "Submit",
  "taskId": "UniqueTaskID",
  "eventTimestamp": 1779708077902,
  "executionId": "UniqueExecutionID",
  "deploymentId": "UniqueDeploymentID",
  "oicProjectId": "OIC_PROJECT_1",
  "updatedTime": null,
  "inputPayload": null,
  "businessKey": "Leave Approval",
  "instanceIdentifier": "ut1",
  "userTaskOutputPayload": {
    "taskExec": {
      "approver": "beb0d855cf9040a3a0fUniqueID",
      "approverDisplayName": "user display name",
      "summary": "",
      "approvedDate": "2026-05-25T11:21:17.577Z",
      "approverComment": "null",
      "assignedDate": "2026-05-25T10:49:01.261Z"
    },
    "taskOutcome": "SUBMIT"
  },
  "outputPayload": null,
  "candidateUsersAndGroups": [
    {
      "name": "sample.user@example.com",
      "type": "USER"
    },
    {
      "name": "sample_group",
      "type": "GROUP"
    }
  ],
  "time": 1779708078285,
  "logGroupOCID": "ocid1.loganalyticsloggroup.oc1.phx.exampleuniqueid",
  "serviceInstanceOCID": "ocid1.processautomationinstance.oc1.phx.exampleuniqueid"
}

As you can see, it includes task name, approver, task outcome etc, as well as the OCID of the OPA service instance.

During the log ingestion process OIC will provide the target log group and the log source.


Create a new Log Group in Log Analytics

Then copy the ocid for future use.


Enable HITL for Log Analytics

This is simple - go to the OCI console page for your OIC instance.


Click Enable and paste the ocid of the log group you just created.

Apply Policy Permissions

Currently HITL runs as part of OPA (Oracle Process Automation). We need to allow this HITL (OPA) instance access to Log Analytics.

I have already enabled such for my OIC instances. I created a dynamic group and added the resource ids of my OIC instances.
This dynamic group will now be augmented with the HITL id. This id can be found in the OCI console for your OIC instance. The entry in the dynamic group should be in the following format -

ALL {resource.id = 'ocid1.processautomationinstance.oc1.phx.nnn'}

The dynamic group is used in a Policy - 

It takes a couple of minutes for the policy to kick in.

After a nice cup of English Breakfast tea - I run an OIC based agent that processes orders. The logic is simple; orders > $1000 need to be manually approved.

I run this a couple of times -

Then move to Log Explorer - 

Check out the HITL Logs - 

Here's the raw log data - 

HITL Dashboard

There is now a HITL dashboard in Log Analytics, let's check it out -

There my 2 tasks, back in the Task list - 

I've approved one of the tasks, let's refresh the dashboard - 

Looks good!

I have one active and one completed instance -

You can check out the queries behind each of the widgets - 

Here is the query - 'Log Source' = 'OCI Integration - HITL Activity Stream Logs' and 'Event Type' = PROCESS | rename 'Process State' as State | stats latest(State) as State by 'Process Name', Instance | stats count(Instance) as 'Instance Count' by State, 'Process Name' | where State not in (created)

Net, net - you can create your own widgets based on your own queries; you can also edit existing queries and save as a different widget.

One thing to note here - the Entity value from the filter is not used. Let's get rid of this, as it may cause confusion. To do this, I duplicate the HITL dashboard and edit the duplicate - 

The Entity is still referenced in the widgets, but we can simply remove it.

Don't enter a value, just click Save changes.


Data is now rendered in the widget - 

Do this for all the widgets - 

I open the original HITL dashboard, to check out the Tasks tab - 

There are 5 widgets - 

  • Tasks by status
  • Tasks by status and workflow
  • Pending tasks by users/groups
  • Completed tasks by approver
  • Tasks by outcome

One thing to note, these tasks are at compartment level; so, if you have 3 OIC instances in your compartment, you could potentially see the aggregation of tasks. as we've already seen there is a serviceInstanceOCID field in the HITL log which can be used to filter on HITL/OPA instance. More about that in a follow-up post.

Summa Summarum

This is a very useful dashboard for those using HITL. It can be exposed to LOB folks easily and avoids having to give them access to OIC/OPA for HITL monitoring purposes.