#1069 OIC invoking OCI GenAI Agents - the basics

Introduction

Welcome to this simple example of invoking an OCI Gen AI Agent from OIC. This post will introduce GenAI Agents and how to leverage them from your integrations.

According to the docs - OCI Generative AI Agents is a fully managed service that combines the power of large language models (LLMs) with AI technologies to create intelligent virtual agents that can provide personalized, context-aware, and highly engaging customer experiences.

 So let's get started with a simple example. I will create an agent that will assist in processing expenses.

Create the OCI GenAI Agent

The menu from Gen AI Agents above gives us an idea of how this will be done. Firstly, I need a knowledge base, e.g. corporate rules around expenses. We will use the following rules from Commiskey Inc.

This is my knowledge base and I'll make it available to the Agent via OCI Object Storage - 

I created the Knowledge Base - 

with its data source - 

The Ingestion Job pulling in the file from Object Storage - 

I then created the Agent - 

The Agent also has an endpoint, which will receive requests - 

I copy the endpoint ocid, as I will need this later.

Let's try out the Chat function - 

Looks good, so now to leveraging the Agent from OIC. But first, let's take a look at the apis we will be invoking from OIC.

OCI GenAI Agent REST APIs

The api docs are here.

As you can see there are 2 groups of api, consider these management and runtime.

Management APIs 

I will invoke some of these from an integrations, using the API Endpoint base url for Chicago, as that's the region hosting my agent.

Runtime APIs 

I will be invoking the createSession and chat api in my integration, but more of that later.

Creating the Connections in OIC 

As you will probably realise, I will need to create 2 connections in OIC, one for the management api and another for the runtime/client api.

Naturally, I will also need to create the relevant Policies to allow my OIC instance to invoke these apis. My esteemed colleague, Harris Q. has documented an easy approach for doing this, so check out his post here. Net, net, here are my 2 connections - 

Note the security policy used, in both connection definitions.

OIC Invoking the Management APIs

So how did I proceed here? I started off with list knowledge bases - the invoke is configured as follows - 

I didn't know the response structure to begin with, so I initially set this to {}. I then ran the integration and copied and pasted the response json from the activity stream. 

I did the same for the other invokes.

I also externalised the ocids for endpoint and compartment -

So let's run this - 

The relevant JSON response is available under Wire Message received...

OIC Invoking the Runtime/Client APIs

The above 2 are the key invokes - create session and then, using the session id, invoke the chat api. 

createSession configuration 

The chat invoke is configured as follows -

Let's run it...

The response - 

{

  "response" : "I'd be happy to help you with your expenses query. 

  However, I have to advise you that the meal you're trying to expense doesn't quite meet our company's expenses rules.

  Firstly, the meal value of €150 exceeds the allowed limit of €100, as per reference 1. 

  Additionally, the beer you've had with your meal is not eligible for expensing, as per reference 2, which clearly states that no alcohol can be expensed.

  Lastly, I notice that the receipt date is May 9th, 2025, which falls on a Friday. According to reference 7, on Fridays, only fish meals can be expensed, 

  and since your meal consisted of steak, it doesn't meet this criteria.\n\nI'm afraid I won't be able to approve this expense. 

  If you have any other receipts that meet our company's expenses rules, I'd be more than happy to help you with those."

}

I can easily add multi-language support to the endpoint - here I use the management api - updateAgentEndpoint -

I re-run the integration with the following request payload - 

{

  "idleTimeoutInSeconds": 3600,

  "expenseNr": "1",

  "details": "Das Datum des Spesenbelegs ist der 9. Mai 2025. Kann ich ein Menü mit Steak und Bier im Wert von 130 € als Spesen abrechnen??"

}

 

The response - 

{

  "response" : "Hallo!\n\nIch verstehe, dass Sie ein Menü mit Steak und Bier im Wert von 130 € als Spesen abrechnen möchten. Leider muss ich Ihnen mitteilen, dass dies nicht möglich ist.\n\nGemäß unseren Spesenregeln (Referenz 1) dürfen Mahlzeiten nicht über 100 € kosten, und Ihr Menü liegt über diesem Betrag. Darüber hinaus ist es nicht erlaubt, Alkohol (Referenz 2) zu expensieren, und Bier fällt unter diese Kategorie.\n\nAußerdem fällt der 9. Mai 2025 auf einen Freitag, und gemäß unserer Regel (Referenz 7) dürfen an Freitagen nur Mahlzeiten mit Fisch expensiert werden, nicht mit Fleisch wie Steak.\n\nIch hoffe, Sie verstehen unsere Regeln und können Ihre Ausgaben entsprechend anpassen. Wenn Sie weitere Fragen haben, stehe ich Ihnen gerne zur Verfügung.\n\nMit freundlichen Grüßen,\nIhr Spesenberater"

}

I can also try in Gaeilge -

{

  "idleTimeoutInSeconds": 3600,

  "expenseNr": "1",

  "details": "Is é dáta Admhála Costais ná 9 Bealtaine 2025. An féidir liom béile steak agus beoir ar €130 a chaitheamh?"

}

{

  "response" : "Dia duit,\n\nTáim buíoch duit as do cheist. Mar chomhairleoir costais, táim ansin chun cabhrú leat a chinntiú go bhfuil do chostais i gcomhréir leis na rialacha atá leagtha amach.\n\nI ndáil le do cheist, táim afraid nach féidir leat béile steak agus beoir ar €130 a chaitheamh. Tá dhá fhadhb ann:\n\n1. Tá an béile thar an teorainn luach, €100, a luaitear i réamhshráid 1.\n2. Tá beoir san áireamh, agus de réir réamhshráid 2, ní ceadmhach é a chaitheamh.\n3. Chomh maith leis sin, tá an béile steak ar siúl ar Dé hAoine, agus de réir réamhshráid 7, ní ceadmhach béile le feoil a chaitheamh ar Dé hAoine - iasc amháin atá ceadmhach.\n\nTáim sásta cabhrú leat a chinntiú go bhfuil do chostais i gcomhréir leis na rialacha. Má tá aon cheist eile agat, ná bíodh drogall orm a chur in iúl.\n\nGo raibh maith agat,\n[Your Name]\nComhairleoir Costais"

} 

Summa Summarum

I accept the use case is banal, but I'm sure you can all extrapolate from it, and apply what I have done to your own use case(s). The post shows you can leverage OCI Gen AI Agents easily, from your OIC integrations. It really only a couple of minutes work. So try it out!

 

 

#1066 - Invoking OCI GenAI from OIC

Introduction 

The OCI GenAI service offers access to pre-trained models, as well as allowing you to host your own custom models.

The following OOTB models are available to you - 

OIC GenAI service is currently available in 5 regions - 

The service offers I will be using the Chicago endpoint in this post. 

The objective of the post is to detail how easy it is to invoke the chat interface from an integration. I will detail this both for Cohere and llama.

Pre-Requisites 

First step - generate private and public keys - 

openssl genrsa -out oci_api_key.pem 2048

and

openssl rsa -pubout -in oci_api_key.pem -out oci_api_key_public.pem

I then login to the OCI Console and create a new api key for my user - 

User Settings - API Keys - 

I then choose and upload the public key I just created - 

I copy the Fingerprint, for later use.

I also created the following Policy - 

allow group myAIGroup to manage generative-ai-family in tenancy

Invoking GenAI chat interface from OIC

Now to OIC where I create a new project - adding 2 connections - 

OCI GenAI connection is configured as follows - note the base url - 

https://inference.generativeai.us-chicago-1.oci.oraclecloud.com

As already mentioned, OCI GenAI is currently available in 5 regions. Check out the documentation for the endpoint for other regions.

This integration will leverage the Cohere LLM - The invoke is configured as follows -

The Request payload is as follows - 

{
  "compartmentId" : "ocid1.compartment.yourOCID",
  "servingMode" : {
    "modelId" : "cohere.command-r-08-2024",
    "servingType" : "ON_DEMAND"
  },
  "chatRequest" : {
    "message" : "Summarize: Hi there my name is Niall and I am from Dublin",
    "maxTokens" : 600,
    "apiFormat" : "COHERE",
    "frequencyPenalty" : 1.0,
    "presencePenalty" : 0,
    "temperature" : 0.2,
    "topP" : 0,
    "topK" : 1
  }
}

  

The Response payload - 

{
  "modelId" : "cohere.command-r-08-2024",
  "modelVersion" : "1.7",
  "chatResponse" : {
    "apiFormat" : "COHERE",
    "text" : "Niall, a resident of Dublin, introduces themselves.",
    "chatHistory" : [ {
      "role" : "USER",
      "message" : "Summarize: Hi there my name is Niall and I am from Dublin"
    }, {
      "role" : "CHATBOT",
      "message" : "Niall, a resident of Dublin, introduces themselves."
    } ],
    "finishReason" : "COMPLETE"
  }
}

I test the integration -

Now to an integration that leverages llama - 

The invoke is configured with the following request / response -

Request - 

{
  "compartmentId" : "ocid1.compartment.yourOCID",
  "servingMode" : {
    "modelId" : "meta.llama-3-70b-instruct",
    "servingType" : "ON_DEMAND"
  },
  "chatRequest" : {
    "messages" : [ {
      "role" : "USER",
      "content" : [ {
        "type" : "TEXT",
        "text" : "who are you"
      } ]
    } ],
    "apiFormat" : "GENERIC",
    "maxTokens" : 600,
    "isStream" : false,
    "numGenerations" : 1,
    "frequencyPenalty" : 0,
    "presencePenalty" : 0,
    "temperature" : 1,
    "topP" : 1.0,
    "topK" : 1
  }
}

 Response - 

{
  "modelId" : "meta.llama-3.3-70b-instruct",
  "modelVersion" : "1.0.0",
  "chatResponse" : {
    "apiFormat" : "GENERIC",
    "timeCreated" : "2025-04-09T09:39:40.145Z",
    "choices" : [ {
      "index" : 0,
      "message" : {
        "role" : "ASSISTANT",
        "content" : [ {
          "type" : "TEXT",
          "text" : "Niall from Dublin introduced himself."
        } ],
        "toolCalls" : [ "1" ]
      },
      "finishReason" : "stop",
      "logprobs" : {
        "1" : "1"
      }
    } ],
    "usage" : {
      "completionTokens" : 9,
      "promptTokens" : 52,
      "totalTokens" : 61
    }
  }
}

I test this integration - 

Maybe it's just me, but the llama response is better.

Now let's make this a bit more interesting - I will pass some context to the LLM -

Here is my input - 

{

  "text": "Here is some additional context Expenses Rules no alcohol can be expensed. bill limit per meal is €100. you cannot expense extreme left wing literature.  Please answer my question: can I expense the book the Anarchist's cookbook?"}

Here is the test result - 

Ok, so let's try with a steak and a beer for €130.

Request -

{

  "text": "Here is some additional context Expenses Rules no alcohol can be expensed. bill limit per meal is €100. you cannot expense extreme left wing literature.  Please answer my question: can I expense a meal of a steak and a beer for €130?"}

Response -

{

  "summary" : "No, you cannot expense a meal of steak and a beer for €130 as it violates two of the Expenses Rules: the bill limit per meal is €100, and no alcohol can be expensed."

}

Finally, let's externalise the expense rules to a file and read that file at runtime - 

The integration flow is as follows - read the expense docs and pass them to the LLM along with the expense request.

The variable v_expensesRules is set as follows - 

oraext:decodeBase64(oraext:encodeReferenceToBase64($GetFileRef/ns18:GetFileReferenceResponse/ns18:File.definitions.getFileReferenceResponse/ns18:fileReference))

Note, I also externalised the chat api field values to integration properties - 

I run the integration with the following request - 

The response - 

{

  "status" : "Based on the provided context, it seems that the company has a strict policy regarding expense claims and the types of purchases that can be reimbursed. \n\nGiven that the company explicitly states that \"no extreme left-wing literature can be expensed,\" it is highly unlikely that the book \"The Anarchist's Cookbook\" would be an allowable expense. This book is often associated with anarchist and radical left-wing ideologies, and as such, it would fall under the category of extreme left-wing literature that is explicitly prohibited from being expensed. \n\nTherefore, it is safe to assume that purchasing \"The Anarchist's Cookbook\" would not be a valid expense claim according to the company's rules."

}

I make this a bit more complex - I add an extra rule to the expense rules -

7. You cannot expense a meal with meat on a Friday - fish only!

I re-run the integration with the following request - 

The Response - 

{

  "status" : "Based on the provided expense rules, you would not be able to expense a meal of steak and chips for €80 on April 4th, 2025. \n\nRule number 1 states that no meals over a value of €100 can be expensed. While this meal is under that value, rule number 7 prohibits the expensing of any meal with meat on a Friday. As such, this meal would not be an allowable expense."

}

 

Summa Summarum

Leveraging OCI GenAI chat interface from OIC is simple. My examples are of course, very simplistic, but I'm sure you can extrapolate from them.

Also, for those interested in the story behind the Anarchist Cookbook, check out this entry on Wikipedia. Written by a disaffected teenager at the time of the Vietnam war, it contained instructions for making lots of illegal stuff. Interestingly, the author found religion and tried to get the book banned. He eventually ended up as a conflict resolution (peaceful) expert, renouncing violence of any sort. For those history buffs amongst you, akin to Mikhail Bakunin morphing into Peter Kropotkin. Kropotkin's most famous work was the book "Mutual Aid", which pitted co-operation, motivated by universal love against cold social Darwinism.