#295 BAM and BPM part 1

In this post I cover the use of BAM OOTB for BPM process analysis.

I have the following simple process that processes an Order -

Here are the DB tables accessed in the process -

Here is the process -

Here is the composite -

I deploy and test and verify that all is working.

Now I start the BAM server.
For this post, I will be leveraging the Sample Monitor Express components.
These need to be installed. Follow the README.txt at -

Now we need to do the BAM adapter setup in the WLS console -

Open the BPM project preferences -

Change as follows -

Login to BAM - http://localhost:9001/OracleBAM/

Start a couple of instances of the process via EM -

Click on the BAM - Active Viewer button

Click - Select Report

Now let's add a counter to the process - to count the number of approval rejections.

re-deploy and create some new instances.
Reject one of them

In BAM, click on Process View and select the process

We see the current value of our counter

#294 Oracle SOA Governance 11g Implementation

The Book

At last over the holiday period I have gotten the opportunity to delve into this worthy tome. This book is excellent in it's coverage of design- and runtime governance. The example business case used for the book's case study is easy to follow. The authors begin by conducting a SOA Maturity Assessment for the fictional company and detail an appropriate reference architecture for them.

Chapter 3 covers Oracle Enterprise Repository - Oracle's flagship product for design time governance. This chapter gives an indepth account of OER architecture. This chapter also covers value-adds such as the solution packs available for OER.

Chapter 4 covers configuring OER for our case study - Service Discovery and Cataloging etc.

Chapter 5 discusses the harvester which allows one to bootstrap OER with existing assets, for example - there is a excellent section covering the OSB harvester. Once harvested - OER then allows us to visualize asset relationships - for me one of the most important features of the product.

Chapter 6 deals with Asset Lifecycle and Workflow - this part contains a valuable section on implementing architecture blueprints - thus defining expected deliverables and enforcing compliance.

Chapter 7 covers Oracle Service Registry (OSR) - Oracle's UDDI implementation. This section also covers the relationship between OSR and OER. OER - desgin time - OSR - runtime. It also covers the OER exchange utility which enables OER / OSR synchronisation. The chapter discusses the usual UDDI concepts in the context of OSR - UDDI Taxanomies, tModel etc. A good refresher!

Chapter 8 covers Design Time Service Promotion and Discovery - essentially applying what we learned in the previous chapter to our use case. This chapter discusses such technical details as automatic workflows for publishing to OSR etc. Again a great feature of this book - the right mix of WHY and HOW.

Chapter 9 brings us to runtime governance - including how to use Oracle Web Services Manager (WSM) to implement security policies as well as monitoring via Enterprise Manager.

Chapter 10 discusses extending runtime governance via EM and BTM (Business Transaction Management)- for infrastructure monitoring. BTM is not a core part of SOA Suite, however it fits very nicely into the mix here as the authors mention.

Chapter 11 covers extending governance with AIA foundation pack - especially interesting in the realm of MDM.

All in all, the book itself is a great asset - with the right mix of answers to to the What, Why and How of SOA Governance. Chapeau Luis and Andrew!

Again the Book

#293 Adaptive Case Management Workshop in sunny Malta Feb 2014

For those interested in attending the workshop - Forum 2014

#292 BPM 11g Human Task - Reassignment via API

Simple scenario:

Task is assigned to CSR role - jcooper is a member.

Create a test instance.
Validate in workspace.

Now I will reassign this task to wshake.

The actual Java API code is as follows -

    public String reassignTask(String taskName, String currentOwner,
                               String newOwner) {
        System.out.println("*** reassignTask() " + taskName + " from " +
                           currentOwner + " to " + newOwner);
        Map properties = new HashMap();
        properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_PROVIDER_URL,
                       "t3://localhost:7001");
        properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_SECURITY_CREDENTIALS,
                       "welcome1");
        properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_SECURITY_PRINCIPAL,
                       "weblogic");

        try {
            //Create JAVA WorflowServiceClient
            IWorkflowServiceClient wfSvcClient =
                WorkflowServiceClientFactory.getWorkflowServiceClient(WorkflowServiceClientFactory.REMOTE_CLIENT,
                                                                      properties,
                                                                      null);

            //Get the task query service
            ITaskQueryService querySvc = wfSvcClient.getTaskQueryService();
            //Login as jcooper
            IWorkflowContext ctx =
                querySvc.authenticate(currentOwner, "welcome1".toCharArray(),
                                      null);
            //Set up list of columns to query
            List queryColumns = new ArrayList();
            queryColumns.add("TASKID");
            queryColumns.add("TASKNUMBER");
            queryColumns.add("TITLE");
            queryColumns.add("OUTCOME");

            Predicate pred =
                new Predicate(TableConstants.WFTASK_TITLE_COLUMN, Predicate.OP_EQ,
                              taskName);

            //Query a list of tasks assigned to jcooper
            List tasks =
                querySvc.queryTasks(ctx, queryColumns, null, //Do not query additional info
                    ITaskQueryService.AssignmentFilter.ALL, null, //No keywords
                    pred, //Custom predicate
                    null, //No special ordering
                    0, //Do not page the query result
                    0);
            //Get the task service
            ITaskService taskSvc = wfSvcClient.getTaskService();

            // create the newAssignee list - in our case only newOwner
            List assigneeList = new ArrayList();

            ITaskAssignee assignee = new TaskAssignee(newOwner, "user");
            assigneeList.add(assignee);
            //


            for (int i = 0; i < tasks.size(); i++) {
                Task task = (Task)tasks.get(i);
                int taskNumber = task.getSystemAttributes().getTaskNumber();
                String title = task.getTitle();
                String taskId = task.getSystemAttributes().getTaskId();
                String outcome = task.getSystemAttributes().getOutcome();
                List taskAssigneeList =
                    task.getSystemAttributes().getAssigneeUsers();
                System.out.println("Found task " + title + " taskId is " +
                                   taskId);

                // do the Reassignment
                for (int j = 0; j < taskAssigneeList.size(); j++) {
                    IdentityType type = (IdentityType)taskAssigneeList.get(j);
                    String name = type.getId();
                    System.out.println("Currently assigned to " + name);
                }
                int taskNr = task.getSystemAttributes().getTaskNumber();

                if (title != null && title.equalsIgnoreCase(taskName)) {
                    System.out.println("Starting reassignment...");
                    Task task4Reassign =
                        querySvc.getTaskDetailsById(ctx, taskId);
                    System.out.println("Task title " +
                                       task4Reassign.getTitle());
                    System.out.println("Reassigning task to " + newOwner +
                                       " for taskNr " + taskNr);
                    taskSvc.reassignTask(ctx, task4Reassign, assigneeList);
                }


            }
        } catch (Exception e) {
            //Handle any exceptions raised here...
            System.out.println("Caught workflow exception: " + e.getMessage());
        }

        return "done";
    }

Run -

login to workspace as wshake

Java project available here

#291 BPEL configuration properties

Rule of thumb for BPEL is - keep those processes short and sweet.
Transient, synchronous is the way to go, if possible.

ORCL docs provide for very good reading in this area - I recommend the FMW Developer Guide for SOA Suite, the FMW SOA Suite Administrators Guide as well as the FMW Performance Guide. The info below, has to a great extent, been taken from there. I also recommend AnTony Reynolds and Matt Wright's - Oracle SOA Suite 11g Developers Guide - available here

Tuning BPEL Engine Threads -

These are the guys who do the actual work -

The dspInvokeThreads property specifies the total number of threads allocated to process invocation dispatcher messages. Invocation dispatcher messages are generated for each payload received and are meant to instantiate a new instance. If the majority of requests processed by the engine are instance invocations (as opposed to instance callbacks), greater performance may be achieved by increasing the number of invocation threads. Higher thread counts may cause greater CPU utilization due to higher context switching costs.


The dspEngineThreads property specifies the total number of threads allocated to process engine dispatcher messages. Engine dispatcher messages are generated whenever an activity must be processed asynchronously. If the majority of processes deployed are durable with a large number of dehydration points (mid-process receive, onMessage, onAlarm, and wait activities), greater performance may be achieved by increasing the number of engine threads. Note that higher thread counts can cause greater CPU utilization due to higher context switching costs.

The dspSystemThreads property specifies the total number of threads allocated to process system dispatcher messages. System dispatcher messages are general clean-up tasks that are typically processed quickly by the server (for example, releasing stateful message beans back to the pool). Typically, only a small number of threads are required to handle the number of system dispatch messages generated during run time.

Start off with the defaults, and tune - if necessary - paying attention to the heuristics above.

These properties can be set in EM. 

As you can see, the Audit Level is set to Inherit as default. Please see my My Post on Logging for more info.

Here is the doc -

For production systems - use Production or Error - depending on your logging requirements.

Audit Detail threshold - under the threshold? - details store in the audit_trail table.
over the threshold? then audit info will be written to the audit_details table.

Large Doc Threshold - same concept -

This is the maximum size (in kilobytes) of a BPEL variable before it is stored in a separate location from the rest of the instance scope data.

Payload Validation - validates incoming and outgoing XML documents 
Incurs a performance hit.

Other Configuration properties

Now let's look at the rest -

As you can see, the descriptions shown in EM are useful and succinct.
You will not change most of these, however - some are worth investigating -


For one way invokes with possible callbacks -

Per default requests to execute a BPEL process are written to the soa_infra.dlv_message table - this causes a message to be generated that is then picked up by a worker thread which then starts executing the process.

Big benefit - Reliability no requests get lost - however we do have an extra DB write.

If set to async.cache then the request is cached in-memory - better Performance

If set to sync -> queuing is bypassed, and the BPEL process is invoked synchronously.
Client has full control of what happens.


1. bpel.config.oneWayDeliveryPolicy
Default value specified in MBean - accessible from em

async.persist: Messages are persisted in the database hash map.
sync.cache: Messages are stored in memory.
sync: Direct invocation occurs on the same thread.

Can be overwritten in the BPEL process definition in composite.xml -

onewayDeliveryPolicy=sync
and
transaction=required
BPEL process runs in the same thread and the same transaction.

onewayDeliveryPolicy=sync
and
transaction=requiresNew
BPEL process runs in the same thread but within a new transaction.

completionPersistPolicy

Here you need to ask yourself, whether the instance data needs to be persisted, and if so to what extent.
If your BPEL process integrates System A with System B. then the fact that the data has been successfully written to B suffices.

Simple example -

here is my one-way BPEL process that writes its input to a file -

Test - here is the output order.

Here is the em trace -

Now I set the the completePersistPolicy to off in composite.xml -
I also need to set inMemoryOptimization = true

Re-test - my output file has been written.

Check the audit trail in em -

Probably faulted would be the best setting here.

The SyncMaxWaitTime property sets the maximum time the process result receiver
waits for a result before returning. Results from asynchronous BPEL processes are
retrieved synchronously by a receiver that waits for a result from Oracle BPEL Server.
The default value is 45 seconds.

If you have sync BPEL processes and you see the need to up this property, then consider making the processes async.

MaxRecoverAttempt - number of times to attempt recovery of invoke/callback messages.

The recovery behavior for invoke and callback messages is different when
MaxRecoverAttempt is set. For example, assume MaxRecoverAttempt is set to 4.
■ Invoke message recovery is retried 4 (N) times before moving the message to
the exhausted state.
■ Callback message recovery is retried 5 times (N + 1) before moving the
message to the exhausted state.
This is the expected behavior. The first attempt is not counted as a recovery
attempt. The recovery attempts are incremented by the BPEL process service
engine. If MaxRecoverAttempt is set to 1, you see one default resolution process
and then one recovery attempt.

Recovery Configuration

This feature is detailed in the Administrator’s Guide for Oracle SOA Suite and Oracle
Business Process Management Suite


e.g. We are trying to update a DB and it is temporarily unavailable.

This covers -
All activities (for example, wait activities and OnAlarm branches of pick activities)
that have an associated expiration date and are scheduled with the SOA
Infrastructure to be rescheduled
■ All activities that are not complete over a provided threshold time

■ All invoke and callback messages that are unresolved

When should auto-recovery be available?
Check out the RecurringScheduleConfig - set start/stopWindowTime

Check out StartupScheduleConfig -
the default config tells us -

maxMessageRaiseSize The maximum number of messages to submit for each startup
recovery attempt. Use this property to limit the impact of recovery on the server. This value specifies the maximum number of messages to filter from activity, invoke, and callback
queries; that is, 50 messages from each of the activity, invoke, and callback tables.

startupRecoveryDuration -
Specifies the number of seconds that the startup recovery period
lasts. After the server starts, it goes into a startup recovery
period. During this period, pending activities and undelivered
callback and invocation messages are resubmitted for
processing.
The default value is 600 (ten minutes). A negative or zero value
disables startup recovery.

Partner Link Properties

The main ones ares -

nonBlockingInvoke=true
PartnerLink Service will executed in a separate transaction.

idempotent=false
Executes on same Thread/transaction

#290 Accessing Process Payload via JDBC

Here is my process -

I instantiate an instance of this using em.

I see the following in the dlv_message table -

I select the GUID, in my case 57D6E....

I now look at the DOCUMENT_DLV_MSG_REF table this will contain a reference to the table storing the incoming payload for the BPM process - key is the above GUID.

This gives me the document_id 57DA6....

I now look in the XML_DOCUMENT table

So now I want to retrieve the value of the payload from a Java client -

Firstly, kudos to Arik for his reply on the forum -

https://forums.oracle.com/thread/2515495

Here's my code -

    public String getProcessPayload(String docId) throws SQLException,
                                                         BinXMLException,
                                                         IOException {
        String rtc = "";
        String selectCmd =
            "select document_id, document_type, document from dev_soainfra.xml_document where document_id = '" +
            docId + "'";
        Connection conn = getConnection(jdbcUrl, dbUser, dbPassword);
        Statement statement;
        statement = conn.createStatement();
        ResultSet rs = null;
        try {
            //       System.out.println("SQL = " + selectCmd);
            rs = statement.executeQuery(selectCmd);
            XMLDOMImplementation domimpl = new XMLDOMImplementation();
            while (rs.next()) {
                BinXMLProcessor proc =  BinXMLProcessorFactory.createProcessor();
                  BinXMLStream inpbin = proc.createBinXMLStream(rs.getBlob("DOCUMENT"));
                  BinXMLDecoder dec = inpbin.getDecoder();
                  InfosetReader xmlreader = dec.getReader();
             
                  XMLDocument doc = (XMLDocument)domimpl.createDocument(xmlreader);
                  doc.print(System.out);
                //
            }
        } catch (SQLException e) {
            e.printStackTrace();
        }
        return rtc;
    }

    private static Connection getConnection(String jdbcUrl, String dbUser,
                                            String dbPassword) {


        Connection connection = null;

        try {
            Class.forName("oracle.jdbc.driver.OracleDriver");
            connection =
                    DriverManager.getConnection(jdbcUrl, dbUser, dbPassword);
        } catch (ClassNotFoundException e) {
            throw new RuntimeException("No Driver");
        } catch (SQLException sqle) {
            throw new RuntimeException("SQL Error", sqle);
        }
        return connection;
    }

Here's the client -

public class Tester {

    public Tester() {

        super();

    }

    public static void main(String[] args) throws WorkflowException,

                                                  StaleObjectException,

                                                  Exception {

        Tester tester = new Tester();

        ITaskQueryAPIDemo itq = new ITaskQueryAPIDemo();

   

        itq.getProcessPayload("57DA64F05B3D11E3BF88096B9553B378");

      

      

                                           

         }

}

Here's the output -

Full JDev Project Here

#289 Human Task API - getting payload values

My process is as follows -

Human task title = ApproveTheOrder for Commiskey

Test Instance -

Task is assisgned to user jcooper.

API to retrieve the above data -

    public String getTaskdetails(String user, String pwd) {
        System.out.println("getTaskdetails()");
        Map properties = new HashMap();
        properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_PROVIDER_URL,
                       "t3://localhost:7001");
        properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_SECURITY_CREDENTIALS,
                       pwd);
        properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_SECURITY_PRINCIPAL,
                       user);

        try {
            //Create JAVA WorflowServiceClient
            IWorkflowServiceClient wfSvcClient =
                WorkflowServiceClientFactory.getWorkflowServiceClient(WorkflowServiceClientFactory.REMOTE_CLIENT,
                                                                      properties,
                                                                      null);

            //Get the task query service
            ITaskQueryService querySvc = wfSvcClient.getTaskQueryService();
            //Login as jcooper
            IWorkflowContext ctx =
                querySvc.authenticate(user, pwd.toCharArray(), null);
            //Set up list of columns to query
            List queryColumns = new ArrayList();
            queryColumns.add("TASKID");
            queryColumns.add("TASKNUMBER");
            queryColumns.add("TITLE");
            queryColumns.add("OUTCOME");

            //
            // also get the payload
            List optionalInfo = new ArrayList();
            optionalInfo.add(ITaskQueryService.OptionalInfo.PAYLOAD);
            //
            //Query a list of tasks assigned to jcooper
            List tasks =
                querySvc.queryTasks(ctx, queryColumns, optionalInfo,
                    ITaskQueryService.AssignmentFilter.MY_AND_GROUP, null,
                    //No keywords
                    null, //No custom predicate
                    null, //No special ordering
                    0, //Do not page the query result
                    0);
            //Get the task service
            ITaskService taskSvc = wfSvcClient.getTaskService();
            //Loop over the tasks, outputting task information, and approving any
            //tasks whose outcome has not been set...
            System.out.println("getTaskdetails() nr of Tasks = " +
                               tasks.size());


            for (int i = 0; i < tasks.size(); i++) {
                System.out.println("In task loop...");
                Task task = (Task)tasks.get(i);
                int taskNumber = task.getSystemAttributes().getTaskNumber();
                String title = task.getTitle();
                String taskId = task.getSystemAttributes().getTaskId();

                String outcome = task.getSystemAttributes().getOutcome();
                int priority = task.getPriority();
                int taskNr = task.getSystemAttributes().getTaskNumber();
                String taskDefId =
                    task.getSystemAttributes().getTaskDefinitionId();
                System.out.println("In task loop... title/taskDefId = " +
                                   title + " / " + taskDefId);
                // Change priority from 3 to 1 for SimpleApprove


                /* if(outcome == null)
{
outcome = "APPROVED";
taskSvc.updateTaskOutcome(ctx,taskId,outcome);
}*/

                System.out.println("Task #" + taskNumber + " (" + title +
                                   ") is " + outcome + " has priority " +
                                   priority + " Id = " + taskId + " taskNr= " +
                                   taskNr);
                if (title.equalsIgnoreCase("ApproveTheOrder for Commiskey")) {
                    System.out.println("*** Getting payload for ApproveTheOrder Human Task...");

                    //
                    Task task2Update =
                        querySvc.getTaskDetailsByNumber(ctx, taskNumber);

                    System.out.println("Getting the Payload...");
                    Element el = task2Update.getPayloadAsElement();
                    System.out.println("el "+el.toString());
                    Element newEl = getPayloadValues(el);
                    task2Update.setPayloadAsElement(newEl);
                    taskSvc.updateTask(ctx, task2Update);
                    //
                }
            }
        } catch (Exception e) {
            //Handle any exceptions raised here...
            System.out.println("Caught workflow exception: " + e.getMessage());
        }

        return "done";
    }

...

    public static Element getPayloadValues(Element pElement) {
        System.out.println("getPayloadValues() for Payload of type " +
                           pElement.getFirstChild().getNodeName());
        NodeList nl = pElement.getChildNodes();

        Node parentNode = nl.item(0);
        NodeList nlChildren = parentNode.getChildNodes();

        for (int i = 0; i < nlChildren.getLength(); i++) {
            Node n = nlChildren.item(i);
            String NodeName = n.getNodeName();
            if (!NodeName.equalsIgnoreCase("#text")) {
                Element myElement = (Element)nlChildren.item(i);
                NodeList nlElement = myElement.getChildNodes();
                String elName = n.getNodeName();
                String elValue = nlElement.item(0).getNodeValue();
                System.out.println("Element is " + elName +
                                   " with a value of " + elValue);
         
            }

        }
        return pElement;
    }


Test output -