Monday, November 11, 2024

#1038 - OIC Integration Basics - Lab 9 File Processing Example

Introduction

Lab9 covers a simple file processing example - I read an xml file from the OIC File Server, I process that file and write it as a zip to another File Server directory. I'm using File Server here for both input and output, but you can extrapolate from this, i.e. output could also be an file hosted on a 3rd party FTP server (using FTP adapter) or a file on my local network (using File adapter). This lab will use the File Server action as well as the Stage File action. 

Lab 9 File Processing Example


Here is the format of the input and output files -

The following folder or directories will be used -

Here are the 2 input order files - these files are from different countries and contain orders from customers in those countries -

Here is an example of such a file - 

The orchestration steps are as follows - 
  • List files in the /ordersInbox
  • For each file
    • Read file contents and transform to the required output format
    • Write the result to a file in a temporary directory
  • After processing all files - zip the temporary directory
  • Write the zipped file to the /ordersOutbox directory
So let's see how we implement such in OIC -

List Files

Here we use the File Server native action - it is configured as follows -













For Each File


Next step is the processing of each file - here I use the For Each action - 





































The For Each is configured as follows - 

Note that the repeating element is set to the files iterator, returned by List Files -


Note the ability to select parallel processing.

Read File

Here I use the File Server native action to get a reference to the file. 
I then use the Stage File action to read the file. 

Check out the mapping - 

Now to the Stage File Read - 

I specify the XSD to be used when reading the file - 

Now that I have the file and its contents - I map its contents to the required output format and write the result to a temp file.

Write File

I use the Stage File action, but with the Write operation - 

Just in case you're wondering where the v_tempDir came from - I created it via an Assign action earlier in the flow. 

Zip the output Directory
























Here again I'm using the Stage File action, with the Zip operation - 


Write Zip to output Directory

I'm back to the File Server native action - 

Check out the mapping for the Write operation - 


Output Directory has been hardcoded, I could also have used a variable for this. 

Before we run, let's ensure the output directory is empty -


Testing the Flow

Let's check out the zip file - 

Summa Summarum

Many integrations are still file driven, so processing files will be a large part of your integration development work. I hope this lab gives you the basics so you can get started and be productive very quickly.

Lab Files

For those who want to try out the lab -

Input XSD

<?xml version="1.0" encoding="windows-1252" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.example.org"
            targetNamespace="http://www.example.org" elementFormDefault="qualified">
  <xsd:element name="OrderList">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="listNr" type="xsd:int"/>
        <xsd:element name="country" type="xsd:string"/>
        <xsd:element name="in_order" type="Order" minOccurs="1" maxOccurs="unbounded"/>
   
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
<xsd:complexType name="Order">
      <xsd:sequence>
        <xsd:element name="orderNr" type="xsd:string"/>
        <xsd:element name="product" type="xsd:string"/>
        <xsd:element name="unitPrice" type="xsd:int"/>
        <xsd:element name="quantity" type="xsd:int"/>
        <xsd:element name="customer" type="xsd:string"/>
        <xsd:element name="customerEmail" type="xsd:string"/>
        <xsd:element name="orderStatus" type="xsd:string"/>
   
      </xsd:sequence>
    </xsd:complexType>
 
</xsd:schema>

Output XSD

<?xml version="1.0" encoding="windows-1252" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://www.example.org"
            targetNamespace="http://www.example.org" elementFormDefault="qualified">
  <xsd:element name="OrderList">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="listNr" type="xsd:int"/>
        <xsd:element name="processingDate" type="xsd:date"/>
        
        <xsd:element name="out_order" type="Order" minOccurs="2" maxOccurs="unbounded"/>
   
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
<xsd:complexType name="Order">
      <xsd:sequence>
        <xsd:element name="orderNr" type="xsd:string"/>
        <xsd:element name="product" type="xsd:string"/>
        <xsd:element name="unitPrice" type="xsd:int"/>
        <xsd:element name="quantity" type="xsd:int"/>
        <xsd:element name="totalPrice" type="xsd:int"/>
        <xsd:element name="customer" type="xsd:string"/>
        <xsd:element name="country" type="xsd:string"/>
        <xsd:element name="orderStatus" type="xsd:string"/>
   
      </xsd:sequence>
    </xsd:complexType>
  
</xsd:schema>

Input XML Sample 

<?xml version="1.0" encoding="UTF-8" ?>
<OrderList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.example.org orderList.xsd" xmlns="http://www.example.org">
  <listNr>1</listNr>
  <country>Ireland</country>
  <in_order>
    <orderNr>1</orderNr>
    <product>iBike</product>
    <unitPrice>2400</unitPrice>
    <quantity>5</quantity>
    <customer>Niall Commiskey Ltd</customer>
    <customerEmail>n@c.com</customerEmail>
    <orderStatus>open</orderStatus>
  </in_order>
  <in_order>
    <orderNr>2</orderNr>
    <product>iScooter</product>
    <unitPrice>1200</unitPrice>
    <quantity>10</quantity>
    <customer>Zappa Inc</customer>
    <customerEmail>z@inc.com</customerEmail>
    <orderStatus>open</orderStatus>
  </in_order>
</OrderList>


Output XML Sample

<?xml version="1.0" encoding="UTF-8" ?>
<ns22:OrderList
xmlns:ns22="http://www.example.org">
<ns22:listNr>1</ns22:listNr>
<ns22:processingDate>2024-11-11Z</ns22:processingDate>
<ns22:out_order>
<ns22:orderNr>1</ns22:orderNr>
<ns22:product>iBike</ns22:product>
<ns22:unitPrice>2400</ns22:unitPrice>
<ns22:quantity>5</ns22:quantity>
<ns22:totalPrice>12000</ns22:totalPrice>
<ns22:customer>Niall Commiskey Ltd</ns22:customer>
<ns22:country>Ireland</ns22:country>
<ns22:orderStatus>pending</ns22:orderStatus>
</ns22:out_order>
<ns22:out_order>
<ns22:orderNr>2</ns22:orderNr>
<ns22:product>iScooter</ns22:product>
<ns22:unitPrice>1200</ns22:unitPrice>
<ns22:quantity>10</ns22:quantity>
<ns22:totalPrice>12000</ns22:totalPrice>
<ns22:customer>Zappa Inc</ns22:customer>
<ns22:country>Ireland</ns22:country>
<ns22:orderStatus>pending</ns22:orderStatus>
</ns22:out_order>
</ns22:OrderList>

















 











 

at

No comments:

Post a Comment