Showing posts with label MFT. Show all posts
Showing posts with label MFT. Show all posts

Friday, April 30, 2021

#843 OIC for File Transfer - Rename - Move to Target - Delete Source

This is a simple scenario from a customer - they need to move files generated by their ERP to OCI Object Storage. 







There are a couple of extras -

  • Source files are on an FTP Server
  • Source file names are in the format Prefix.FileName.Suffix for example - SAP001.Mahnwesen012021.mnf
  • Customer wants the file written to object storage without prefix and suffix,
  • Customer wants files deleted from FTP Server, once they have been saved to object storage
Ok, so here's my quick demo for such - 
Files on my ftp server - 




















Here is the integration - 












































So what are the steps here?

1. ListFiles - FTP Adapter --> List File operation - this will return a list of all the files to be moved.
2. For EachFile loop - 
2.1. Read the file from FTP (pass by reference)
2.2. WriteFile to Object Storage
3. Check result of WriteFile
3.1. If success (http code = 200) -- FTP Adapter -- Delete File  

I test the integration - 

























I check the ftp directory - 

























Now to the OIC configuration details - 

ListFiles - List the files in the source directory of the ftp server -

















For Each Loop configuration - 














Read File configuration - 




















Note the dummy file name. I will set this to the name of the current file -

Map to Read File configuration - 



 




WriteFile2Storage configuration - 



















Map to Write2Storage configuration - 

first step - pass file by reference -








second step - rename - 
















I do the substring magic to extract the file name without prefix and suffix - 

concat ("https://objectstorage.us-phoenix-1.oraclecloud.com/n/tenancy/b/bucket/o/", substring-before (substring-after ($currentFile/nsmpr3:File/nsmpr3:filename, "." ), "." ) )

SWITCH - check for success configuration - 





















DeleteFile - FTP connection configuration - 




















Map to DeleteFile - same modus operandi - pass file name and directory from the currentFile variable -







Tuesday, November 29, 2016

#536 Creating an MFT Cloud Service Instance

Begin by creating the backup storage containers -





















Start the MFT Service creation wizard -














































Do a sanity test on the environment.

ssh into the MFT vm and create 2 directories
/in and /out










create a file in /in e.g. touch myReport.txt











Create a simple file2file integration -















Monitor -


Thursday, January 15, 2015

#374 MFT --> Creating Custom Callouts - A simple example

Introduction 

Let's go straight to the MFT docs -


Oracle Managed File Transfer provides built-in compression, decompression, encryption, and decryption actions for transfer preprocessing and postprocessing. See Setting Up Transfer Preprocessing and Postprocessing Actions and Setting Up Source Processing Actions for details.

You can create new preprocessing and postprocessing actions, which are called custom callout
Custom callouts can be associated with either the source or the target. The sequence of processing action execution during a transfer is as follows:

Source preprocessing actions
Target preprocessing actions

Payload delivery

Target postprocessing actions
Source postprocessing actions








Here is the jar containing the interfaces -
YourFMW4MFTDir\mft\modules\oracle.mft_12.1.3.0\core-12.1.1.0.jar



Now before I get to my simple example, I just want to highlight the
excellent MFT resources available on the Oracle SOA PM blog.
Check it out right here

The Simple Example 


So now to my simple example -
I need to parse the files and convert the contents as follows -

Change any occurrences of Ireland to Eire
or
Change any occurrences Eire to Ireland and visa-versa.

The test file I use is as follows -









So I will expect the output to have Eire instead of Ireland.
Eire, for those interested is the Gaelic name for Emerald Isle.

Implement the Callout


I followed this section from the MFT docs to create the Java class that will
do the conversion.

The code is very simple - the main jar required is here -



I add this to the JDev project -










Here is the implementation -

package mftcallout;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;

import java.util.Map;

import oracle.tip.mft.engine.processsor.plugin.PluginContext;
import oracle.tip.mft.engine.processsor.plugin.PluginOutput;
import oracle.tip.mft.engine.processsor.plugin.PreCalloutPlugin;

public class NcIECallout implements PreCalloutPlugin {
    public NcIECallout() {
        super();
    }
    // Callout changes the payload
    @Override
        public boolean isPayloadChangeRequired(PluginContext context,
                Map calloutParams) {
            return true;
        }
    
    @Override
       public PluginOutput process(PluginContext context, InputStream input,
               Map calloutParams) {
           return null;
       }
    
    @Override
       public PluginOutput process(PluginContext context, InputStream input,
               OutputStream out, Map calloutParams) {
           String type = calloutParams.get("Type");
           if ("Eire2Ireland".equals(type)) {
               doConversionEire2Ireland(input,out);
               
           }
           else{
               doConversionIreland2Eire(input,out);
           }
           return new PluginOutput();
       }
    
    private void doConversionIreland2Eire(InputStream in, OutputStream out) {
            BufferedReader bufferIn = null;
            BufferedWriter bufferOut = null;
            try {
                DataInputStream dataIn = new DataInputStream(in);
                bufferIn = new BufferedReader(new InputStreamReader(dataIn));
                DataOutputStream dataOut = new DataOutputStream(out);
                bufferOut = new BufferedWriter(new OutputStreamWriter(dataOut));
                // For each line in the un-normalized file
                String line;
                while ((line = bufferIn.readLine()) != null) {
                    if(line.equalsIgnoreCase("Ireland")){
                        line = "Eire";
                    }
                    bufferOut.write(line);
                    bufferOut.write("\r\n");
                          }
            } catch (Exception e) {
                throw new RuntimeException(e);
            } finally {
                try {
                    bufferIn.close();
                    bufferOut.close();
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            }
        }
    private void doConversionEire2Ireland(InputStream in, OutputStream out) {
            BufferedReader bufferIn = null;
            BufferedWriter bufferOut = null;
            try {
                DataInputStream dataIn = new DataInputStream(in);
                bufferIn = new BufferedReader(new InputStreamReader(dataIn));
                DataOutputStream dataOut = new DataOutputStream(out);
                bufferOut = new BufferedWriter(new OutputStreamWriter(dataOut));
                // For each line in the un-normalized file
                String line;
                while ((line = bufferIn.readLine()) != null) {
                    if(line.equalsIgnoreCase("Eire")){
                        line = "Ireland";
                    }
                    bufferOut.write(line);
                    bufferOut.write("\r\n");
                }
            } catch (Exception e) {
                throw new RuntimeException(e);
            } finally {
                try {
                    bufferIn.close();
                    bufferOut.close();
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            }
        }
}

Now to the creation of the callout definition file -
The details here surface in the MFT console - i.e. the MFT user will be
able to add a pre-processing action - "Eire to Ireland text conversion"









I jar up the Java class and copy it, along with the xml file above to the following directory -
YourFMW4MFTDir\user_projects\domains\mft_domain\mft\callouts e.g.










Create this directory, is it doesn't already exist.








Now the next step is to register the new callout with MFT. We do this via WLST -



wls:/mft_domain/serverConfig> crtCalls('D:\Work\envs\FMW12c\MFT\user_projects\do
mains\mft_domain\mft\callouts\EireCalloutDef.xml')

Create a simple MFT Transfer to leverage the callout


Now I create a new source (File) in MFT and add a pre-processing step -










Now you can see how all of these values come from the callout definition file.
I then create a target and a transfer and test with the following file -







the output file is as follows -







btw. the Gaelic for England is Sasana, for France, is an Fhrainc.
  
Java Project here