Sunday, March 1, 2020

Getting Started with Logic Apps - EDI X12 to XML

Introduction

Any organisation which needs to do Business to Business communication using EDI Standards (EDI X12, EDIFACT, TRADACOMS etc) need to have some EDI System/Tool/Processor/Translator which does understand EDI standards and capable of receiving and sending the EDI messages.

In following post I  have explained the basics of EDI and EDI X12 standards 

And before starting the communication the business partners decide on how communication will be (transport protocol), what data format will be used , whether Acknowledgement is required etc etc and all this becomes the part of Agreement between them.

Following is high level picture of B2B EDI communication
B2B EDI Communication

When a EDI document is to be sent the Partner 1 EDI system refers to Agreement and creates the documents as per the instructions in it.

When a EDI document is to be received by the Partner 2, it's EDI system refers to Agreement to Validate the incoming document and to check if Acknowledgement - Technical(TA1)/Functional(997/999) is to be created or not.

Let’s  see how we can enable EDI communication with Logic Apps

Logic Apps and EDI together


Earlier it was BizTalk from Microsoft which has all capabilities around Enterprise Application Integration(EAI) and Electronic Data Interchange (EDI)(which is server product) but with the cloud shift where everything is desired as service, Microsoft provided Logic Apps to cater the integration needs for cloud users and it is continuously getting matured (most of BizTalk capabilities already added).

Read more about   Logic App Fundamentals 

To enable the logic apps to be leveraged in Business to Business EDI communication Microsoft introduced Enterprise Integration Pack,  which adds B2B capabilities like AS2 and X12, EDI standards support , XML capabilities like XML Validation, XSLT Transformation, Flat file to XML encode/decode etc.(Available as actions in Logic app e.g., AS2 Decode, Encode to X12 etc)

And an Integration Account which can be used to store all the artifacts required to run the Enterprise Integration like schema, maps(xslt), partners, agreements, certificates etc.



To make Logic App for B2B communications use of Enterprise Integration Pack is to be done and below are the steps to be followed
     1. Create Integration Account
     2. Add the artifacts which you would be needing in Integration account i.e.,     Schemas,Maps, Partners, Agreement etc.
     3. Create a Logic App
     4. Link the Integration Account to Logic App
     5. Use the artifacts and design the process/workflow

Let’s consider following scenario and create the solution – enabling Logic App for EDI X12 communication


Scenario


Say there is a company Techfindings and it wants to receive EDI X12 850 from its business partner and after getting it, it needs to convert it XML Purchase Order format which is expected by it's inhouse application for further processing.Inhouse application wants the file to be placed in a Blob Container.

And the business partner don't need any acknowledgement.

So we need to create a solution which will accept EDI X12 850 message and convert it to XML Purchase Order and forward it to Blob location for Inhouse application.

Steps in creating solution

    
    1. Create Integration Account
     
     On Azure portal, select All services and search for  "integration account" as your filter, and select Integration Account

create Integration account

Give Name , select subscription and resource group. Under Pricing Tier as Free and location where it is to be kept.

Note: Use same location when you create Logic App as integration accounts, must exist in the same region as your logic apps.


     2. Create/obtain the schemas

If you don't have BizTalk dev machine then you would need Visual Studio 2015 to create/edit the schemas and maps.

You can download community edition from - https://www.visualstudio.com/vs/older-downloads/

Also you will have to add following extension - Microsoft Azure Logic Apps Enterprise Integration Tools for Visual Studio 2015 2.0


Note: No extension available in later version above 2015, as the extension is borrowed from BizTalk  2016.Latest release of BizTalk 2020 happened recently but yet extension is not rolled out by Microsoft.


xml orders schema

Above is  XML Order Schema and below is the XML representation of EDI X12 850 schema.

EDI Schemas can be downloaded from -  https://www.microsoft.com/en-us/download/details.aspx?id=39087

EDI X12 850 schema


     3. Create map
     
    As we need to produce  Order XML select EDI X12 850 as source and XML Order as destination schema


EDI 850 to XML Order Map

  Once mapping is done, we need to get the xslt file , for that right click on map file -->Select Debug and in output window you can see the xslt path

xslt path


  make note of path where xslt is stored,as it is this which is to be uploaded to integration account (.btm file is not supported). 
  


  Following is the xslt

<?xml version="1.0" encoding="UTF-16"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:var="http://schemas.microsoft.com/BizTalk/2003/var" exclude-result-prefixes="msxsl var s0" version="1.0" xmlns:ns0="http://EDIToXMLDemo.XML_Orders" xmlns:s0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006">
  <xsl:output omit-xml-declaration="yes" method="xml" version="1.0" />
  <xsl:template match="/">
    <xsl:apply-templates select="/s0:X12_00504_850" />
  </xsl:template>
  <xsl:template match="/s0:X12_00504_850">
    <ns0:Orders>
      <Order>
        <Header>
          <PONumber>
            <xsl:value-of select="s0:BEG/BEG03/text()" />
          </PONumber>
          <PODate>
            <xsl:value-of select="s0:BEG/BEG05/text()" />
          </PODate>
        </Header>
        <xsl:for-each select="s0:PO1Loop1">
          <LineItems>
            <xsl:if test="s0:PO1/PO101">
              <ItemId>
                <xsl:value-of select="s0:PO1/PO101/text()" />
              </ItemId>
            </xsl:if>
            <xsl:if test="s0:PO1/PO102">
              <Quantity>
                <xsl:value-of select="s0:PO1/PO102/text()" />
              </Quantity>
            </xsl:if>
            <xsl:if test="s0:PO1/PO104">
              <UnitPrice>
                <xsl:value-of select="s0:PO1/PO104/text()" />
              </UnitPrice>
            </xsl:if>
            <xsl:if test="s0:PO1/PO107">
              <ItemDescription>
                <xsl:value-of select="s0:PO1/PO107/text()" />
              </ItemDescription>
            </xsl:if>
          </LineItems>
        </xsl:for-each>
      </Order>
    </ns0:Orders>
  </xsl:template>
</xsl:stylesheet>

     4. Add schemas and map to Integration Account

Go to the integration account created earlier in step1  and select the Schemas tiles and click on add button.


add schema

Give name and browse to schema path and select the schemas, below is how it should look

Add schemas in integration account

Read related error -   Selected file must be between 1 and 2097152 bytes


Go to the integration account created earlier in step1  and select the Maps tiles and click on add button. 
   Give name and browse to xslt path(saved in step 3) and below is how it should look
upload map in Integration Account






     5. Add Partners

Go to the integration account created earlier in step1  and select the Partners tiles and click on add button.



            
The partner which represents your organization is known as host partner -Techfindings is identity value given which is mutually defined thus qualifier is ZZ and the second partner is known as the guest partner - EDIPartner is identity value given which is mutually defined thus qualifier is ZZ. The guest partner can be another company, or even a department within your own organization.

Add partner in integration account


     6.  Add Agreement

Go to the integration account created earlier in step1  and select the Agreement tiles and click on add button.


Add agreement in integration account

You and your trading partner mutually decide about how the communication has to be done i.e. protocol or transport-specific. In our scenario its X12 protocol based communication.

For transport specific B2B communication read - Getting Started with Logic Apps - AS2

Here as we are going to receive EDI document, so Receive settings of the agreement has to be defined.

Receive Settings is organized into these sections:

1. Identifiers -we have not use any Identifier
2. Acknowledgmentwe have not chosen any acknowledgement as we don't need it (Partner doesn't want)
3. Schemas - we have selected 00504 as version, 850 as Transaction type and EDI_850 as schema

Note: In case where some other EDI documents is to be from  same Partner, then we need to select Schema and Envelope for that as well . In the image below you can see another edi document configured i.e. EDI_315



receive settings
4.Envelopes - Standard Identifier and default values for GS
5.Character Sets and Separators - UTF 8
6.Control Numbers - Control numbers are supposed to be unqiue for each request. Select the desired one from below

Disallow Interchange Control Number duplicates - Stops duplicate interchanges. Checks the interchange control number (ISA13) for the received interchange control number. If a match is detected, within the specified number of days
Disallow Group control number duplicates - Stops interchanges with duplicate group control numbers.

Disallow Transaction set control number duplicates - Stop interchanges
 number is added when edi doc leaves, thus a range is to be defined, here it is starting from 1 till 999999999
7.Validation - You can configure the level of validation needed, have kept default 
8.Internal Settings - Here you specify how to process the interchange batch, what to do if Validation fails
receive setting in agreement logic app



Check following Microsoft article for more details about Agreement Settings - https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-enterprise-integration-x12#configure-how-your-agreement-sends-messages



     7. Create Logic App and link Integration Account

Create logic app, add http trigger and save it. Go to workflow settings and link the integration account created in step 1
link IA

Note: The way to receive EDI document can vary based on devlopment strategy, like FTP or Blob location can also be used





     8. Design the logic App


First step is to add Http request  followed by Decode X12 message

Decode X12 message action needs inputs from integration account i.e., schemas, maps and agreement. Thus we need to create a connection to Integration Account. Provide Name, select the Integration account and click on Create.
create connection to Integration account

And then provide it the message received by the trigger (http request) 
trigger body

Next is add For each action and Provide it output of above step -  goodmessages followed by Transform XML action
add transform xml action in foreach

Against content add following expression -- xml(base64ToBinary(item()?['Payload']))

Against map select EDI850_toXMLOrder

Last step is  to add Create Blob action. Before using this action, you need to create a storage account and container. See following - https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal


Create connection to blob and then select the location, give the name to the file/blob to be created and content would be whatever above map gives us

Below is the final Logic app

EDI to XML Logic APP

The logic app receives EDI 850 messager over http, then it Decodes EDI X12 message  and then  transforms it  in xml Order  with help of Transform XML action and finally places the converted file in blob container.

Testing


Below is EDI X12 850 test input posted using ARC to logic app



Test input

and xml file was created in blob container 


file in blob container

And below is what it has
final output



How it Works 


The logic app receives EDI 850 messager over http, then it moves to Decode X12  message action and it is here where all the magic happens.

It first reads the identities from the interchange and goes for searching an Agreement from the Integration Account to which we had created a connection to. 

With reference to found Agreement it validates if all is as expected i.e. if edi version is correct or not, are the delimiters in adherence to agreement, are values as per data type/restrictions or not etc.

It creates payload as per the option choosen in Internal settings(Internal batch Processing), the payload is xml representation of edi file with root node added to it and it also generates acknowledgement if agreement says so.

The valid documents are added in goodMessages(array of valid documents) else in badMessages(array of invalid documents).

To process the valid documents we need to go through each item in goodMessages thus ForEach action is used.


The payload created by Decode X12 message action is base64 encoded so we
need to decode it first and as map expects xml representation of the ST document so it is to be converted
into xml. Thus following expression is used in Transform XML action

                         xml(base64ToBinary(item()?['Payload']))

And finally the transformed xml is placed on the blob location.


In this post we covered the fundamentals of B2B EDI communication and walkthrough the steps on  how to make it cloud based using Logic Apps and Enterprise Integration pack.






Related Post 




No comments:

Post a Comment

If you have any suggestions or questions or want to share something then please drop a comment