Sunday, November 3, 2019

Getting Started with Logic Apps - XML to EDI X12

Continuing from last post where I explained the basics of EDI and EDI X12 standards, let’s  see how we can enable EDI communication with Logic Apps

Logic Apps and EDI together


Earlier it was BizTalk from Microsoft which has all capabilities around Enterprise Application Integration(EAI) and Electronic Data Interchange (EDI)(which is server product) but with the cloud shift where everything is desired as service, Microsoft provided Logic Apps to cater the integration needs for cloud users and it is continuously getting matured (most of BizTalk capabilities already added).

Read more about Logic App Fundamentals

To enable the logic apps to be leveraged in Business to Business EDI communication Microsoft introduced Enterprise Integration Pack,  which adds B2B capabilities like AS2 and X12, EDI standards support , XML capabilities like XML Validation, XSLT Transformation, Flat file to XML encode/decode etc.(Available as actions in Logic app e.g., AS2 Decode, Encode to X12 etc)

And an Integration Account which can be used to store all the artifacts required to run the Enterprise Integration like schema, maps(xslt), partners, agreements, certificates etc.


Note : If you have worked on BizTalk, then you know that it had capability of storing artifacts as it had dedicated database for that  whereas Logic App is way to orchestrate a business process in which you connect various applications/services/components and for that you create API connections, and only those connection information is stored in Logic App and not any artifacts. Thus the need of Integration Account.

To make Logic App for B2B communications use of Enterprise Integration Pack is to be done and below are the steps to be followed
     1. Create Integration Account
     2. Add the artifacts which you would be needing in Integration account i.e.,     Schemas,maps, Partners etc.
     3. Create a Logic App
     4. Link the Integration Account to Logic App
     5. Use the artifacts

Now, let’s focus on topic of this post of – enabling Logic App for EDI X12 communication


Scenario


Say there is a company Techfindings and it wants to place an order with it’s partner company. But the partner company accepts Purchase order only in EDI X12 850 format whereas Techfindings system generates XML based order.


Steps in creating solution

    
    1. Create Integration Account

     On Azure portal, select All services and search for  "integration account" as your filter, and select Integration Account

create Integration account

Give Name , select subscription and resource group. Under Pricing Tier as Free and location where it is to be kept.


     2. Create/obtain the schemas

You would need Visual Studio to create/edit the schemas and maps. Also you will have to add following extension - Microsoft Azure Logic Apps Enterprise Integration Tools for Visual Studio 2015 2.0


Note: No extension available in later version above 2015, as the extension is borrowed from BizTalk and last release of it is BizTalk 2016(extension is based on the BizTalk components and those components are only available for Visual Studio 2015). As soon as new version of BizTalk is out then the extension for BizTalk associated Visual studio would be available.


sample XML Order Schema

Above is  XML Order Schema and below is the XML representation of EDI X12 850 schema, it can be downloaded from https://www.microsoft.com/en-us/download/details.aspx?id=39087

EDI X12 850 schema


     3. Create map
     
    As we need to produce EDI X12 850, select Order XML as source and EDI 850 as destination schema
order xml to edi 850 map

  Once mapping is done, test and validate it (make sure all warnings and errors are resolved)

validate map


    and make note of path where xslt is stored,as it is this which is to be uploaded to integration account (.btm file is not supported).

xslt path

  Following is the xslt

  <?xml version="1.0" encoding="UTF-16"?>
   <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:msxsl="urn:schemas-microsoft-com:xslt" xmlns:var="http://schemas.microsoft.com/BizTalk/2003/var" exclude-result-prefixes="msxsl var s0" version="1.0" xmlns:s0="http://Inbound_EDI.OrderFile" xmlns:ns0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006">
  <xsl:output omit-xml-declaration="yes" method="xml" version="1.0" />
  <xsl:template match="/">
    <xsl:apply-templates select="/s0:OrderFile" />
  </xsl:template>
  <xsl:template match="/s0:OrderFile">
    <ns0:X12_00401_850>
      <ST>
        <ST01>
          <xsl:text>850</xsl:text>
        </ST01>
        <ST02>
          <xsl:text>1000</xsl:text>
        </ST02>
      </ST>
      <ns0:BEG>
        <BEG01>
          <xsl:text>00</xsl:text>
        </BEG01>
        <BEG02>
          <xsl:text>PO</xsl:text>
        </BEG02>
        <BEG03>
          <xsl:value-of select="Order/Header/PONumber/text()" />
        </BEG03>
        <BEG05>
          <xsl:value-of select="Order/Header/PODate/text()" />
        </BEG05>
      </ns0:BEG>
      <ns0:N1Loop1>
        <ns0:N1>
          <N101>
            <xsl:text>BT</xsl:text>
          </N101>
          <N102>
            <xsl:value-of select="Order/Header/CustomerContactName/text()" />
          </N102>
          <N104>
            <xsl:value-of select="Order/Header/CustomerID/text()" />
          </N104>
        </ns0:N1>
        <ns0:PER_2>
          <PER01>
            <xsl:text>BT</xsl:text>
          </PER01>
          <PER04>
            <xsl:value-of select="Order/Header/CustomerContactPhone/text()" />
          </PER04>
        </ns0:PER_2>
      </ns0:N1Loop1>
      <ns0:PO1Loop1>
        <xsl:for-each select="Order/LineItems">
          <ns0:PO1>
            <PO101>
              <xsl:value-of select="PONumber/text()" />
            </PO101>
            <PO102>
              <xsl:value-of select="Quantity/text()" />
            </PO102>
            <PO103>
              <xsl:value-of select="UOM/text()" />
            </PO103>
            <PO104>
              <xsl:value-of select="Price/text()" />
            </PO104>
            <PO105>
              <xsl:text>EA</xsl:text>
            </PO105>
            <PO107>
              <xsl:value-of select="ItemOrdered/text()" />
            </PO107>
            <PO109>
              <xsl:value-of select="Description/text()" />
            </PO109>
          </ns0:PO1>
        </xsl:for-each>
      </ns0:PO1Loop1>
      <SE>
        <SE01>
          <xsl:text>3</xsl:text>
        </SE01>
        <SE02>
          <xsl:text>1000</xsl:text>
        </SE02>
      </SE>
    </ns0:X12_00401_850>
  </xsl:template>
  </xsl:stylesheet>

     4. Add schemas and map to Integration Account

Go to the integration account created earlier in step1  and select the Schemas tiles and click on add button.


add schema

Give name and browse to schema path and select the schemas, below is how it should look

Add schemas in integration account


Go to the integration account created earlier in step1  and select the Maps tiles and click on add button. 
   Give name and browse to xslt path(saved in step 3) and below is how it should look


Add map in integration account



     5. Add Partners

Go to the integration account created earlier in step1  and select the Partners tiles and click on add button.



            
The partner which represents your organization is known as host partner -Techfindings is identity value given which is mutually defined thus qualifier is ZZ and the second partner is known as the guest partner - EDIPartner is identity value given which is mutually defined thus qualifier is ZZ. The guest partner can be another company, or even a department within your own organization.

Add partner in integration account


     6.  Add Agreement

Go to the integration account created earlier in step1  and select the Agreement tiles and click on add button.


Add agreement in integration account

You and your trading partner mutually decide about how the communication has to be done i.e. protocol or transport-specific. In our scenario its X12 protocol based communication.

For transport specific B2B communication read - Getting Started with Logic Apps - AS2

Here as we are going to send EDI document, Send settings of the agreement has to be defined.

Send Settings is organized into these sections:

1. Identifiers -we have not use any Identifier
2. Acknowledgmentwe have not chosen any acknowledgement as we don't need it 
3. Schemas - we have selected 00401 as version, 850 as Transaction type and EDI_850 as schema

Note: In case where some other EDI documents is to be sent to same Partner, then we need to select Schema and Envelope for that as well(in same Agreement).


4.Envelopes - Standard Identifier and default values for GS
5.Character Sets and Separators - UTF 8
6.Control Numbers - Control number is added when edi doc leaves, thus a range is to be defined, here it is starting from 1 till 999999999
7.Validation.


EDI x12 agreement send setting

Check following Microsoft article for more details about Send Settings - https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-enterprise-integration-x12#configure-how-your-agreement-sends-messages



     7. Create Logic App and link Integration Account

Create logic app, add http trigger and save it. Go to workflow settings and link the integration account created in step 1


link integration account


     8. Design the logic App

XMl to EDI X12 Order Processing Logic App

The logic app receives xml based Purchase Order over http, then it is transformed in xml representation of EDI X12 850 with help of Transform XML action. 

Provide triggerbody() as  content and select OrderFile_to_EDIX12_850 as Map (xslt )
Transform xml to EDI 850 XML

transformed EDI 850 XML is passed on to Encode to X12 message action.

Here we can use either of the option
i. Encode to X12 message action by identities
ii.Encode to X12 message action by Agreement 

In any option first we need to create connection to Integration account - it is via this connection, Agreement is referred by this action from Integration Account


create IA connection

Option 1 - Here the identifiers are provided based on it Agreement is selected and  EDI XML is encoded to EDI text file alongwith the applied envelopes(ISA-IEA and GS-GE).
encode to x12

Option 2 - Here you directly give the name of Agreement which is to be referred for encoding EDI XML to EDI text file




and in last step the EDI file is sent as an attachment via email . And content for that is Payload of above step (Encode to X12 message)


send email setting

Although I have used email, the partners can choose http based service to receive file,FTP, SFTP and FTPS etc.

Testing


Below is xml order test input posted using ARC


test input


and email is received with EDI 850 file as an attachment


output


Conclusion


In this post we covered the fundamentals of B2B EDI communication and walkthrough the steps on  how to make it cloud based using Logic Apps and Enterprise Integration pack.





Saturday, November 2, 2019

Getting Started with Logic Apps - EDI X12 Fundamentals

Few days back I was contacted by a beginner asking to guide on getting started on Logic Apps and EDI X12, although I tried to help him over call but thought of writing a post so that other beginners can be benefited.

Have tried to cover basics of EDI X12 in this post and how to use it in context to Logic app in following post - Getting Started with Logic Apps - EDI X12 Fundamentals



What is EDI?

In Layman term any data which is interchanged using electronic medium can be said as Electronic data interchange (EDI) .

Wikipedia says - Electronic data interchange (EDI) is the concept of businesses electronically communicating information that was traditionally communicated on paper, such as purchase orders and invoices.

Traditional way


Here the documents are created, put together in an envelope and sent across via mail/courier service and delivered to the receiver. In acknowledgment to receipt of documents, the receiver would sign on the delivery receipt and the sender may be informed back by mail/courier service provider.


Traditional Way of business


The sender can send a single document, say a single purchase order or multiple Purchase orders in an envelope. Also they can send Purchase order and Invoice document in an envelope to same receiver.

Details like whom to deliver, who is sender (To and From address) were provided over on an envelope by the sender and the mail service provider would stamp it which would then imply the date and time etc.

EDI way


EDI way can be defined as the transfer of structured data, by agreed message standards, from one computer system to another without human intervention.


EDI Way of business



There are many standards like  X12, EDIFACT, TRADACOM,ODETTE, etc. based on regions, business domains, government laws and lot of research but the intention of all is to define structured way for the entities/organisations/business  to create and send the documents.

The EDI standards are designed in such a way that it helps to create documents with most of the details along with the Interchange details (envelope details like To, From, Timestamp etc), and are released for use as a specific version e.g., 4010, 5020, 6010 etc.

Each time there is change i.e., amendments or reductions in the fields, change of data types etc then a new version is released, it can be major or minor release.

Say 4010 is current version in use and if some major changes are introduced then next version becomes 5010(version 5 Release 1) else if it is minor version then next version becomes 4020 (version 4 Release 2).


And certainly adding following benefits
  • Time and cost saving
  • Increase processing Speed
  • More Accuracy due to elimination of Manual errors
  • Secure and simple


EDI X12 Standard


EDI X12 is governed by standards released by ASC X12 (The Accredited Standards Committee) mainly used in US. Each release contains set of message types like invoice, purchase order, healthcare claim, etc. Each message type has specific number assigned to it instead of name. e.g., purchase order is 850, an invoice is 810 etc.

List of EDI X12 documents -  EDI X12 Documents List

Standard EDI X12 format data is text file separated by segment, segment delimeter, element , element delimeter and sub-element delimiters.
Segments always begin with a 2-3 letter code like ST or ISA. Some segments are mandatory, while some are not.

The mandatory segments of X12 are:


1.ISA(Interchange header) - It is the first segment and unlike any other segment in X12, ISA is fixed length of 105. This has information about the sender and recipient, version number.

The ISA Segment has the following structure

ISA01 Authorization Information Qualifier (length – 2)
ISA02 Authorization Information (length – 10)
ISA03 Security Information Qualifier (length – 2)
ISA04 Security Information (length – 10)
ISA05 Interchange ID Qualifier (length – 2)
ISA06 Interchange Sender ID (length – 15)
ISA07 Interchange ID Qualifier (length – 2)
ISA08 Interchange Receiver ID (length – 15)
ISA09 Interchange Date (length – 6)
ISA10 Interchange Time (length – 4)
ISA11 Interchange Control Standards ID (length – 1)
ISA12 Interchange Control Version Number(length – 5)
ISA13 Interchange Control Number (length – 9)
ISA14 Acknowledgment Requested (length – 1)
ISA15 Test Indicator (length – 1)
ISA16 Subelement Separator (length – 1)


   2. GS(Functional group header) - It is the second segment and is a set of X12 documents of the same document type (transaction set). It has information about the functional group, e.g. codes for the sender, recipient, and date/time of preparation.

The GS Segment has the following structure

GS01 - Functional Group Header Code(min/max – 2/2)
GS02 - Application Sender's Code(min/max – 2/15)
GS03 - Application Receiver's Code(min/max – 2/15)
GS04 - Date(min/max – 8/8)
GS05 - Time - HHMM(min/max – 4/8)
GS06 - Group Control Number(min/max – 1/9)
GS07 - Responsible Agency Code(min/max – 1/2)
GS08 - Version Code(min/max – 1/12)


    3. ST(transaction set header) -  It is the third segment, this marks the start of a transaction set in an interchange.

The ST Segment has the following structure

ST01 - Transaction Set Identifier Code - A three digit numeric code identifying the Transaction Set type
ST02 - Transaction Set Control Number

    4. SE(Transaction set trailer)- This marks the end of a transaction set.

The SE Segment has the following structure

SE01 - Number of included segments
SE02 - Transaction Set Control Number - Must match the Transaction Set Control Number in the ST.

    5. GE(Functional group trailer) - Segment of a set of X12 documents of the same transaction set. It tells you the number of functional groups and the control group number. It also marks the end of a group of transaction sets.

The GE Segment has the following structure

GE01 - Number of included Transaction Sets
GE02 - Group Control Number - Must match the group control number of the GS

    6. IEA(Interchange trailer)- this marks the end of the interchange. It contains information about the number of functional groups and the interchange control reference number. 

The IEA Segment has the following structure

IEA01 - Number of Included Functional Groups
IEA02 - Interchange Control Number - Must match the control number in the ISA.


Note:
i. There will be only one ISA and IEA segments present in an Envelope
ii. There can be multiple GS in an envelope.
iii.There can be multiple ST's

Above mandatory segments will be part of  every EDI X12 document no matter if it is EDI X12 850, 855, 810, 856 etc but the segments between ST and SE can vary from document to document.

If compared to XML structure - a segment is equivalent to a record type and each segment is separated by segment delimeter .A segment is composed of one or more data elements or composite data structures, which are equivalent to the fields in a record and each element is separated by element delimeter.

EDI X12 850 Sample


In the EDI X12 850 sample shown below "*" is the element separator and "~" is the segment separator. It has details of a single purchase order with two items.

ISA*00*          *00*          *ZZ*Techfindings   *ZZ*EDIPartner     *191031*0943*U*00401*000000010*0*T*:~
GS*PO*BTS-SENDER*RECEIVE-APP*20191031*0943*10*T*00401~
ST*850*0010~
BEG*00*PO*123**31/10/2019~
N1*BT*Maheshkumar Tiwari**2341~
PER*BT***9856543261~
PO1*1234*1000*LT*80000*EA**Petrol**Fuel~
PO1*1235*1000*Lt*65000*EA**Diesel**Fuel~
SE*7*0010~
GE*1*10~
IEA*1*000000010~


Summary


EDI X12 850 sample


Related Post