Sunday, October 26, 2014

EDIFACT UNB and UNG Segment Fetcher Helper Class


Many times there is requirement to pass or store the values from UNB(Interchange Control Header) and UNG (Functional group) segments. This helper class can be called from map and individual values can be fetched.

How To Do

To create helper class do following:

1. Open Visual Studio
2. Add new C# library project
3. Add below code


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace EDIFACTSegmentValueFetcher

    public class SegmentValueFetcher
        public string fetchvalue(string segment, string elemsptr, string subelemsptr, int pos, int subpos)
            string subsegment;
            char esptr = elemsptr[0];
            char sesptr = subelemsptr[0];

            var lst = segment.Split(esptr);

            subsegment = lst[pos];
            if (subsegment.Contains(subelemsptr))
                var sublst = subsegment.Split(sesptr);
                return sublst[subpos - 1].Replace("'", "");
                return lst[pos].Replace("'","");

4. Sign the project
5. Add it to GAC

Add to Gac

How It Works

EDIFACT  messages are wrapped with envelope, at header it has UNB segment (Interchange control header - Mandatory), UNG (Functional group - Conditional) and UNH (Transaction Set) and at the trailer it has UNZ, UNE and UNT. Enveloping segments work in pairs, UNB-UNZ represents an interchange, UNG-UNE is a functional group inside of the interchange and UNH-UNT is a transaction inside the group.

Many times there is required to pass or store the values from UNB (Interchange Control Header) and UNG (Functional group) segments. But this is not part of the data and not all properties from Envelope are promoted or written in the context of the message. Segments have data elements which are separated by element separator, also sub data elements which are separated by sub-element separator.

This helper class helps in fetching the individual segment and subsegment values. You need to provide the whole segment, element separator, subelement separator and the number of field you want and number of subfield.

Example(Header segments of INVOIC):



Suppose from above sample,
1. If UNG7.1 value is to be fetched, in that case UNG segment, element separator, subelement separator, field value as "7" and subfield value as "1" will be provided as input and the output will be "D"
2. If UNG7.2 value is to be fetched, in that case UNG segment, element separator, subelement separator, field value as "7" and subfield value as "2" will be provided as input and the output will be "96A"

BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping - Configuring and Testing

Step 6: Configure and test the application

Create receive port

create receive port

Create receive location

create receive location

Create send port

create send port

Set the destination URI

destination uri

Configure the Orchestration

configure application

4. Create drop location
4. Drop the sample input file at input location
5. Check the output

output file

Note: Create a party and agreement and use sample file as per it. OR Enable EDIFACT fallback settings and configure it and use values in sample file as per it. As EDI receive pipeline first checks Party for the configuration, if no matching party found then it looks up for Fallback settings.

Possible Error

In a scenario where the particular element is not present in the segment (input) and if we try to fetch that particular element then following error is encountered.
Say, if UNB8 is not there in input and if tried to fetch that data then error occurs.


BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping - Orchestration

Step 5: Create Orchestration(FetchInterchangeData)

Create messages

  • INVOICmsg - Conforming to EDIFACT INVOIC Schema
  •  InterchangeContextmsg - Conforming to InterchangeContext Schema
  •  Invoicemsg - Conforming to CanInvoice Schema

Create variables

  • UNBString - of type String
  •  xmlDoc - of type System.Xml.Document
3. Create Receive shape and configure it to accept INVOICmsg
4. Construct the context message(InterchangeContext) and the Invoice message which is the final output(Invoicemsg) using ConstructMessage shape with following :

Message Assignment shape

Here a variable xmlDoc of type System.Xml is used and its LoadXml method is used to construct the context message.

Why XmlDocument variable? The class "XmlDocument" is a super class for all XML messages, and hence it can hold any type of XML message and subsequently any type of orchestration message.

InterchangeContextmsg = xmlDoc;
UNBString = INVOICmsg(EDI.UNB_Segment);

Assignment of EDI context values to the distinguished fields of context message

InterchangeContextmsg.UNB = INVOICmsg(EDI.UNB_Segment);
InterchangeContextmsg.UNG = INVOICmsg(EDI.UNG_Segment);

Fetching the element separator and sub element separator

InterchangeContextmsg.ElementSeparator = UNBString.Substring(3,1);
InterchangeContextmsg.SubElementSeparator = UNBString.Substring(8,1);


Here we created a intermediate message and populated it with the values coming from EDI Envelope header(which are detach during pipeline processing), so that we can use it multi-input map.

Transform Shape

Here two source are used, the incoming INVOICmsg(for the business data) and the InterchangeContextmsg(for the segment/context data) which was created in above message assignment shape

Transform Shape

And destination as Invoicemsg
Transform Shape select destination

Suppose UNB1.1 segment is to be fetched then following is to be done

Configuring Scripting Functoid

1. Select Script type as External assembly

2. Select the script assembly EDIFACTSegmentValueFetcher
3. Select the script class as SegmentValueFetcher
4. Select the script method as fetchvalue

script  Functoid

Note: Assemblies will only be available for selection if they are present in GAC.

Passing inputs to Scripting Functoid

1. Input[0] as UNB, link from InterchangeContext UNB element

2. Input[1] as ElementSeparator link from InterchangeContext ElementSeparator element
3. Input[2] as SubElementSeparator link from InterchangeContext SubelementSeparator element 
4. Input[3] as "1" hard code value
5. Input[4] as "1" hard code value

script  Functoid configuration

5. Create Send shape and configure it to Invoicemsg
6. Configure the logical port

build project

7. Sign the project, build it and deploy it

BizTalk Server : Fetching UNB and UNG Segment Values From Interchange Envelope and Mapping


To accept/process an EDIFACT message in BizTalk, EDI Receive Pipeline is used and in it the actual data (message -- UNH to UNT) is extracted after detaching envelope part (header and trailer) and added to message context. Only few UNB and UNG segments are promoted, whereas UNB_Segment and UNG_Segment are written, rest properties are not promoted due to performance reason.
This article intends to show how to fetch the individual UNB and UNG segment values using helper class and mapping those to the destination structure.


For this walkthrough have considered to use EDIFACT INVOIC  which are received and data from it has to be mapped to the destination structure, also Interchange Control and Functional Group Header values. Destination structure can be XML, Flat file or schema generated from database. For this post, destination structure is  XML Canonical Invoice.


EDIFACT  messages are wrapped with envelope, at header it has UNB segment (Interchange control header - Mandatory), UNG (Functional group - Conditional) and UNH (Transaction Set) and at the trailer it has UNZ, UNE and UNT. Enveloping segments work in pairs. UNB-UNZ represents an interchange. UNG-UNE is a functional group inside of the interchange and UNH-UNT is a transaction inside the group.

Many times there is required to pass or store the values from UNB (Interchange Control Header) and UNG (Functional group) segments. But this is not part of the data and not all properties from Envelope are promoted or written in the context of the message. Segments have data elements which are separated by element separator, also sub data elements which are separated by subelement separator.

In the input sample shown below "+" is the element separator, ":" is the subelement separator and "'" is the segment separator.

edifact file

How To Do

Step 1: Helper class which will split the segment and return individual value

Make this independent project so that it can be used wherever required without any dependency.

Step 2: Create BizTalk Project

  • Add EDIFACT INVOIC schema (EDI schema is shipped in as a self-extracting executable and when extracted, schemas can be found at drive:\Program Files\Microsoft BizTalkServer2010\XSD_Schema\EDI).

Edifact Invoic

  • Add/Create Output schema CanInvoice as shown below.
Cannonical Invoic

Step 3: Context Schema which will hold the UNB,UNG,ElementSeparator and SubelementSeparator

  • Add a schema to project (InterchangeContext).
  • Add four elements of type string 
    • UNBSegment
    • UNGSegment
    • ElementSeparator
    • SubElementSeparator
  • Make all four elements as Distinguished field.

mark distinguished feild

Step 4: Add references

  • EDIFACTSegmentValueFetcher.dll (Helper class) : This will help in using the method to split the segment and return individual values.

dll file location

  • Microsoft.BizTalk.Edi.BaseArtifacts.dll  : This will help in accessing the EDIContext properties in Orchestration.
edi.baseartifact dll

Saturday, October 11, 2014

The Filename, directory name, or volume label syntax is incorrect


In an application, it was required to set the file name of outgoing message with the current date.
And for that in the Orchestration, I set the FILE.ReceivedFileName context property with the current date.


And on send port used %SourceFileName% macro, thus expected was file with today’s date in the Out folder.

But while testing, no files were dropped in Out folder. This meant something was done wrong and to find it checked the Event Log, and I see there were no errors instead Warning. 

event log viewer

Why it happened:  

As can be seen in above figure, the details of the warning states that there was something incorrect in Filename, directory name. The issue was semicolon(“:”). Semicolon is not accepted as a part of FileName.

error in event viewer

What to do: 

To get rid of it and also as only date was needed, did following:

substring to get date

And now the FileName was set as Date only (no time, which had semicolon).
Date in event viewer

What are other characters which are not allowed?

To find this I tried with all characters and found some more which are not allowed/accepted. It’s easy or less error prone when setting file name to know what are not allowed, but it won’t be easy if value is set somewhere else.
invalid property value

invalid property value 1

invalid property value 2

invalid property value 3

invalid property value 4

invalid property value 5

invalid property value 6

Tuesday, October 7, 2014

Begineer to BizTalk Expert : Interview with Sandro Pereira

Welcome to fourth interview of the series, today's expert is Sandro Pereira.

Sandro works as a consultant at DevScope. His main focus is on Integration Technologies where he have been using .Net, BizTalk and SOAP/XML/XSLT since 2002. He is an active blogger, member and moderator on the MSDN BizTalk Server Forums and Code Gallery contributor. He has been awarded the Microsoft Most Valuable Professional (MVP) award for four consecutive years (2011,2012,2013 and 2014) based on his contributions to the world-wide BizTalk Server community.

Sandro is also a member and co-founder of BizTalkCrew along with Nino Crudele, Saravana Kumar, Steef-Jan Wiggers and Tord Glad Nordahl that are responsible for organizing BizTalk Innovation Day event all across Europe. Before starting the interview, I would congratulate Sandro for the success of his Book BizTalk Mapping Patterns & Best Practices.

Mahesh: Who are you and what you do? 
Sandro My name is Sandro Pereira, 36-year-old Portuguese guy and I’m living in the beautiful Porto region in the north of the country, a place of good food and Port wine. I am a BizTalk Consultant at DevScope ( and of course I am a community and technology enthusiast, working mainly focus with Microsoft Integration technologies and writing about it in several places: my personal blog, TechNet Wiki, Forums and several other communities’ platforms.

Mahesh: When did you start working on BizTalk?
Sandro: I started working with BizTalk Server, I think in June 2006 at DevScope as a junior BizTalk Consultant, I started optimizing and finishing some projects in progress in BizTalk Server 2004, and then installing new environments and perform a lot of project migrations to 2006. Which was nice because I was able to learn quite a few things and gave me a vast and important knowledge for what was to follow.

Mahesh: How did you mastered BizTalk (Learning path, amount of time)? 
Sandro: The reason I love working as an integration is that there is always something new for you to learn so… I’m still learning J. When I started there wasn’t too much information as exists today on BizTalk Server, but I think the learning path is the same as any other technology: by reading books, following some blogs, forums and mainly by don’t be afraid to try and fail. I learned quite a lot by myself working on the field and experience different things in my projects. But in a way I think now is a little easier to start developing and learn BizTalk Server.

Mahesh: Which are the major projects you handled so far? 
Sandro: In the last few years I'm been working on integration scenarios and Cloud Provisioning implementations at a major telecommunications service provider in Portugal. Is not only one project but a set of BizTalk projects that ultimately form a platform for supporting the entire Cloud infrastructure 
and integrate the Cloud platform with several internal legacy systems, such as HP Service Desk, billing systems… and other external partner systems. 
And also implementing outbound and inbound flows, archive and integration of thousands of electronic invoices, orders and status in EDI format with several partners and integrate these documents with SAP ERP system.

Mahesh: How do you see BizTalk compare to other integration platform? 
Sandro: Well to honest reply to this question I would have to know about all the other tools/platforms and I don’t. But sometimes I see clients and/or consultant companies trying to implement everything that is "integration" with BizTalk Server which in reality is not the right thing to do. For example the Microsoft integration stack in composed by Microsoft BizTalk Server (main integration product) but also with Microsoft Stream Insight, SQL Server Integration Services, SQL Server Master Data Services, Windows 
Server AppFabric, SharePoint Business Connectivity Services, Microsoft Azure Service Bus, a combination of . NET, WCF and AppFabric and recently Microsoft Azure BizTalk Services and so on. And each one of them have their purpose and their advantages and disadvantages and us, as consults, we 
should analyze the requirements and suggest the best tools/technologies/platform to fulfil them.

But I love BizTalk Server and I personal think that in general it is one of the best choice as an integration platform in a heterogeneous environment. But I'm not the best person to compare it with other competing platforms such as TIBCO, however I think that all of them have advantages and disadvantages. Probably TIBCO could have better performance in solutions that require low latency, but again probably will be more difficult to handle long running transactions.

Mahesh: What as per you is must to know to become an Integration (BizTalk) Expert? 
Sandro: As time passes, the normal process of life, we get more mature and we start to look some things differently. I began, for example, to have a great respect for this sentence "integration expert" and I honest don’t like to use it. Because if you work in integration it is almost impossible to know 
everything about it, there are thousands of different systems, protocols, formats and so on. You can really be very good in mapping, orchestrations, WCF, EDI or other common task but knowing nothing about RosettaNet, HL7 or legacy mainframes… and this is the reason because I love it. A new project 
may literally means that I have to be able to adapt and learn new things.

So in my honest suggestion the best thing that you should learn is to leave your comfort zone and try/learn new things, do not be afraid to try and fail and learn from that experience. You can start with easy things like if you are a BizTalk developer try to spend some time in BizTalk administration tasks or if you are an admin try to developer something by our own.

Mahesh: What are your thoughts on forums, blogs and articles etc.? 
SandroEssential resources for any community.

I think that all platforms have their purpose, Blogs, sites and magazines are good to share information and express the point of view of different authors, however has the disadvantage that its contents are immutable or changeable only by the author and reaches maybe a few thousand people. MSDN 
library is a great place to check official documentation. Forums is an excellent platform and great for helping people obtain answers for their problems and TechNet Wiki is and open platform that complements all the previous in which the greatest advantages is that anyone can pick up an 

article, fix it or add relevant content to the topic, making it always up to date.

And again when I started there wasn’t too much information as exists today, the number of resources grew substantially on BizTalk the last couple of years, and I'm not referring only to blogs and articles, open tools, scripts and so on. Last time I check there was 188 projects in CodePlex, 197 demos in Microsoft Code Gallery and 94 in TechNet Gallery.

Besides sending a clear messages to the community that BizTalk Server is alive and is here to stay it plays a vital role in the process of training and form new BizTalk developers or administrators but also to improve the existent skills of all of us.

Even me when I start to write my own posts or articles sometimes, or several times, I end up learning new stuff that I wasn't expecting, in this way, writing could also be a good way for us to learn new things about BizTalk Server and integration in general.

Mahesh: Your suggestion to a newcomers? What should be approach to get sound knowledge in Biztalk? 
Sandro: Get a book, there are several good books about BizTalk Server; start following some blogs; go to forums, not only to place questions but also for searching for problems; consume the MSDN and TechNet Wiki… and make a lot of samples, virtual labs and work with it.

Steef-Jan Wiggers wrote some while ago a Beginners Guide , if you are a beginner you should read that. And also this articles that contain a list of training resources for developers and administrators


Mahesh: What are your thoughts around BizTalk certification? 
Sandro: In general, certification is more valuable to present to clients as a way to "prove" that you have knowledge of the product. But not really a way or an indicator that you have experience and know-how. Your community activity, your projects or endorsements from community members has, in my personal opinion, more value.

Mahesh: What is the future of BizTalk? 
Sandro: Microsoft is putting a lot of effort and focus in the cloud, in this case, Microsoft Azure BizTalk Service where we will see a shorter cadence of releases than with BizTalk Server but on premise will stay, Microsoft will not move away from the on premise part and it will continue to invest and improve the platform. The release cadence of the Microsoft Integration is as follows:

* Major Version of BizTalk Server every 2 years – last major release was BizTalk Server 2013
* Minor Release of BizTalk Server every alternate year - BizTalk Server 2013 R2
* BizTalk Services will follow a release cadence of 6 months

At the end, not in the next 2/3 years in my opinion, this two platform (on premise and in the cloud) will converge and we will probably have again only one platform that can run on premises and in the cloud.

Mahesh: Any thoughts on cloud? 
SandroMicrosoft Azure BizTalk Services is starting and is young, again Microsoft is putting a lot of effort and focus on it, but it still have a long way to run and to become a powerful and mature integration platform. At the moment I suggest to use it to implement hybrid solutions like for example migrate small parts of process that requires low latency to cloud. But I think Microsoft is doing a good job.

Mahesh: What motivates you to do the community work?
Sandro: Well good question. I am a very active in the BizTalk community, I have my personal blog where I write an average of 100 post per year:  , I’m a member and moderator on the MSDN BizTalk Server Forums, TechNet Wiki author, Code Gallery and CodePlex contributor, member of BizTalk Brazil community where I write content in Portuguese, member of NetPonto community , BiztalkAdminsBlogging  community, editor of the magazine “Programar ”, public speaker and recently author of the book “BizTalk Mapping Patterns & Best Practices ”.

And I do all of this because… I like, writing is a hobby for me and a way to escape the stress of our work and our lives in general. Some persons love to run or playing other sports, others reading and so on. In my spare time: I enjoy traveling; play soccer with my friends; go to the cinema; read technical literature and write mainly in my blog about BizTalk.

But a key factor for all of that was José Antonio Silva, R&D Director at DevScope, it was he who encouraged me (almost forced me) to start contributing to the community. One day in mid-2008, he came to my side and told me I had reached a certain level in my career that I should start giving to the community what she had given me all these years and that if I did that, I would start to like it and he was right J. And it is healthy hobby, I don’t consider it work because I have no deadlines or requirements, I’m just playing with a technology that I like and write about it, I got the chances to improve myself and learn new things… and that led me to become Microsoft Integration Most Valuable Professional (MVP) since January 2011.

But most important there is nothing that can prepare you when someone that you don’t know reach to you and says "Thank you" it is priceless and makes me want to continue.

Thanks a lot Sandro, great insights , this will surely benefit many.

Feel Free to ask questions to Sandro in the comments!!!!!!!!

Related Post: