Wednesday, December 28, 2016

Begineer to BizTalk Expert : Interview with Tord G Nordhal

Happy New Year to all and Welcome to the special 25th interview of the series, today's expert is Tord Glad Nordhal - who has been interviewed earlier before joining Microsoft, following is the link : http://tech-findings.blogspot.in/2014/10/begineer-to-biztalk-expert-interview_31.html


With a passion for Technology and specifically the cloud and its services Tord have acquired skills to be able to utilize the technologies that is provided out there. With the main focus of the product within the Microsoft Integration Stack. With experience within system architecture, solution architecture, and setting up and configuring large scale software for different clients worldwide,helping the customers to achieve more..


His current position requires him to have deep technical knowledge of BizTalk and Logic Apps, but most importantly what the customers expect and need to help them grow business, solutions running on-prem, hybrid or cloud. Tord held courses and been active with both online and offline communities within both BizTalk and Azure and have spoken at different events all over the world. He created products and methods as a system developer to increase productivity within the different technologies he has worked with and shared all these with the community. Sharing is caring!


Prior to the role in Microsoft TORD was awarded as MVP within the Microsoft Integration Stack.

Let's begin the interview.....

Mahesh:Who are you and what you do? When did you start working on BizTalk? 
Tord: I’m a family father of 4, I work as a Program Manager in Microsoft focusing on BizTalk. I started working with BizTalk way back and have worked with BizTalk versions from 2000 to our latest release BizTalk Server 2016

Mahesh:How did you mastered BizTalk (Learning path, amount of time)? 
Tord: I don’t know how much time I’ve spent learning BizTalk, but I found the product interesting and learned a lot around it. However I do love to share my findings with the community. Now as the Program Manager for BizTalk I continue my investment and passion for the product to build and make it better than it was, one of these things is to listen more to the community. We are coming out with a user voice page very soon to get more feedback from our customers, partners and community!

Mahesh: Which are the major projects you handled so far?
Tord: There are too many to mention and the too exiting ones I can’t talk too much about. However I’m more fan of listening to customers and partners (and this blog) to see what other people are using our product for. Understanding the stories around it helps us make it better.

Mahesh:How do you see BizTalk compare to other integration platform?
Tord: BizTalk is unique, both when it comes to the community and to the way it works. It’s a product that has been on the market for so many years that any child disease is gone. Customers are still investing in it and it plays a huge role to a lot of businesses around the world. Did you know that BizTalk has never shipped a security update? Not only that but with the unified story in the integration space at Microsoft we are even making a bigger effort. Logic App is our cloud iPaaS solution, very powerful and very easy to use, our on premise solution is BizTalk and we are heavily investing in both the products. API Management is also one of the products in our team, that can help us make a complete integration story both in the cloud and on premise.

Mahesh: What as per you is must to know to become an Integration(BizTalk) Expert?
Tord: Learn integration and understand why it’s so important to think one step ahead. No matter if you’re going to work on BizTalk or Logic apps (or any other) the mindset is the most important. Understanding the systems you’re communicating with is important. The other thing that I find very important is to always think that whatever I am making is going to run in production for a long time, and you need to make it perfect for a production scenario, no shortcuts!

Mahesh:What are your thoughts on forums,blogs and articles etc.?
Tord: So forums, blogs, articles, TN Wiki are all super important, this is where we share our knowledge (and where I spend a lot of my time looking for juicy information, like bugs, issues, concerns or even wishes for vNext). We reach out to people who have done different types of presentations or articles to get more information when we are interested in the feature (maybe for vNext). We are very proud of our community is huge, and the amount of data available is mind-blowing . I am very grateful for our community.

Mahesh:Your suggestion to a newcomers? What should be approach to get sound knowledge in BizTalk?
Tord: Read up and talk to the community, find your peers and mentors and start off with something simple.

Mahesh: There are many tools from community which support BizTalk in some or the other way(like BTDF, Bizunit etc), what do you say about it? Which ones you would recommend?Why?
Tord: I love when the community find ways to improve our product (in some areas so well that it blows us away). We got partners worldwide that make a living for themselves creating tools like BizTalk360, nSoftware etc. Although some of their products goes outside the area we’re focusing on we can see as the products get used heavily that there is a need for them. We appreciate any contributions customers, partners and FTEs (Full-time employees at Microsoft) do for our products. And we are more than eager to understand and help take this to the next step.

Mahesh:There are monitoring tools developed by other companies around BizTalk, is there any plan if Microsoft plans to acquire them and provide it out of box or any plans to develop monitoring tool?
Tord: No, there are no plans for this.

Mahesh:What are your thoughts around BizTalk certification?
Tord: BizTalk and integration in general has had a lack of certification the last releases. However I find that the tests are not mandatory (sadly you can go an buy all the answers a few days after we launch a new certificate). But I do think they can help you out to see where you should invest more time to learn more about our products, and understand more around them. And some customers do require certifications when they are looking for new vendors or partners.

Mahesh: Why certification around BizTalk is stopped?
Tord: Who said it stopped? We are working on something new but it will most likely be involving more the entire integration story, both on premise and in the cloud. However this is very time consuming and we have a strong focus on developing the products.

Mahesh:What is the future of BizTalk?
Tord: Brighter than ever I’d say. With the recent release of BizTalk Server 2016 release, and our continued investment and announcement of vNext, we are bringing the product one step further. We continue to state that our on-prem investment is to BizTalk Server while our cloud story is Logic apps and Enterprise Integration Pack.

Mahesh:No where it appears that Microsoft is focussing on BAM, ESB portal upgardation based on latest .Net version. What about supporting xslt 2.0?
Tord:So, we are looking into this to see how we can enhance these experiences. It’s not an easy job to pick the right paths to move towards, but we’re hoping to surprise you with some great features coming later.

Mahesh: Any thoughts on cloud, in particular Azure? 
Tord: For some companies this is the way going forward, the cloud is powerful and provides functionality you won’t be able to get on premise as easy as in the cloud. We also believe that in order to empower every person and every organization on the planet to achieve more we need to be agile and provide first class services, both in the cloud and on premise.

Mahesh:You were MVP before joining Microsoft, do you feel that responsibilities get added when you are MVP? What is your thought on MVP now?
Tord: MVPs are awesome, it requires a lot of work that we cherish, coming from the MVP area myself. I know how important it is to stay on top of things and being aware of what is happening in the field and at Microsoft. We believe that if we can continue to add value to the MVP program by being open and sharing information you will get more out of it, we also like to get your feedback on how you think we are doing. We have started monthly PGIs with the MVPs.

Mahesh: As per the Roadmap provided by Microsoft,LogicApps can be run on-premise in addition to Azure.Do you think Azure Stack Logic Apps on prem will supersede BizTalk Server?
Tord: No, this is not right anymore. BizTalk Server will be our on premise integration technology going forward, Logic apps is our cloud offering.

Mahesh: From my perspective, Microsoft keeps coming up with Overlapping technologies like recent ones MABS and Logic Apps, in some situation it gets puzzling. What you say?(MABS seems to be dead)
Tord: Yes, the cloud is confusing, even with technologies like ServiceBus, Data Factory, Stream Analytics etc. However some of these may have overlapping features they are all individually unique. They are designed to do different tasks and to help you integrate your systems better.

Mahesh: Last question but not least, this question is been asked every now and then -- maximum messages in a second BizTalk can process/What is the maximum size supported (I know it depends on machine config), would be great to get stats from you both :).
Tord: It all depends on your setup and configuration, I’ve seen super-fast BizTalk Environments, and I’ve seen environments pup 100gb files through BizTalk, but I wouldn’t recommend that. There is not hard limit on either one of these.

Thanks a lot Tord for taking out time and sharing your insights,experiences and info around Microsoft future plans, this will surely benefit many !!!

Feel Free to ask questions to Tord in the comments!!!!!!!! 






Related Post:


Tuesday, December 27, 2016

SqlDateTime Overflow. Must be between 1/1/1753 and 12/31/9999


In current application I have a call to Stored Procedure which updates a table which has some DATETIME column as well. I was testing for various combination and as per test cases all was as expected, except one scenario where my orchestration remained in Started mode itself.  

On seeing the Tracked Message Events, found that message has been sent to SP but no response is received (that's why Orchestration is in started mode).

So, next was to check why there was no response from SP and it was the below errors which made SP to break


Event 5754

Event 10034


Why it happened


As the error says, the date passed against the datetime parameter was out of accepted range. I had set default date to 0001-01-01 but the range starts from 1753-01-01

Invalid date range

What to do


I changed the default date to  1900-01-01 00:00:00.000 and all worked fine.


One more thing to do is, to have exception block for SQL exception :) 


.

Friday, December 9, 2016

Operation element is not declared

We were doing load testing of one of the interfaces which sends message to destination using WCF-WebHttp adapter configured dynamically in orchestration. Not much load - just 100 messages in few minutes, but found that 5 of the messages got suspended.

This was bit confusing as until now all went fine in all the env and no suspended messages, except here in Performance env.

Operation element is not declared


Why it happened

After looking out for this error, found that this is known issue with WCF-WebHTTP adapter along with Dynamic send Port when under load.

What to do

You need to apply the cumulative update provided by Microsoft, which can be found at following link:

Sunday, November 27, 2016

Begineer to BizTalk Expert : Interview with Rachit Sikroria

Welcome to 24th interview of the series, today's expert is Rachit Sikroria.

Rachit Sikroria is a Senior BizTalk Consultant at Tata Consultancy Services (TCS).  He has 6 years of experience with EAI and ETL technologies. He is specialized in Microsoft BizTalk Server 2006 - 2016, Microsoft Azure, Business processes (EAI / B2B, ETL & BPM), Microsoft .NET Framework & ESB and SOA solutions. He enjoys supporting the BizTalk Community through continued participation on the MSDN and TechNet Forums. He is an individual who can wear many hats in the application life cycle, but has a focus on design and development, with the goal of becoming more of an Architect. 
Microsoft awarded him the prestigious Azure MVP of the year 2016 for his outstanding contribution to the Microsoft BizTalk Server community.

One applaudable thing is that Rachit is been top answerer on MSDN forums:



Let's begin the interview.....

Mahesh: Who are you and what you do? 
Rachit: My name is Rachit Sikroria. I am based in Gurgaon, India where I serve Tata Consultancy Services as a Senior BizTalk Consultant, specialized in Integration and Cloud technologies on the Microsoft technology platform. I have had the opportunity of working in many exciting industries such as Commercial Aviation, Banking, Finance & Energy and Utilities.

Mahesh: When did you start working on BizTalk?
Rachit: It was 31st Jan, 2011 when I was hired by a consulting firm. I was a novice at that time with only a training of a fortnight. I was then assigned a project for one of our clients where I started learning and exploring with BizTalk Server 2006.
Throughout my career, I have been opportune to work on latest technologies with a lot of sleepless and tough yet exciting moments in service.


Mahesh: How did you Master BizTalk Server?
Rachit: No one can really master BizTalk. It is just that we can become better and better with every successive project. In the initial years of my career, I was required to read a lot of books and blogs but with time I realized that reading is not enough, it is important that one should start practically implementing things too. So, what counted as actual learning for me were the real situations that I bumped into almost every now and then. My knowledge came through the success that I found in those hit and trials.
Also, throughout my career I got the opportunity to work with different Industries with different integration requirements which helped me gain the expertise while working out the best fit solutions.
At this stage of my career I can say one thing about BizTalk, that, I live it, breathe it, and absolutely love it.

Mahesh: What are your major projects?
Rachit: I have completed 2 major projects which lasted for a year each. One of them was for a well-known financial company where we had to bridge the front-end website with the back-end systems and help in building their middleware/BPM platform using BizTalk Server.  The other was for one of the major airlines of UK where we had to setup their Booking and E-Ticketing system. 
These two have been the most demanding and inspiring projects I was involved in so far where we faced challenges in removing blocking issues, and satisfying low-latency requirements by implementing different design patterns within the stringent SLAs. 
I also got a chance to integrate BizTalk with different software packages like SharePoint, SAP, Oracle, SQL and CRM Dynamics. 

Mahesh: How you compare BizTalk with other Integration products? 
Rachit: I have not used any other integration platform so far. Therefore, I can’t really compare! But I can surely say that BizTalk Stack (BizTalk Server, Windows, SQL Server, Visual Studio, .Net, SharePoint, IIS) is much more than just a BizTalk Server. You can do a lot more with BizTalk than any other offering. 
Another advantage is of course the encouraging community support we have. 

Mahesh: What as per you is a must to know to become an Integration (BizTalk) expert.
Rachit: Fore-mostly, the professional should have a passion for technology and BizTalk/Integration in particular. The second is the skill to analyze the requirement and have the capability to decide when to use BizTalk and when not to. We come across a lot of questions on the MSDN forum where professionals have implemented BizTalk instead of the ETL tool like SSIS that was more suited.  
Not to forget in today's cloud-enabled world, it is important that the experts should evolve with the new offerings. With the recent release of BizTalk Server 2016, we can now connect both traditional on-premises applications to cloud-native applications seamlessly. We have BizTalk Azure VMs available to move our on-premises setup to cloud as well. So, there is a lot to explore and as an integration expert, one should be ready to stay abreast of the latest technologies.

Mahesh: What are your thoughts on the forums, blogs and articles etc.?
Rachit: I think that the existence of platforms like Microsoft MSDN Forums, Blogs, Code Gallery, and TechNet Wiki have played a crucial role behind the success of Microsoft technologies. They are sources of readily available expertise and you get solutions for most of the issues in just few clicks over the Internet. 


Mahesh: Your suggestion to a newcomer? What should be their approach to get sound knowledge in BizTalk?
Rachit: I always say that Knowledge is no fortune that happens through a windfall gain. One gets in the cycle of learning, remembering and sharing and it is a long-devoted process. Work with lots of learned techies, read and observe. Like in any other discipline, start with the basics and in case of severity, come back to them. Chances of failing minimize if one has a strong base knowledge. Perfection comes later!
For one stop shop for BizTalk, I would refer the following link. This link holds all the excellent articles written by MVPs and other BizTalk Community members across the globe.
http://social.technet.microsoft.com/wiki/contents/articles/2240.biztalk-server-resources-on-the-technet-wiki.aspx

Mahesh: There are many tools from community which support BizTalk in one or the other way (like BTDF, Bizunit etc), what do you say about it? Which ones you would recommend? Why?
Rachit: I would say that they all are life savers. Tools like BTDF, BizUnit, BizTalk Server Pipeline Component Wizard, BizTalk CAT Instrumentation Framework Controller, BizTalk Mapper Extensions UtilityPack and many more are highly useful tools.

Mahesh: What are your thoughts about BizTalk certification?
Rachit: If I were to choose, I would rather prefer a professional having Microsoft Forums and Technet contributions listed in his/her resume than a certification. The former reflects more about the person’s capability and penchant.

Mahesh: What is the future of BizTalk?
Rachit: People keep asking question like “Is BizTalk Server Going Away at Some Point”, “Does Microsoft BizTalk Server Have a Future?” and “Is BizTalk dead?”. My answer to them is NO, BizTalk is not dead, it is very much alive and kicking. BizTalk server is still the best on-premises integration platform available, and it is not changing anytime soon. Microsoft is still committed to release a new version of BizTalk Server around about every 2 years. BizTalk and Logic apps, both are integral part of Microsoft integration roadmap. With the release of a new Logic Apps adapter for BizTalk and the BizTalk Connector for Logic Apps the integration between BizTalk Server and Logic Apps is fulfilled.


Mahesh: What motivates you to do the community work?
Rachit: For me, the highest compliment is to see people finding their solutions through my answers. I am in awe of other contributors who have achieved intellectual zenith. The biggest motivation is that this community work has helped me both, share and receive knowledge. So, when I am answering someone’s question, I’m also learning the many things I am ignorant of. Recently, I even started sharing code samples and writing Wiki articles. 


Mahesh: Being an MVP, do you feel that responsibilities get added? What is your thought on MVP?
Rachit: May be not a responsibility, but it is definitely an impetus to do more in this field. To learn that Microsoft recognizes our efforts, a virtual leadership is created in us that makes us go on and on with our services. MVP award has given me that extra connection with the Microsoft Product team and it leads to professional growth. Being a part of the Private mailing-list that has all Azure MVPs together and several Microsoft employees too (including members of the Azure team), is like a dream come true! I am now offered invitations for all the webcasts and tech session which keep me abreast of the latest progress in Microsoft technologies.


Mahesh: From my perspective, Microsoft keeps coming up with Overlapping technologies like recent ones MABS and Logic Apps, in some situation it gets puzzling. What you say? 
Rachit: Honestly, this strategy left me quite disappointed too, it seems a little attention was paid to MABS by the product team. Now that MABS is being phased out in favor of Logic and API Apps, and rightly so, as MABS never had the full capabilities of BizTalk Server. For example, you don't have orchestrations/long-running workflows, there was no way to have parallel and conditional execution, No in-built business process management capabilities or business rules engine. It was just good for lightweight integration scenarios at best.


Thanks a lot Rachit for sharing your insights and experiences, this will surely benefit many !!!

Feel Free to ask questions to Rachit in the comments!!!!!!!! 






Related Post:



Wednesday, November 23, 2016

How to install BizTalk Health Monitor v3.2

In previous version of BHM, you had to register the MBVSnapIn.dll 🙂

Here are the steps:

  • Open a command prompt as an administrator
  • Navigate to the BizTalkHealthMonitor folder
  • Run InstallUtil.exe MBVSnapIn.dll


And followed by adding it to MMC, but this is lot simplied in version 3.2 .

Following is the Link to download setup file:
https://www.microsoft.com/en-us/download/details.aspx?id=43716



download page

After download, unzip the package to your preferred location – I have unzipped it in 

unzipped files

Click on BHMSetup and Run as Administrator

BHM Setup

You will see Installation page
BHM installation start page

Click on Start Installing 

BHM installation start page1

In few seconds you will see installation is completed

BHM installation start page installation complete

As can be seen I have checked all the check boxes, you can uncheck based on your preferences. I clicked on Exit and following error popped out
BHM Error

This is a known issue with Windows Server 2012 and Windows 10 and to resolve this issue, steps are mentioned in the error page (see above).
MYHCQueries

Click on properties and  go to General Tab
MYHCQueries Properties----> MYHCQueries Properties1

You see against Security the Unblock button, we need to click on it.


Same process has to be followed for MaintenanceRep.dll

MaintenanceRep MaintenanceRep properties

Now repeat the initial step of installation and after installation now you should be able to see following, BizTalk Health Monitor MMC having BizTalk Server Administration and Event Viewer (it is because we had checked the option to include while installation)

BHM profile

As I installed it on my personal demo machine, only one profile is shown, click on Analyze Now
BHM Dashboard
On the Dashboard you can see the current analysis report and in case you want to analyse later sometime -- you can simply right click and choose Analyze now

BHM Dashboard Analyse now


Note that whenever you run Analyzer, log for that is maintained as can be seen in image below(marked in yellow)

BHM Report








Thursday, November 17, 2016

How to get Retry Count of Successful sent attempt -Two way Scenario

In the last post we saw how to get retry count of successful attempt of transmission in one way scenario, there we used Delivery Notification property of logical send port where NACK actually generates Delivery Failure exception, but that property doesn't work with web service call i.e. request- response port.

Why?

In case of two way port the "Delivery Notification" is implicit in the Receive Operation, so either you receive a service response message or an error message thus completing the call i.e. confirming Delivery. 

But what if the service you are calling is stopped or unavailable temporarily? Do we get Delivery Failure exception? 

To demonstrate what exception we get and how to get Retry count of the successful attempt in case of webservice call, have created very simple WCF Service with four methods and deployed it on the same machine, followed by consuming it using BizTalk WCF consuming Wizard.

WCF service code



published webservicebrowsing the service








How to do 



Scenario:


The scenario is simple, you have to invoke a Add method of HelloWordWCFservice and if there is failure then retry has to be done and log the retrycount when service call was successful.




Input schema 

input schema

Output schema and generated files by wizard

artifacts generated by Consuming wizard

Creating Orchestration


Orchestration with retry logic

The steps remain same as demonstrated in earlier post,the change here is we have request-response port, one send port where the response message from service call and have included following exception blocks 

exception blocks


Here if you see, I have placed the exception blocks based on relevance, of most likely exception to be raised.

Testing



1. Service is up and running


In this scenario, the call was successful and the response message was saved at the configured folder.


event log for successful transmission

2. Service is stopped with Retry count set to 3 and retry interval set to 1 minute (Service started after 1 minute)


event log for exception caught

As can be seen above the type of exception raised was Soap Exception when the service was stopped and we get the RetryCount of the successful attempt i.e. 1 when service started.

Note: The Delivery Notification property(either None or Transmitted) has no effect on the behavior


Download Sample



Wednesday, November 16, 2016

How to get Retry Count of Successful sent attempt - One way Scenario

There is a provision in BizTalk to set Retry Interval and Retry Count for both scenarios i.e. Static and Dynamic. For static we set it on the send ports and in case of dynamic we set the context properties in Orchestration/Pipeline.

But, I found no where or no way to get the Retry count of the successful attempt - when message was successfully delivered to destination system/service. If anyone knows, please let me know in comments :) .

So what if there is need to get that Retry count, say if business team wants to track and log it. 


How to do

To explain the process will use simple scenario file routing as below

Scenario: 


We receive files at a folder which is to be routed to different folder and in case of transmission failure there has to be retries done for every minute but only three times and log the retry count when transmission was successful.

To implement this I have used Orchestration, where I receive a file and send it as it is inside a scope with transaction type set to None and an excpetion handler to catch Delivery Failure exception.

Creating Orchestration

Orchestration with retry logic

Here we are not going to use the Retry mechanism provided by BizTalk, instead we build custom Retry logic with help of two variables, Loop Shape and Delay Shape.

Var_RetryCount and Var_RetryInterval are variables set in orchestration (hard coded just for the post, ideally it should be configurable like SSO application/Config files or DB).

Retry count variable declaration
                     retry interval variable


The logical send port has Delivery Notification set to Transmitted, doing this makes orchestration to wait for an acknowledgement from physical send port - ACK/NACK. If transmission goes ok then positive ACK is received  and if something goes wrong then exception is thrown to orchestration which is caught by exception handler (provided exception handling is implemented).
Delivery notification on logical port

Question might arise, what happens if Delivery Notification is set to None on the Logical send Port and address is incorrect.

The message is sent and the flow continues and process ends as successful,no exception is caught even though Exception handler is there(read no exception raised), but there is error logged in Event Viewer. 


The system cannot find the path specified

It is perfectly valid behavior as the orchestration is not set to hear back from Send Port.In this case Orchestration is instructed to submit message to Message Box (to which the send port has subscription) and continue to check if exception occurred, if not then write entry to event log.

The logic here is, the Loop shape here is going to do retry based on Var_Retrycount(number of times).
Loop counter

And Delay shape is to induce the interval between the retries based on Var_RetryInterval.

retry interval in Delay


And to decide whether retry is to be done is based on Var_ExceptionOccurred, which gets set to true in Exception block whenever exception is encountered.

Exception occurred flag set to true


check if exception occurred

If exception occurred, then we just increment the loop counter, wait for the delay set and control is given back to loop shape for another retry(provided retry count is not reached).


Increment loop counter



 And if no exception i.e. the transmission was successful, then the Var_SuccessRetryCount is set to loop counter and Loop counter is set to Var_RetryCount+1 so that no more retry is done. 
set success retry countSo when Var_SuccessRetryCount is zero it means no retry was done and transmission was successful, when it is one - means transmission was successful at 1st attempt of retry and so on.



Testing



1. Correct address

In this scenario, the input file was successfully dropped to the destination folder

2. Incorrect address with Retry count set to 3 and retry interval set to 1 minute (address corrected after 1 minute)


incorrect address

 As can be seen in below image, there was delivery exception and current retry count and when will retry done is logged to event log, same happens to next retry also
current retry count

Then after I correct the address 


correct retry count

Thus, message was sent successfully at second retry .

Message sent successfully at retry count


file at destination


So now we have the RetryCount when transmission was successful.



Download Sample

Related Post