Begineer to BizTalk Expert : Interview with Wagner Silveira

Welcome to the 28th interview of the series, today's expert is Wagner Silveira

Wagner has been working in the New Zealand market since 2002, focusing on integration and services oriented architecture, working across various industries, including Healthcare, Education, Transports, Utilities and Retail.He is passionate about finding the right solution for the problem.
Specialties: Service Oriented Architecture, Software Development with Focus on Microsoft Technologies, Distributed Applications Design and Implementation, Health IT.

Very active in the integration community by blogging, writing, creating code samples, and speaking at events internationally. Recognized by Microsoft as a Microsoft Azure MVP for these efforts

Let's Begin Interview..

Mahesh: Who are you and what you do? When did you start working on BizTalk? 
Wagner: My name is Wagner Silveira and I'm the Principal Integration Architect at Theta. I am also a Azure MVP, specialized in cloud and hybrid integration. My first experience with BizTalk was about twelve years ago - I was just starting a in a new company, and my first project was to replace a consultant in a project that was about to start. "And by the way, the integration is using BizTalk Server 2006." the technical lead told me. I have never heard about BizTalk until that point. The solution evolved into the Health Connection Engine (https://hce.codeplex.com/) - awarded me my first Microsoft Partner Solution of the Year and taught me a lot about integration patterns and BizTalk Server.

Mahesh: How did you mastered BizTalk (Learning path, amount of time)?
Wagner: As I've jumped head first into that initial project, I had to learn on the job. So I had to get every single book, article and forum questions I got my hands on. I still remember being a proud owner of Professional BizTalk Server 2006 from Wrox Press and BizTalk Server 2006 Unleashed from Sams. But what really helped was a variety of blog posts and articles from community members, which are today recognized MVPs like Kent Weare, Stephen Thomas, Sandro Pereira, to name a few. I've worked on the HCE project for almost three years on and off, as Microsoft bought the IP from the original proof of concept and had plans to use it as some form of accelerator. That was a time where I was exposed to a lot of great design patterns, which solidified my view of how to design/implement BizTalk projects.

Mahesh: Which are the major projects you handled so far?
Wagner: If I had to choose three of my most interesting projects it would be:

·        The back end of a large ePrescribing implementation in Australia, where I've used the concepts of HCE of bespoke adapters talking to a central integration point (in this case a WCF service back end). This project taught me a lot about performance, security and service patterns.
·        The integration platform for a utilities company managing smart meters. This project was fully implemented in BizTalk Server 2010, and had some very interesting requirements, like processing 1.5 GB files through BizTalk Pipelines, and managing a variety of client message formats at the edges, while maintaining a core solution at its centre. I've learned a lot about canonical models, publishing/subscribing and streaming processing during this project.
·        Recently, a large implementation of cloud integration for another utilities company, this time in the gas industry. This used the latest cloud integration platform from Microsoft Azure - logic apps, API Management, Azure Functions, Service Bus - and taught me how incredible and full of possibilities this new platform can be.

Mahesh: What do you think is the most challenging part while working on Integration project?
Wagner: The most challenging for me while working with new integration projects is making new stakeholders understand the importance of integration, and how complex it can become. Usually integration ends up being an afterthought, so you are brought to the party late. That makes the work to raise the right requirements, to support your design challenging. Fortunately I started to see a trend of people understanding how important the integration side of a solution is, and starting to engage earlier. The good thing about this challenge is that most of the times, at the end of the project you got a very grateful and loyal client - creating a long and successful business relationship.

Mahesh:How do you see BizTalk compare to other integration platform?
Wagner:BizTalk excel at on-premises integration, and have a very powerful extensibility framework, which makes it arguably the most powerful on-premises integration platform in the market. But all that power and extensibility comes with a cost - it is more complex to implement than some of the other tools in the market. Without doing direct comparisons I think that there are other tools in the market that have a lower entry threshold for simpler, point to point integration. But as the complexity increases, it start to rely more and more in bespoke or external components. BizTalk can handle pretty much anything that can be throw at it from an integration point of view. And now with the latest version, it becomes - together with Logic Apps - the perfect platform for hybrid integration solutions.

Mahesh:What as per you is must to know to become an Integration(BizTalk) Expert?
Wagner:The two things that I had to learn before becoming proficient with BizTalk were:

·        Learn to change my thinking from designing procedural processes to workflows - really understanding the concepts of publishing/subscribing patterns and how that make your workflows generic enough to expand with time without having to recreate a whole solution.
·        From a coding point of view, XPATH and XSLT were the two things that gave me an advantage. I learned XSLT and XPATH before getting into BizTalk and that was definitely an advantage.

Mahesh:What are your thoughts on forums, blogs and articles etc.?
Wagner:Those resources are invaluable! I am always impressed with the amount of energy and comradery I find in the Integration community. I can't remember how many times my answers/frustrations were answered in a blog of a forum post, and that is why I try to pay it back being as active in the community as I can - writing blog posts, being active in the forums (I'm being particularly active in the Logic Apps forums, which is one of the topics I'm dedicating some time at the moment) and presenting or helping with the organization of local events. On that note it is quite cool to see that this integration community today also extends to the professional integration product group. The whole team is quite active in twitter, posts, forums, conferences. This creates a unique opportunity to learn from the source for everyone, not just a select few that have the right contacts.

Mahesh:Your suggestion to a newcomers? What should be approach to get sound knowledge in BizTalk?
Wagner:Start understanding the basic concepts. When I first started with BizTalk, my first thought was to create an orchestration, without understanding that there is a lot of preparation work you need to do before orchestrating anything. Once you understand those concepts, try to get some kind of integration work you did before and try to replicate it using those concepts in BizTalk. So you can start to draw parallels from something you are familiar with and this new technology.

By the time you are familiar with the basics, I would say that the next steps in the natural progression would be:

·        Learn more about XML/XPATH/XSLT
·        Understand more about the main integration patterns - canonical model, publishing/subscribing, content based routing are some that comes to my mind straight away.
·        Understand how you could implement those patterns in BizTalk. You will see that this will help solve a lot of recurring problems.

This might be a lot of work to learn on your own - so if you can, try to get someone to mentor you. Try to find opportunities to work with people that you know have a good knowledge of BizTalk (or integration in general) and learn as much as possible from those opportunities. And also remember that Microsoft Integration have arguably the best peer community in the world. Don't be afraid to reach out to the community members with questions and advice if you get stuck

Mahesh:There are many tools from community which support BizTalk in some or the other way(like BTDF, Bizunit etc), what do you say about it?
Wagner:Again it shows the breadth and strength of this community. BTDF is a great example of getting real life gap and turning it into something that became pretty much the standard. Community tools are quite important on a product that have a life-cycle of every couple of years. Until now we wouldn't expect to see anything new from BizTalk in-between releases, so it was up to the community to plug those gaps. This has been alleviated by the Feature Pack releases, but I can still see real value in creating tools and components to address existing problems, in a faster, or some times more pragmatic way.

Which ones you would recommend? Why? No Microsoft support available?

As I've said, we are using BTDF in pretty much every project we deploy today. We also used from the beginning the ESB Toolkit Exception Management Framework (which later was picked up by Microsoft), and we also forked a series of pipeline components like the archiver, context property writer, etc that we use on a regular basis.

Mahesh:What are your thoughts around BizTalk certification?
Wagner:As any other certification BizTalk certifications (and official training) have its merits. My approach to certification is to sit them after I am working on a technology for some time, so I can prove to myself that I am proficient on it. Unfortunately BizTalk server certification has been discontinued during the black hole that was the period between 2012 and 2015 - the same time where we didn't had any roadmap or compelling strategy for BizTalk and Integration in general for BizTalk Server. Hopefully we will see a revamped Enterprise Integration certification path in the future.

Mahesh:What is the future of BizTalk?
Wagner:Today, more than ever, I can see integration scenarios moving from either on-premises only or cloud only to hybrid scenarios. In that scenario I can still see BizTalk in a prominent place as the on-premises integration platform and the gateway between on-premises and cloud integration artefacts. The picture that Jim Harrer painted in the Integrate 2016 keynote, and confirmed a year later during the Integrate 2017 keynote is where I see BizTalk (and MS Integration in general) going, with a very strong partnership between BizTalk Server and Logic Apps - each one feeding from each other's strengths to create value through integration. BizTalk with its mature on-premises adapters ecosystem and reliable platform and Logic Apps with the always growing SaaS and cloud resources adapters and the ability to stand up cloud integration workflows in hours instead of days.

Mahesh:Any thoughts on cloud?
Wagner:As you can see by my previous answer I think the cloud will be an ongoing part of our life as integration professionals. We will be more and more involved in scenarios where knowledge about cloud and cloud integration technologies will be required to match our on-premises knowledge. Understanding when to use each one and when combine both will be crucial in the future.

Mahesh:What motivates you to do the community work?
Wagner:Three things are the my main drivers to do community work:

The first one and probably the most important, is my wish to pay back to the integration community  - I've came from no BizTalk knowledge to be considered an "expert" thanks to the amount of information that I could find along the way to help me solve problems/road blocks. And the lion share of that was from community content.

The second driver is the ability to learn new technologies, or at least try new things that I wouldn't have access in my day job. The forum questions that are more rewarding from me, for example, are the ones where I didn't have the answer from the tip of my tongue - where I had to go to back to the tools and try new things, and go through a journey with someone that I didn't even met in person, most of the time - bouncing ideas back and forth makes me a better professional, so it is a win-win situation.

And finally, I miss the classroom! I used to be a teacher before becoming a developer full time, so I miss the ability to be in front of a class and show things I've learned and engage, discuss and learn from the audience as much as I'm teaching. Presenting a various events gives me the ability to step into that role again for 30-60 minutes at a time. This gives me the opportunity to mix two of my passions – my work as integration architect/developer and teaching.

Mahesh:As per the Roadmap provided by Microsoft, Logic Apps will be able to run on-premise in addition to AZURE. Do you think Azure Stack Logic Apps on prem will supersede BizTalk Server?
Wagner:I think that BizTalk and Logic Apps will still work hand in hand. Here are some reasons:

·        Adapter ecosystems - both technologies have a complimentary set of adapters (or connectors). Apart from a core which is kind of shared by both, Logic Apps is geared towards SaaS applications where BizTalk Server has been historically geared towards Enterprise back end systems and Line of Business Systems. Having both available means that you can use the best of both worlds.
·        Some of the integration technologies for back end systems would make adapters for Logic Apps very complex.
·        Logic Apps is just one of the components of an Enterprise Level integration platform. It excel at orchestrating APIs, and taps on other technologies like Service Bus, API Management and Azure Functions to complete the platform. So again, using Logic Apps with BizTalk would make sense, as some of those functions, in special the role of Service Bus, is core to BizTalk.

Mahesh: From my perspective, Microsoft keeps coming up with Overlapping technologies like recent ones MABS, Microsoft Flow and Logic Apps, in some situation it gets puzzling. What you say?
Wagner:Microsoft took some time to consolidate its strategy around Hybrid Integration. Between 2015 and 2016 that picture was quite confusing, but by the time that Logic Apps went GA that picture started to consolidate. I see a good range of complementary technologies now, instead of competing. I see the spectrum as:

·        Microsoft Flow - aimed at the "citizen integrator" - that super user that needs to automate day to day processes, and would try to create that himself using office automation, some code or would have to wait in line until the development team would be able to help him.
·        Logic Apps - aimed at both application and integration developers - those who need to integrate systems in the cloud as part of their day jobs, sometimes with a roundtrip on-premises.
·        BizTalk - aimed at the integration developer, which need to integrate on-premises systems in a mature and scalable way.

The great thing about those products is that they can actually work together to create very powerful hybrid scenarios. And I didn't even include the other cloud integration components in that picture.

But I agree that exactly because that vision is still relatively new, we need more and more examples, documentation, patterns to help existing and new customers to understand how to take the most out of those tools.

Mahesh: Do you think BizTalk in cloud (IAAS) is accepted over BizTalk on Prem? Which one you prefer - what pros and cons you see?
Wagner: I think that both have their value. I have clients today running BizTalk 2016 using the IaaS model, and it worked for them because they didn't have to upgrade their existing virtualization platform on-premises to support BizTalk pre-requisites. Other clients wants to have BizTalk production environment on-prem, but need the ability to stand up, or turn on development and test environment at will. So having the ability to do so is quite useful.

Mahesh:Being MVP, do you think responsibilities get added?
Wagner:I think being recognized as a leader in your area of expertise makes you think even more how to help and provide good information to the community. So in this sense you feel you have extra responsibility.

Mahesh: How do you see the step of Microsoft releasing Feature Pack for BizTalk?
Wagner:This is a fantastic way for Microsoft to finish once and for all the myth that BizTalk was dead. Not only the product is very mature in its 10th edition and its embracing the hybrid integration paradigm with BizTalk Server 2016, it is now start to gear up towards a devops mentality with continuous integration pipeline being integrated and full scale analytics with Applications Insight Support.

Mahesh:Microsoft is keeping on adding many things under Hybrid Integration, where should be Integration developers be focussing on?
Wagner:In my talk in Integrate 2017, I've suggested that BizTalk Server 2016 should be your on-premises gateway to hybrid integration space as it have all the capabilities required to integrate with the main cloud integration components, like Logic Apps, Service Bus - including WCF Relay, Azure Functions and API Management.

If you can't or is not prepared to use BizTalk Server 2016 on-premises, then I would concentrate first on the pairing of Logic Apps with on-premises data gateway whenever I can, and use Azure Relays to close any of the gaps.

Mahesh:Do you think the cost of using cloud services(Logic App, Service Bus etc) will be more than having infrastructure on prem.(Where transactions are in millions or more)?
Wagner:My gut feeling is that the cloud services will still be cheaper than maintaining your own infrastructure, when you add the other costs to the equation, like outages for maintenance, engineering costs to maintain the platform and support, issues generated or revenue loss due to unexpected peaks. So even if consumption costs would equal to licensing - which I don't think it is the case today or would be the case in the future - there are extra costs associated to maintaining your own infrastructure that is usually forgotten when doing the maths.

Thanks a lot Wagner for taking out time and sharing your insights,experiences, this will surely benefit many !!!

Feel Free to ask questions to Wagner in the comments!!!!!!!! 






Related Post:



Post a Comment

If you have any suggestions or questions or want to share something then please drop a comment

Previous Post Next Post