moving files into folders (yyyy-mm)

$StartInPath = "c:\Anoop\test"

$DirectoryList = Get-ChildItem $StartInPath -Directory -Recurse | Where-Object {
($_.FullName) -notmatch "archived" } | Where-Object {
($_.GetFiles().Count) -ne 0}

#Write-Output $DirectoryList


foreach ($directory in $DirectoryList) {
$fileList = Get-ChildItem $directory.FullName -File
foreach ($file in $fileList){
$destinationDir = $directory.FullName + "\archived\" + $file.LastWriteTime.Date.ToString('yyyy') + "-" + $file.LastWriteTime.Date.ToString('MM')

Write-Output $destinationDir

if (!(Test-Path $destinationDir))
        {
            New-Item $destinationDir -type directory
        }
        Move-Item $file.fullname $destinationDir
}
}

Expressions to deal with SoapException object (BizTalk 2006)

• ((Microsoft.XLANGs.Core.XSDPart)(((Microsoft.BizTalk.XLANGs.BTXEngine.SoapExceptionMsg)(((Microsoft.XLANGs.Core.XlangSoapException)(_soapEx))._msg)).faultPart)).TypedValue.InnerXml

• ((Microsoft.XLANGs.Core.XlangSoapException)(((Microsoft.BizTalk.XLANGs.BTXEngine.SoapExceptionMsg)(((Microsoft.XLANGs.Core.XlangSoapException)(_soapEx))._msg)).TheException)).ErrorDescription

• ((Microsoft.XLANGs.Core.XlangSoapException)(_soapEx)).ErrorDescription

• Microsoft.BizTalk.XLANGs.BTXEngine.SoapExceptionMsg.SoapExceptionPart.PartSchema.XmlContent

XSLT Template to split a string received in node and create repeating node on destination end

Use the following XSLT Call Template in scripting functoid to split a ; (semi-colon) separated string and create a repeating structure on the destination end using the values retrieved from split

<xsl:template match="ns0:QI_Notifications/ns0:QICountry__c/text()" name="split" xmlns:ns0="http://soap.sforce.com/2005/09/outbound" xmlns:ns1="http://webservice.mks.com/2009/Integrity/schema">
<xsl:param name="pText" select="."/>
<xsl:if test="string-length($pText)">
<ns1:value>
<xsl:value-of select="substring-before(concat($pText,';'),';')"/>
</ns1:value>
<xsl:call-template name="split">
<xsl:with-param name="pText" select="substring-after($pText, ';')"/>
</xsl:call-template>
</xsl:if>
</xsl:template>

**template matches the node from source which contains semi-colon separated string

Control Back From Start Orchestration

In order to make your "Start Orchestration" work like a "Call Orchestration" shape (that is, you send the control back to the caller orchestration), you can achieve this by means of Self-Correlating port.

Say, from OrchA you start OrchB, now when OrchB completes, you want something to be done in OrchA. To make it working, 

In OrchB, create an input orchestration port parameter with following settings:

Communication Pattern = One-Way

Communication Direction = Send

For the port type settings, you can give it any operation name and specify the messagetype on request. This messagetype will be the one that will help you to send control back from OrchB to OrchA. In my case, I created a new schema under my Schemas project by the name of Notifications that would exhibit the following XML structure

<notifications><status>SUCCESS</status></notifications>

Once your port configuration is completed, this port shall appear on the port surface in the orchestration designer view. You can then add your piece of work (whatever you want to do) in OrchB and then initialize the notifications message using a message assignment shape and then send the notifications message through this port.

In OrchA, add a new port to your port surface and select choice option "Use and existing port-type". This shall show you all the available port-types listed from within all your orchestrations. Select the port-type that you created in OrchB (in previous steps). Select the binding as Direct (Self Correlating) and Communication Direction = Receive as you will be receiving notifications message on this port.

Once all set, drop a receive shape to your orchestration designer, create a message of messagetype notifications and assign that to the Message property of the receive shape.

Now whenever orchB would send a notification message, it would be appropriately received by it's caller orchA.

How to send Basic Authentication credentials in HTTP POST

You need to set the Request Headers as:

Authorization: Basic

** UsernamePassoword should be separated with a colon e.g., Username:Password and then encoded to Base64

Setting up HTTP Receive Location in 2010 & IIS7

Microsoft has changed the IIS like anything. So has the method of configuring the BizTalk HTTPReceive.dll

See the following link: http://msdn.microsoft.com/en-us/library/bb246041(BTS.10).aspx

How to convert text to Base64

There's a utility included in Fiddler that can help you convert text to any encoding format. It's the TextWizard in Fiddler that you can use to convert.

BTS.RouteDirectToTP

Message Transformation shape

I recently used a message transformation shape in one of my orchestration.

The orchestration initialized with receiving a message, which was converted to a webservice request and the response from webservice was then need to be transformed to response message.

The final response message was made up of 2 source messages (1 received in the beginning and 1 received as response from webservice)

I was required to copy the context properties of the very first message received to the newly created message (post transformation). In order to achieve the same, I followed the simple method i.e., DestinationMsg(*) = SourceMsg(*).

For some reason I later commented this line and still could find the context properties on the destination message intact. What I can assume is, it's the message transformation shape which is in some way copying the context properties even ... strange, but it is true.

Filters on Party (EDI)

Did you know that more than 12 filters applied on EDI Party Batch Creation dialog box leaves you with no filters at all ??

Seems like a possible bug, I hope Microsoft fixes the issue in 2010 Final Release.

biztalk-ing EDI

I've been lately working on an EDI implementation using BizTalk Server 2006 R2 and have enjoyed some peices of it so far. The challenge so far I've faced is with Mapping ... believe me, this isn't as simple as it looks like when you're working with EDI (using BizTalk).

There have been a number of cases with me when I try to do a map using functoids which don't seem to be happy enough to let things work. XSL definitely is a life-saver. Had Microsoft not supported xsl in the maps, I don't think it would have been any possible to do maps for EDI.

If you are struggling with maps, I would personally suggest to go through Apress Pro Mapping In BizTalk Server 2009. This is one of those books which should be by your side always when doing maps.

Screwed-up Deployment

All of us know BizTalk can be tough to handle when its in no mood to help you out understanding the problem.

That's what happened with me this week.

I was performing a deployment on my local box which included a number of schemas, orchestrations etc. To accomplish my deployment I generaly make use of custom-built MSI kit. Unknowingly, somehow I initiated the deployment of the document schema first(which was expecting a property-schema to be already available) which never completed and neither did failed. Probably another add-on I need to take care for my custom installer scripts!!

Further to this, when I saw the installer not completing itself, I clicked the cancel button hoping to rollback the whole thing but that even didn't work. Thus, I had to manually terminate the "msiexec" process which forcibly killed the installer. Soon I realised, I WAS IN A MESS !! BizTalk was out-of control :(

This all lead to a partial deployment of the assembly where it successfully was able to populate some of the tables in MgmtDb and failed for some, as the Property schema was never deployed to it.

I tried hitting the MgmtDb tables and deleting the newly added assembly records in the MgmtDb tables but was not much luck ... you see, I'm not that genius !! Simplest option was, restore the databases backed-up earlier in the day prior to the deployment activity was initiated.


Moral of the story: Always keep in mind the order of deployment or else BizTalk is all gone !!

The DR experience !!

Last weekend I was participating in the DR activity for my client. The experience was as amazing as alwasy :-). From the very first day when the plan for all this DR thing started, don't know why, but I had a feeling that what my client is planning to do will definitely work but there would definitely some problem with the SSO thing for BizTalk. And yes, that happened !!

Plan was: backup the running BizTalk Server box as a virtual server. Restore the database backups to new virtual sql servers. This was done without a flaw.

The problem came in when the servers were turned on, in the order of SQL boxes first, then the BizTalk box. BizTalk host instances were unable to start or stop. It everytime errored saying something related to encryption/decryption and "unable to retrieve master secret". The latter was the actual problem. Yes, the master secret for BizTalk had corrupted or one would rather say "it wasn't there at all". I am still not aware why, but may be because the SQL Server boxes were not restored as existing servers' virtaul image as was the case with BizTalk box.

I resolved the problem by installing the ENTSSO Tools/Services on the SQL Server box, which was actually the ONE (master secret server). Installation is never a tough job with Micrsoft products but what amazed me was configuring BizTalk ENTSSO. I was a bit confused at this point of time. Should we be creating a new SSO Master Secret or Join the existing one? Giving a peice of load to my brain, I realised, JOINing to the existing group would work since I already had the SSO database hosted (this was also restored using the existing backups). But me still fighting with the contradicting statements going on within me, thought of why not restore the whole config from where the backups have been restored ??

This was the hit !! It worked !! I simply imported the BizTalk configuration of the environment into DR, which backed my statement of JOINing the SQL box to existing SSO group.

Next was nothing but restoring the master secret file which we had already backed up earlier safely :) using ssoconfig -restoreconfig

Quick Handy SQL Script to get the instance count

select sc.nvcName "Service Name", [Status]=CASE
WHEN i.nState IN(1) THEN 'Ready To Run'
WHEN i.nState IN(2) THEN 'Active'
WHEN i.nState IN(4) THEN 'Suspended (Resumable)'
WHEN i.nState IN(8) THEN 'Dehydrated'
WHEN i.nState IN(16) THEN 'Completed With Discarded Messages'
WHEN i.nState IN(32) THEN 'Suspended (Not Resumable)'
WHEN i.nState IN(64) THEN 'In Breakpoint'
END, count(i.uidInstanceID) "MsgCount" from [Instances] AS i with(nolock)
JOIN [ServiceClasses] AS sc on sc.uidServiceClassID = i.uidClassIDwhere i.uidServiceID in
(select distinct uidServiceID from [Services] with(nolock))
OR i.uidServiceID not in
(select distinct uidServiceID from [Services] with(nolock))
group by sc.uidServiceClassID, sc.nvcName, i.nState
This might not be optimized AT ALL !!

Message receive in BizTalk

A few days back, I was wondering what exactly could happen to pull in the message to the MessageBoxDb ... though I had a fair knowledge on this but I was looking for something concrete. I asked a friend of mine and he passed on this to me.


Each adapter has its own instance of a TransportProxy object that it uses to interact with the Messaging Engine. Adapters perform work against the Messaging Engine in batches, which are processed atomically. A batch is a collection of operations such as SubmitMessage, SuspendMessage, or DeleteMessage. The following is the sequence of events for the scenario where an adapter submits a message to the Messaging Engine:

1. The adapter creates a new message and connects the data stream to the message.
2. The adapter gets a new batch from the Messaging Engine.
3. The adapter adds the message to the batch to be submitted.
4. The batch is committed and queued up on the Messaging Engine thread pool.
5. The Messaging Engine thread pool starts processing the new batch.
6. The message is processed in the receive pipeline.
7. The receive pipeline produces zero or more messages. Pipelines can consume messages providing they do not return any errors. Receive pipelines can produce more than one message; typically this happens when the dissassembler component disassembles a single interchange into many messages. Typically the receive pipeline normalizes the submitted message into XML.
8. The message(s) produced by the pipeline will be processed in the mapper if mapping is configured.
9. The message(s) are published to the Message Agent or to the MessageBox database.
10. The Messaging Engine calls back the adapter to notify it of the outcome of the batch of work.

R2 Upgrade experience

Last week, I was performing an upgrade on existing BizTalk Server 2006 to R2. Phew... Though this was pretty easy...but I had to struggle a lot with it. The problem came in when I ran the installer and it showed only MgmtDb for performing upgrade. I checked out the Setup logfile and found that installer was unable to resolve the names for MsgBoxDb and DTADb (as I am not using BAM, BAS, Rule Engine & HWS etc..). I then went off with upgrading this onto a local system and noted down all the steps carefully, where the upgrade (as told by most of the people) was seamless. After banging my head with the issue, I just went across and checked the name for BizTalk Server Group in the Admin console and found it to be a custom one. This was the "Catch" :0) I changed this to "BizTalk Group" which was itself updated into adm_Group table in the mgmtdb and ran the installer...and "YES" .. the installer was able to resolve the db names for MsgBoxDb and DTADb this time :-).

Suspended Message Writer for BizTalk

OK ... so finally after a long wait ... I'm here with the ideas of the code... Sorry, but I can't paste it all out here.

Writing suspended messages to disk into the folder name of its own application is something that can be easily achieved by using WMI classes and some peices of .NET programming. I know it isn't sounding that great ... but beleive me ... I had to struggle coz I never got a chance to look at this view of BizTalk ... --the WMI classes under. (root\MicrosoftBizTalkServer)

My peice of code also uses a lot of SQL queries and # tables. This was used to club up the Service Names (application names) and get their GUIDs. I then relate the suspended instances with their owner services. Once this is all available, you can now write down the peices of code to fetch the Application name for each service instance that is suspended, create the folder to some location if not already existing, and save the messages to disk.

The idea may not be that great ... but I hope it would be quite some helpful for a few out there.

FTP Gauranteed Delivery

The BizTalk Server FTP adapter offers a 'guaranteed delivery' feature, the details of which are not documented by Microsoft. The mechanism is very straightforward, however, and involves the use of a temporary folder. Files are staged by BizTalk in this folder before being moved to the destination folder. BizTalk supports this for both Send and Receive handlers. To use this feature, simply set the "Temporary Folder" property of the FTP adapter. For Send ports, this will be the name of an FTP folder on the server, rather than a local folder.
The main advantage of using this feature is that retries can be performed in the event of a transport failure, with multiple copies of the same file being sent to the temporary folder. However, at most, only one copy of the file will ever be moved to the destination folder or receive location. The use of staging eliminates the possibility (in theory) of the FTP server (Send port) or BizTalk (Receive port) receiving multiple copies of the same file. Another advantage is that you can ensure that a file is received in entirety, or not at all. FTP allows files to be partially transferred, and supports transfer recovery where a partially completed transfer can be re-started from the point of failure (FTP regets and reputs). While this is a great feature in some scenarios, it is potentially a killer in BizTalk, because it can lead to BizTalk believing it has received or sent a file, when in fact the file has only been partially transferred. The use of a temporary staging folder allows a file to be fully transported before being transferred to its destination folder or receive location. This is particularly useful when transporting large files or transporting files over un-reliable networks.
You do need to be very careful when using the feature on a receive port. You configure the temporary folder at the host level via the BTS Administrative console. If you have multiple servers in your BizTalk group, and run multiple instances of you host across those boxes, each instance will point to the same temporary folder. In this case, things go badly wrong. Unlike the File adapter, instances of the native FTP adapter do not lock or rename files in order to prevent other instances from processing the same input. Hence, bad things happen. It is possible to have multiple instances of the same message delivered to the MessageBox, even though you are using the 'guaranteed deliver' feature. In addition, when one instance of the adapter moves a file from the temporary folder, a second instance then fails when it tries to do the same thing, and cannot find the file. Always isolate your FTP adapter in a separate host, and always run just one instance for a receive handler. One idea might be to assign your FTP-receiving host to multiple servers, but only enable one of the host instances. You could then use WMI to automate a fail-over to another instance in the event of a failure.

Content owner: http://geekswithblogs.net/cyoung/archive/2004/08/10/9524.aspx

Deadlock due to Body Tracking - BizTalk Server 2006

You receive a similar alert as this one
ISSUE:
Event Type: Error
Event Source: BizTalk Server 2006
Event Category: BizTalk Server 2006
Event ID: 6912Date: 19/12/2007
Time: 08:09:11
User: N/A
Computer: SOMESERVER
Description:The following stored procedure call failed: " { call [dbo].[bts_UpdateMsgbox_HostInstanceName]( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}". SQL Server returned error string: "Transaction (Process ID 102) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.;Duplicate key was ignored.".

RESOLUTION:
Disable body tracking!!

Problem was due to problems in BizTalk 2006 when you have multiple tracking on the same message. E.g. Before and After without changing the message (Pass-through) and in multiple place like when sending to many send ports/send port groups and tracking in all of them or tracking pipelines and schemas as well. Only tracking the same message in one place solved the problem. This was not fixed due to too complex and not happening too often

Routing Failures

http://biztalkhotrod.com/RoutingFailures.aspx