WCF polling adapter

This post shows how to Consume a BizTalk WCF-Custom adapter with sqlbindings to create a polling mechanism from a SQL table.

A bit about polling.

In general I am a big fan of polling from SQL, compare to having a receive port within a listen shape and a delay, which will cause dehydration. In cases of a short time correlation when the response is expected in a fairly short time a listen shape is a good solution especially if we might not be notified of an exception happening later on or when implementing scatter gather patterns, but if waiting time can exceed minutes then “waiting in the DB”, to my opinion, is the correct approach.

I did come upon solutions with messages that could wait for a correlation response from an external system using a delay shape for month, which resulted in thousands of dehydrated messages, that stuffs up the message box, it’s hard to keep track of the messages and resolve failure scenarios, not to mention the overhead on the entire system.

The benefit of waiting in the DB and polling when a state was reached reduces message box overhead and hands a lot of control over the message in the sense of knowing what state a process is in, having the ability to store information from the message in additional columns (assuming for example the message is serialized to an XML column and key fields are saved in additional columns) and the obvious ability to query those fields, being able to initiate polling only when several conditions are met(for example values or rows form other tables), and easy to amend rules having the polling logic in a stored procedure.

There are several polling techniques available in the 2009 WCF-custom Adapter with SQL bindings as Polling, TypedPolling, XMLPolling and notification can be found here.

In this post I will show polling using the XMLPolling type.

A look in to the stored procedure:

      --how many messages in each batch
      declare @BatchSize int = 5;
      declare @tblIDs table (BatchID int)
      declare @BatchCount int = 0;

    insert into @tblIDs
     select top(@BatchSize)  BatchID from Batch WITH(NOLOCK) where Operation = @Operation and Processed = 0

      IF (select count(BatchID) from @tblIDs) > 0
      BEGIN
            update Batch set Processed = 1 where BatchID in(select BatchID from @tblIDs)
            set @BatchCount = (select count(BatchID) from @tblIDs)
      END

      declare @xVar XML
      SET @xVar =
      (
            select [xml] from Batch where BatchID in(select BatchID from @tblIDs)
            FOR XML AUTO
      );

      WITH XMLNAMESPACES ('http://BizTalk/BatchsMessages' as ns1)
       Select @Operation as Operation, @BatchCount as BatchCount ,@xVar as 'ns1:xmlMessage'
      FOR XML RAW('ns1:BatchsMessages')

The schema:

Use the xmlschema command to generate an XSD, use this XSD schema in the BizTalk project as the polling schema.

You need to change the import schemaLocation=”sqltypes.xsd” in the generated XSD to point to a local XSD schema, you can get it here.

Create a schema Envelope point the body XPath to the root element as shown below:

clip_image001

The properties of the schema and the root element should look something like this.

clip_image001[7] clip_image001[9]

PipeLine:

Create a receive pipeline and add a dissembler component, add the pooling schema to the Document schemas (collection) and the envelope schema to the Envelope schema (collection).

This should strap the wrapper element the adapter will add to the message.

Orchestration:

In the orchestration create an activate receive port specified to receive a messages of the pooling schema type.

clip_image001[12]

The port message type should be set to the polling schema type.

clip_image001[14]

Physical port binding:

Create a WCF-Custom receive location with sqlbinding as shown bellow.

clip_image001[16]

The root node and namespace must match the ones on the envelope schema, the pipeline will strip off that root element passing in to the orchestration the polling schema.

Attach the pipeline to the receive location.

Note you must have some select statement or a SP call in the polledDataAvailableStatement property, should be some sort of count, if the result from that statement is >0 a call is made to the next statement ”pollingStatement” (tip: you can initiate polling by specifying “select 1”).

You are good to go .

A walk through how to create XMLPolling WCF custom with SQL binding can be found here.

Advertisements

Monitoring BizTalk Performance Counters for throttling

In occasions when a BizTalk server has a big load to process often we will notice every thing slows down and it seems BizTalk is not processing any more messages.

This happens when a throttling state was achieved, and BizTalk will stop processing messages, In my case the Message delivery throttling state had the value of 3 meaning “Throttling due to high in-process message count” we had too many messages going in, allowing the host to use more memory(see below) solved the problem for the amount of messages that where input to BizTalk.

To monitor BizTalk throttling states we can use some performance counters, to add the counters:

go to:  Performance monitor -> BizTalk:MessageAgent

The description for each of the performance counters can be found here msdn.

Tip: on 64 bit machines we can allow the host to use up to 50% of the memory on the server that will allow BizTalk the ability to have a higher throughput.

On the host configuration click advanced -> throttling trash hold -> Process memory usage (on 64 bit can increase to 50%)

Using the Tuning Advisor to to improve SQL performance

This post is about how to use the DataBase Engine Tuning Advisor.

Performance is a key issue in solutions and performance has very much to do with calling SQL procedures and avoiding deadlocks so tuning up SP can significantly boost performance.

SQL provides the DataBase Engine Tuning Advisor which is a tool to help indexing SQL tables.

In order to evaluate the DB performance we need to have a sample to tune on, this is where the SQL Sever profiler comes in.

How to create a Trace file:

1. On the SQL management studio go to tools -> SQL Profiler.

2. Start a new trace make sure you specify a file to save to and use the Tuning template

clip_image0018_thumb4[10]

3. Select the Events Selection tab and click on Column Filters to bring up the Edit Filter window, now specify the name of the DB to trace

clip_image002_thumb6

4. Press run to start tracing, now is the point where you need to start using your application, in my case pushing messages in to the BizTalk server and the more the better try to cover as much scenarios as possible especially scenarios that interact with SQL Server so the trace can have good statistics.

clip_image003_thumb2

5. After letting the systems run for some time (In my case I let the system run for a couple of minutes)  we can stop the trace and now its time to use the tuning advisor, From the SQL management studio/or the profiler itself go to tools -> DataBase Engine Tuning Advisor.

6. Create a new session specify the trace file created in the Profiler, select the workload database which is you DB and select the same DB to tune(I selected all tables) then press the Start Analysis button. should look like the following

clip_image004_thumb6

7. When the analysis is finished we get a recommendation of which tables to index and actually the script itself, notice the estimated improvement percentage on the top left and bellow are some other screen shots of the report that was generated

clip_image005_thumb2

The reports, other reports are available in the select report drop down

clip_image006_thumb2

After creating the indexes and running the tuning advisor again I managed to get 0% estimated improvement, Note before creating the indexes one of the SP had occasional Dead Locks, after indexing that never happened again.

A good reason to use the tuning Advisor is to sort out Dead Lock errors (in my case I over indexed several columns which can actually decrease performance).

Schema validation component

In my latest project we had a requirement to validate schemas, In several locations within the process flow schemas needed validation.

Now I know validation can be performed in an xml pipeline, but then again when looking in to the pipeline code using reflection validation happens using an XmlReader against the XSD schema, there are several benefits in writing a custom component.

The XmlReader takes in its constructor an XmlReaderSettings component, the XmlReaderSettings exposes an event we can subscribe to which will trigger if the schema validation fails, This allows to create an ESB fault message(we where using the ESB Exception management portal) without running within a try catch block, which is more preferment.

The XmlReaderSettings uses the XmlSchemaSet object to store the schemas it will validate against which allows the ability to cache the schemas and thus improve performance. Note that XmlSchemaSet is guaranteed to be thread safe only when used as a static field(singleton), all schemas that need validation must be loaded in to the XmlSchemaSet in the static constructor and thus create a caching effect.

The code is available here SchemaValidation.cs

Enumerating context properties

Enumerating the context properties of a BizTalk message can be useful when trying to solve an error in an orchestration.

I wrote the following method to our tracing component that will loop through the context properties of an XLANGMessage.

because the context properties are hidden within the XMessage object in the Microsoft.XLANGs.Core name space, I had to use reflection to unwrap  XMessage in order to expose the GetContextProperties() method.

Important:

This method uses the internal method “Unwrap” in the XMessage object that is NOT intended to be used by user code DO NOT USE this method in a production environment, I use it only for testing purposes and remove the call to the method when finished.

public void WriteMessageProperties(XLANGMessage message)
{
    try
    {
        StringBuilder sb = new StringBuilder();
        sb.Append(Environment.NewLine);
        if (message is Microsoft.XLANGs.Core.MessageWrapperForUserCode)
        {
            Microsoft.XLANGs.Core..MessageWrapperForUserCode mwu = (Microsoft.XLANGs.Core..MessageWrapperForUserCode)message;
            Microsoft.XLANGs.Core.XMessage xmessage = (Microsoft.XLANGs.Core.XMessage)mwu.GetType()
            .GetMethod("Unwrap", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic)
            .Invoke(mwu, null);
            if (xmessage != null)
            {
                System.Collections.Hashtable ht = xmessage.GetContextProperties();
                Microsoft.XLANGs.Core.XmlQNameTable tbl = new Microsoft.XLANGs.Core.XmlQNameTable(ht);
                foreach (System.Collections.DictionaryEntry dic in tbl)
                {
                    Microsoft.XLANGs.BaseTypes.XmlQName N = (Microsoft.XLANGs.BaseTypes.XmlQName)dic.Key;
                    sb.Append(N.Name + " : " + dic.Value + Environment.NewLine);
                }
            }
        }
        LogEntry entry = new LogEntry();
        entry.Message = sb.ToString();
        entry.Categories = new string[1] { sourceType };
        entry.Severity = System.Diagnostics.TraceEventType.Verbose;
        Logger.Write(entry);
    }
    catch (Exception ex)
    {
        Debug.WriteLine(ex.ToString(), "Error");
    }
    finally
    {
        // Call Dispose on the XLANGMessage object 
        // because the message doesn't belong to the 
        // .NET runtime - it belongs to the MessageBox database 
        message.Dispose();
    }
}

catch the close button event on a windows form

In order to catch the close button event of the form we need to override the WndProc method.

Code:

public partial class MyForm : Form
{
    public MyForm()
    {
        InitializeComponent();
    }

    protected override void WndProc(ref Message M)
    {
        const int WM_SYSCOMMAND = 0x112;
        const int SC_CLOSE = 0xf060;

        if (M.Msg == WM_SYSCOMMAND & M.WParam.ToInt32() == SC_CLOSE)
        {
            // User clicked the X - close button
            // Any code handling goes here:

            return;
        }

        base.WndProc(ref M);
    }
}

Balls bouncing and colliding on the screen

Just a small graphical application i made in my free time.
The program has a ball form which is called in a loop from the void main method.

We catch the paint event which is where the ball is drawn:

private void BallForm_Paint(object sender, PaintEventArgs e)
{
    Width = Convert.ToInt32(radios * 2); //24;
    Height = Convert.ToInt32(radios * 2); //24;

    Graphics g = e.Graphics;
    g.Clear(Color.Cyan);
    g.FillPie(new SolidBrush(col), 0, 0, Width - 1, Height - 1, 0, 360);
    g.DrawArc(new Pen(Color.Black,2), 0, 0, Width - 1, Height - 1, 0, 360);
}

Gravity:
the balls are under the force of gravity there for always pulled down, to get this effect every tick  the Y vector decreased in just a bit which simulated gravity and can be modified to have a stronger gravitational pull if increased

double gravity = 0.1;
moveY += gravity;

also the X vector is changed to simulate fraction
moveX *= 0.75;

Collision:
balls that collide with one each other are calculated to bounce away in the opposite direction of the impact, in this case some physics is needed to calculate the direction of 2 collisions using the radios of the circles equation to find if one ball entered another balls radios space

double distIn = Math.Sqrt(Math.Pow((int)x1 - (int)x2, 2) + Math.Pow((int)y1 - (int)y2, 2));
m = (y2 - y1) / (x2 - x1);
double b = y1 - (m * x1);

and the velocity of the ball which absorbs the velocity from the ball it collided with
double difX = Math.Abs(oBall.moveX – moveX);
double difY = Math.Abs(oBall.moveY – moveY);

if right clicking on a ball a small menu will open and allow some functionality like make it bounce again(which will any way happen when the ball stops movement) the tricky part is to catch the right click event exactly when the pointer is over the ball good luck 🙂

here is the source code:

Visual studio 2005: BouncingBall-SRC.zip