Showing posts with label EVENTS. Show all posts
Showing posts with label EVENTS. Show all posts

Tuesday, 1 July 2014

Prevent events executing multiple times

Case
I have a log task in an OnPreExecute and OnPostExecute event handler, but it executes multiple times. Why is that and how can I prevent it?

Solution
This is because all tasks and containers fire events and these events are propagated to their parent container and then to their parent container and so on. This means that if you have a package with a Sequence Container with and with an Execute SQL Task in it, that each of them fires events. Let's test that.

For testing purposes I added an Execute SQL Task in each event handler with an insert query to show which events are fired. Each task inserts the name of the event and the value of the System Variable SourceName in a log table with an identity column LogId.
Log all events, executable is package



















When you run the package you can see that for example the OnPreExecute event has three records:
1) the Package
2) the Sequence Container
3) the Execute SQL Task
Event handlers executing multiple times



















The trick
The trick to execute the Execute SQL Task in the event handler(s) only once, is to check whether the source of the event is the package and not one of it's children (containers/tasks).

Add a dummy Script Task or an empty Sequence Container in front of the Execute SQL Task and add a Precedence Constrain expression between them: @[System::SourceName] ==  @[System::PackageName]
Expression to filter events that are not from the package




















Sunday, 1 June 2014

Retry a Task on failure

Case
I have a webservice task that sometimes fails (due an external cause) and I want to retry it a couple of times before failing the entire package. How do I create a retry construction for a task?

Solution
A solution could be to add a FOR LOOP around the task you want to retry. This works for all tasks, not just the webservice task.
A For Loop Container for retrying a task















1) Variables
We need a couple of variables for the For Loop Container.
- RetryMax: an integer indicating the maximum number of attempts
- RetryCounter: an integer to keep track of the number of attempts
- QuitForLoop: a boolean for quiting loop before reaching the maximum number of attempts
- RetryPause: an integer for storing the number of pause seconds before retry

Variables for the FOR LOOP























2) For Loop
Add a For Loop Container and move the task you want to retry inside the container. Edit the For Loop Container and set the following properties:

InitExpression: @[User::RetryCounter] = 1
This will give the RetryCounter an initial value.

EvalExpression: @[User::RetryCounter] <= @[User::RetryMax] &&
                             @[User::QuitForLoop] == false
This will indicate when the For Loop Container stops looping. In this case reaching the MaxRetry or when the Boolean variable is filled with True.

AssignExpression: @[User::RetryCounter] = @[User::RetryCounter] + 1
This will increase the RetryCounter.
The FOR LOOP expressions
















3a) Pause Task
For this example I will use a Script Task for waiting a couple of seconds/minutes before a retry . If you don't like scripting there are alternatives for a pause. Add a Script Task, give it a useful name and connect it to the task you want to retry. Make sure the Script Task only executes on error by setting the Constraint Option to Failure.
Pause after failure
















3b) The Script
Edit the script Task and add the 3 integer variables as read-only variables. Then hit the edit button and copy the contents of my main method to your main method. The example code is in C#. For a VB.Net version you can use this conversion tool.
Read-only variables

















// C# Code
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;

namespace ST_1c11fe9f84ce4662bdc37ece5316e04d
{
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {

        public void Main()
        {
            if (Dts.Variables["RetryCounter"].Value.ToString() != Dts.Variables["RetryMax"].Value.ToString())
            {
                // Fire warning message that the previous task failed
                Dts.Events.FireWarning(0, "Wait", "Attempt " + Dts.Variables["RetryCounter"].Value.ToString() + " of " + Dts.Variables["RetryMax"].Value.ToString() + " failed. Retry in " + Dts.Variables["RetryPause"].Value.ToString() + " seconds.", string.Empty, 0);

                // Wait x seconds
                System.Threading.Thread.Sleep(Convert.ToInt32(Dts.Variables["RetryPause"].Value) * 1000);

                // Succeed Script Task and continue loop
                Dts.TaskResult = (int)ScriptResults.Success;
            }
            else
            {
                // Max retry has been reached. Log, fail and quit
                Dts.Events.FireError(0, "Wait", "Attempt " + Dts.Variables["RetryCounter"].Value.ToString() + " of " + Dts.Variables["RetryMax"].Value.ToString() + " failed. No more retries.", string.Empty, 0);

                // Fail Script Task and quit loop/package
                Dts.TaskResult = (int)ScriptResults.Failure;
            }
        }

        #region ScriptResults declaration
        /// 
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        /// 
        /// This code was generated automatically.
        /// 
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion
    }
}


4a) Quit loop on success
To quit the loop when your task executes successfully we are setting the Boolean variable QuitForLoop to true. I'm using a Script Task for it, but you could also use an Expression Task if you're using 2012 and above or a custom Expression Task for 2008.

Add a Script Task, give it a useful name and connect it to the task you want to retry. Edit the Script Task and add the integer variables RetryCounter and RetryMax as read-only variables and the Boolean variable QuitForLoop as read-write variable. After this hit the Edit button and go to the next step.
read-only and read-write variables


















4b) The Script
Copy the contents of my main method to your main method. This code will set the Boolean variable to true causing the loop to stop. The example code is in C#. For a VB.Net version you can use this conversion tool.
// C# Code
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;

namespace ST_a72ebc0827b64c0f8a1083951014129c
{
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {
        public void Main()
        {
            // Fire information message that the previous task succeeded
            bool FireAgain = true;
            Dts.Events.FireInformation(0, "Succeeded", "Attempt " + Dts.Variables["User::RetryCounter"].Value.ToString() + " of " + Dts.Variables["User::RetryMax"].Value.ToString() + " succeeded. Quiting loop", string.Empty, 0, ref FireAgain);

            // Fill boolean variable with true so that the FOR LOOP EvalExpression will evaluate false and quit
            Dts.Variables["User::QuitForLoop"].Value = true;

            Dts.TaskResult = (int)ScriptResults.Success;
        }

        #region ScriptResults declaration
        /// 
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        /// 
        /// This code was generated automatically.
        /// 
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion
    }
}

5) Event handler propagate
Now the most important step! A failure on one of the tasks within a parent container (package, sequence, for loop or foreach loop) will also cause the parent container to fail, but we want to continue after an error. The trick is to add an empty OnError event handler on the task that could fail and change the value of the propagate variable .

In this example we want to ignore errors on the webservice task. Go to the event handlers. Select the webservice task as executable and select OnError as eventhandler and then create the event handler by clicking on the link in the middle of the screen: "Click here to create an 'OnError' eventhandler for executable 'WST - Call webservice'.

Next go to the variables pane and hit the Variable Grid Option button to also show system variables (SSIS 2008 has a different button). Then find the system variable Propagate and set it to false.

Last step is to add an annotation in the empty event handler to explain why it's empty.

Empty OnError event handler with propagate set to false


















Show system variables for SSIS 2008








Note: propagate will only work within a package not in a parent-child package construction, but there is a workaround available.

6) testing
In the first run the third attempt was successful and the package succeeded. In the second run all five attempts failed and so did the package.
Third attempt was successful

All five attempts failed

Tuesday, 13 May 2014

Continue Loop after error - Part II

Case
I used the propagate variable trick to continue a loop on error. That works within the package, but the parent package calling the package with the loop still fails.
Child succeeds, parent fails

















Solution
This is by design according Microsoft. The workaround they suggest is setting the DisableEventHandlers property of the Execute Package task to True. This should ignore all errors in the child package. Below here is an alternative solution to only ignore expected errors. Don't hesitate to post your own solution in the comments.

A parent package variable will be filled by the child package in case of an unexpected error. In the parent package it will be used to throw an error when it's filled.

1) Variable - Parent Package
Add a string variable in the parent package called childError.
Add string variable


























2) OnError - Child Package
Go to the child package where you used the propagate variable 'trick' and add an OnError event handler on package level. Go to Event Handlers tab. Make sure the package is selected as Executable and OnError as Event handler. Then click on the link in the middle of the page: Click here to create an 'OnError' event handler for the executable 'Child1'.
OnError event handler for package


















3a) Script Task - Child Package
Add a Script Task to the event handler and give it a suitable name. Then edit it and add the System variables ErrorCode and ErrorDescription as read only variables and type the name of the parent package in the read write box. You can't select it because it's only known at runtime.
Add the variables




















3b) The Script
Hit the edit button and copy the code of my main method to your main method. The example code is in C#. For a VB.Net version you can use this conversion tool.
// C# Code
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;

namespace ST_ace2311e5a4c4bbb98101cd54888c7c9
{
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {
        public void Main()
        {
            // Fill parent package variables with error message from this child package.
            Dts.Variables["User::ChildError"].Value = Dts.Variables["System::ErrorCode"].Value.ToString() + " - " + Dts.Variables["System::ErrorDescription"].Value.ToString();
            Dts.TaskResult = (int)ScriptResults.Success;
        }

        #region ScriptResults declaration
        /// 
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        /// 
        /// This code was generated automatically.
        /// 
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

 }
}

4) Propagate - Parent Package
Go back to the parent package and add an empty event handler on the Execute Package Task so that it won't fail if the child package has an error. Go to the event handlers. Select the Execute Package Task as executable and select OnError as eventhandler and then create the event handler by clicking on the link in the middle of the screen: "Click here to create an 'OnError' event handler for executable 'EPT - Child1'.

Next go to the variables pane and hit the Variable Grid Option button to also show system variables (SSIS 2008 has a different button). Then find the system variable Propagate and set it to false.

Last step is to add an annotation in the empty event handler to explain why it's empty.
OnError Event Handler

















Show system variables for SSIS 2008








5a) Script Task - Parent Package
Go back to the Control Flow of your parent package and add a Script Task to fire an error. Connect it to the Execute Package Task with an expression only on the precedence constraint:
@[User::ChildError] != ""
Fire Child Error























5b) The Script
Edit the Script Task and add the string variable ChildError as read-only variable. After that hit the edit button and copy the content of my main method to your main method. The example code is in C#. For a VB.Net version you can use this conversion tool.
Read-only variable























// C# Code
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;

namespace ST_7b522ca79c9f4428a233a100bfc66e6e
{
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {
        public void Main()
        {
            // Fire error with error message from child package and fail script task
            Dts.Events.FireError(0, "Child package", "Child package error: " + Dts.Variables["User::ChildError"].Value.ToString(), string.Empty, 0);
            Dts.TaskResult = (int)ScriptResults.Failure;
        }

        #region ScriptResults declaration
        /// 
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        /// 
        /// This code was generated automatically.
        /// 
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

 }
}


6) Precedence Contraint - Parent Package
If there is no unexpected error in the child package then we continue with the next task(s). Each task that is connected to the Execute Package Task with a Success Precedence Constraint should be changed to an expression: @[User::ChildError] == ""
Continue if there are no unexpected errors in the child package


























7) The result
I added a Script Task in the child package that always fails to simulate an unexpected error. On the second run there where only expected errors in the loop, so the child package was successful. In the parent package there is still a red cross, but it continues without (unexpected) errors.
The result

Continue Loop after error - Part I

Case
I want to continue my loop when in one of the iterations a task fails. It should continue with the next iteration/file. I tried changing the task properties FailParentOnFailure and MaximumErrorCount, but nothing seems to work.
Task fails, so loop fails

















Solution
There are a couple of solutions. You could set the MaximumErrorCount to a higher number on the parent container (not on the task that fails). With FailParentOnFailure property on the failing task you can override that setting and fail the parent on the first error.

The easiest/stable solution is to use an empty OnError event handler on the failing task with the system variable Propagate set to false.

1) Empty event handler
Go to the event handlers tab. Select (1) the failing task as Executable and (2) OnError as the event handler. Next (3) click on the link in the middle of the page to create the event handler: Click here to create an 'OnError' event handler for executable 'DFT - Stage files'.
Create empty event handler
















2) System variable propagate
Open the variable pane in the event handler. Click on (1) the Variable Grid Options button. Check (2) the radio button "Show system variables". Search for the Propagate variable and set (3) it to false.
System variable Propagate

























Show system variables for SSIS 2008








3) Annotation
An empty event handler could confuse other developers. Adding a simple annotation could solve that.
Add an annotation























4) The result
Now an error in the Data Flow Task won't fail the Foreach Loop Container. This solution works with all containers (Sequence, For Loop, Foreach Loop and package).
























Note: this solution wont work with parent child packages. Propagate can't be disabled from a child package to a parent package. This is by design according Microsoft. Here is a workaround for that.

Monday, 14 February 2011

Script Task and Component Logging

Case
I want to add logging to my Script Tasks and Script Components. How do I implement that?

Solution
You can raise events in the Script Task / Script Component and those events can be logged by SSIS. This article will explain the event raising.

Script Component
The Script Component has the following events:
  • FireCustomEvent
  • FireError
  • FireInformation
  • FireProgress
  • FireWarning
Fire Event in Script Component






















You can raise an error event with the following code
//C# code 
bool pbCancel = false;
this.ComponentMetaData.FireError(0, "myScriptComponent", "An error occurred.", "", 0, out pbCancel);

But you can also combine the event raising with the try catch statements from .Net:
//C# code 
try
{
    // your code
}
catch(Exception e)
{
    bool pbCancel = false;
    this.ComponentMetaData.FireError(0, "myScriptComponent", "An error occurred: " + e.Message, "", 0, out pbCancel);
}

You can even combine that with SSIS System Variables:
//C# code
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    // Declare script variable
    private string myTask;

    public override void PreExecute()
    {
        base.PreExecute();
        // Fill script variable with SSIS variable
        // Don't forget to add the system variable
        // to the ReadOnlyVariables in the Script
        // Component.
        myTask = Variables.TaskName; 
    }

    public override void Input0_ProcessInputRow(Input0Buffer Row)
    {
        try
        {
            // Throwing exception for testing
            // Add your own code here.
            throw new ArgumentNullException();
        }
        catch (Exception e)
        {
            // Raise event so it can be logged by SSIS
            bool pbCancel = false;
            this.ComponentMetaData.FireError(0, myTask, "An error occurred: " + e.Message.ToString(), "", 0, out pbCancel);
        }
    }
}


Script Task
The Script Task has the following events:
  • FireBreakpointHit
  • FireCustomEvent
  • FireError
  • FireInformation
  • FireProgress
  • FireQueryCancel
  • FireWarning  
Fire Event in Script Task






















You can combine the event raising with the try catch statements from .Net and the SSIS System Variables:
//C# code 
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;

namespace ST_c9f1ec34c2ee4dbbb0bf0b0db3f3ae58.csproj
{
    [System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {

        #region VSTA generated code
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

        public void Main()
        {
            // Don't forget to add the system variable to the ReadOnlyVariables in the Script Task
            bool fireAgain = true;
            Dts.Events.FireInformation(0, Dts.Variables["System::TaskName"].Value.ToString(), "Starting", string.Empty, 0, ref fireAgain); 
            try
            {
                // Throwing exception for testing
                // Add your own code here.
                // Dts.TaskResult = (int)ScriptResults.Success;
                Dts.Events.FireWarning(0, Dts.Variables["System::TaskName"].Value.ToString(), "About to crash", string.Empty, 0);
                throw new ArgumentNullException();
            }
            catch (Exception e)
            {
                // Raise event so it can be logged by SSIS
                Dts.Events.FireError(0, Dts.Variables["System::TaskName"].Value.ToString(), "An error occurred: " + e.Message.ToString(), "", 0);
                Dts.TaskResult = (int)ScriptResults.Failure;
            }
        }
    }
}




Related Posts Plugin for WordPress, Blogger...