Marc Clifton's Blog
Commentary on Windows software development and our insane government
.NET (5)
C# Programming (2)
Concurrent Programming (1)
Design And Architecture (7)
Entity Framework (1)
General (6)
Linq (3)
Microsoft (1)
Mono (2)
Office 2007 (3)
Politics (9)
Software Development (1)
Vista (10)
Wisdom (10)
XNA (19)
Why CAB Sucks
Items vs. Service collection
5/21/2009 10:44:06 AM

Using the Composite Application Block, you can't do dependency injection on components that are decoupled with interfaces using the Items collection.  You have to use the Service collection.  Why isn't there a single process for reflecting the properties so that both the Items and Services collection work the same way when doing dependency injection?

For example, given the following code and three separate modules (Component1, Component2, and Interfaces), this fails with a DependencyMissingException: "Could not locate dependency Interfaces.IComponent1".

Here's my startup code:

protected override void AfterShellCreated()
{
  IComponent1 comp1 = new Component1();
  RootWorkItem.Items.Add(comp1, "FirstComponent1");
  Component2 component2 = new Component2();
  RootWorkItem.Items.Add(component2);
}

Component1:

public class Component1 : IComponent1
{
}

Component2:

public class Component2 : IComponent2
{
  [ComponentDependency("FirstComponent1")]
  public IComponent1 Component1 { get; set; }
}

Interface:

public interface IComponent1
{
}

public interface IComponent2
{
 IComponent1 Component1 { get; set; }
}

Instead, you have to use the Services collection:

RootWorkItem.Services.AddNew<Component1, IComponent1>();
IComponent2 component2 = new Component2();
RootWorkItem.Items.Add(component2);

and set up a ServiceDependency attributed property:

[ServiceDependency]
public IComponent1 Component1 { get; set; }

Frankly, this makes no sense.  The resolver that determines that the object being injected is of the interface type should be the same regardless of whether it's being managed by the Items collection or the Service collection.

--Marc

Does delegate.BeginInvoke use the ThreadPool?
5/14/2009 5:51:34 PM

I wanted conclusive proof that a ThreadPool is indeed used when making an asynchronously call using a delegate's BeginInvoke method.  Indeed, it does.  The behavior of the Test thread is identical to using the ThreadPool, as illustrated by the commented out line in the code below.  Specifically:

  • By blocking the worker thread permanently, we can force the ThreadPool to exhibit the behavior in which a new worker thread does not start for approximately 500ms. 
  • By forcing the maximum number of threads to 25, the console window outputs only 0 through 24.  The final 5 queued work items do not execute because the ThreadPool has run out of available threads, since none of the worker threads exit.

Here's the test code:

using System;
using System.Threading;

namespace ThreadPoolTest
{
  class Program
  {
    public delegate void MyDlgt(object state);
    public static MyDlgt myDlgt;
    public static Mutex mutex;

    static void Main(string[] args)
    {
      mutex = new Mutex();
      mutex.WaitOne();
      ThreadPool.SetMaxThreads(25, 1000);
      myDlgt = Test;

      for (int i = 0; i < 30; i++)
      {
        // ThreadPool.QueueUserWorkItem(new WaitCallback(Test), i);
        myDlgt.BeginInvoke(i, null, null);
      }

      Console.ReadLine();
    }

    static void Test(object state)
    {
      int n = (int)state;
      Console.WriteLine(n);
      mutex.WaitOne();
    }
  }
}

Why is this useful information? Because we should understand the underlying mechanism for asynchronous method calls, as it can affect the architecture of the application if you know that the asynchronous call is using a ThreadPool.

--Marc

Recovery Service Library
Part I
5/14/2009 5:19:01 PM

Abstract

A Recovery Service is a reusable component for managing connectivity and communication errors without forcing the application to terminate a workflow process.  The Service can be used directly by an application to facilitate managing the connectivity state of a database, web service, streaming media source, etc.  The Service can also be used in conjunction with Microsoft's Workflow Foundation to help manage document-related service recovery.  Thus, the Recover Service can work with both document-centric and process-centric workflows and applications.  Utilizing an Inversion of Control (IoC) pattern, the Recovery Service can wire up recovery routines and define other recovery parameters using metadata, providing further abstraction and decoupling of the application from third party services whose connectivity cannot be guaranteed.

Introduction

A while ago, I wrote about a Tiered Error Management and Recovery Service and, the backbone to it, a Safe Method Caller.  I've found that, two years later, I want to revisit these implementations to create a true Recovery Service Library (RSL) and to rethink some of the design and subsequent implementation, especially regarding features and method interruption.

Part I describes the general problem that the Recovery Service solves and looks at several use cases.  In Part II, the requirements and features of the RSL are determined from the use cases and the architecture of the RSL is determined, along with some initial unit tests and prototype code investigations.  Part III, the final installment, will define further unit tests, complete the implementation, and then review the RSL against the requirements and features stated in Part II.

What is the Problem?

Any time a method has to call outside of its own domain (by domain, I usually mean the protected and private methods of the same class), there is the possibility of an error occurring that is not under the control of the application.  Something we can all identify with is querying a database--we use ADO.NET or some other API/framework to communicate with the database, and this leaves our application vulnerable to connectivity problems, schema changes, and so forth.  Another example is when our application communicates with a web service--again, there are potential connectivity problems, changes to the WSDL, etc.  A common thread in this problem set is that our application is connecting to another application (a database server, a web service, an RSS feed, etc.)

In many cases (for example, applications that perform automated tasks) we don't want to terminate the workflow that the application is attempting to accomplish.  Rather, we'd like to give the application the chance to try again.  By "try again", this could mean trying the same connection again after a suitable wait period, perhaps trying a different connection, perhaps requesting some alternate solution from the user.

This recovery attempt can be performed in a completely automated fashion without the user even knowing that there is a problem (however the recovery attempts should probably be logged), or with some feedback to the user, in which the user can "take control" and terminate the workflow manually.

Why is This an Important Problem to Solve?

An RSL improves the robustness of the application's design and implementation:

  • The availability of an RSL brings to the developer's consciousness the question "should I always terminate the workflow if a critical error occurs?" and provides a tool that the developer can use if the answer is "no."
  • Rather than custom coding individual sections of the application that require a recovery service (and the implicit complexities of state management), the RSL provides a common and unit-tested implementation that can be quickly leveraged for the required functionality.

An RSL improves the usability of an application:

  • The RSL gives the application a chance to succeed in the workflow by retrying connections and alternate endpoints.
  • The RSL improves the feel of the application by automating recovery and/or providing the user with feedback as to the recovery state and allowing the user to intervene (or not) in the recovery process.
  • The RSL can work in conjunction with error logging to produce coherent event logs.

An RSL improves the maintainability of the application:

  • The binding of the recovery parameters (timeouts, alternates, internal methods) can be done at runtime.
  • The RSL metadata can be reconfigured in response to changes in third party services, features and capabilities, as well as user-specific requirements.

Use Case Example

I'm going to discuss one use cases here, posting transactions to a middle tier broker.  This is a fairly complex use case from which we can derive the most important requirements of the RSL.

In this use case, the client, rather than posting transactions directly to a database, instead posts transactions to a middle tier broker.  This decouples the client from the persistence implementation--all the client needs is an acknowledgement from the broker that the transaction information was received.  Let's say that the connection with the broker fails in one of two failure modes:

  • the connection with the broker cannot be established, which is determined immediately on the connection attempt
  • the connection is up but the broker fails to respond after the client posts the transaction

Regardless of the failure mode:

  • the application should attempt to connect to a secondary, backup middle tier broker
  • upon failing that attempt, the application should persist the transaction locally
  • once the connection is re-established, post all locally persisted transactions. 

In the first failure mode (connection fails), the client should attempt to connect to the middle tier several times (configurable) after an appropriate time delay (also configurable) but without affecting the response of the application's UI.  Similarly, in the second failure case (no response after posting transactions), waiting for the broker's response before determining failure should also not affect the responsiveness of the UI.  Furthermore, the RSL should be used to interrupt or terminate the "wait for response" method.

We can determine from these minimal requirements that the RSL can assist the application in several ways:

  • Managing the transaction state
    • Connection
    • Transmitting transaction data
    • Waiting for response
    • Attempting connection to the backup middle tier
    • Locally persisting the transaction
    • Executing a background thread that continues the reconnect attempt
  • Reporting state
    • Allows the application to report the transaction state to the UI
    • Allows the application to log errors
  • Specifying information about each state
    • The method to be called
    • Whether the method is called synchronously or asynchronously
    • Number of attempts
    • Timeout between each attempt
    • Maximum time to wait for a method to return
    • Termination approach:
      • Thread interrupt
      • Thread abort
  • State Management
    • The successful completion of a method advances to the specified success state
    • The failure of a method advances to the specified failure state (after retries)
    • A method can forcibly set the next state
    • A method can forcibly terminate the entire RSL
  • Background Processing Management
    • Execute a method asynchronously
    • Interrupt or abort the thread executing the asynchronous method
  • Callbacks
    • The RSL provides a callback whenever a failure state transition occurs
    • The RSL provides a callback for a retry attempt
    • The RSL provides a callback for successful completion
    • The RSL provides a callback for failure termination

Architecture Concerns

Using an RSL decouples the workflow into discrete states.  While this relieves the application from managing state and allows the whole recovery process to be potentially expressed in metadata, making it easily customizable, it does however add additional burdens to the application in two regards:

  • managing information between states within the lifetime of the RSL process
  • managing that information in a thread-safe manner

These concerns can be addressed by adding the following requirements to the RSL:

  • Construction of the RSL includes a thread-safe object representing the data (for document-centric processes) on which the states defined in the RSL operate, for the lifetime of the RSL.  If necessary, the application can perform a deep clone of the object to guarantee that the object is completely decoupled from further manipulation by the application.
  • Consider specifying the data type as a generic parameter to the RSL class.  This might improve typing issues when passing the data object to the state's method.
  • Consider encapsulating all operations on the data as part of the class definition for the data object.  The methods defined in the data class would therefore be wired up to the RSL rather than having separate class methods that are passed the data object.

Regarding the last two points above, I feel there is some prototyping that should be done to explore the technical issues and architectural pros and cons before deciding on a which option is best or whether the RSL should be flexible to accommodate all options.

Conclusion

The RSL is very powerful concept, however it should be clear from this discussion that it can also be complex, perhaps overly complex.  Therefore, it is very important that a specific set of requirements is defined, that the requirements do not get succumb to "feature creep", and that certain technical issues be investigated before the final requirements and architecture is determined.  This will be the topic of Part II.

--Marc

Thread Abort and Thread Pools
Beware of your thread pool implementation
5/14/2009 4:23:31 PM

I just did a quick investigation on what happens when you abort a thread managed by the .NET ThreadPool.  I was concerned that aborting the thread would, for each aborted thread, reduce the number of threads available in the ThreadPool.  It appears that this is not the case. Here's my test code:

using System;
using System.Threading;

using Clifton.Threading;

namespace ThreadPoolTest
{
  class Program
  {
    static void Main(string[] args)
    {
      ThreadPool.SetMaxThreads(25, 1000);

      for (int i = 0; i < 30; i++)
      {
        ThreadPool.QueueUserWorkItem(new WaitCallback(Test), i);
        // ManagedThreadPool.QueueUserWorkItem(new WaitCallback(Test), i);
      }

      Console.ReadLine();
    }

    static void Test(object state)
    {
      int n = (int)state;
      Console.WriteLine(n);
      Thread.CurrentThread.Abort();
    }
  }
}

A few things to note about this code:

  • I'm forcing the thread pool to a maximum of 25 threads (the default on my machine is 500)
  • I attempt to queue 30 work items
  • The thread aborts itself

When using the .NET ThreadPool (using .NET 3.5), I'm pleasantly surprised that each thread executes--in other words, the console window outputs the numbers 0 through 29, not necessarily in order.

Now, notice the commented line, which instead uses a ManagedThreadPool class. This class is an alternative to the .NET ThreadPool, and was written by Stephan Stoub. A link to this code can be found here. By changing the test program to use the ManagedThreadPool:

// ThreadPool.QueueUserWorkItem(new WaitCallback(Test), i);
ManagedThreadPool.QueueUserWorkItem(new WaitCallback(Test), i);

I then observe the following behavior: the console only outputs 0-14. Keep in mind that the ManagedThreadPool only creates 15 threads, and the resulting console output clearly shows that aborting the thread kills the ability to queue additional work, because the ManagedThreadPool runs out of threads.

The problem can be fixed in Stoub's code by replacing the thread about to be terminated with a new worker thread in the ProcessQueueItems() method:

try
{
  Interlocked.Increment(ref _inUseThreads);
  callback.Callback(callback.State);
}
catch (ThreadAbortException)
{
  // Remove this thread
  _workerThreads.Remove(Thread.CurrentThread);
  // Replace this thread!
  Thread newThread = new Thread(new ThreadStart(ProcessQueuedItems)); 
  _workerThreads.Add(newThread);
  // Configure the new thread and start it
  newThread.Name = "ManagedPoolThread";
  newThread.IsBackground = true;
  newThread.Start();
}
catch
{
  // Make sure we don't throw here. Errors are not our problem.
}
finally 
{
  Interlocked.Decrement(ref _inUseThreads);
}

Once we do this, the ManagedThreadPool behaves the same as the .NET ThreadPool class.

So, the point here is, one has to be careful of the implementation of the underlying thread pool if one is expecting to have to abort or interrupt a worker thread being managed by a thread pool. In fact, the above code should also handle the ThreadInterruptException so the thread doesn't terminate when interrupted.

--Marc

Transform Vista into XP
Some useful tips
12/20/2008 1:15:26 PM

I don't really want to transform Vista into XP, but there are some things that the article mentions that I think are useful, so I'm extracting some specific stuff (redundant as it may be with other blog posts:

From TechRepublic:

Disable UAC

  1. Access the Control Panel and select Classic View. (That fancy Control Panel Home look is for pansies anyway, right?)
  2. Locate and click User Accounts.
  3. Select the Turn User Account Control on or off and respond appropriately to the UAC.
  4. Clear the Use User Account Control (UAC) to help protect your computer.
  5. Click OK.
  6. Restart Windows.

Disable Notification Balloons

  1. Access the Run dialog box, type Regedit in the Open text box and click OK.
  2. Locate the following key: HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced
  3. Click on the right-hand pane
  4. Pull down the Edit menu and select the New | DWORD (32-bit) Value command.
  5. Name the new key EnableBalloonTips.
  6. Make sure the value is set to 0.
  7. Click OK.
  8. Log off and then back on.

Disable Windows Defender

  1. Access Windows Defender on the Start | All Programs menu.
  2. Click the Tools button.
  3. Click Options in the Settings section.
  4. Clear the following check boxes:
    1. Automatically scan my computer
    2. Use real-time protection
    3. Scan the contents of archived files and folders for potential threats
    4. Use heuristics to detect potentially harmful or unwanted behavior by software that hasn’t been analyzed for risks
    5. Create a restore point before applying actions to detected items
    6. Use Windows Defender
  5. Click Save.
  6. Access the Run dialog box, type Services.msc in the Open text box, and click OK.
  7. Locate and click the Windows Defender service.
  8. Select Disabled in the Startup type drop-down list, click the Stop button, and then click OK.

Disable Automatic Defragging

  1. Access Disk Defragmenter (Start|All Programs|Accessories|System Tools).
  2. Clear the Run on a Schedule check box.
  3. Click OK.

Enable the Explorer Menu Bar

  • Access the Control Panel.
  • Click Folder Options.
  • Select the View tab.
  • Select the Always Show Menus check box.
  • Click OK.

Disable the Sidebar

  1. Right-click in the Sidebar and choose Properties.
  2. Clear the Start Sidebar when Windows Starts check box.
  3. Click OK.
  4. Right-click in the Sidebar again and choose Close Sidebar.
Workflows
12/20/2008 1:05:59 PM

Richard Abbott on Code Project made it clearer for me as to the terminology to describe workflows: sequential and state machine workflows. I was using the terms "event" and "document" respectively, but sequential and state machine are better, as they are more abstract. Note what Richard said about these two types of workflows:

Sequential is not necessarily linear or predictable and may include branches and loops and may execute in parallel on separate threads. Whereas State Machine can be typical where human interaction is to be expected using states, events and triggers and as such do not follow a simple linear path.

Malware Removal Guide
10/25/2008 2:27:07 PM

Link here: http://forums.majorgeeks.com/showthread.php?t=35407

Flash LSO's
Disabling them
10/24/2008 2:11:09 PM

I wanted to make sure that I preserved this post by Hans Dietrich on Code Project:

When I was looking around for more info on the new security alert, I learned what LSOs are. Just incredible. I can't believe they think it's ok to store this crap on my computer. It turns out that Adobe's Flash Player maintains its own cookies called Local Shared Objects. They are not cookies, so your browser has no control over them. While cookies are limited to 4KB of text, LSOs can be as large as 100KB. Cookies are controlled by your browser, but LSOs are controlled by the Flash player, using obscure, hidden settings. LSOs can be set and read by web pages, even if you can't see a Flash animation on the page. If you look, you will find sites devoted to explaining how to use LSOs to track user movements online, and store small databases on the user's computer, to eliminate the need for making a round-trip back to the web server. I have even seen a posting from a user complaining that his bank was using LSOs to store his personal information, even though there was no Flash animation on the bank's site. Here's how to stop this nonsense: by default, Flash accepts all third-party LSOs. You have to go to Adobe's Flash Player Settings Manager site On the left you will see a Table of Contents. Under that, click on Website Privacy Settings Panel. What you will see displayed is the actual management console to manage the settings on your computer. If you don't recognize a site in the list, that's not surprising - you were never asked for permission to store this crap on your computer. What I did was simply click on Delete all sites. You may want to be more selective. OK, now click on Global Storage Settings Panel in the Table of Contents. Again, you're looking at the actual Settings Manager. Now uncheck the box that says "Allow third-party Flash content to store data on your computer". This should take care of LSOs.

Obama and Public Funds
6/19/2008 10:00:35 PM

It's amusing how the Republicans go after Obama for changing his position on public funds. I say, good for him, recognizing that you sometimes have to change directions in order to meet your goals. I say it's the sign of a good leader that can realistically re-evaluate previous "promises" rather than stick to a losing strategy.

The Truth Comes Out
1/23/2008 7:14:53 PM

President Bush and his top aides publicly made 935 false statements about the security risk posed by Iraq in the two years following September 11, 2001, according to a study released Tuesday by two nonprofit journalism groups.

"In short, the Bush administration led the nation to war on the basis of erroneous information that it methodically propagated and that culminated in military action against Iraq on March 19, 2003," reads an overview of the examination, conducted by the Center for Public Integrity and its affiliated group, the Fund for Independence in Journalism.

...

The quotes in the study include an August 26, 2002, statement by Cheney to the national convention of the Veterans of Foreign Wars.

"Simply stated, there is no doubt that Saddam Hussein now has weapons of mass destruction," Cheney said. "There is no doubt he is amassing them to use against our friends, against our allies, and against us."

(from CNN)

Read more here, from The Center for Public Integrity.

Bush shouldn't be impeached, he and his cronies should be thrown in jail.

--Marc

RSS
May 2009 (4)
December 2008 (2)
October 2008 (2)
June 2008 (1)
January 2008 (4)
November 2007 (4)
September 2007 (2)
August 2007 (4)
July 2007 (37)
June 2007 (8)
May 2007 (5)
February 2007 (7)

(c) 2007 Marc Clifton
MBlog (c) 2007 Marc Clifton