More flexible polling/scheduling support for receivers

Would be nice if the polling strategy could work hand in hand with ScheduledExecutor. It is possible to dynamically reschedule a Callable (I have code for that) so the initially fixed sleep time should not be a problem.

That would be a useful improvement: I have a JDBCConnector that needs to be polling the table only during some time window.
In this moment I've found just 2 ways:

• implement a JDBCComponent that receives events from an inbound Quartz endpoint (but it duplicates the code and the function of the JDBCConnector)
• configure a Component with an inbound JDBC endpoint, initially in status "stopped". Then I had to configure two other component with an inbound Quartz endpoint that start and stop the first component at the correct hours.

With an embedded polling/scheduling for receivers, I'd just need to set a property of my JDBCConnector..

I've submitted a patch that implements the concept of a polling strategy for the AbstractPollingMessageReceiver class.

The only polling strategy provider I have is a Quartz provider using cron expressions.

After applying the patch you get somes changes to

• the mule-core,
• the mule config dtd,
• a new module calling polling, and
• and integration test case called QuartzProviderTestCase in the integration tests module.

You can configure a transport that requires cron-like polling behaviour in the mule config (e.g. polling the file system):

<connector name="fileConnector" className="org.mule.providers.file.FileConnector">
<properties>
<container-property name="pollingStrategy" reference="pollingStrategy"/>
</properties>
</connector>

This refers to a polling strategy defined as a spring bean:

<bean id="pollingStrategy"
class="org.mule.polling.CronPollingStrategy">
<property name="expression" value="0/3 * * * * ?"/>
</bean>

This example shows a cron trigger to fire every 3 seconds (hardly the average use case, but it's a valid crontab expression).

You then define a scheduling provider for the polling interface in the mule config:

<scheduler providerType="quartz" className="org.mule.polling.QuartzScheduler"/>

And that's it.

This patch is of POC quality and has only been tested with a file based receiver. I wanted to submit the basic idea before I start refining anything. Here are areas that may require some attention:

• Fully test the lifecycle of the scheduling provider
• Fully test the registration and deregistration of receivers with sechuling providers
• Register multiple receivers with the same scheduling provider
• Is the default behaviour in the AbstractPollingMessageReceiver class correct: if a polling strategy is defined then use it, otherwise fall back to a spinning loop?
• Persistence of jobs for fault tolerance
• Implications for clustering: what happens with federated Mule instances and do they lock globally via the scheduler?
• Passing extra parameters to the scheduler (is there a use case)?
• Can you change the config (e.g. cron tab expression) at run time?

Hi Ben, thanks a ton for the code but sad to say this approach is exactly what I wanted to avoid. There are many reasons why we cannot use "global" schedules like you used, most notably OSGi and classloading. MuleManager is going away too. The whole problem is really much more difficult than it seems.
pollingStrategy should not be on the connector but rather per-endpoint (receiver), and it must also take into account overruns and other hazards (though yes they are unlikely when you only poll once a day). In a nutshell what we should not do under any circumstances is duplicate the actual functionality of the per-connector ScheduledExecutor. Rather we should trigger a non-scheduled one-shot execution by simply sending a message (calling a method?) to the connector and/or receiver, which can then submit itself for execution. The triggering message might well be generated by Quartz.
With all that I think we might be able to simply reuse the existing Quartz provider and send events directly to the connector or endpoint. Remember that endpoints can now receive().

Btw Ben, I hope you're not too disappointed about my comment; when we discussed this first back in December, most of the ScheduledExecutor stuff was not yet in place and many small but important things were added since then. When I realized just how complicated the cron-based triggers really are I put the problem on hold and concentrated on getting the regular (periodic) polling right first.

Hi Holger, no worries, I can see where you're coming from. As an outsider, its difficult to judge the actual status quo of the Mule code base and what is the cutting edge, so therefore I just went for a solution that works now with the current code. At our company, we developed a solution that solves the cron problem on the UMO level using a buffering outbound router as a countdown latch, but I wanted to solve the problem more wholistically (but sadly didn't . Is there anything one can look at to maybe help out with solving this problem, or are there simply too many things in flux at the moment?

Having said that, I didn't know that endpoints can now receive(), so I might look into that.......

Holger, does this still apply after all the changes that were made to the scheduling/polling?

Yes, this is about something different (though related). By default a PollingMessageReceiver either polls periodically or not; the idea is to make this behaviour pluggable to allow polling e.g. once-a-day. See the comment above about triggering this via the quartz "transport".
Essentially this is one of those "would be nice" features that are a lot more difficult than they seem

Btw this qualifies as "new feature" and could easily be transferred into 2.x.

The quartz support in Mule 2.0 can be used to achieve the behaviours described here. http://mule.mulesource.org/display/MULE2USER/Quartz+Transport (Check out the Endpoint Polling Job). I agree with Holger's assessment about how scheduling could be integrated into Message Receivers. However, since we already have the functionality, I don't see any point progressing with this.

Ben, thank you for you contributions here.