Faster processing using the Worker class

A Worker is a special object that you can add to your project. Code in the Worker's JobRun event runs as a separate process in a Console application to allow multiple CPU cores to be used.

When your project is Run in debug mode, Worker uses threads to simulate separate console app processes to help simplify debugging. So when debugging you will not see any benefit of using multiple cores since Xojo threads are limited to a single CPU core.

Worker can be used with Desktop projects.

Worker class

Events

Name	Description
JobCompleted(jobResult As String)	Called when JobRun completes and contains the results returned by JobRun.
JobProgressed(progressData As String)	Called by SendProgress() method to provide information about the progress of the job.
JobRequested() As String	Return something that indicates the job to process. This information is supplied as the parameter to JobRun. You should avoid passing around large chunks of data as that will not be memory efficient. If you have large data (pictures, data, databases, etc.) then pass along information on how to get the data using other means such as file access or connecting to the database and fetching what is needed.
JobRun(jobData As String) As String	When a job is completed, the JobCompleted event is called passing along the results returned from JobRun.
Error(jobData As String)	If there is an error, the job data is provided here so you can determine which job caused the error.

Properties

Name

Description

CorePercent As Integer

The the maximum number of cores to use expressed as a percent of total CPU cores.

MaximumCoreCount As Integer

The maximum number of cores to use. Regardless of the percent, this is the most CPU cores that will be used. This should be 1 or greater.

ProjectItemsToInclude As String (Design-Time)

List of Project Items to include in the Worker Console app. No classes or modules are included by default. Enter one project item per line. To include all classes and modules, enter IncludeAllClassesAndModules.

In addition you can use the Compatibility Flags to mark individual methods or properties so that they are not included in the Worker Helper. For example, to exclude a method, select the method and click the Advanced tab in its Inspector and turn off the checkboxes next to Console. This will prevent the method from being included in the Worker Helper. You may find this useful if you have classes or modules that have some methods that use desktop features (for example, FolderItem.ShowOpenFileDialog) but you don't want to refactor everything out to separate project items.

Methods

Name	Description
Start	Starts the worker by creating the separate helpers based on your core settings.
Stop	Stops the worker and all its helpers.
SendJobCompleted(jobResult As String)	Manually call JobCompleted event sending along the result in the parameter.
SendProgress(progressData As String)	Calls the JobProgressed event sending along the data in the parameter. This should only be called from the JobRun event (or methods it calls from the event).

Usage

Add a Worker to your project using Insert (menu or toolbar) and name it as necessary.
Add the events. You will need JobRequested and JobRun at a minimum. You'll probably also want JobCompleted to know when jobs are finished.
Start your worker. For example: Worker1.Start

Notes

The code in the JobRun events runs separately in each Worker Helpers (Console app). This means that this code (and any code that it calls) has to be Console-safe and cannot make use of Desktop commands or features. If you use these, you will get compile errors.

The other events all run on the desktop app.

This distinction is important. For example, if you create a property on your Worker (TestValue for example) and assign it a value in the JobRequested event (which ran in the desktop app), you will find that that property value is not set when that property is accessed from the JobRun event (which runs in the Helper console app). All data to be passed between the Worker in the desktop app and any of its Worker Helpers must use the parameters and return values of the events.

This also applies to project constants. For example, if you have a constant on your main App, that constant is not available to the Worker. Instead put the constant on the Worker itself.

Excluding project items from Worker helpers

By default, all modules and non-UI classes are copied into your Worker Helper so they can be used. However, if any of those classes refers to Desktop commands or features you will get compile errors. You can choose to exclude entire project items by listing them in the Project Items to Include property in the Inspector for the Worker. Enter one project item per line. For nested modules and classes, list each module and class separately on its own line.

In addition you can use the Compatibility Flags to mark individual methods or properties so that they are not included in the Worker Helper. For example, to exclude a method, select the method and click the Advanced tab in its Inspector and turn off the checkboxes next to Console. This will prevent the method from being included in the Worker Helper. You may find this useful if you have classes or modules that have some methods that use desktop features (for example, FolderItem.ShowOpenFileDialog) but you don't want to refactor everything out to separate project items.

Handling errors

A variety of things can cause an error, including:

A Worker Helper losing connection with its Worker in the main app. This can happen if the main app crashes, quits or is force-quit.
An unhandled exception in a Worker Helper.
A Worker losing connection with one of its Helpers. This can happen if the Worker Helper crashes or is force-quit for some reason.

When an error occurs, the Error event in the Worker is called. The parameter has details about the job that was being processed when the error occurs. When an error occurs in a Worker's Helper, the Helper app quits.

You can restart Worker Helpers by calling the Worker.Start method. This method will restart any Worker instances that have stopped normally or because of an error and they will call JobRequested. If you find that a Worker Helper is failing repeatedly for the same JobData then you should probably remove that JobData (and perhaps log that it was not processed) so that it is not sent out when a job is requested.

Be sure you are catching possible exceptions in your JobRun code. To help diagnose errors in Worker Helper apps, try using System.DebugLog to write information to the system log.

Since Worker Helpers run as separate console apps (when built) they do not retain any state from the main app. If you have to do activation, such as with plugins, then you will also need to ensure the Activate code is called by JobRun.