Open source tools written in Java. Sponsored by Acelet corporation
|Home | Products | Documents | Downloads | Partners | Contact |
SuperScheduler

Table of Contents


Overview
SuperScheduler is a full-featured job scheduler for all applications job scheduling, supporting J2EE tasks and Web service tasks. When it is scheduled for these tasks, it runs as a client of your server. It runs in its own JVM and never interferes the state of your server.

SuperScheduler is the twin software of SuperWatchdog, which is event-action job scheduler.

SuperScheduler is entirely written in Java. It is platform neutral, J2EE vendor neutral, and Web service server neutral. 

Main features

Configuration

Your system can be arranged in different configurations. For a small business, you can run SuperScheduler on the same machine as the J2EE application server. For an enterprise environment, you can run an instance of SuperScheduler in your remote data center, asking operators to take care of it, while run another instance of SuperScheduler on a manager's desk. Managers can schedule and monitor tasks for their departments concurrently. These managers do not need to know which J2EE application runs on which machine in the cluster. For example, warehouses run inventory checking tasks while sales run sale analysis tasks. If anything is abnormal at application level, an alarm email will be send to managers. If there are errors on system level, such as database is down, the alert notification goes to system or database administrators immediately.

A task is guaranteed to run only once at its scheduled time, regardless how many instances of SuperScheduler (either Doer or Talker) run at the same time.

SuperScheduler is thread safe and network safe. Different instances of SuperScheduler will not corrupt each other. If two users edit the same task at the same time, either in the same office or on the other side of the world, only one can be successful. After one of them saves data into database, the other user's data is stale. If other user tries to save stale data he or she will get an error.

Prerequisite

Network

All machines in your network must be time synchronized.

Software

SuperScheduler is optimized by using on the following technologies from Super.:

The logging facilities are important for SuperScheduler. You can use either SuperLogging, or define your own LogAgent. If you use SuperLogging, it must be installed, up and running before SuperScheduler runs.

EasyJMX is for you to set JMX (MBean) tasks visually. If you do not run JMX tasks, you do not need it.

EasyJms is for you to set JMS tasks visually. If you do not run JMS tasks, you do not need it.

EasyEJB is for you to set EJB tasks visually. If you do not run EJB tasks, you do not need it.

EasyWebservice is for you to set Web Service tasks visually. If you do not run Web Service tasks, you do not need it.

See Prerequisite in Installation for general prerequisite.

Scheduler mode

There are two scheduler modes: Doer and Talker. 

Doers run with full power of SuperScheduler. It is the primary scheduler. It has an executer running with it on the background. When the scheduled time is due for a task, the executer executes the task in the background.

Talkers are designed for remotely scheduling, managing and monitoring tasks only. You can add, edit, delete and suspend tasks using Talker. Talker runs without the executer, so Talker can NOT execute scheduled tasks, it means that scheduled tasks DO NOT run on Talker. For your convenient. You can manually run a task "immediately once" on either Doer or Talker, but the real execution will be on a Doer, using Doer's CPU and memory. The "immediately once" is actually a request. A Doer will take the order and execute the request promptly (in interval - the default value is 2 seconds).

You can run as many instances of SuperScheduler as you like, in either Doer mode or in Talker mode within your network. All Doers in the system are identical and they have no dependency on each other. The same is true for all Talkers.

If there are Doers running, a task will be executed once and only once at the scheduled time. The Doers which use the same task database compete with each other. Only the early bird catches the worm. But which Doer runs which task may or may not be predictable.

If there is no Doer running, no any scheduled  tasks will be executed. If there is no Doer running at a task scheduled time, the task will become a missed task. Doers detect missed tasks. If the Doer finds any missed tasks, it will send an alarm message.

To guarantee the scheduled tasks running on time, you should keep at least one Doer running all the time.

If you only want to schedule, manage or monitor tasks and do not intent to actually run scheduled tasks on your desktop machine,  you should run Talker on the machine, instead of Doer.

Task

A task is an object with one target job and one scheduled term. A task will be executed once at its scheduled time.  

Note: a Task of SuperScheduler is similar with a Task of SuperWatchdog. But they are different: the scheduled terms are different. The scheduled term of SuperScheduler is by timing, while the scheduled term of SuperWatchdog is by event.

The status of the task is about the nature of the task. A task can have the following statuses:

 Active The scheduled term is future runnable.
 Inactive The scheduled term is Standby or out of the effective running period.
 Suspended The task is suspended. 
 Error Error occurs when the task is created. The task will not run.

A task will never be scheduled if its status is not Active. Non-active tasks can still be run manually. Non-active tasks may be run programmatically or as a member of a composite job or retry job.

Working Task

A working task is a mirror version of the original task for one execution only. A working task is created and maintained during the execution of a task. 

The status of the working task is about the execution status of the task. A working task have the following statuses:

Term

The term is the timing information about the task. It consists of  the following attributes: Repeating, Start time, Holiday/Weekend policy and Expiration time.

The repeating attribute of a task can be:

Advanced Attributes

A task will run at due time, if it satisfies all advanced attributes. There are following attributes:

When a scheduled task falls on a holiday or weekend, it will be adjusted by the Holiday/Weekend policy. The options are:

Note: Holiday/Weekend Policy is used to adjust the original scheduled time. If a task does not run at the original time, there will be no adjustment. 

Example 1: A task runs on the first day of each month with Previous Business Day policy. It will not run on January 1 (New Year Day holiday), but will be adjusted to run on December 31, if that day is a holiday as well, SuperScheduler will try December 30,... until find a business day.

Example 2: A task runs on 29th of each month with Next Business Day policy. It will not be scheduled on February 29, 2003 (no such day). It will not run on March 1 either: there will be no adjustment at all.

The expiration time defines an effective period for a task: A task will not start after the expiration time.

Example: Suppose you have a task scheduled to run at each hour and the task takes  40 minutes to finish. Suppose the expiration time is 22:30. The last task will start at 22:00 (in its active period) and run to its completion at 22.40.

Brief information of Task

The format of summary information about schedule terms is:

    Repeating; Holiday/Weekend-Policy; ExpirationTime;

Some parts of them may be empty if they are not defined.

Summary information

SuperScheduler logs major events using brief information of a task. The format of brief task information logged is:

!! "working task status" > "task name" > "term" > "task status" > "host" > "job brief information"

Many of the above attribute can be empty.

Example:

!! Task done > showLuckyNumber > Minutely: 1 minute(s); The same day; 2002-05-24; hp1 > Active > Test

Period

The period is the effective period for a task: A task will not be scheduled out of its period.

Suspend task

A task can be suspended and un-suspend at any time you want. A suspended task will never be run at the scheduled time. A suspended task can be manually run immediately.

Reactivate task

If the status of a task is Suspended or Error, it can be reactivated by re-save it. Or click the Activate button on the SuperScheduler panel.

Missed task

A missed task is a task which did not run at its scheduled time.

A missed task will be scheduled to run at the next possible time. 

SuperScheduler does not provide any remedy for a missed task. For example, if you schedule a task to run every day and Monday's task is missed, the task will run six times this week, instead of seven times. Of course, you can always provide your manual remedy.

Killed and Lost task

A working task can be killed manually or programmatically. If you set the option of "Kill if exceeds duration" for a task and the task runs longer than expected duration, SuperScheduler will kill the working task.

Kill or Lost is a status of a working task. It does not affect the status of the original task.

Note 1: When a working task is killed, the behavior of the working task depends on the nature of the job. For Operating System jobs, it is just like you press Control-c on the window. For remote execution jobs, such like EJB, the real job may still goes on.

Note 2: The status Lost is only a mark for close. If a machine crashes while running a task, the status of the working task will remain as Started for ever by itself. You can mark it as Lost and close, from other machine, or when the machine is up and running again.

Run a task immediately

Any task can be manually run immediately. The schedule of the original task will not be changed.

Actually, the "command" to run immediately is a "request". A Doer will pick up the request and run the task promptly (in interval - the default value is 2 seconds).

Run a task programmatically 

To run a task programmatically you need to do as the following:

1. First, you need to manually, visually set the tasks using SuperScheduler GUI.  It is recommended that the task's repeating attribute to be Standby. 

2. Then, you activate the task in your program by calling methods in class SchedulerControl. The SchedulerControl is designed for simplicity without losing power. It is very simple and less error-prone. To activate a task, you call

public void requestRunTask(String taskName);

Example: 
    String driverName = "org.hsql.jdbcDriver";
    String databaseUrl = "jdbc:HypersonicSQL:hsql://localhost:23536";
    String userName = "acelet";
    String password = "acelet";
    Class.forName(driverName);
    Connection connect = 
      DriverManager.getConnection(databaseUrl, userName, password);

    Properties properties = new Properties();
    properties.setProperty("java.naming.factory.initial", "weblogic.jndi.WLInitialContextFactory");
    properties.setProperty("java.naming.provider.url", "t3://localhost:7001");
    properties.setProperty(Context.SECURITY_PRINCIPAL, "system");
    properties.setProperty(Context.SECURITY_CREDENTIALS, "password");
    InitialContext context = new InitialContext(properties);
    String logAgentName = "com.acelet.s.scheduler.LogAgentUsingSuperLogging";
    LogAgent logAgent = new LogAgentUsingSuperLogging(context);

    SchedulerControl SchedulerControl = new SchedulerControl(logAgent, connect);

    SchedulerControl.changeTaskStatus("myFirstTask", STATUS_ACTIVE);
    SchedulerControl.requestRunTask("secondTask");
    Thread.currentThread( ).sleep(1000);
    SchedulerControl.requestRunTask("myFirstTask");
    Thread.currentThread( ).sleep(1000);
    SchedulerControl.requestRunTask("goodTask");
    SchedulerControl.deleteTask("abc");

The control methods in SchedulerControl are available from SuperSchedulerLibrary. You need to install that library, so they are available for your applications.

Date and time format

The date and time format is ISO 8601 format, not locale dependent. The format is: YYYY-MM-DD hh:mm:ss. 

Target jobs

A job is an executable process. A job of SuperScheduler is identical with the same job of SuperWatchdog.

Currently supported target jobs are:

Note 1: J2EE is in distributed computing environment with no machine dependency. The Java Application jobs and Operating Systems jobs may be machine dependent. You may need to specify desired host, environment and working directory for running those jobs. Theses jobs are provided as a convenient extension. 

Note 2: The following jobs

require pre-saved data files from other modules. The job files are XML formatted ASCII files. You can generate those data files on your working machine or on any machine running Super and copy these data files to your working machine. When the task is created, SuperScheduler saves a private copy of the data file into the task database. The original data files are not needed for scheduled execution. If you modify the original data file, the scheduled execution will not be affected until you update the scheduled task.

Any Exception thrown from a job will be considered as error. For Operating System and Java jobs, the exit code of a job is checked. SuperScheduler considers non-zero exit code as an error. If there is any error, an email alarm will be triggered.

Composite jobs 

SuperScheduler also supports composite jobs which consist of any combination of the above jobs. 

It consists of one or more member jobs. The member jobs are executed according to the order specified. If any member job generates errors during the execution, the rest of the member jobs will not be executed. The error is recorded as the result of the member task. An alarm email will be sent as the alarm of the composite job, if the email attribute is set.

A member job must be an atomic job: composite jobs and retry jobs are not eligible to be members of another composite job.

Example of composite application:

  1. Send email (to a pager) asking to prepare printers.
  2. Wait 10 minutes.
  3. Start print job.
  4. Send email signal the end of the print job.

Retry jobs 

SuperScheduler also supports retry jobs which consist of any combination of the above jobs. 

It consists of one or more member jobs. The member jobs are executed according to the order specified. If any member job is executed successfully, the rest of the member jobs will not be executed. 

If all member jobs are executed and none of them was successful, then the working status of the composite job is Error. An alarm email will be sent as the alarm of the composite job, if the email attribute is set.

A member job must be an atomic job: composite jobs and retry jobs are not eligible to be members of another retry job.

Example of retry application:

  1. FTP file to address A, the first time.
  2. FTP file to address A, the second time.
  3. FTP file to alternative address B, because something was wrong for address A.

Log 

SuperScheduler logs important messages by calling LogAgent. The default LogAgent writes log record using SuperLogging to record major task changes, such as status changes and core attribute modification. 

The information logged is the Summary information of Task for detail information. If you use the default LogAgent (underlying facility is SuperLogging), there will be more information logged by SuperLogging.

If you want to log schedule events using your favorite logging software, you need to:

  1. Provide your version of LogAgent. Your implementation must provide a default constructor (without parameter list). Note: your default constructor may be called more than once.
  2. Install your version of LogAgent:

Modify scheduler.properties under beanbox directory of installed directory (default is AceletSuper/beanbox). You will find the following line:

    SuperSchedulerLogAgentName=com.acelet.s.logging.LogAgentUsingSuperLogging

Replace the right part (after the '=' character) by your full class name. You need to put your implementation on the CLASSPATH. See Environment variables for details lo

If you do not want to log, you need to remove this line, or just implement LogAgent.log( ) as an empty method.

Note: If you use other logging system, you will lose some facilities of SuperScheduler, such as Log History.

Alert email and Alarm email 

Alert email:  If there are errors on the system level, such as database is down, the alert email notification goes to system or database administrators immediately.

Alarm email:  If anything is abnormal at application levels, an alarm email notification will be send to managers immediately. The alarm email address is specified for each task. 

Alarm emails will be sent in the following situation:

Note: Mail Server must be set. See Mail Server Panel from the main menu > System > Mail Server for related information.

Refresh interval

Many copies of SuperScheduler can run at the same time on different computers. When:

on one instance of SuperScheduler, other instances of SuperScheduler will be refreshed promptly through heart-beating. The period of heart-beating is interval.

The interval is configurable time period (default value is 2 seconds). The interval does not affect the execution or schedule of the system. It only affect the "freshness" of the data for each instance of SuperScheduler.

You can manually refresh the GUI panel any time by click the Refresh button as well.

Task database

Task database is the database which stores task information. The default task database is the database comes with Evaluation Edition of Super for demo purpose. 

A task database here means a database connection or a database login. It does not matter if those connections are for one physical database or not. If you have many departments and want each department takes care its own tasks, each department should use its own task database.

Be sure there is only one log database running for SuperLogging, although you can have more than one task database for different instances of SuperScheduler.

SuperScheduler and SuperWatchdog can share the same task database or use separated two databases. The database tables for the two are similar, but different.

SuperScheduler is an administration tool. It should run within trusted network where Direct database connect is convenient. If the direct connection to database is not practical for whatever reasons, you need to use VNC (Virtual Network Computing). 

SuperScheduler uses standard ANSI SQL database as task databases. SuperScheduler can work with any ANSI SQL 92 entry level database. The task database schema has been tested using the following databases:

If you want to use other database as your task database, please contact us.

Customize the length of some columns of the task database

If your database is written in Java and the text data type is java.lang.String, you do not need to worry about the length of columns. But, if you use traditional databases, such as Oracle, DB2, Sybase, the data type of text columns is VARCHAR or CHAR based. You may have to specify the maximum length for each column. The lengths of columns of database schemas that Acelet provides are for demonstration purpose. You may want modify these values according to your applications.

Direct database connect

With direct task database connection, SuperScheduler connects to the task database directly, not through server (J2EE application server) side components. 

If you do not use the default database as your task database, you must specify your database parameters on the Direct Task Database Connection Panel from the main menu. The system will store all parameters in plain text. We assume that your computers are located in trusted areas and protected by passwords. Please see Security for more information.

Get around the firewall

See Get around the firewall in SuperArch for detailed information.

JMS message job

JMS message jog is for sending messages for Java Message Service. It uses unified, common interface for J2EE 1.4 application servers. You can use any connection factory.

JMS Queue and Topics

JMS Queue and JMS Topic jobs are deprecated. For J2EE 1.4 application servers, please use JMS message job instead.

JMS (Java Message Service) Queue and Topic jobs are among schedule-able jobs of SuperScheduler. You need to configure your J2EE server to provide message factories. 

The default JNDI name for queue factory is: SuperJmsQueueConnFactory.

The default JNDI name for topic factory is: SuperJmsTopicConnFactory.

If you want different JNDI name, you need to specify them in scheduler.properties which is under Super home directory (default is AceletSuper). For example: 

SuperJmsQueueConnFactory=MyQueueFactory
SuperJmsTopicConnFactory=YourTopicFactory

Of course, these JNDI names and objects must be set on your J2EE server.

Holidays sets, holiday rules and holidays

The word holiday here is a generic word. It applies to all business idle situations, for example, weekends, business shutdown, labor strike, ... 

Holiday rules are algorithms. They do not belong to any particular year. They are optional. At run time SuperScheduler only consults holidays to adjust the actual next run time.  

Holidays can be generated from holiday rules by clicking a button. It is recommended that the user generate next year's holidays at the end of the current year. 

From time to time, you may want to manually input a few special days as holidays. The GUI holiday facilities is very convenient to do so.

All tasks which use holiday policies will be promptly (in interval, the default is 2 seconds) updated according to any holiday changes automatically regardless of on which machine they run.

Not all tasks are equal to all holidays. Some tasks are sensitive to a group of holidays, other tasks may be sensitive to another group of holidays. Each group of holidays is a holiday set. 

The majority of users uses only one holiday set.  If you are one of them, you do not need to worry about holiday sets. By default, you just use the default holiday set: a no-name holiday set. You will not see unnecessary GUI components.

If you want to use multiple holiday sets, you need to add a line on scheduer.properties file which is located under the directory beanbox of the Super home directory (default is AceletSuper):

useMultipleHolidaySets=true

See Properties of SuperComponents for related information.

Then, related GUI components will appear. If you add more holiday sets, your co-workers will see those holiday sets even if they do not add that line on their copies of scheduler.properties.

You can use as many holiday sets as you like. Each holiday set may have a group of holiday rules, which define the rule for generating the holidays for the holiday set. SuperScheduler provides some holiday rules.  Users can define their own holiday rules.

There is no relationship among holiday sets, each holiday set is an independent group.

 

 

Kill working task

Working tasks can be terminated manually, programmatically or automatically if they exceeds expected duration.

See Killed and Lost task for related information.

Graceful and forceful shutdown

If you terminate a Talker, there is no impact on the system: the schedule engine is not running on the instance of your SuperScheduler.

If you run a Doer, either Doer window of SuperScheduler or command line executor, there is a schedule engine running in the background,  you have choices to terminate it gracefully or forcibly.

When SuperScheduler engine exits, either gracefully or forcibly, all scheduled tasks on that instance of SuperScheduler will be canceled. There will be no future scheduled tasks on that instance of SuperScheduler. Shutting down one Doer will not affect the state of another Doer. But other Doers will take the slack automatically and immediately for future schedule. 

The differences between graceful and forceful shutdown are the way to treat currently running tasks. Graceful shutdown terminates when all running tasks are finished naturally. Forceful shutdown terminates the JVM unconditionally making the states of running tasks unknown.

If you terminate the Doer window, but keep the main window of Super open,  then graceful shutdown procedure starts. You can open the Talker window immediately to monitoring the shutdown process. All other ways to shutdown SuperScheduler are forceful shutdown.

Security

Direct database connection

Users need sufficient privileges on the J2EE server, the log database, the task database and all underlying facilities. The system is protected by these accounts and passwords.

When you input password on the database parameter panel, it does not show literally. When you save it, it will be saved as plain text. If you have security concerns, do not save your password. See Security guideline for logins of SuperArch for detailed information.

Access logging system

SuperScheduler logs major event on the logging system, so the user of SuperScheduler needs privileges of access logging system. Of course, if you do not use LogAgent (set it null explicitly) on configuration files, this will not be an issue. See Log for related information.

Role based entitlement

The Role based entitlement  a user base entitlement for operating GUI panels.

When a user is assigned to a particular role, the user is entitled for operations it entitled only.

There are three default roles: Admin, Manager and Operator. The administrator of entitlement can define or edit roles with any combination of rights for all GUI operation.

When a role is specified, the role based entitlement will automatically take effect. When you delete all roles, the role based entitlement will be disabled automatically. As default, there is no user assigned to any role, so the role based entitlement is disabled as default.

The role based entitlement does not provide entitlement management for the scheduler engine. If you have tasks needed to run for different users, you need to use different task database. The different task database can be the same logical database, but with different connection (login).

There are two tables in the task database which store data for the role based entitlement:

Every one use the task database needs privilege to SELECT entitlement related tables, only the administrator needs INSERT and UPDATE privilege to related tables. If every user is assigned all privilege to the related table, there will be no extra security, because every one can be the administrator and change entitlement. But the role based entitlement still provides valuable features. For example, an operator will not accidentally delete a task: he or she does not have that entitlement.

Command line executer (the daemon)

You can run the Scheduler executer as a command line program (the daemon) under <Super Home>, the default directory is AceletSuper. You need to run run.bat (Windows) or run.sh (Unix) scripts. These scripts set required environment variables for using some jar files.

You can do either of the followings:

On Unix:

     > cd AceletSuper
     > sh runSuperSchedulingDaemon.sh

On Windows:

     > cd AceletSuper
     > runSuperSchedulingDaemon.bat

To run SuperScheduler command line executor (daemon) you still need a full installation of Super. When the command line executor starts, it uses parameters stored in properties files to connect to server and databases. The files and property values are: 

 super.properties, for application server connection: 
serverLoginName
serverPassword
 beanbox/scheduler.properties, for task database connection:
taskDatabaseUserName
taskDatabasePassword

These parameters are stored as plain text. If you have security concerns, do not save them. When the command line executor starts, it tries to connect to the database. If the connection fail, it will prompt message and ask you to key in user names and passwords. See security for more information.

To shutdown the command line executor, you need to use facilities from the operating system, such as Control-c, kill (Unix) or Task Manager (Windows). See Graceful and forceful shutdown for related information.

Schedule tasks from programs written in other languages

You can schedule tasks from any program written in any language, for example C/C++. For end users, we recommend that you use SuperScheduler to set a task visually, than activate the task in your C/C++ by inserting a row in task database. Specifications are available for prospective licensees. Please contact us.

SendDataToHost interface

For Servlet/JSP (URL) job, if the destination servlet or JSP uses POST method. The required data must be sent to the J2EE application server. Each application is different and there is no standard way to send data. So you need to implement SendDataToHost interface. When SuperScheduler execute a Servlet/JSP (URL) job, it will call SendDataToHost.send() to send data to the destination URL. You need to put your implementation of SendDataToHost on the CLASSPATH of Super. See Environment variables for related information.

Arrange your schedule wisely 

Avoid starting jobs at the same time

When a task starts, it needs a lot of computing power from client, server and network, so avoid starting tasks at the same time, if you can. SuperScheduler does not impose any restriction on that.

Figure out the power your tasks need

Different tasks need different computing power. 

EJB (Poke) tasks are running on the application server as EJB services and do not need much power from SuperScheduler (the client). 

MBean (JMX) tasks are running on the application server as JMX services and do not need much power from SuperScheduler (the client). 

Other tasks are very from job to job.

Generally speaking, if your computer is capable to run the job out side of SuperScheduler, it can be scheduled with SuperScheduler. SuperScheduler itself does not need much computing power.

Hibernation

SuperScheduler is hibernate-able. See Hibernation for details.

Note: the hibernation saves and restores applicable panels only. If there is an input data panel open when Super is shutting down by Shutdown Super job, the data in the open panel will not be saved automatically, the panel will not be open automatically when Super restores the state of the system.

Bounce the whole system

Many companies have policies to shutdown-restart their whole system regularly to make sure the whole system is always "fresh". SuperScheduler and SuperWatchdog provide very convenient tools to do such tasks, by providing  Shutdown Super facilities with Hibernation and auto-login. 

Unix (POSIX) cron schedule

The native scheduler on Unix is cron. It allows multiple choices. It runs on the specified hour and minute on specified days. It consists of five sections: Minute, Hour, Monthday, Month and Weekday.

SuperScheduler provides a similar term for people who are familiar with cron. The "At specified times" repeating attribute of the task implements "UNIX Specification, Version 2" of POSIX. The core specification is:

If month, day of month and day of week are all asterisks, every day will be matched. If either the month or day of month is specified as an element or list, but the day of week is an asterisk, the month and day of month fields will specify the days that match. If both month and day of month are specified as asterisk, but day of week is an element or list, then only the specified days of the week will match. Finally, if either the month or day of month is specified as an element or list, and the day of week is also specified as an element or list, then any day matching either the month and day of month or the day of week, will be matched.

   © Copyright 1999-2008 Acelet Corporation. All rights reserved.