Friday, May 4, 2012

WCS Schedulers


Introduction

Regular Scheduler Jobs: Any long running offline process can be run as a WCS scheduler job, think about them as regular WCS commands which gets triggered by WCS scheduler engine and performs a specific task based on your custom implementation, the jobs could be one time only or can be executed at a regular frequency.

Broadcast jobs: These are WCS jobs which run on every single WCS JVM in the cluster at least once,  Refresh Registry is one such job, this Job ensures registry which resides in WCS JVM on every instance is refreshed.

All WCS jobs require WCS runtime for there execution, which means they can not be run outside of WCS server or container.

In terms of datamodel, there are few key tables you need to be aware of
SCHCONFIG
SCHSTATUS
SCHACTIVE
SCHBRDCST

Creating a custom Scheduler Job

//Define the custom scheduler interface
package com.ibm.commerce.samples;
import com.ibm.commerce.command.ControllerCommand;
public interface MyCustomSchedulerCmd extends ControllerCommand {

public static final String defaultCommandClassName="com.ibm.commerce.samples.MyCustomSchedulerCmdImpl";

}

//Define the custom scheduler implementation class
package com.ibm.commerce.samples;
import java.util.logging.Logger;
import com.ibm.commerce.command.ControllerCommandImpl;
import com.ibm.commerce.exception.ECException;

public class MyCustomSchedulerCmdImpl extends ControllerCommandImpl implements
        MyCustomSchedulerCmd {
  
    Logger logger = Logger.getLogger(MyCustomSchedulerCmdImpl.class.getName());

    public void performExecute() throws ECException {
        //implement custom logic
        logger.info("Hello World from custom scheduler");
    }
}

//Make following Struts Config entry

<action parameter="com.ibm.commerce.samples.MyCustomSchedulerCmd" path="/MyCustomScheduler" type="com.ibm.commerce.struts.BaseAction">
<set-property property="authenticate" value="0:0"/>
<set-property property="https" value="0:1"/>
</action>

//Make following DB entries to register the new JOB.

-- Use following queries for Apache Derby DB
insert into schconfig (sccjobrefnum,scchost,member_id,storeent_id,sccrecdelay,sccrecatt,sccpathinfo,sccquery, sccstart,sccinterval,sccpriority,sccsequence,sccactive,sccapptype,interfacename,optcounter)
values
((select Max(sccjobrefnum)+1 FROM schconfig),null, -1000,10051, 100,0,'MyCustomScheduler',null, CURRENT_TIMESTAMP, 100, 5,0,'A',default,'com.ibm.commerce.samples.MyCustomSchedulerCmd',0);

insert into schactive (scsinstrefnum,scsjobnbr,scsactlstart,scsattleft,scsend, scsinstrecov,scsprefstart,scsqueue,scsresult,scssequence,scsstate,scspriority,optcounter)
 values
 ((select Max(scsinstrefnum)+1 FROM schactive),(select max(sccjobrefnum) FROM schconfig), CURRENT_TIMESTAMP,1,null,null,CURRENT_TIMESTAMP,null,null,0,'I',default,0);

-- Oracle DB
insert into schconfig (sccjobrefnum,scchost,member_id,storeent_id,sccrecdelay,sccrecatt,sccpathinfo,sccquery, sccstart,sccinterval,sccpriority,sccsequence,sccactive,sccapptype,interfacename,optcounter)
 values
((select Max(sccjobrefnum)+1 FROM schconfig),null, -1000,10051, 100,0,'MyCustomScheduler',null, CURRENT_TIMESTAMP, 100, 5,0,'A',default,'com.ibm.commerce.samples.MyCustomSchedulerCmd',0);

insert into schactive (scsinstrefnum,scsjobnbr,scsactlstart,scsattleft,scsend, scsinstrecov,scsprefstart,scsqueue,scsresult,scssequence,scsstate,scspriority,optcounter)
 values
 ((select Max(scsinstrefnum)+1 FROM schactive),(select max(sccjobrefnum) FROM schconfig), SYSDATE,1,null,null,CURRENT_TIMESTAMP,null,null,0,'I',default,0);

Sticky configuration for Scheduler jobs

A regular scheduler job by default may run randomly on any WCS instance in a cluster, you should use following configuration for sticky configuration of a Scheduler, this will allow it to run always from a given WCS instance in a cluster.

a. Define a JVM Property on the instance which will run the Scheduler Job
"com.ibm.commerce.scheduer.SchedulerHostName" and provide an identifier for the JVM that should run the job.

Refer following link for more details.
http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.admin.doc/tasks/tjsinstschjob.htm


b. While creating the Job replace the value for SCHCONFIG.SCCHOST with the value of the property, in this case as "mySchedulerJVM"

E.g. note that scchost has been replaced with 'mySchedulerJVM'


insert into schconfig (sccjobrefnum,scchost,member_id,storeent_id,sccrecdelay,sccrecatt,sccpathinfo,sccquery, sccstart,sccinterval,sccpriority,sccsequence,sccactive,sccapptype,interfacename,optcounter)
values
((select Max(sccjobrefnum)+1 FROM schconfig),'mySchedulerJVM', -1000,10051, 100,0,'MyCustomScheduler',null, CURRENT_TIMESTAMP, 100, 5,0,'A',default,'com.ibm.commerce.samples.MyCustomSchedulerCmd',0);



Monitoring and Maintaining Scheduler Jobs

WCS provides WAS Admin console from which you can validate the Job status, this tool essentially lists down every single job status from SCHSTATUS table and is not a very user friendly tool, for some reason the pagination and sorting feature does not work very well.
as an alternative option you can query OOB tables with your job name and validate the status of the job.

//This Query validates if you have the scheduler configured with the Active status
select sccjobrefnum from wcsadm.schconfig where sccpathinfo like '% MyCustomScheduler%' and sccactive='A'

select * from wcsadm.schactive where scsjobnbr IN
(select sccjobrefnum from wcsadm.schconfig where sccpathinfo like '% MyCustomScheduler%' and sccactive='A');

//This Query validates if an entry is created for the last run of Scheduler, if you don't see an entry here based on the job frequency of the scheduler it would mean the scheduler is not running successfully
select * from wcsadm.schstatus where scsjobnbr in (select sccjobrefnum from wcsadm.schconfig where sccpathinfo like '% MyCustomScheduler%' and sccactive='A')
order by scsactlstart desc;

Tuning WCS Schedulers

As of WCS 7, fix pack 2, we have few additional options to fine tune the scheduler jobs.
you can define a custom transaction timeout for scheduler Jobs which will be independent of your WAS level setting for global transaction timeout.

a. Disabling Scheduler in WCS instance
follow these steps If you have a need to not run any scheduler jobs from a given WCS instance,
Edit the wc-server.xml file of a given instance and edit the value enabled=false, this configuration will disable scheduler jobs from running on this instance.
 It is important to note that by doing this you are disabling the ability of this JVM to run any Broadcast event Jobs as well.

        <component
            compClassName="com.ibm.commerce.scheduler.SchedulerComm"
            enable="false" name="Scheduler">
            <property autoClean="off" broadcastExpireTime="1800"
                contextSetName="Authoring" cycleTime="600" display="false"/>
        </component>

b. Defining transaction timeout for Scheduler Jobs

Here are the steps which is part of APAR IZ02770, which explains how to specify the         
scheduler-specific transaction timeout settings:                       
                                                                        
Update the WC_installdir/instances/instanceName/xml/instanceName.xml
to include the new transactionTimeout field. Set the value in seconds  
for your desired transaction timeout for scheduled jobs                
                                                                       
<component                                                             
compClassName="com.ibm.commerce.scheduler.SchedulerComm"               
enable="true" name="Scheduler">                                         
<property autoClean="off"                                              
broadcastExpireTime="1800"                                             
transactionTimeout="1800"                                               
contextSetName="Authoring"                                             
cycleTime="600"                                                        
display="false">                                                       
</property>                                                             
</component>                                                           
                                                                       
Restart the application and test your scenario

2. Defining dedicated Container threads for Jobs
This setting might be useful if you are using WCS JVM to serve external traffic and also run some of the Scheduler jobs, the configuration defines maximum threads for various Jobs.

<component                                                             
compClassName="com.ibm.commerce.scheduler.SchedulerComm"               
enable="true" name="Scheduler">                                         
<property autoClean="off"                                              
broadcastExpireTime="1800"                                             
transactionTimeout="1800"                                              
contextSetName="Authoring"                                             
cycleTime="600"                                                        
display="false"> 
<applicationType applicationName="default" maxNumofThreads="10"/>      
<applicationType applicationName="broadcast" maxNumofThreads="10"/>    
<applicationType applicationName="auction" maxNumofThreads="10"/>      
<applicationType applicationName="inventory" maxNumofThreads="10"/>    
</property>                                                             
</component>

Refer following link for more details
http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/topic/com.ibm.commerce.admin.doc/concepts/cjsconfig.htm
 
More Reading

Maintaining Scheduler
http://publib.boulder.ibm.com/infocenter/wchelp/v7r0m0/index.jsp?topic=/com.ibm.commerce.admin.doc/tasks/tjsmaint.htm

Websphere Commerce Scheduler Mustgather
https://www-304.ibm.com/support/docview.wss?uid=swg21454411

Use following trace component for troubleshooting and scheduler related issues.
*=info: enable.trace.log.*=all: com.ibm.websphere.commerce.WC_THREAD=all: com.ibm.websphere.commerce.WC_SERVER=all

Maintaining the WebSphere Commerce Scheduler tables
https://www-304.ibm.com/support/docview.wss?uid=swg21397348

37 comments:

  1. Thanks for the article. I was wondering why you need to set a JVM property to run a scheduler on a given server within a cluster all the time. Why not use the allowed host feature within the scheduler configuration which can be changed anytime without server/JVM restart?

    ReplyDelete
  2. When you say scheduler configuration I'm assuming you are refering to schconfig.scchost, You need to do both, essentially the JVM property gives a name/identifier for your WCS JVM instance and then you set this value in schconfig. scchost

    ReplyDelete
  3. Really very Helpfull..!!!
    Thanks Hari..

    ReplyDelete
  4. Excellent article, thanks a lot Hari! If only the information provided in InfoCenter could be more like this.

    ReplyDelete
  5. Don't we need to make an entry to CMDREG for the custom Scheduler command? Just curious.

    ReplyDelete
    Replies
    1. Insert statement into the CMDREG table is not absolutely necessary. By default, the interface uses the default implementation, and as such, this association between the interface and implementation class does not really need to be specified in the command registry.

      In my sample code above I have defined the implementation class name in interface definition and hence you don't need a CMDREG entry


      MyCustomSchedulerCmd extends ControllerCommand {

      public static final String defaultCommandClassName="com.ibm.commerce.samples.MyCustomSchedulerCmd";

      Delete
  6. if a company has an enterprise scheduling software should we use that vs WCS scheduler?

    Ex there is an external service that is to be invoked for order status which in turn update orders in Commerce. Would you recommend WCS scheduler or can this be outside?

    ReplyDelete
    Replies
    1. Typically the Scheduler jobs that run in WCS would require WC Runtime support and hence you cannot run them outside WC runtime environment.
      for your example you may have to analyze and see if your Order update flow just includes a simple DB status update or does it require other API / runtime support provided by WC Runtime.

      Delete
  7. thanks for the scheduler sql - very helpful.

    ReplyDelete
  8. Hello Hari,
    In WCS admin console, after clicking on the Scheduler link no jobs are listing on the console window. Can you please share your views ?


    Thanks,
    Raja

    ReplyDelete
    Replies
    1. not sure if you are selecting site/store , it could be all jobs are site level.

      Delete
  9. Hey - FYI to others following this post - there is a bug in the example code. In the Interface you need to use:
    public static final String defaultCommandClassName="com.ibm.commerce.samples.MyCustomSchedulerCmdImpl";
    instead of
    public static final String defaultCommandClassName="com.ibm.commerce.samples.MyCustomSchedulerCmd";

    the defaultCommandClassName needs to point to the Implementation Class, otherwise the scheduler will not know how to call it. After making that minor change the above code and DB updates work great for me. Thanks for the example.

    ReplyDelete
  10. We have written a scheduler to pick up all orders from wcs db and one by one call an external SAP webservice on ever order.
    Get the response back form SAP and update the wcs db table. Many a times our scheduler is timing out waiting for webservice response.
    Please help.

    ReplyDelete
  11. Hi ,

    I have scheduler which will run every one hour and pick the orders from orders table with status 'M' and by web service call to a external system and after the response will update the DB.For this i have configured the scheduler to run on one server in the cluster envirnoment by giving the "AllowedHostName" while configuring the scheduler.But sometimes the scheduler is picking the new orders and sometimes not..many times not picking.But while i am running manually it is picking. Please help me what needs to be done so that it will pick everytime.

    ReplyDelete
  12. Hi Hari, I see one of my scheduler job running in Inactive JVM. Can you Please help.

    ReplyDelete
  13. Hi Hari,
    Do you know if we can implement multi threading solution with WCS Scheduler? Basically we need to implement multi-threading solution where we have a single scheduler job with multiple worker threads.

    ReplyDelete
    Replies
    1. That would be an anti pattern for J2EE, best way to implement this is by using the WAS worker thread feature, but that will run outside the WCS context.
      I would not recommend multi threaded programming in WCS scheduler jobs. if you just need more schedulers then you can always create multiple entries for the same job so they all process different set of records for you.

      Delete
    2. Thank you Hari

      Delete
  14. Super article looking to create a scheduler through queries, got correct article
    Thank u very much.

    ReplyDelete
  15. Visit this amazingly helpful website for real estates!!
    Real estate

    ReplyDelete
  16. Hello Hari, are Schedulerhosts runing only in the WSC installation Server ? we have a problem in the Cluster if the Schedulerhost not define on the primary WCS Server and the WCS Server are down (IP not available )? Can you help ? thank you
    HUONG

    ReplyDelete
    Replies
    1. yes scheduler runs within the wcs server, one option for high availability design could be to have more than one server running the scheduler jobs, you will have to customize the job to ensure they do not run from both the servers else you will have duplicate jobs, this should help you with the high availability architecture for scheduler jobs.

      Delete
  17. Hi Hari, thank you for reply. Is the Service for Scheduler Jobs differently from WCS Tools like Accelerator, MC ...?
    We have Server A and Serve B. On Server A is the WCS Installation and we define strictly a Instance S0 as the Schedulerhost. Our idea for HA we define on Server B Instance S0 with the same Schedulerhostname and started only if Server A are down. This idea working fine if the IP of Server A are available. If Server A power off (IP are not available) WCS Tools and Shop working through Server B but Schedulerjobs not working. We dont understand why this defferent. Is our problem because we define the Schedulerhost strictly ? HUONG

    ReplyDelete
  18. for your configuration I think you should keep both S0 on Server A and S0 on Server B running, and have same scheduler hostnames defined at the JVM property level, WCS will automatically ensure that only one JVM picks up the JOB, this will also handle your HA, I'm not sure why you have to manually start the servers for HA.

    ReplyDelete
    Replies
    1. Hello Hari,for newbie like me is your Site a big help ,very thanks to you :* Our Problem with the Schedulerjobs we will try go back to Standard without sticky definition the Schedulerhost and hope to resolve this problem

      Thx HUONG

      Delete
  19. I have a question. When i run my sitemap job, i used to get the servername in the sitemap.xml instead of the domain name. I do see the host name parameter passed from the scheduler is wcserver1. Do you have any idea where should we need to change to get the domain name instead of the servername?

    ReplyDelete
  20. Thanks for sharing this Informative content. Well explained. Got to learn new things from your Blog on SAP MM.SAP MM

    ReplyDelete
  21. Can't we create Scheduler from WCS Admin Console tool?

    ReplyDelete
  22. Hi , When we create a schduler by making entry in DB directly then this scheduler is not shown in Admins console. Can u tell me what more entry is required to get its visibility in admin console also.

    ReplyDelete
    Replies
    1. Add entry in struts configuration file.

      Delete
  23. Hi this can be my terribly initial inquire into your web site. ,I have been reading your journal for a moment and thought i'd fully move into and drop a friendly note. . it's nice stuff so. I conjointly wished to raise..is there the way to buy your web site via email?
    free classified sites in pakistan

    ReplyDelete
  24. I really appreciate information shared above. It’s of great help. If someone want to learn Online (Virtual) instructor lead live training in TECHNOLOGY , kindly contact us http://www.maxmunus.com/contact
    MaxMunus Offer World Class Virtual Instructor led training on TECHNOLOGY. We have industry expert trainer. We provide Training Material and Software Support. MaxMunus has successfully conducted 100000+ trainings in India, USA, UK, Australlia, Switzerland, Qatar, Saudi Arabia, Bangladesh, Bahrain and UAE etc.
    For Demo Contact us.
    Saurabh Srivastava
    MaxMunus
    E-mail: saurabh@maxmunus.com
    Skype id: saurabhmaxmunus
    Ph:+91 8553576305 / 080 - 41103383
    http://www.maxmunus.com/


    ReplyDelete
  25. This comment has been removed by the author.

    ReplyDelete
  26. Hi Hari,

    Firstly, many thanks for the content shared. I am facing a weird issue in my website's production environment while setting a scheduler.

    Problem: Assume a clustered / multi node environment and that we run the Administration Console with host name 'xyz'. The website runs with host 'abc'. So, when the scheduler is run, it is taking the host name of the Administration Console (i.e., 'xyz') only. Due to this, the product links we are building in the email template are getting this ('xyz') as host name. Instead, we want the links to be with 'abc'.

    Could you please guide me how can I resolve this.

    Regards,
    Abdul

    ReplyDelete
  27. Hi, I have executing a scheduler job programmatically and I have the jobId. How can I get the status of that job in command.

    ReplyDelete