Re: [geclipse-dev] Job Status

[Mariusz Wojtysiak <mariusz.wojtysiak@xxxxxxxxxxxxx>]

Hello Romain!

Previously every update of glite job status downloaded full information 
about job including: JDL, RSL, ClassAd and much more data.

We changed methods to update job status. Now it's possible to pass 
parameter how many information should be updated during updating job status.

Backgroud updaters just updating only basic information (status-type, 
status-name, reason), whereas updating called by the user from Jobs view 
and Job Details View checking for all available data about job.

Additionally we declared some common objects as static, what should also 
improve memory usage.


Also regarding the overheat when you want to check statuses of thousands 
of jobs, gEclipse provides methods to support you. I assume that you use 
code similar to the one on wiki page:

 while( job.getJobStatus().canChange() ) {
                try {
                        Thread.sleep(5000);
                        job.updateJobStatus();
                        System.out.println(job.getJobStatus().getName() 
+ ": " + job.getJobStatus().getReason());
                } catch (InterruptedException e) {
                        e.printStackTrace();
                }
        }

gEclipse can handle the updating status of the job for you. All you need 
to do is to register a job in JobManager using the following code:

  IGridJobID id = js.submitJob(jsdl, new NullProgressMonitor());
  IGridJob job = GridJob.createJobStructure( folder,
                                                     ( GridJobID )id,
                                                     wms,
                                                     jsdl,
                                                     jsdl.getName() );
  IGridJobManager manager = GridModel.getJobManager();
  manager.addElement( job );

After job is registered in JobManager, JobStatusUpdater is created for 
each job which checks status of the job periodically. Developer can set 
various parameters of the updating mechanism by calling following 
methods of eu.geclipse.core.Preferences class:

- setUpdateJobsStatus( final boolean status )
- setUpdateJobsPeriod( final int period )
- setUpdatersLimit( final int limit )

Especially the last one might be usefull in your case, where you can set 
the maximum number of updaters working in parallel. The rest of the 
updaters are not active when the limit is reached and they are resumed 
gradually when jobs which are running finish. Default values are: job 
updating is active, 30 seconds between each update, 30 limit to parallel 
job updates.

After the job is registered you need to add your listener which will be 
notified when job status is changed or updated:

manager.addJobStatusListener( new IGridJobStatusListener(){

            public void statusChanged( final IGridJob job ) {
              System.out.println( "Status of the job has changed!" );
              System.out.println( job.getID().getJobID() + " :" + 
job.getJobStatus().getName() );
              if( !job.getJobStatus().canChange() ) {
                //Do something when job status can no longer change
              }
            }

            public void statusUpdated( final IGridJob job ) {
              System.out.println( "Status of the job was updated but 
did not change" );
              System.out.println( "Last update time: " + 
job.getJobStatus().getLastUpdateTime() );
            }

          });

Hope this changes will improve the performance of your application. If 
you have any more questions we will provide further assistance.

--
Regards,
Mariusz Wojtysiak

Romain pisze:
> Hello,
>
> Actually  several million instances of GridJobStatus are potentially 
> created during one execution. By consequence, from my point of view it 
> is better to manage their life-cycles and reuse the old instances. 
> Furthermore it is not a lot of work to do.
>
> Rom.
>
> Mariusz Wojtysiak a écrit :
> > Hello!
> >
> > > Yes, at least a new JobStatusUpdater is created for each new Job, not 
> > > a Status itself as far as i see. But for sure Mariusz and/or Pawel 
> > > will have a more authoritative answer.
> >
> > For every GridJob, which status can change (!= DONE, ABORTED, PURGED) 
> > we create JobStatusUpdater object. But those objects are reused during 
> > every status update for given job.
> >
> > As Romain suggested, in fact: after every status update we create new 
> > GridJobStatus object.
> >
> > But I'm not sure that IGridJobStatus implementations causes problem 
> > with garbage collector. Before changing strategy of creation 
> > IGridJobStatus objects I suggest to start profile tool and check which 
> > classes causes problems for GC.
>
> _______________________________________________
> geclipse-dev mailing list
> geclipse-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/geclipse-dev