[drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions

Tim Harsch harsch1 at llnl.gov
Fri Mar 4 16:02:59 CST 2005

Thanks Peter,
    I'm a little rusty on what DRMAA on SGE is doing for sure, perhaps Ancor
can be a bit more specific as to the behaviour he is seeing.

----- Original Message ----- 
From: "Peter Troeger" <peter.troeger at hpi.uni-potsdam.de>
To: "Tim Harsch" <harsch1 at llnl.gov>
Cc: "Áncor González Sosa" <ancorgs at softhome.net>; <drmaa-wg at ggf.org>
Sent: Friday, March 04, 2005 1:56 PM
Subject: Re: [drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions

Sorry for the late answer. I'm not a SGE expert, but DRMAA section 3.1.2
is very clear about this:

"... Job Id’s SHALL remain valid from one session to another. Job
control routines SHOULD work
correctly if a job Id came from a previous DRMAA session, provided the
current DRMAA session
knows how to resolve this job Id. The burden is on the user to match
previous job Id’s with
appropriate DRMAA sessions (i.e., DRMAA implementations). It is
restartable applications make job Id’s persistent in order to access the
already submitted jobs."

In sum, you need to persist your job id's with an own mechanism. It
SHOULD be allowed to re-use old job id's in a new session. This approach
ensures that the DRMAA library is not enforced to do any kind of data
If SGE works as described, than the "SHOULD" keyword from the
specification is ignored. If this was intentional, then I would be
interested in the reasons (Andreas ? Dan ?). I really agree to the
argumentation of Tim, so maybe the "SHOULD" keyword has to be a "SHALL".


Tim Harsch schrieb:

>    Someone who knows the spec better than I should probably confirm but I
>believe drmaa_job_ps is indeed only tracking the jobs submitted from within
>the session.  In my opinion, for drmaa_job_ps, it is indeed how the
>behaviour should be from a sort of object oriented perspective... however,
>that doesn't meand there shouldn't perhaps be a sessionless set of API
>functions designed for manipulating jobs on the grid as a whole.  I don't
>know if this has been discussed for DRMAA before or not.
>    I can see how this would be a problem for a set of CGIs, since each are
>a new process you'll not be able to maintain the drmaa_session accross
>processes.  I don't know if there is a way around it...
>    You might have to consider maintaining your job ids in a persistent
>store of your own and backtick calls to qsub, etc.  Its ugly in that you'll
>have the process overhead each time, but you'll get more of the SGE
>features that way.
>    Perhaps the group can help tell us if there is a way to set up a
>persistent DRMAA session?
>    Wouldn't this problem also be a problem for Globus?  At one of the
>conferences they were saying that Globus would (in the future) perhaps use
>DRMAA as the tie in to the DRM, seems like without persistent sessions this
>might be a bit difficult in practice.
>----- Original Message ----- 
>From: "Áncor González Sosa" <ancorgs at softhome.net>
>To: <drmaa-wg at ggf.org>
>Sent: Wednesday, March 02, 2005 3:34 PM
>Subject: [drmaa-wg] SGE, drmaa_job_ps and DRMAA sessions
>I'm developing a webmin module for submiting jobs and fetching the results
>a cluster with Sun Grid Engine 6.0u1. I'm using Scheduler::DRMAAc, the Perl
>wrapper of the C binding of the DRMAA specification.
>My problem is that drmaa_job_ps() only works for jobs that have been
>in the current DRMAA session. Is this the expected behavior in any DRMAA
>implementation or is a feature/bug of SGE 6? I'm not sure, even after
>the DRMAA 1.0 specification paper. If it is the normal behavior of a DRMAA
>system, I find DRMAA really useless for "unattended" execution.

More information about the drmaa-wg mailing list