pgi-wg

Production Grid Infrastructure WG

Architecture

The objective of this working group is to formulate a well-defined set of profiles, and additional specifications if needed, for job and data management that are aligned with a Grid security and information model that addresses the needs of production grid infrastructures.

The origin of this working group is from the Grid Interoperation Now (GIN) community group. The GIN-CG has representatives from production grid infrastructures (such as DEISA, EGEE, OSG, NAREGI, NorduGrid, TeraGrid, NGS, etc) and supporting organizations (such as OGF-Europe, OMII-Europe, OMII-UK, etc.). Over the last two years the members of GIN that are working daily in world-wide production grids have been working within the OGF to explore the interoperability issues in:
* jobs submission and job monitoring
* data access and movement
* authentication and authorization
* information representation and discovery
As a result of this collaborative activity the GIN-CG now feels there is sufficient understanding and motivation within the group to define a set of profiles around a well-defined use case. This use case should use open standards from different areas of OGF to facilitate interoperability among production Grid infrastructures. This is the focus of this working group.


This basic use case identified in production Grids is as follows: on behalf of an authorized user, a tool interrogates the information system (representing a grid or a single resource), locates an optimal execution resource, submits the job to the execution resource, which in turn interprets the submitted job description and locates and fetches the necessary input data from a remote storage - also on behalf of the user. Upon the completion, the newly created data is uploaded to a storage resource where this user is authorized (as a member of a Virtual Organization), registered in the necessary data indexing catalogs, and the job record is updated in the accounting and monitoring system.

Thus the milestones this working group must meet are particularly ambitious due to the needs of the production Grid infrastructures. There is a need to come to a common proven end-to-end interoperable definition of the services and their interactions within the next 18 months for the European middleware developers. This development work will take place within this time period – we feel it is imperative to attempt to do this work within OGF.

Scope
The scope of this working group is restricted to dealing with job and data management issues found in production grid infrastructures that together use a limited number of security specifications and information models. The resulting Web service interfaces and schemas defined by this working group will be a set of profiles of open standards and specifications such as OGSA-Basic Execution Services (GFD.108), Job Submission Description Language (GFD.136), GridFTP (GFD.20), Storage Resource Manager (GFD.129), and efforts of the GLUE2 (GFD.147) working group. The additional work bv this group will go beyond the current specifications and extensions to include an authentication and authorization model, an extended state model, an integrated resource and informational model, and a data transfer and storage model – all tuned to the use cases found in production grid infrastructures.

Goals
This group will deliver the following document:

* Production Grid Infrastructure Roadmap Document (GFD-I)
* Provides an overview of missing links between open standards and the fundamental motivation for standardization of the production grid infrastructure profiles
* Secure Job and Data Management Profile in Production Grids (GFD-R.P)
* Develop a job/data/security profile assuming an deployed computing endpoint is already known by an end-user
* Secure Information Profile in Production Grids (GFD-R-P)
* Allows a user to discover resources that are appropriate for their request and that they are authorized to access
* Secure Accounting Profile in Production Grids (GFD-R-P)
* Develop an accounting profile that allows services to securely update a service within information about the resources they have used on behalf of a user.
* Secure Monitoring Profile in Production Grids (GFD-R-P)
* Develop a profile that allows services to securely record in a service the progress of an activity.

When the recommendation documents are complete and have passed through necessary public comments and editor reviews, the working group will be dissolved.

Title:
Title:

1. Is the scope of the proposed group sufficiently focused?
The working group is focused just on formalizing the experience gained in batch job management within the deployed production grids taking commonly used data management and information systems technologies and security setups into account.

2. Are the topics that the group plans to address clear and relevant for the Grid research, development, industrial, implementation, and/or application user community?
Yes. Job and data management through a Web service or dedicated protocols (i.e. GridFTP) is central to all production grid infratructures.

3. Will the formation of the group foster (consensus-based) work that would not be done otherwise?
The requirements from the priduction grid communities to achieve interoperability is well established within the GIN-CG. Bringing this standardisation work into the OGF will provide a focus for the production grid comminity already represented within the OGF to do this work.

4. Do the group's activities overlap inappropriately with those of another OGF group or to a group active in another organization such as IETF or W3C?
No, this work will build upon work that has already taken place within the OGF through the BES, JSDL, SRM, GridFTP, and GLUE2 specifications. The HPC Basic Profile specification has addressed similar use cases around job submission for exposing single clusters for direct job submission. This work is however focused around the needs of production grid infrastructures which have different priorites and use cases to those of the HPC Basic Profile specification. This work is independent of the OGSA activities.

5. Are there sufficient interest and expertise in the group's topic, with at least several people willing to expend the effort that is likely to produce significant results over time?
Yes. The people involved have implemented and are running services of this type within their production grids. They have found that addtional standardisation is needed beyond the functionality captured within the HPC Basic Profile specification.Such standarisation is needed to support upcoming interoperability needs and they are therfore eager to drive forward this standardisation activity. The active participants in the working group (both within the leadership and those contributing technically to its work) have the support of their funding organisations and projects to work in this area.

6. Does a base of interested consumers (e.g., application developers, Grid system implementers, industry partners, end-users) appear to exist for the planned work?
Yes. This community is drawn from the software providers to the production grid community.

7. Does the OGF have a reasonable role to play in the determination of the technology?
Yes. In particular, the proposed work expects to build on existing OGF activities such as JSDL, BES, GridFTP, SRM, and GLUE2.

Active

The objective of this working group is to formulate a well-defined set of profiles, and additional specifications if needed, for job and data management that are aligned with a Grid security and information model that addresses the needs of production grid infrastructures.