From urbah at lal.in2p3.fr Thu Sep 1 13:45:21 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Thu, 01 Sep 2011 20:45:21 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service Message-ID: <4E5FD2C1.8050400@lal.in2p3.fr> Andrew, Steven and all OGSA-BES stakeholders, At the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011, we had agreed to propose for the beginning of September 2011 improvements to the current specification of BES 1.0 described in GFD.108 But I am unable to define specifications before having a clear picture of requirements. Therefore, I am currently writing down a document named 'Requirements for an improved Basic Execution Service (BES)'. This document is NOT finished, but the first 10 chapters are in a readable and consistent state. The current version of this document is in the attached 'BES-Improvements-Requirements.doc' file. Thank you in advance for reading and criticizing this document. When GridForge will be up again, I will save the latest version there. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: BES-Improvements-Requirements.doc Type: application/msword Size: 364032 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110901/e1210b45/attachment-0001.doc -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110901/e1210b45/attachment-0001.bin From urbah at lal.in2p3.fr Wed Sep 7 12:33:27 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Wed, 07 Sep 2011 19:33:27 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <4E5FD1DE.6000404@lal.in2p3.fr> References: <4E5FD1DE.6000404@lal.in2p3.fr> Message-ID: <4E67AAE7.9070606@lal.in2p3.fr> Andrew, Steven and all OGSA-BES stakeholders, At the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011, we had agreed to propose for the beginning of September 2011 improvements to the current specification of BES 1.0 described in GFD.108 But I am unable to define specifications before having a clear picture of requirements. Therefore, I have written down a document named 'Requirements for an improved Basic Execution Service (BES)' and available at http://forge.gridforum.org/sf/go/doc16306 This document is almost finished, with most chapters in a readable and consistent state. The only exception is the chapter about data management, which I hope to write down very soon. Thank you in advance for reading and criticizing this document. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- On Thu, 01/09/2011 20:41, Etienne URBAH wrote: > Andrew, Steven and all OGSA-BES stakeholders, > > At the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011, we > had agreed to propose for the beginning of September 2011 improvements > to the current specification of BES 1.0 described in GFD.108 > > But I am unable to define specifications before having a clear picture > of requirements. > > Therefore, I am currently writing down a document named 'Requirements > for an improved Basic Execution Service (BES)'. > > This document is NOT finished, but the first 10 chapters are in a > readable and consistent state. > > The current version of this document is in the attached > 'BES-Improvements-Requirements.doc' file. > > Thank you in advance for reading and criticizing this document. > > When GridForge will be up again, I will save the latest version there. > > Best regards. > > ----------------------------------------------------- > Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS > Bat 200 91898 ORSAY France > Tel: +33 1 64 46 84 87 Skype: etienne.urbah > Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr > ----------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110907/4b3f9f0b/attachment.bin From m.riedel at fz-juelich.de Thu Sep 8 05:51:27 2011 From: m.riedel at fz-juelich.de (Morris Riedel) Date: Thu, 8 Sep 2011 12:51:27 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <4E67AAE7.9070606@lal.in2p3.fr> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> Message-ID: <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> Hi Etienne, thanks for your input. At the next OGF, we will have several PGI/BES/JSDL/GLUE2. We thus not only are able to clarify questions, but also will make a progress towards commonly agreed spec based on all of our inputs. Take care, Morris >-- -----Urspr?ngliche Nachricht----- >-- Von: ogsa-bes-wg-bounces at ogf.org [mailto:ogsa-bes-wg-bounces at ogf.org] Im Auftrag von Etienne URBAH >-- Gesendet: Mittwoch, 7. September 2011 19:33 >-- An: Andrew GRIMSHAW; steven.newhouse at egi.eu; ogsa-bes-wg at ogf.org >-- Cc: pgi-wg at ogf.org; edgi-na2 at mail.edgi-project.eu; lodygens at lal.in2p3.fr >-- Betreff: Re: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service >-- >-- Andrew, Steven and all OGSA-BES stakeholders, >-- >-- At the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011, we >-- had agreed to propose for the beginning of September 2011 improvements >-- to the current specification of BES 1.0 described in GFD.108 >-- >-- But I am unable to define specifications before having a clear picture >-- of requirements. >-- >-- Therefore, I have written down a document named 'Requirements for an >-- improved Basic Execution Service (BES)' and available at >-- http://forge.gridforum.org/sf/go/doc16306 >-- >-- This document is almost finished, with most chapters in a readable and >-- consistent state. >-- The only exception is the chapter about data management, which I hope to >-- write down very soon. >-- >-- Thank you in advance for reading and criticizing this document. >-- >-- Best regards. >-- >-- ----------------------------------------------------- >-- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >-- Bat 200 91898 ORSAY France >-- Tel: +33 1 64 46 84 87 Skype: etienne.urbah >-- Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >-- ----------------------------------------------------- >-- >-- >-- On Thu, 01/09/2011 20:41, Etienne URBAH wrote: >-- > Andrew, Steven and all OGSA-BES stakeholders, >-- > >-- > At the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011, we >-- > had agreed to propose for the beginning of September 2011 improvements >-- > to the current specification of BES 1.0 described in GFD.108 >-- > >-- > But I am unable to define specifications before having a clear picture >-- > of requirements. >-- > >-- > Therefore, I am currently writing down a document named 'Requirements >-- > for an improved Basic Execution Service (BES)'. >-- > >-- > This document is NOT finished, but the first 10 chapters are in a >-- > readable and consistent state. >-- > >-- > The current version of this document is in the attached >-- > 'BES-Improvements-Requirements.doc' file. >-- > >-- > Thank you in advance for reading and criticizing this document. >-- > >-- > When GridForge will be up again, I will save the latest version there. >-- > >-- > Best regards. >-- > >-- > ----------------------------------------------------- >-- > Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >-- > Bat 200 91898 ORSAY France >-- > Tel: +33 1 64 46 84 87 Skype: etienne.urbah >-- > Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >-- > ----------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3550 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110908/a8fcf0c0/attachment.bin From urbah at lal.in2p3.fr Fri Sep 9 15:51:53 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Fri, 09 Sep 2011 22:51:53 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> Message-ID: <4E6A7C69.9010308@lal.in2p3.fr> Andrew, Steven and all OGSA-BES stakeholders, I have finished written down a document named 'Requirements for an improved Basic Execution Service (BES)' and available at http://forge.gridforum.org/sf/go/doc16306 I will use this requirements document to propose improvements to the current specification of BES 1.0 described in GFD.108, as agreed at the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011. Thank you in advance for reading and criticizing my requirements document. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- On Thu, 08/09/2011 12:51, Morris Riedel wrote: > Hi Etienne, > > thanks for your input. > > At the next OGF, we will have several PGI/BES/JSDL/GLUE2. > > We thus not only are able to clarify questions, but also will make a progress towards commonly agreed spec based on all of our inputs. > > Take care, > Morris > > >> -----Urspr?ngliche Nachricht----- >> Von: ogsa-bes-wg-bounces at ogf.org [mailto:ogsa-bes-wg-bounces at ogf.org] Im Auftrag von Etienne URBAH >> Gesendet: Mittwoch, 7. September 2011 19:33 >> An: Andrew GRIMSHAW; steven.newhouse at egi.eu; ogsa-bes-wg at ogf.org >> Cc: pgi-wg at ogf.org; edgi-na2 at mail.edgi-project.eu; lodygens at lal.in2p3.fr >> Betreff: Re: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service >> >> Andrew, Steven and all OGSA-BES stakeholders, >> >> At the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011, we >> had agreed to propose for the beginning of September 2011 improvements >> to the current specification of BES 1.0 described in GFD.108 >> >> But I am unable to define specifications before having a clear picture >> of requirements. >> >> Therefore, I have written down a document named 'Requirements for an >> improved Basic Execution Service (BES)' and available at >> http://forge.gridforum.org/sf/go/doc16306 >> >> This document is almost finished, with most chapters in a readable and >> consistent state. >> The only exception is the chapter about data management, which I hope to >> write down very soon. >> >> Thank you in advance for reading and criticizing this document. >> >> Best regards. >> >> ----------------------------------------------------- >> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >> Bat 200 91898 ORSAY France >> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >> ----------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110909/2a87a267/attachment-0001.bin From urbah at lal.in2p3.fr Mon Sep 12 11:33:26 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Mon, 12 Sep 2011 18:33:26 +0200 Subject: [OGSA-BES-WG] Fwd: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) In-Reply-To: <201109121558.p8CFwM24004923@evo04.caltech.edu> References: <201109121558.p8CFwM24004923@evo04.caltech.edu> Message-ID: <4E6E3456.200@lal.in2p3.fr> Philipp and all, Concerning the preparation of the PGI / BES / JSDL sessions at OGF 33 in Lyon next week : I booked an EVO teleconference named 'OGF preparation of PGI / BES / JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and Oracle Java 1.7.0, so that you may have to use EVO beta at http://evo.caltech.edu/evoBeta/ which I tested successfully. Access to the EVO teleconference will be provided at http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s with password !!jsdlftw Thanks in advance to Philipp for providing the agenda. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- -------- Original Message -------- Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) Date: Mon, 12 Sep 11 17:58:20 +0200 From: evosupport at vrvs.org To: urbah at lal.in2p3.fr Hello Etienne URBAH, You have BOOKED a meeting in EVO (http://evo.caltech.edu). Title: OGF preparation of PGI / BES / JSDL Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in Lyon next week Community: Universe Password: !!jsdlftw Meeting Access Information: - Meeting URL http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s - Password: !!jsdlftw - Phone Bridge ID: 399 7332 Password: 0162 Central European Summer Time (+0200) Start 2011-09-14 16:00 End 2011-09-14 18:00 Japan Standard Time (+0900) Start 2011-09-14 23:00 End 2011-09-15 01:00 Eastern Daylight Time (-0400) Start 2011-09-14 10:00 End 2011-09-14 12:00 Pacific Daylight Time (-0700) Start 2011-09-14 07:00 End 2011-09-14 09:00 EVO Phone Bridge Telephone Numbers: --------------- - USA (Caltech, Pasadena, CA) +1 626 395 2112 - Switzerland (CERN, Geneva) +41 22 76 71400 - Slovakia (UPJS, Kosice) +421 55 234 2420 - Italy (INFN, several cities) http://server10.infn.it/video/index.php?page=telephone_numbers Enter '4000' to access the EVO bridge - Germany (DESY, Hamburg) +49 40 8998 1340 - USA (BNL, Upton, NY) +1 631 344 6100 - United Kingdom (University of Manchester) +44 161 306 6802 - Australia (ARCS) +61 Adelaide 08 8463 1011 Brisbane 07 3139 0705 Canberra 02 6112 8742 Hobart 03 623 70281 Melbourne 03 8685 8362 Perth 08 6461 6718 Sydney 02 8212 4591 - Netherlands (Nikhef, Amsterdam) +31 20 7165293 Dial '2' at the prompt - Canada (TRIUMF, Vancouver) +1 604 222 7700 - Czech Republic (CESNET, Prague) +420 95 007 2386 - USA (MIT, Cambridge, MA) +1 617 715 4691 - France (RAP, Paris) +33 144 27 81 50 - Skype (tm) (World-wide) evo.phone See: http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html -------------- next part -------------- A non-text attachment was scrubbed... Name: EVO_meeting_1316008800000.ics Type: text/calendar Size: 813 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110912/f6c67918/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110912/f6c67918/attachment-0001.bin From philipp.wieder at tu-dortmund.de Tue Sep 13 13:29:48 2011 From: philipp.wieder at tu-dortmund.de (philipp.wieder at tu-dortmund.de) Date: Tue, 13 Sep 2011 18:29:48 +0000 Subject: [OGSA-BES-WG] EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) In-Reply-To: <4E6E3456.200@lal.in2p3.fr> Message-ID: Dear Etienne, thank you very much for setting this up. The agenda for tomorrow is brief: - Assess status of input material for the whole day BES/JSDL/GLUE/PGI session in Lyon (some material is already here [1]) - Define tentative agenda for the respective session including + Objectives + Actions + Responsibilities + Roadmap - Check availability for Lyon - AOB Since I am traveling tomorrow I may not be able to join (I will try via a mobile connection). Etienne, is it possible for you to lead the conference in this case and take minutes? Best regards, Philipp. [1] https://forge.gridforum.org/sf/docman/do/listDocuments/projects.jsdl-wg/doc man.root.background_docs.2011 Am 12.09.11 18:33 schrieb "Etienne URBAH" unter : >Philipp and all, > >Concerning the preparation of the PGI / BES / JSDL sessions at OGF 33 in >Lyon next week : > >I booked an EVO teleconference named 'OGF preparation of PGI / BES / >JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). > >BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and >Oracle Java 1.7.0, so that you may have to use EVO beta at >http://evo.caltech.edu/evoBeta/ which I tested successfully. > >Access to the EVO teleconference will be provided at >http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >with password !!jsdlftw > >Thanks in advance to Philipp for providing the agenda. > >Best regards. > >----------------------------------------------------- >Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS > Bat 200 91898 ORSAY France >Tel: +33 1 64 46 84 87 Skype: etienne.urbah >Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >----------------------------------------------------- > > >-------- Original Message -------- >Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 >Sep 11 16:00 +0200 (CEST) >Date: Mon, 12 Sep 11 17:58:20 +0200 >From: evosupport at vrvs.org >To: urbah at lal.in2p3.fr > >Hello Etienne URBAH, >You have BOOKED a meeting in EVO (http://evo.caltech.edu). > >Title: OGF preparation of PGI / BES / JSDL >Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in >Lyon next week >Community: Universe >Password: !!jsdlftw > >Meeting Access Information: >- Meeting URL > http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s > >- Password: !!jsdlftw >- Phone Bridge > ID: 399 7332 > Password: 0162 > >Central European Summer Time (+0200) > Start 2011-09-14 16:00 > End 2011-09-14 18:00 > >Japan Standard Time (+0900) > Start 2011-09-14 23:00 > End 2011-09-15 01:00 > >Eastern Daylight Time (-0400) > Start 2011-09-14 10:00 > End 2011-09-14 12:00 > >Pacific Daylight Time (-0700) > Start 2011-09-14 07:00 > End 2011-09-14 09:00 > > >EVO Phone Bridge Telephone Numbers: >--------------- >- USA (Caltech, Pasadena, CA) > +1 626 395 2112 > >- Switzerland (CERN, Geneva) > +41 22 76 71400 > >- Slovakia (UPJS, Kosice) > +421 55 234 2420 > >- Italy (INFN, several cities) > http://server10.infn.it/video/index.php?page=telephone_numbers > Enter '4000' to access the EVO bridge > >- Germany (DESY, Hamburg) > +49 40 8998 1340 > >- USA (BNL, Upton, NY) > +1 631 344 6100 > >- United Kingdom (University of Manchester) > +44 161 306 6802 > >- Australia (ARCS) > +61 > Adelaide 08 8463 1011 > Brisbane 07 3139 0705 > Canberra 02 6112 8742 > Hobart 03 623 70281 > Melbourne 03 8685 8362 > Perth 08 6461 6718 > Sydney 02 8212 4591 > >- Netherlands (Nikhef, Amsterdam) > +31 20 7165293 > Dial '2' at the prompt > >- Canada (TRIUMF, Vancouver) > +1 604 222 7700 > >- Czech Republic (CESNET, Prague) > +420 95 007 2386 > >- USA (MIT, Cambridge, MA) > +1 617 715 4691 > >- France (RAP, Paris) > +33 144 27 81 50 > > >- Skype (tm) (World-wide) > evo.phone > See: >http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html > From urbah at lal.in2p3.fr Tue Sep 13 13:51:12 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Tue, 13 Sep 2011 20:51:12 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <4E6A7C69.9010308@lal.in2p3.fr> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> <4E6A7C69.9010308@lal.in2p3.fr> Message-ID: <4E6FA620.8020307@lal.in2p3.fr> Andrew, Steven and all OGSA-BES stakeholders, In the document named 'Requirements for an improved Basic Execution Service (BES)' and available at http://forge.gridforum.org/sf/go/doc16306 I have performed following improvements : - For each functionality, clearly separate : - the requirements about the corresponding Client interface (often 'MUST'), - the requirements about the implementation of the functionality (sometimes 'MAY'). - Add short titles in bold to improve readability. Thank you in advance for reading and criticizing this requirements document. Best regards. Etienne URBAH On Fri, 09/09/2011 22:51, Etienne URBAH wrote: > Andrew, Steven and all OGSA-BES stakeholders, > > I have finished written down a document named 'Requirements for an > improved Basic Execution Service (BES)' and available at > http://forge.gridforum.org/sf/go/doc16306 > > I will use this requirements document to propose improvements to the > current specification of BES 1.0 described in GFD.108, as agreed at the > OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011. > > Thank you in advance for reading and criticizing my requirements document. > > Best regards. > > ----------------------------------------------------- > Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS > Bat 200 91898 ORSAY France > Tel: +33 1 64 46 84 87 Skype: etienne.urbah > Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr > ----------------------------------------------------- > > > On Thu, 08/09/2011 12:51, Morris Riedel wrote: >> Hi Etienne, >> >> thanks for your input. >> >> At the next OGF, we will have several PGI/BES/JSDL/GLUE2. >> >> We thus not only are able to clarify questions, but also will make a >> progress towards commonly agreed spec based on all of our inputs. >> >> Take care, >> Morris >> >> >>> -----Urspr??ngliche Nachricht----- >>> Von: ogsa-bes-wg-bounces at ogf.org [mailto:ogsa-bes-wg-bounces at ogf.org] >>> Im Auftrag von Etienne URBAH >>> Gesendet: Mittwoch, 7. September 2011 19:33 >>> An: Andrew GRIMSHAW; steven.newhouse at egi.eu; ogsa-bes-wg at ogf.org >>> Cc: pgi-wg at ogf.org; edgi-na2 at mail.edgi-project.eu; lodygens at lal.in2p3.fr >>> Betreff: Re: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an >>> improved Basic Execution Service >>> >>> Andrew, Steven and all OGSA-BES stakeholders, >>> >>> At the OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011, we >>> had agreed to propose for the beginning of September 2011 improvements >>> to the current specification of BES 1.0 described in GFD.108 >>> >>> But I am unable to define specifications before having a clear picture >>> of requirements. >>> >>> Therefore, I have written down a document named 'Requirements for an >>> improved Basic Execution Service (BES)' and available at >>> http://forge.gridforum.org/sf/go/doc16306 >>> >>> This document is almost finished, with most chapters in a readable and >>> consistent state. >>> The only exception is the chapter about data management, which I hope to >>> write down very soon. >>> >>> Thank you in advance for reading and criticizing this document. >>> >>> Best regards. >>> >>> ----------------------------------------------------- >>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >>> Bat 200 91898 ORSAY France >>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >>> ----------------------------------------------------- > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110913/0ea46c10/attachment.bin From urbah at lal.in2p3.fr Tue Sep 13 14:26:45 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Tue, 13 Sep 2011 21:26:45 +0200 Subject: [OGSA-BES-WG] EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) In-Reply-To: References: Message-ID: <4E6FAE75.6070906@lal.in2p3.fr> Philipp, Steven, Morris, Andrew and all, Concerning the EVO teleconference named 'OGF preparation of PGI / BES / JSDL' on Wednesday 14 September 2011 at 16h CEST (14h UTC) : - I remind everyone that it will be held this Wednesday at http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s with password !!jsdlftw - The scope of the agenda is NOT limited to JSDL, but really encompasses BES / JSDL / GLUE / PGI. - Even if I have technical knowledge on all these subjects, I am NOT highly talented as group leader, and I would greatly prefer that the conference leader is a chair of one of the above OGF working groups. In particular, if one of them participates, I would suggest Steven NEWHOUSE, Morris RIEDEL or Andrew GRIMSHAW. - I am quite slow at taking notes, and I would be able to perform this task only if I am NOT the conference leader. Anyway, can anyone volunteer taking notes ? - As input material, I suggest to read 'Requirements for an improved Basic Execution Service (BES)' at http://forge.gridforum.org/sf/go/doc16306 Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- On Tue, 13/09/2011 20:29, philipp.wieder at tu-dortmund.de wrote: > Dear Etienne, > > thank you very much for setting this up. > > The agenda for tomorrow is brief: > - Assess status of input material for the whole day BES/JSDL/GLUE/PGI > session in Lyon (some material is already here [1]) > - Define tentative agenda for the respective session including > + Objectives > + Actions > + Responsibilities > + Roadmap > - Check availability for Lyon > - AOB > > Since I am traveling tomorrow I may not be able to join (I will try via a > mobile connection). Etienne, is it possible for you to lead the conference > in this case and take minutes? > > Best regards, Philipp. > > > [1] > https://forge.gridforum.org/sf/docman/do/listDocuments/projects.jsdl-wg/doc > man.root.background_docs.2011 > > Am 12.09.11 18:33 schrieb "Etienne URBAH" unter: > >> Philipp and all, >> >> Concerning the preparation of the PGI / BES / JSDL sessions at OGF 33 in >> Lyon next week : >> >> I booked an EVO teleconference named 'OGF preparation of PGI / BES / >> JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). >> >> BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and >> Oracle Java 1.7.0, so that you may have to use EVO beta at >> http://evo.caltech.edu/evoBeta/ which I tested successfully. >> >> Access to the EVO teleconference will be provided at >> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >> with password !!jsdlftw >> >> Thanks in advance to Philipp for providing the agenda. >> >> Best regards. >> >> ----------------------------------------------------- >> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >> Bat 200 91898 ORSAY France >> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >> ----------------------------------------------------- >> >> >> -------- Original Message -------- >> Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 >> Sep 11 16:00 +0200 (CEST) >> Date: Mon, 12 Sep 11 17:58:20 +0200 >> From: evosupport at vrvs.org >> To: urbah at lal.in2p3.fr >> >> Hello Etienne URBAH, >> You have BOOKED a meeting in EVO (http://evo.caltech.edu). >> >> Title: OGF preparation of PGI / BES / JSDL >> Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in >> Lyon next week >> Community: Universe >> Password: !!jsdlftw >> >> Meeting Access Information: >> - Meeting URL >> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >> >> - Password: !!jsdlftw >> - Phone Bridge >> ID: 399 7332 >> Password: 0162 >> >> Central European Summer Time (+0200) >> Start 2011-09-14 16:00 >> End 2011-09-14 18:00 >> >> Japan Standard Time (+0900) >> Start 2011-09-14 23:00 >> End 2011-09-15 01:00 >> >> Eastern Daylight Time (-0400) >> Start 2011-09-14 10:00 >> End 2011-09-14 12:00 >> >> Pacific Daylight Time (-0700) >> Start 2011-09-14 07:00 >> End 2011-09-14 09:00 >> >> >> EVO Phone Bridge Telephone Numbers: >> --------------- >> - USA (Caltech, Pasadena, CA) >> +1 626 395 2112 >> >> - Switzerland (CERN, Geneva) >> +41 22 76 71400 >> >> - Slovakia (UPJS, Kosice) >> +421 55 234 2420 >> >> - Italy (INFN, several cities) >> http://server10.infn.it/video/index.php?page=telephone_numbers >> Enter '4000' to access the EVO bridge >> >> - Germany (DESY, Hamburg) >> +49 40 8998 1340 >> >> - USA (BNL, Upton, NY) >> +1 631 344 6100 >> >> - United Kingdom (University of Manchester) >> +44 161 306 6802 >> >> - Australia (ARCS) >> +61 >> Adelaide 08 8463 1011 >> Brisbane 07 3139 0705 >> Canberra 02 6112 8742 >> Hobart 03 623 70281 >> Melbourne 03 8685 8362 >> Perth 08 6461 6718 >> Sydney 02 8212 4591 >> >> - Netherlands (Nikhef, Amsterdam) >> +31 20 7165293 >> Dial '2' at the prompt >> >> - Canada (TRIUMF, Vancouver) >> +1 604 222 7700 >> >> - Czech Republic (CESNET, Prague) >> +420 95 007 2386 >> >> - USA (MIT, Cambridge, MA) >> +1 617 715 4691 >> >> - France (RAP, Paris) >> +33 144 27 81 50 >> >> >> - Skype (tm) (World-wide) >> evo.phone >> See: >> http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110913/c63c14a5/attachment-0001.bin From grimshaw at virginia.edu Tue Sep 13 14:57:39 2011 From: grimshaw at virginia.edu (Andrew Grimshaw) Date: Tue, 13 Sep 2011 15:57:39 -0400 Subject: [OGSA-BES-WG] [jsdl-wg] EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) In-Reply-To: References: <4E6E3456.200@lal.in2p3.fr> Message-ID: <032a01cc724f$64d5fc70$2e81f550$@edu> All, I will not be able to attend either. In terms of agenda for the F2F I suggest Generally put together the laundry list of things and how they relate to the requirements, e.g., GLUE2? Which rendering? OR - as an element Matching parameters More clarity on file systems A -----Original Message----- From: jsdl-wg-bounces at ogf.org [mailto:jsdl-wg-bounces at ogf.org] On Behalf Of philipp.wieder at tu-dortmund.de Sent: Tuesday, September 13, 2011 2:30 PM To: urbah at lal.in2p3.fr; jsdl-wg at ogf.org Cc: ogsa-bes-wg at ogf.org; edgi-na2 at mail.edgi-project.eu; lodygens at lal.in2p3.fr; pgi-wg at ogf.org Subject: Re: [jsdl-wg] EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) Dear Etienne, thank you very much for setting this up. The agenda for tomorrow is brief: - Assess status of input material for the whole day BES/JSDL/GLUE/PGI session in Lyon (some material is already here [1]) - Define tentative agenda for the respective session including + Objectives + Actions + Responsibilities + Roadmap - Check availability for Lyon - AOB Since I am traveling tomorrow I may not be able to join (I will try via a mobile connection). Etienne, is it possible for you to lead the conference in this case and take minutes? Best regards, Philipp. [1] https://forge.gridforum.org/sf/docman/do/listDocuments/projects.jsdl-wg/doc man.root.background_docs.2011 Am 12.09.11 18:33 schrieb "Etienne URBAH" unter : >Philipp and all, > >Concerning the preparation of the PGI / BES / JSDL sessions at OGF 33 in >Lyon next week : > >I booked an EVO teleconference named 'OGF preparation of PGI / BES / >JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). > >BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and >Oracle Java 1.7.0, so that you may have to use EVO beta at >http://evo.caltech.edu/evoBeta/ which I tested successfully. > >Access to the EVO teleconference will be provided at >http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >with password !!jsdlftw > >Thanks in advance to Philipp for providing the agenda. > >Best regards. > >----------------------------------------------------- >Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS > Bat 200 91898 ORSAY France >Tel: +33 1 64 46 84 87 Skype: etienne.urbah >Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >----------------------------------------------------- > > >-------- Original Message -------- >Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 >Sep 11 16:00 +0200 (CEST) >Date: Mon, 12 Sep 11 17:58:20 +0200 >From: evosupport at vrvs.org >To: urbah at lal.in2p3.fr > >Hello Etienne URBAH, >You have BOOKED a meeting in EVO (http://evo.caltech.edu). > >Title: OGF preparation of PGI / BES / JSDL >Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in >Lyon next week >Community: Universe >Password: !!jsdlftw > >Meeting Access Information: >- Meeting URL > http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s > >- Password: !!jsdlftw >- Phone Bridge > ID: 399 7332 > Password: 0162 > >Central European Summer Time (+0200) > Start 2011-09-14 16:00 > End 2011-09-14 18:00 > >Japan Standard Time (+0900) > Start 2011-09-14 23:00 > End 2011-09-15 01:00 > >Eastern Daylight Time (-0400) > Start 2011-09-14 10:00 > End 2011-09-14 12:00 > >Pacific Daylight Time (-0700) > Start 2011-09-14 07:00 > End 2011-09-14 09:00 > > >EVO Phone Bridge Telephone Numbers: >--------------- >- USA (Caltech, Pasadena, CA) > +1 626 395 2112 > >- Switzerland (CERN, Geneva) > +41 22 76 71400 > >- Slovakia (UPJS, Kosice) > +421 55 234 2420 > >- Italy (INFN, several cities) > http://server10.infn.it/video/index.php?page=telephone_numbers > Enter '4000' to access the EVO bridge > >- Germany (DESY, Hamburg) > +49 40 8998 1340 > >- USA (BNL, Upton, NY) > +1 631 344 6100 > >- United Kingdom (University of Manchester) > +44 161 306 6802 > >- Australia (ARCS) > +61 > Adelaide 08 8463 1011 > Brisbane 07 3139 0705 > Canberra 02 6112 8742 > Hobart 03 623 70281 > Melbourne 03 8685 8362 > Perth 08 6461 6718 > Sydney 02 8212 4591 > >- Netherlands (Nikhef, Amsterdam) > +31 20 7165293 > Dial '2' at the prompt > >- Canada (TRIUMF, Vancouver) > +1 604 222 7700 > >- Czech Republic (CESNET, Prague) > +420 95 007 2386 > >- USA (MIT, Cambridge, MA) > +1 617 715 4691 > >- France (RAP, Paris) > +33 144 27 81 50 > > >- Skype (tm) (World-wide) > evo.phone > See: >http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html > -- jsdl-wg mailing list jsdl-wg at ogf.org http://www.ogf.org/mailman/listinfo/jsdl-wg From urbah at lal.in2p3.fr Wed Sep 14 09:08:27 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Wed, 14 Sep 2011 16:08:27 +0200 Subject: [OGSA-BES-WG] [jsdl-wg] EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 Sep 11 16:00 +0200 (CEST) In-Reply-To: <4E6FAE75.6070906@lal.in2p3.fr> References: <4E6FAE75.6070906@lal.in2p3.fr> Message-ID: <4E70B55B.7010405@lal.in2p3.fr> On Tue, 13/09/2011 21:26, Etienne URBAH wrote: > Philipp, Steven, Morris, Andrew and all, > > Concerning the EVO teleconference named 'OGF preparation of PGI / BES / > JSDL' on Wednesday 14 September 2011 at 16h CEST (14h UTC) : > > - I remind everyone that it will be held this Wednesday at > http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s > with password !!jsdlftw THAT IS IMMEDIATELY. > > - The scope of the agenda is NOT limited to JSDL, but really encompasses > BES / JSDL / GLUE / PGI. > > - Even if I have technical knowledge on all these subjects, I am NOT > highly talented as group leader, and I would greatly prefer that the > conference leader is a chair of one of the above OGF working groups. > In particular, if one of them participates, I would suggest Steven > NEWHOUSE, Morris RIEDEL or Andrew GRIMSHAW. > > - I am quite slow at taking notes, and I would be able to perform this > task only if I am NOT the conference leader. > Anyway, can anyone volunteer taking notes ? > > - As input material, I suggest to read 'Requirements for an improved > Basic Execution Service (BES)' at http://forge.gridforum.org/sf/go/doc16306 > > Best regards. > > ----------------------------------------------------- > Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS > Bat 200 91898 ORSAY France > Tel: +33 1 64 46 84 87 Skype: etienne.urbah > Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr > ----------------------------------------------------- > > > On Tue, 13/09/2011 20:29, philipp.wieder at tu-dortmund.de wrote: >> Dear Etienne, >> >> thank you very much for setting this up. >> >> The agenda for tomorrow is brief: >> - Assess status of input material for the whole day BES/JSDL/GLUE/PGI >> session in Lyon (some material is already here [1]) >> - Define tentative agenda for the respective session including >> + Objectives >> + Actions >> + Responsibilities >> + Roadmap >> - Check availability for Lyon >> - AOB >> >> Since I am traveling tomorrow I may not be able to join (I will try via a >> mobile connection). Etienne, is it possible for you to lead the >> conference >> in this case and take minutes? >> >> Best regards, Philipp. >> >> >> [1] >> https://forge.gridforum.org/sf/docman/do/listDocuments/projects.jsdl-wg/doc >> >> man.root.background_docs.2011 >> >> Am 12.09.11 18:33 schrieb "Etienne URBAH" unter: >> >>> Philipp and all, >>> >>> Concerning the preparation of the PGI / BES / JSDL sessions at OGF 33 in >>> Lyon next week : >>> >>> I booked an EVO teleconference named 'OGF preparation of PGI / BES / >>> JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). >>> >>> BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and >>> Oracle Java 1.7.0, so that you may have to use EVO beta at >>> http://evo.caltech.edu/evoBeta/ which I tested successfully. >>> >>> Access to the EVO teleconference will be provided at >>> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >>> with password !!jsdlftw >>> >>> Thanks in advance to Philipp for providing the agenda. >>> >>> Best regards. >>> >>> ----------------------------------------------------- >>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >>> Bat 200 91898 ORSAY France >>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >>> ----------------------------------------------------- >>> >>> >>> -------- Original Message -------- >>> Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 >>> Sep 11 16:00 +0200 (CEST) >>> Date: Mon, 12 Sep 11 17:58:20 +0200 >>> From: evosupport at vrvs.org >>> To: urbah at lal.in2p3.fr >>> >>> Hello Etienne URBAH, >>> You have BOOKED a meeting in EVO (http://evo.caltech.edu). >>> >>> Title: OGF preparation of PGI / BES / JSDL >>> Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in >>> Lyon next week >>> Community: Universe >>> Password: !!jsdlftw >>> >>> Meeting Access Information: >>> - Meeting URL >>> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >>> >>> - Password: !!jsdlftw >>> - Phone Bridge >>> ID: 399 7332 >>> Password: 0162 >>> >>> Central European Summer Time (+0200) >>> Start 2011-09-14 16:00 >>> End 2011-09-14 18:00 >>> >>> Japan Standard Time (+0900) >>> Start 2011-09-14 23:00 >>> End 2011-09-15 01:00 >>> >>> Eastern Daylight Time (-0400) >>> Start 2011-09-14 10:00 >>> End 2011-09-14 12:00 >>> >>> Pacific Daylight Time (-0700) >>> Start 2011-09-14 07:00 >>> End 2011-09-14 09:00 >>> >>> >>> EVO Phone Bridge Telephone Numbers: >>> --------------- >>> - USA (Caltech, Pasadena, CA) >>> +1 626 395 2112 >>> >>> - Switzerland (CERN, Geneva) >>> +41 22 76 71400 >>> >>> - Slovakia (UPJS, Kosice) >>> +421 55 234 2420 >>> >>> - Italy (INFN, several cities) >>> http://server10.infn.it/video/index.php?page=telephone_numbers >>> Enter '4000' to access the EVO bridge >>> >>> - Germany (DESY, Hamburg) >>> +49 40 8998 1340 >>> >>> - USA (BNL, Upton, NY) >>> +1 631 344 6100 >>> >>> - United Kingdom (University of Manchester) >>> +44 161 306 6802 >>> >>> - Australia (ARCS) >>> +61 >>> Adelaide 08 8463 1011 >>> Brisbane 07 3139 0705 >>> Canberra 02 6112 8742 >>> Hobart 03 623 70281 >>> Melbourne 03 8685 8362 >>> Perth 08 6461 6718 >>> Sydney 02 8212 4591 >>> >>> - Netherlands (Nikhef, Amsterdam) >>> +31 20 7165293 >>> Dial '2' at the prompt >>> >>> - Canada (TRIUMF, Vancouver) >>> +1 604 222 7700 >>> >>> - Czech Republic (CESNET, Prague) >>> +420 95 007 2386 >>> >>> - USA (MIT, Cambridge, MA) >>> +1 617 715 4691 >>> >>> - France (RAP, Paris) >>> +33 144 27 81 50 >>> >>> >>> - Skype (tm) (World-wide) >>> evo.phone >>> See: >>> http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110914/f9a3295a/attachment.bin From urbah at lal.in2p3.fr Wed Sep 14 11:26:25 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Wed, 14 Sep 2011 18:26:25 +0200 Subject: [OGSA-BES-WG] OGF33 Lyon - BES/JSDL/GLUE/PGI session on Wednesday 21 September 2011 In-Reply-To: <4E70B55B.7010405@lal.in2p3.fr> References: <4E6FAE75.6070906@lal.in2p3.fr> <4E70B55B.7010405@lal.in2p3.fr> Message-ID: <4E70D5B1.20100@lal.in2p3.fr> Philipp, Steven, Morris, Andrew and all, Concerning the BES/JSDL/GLUE/PGI session at OGF33 in Lyon on Wednesday 21 September 2011 : Teleconference named 'OGF preparation of PGI / BES / JSDL' ---------------------------------------------------------- This teleconference was proposed by Philipp WIEDER for Wednesday 14 September 2011 at 16h CEST (14h UTC), and I only performed the booking at EVO. Even after 20 minutes waiting, the only participants were Michael SARAVO and myself. Michael said that he will ask Andrew GRIMSHAW to move the JSDL session from Monday 19 to Tuesday 20. After that, I closed the teleconference. Agenda of the BES/JSDL/GLUE/PGI session at OGF33 ------------------------------------------------ As input material, I propose : - http://forge.gridforum.org/sf/go/doc15990 PGI Useful Official and De facto Standards - http://www.ogf.org/documents/GFD.181.pdf OGF PGI Glossary of Acronyms and Terms - http://www.ogf.org/documents/GFD.180.pdf OGF PGI Use Case Collection - http://www.ogf.org/documents/GFD.180.pdf GLUE Specification 2.0 - http://www.ogf.org/documents/GFD.182.pdf The VOMS Attribute Certificate Format - http://www.ogf.org/documents/GFD.98.pdf Usage Record Format - http://www.ogf.org/documents/GFD.136.pdf JSDL 1.0 - http://www.ogf.org/documents/GFD.135.pdf HPC File Staging Profile 1.0 - http://www.ogf.org/documents/GFD.115.pdf JSDL SPMD Application Extension 1.0 - http://www.ogf.org/documents/GFD.111.pdf JSDL HPC Profile Application Extension 1.0 - http://www.ogf.org/documents/GFD.108.pdf OGSA-BES 1.0 - http://www.ogf.org/documents/GFD.114.pdf HPC Basic Profile 1.0 - http://www.ogf.org/documents/GFD.151.pdf HPCBP Advanced Filter Extension - http://forge.gridforum.org/sf/go/doc16306 Requirements for an improved BES - http://forge.gridforum.org/sf/go/doc16186 GLUE 2 Attributes for Use in JSDL Taking into account my experience at OGF, I consider that the main issue is : WHO is able to devote HOW MUCH TIME to real technical work, in particular to the review of input material ? Lot of thanks in advance to any volunteer. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- On Wed, 14/09/2011 16:08, Etienne URBAH wrote: > On Tue, 13/09/2011 21:26, Etienne URBAH wrote: >> Philipp, Steven, Morris, Andrew and all, >> >> Concerning the EVO teleconference named 'OGF preparation of PGI / BES / >> JSDL' on Wednesday 14 September 2011 at 16h CEST (14h UTC) : >> >> - I remind everyone that it will be held this Wednesday at >> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >> with password !!jsdlftw > THAT IS IMMEDIATELY. > >> >> - The scope of the agenda is NOT limited to JSDL, but really encompasses >> BES / JSDL / GLUE / PGI. >> >> - Even if I have technical knowledge on all these subjects, I am NOT >> highly talented as group leader, and I would greatly prefer that the >> conference leader is a chair of one of the above OGF working groups. >> In particular, if one of them participates, I would suggest Steven >> NEWHOUSE, Morris RIEDEL or Andrew GRIMSHAW. >> >> - I am quite slow at taking notes, and I would be able to perform this >> task only if I am NOT the conference leader. >> Anyway, can anyone volunteer taking notes ? >> >> - As input material, I suggest to read 'Requirements for an improved >> Basic Execution Service (BES)' at >> http://forge.gridforum.org/sf/go/doc16306 >> >> Best regards. >> >> ----------------------------------------------------- >> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >> Bat 200 91898 ORSAY France >> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >> ----------------------------------------------------- >> >> >> On Tue, 13/09/2011 20:29, philipp.wieder at tu-dortmund.de wrote: >>> Dear Etienne, >>> >>> thank you very much for setting this up. >>> >>> The agenda for tomorrow is brief: >>> - Assess status of input material for the whole day BES/JSDL/GLUE/PGI >>> session in Lyon (some material is already here [1]) >>> - Define tentative agenda for the respective session including >>> + Objectives >>> + Actions >>> + Responsibilities >>> + Roadmap >>> - Check availability for Lyon >>> - AOB >>> >>> Since I am traveling tomorrow I may not be able to join (I will try >>> via a >>> mobile connection). Etienne, is it possible for you to lead the >>> conference >>> in this case and take minutes? >>> >>> Best regards, Philipp. >>> >>> >>> [1] >>> https://forge.gridforum.org/sf/docman/do/listDocuments/projects.jsdl-wg/doc >>> >>> >>> man.root.background_docs.2011 >>> >>> Am 12.09.11 18:33 schrieb "Etienne URBAH" unter: >>> >>>> Philipp and all, >>>> >>>> Concerning the preparation of the PGI / BES / JSDL sessions at OGF >>>> 33 in >>>> Lyon next week : >>>> >>>> I booked an EVO teleconference named 'OGF preparation of PGI / BES / >>>> JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). >>>> >>>> BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and >>>> Oracle Java 1.7.0, so that you may have to use EVO beta at >>>> http://evo.caltech.edu/evoBeta/ which I tested successfully. >>>> >>>> Access to the EVO teleconference will be provided at >>>> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >>>> >>>> with password !!jsdlftw >>>> >>>> Thanks in advance to Philipp for providing the agenda. >>>> >>>> Best regards. >>>> >>>> ----------------------------------------------------- >>>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >>>> Bat 200 91898 ORSAY France >>>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >>>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >>>> ----------------------------------------------------- >>>> >>>> >>>> -------- Original Message -------- >>>> Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 >>>> Sep 11 16:00 +0200 (CEST) >>>> Date: Mon, 12 Sep 11 17:58:20 +0200 >>>> From: evosupport at vrvs.org >>>> To: urbah at lal.in2p3.fr >>>> >>>> Hello Etienne URBAH, >>>> You have BOOKED a meeting in EVO (http://evo.caltech.edu). >>>> >>>> Title: OGF preparation of PGI / BES / JSDL >>>> Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in >>>> Lyon next week >>>> Community: Universe >>>> Password: !!jsdlftw >>>> >>>> Meeting Access Information: >>>> - Meeting URL >>>> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >>>> >>>> >>>> - Password: !!jsdlftw >>>> - Phone Bridge >>>> ID: 399 7332 >>>> Password: 0162 >>>> >>>> Central European Summer Time (+0200) >>>> Start 2011-09-14 16:00 >>>> End 2011-09-14 18:00 >>>> >>>> Japan Standard Time (+0900) >>>> Start 2011-09-14 23:00 >>>> End 2011-09-15 01:00 >>>> >>>> Eastern Daylight Time (-0400) >>>> Start 2011-09-14 10:00 >>>> End 2011-09-14 12:00 >>>> >>>> Pacific Daylight Time (-0700) >>>> Start 2011-09-14 07:00 >>>> End 2011-09-14 09:00 >>>> >>>> >>>> EVO Phone Bridge Telephone Numbers: >>>> --------------- >>>> - USA (Caltech, Pasadena, CA) >>>> +1 626 395 2112 >>>> >>>> - Switzerland (CERN, Geneva) >>>> +41 22 76 71400 >>>> >>>> - Slovakia (UPJS, Kosice) >>>> +421 55 234 2420 >>>> >>>> - Italy (INFN, several cities) >>>> http://server10.infn.it/video/index.php?page=telephone_numbers >>>> Enter '4000' to access the EVO bridge >>>> >>>> - Germany (DESY, Hamburg) >>>> +49 40 8998 1340 >>>> >>>> - USA (BNL, Upton, NY) >>>> +1 631 344 6100 >>>> >>>> - United Kingdom (University of Manchester) >>>> +44 161 306 6802 >>>> >>>> - Australia (ARCS) >>>> +61 >>>> Adelaide 08 8463 1011 >>>> Brisbane 07 3139 0705 >>>> Canberra 02 6112 8742 >>>> Hobart 03 623 70281 >>>> Melbourne 03 8685 8362 >>>> Perth 08 6461 6718 >>>> Sydney 02 8212 4591 >>>> >>>> - Netherlands (Nikhef, Amsterdam) >>>> +31 20 7165293 >>>> Dial '2' at the prompt >>>> >>>> - Canada (TRIUMF, Vancouver) >>>> +1 604 222 7700 >>>> >>>> - Czech Republic (CESNET, Prague) >>>> +420 95 007 2386 >>>> >>>> - USA (MIT, Cambridge, MA) >>>> +1 617 715 4691 >>>> >>>> - France (RAP, Paris) >>>> +33 144 27 81 50 >>>> >>>> >>>> - Skype (tm) (World-wide) >>>> evo.phone >>>> See: >>>> http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html >>>> >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110914/97910068/attachment-0001.bin From m.riedel at fz-juelich.de Thu Sep 15 03:55:00 2011 From: m.riedel at fz-juelich.de (Morris Riedel) Date: Thu, 15 Sep 2011 10:55:00 +0200 Subject: [OGSA-BES-WG] OGF33 Lyon - BES/JSDL/GLUE/PGI session on Wednesday 21 September 2011 In-Reply-To: <4E70D5B1.20100@lal.in2p3.fr> References: <4E6FAE75.6070906@lal.in2p3.fr> <4E70B55B.7010405@lal.in2p3.fr> <4E70D5B1.20100@lal.in2p3.fr> Message-ID: <002c01cc7385$26dd8c80$7498a580$@riedel@fz-juelich.de> Dear Etienne, >-- Even after 20 minutes waiting, the only participants were Michael SARAVO >-- and myself. Well, I never agreed to this particular time slot nor confirmed my participation in this telcon where key folks already informed that they also not present. In next events this should be clarified and ensured that we are all there to make progress. Apart from this, you, me, and Michael discussed all relevant work already as part of PGI and from my perspective we agree to our major concepts/requirements (e.g. manual data-staging, etc.), but important is now that JSDL/BES chairs are represented to get specification work forward, including decisions! >-- WHO is able to devote HOW MUCH TIME to real technical work, in >-- particular to the review of input material ? I personally understand your frustration and we need to explore how we can change the situation in the next months/years across our groups. Perhaps as a first step, I'm able to state that EMI is in the process of re-structuring itself and as part of this process EMI members, including me, can devote much more time to real specification work as before. Although pending EMI structural agreements, I would assume that more time is available for our standardization process starting next month onwards. More at OGF next week. Take care, Morris >-- -----Urspr?ngliche Nachricht----- >-- Von: Etienne URBAH [mailto:urbah at lal.in2p3.fr] >-- Gesendet: Mittwoch, 14. September 2011 18:26 >-- An: philipp.wieder at tu-dortmund.de; steven.newhouse at egi.eu; Riedel, Morris; Andrew GRIMSHAW >-- Cc: jsdl-wg at ogf.org; ogsa-bes-wg at ogf.org; pgi-wg at ogf.org; glue-wg at ogf.org >-- Betreff: Re: OGF33 Lyon - BES/JSDL/GLUE/PGI session on Wednesday 21 September 2011 >-- >-- Philipp, Steven, Morris, Andrew and all, >-- >-- Concerning the BES/JSDL/GLUE/PGI session at OGF33 in Lyon on Wednesday >-- 21 September 2011 : >-- >-- >-- Teleconference named 'OGF preparation of PGI / BES / JSDL' >-- ---------------------------------------------------------- >-- This teleconference was proposed by Philipp WIEDER for Wednesday 14 >-- September 2011 at 16h CEST (14h UTC), and I only performed the booking >-- at EVO. >-- >-- Even after 20 minutes waiting, the only participants were Michael SARAVO >-- and myself. >-- >-- Michael said that he will ask Andrew GRIMSHAW to move the JSDL session >-- from Monday 19 to Tuesday 20. >-- >-- After that, I closed the teleconference. >-- >-- >-- Agenda of the BES/JSDL/GLUE/PGI session at OGF33 >-- ------------------------------------------------ >-- As input material, I propose : >-- >-- - http://forge.gridforum.org/sf/go/doc15990 PGI Useful Official and De >-- facto Standards >-- >-- - http://www.ogf.org/documents/GFD.181.pdf OGF PGI Glossary of >-- Acronyms and Terms >-- - http://www.ogf.org/documents/GFD.180.pdf OGF PGI Use Case Collection >-- >-- - http://www.ogf.org/documents/GFD.180.pdf GLUE Specification 2.0 >-- - http://www.ogf.org/documents/GFD.182.pdf The VOMS Attribute >-- Certificate Format >-- - http://www.ogf.org/documents/GFD.98.pdf Usage Record Format >-- >-- - http://www.ogf.org/documents/GFD.136.pdf JSDL 1.0 >-- - http://www.ogf.org/documents/GFD.135.pdf HPC File Staging Profile 1.0 >-- - http://www.ogf.org/documents/GFD.115.pdf JSDL SPMD Application >-- Extension 1.0 >-- - http://www.ogf.org/documents/GFD.111.pdf JSDL HPC Profile >-- Application Extension 1.0 >-- >-- - http://www.ogf.org/documents/GFD.108.pdf OGSA-BES 1.0 >-- - http://www.ogf.org/documents/GFD.114.pdf HPC Basic Profile 1.0 >-- - http://www.ogf.org/documents/GFD.151.pdf HPCBP Advanced Filter Extension >-- >-- - http://forge.gridforum.org/sf/go/doc16306 Requirements for an >-- improved BES >-- - http://forge.gridforum.org/sf/go/doc16186 GLUE 2 Attributes for Use >-- in JSDL >-- >-- >-- Taking into account my experience at OGF, I consider that the main issue >-- is : >-- >-- WHO is able to devote HOW MUCH TIME to real technical work, in >-- particular to the review of input material ? >-- >-- Lot of thanks in advance to any volunteer. >-- >-- >-- Best regards. >-- >-- ----------------------------------------------------- >-- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >-- Bat 200 91898 ORSAY France >-- Tel: +33 1 64 46 84 87 Skype: etienne.urbah >-- Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >-- ----------------------------------------------------- >-- >-- >-- On Wed, 14/09/2011 16:08, Etienne URBAH wrote: >-- > On Tue, 13/09/2011 21:26, Etienne URBAH wrote: >-- >> Philipp, Steven, Morris, Andrew and all, >-- >> >-- >> Concerning the EVO teleconference named 'OGF preparation of PGI / BES / >-- >> JSDL' on Wednesday 14 September 2011 at 16h CEST (14h UTC) : >-- >> >-- >> - I remind everyone that it will be held this Wednesday at >-- >> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >-- >> with password !!jsdlftw >-- > THAT IS IMMEDIATELY. >-- > >-- >> >-- >> - The scope of the agenda is NOT limited to JSDL, but really encompasses >-- >> BES / JSDL / GLUE / PGI. >-- >> >-- >> - Even if I have technical knowledge on all these subjects, I am NOT >-- >> highly talented as group leader, and I would greatly prefer that the >-- >> conference leader is a chair of one of the above OGF working groups. >-- >> In particular, if one of them participates, I would suggest Steven >-- >> NEWHOUSE, Morris RIEDEL or Andrew GRIMSHAW. >-- >> >-- >> - I am quite slow at taking notes, and I would be able to perform this >-- >> task only if I am NOT the conference leader. >-- >> Anyway, can anyone volunteer taking notes ? >-- >> >-- >> - As input material, I suggest to read 'Requirements for an improved >-- >> Basic Execution Service (BES)' at >-- >> http://forge.gridforum.org/sf/go/doc16306 >-- >> >-- >> Best regards. >-- >> >-- >> ----------------------------------------------------- >-- >> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >-- >> Bat 200 91898 ORSAY France >-- >> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >-- >> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >-- >> ----------------------------------------------------- >-- >> >-- >> >-- >> On Tue, 13/09/2011 20:29, philipp.wieder at tu-dortmund.de wrote: >-- >>> Dear Etienne, >-- >>> >-- >>> thank you very much for setting this up. >-- >>> >-- >>> The agenda for tomorrow is brief: >-- >>> - Assess status of input material for the whole day BES/JSDL/GLUE/PGI >-- >>> session in Lyon (some material is already here [1]) >-- >>> - Define tentative agenda for the respective session including >-- >>> + Objectives >-- >>> + Actions >-- >>> + Responsibilities >-- >>> + Roadmap >-- >>> - Check availability for Lyon >-- >>> - AOB >-- >>> >-- >>> Since I am traveling tomorrow I may not be able to join (I will try >-- >>> via a >-- >>> mobile connection). Etienne, is it possible for you to lead the >-- >>> conference >-- >>> in this case and take minutes? >-- >>> >-- >>> Best regards, Philipp. >-- >>> >-- >>> >-- >>> [1] >-- >>> https://forge.gridforum.org/sf/docman/do/listDocuments/projects.jsdl-wg/doc >-- >>> >-- >>> >-- >>> man.root.background_docs.2011 >-- >>> >-- >>> Am 12.09.11 18:33 schrieb "Etienne URBAH" unter: >-- >>> >-- >>>> Philipp and all, >-- >>>> >-- >>>> Concerning the preparation of the PGI / BES / JSDL sessions at OGF >-- >>>> 33 in >-- >>>> Lyon next week : >-- >>>> >-- >>>> I booked an EVO teleconference named 'OGF preparation of PGI / BES / >-- >>>> JSDL' for Wednesday 14 September 2011 at 16h CEST (14h UTC). >-- >>>> >-- >>>> BEWARE that EVO 2.8 fails on MS-Windows XP SP3 with Firefox 6.0.2 and >-- >>>> Oracle Java 1.7.0, so that you may have to use EVO beta at >-- >>>> http://evo.caltech.edu/evoBeta/ which I tested successfully. >-- >>>> >-- >>>> Access to the EVO teleconference will be provided at >-- >>>> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >-- >>>> >-- >>>> with password !!jsdlftw >-- >>>> >-- >>>> Thanks in advance to Philipp for providing the agenda. >-- >>>> >-- >>>> Best regards. >-- >>>> >-- >>>> ----------------------------------------------------- >-- >>>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >-- >>>> Bat 200 91898 ORSAY France >-- >>>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >-- >>>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >-- >>>> ----------------------------------------------------- >-- >>>> >-- >>>> >-- >>>> -------- Original Message -------- >-- >>>> Subject: EVO Booking : OGF preparation of PGI / BES / JSDL from Wed, 14 >-- >>>> Sep 11 16:00 +0200 (CEST) >-- >>>> Date: Mon, 12 Sep 11 17:58:20 +0200 >-- >>>> From: evosupport at vrvs.org >-- >>>> To: urbah at lal.in2p3.fr >-- >>>> >-- >>>> Hello Etienne URBAH, >-- >>>> You have BOOKED a meeting in EVO (http://evo.caltech.edu). >-- >>>> >-- >>>> Title: OGF preparation of PGI / BES / JSDL >-- >>>> Description: Preparation of the PGI / BES / JSDL sessions at OGF 33 in >-- >>>> Lyon next week >-- >>>> Community: Universe >-- >>>> Password: !!jsdlftw >-- >>>> >-- >>>> Meeting Access Information: >-- >>>> - Meeting URL >-- >>>> http://evo.caltech.edu/evoBeta/koala.jnlp?meeting=MsMiMI282BDMDe9l92Dt9s >-- >>>> >-- >>>> >-- >>>> - Password: !!jsdlftw >-- >>>> - Phone Bridge >-- >>>> ID: 399 7332 >-- >>>> Password: 0162 >-- >>>> >-- >>>> Central European Summer Time (+0200) >-- >>>> Start 2011-09-14 16:00 >-- >>>> End 2011-09-14 18:00 >-- >>>> >-- >>>> Japan Standard Time (+0900) >-- >>>> Start 2011-09-14 23:00 >-- >>>> End 2011-09-15 01:00 >-- >>>> >-- >>>> Eastern Daylight Time (-0400) >-- >>>> Start 2011-09-14 10:00 >-- >>>> End 2011-09-14 12:00 >-- >>>> >-- >>>> Pacific Daylight Time (-0700) >-- >>>> Start 2011-09-14 07:00 >-- >>>> End 2011-09-14 09:00 >-- >>>> >-- >>>> >-- >>>> EVO Phone Bridge Telephone Numbers: >-- >>>> --------------- >-- >>>> - USA (Caltech, Pasadena, CA) >-- >>>> +1 626 395 2112 >-- >>>> >-- >>>> - Switzerland (CERN, Geneva) >-- >>>> +41 22 76 71400 >-- >>>> >-- >>>> - Slovakia (UPJS, Kosice) >-- >>>> +421 55 234 2420 >-- >>>> >-- >>>> - Italy (INFN, several cities) >-- >>>> http://server10.infn.it/video/index.php?page=telephone_numbers >-- >>>> Enter '4000' to access the EVO bridge >-- >>>> >-- >>>> - Germany (DESY, Hamburg) >-- >>>> +49 40 8998 1340 >-- >>>> >-- >>>> - USA (BNL, Upton, NY) >-- >>>> +1 631 344 6100 >-- >>>> >-- >>>> - United Kingdom (University of Manchester) >-- >>>> +44 161 306 6802 >-- >>>> >-- >>>> - Australia (ARCS) >-- >>>> +61 >-- >>>> Adelaide 08 8463 1011 >-- >>>> Brisbane 07 3139 0705 >-- >>>> Canberra 02 6112 8742 >-- >>>> Hobart 03 623 70281 >-- >>>> Melbourne 03 8685 8362 >-- >>>> Perth 08 6461 6718 >-- >>>> Sydney 02 8212 4591 >-- >>>> >-- >>>> - Netherlands (Nikhef, Amsterdam) >-- >>>> +31 20 7165293 >-- >>>> Dial '2' at the prompt >-- >>>> >-- >>>> - Canada (TRIUMF, Vancouver) >-- >>>> +1 604 222 7700 >-- >>>> >-- >>>> - Czech Republic (CESNET, Prague) >-- >>>> +420 95 007 2386 >-- >>>> >-- >>>> - USA (MIT, Cambridge, MA) >-- >>>> +1 617 715 4691 >-- >>>> >-- >>>> - France (RAP, Paris) >-- >>>> +33 144 27 81 50 >-- >>>> >-- >>>> >-- >>>> - Skype (tm) (World-wide) >-- >>>> evo.phone >-- >>>> See: >-- >>>> http://evo.caltech.edu/evoGate/Documentation/extclient/skype/skype.html >-- >>>> >-- >>> >-- >> >-- > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3550 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110915/9a97454e/attachment.bin From b.schuller at fz-juelich.de Thu Sep 15 06:00:55 2011 From: b.schuller at fz-juelich.de (Bernd Schuller) Date: Thu, 15 Sep 2011 13:00:55 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <4E6FA620.8020307@lal.in2p3.fr> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> Message-ID: <1316084455.1787.18.camel@zam994> Hi Etienne, thanks for creating this consolidated requirements document, it is very interesting to read. I have a few comments, mostly related to whether certain features are a "MUST" or not. In many cases it was not clear to me whether you talk about the BES interface specification, or about the way a BES instance should be deployed and operated. It would be good to remove the operational and deployment concerns, so that only the specification parts remain. I hope my comments can be used to make the document clearer and easier to digest. So, in detail: 1.4 Methodology * ref 4: glite user guide -> out of scope, gLite is too complex and too specific in its architecture (if there is such a thing) to be useful as a base for BES 2.4 Collaboration with other services While this is important for interoperability, it is unimportant for the specification of a BES. The BES spec should NOT try to specify all the interaction with the rest of the world. This is the task of a "grid architecture specification" like OGSA. Specifically, the interactions with security, monitoring, accounting and logging framework are OPERATIONAL concerns that MUST NOT be a mandatory part of a BES specification. 4. BES non-functional requirements 4.1.2 Traceability - should be SHOULD not MUST 4.1.3 Security - how can a specification "implement" a policy? You probably meant "BES implementations SHOULD ..." - availability/reliability: while I agree, you cannot enforce quality of an implementation via the specification. So all of 4.1.3 is SHOULD in my opinion. 4.2 all of this is out of scope. For example the UNICORE service container hosts a number of services including an execution service. Probably you mean that the execution service SPECIFICATION should be limited to the execution service and MUST NOT specify accounting, security etc. 5 BES requirements applying to the info system Introduction: this is a requirement on the grid, not on the BES. 6 BES requirements applying to security Introduction: when you say "The Execution Service MUST NOT be overloaed by implementing a security framework", it is not clear what you mean by "service". Do you mean the service as defined by its interfaces, or do you mean the actual BES process or even machine which is running BES? For example, a BES may be one of many services hosted by a service container, which may include a full featured security framework. This should be clarified. 6.1 * "SSL certificates MUST be signed by a CA..." this is an operational decision, and has nothing to do with the BES spec. For example, a site may run an inhouse deployment of BES using an in-house CA. This requirement should be deleted. 6.3 * "For Client authentication, the Execution Service MUST accept all following authentication methods: Full X509, RFC-3820-compliant X509 Proxy" This requirement is invalid. I agree that it would be nice to be able to specify authentication methods, but it is impossible. For example Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be valid authentication methods in some circumstances. Furthermore, making proxies a MUST implies that nonstandard authentication libraries instead of TLS/SSL must be used, making the BES implementation insecure. For some implementors (like UNICORE) proxies on the transport level are very much a no-go. 6.4. "This authorization mechanism MUST be consistent across all instances of the Execution Service" This violates the autonomy of a site. Site administrators often wish to stay in control of their resources, and do not accept external authorisation decision points. And anyway, who cares? Since the AuthZ mechanism is internal to the BES, it cannot be specified in the BES spec as such. 6.5 These are reqiurements on the security layer (or framework) and should not be used as requirements on BES. 7 BES requirements related to "Application Repositories" While I agree that BES should understand the notion of an "Application" (see e.g. JSDL ApplicationName), I don't agree that the BES should use these for Scheduling. Rather, this is the job of a broker. 8 BES requirements applying to Accounting As a "MUST", these are out of scope, and should be made "SHOULD". 9 Logging/Bookkeeping Same as 8. 10 Jobs 10.1 Types of job Support for parallel jobs: it should be "MUST" :-) Best regards, Bernd. On Tue, 2011-09-13 at 20:51 +0200, Etienne URBAH wrote: > Andrew, Steven and all OGSA-BES stakeholders, > > In the document named 'Requirements for an improved Basic Execution > Service (BES)' and available at > http://forge.gridforum.org/sf/go/doc16306 I have performed following > improvements : > > - For each functionality, clearly separate : > - the requirements about the corresponding Client interface (often > 'MUST'), > - the requirements about the implementation of the functionality > (sometimes 'MAY'). > > - Add short titles in bold to improve readability. > > Thank you in advance for reading and criticizing this requirements document. > > Best regards. > > Etienne URBAH > > > On Fri, 09/09/2011 22:51, Etienne URBAH wrote: > > Andrew, Steven and all OGSA-BES stakeholders, > > > > I have finished written down a document named 'Requirements for an > > improved Basic Execution Service (BES)' and available at > > http://forge.gridforum.org/sf/go/doc16306 > > > > I will use this requirements document to propose improvements to the > > current specification of BES 1.0 described in GFD.108, as agreed at the > > OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011. > > > > Thank you in advance for reading and criticizing my requirements document. > > > > Best regards. > > > > ----------------------------------------------------- > > Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS > > Bat 200 91898 ORSAY France > > Tel: +33 1 64 46 84 87 Skype: etienne.urbah > > Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr > > ----------------------------------------------------- > > > > [...] -- Dr. Bernd Schuller Federated Systems and Data Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc Phone: +49 246161-8736 (fax -8556) ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ From urbah at lal.in2p3.fr Thu Sep 15 12:27:00 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Thu, 15 Sep 2011 19:27:00 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <1316084455.1787.18.camel@zam994> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> Message-ID: <4E723564.70407@lal.in2p3.fr> Bernd, Concerning the document named 'Requirements for an improved Basic Execution Service (BES)' and available at http://forge.gridforum.org/sf/go/doc16306 : THANK YOU VERY MUCH for having taken the time to read this document, and for having taken the time to provide comments : Such comments are very useful for the improvement of documents, permit convergence and prepare later agreement. My answers are interspersed in your comments. Please note that the chapter numbering has been modified. Besides, in the chapter about 'Data Management', I have added a description of the context and the workflows for automatic and manual data staging. The new version of the document is available at the location given above. In order that we all make progress, please continue sending comments. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- On Thu, 15/09/2011 13:00, Bernd Schuller wrote: > Hi Etienne, > > thanks for creating this consolidated requirements document, it is very > interesting to read. I have a few comments, mostly related to whether > certain features are a "MUST" or not. > > In many cases it was not clear to me whether you talk about the BES > interface specification, or about the way a BES instance should be > deployed and operated. It would be good to remove the operational and > deployment concerns, so that only the specification parts remain. > > I hope my comments can be used to make the document clearer and easier > to digest. > > So, in detail: > > 1.4 Methodology > > * ref 4: glite user guide -> out of scope, gLite is too complex and > too specific in its architecture (if there is such a thing) to be useful > as a base for BES Yes, this is a very large user guide for specific usage of gLite by users, and it does NOT provide a clear description of the architecture and of the functionalities. But it is NOT so complex. As soon as I finished reading this guide, it was easy for me to perform reverse engineering and extract the effective architecture (SOA with internal interfaces) and the implied functionalities (which are to be improved). In the text, I have prepended a few words to explain that. > > 2.4 Collaboration with other services > > While this is important for interoperability, it is unimportant > for the specification of a BES. The BES spec should NOT try to specify > all the interaction with the rest of the world. This is the task of a > "grid architecture specification" like OGSA. My document is NOT targeted only to the specification of the BES Client interface, but to the clear and consistent description of BES context and functional + operational requirements which are really necessary for interoperability. As far as I know, OGSA does NOT take into account GLUE 2.0 yet. Therefore, an up to date 'grid architecture specification' is absolutely necessary. If OGSA members consider chapters 2.3 and 2.4 of my document as a 'grid architecture specification' which updates and improves OGSA, I thank them. If they consider that this 'grid architecture specification' does NOT comply with OGSA and competes with it, then I assert that it obsoletes OGSA. > > Specifically, the interactions with security, monitoring, accounting and > logging framework are OPERATIONAL concerns that MUST NOT be a mandatory > part of a BES specification. FAILURE of practical operations is often caused by LACK of early care about operational concerns during specification phase. As GIN-GC has proven and documented, this is even more true for interoperability on the field (as opposed to theoretical interoperability at the WSDL level). I confirm that care about operational concerns is REQUIRED for real operations and for practical interoperability on the field. Although operational concerns are NOT part of the BES Client interface, they are REQUIRED for the overall specifications of BES in its context. In the text, I have stressed that the document DOES take into account operational concerns. > > 4. BES non-functional requirements > > 4.1.2 Traceability - should be SHOULD not MUST This is an operational concern : Would you really take the risk that the whole EGI becomes a spambot or a scambot ? No traceability --> No post mortem analysis after attack --> Large infection --> Panic --> Abrupt and very long shutdown of all services. I fully confirm that traceability is a MUST. > > 4.1.3 Security > - how can a specification "implement" a policy? You probably > meant "BES implementations SHOULD ..." The text now is 'BES specifications MUST fully take into account the Security Policies ...' > - availability/reliability: while I agree, you cannot enforce > quality of an implementation via the specification. YES. Thank you. > So all of 4.1.3 is SHOULD in my opinion. Security policies stay a MUST. > > 4.2 > all of this is out of scope. For example the UNICORE service > container hosts a number of services including an execution service. > Probably you mean that the execution service SPECIFICATION should be > limited to the execution service and MUST NOT specify accounting, > security etc. 'Well defined and narrow scope' is a general engineering requirement. It is fundamental concern crossing requirements, design, specifications, fabrication, tests, operations, user experience and product maintenance for all types of products, even outside software engineering. I confirm that 'Well defined and narrow scope' is absolutely REQUIRED for sound software design, implementation, deployment and maintainability. From your comment, I assume that the Execution Service of UNICORE does have a 'Well defined and narrow scope', does have precise interfaces with other UNICORE services, and minimizes overlaps with them. The text now is 'The Execution Service is defined by its functionalities (efficiently manage Jobs, which are transient entities) and its operational constraints : For sound software design, implementation, deployment and maintainability, ...' > > 5 BES requirements applying to the info system > > Introduction: this is a requirement on the grid, not on the BES. This is NOT a requirement, this is a DESCRIPTION. The importance of the Information System as foundational block is underestimated, and precise knowledge of GLUE 2.0 is poor inside OGF. I am trying to improve the situation by providing clear explanations. > > 6 BES requirements applying to security > > Introduction: when you say "The Execution Service MUST NOT be overloaed > by implementing a security framework", it is not clear what you mean by > "service". Do you mean the service as defined by its interfaces, or do > you mean the actual BES process or even machine which is running BES? > For example, a BES may be one of many services hosted by a service > container, which may include a full featured security framework. This > should be clarified. The text now is 'The Execution Service is defined by its functionalities (efficiently manage Jobs, which are transient entities) and its operational constraints. For sound software design, implementation, deployment and maintainability, BES specifications and instances of the Execution Service ...' > > 6.1 > > * "SSL certificates MUST be signed by a CA..." this is an operational > decision, and has nothing to do with the BES spec. > For example, a site may run an inhouse deployment of BES using an > in-house CA. This requirement should be deleted. This operational concern is REQUIRED for practical interoperability on the field. I have prepended : * Authentication of Servers : The Execution Service SHOULD permit Clients to authenticate it. If an Execution Service authenticates itself to clients, it MUST permit Clients to really perform this authentication. > > 6.3 > > * "For Client authentication, the Execution Service MUST accept all > following authentication methods: Full X509, RFC-3820-compliant X509 > Proxy" > > This requirement is invalid. I agree that it would be nice to be able to > specify authentication methods, but it is impossible. For example > Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface > over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be > valid authentication methods in some circumstances. There are 2 separate requirements : - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy - 1 'MAY' for all other ones. > > Furthermore, making proxies a MUST implies that nonstandard > authentication libraries instead of TLS/SSL must be used, making the BES > implementation insecure. For some implementors (like UNICORE) proxies on > the transport level are very much a no-go. I had clearly specified RFC-3820-compliant X509 Proxy, which ARE standard. Your critics are valid for GSI proxies, which I have taken care NOT to mention. > > 6.4. > > "This authorization mechanism MUST be consistent across all instances of > the Execution Service" > > This violates the autonomy of a site. Site administrators often wish to > stay in control of their resources, and do not accept external > authorisation decision points. And anyway, who cares? Since the AuthZ > mechanism is internal to the BES, it cannot be specified in the > BES spec as such. Interoperability requires a federation of independent administrative domain to agree on common functionalities, interfaces and operations. This DOES sometime violate the autonomy of each individual site. The requirement is NOT that the AUTHZ decision point is external to any site, but that all participating site MUST accept to install inside their site an instance of a commonly validated software implementing the decision point. The AUTHZ mechanism MUST NOT be internal to the BES : For example, in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package is described as 'The security subsystem of UNICORE/X', and is NOT internal to 'de.fzj.unicore.uas.impl.job'. > > 6.5 > > These are reqiurements on the security layer (or framework) and should > not be used as requirements on BES. These security requirements DO have impacts on the BES Client interface and on the Job Description document. In the text, I have made it clear. > > 8 BES requirements related to "Application Repositories" > > While I agree that BES should understand the notion of an "Application" > (see e.g. JSDL ApplicationName), I don't agree that the BES should > use these for Scheduling. Rather, this is the job of a broker. The text is now : * Resource selection : The Execution Service MUST use, among others, these references to 'Installed Applications' in order to select the most adequate computing resource for the Job. > > 9 BES requirements applying to Accounting > > As a "MUST", these are out of scope, and should be made "SHOULD". No Accounting --> No precise reporting to funding agencies --> No funding --> Abrupt and very long shutdown of all services. I fully confirm that Accounting is a MUST. > > 10 Logging/Bookkeeping > > Same as 9. Same as 'Traceability' : This is an operational concern : Would you really take the risk that the whole EGI becomes a spambot or a scambot ? No Logging and Bookkeeping --> No post mortem analysis after attack --> Large infection --> Panic --> Abrupt and very long shutdown of all services. I fully confirm that Logging and Bookkeeping is a MUST. > > 12 Jobs > > 12.1 Types of job > > Support for parallel jobs: it should be "MUST" :-) The text is now : - The concept of 'Single Job' includes Jobs running massively-parallel processes using MPI on one large-scale HPC System. The Execution Service MUST understand instructions for usage of MPI inside the Job Description document. The Execution Service SHOULD transmit these instructions to the Batch System, or return an explicit error message if not supported. > > > > > Best regards, > Bernd. > > > > > On Tue, 2011-09-13 at 20:51 +0200, Etienne URBAH wrote: >> Andrew, Steven and all OGSA-BES stakeholders, >> >> In the document named 'Requirements for an improved Basic Execution >> Service (BES)' and available at >> http://forge.gridforum.org/sf/go/doc16306 I have performed following >> improvements : >> >> - For each functionality, clearly separate : >> - the requirements about the corresponding Client interface (often >> 'MUST'), >> - the requirements about the implementation of the functionality >> (sometimes 'MAY'). >> >> - Add short titles in bold to improve readability. >> >> Thank you in advance for reading and criticizing this requirements document. >> >> Best regards. >> >> Etienne URBAH >> >> >> On Fri, 09/09/2011 22:51, Etienne URBAH wrote: >>> Andrew, Steven and all OGSA-BES stakeholders, >>> >>> I have finished written down a document named 'Requirements for an >>> improved Basic Execution Service (BES)' and available at >>> http://forge.gridforum.org/sf/go/doc16306 >>> >>> I will use this requirements document to propose improvements to the >>> current specification of BES 1.0 described in GFD.108, as agreed at the >>> OGSA-BES session at OGF32 in Salt Lake City on 17 July 2011. >>> >>> Thank you in advance for reading and criticizing my requirements document. >>> >>> Best regards. >>> >>> ----------------------------------------------------- >>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >>> Bat 200 91898 ORSAY France >>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >>> ----------------------------------------------------- >>> >>> > [...] > > -- > Dr. Bernd Schuller > Federated Systems and Data > Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc > Phone: +49 246161-8736 (fax -8556) > > > > ------------------------------------------------------------------------------------------------ > ------------------------------------------------------------------------------------------------ > Forschungszentrum Juelich GmbH > 52425 Juelich > Sitz der Gesellschaft: Juelich > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher > Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, > Prof. Dr. Sebastian M. Schmidt > ------------------------------------------------------------------------------------------------ > ------------------------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110915/3474996d/attachment-0001.bin From andre at merzky.net Thu Sep 15 13:43:13 2011 From: andre at merzky.net (Andre Merzky) Date: Thu, 15 Sep 2011 20:43:13 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <4E723564.70407@lal.in2p3.fr> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> Message-ID: Hi Etienne, all, On Thu, Sep 15, 2011 at 7:27 PM, Etienne URBAH wrote: > > On Thu, 15/09/2011 13:00, Bernd Schuller wrote: >> >> In many cases it was not clear to me whether you talk about the BES >> interface specification, or about the way a BES instance should be >> deployed and operated. It would be good to remove the operational and >> deployment concerns, so that only the specification parts remain. >> >> 2.4 Collaboration with other services >> >> While this is important for interoperability, it is unimportant >> for the specification of a BES. The BES spec should NOT try to specify >> all the interaction with the rest of the world. This is the task of a >> "grid architecture specification" like OGSA. > > ?My document is NOT targeted only to the specification of the BES Client > interface, but to the clear and consistent description of BES context and > functional + operational requirements which are really necessary for > interoperability. I would like to put forward a motion: "The discussion about BES interface specification should be separated from the discussion about interoperation of BES services." Motivation: Both discussions are important and necessary to have. Discussing both topics at once, however, will convolute the BES interface specification, and will delay overall progress. I do not mean that interoperation can only be discussed after the BES interface spec is finished, not at all - but each argument should clearly marked as belonging to *either* discussion, not both. Some more comments inlined... >> Specifically, the interactions with security, monitoring, accounting and >> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory >> part of a BES specification. > > ?FAILURE of practical operations is often caused by LACK of early care about > operational concerns during specification phase. ?As GIN-GC has proven and > documented, this is even more true for interoperability on the field (as > opposed to theoretical interoperability at the WSDL level). > > ?I confirm that care about operational concerns is REQUIRED for real > operations and for practical interoperability on the field. ?Although > operational concerns are NOT part of the BES Client interface, they are > REQUIRED for the overall specifications of BES in its context. "REQUIRED for the overall specifications of BES" - I assume that this does *not* mean the BES Service Interface specification (which I think you refer to as 'BES client interface', as it is consumed by a non-service / client)? > ?In the text, I have stressed that the document DOES take into account > operational concerns. 'the document' - I assume you mean the present requirement document? If so, I agree - it is useful to capture operational requirements. I also agree with Bernd though, that those should not directly influence the BES service interface specification, but rather are a separate concern. You cannot foresee the requirements of all implementations, nor the boundary conditions of all deployments - adding operational features to the BES Service interface specification *will* limit its applicability. Thus, those issues must IMHO be addressed in a separate document. And no, I don't expect EGI to use my service implementation ;-) My $0.02, Andre. -- Nothing is ever easy... From b.schuller at fz-juelich.de Thu Sep 15 15:44:35 2011 From: b.schuller at fz-juelich.de (Bernd Schuller) Date: Thu, 15 Sep 2011 22:44:35 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <4E723564.70407@lal.in2p3.fr> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> Message-ID: <1316119475.24230.60.camel@zam994> Hi Etienne, thanks for the clarifications. So indeed your document is aimed at both: 1) providing requirements for the actual BES specification ("client interface" in your terminology) 2) the operation and deployment issues that have to be solved for interoperability on an infrastructure level (say EGI and EDGI). It would be very beneficial for further progress if these two distinct concerns could be separated, at least CLEARLY marked in your document. I have added some more comments inline. On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: > Concerning the document named 'Requirements for an improved Basic > Execution Service (BES)' and available at > http://forge.gridforum.org/sf/go/doc16306 : > > THANK YOU VERY MUCH for having taken the time to read this document, and > for having taken the time to provide comments : > > Such comments are very useful for the improvement of documents, permit > convergence and prepare later agreement. > [...] > On Thu, 15/09/2011 13:00, Bernd Schuller wrote: > > > >[...] > > > 1.4 Methodology > > > > * ref 4: glite user guide -> out of scope, gLite is too complex and > > too specific in its architecture (if there is such a thing) to be useful > > as a base for BES > Yes, this is a very large user guide for specific usage of gLite by > users, and it does NOT provide a clear description of the architecture > and of the functionalities. > But it is NOT so complex. As soon as I finished reading this guide, > it was easy for me to perform reverse engineering and extract the > effective architecture (SOA with internal interfaces) and the implied > functionalities (which are to be improved). Indeed it appears that you try to impose gLite specifics (like a logging & bookkeeping service or proxy certificates on the transport level) as requirements. This would severly limit the BES specification effort, and will not be accepted (I hope) by other stakeholders. > In the text, I have prepended a few words to explain that. Basically you imply that the "architecture and functionalities" of the gLite execution system (together with the PGI work) is somehow the guideline to be followed, which I fully disagree with. > > > > 2.4 Collaboration with other services > > > > While this is important for interoperability, it is unimportant > > for the specification of a BES. The BES spec should NOT try to specify > > all the interaction with the rest of the world. This is the task of a > > "grid architecture specification" like OGSA. > My document is NOT targeted only to the specification of the BES > Client interface, but to the clear and consistent description of BES > context and functional + operational requirements which are really > necessary for interoperability. > As far as I know, OGSA does NOT take into account GLUE 2.0 yet. > Therefore, an up to date 'grid architecture specification' is absolutely > necessary. Glue2 is just an information model, not necessarily a perfect one nor the only one. However, I agree an information model has to be adopted for BES and any associated information systems. > If OGSA members consider chapters 2.3 and 2.4 of my document as a > 'grid architecture specification' which updates and improves OGSA, I > thank them. If they consider that this 'grid architecture > specification' does NOT comply with OGSA and competes with it, then I > assert that it obsoletes OGSA. I can't really say, the OGSA group stopped its work a long time ago and it's a long time that I looked at the documents. > > > > Specifically, the interactions with security, monitoring, accounting and > > logging framework are OPERATIONAL concerns that MUST NOT be a mandatory > > part of a BES specification. > FAILURE of practical operations is often caused by LACK of early care > about operational concerns during specification phase. Agreed. > As GIN-GC has > proven and documented, this is even more true for interoperability on > the field (as opposed to theoretical interoperability at the WSDL level). > I confirm that care about operational concerns is REQUIRED for real > operations and for practical interoperability on the field. Although > operational concerns are NOT part of the BES Client interface, they are > REQUIRED for the overall specifications of BES in its context. > In the text, I have stressed that the document DOES take into account > operational concerns. > > > > > 4. BES non-functional requirements > > > > 4.1.2 Traceability - should be SHOULD not MUST > This is an operational concern : Would you really take the risk that > the whole EGI becomes a spambot or a scambot ? Isn't it already, powered by gLite and used by the wlcg botnet (just kidding of course) :-) > No traceability --> No post mortem analysis after attack --> Large > infection --> Panic --> Abrupt and very long shutdown of all services. > I fully confirm that traceability is a MUST. It is an internal detail which any good implementation will provide. If BES-A is much easier and more secure to operate than BES-B, admins can choose which to install. > > > > 4.1.3 Security > > - how can a specification "implement" a policy? You probably > > meant "BES implementations SHOULD ..." > The text now is 'BES specifications MUST fully take into account the > Security Policies ...' Still no understanding here... let's take traceability The relevant EGI policy says "[...] software deployed in the Grid MUST include the ability to produce sufficient and relevant logging [...] The level of the logging MUST be configured by all service providers, including but not limited to the Sites, to produce the required information which MUST be retained for a minimum of 90 days." For example all UNICORE services can be made to comply with this by configuration of the logging library we use (Apache Log4j), and by not deleting log files for 90 days. So this is a feature of the implementation and the administrator in charge, not the specification. Thus, your sentence should read "BES implementations SHOULD ..." (It is MUST of course only if they want to be deployed in EGI) One does not try to specify implementation details, at least not in any specification I've ever seen (e.g. does the HTTP specification say anything about server logging or accepted CAs?). >[...] > > > > 4.2 > > all of this is out of scope. For example the UNICORE service > > container hosts a number of services including an execution service. > > Probably you mean that the execution service SPECIFICATION should be > > limited to the execution service and MUST NOT specify accounting, > > security etc. > 'Well defined and narrow scope' is a general engineering requirement. > It is fundamental concern crossing requirements, design, > specifications, fabrication, tests, operations, user experience and > product maintenance for all types of products, even outside software > engineering. exactly. > I confirm that 'Well defined and narrow scope' is absolutely REQUIRED > for sound software design, implementation, deployment and maintainability. > From your comment, I assume that the Execution Service of UNICORE > does have a 'Well defined and narrow scope', does have precise > interfaces with other UNICORE services, and minimizes overlaps with them. of course. And UNICORE does not include a L&B service :-) > [...] > > 6.1 > > > > * "SSL certificates MUST be signed by a CA..." this is an operational > > decision, and has nothing to do with the BES spec. > > For example, a site may run an inhouse deployment of BES using an > > in-house CA. This requirement should be deleted. > This operational concern is REQUIRED for practical interoperability > on the field. I have prepended : > * Authentication of Servers : The Execution Service SHOULD permit > Clients to authenticate it. If an Execution Service authenticates > itself to clients, it MUST permit Clients to really perform this > authentication. This sentence makes no sense to me, sorry. Maybe "Server and client SHOULD communicate via a secure channel (SSL/TLS)". Even this may not be true in the future, though it is for all(?) the Grid systems currently. > > > > 6.3 > > > > * "For Client authentication, the Execution Service MUST accept all > > following authentication methods: Full X509, RFC-3820-compliant X509 > > Proxy" > > > > This requirement is invalid. I agree that it would be nice to be able to > > specify authentication methods, but it is impossible. For example > > Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface > > over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be > > valid authentication methods in some circumstances. > There are 2 separate requirements : > - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy > - 1 'MAY' for all other ones. > > > > Furthermore, making proxies a MUST implies that nonstandard > > authentication libraries instead of TLS/SSL must be used, making the BES > > implementation insecure. For some implementors (like UNICORE) proxies on > > the transport level are very much a no-go. > I had clearly specified RFC-3820-compliant X509 Proxy, which ARE > standard. > Your critics are valid for GSI proxies, which I have taken care NOT > to mention. by "standard" I did not mean that it is an RFC, but software support. As opposed to standard SSL/TLS, proxies are almost not supported by industry standard tools, for example Apache httpd or the Java JDK. One has to rely on custom code, which is notoriously buggy and error prone. Since one important non-functional requirement (for me at least) is to be able use standard (off-the-shelf) open source software, having to support proxies is a big limitation. > > 6.4. > > > > "This authorization mechanism MUST be consistent across all instances of > > the Execution Service" > > > > This violates the autonomy of a site. Site administrators often wish to > > stay in control of their resources, and do not accept external > > authorisation decision points. And anyway, who cares? Since the AuthZ > > mechanism is internal to the BES, it cannot be specified in the > > BES spec as such. > Interoperability requires a federation of independent administrative > domain to agree on common functionalities, interfaces and operations. > This DOES sometime violate the autonomy of each individual site. > The requirement is NOT that the AUTHZ decision point is external to > any site, but that all participating site MUST accept to install inside > their site an instance of a commonly validated software implementing the > decision point. No. Each site may choose their own authz decision point, IMO. > The AUTHZ mechanism MUST NOT be internal to the BES : Maybe I was not clear. The authz mechanism is invisible for outside parties (like clients). It can be an external component, an internal component, whatever, it is up to the BES implementor and the site admin. > For example, > in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package > is described as 'The security subsystem of UNICORE/X', and is NOT > internal to 'de.fzj.unicore.uas.impl.job'. In UNICORE site admins can choose what attribute sources and XACML decision points they want to use, but the clients (including other services) do not need to know this. That is what I meant by "internal". > > > > 6.5 > > > > These are reqiurements on the security layer (or framework) and should > > not be used as requirements on BES. > These security requirements DO have impacts on the BES Client > interface and on the Job Description document. > In the text, I have made it clear. indeed while preparing the EMI-ES specification, we came to the following conclusions 1) when using proxies for delegation, it is necessary to map each data staging item to a delegated credential (you can check the EMI-ES job description for details) 2) the delegation operations are separate from the job management operations, so they do not necessarily have to be part of the BES client interface. Also, there are existing implementations (UNICORE and Genesis come to my mind) that do not need this at all, because they do delegation without proxies. So I disagree, 6.5 mostly describes features of the particular security framework that is used. > > > > 8 BES requirements related to "Application Repositories" > > > > While I agree that BES should understand the notion of an "Application" > > (see e.g. JSDL ApplicationName), I don't agree that the BES should > > use these for Scheduling. Rather, this is the job of a broker. > The text is now : > * Resource selection : The Execution Service MUST use, among others, > these references to 'Installed Applications' in order to select the most > adequate computing resource for the Job. > > > > 9 BES requirements applying to Accounting > > > > As a "MUST", these are out of scope, and should be made "SHOULD". > No Accounting --> No precise reporting to funding agencies --> No > funding --> Abrupt and very long shutdown of all services. > I fully confirm that Accounting is a MUST. > ... operational > > > > 10 Logging/Bookkeeping > > > > Same as 9. > Same as 'Traceability' : > This is an operational concern : Would you really take the risk that > the whole EGI becomes a spambot or a scambot ? > No Logging and Bookkeeping --> No post mortem analysis after attack > --> Large infection --> Panic --> Abrupt and very long shutdown of all > services. > I fully confirm that Logging and Bookkeeping is a MUST. > an operational MUST maybe for some infrastructures, not all. E.g. a typical HPC site has its own accounting, its own logging systems independent of the (Grid) software used to submit jobs to it. > > > > 12 Jobs > > > > 12.1 Types of job > > > > Support for parallel jobs: it should be "MUST" :-) > The text is now : > - The concept of 'Single Job' includes Jobs running > massively-parallel processes using MPI on one large-scale HPC System. > The Execution Service MUST understand instructions for usage of MPI > inside the Job Description document. The Execution Service SHOULD > transmit these instructions to the Batch System, or return an explicit > error message if not supported. OK Best regards, Bernd. -- Dr. Bernd Schuller Federated Systems and Data Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc Phone: +49 246161-8736 (fax -8556) ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ From urbah at lal.in2p3.fr Fri Sep 16 06:32:46 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Fri, 16 Sep 2011 13:32:46 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <1316119475.24230.60.camel@zam994> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> <1316119475.24230.60.camel@zam994> Message-ID: <4E7333DE.5040809@lal.in2p3.fr> Bernd and Andre, Concerning the document named 'Requirements for an improved Basic Execution Service (BES)' and available at http://forge.gridforum.org/sf/go/doc16306 : Thank you very much for your comments. BES Client interface and BES operational requirements ----------------------------------------------------- On Thu, 15/09/2011 20:43, Andre Merzky wrote: > "The discussion about BES interface specification should be separated > from the discussion about interoperation of BES services." I agree, and I will create a separate chapter gathering all BES operational requirements having NO direct impact on the BES Client interface (this will take a little time). BES requirements which seem to be specific to gLite --------------------------------------------------- If a requirement corresponds to NO real functional or operational need, then we should simply remove it. If a requirement corresponds to a real functional or operational need, but is expressed in way too specific to gLite, then we should reformulate it in a more generic way. Please provide suggestions. RFC-3820-compliant X509 Proxies ------------------------------- The RFC-3820-compliant X509 proxies are fully supported by the jLite library written in Java by Oleg SUKHOROSLOV and available at http://code.google.com/p/jlite/ Dependency of BES on other grid software components / operational issues ------------------------------------------------------------------------ It is very good that we know agree on GLUE 2.0 as base for the Information System. Otherwise, we could NOT agree on the way to express references to grid entities in the Job Description document. Your comments about chapter 6.5 confirm that the specifications of the BES Client interface DEPENDS on whether BES supports X509 proxies for delegation of Security credentials or NOT. Since delegation of Security credentials is a MUST for BES, my conclusion is that we MUST agree on SECURITY issues (even if some are operational issues) BEFORE trying to write down BES requirements. In the same way, we know that Clients need to perform complex queries on Jobs. The BES Client interface DEPENDS on whether such queries are accepted by BES itself, of by a separate Logging and Bookkeeping service. So, I think that we have to agree on the existence or absence of a separate Logging and Bookkeeping service BEFORE trying to write down BES requirements about queries. As a summary : - Some BES requirements are quite independent from other grid components, and we can discuss on them immediately. - But some other BES requirements are DEPENDENT from foundational grid components or operational issues, in particular Information System, Security, Logging and Bookkeeping, ... - Therefore, we have to agree on these other grid components or operational issues FIRST. This is a critical issue, and I propose that we discuss on it at OGF33. I will answer to your other comments later. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- On Thu, 15/09/2011 22:44, Bernd Schuller wrote: > Hi Etienne, > > thanks for the clarifications. So indeed your document is aimed at both: > > 1) providing requirements for the actual BES specification ("client > interface" in your terminology) > 2) the operation and deployment issues that have to be solved for > interoperability on an infrastructure level (say EGI and EDGI). > > It would be very beneficial for further progress if these two distinct > concerns could be separated, at least CLEARLY marked in your document. > > I have added some more comments inline. > > On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: > >> Concerning the document named 'Requirements for an improved Basic >> Execution Service (BES)' and available at >> http://forge.gridforum.org/sf/go/doc16306 : >> >> THANK YOU VERY MUCH for having taken the time to read this document, and >> for having taken the time to provide comments : >> >> Such comments are very useful for the improvement of documents, permit >> convergence and prepare later agreement. >> [...] >> On Thu, 15/09/2011 13:00, Bernd Schuller wrote: >>> >>> [...] >> >>> 1.4 Methodology >>> >>> * ref 4: glite user guide -> out of scope, gLite is too complex and >>> too specific in its architecture (if there is such a thing) to be useful >>> as a base for BES >> Yes, this is a very large user guide for specific usage of gLite by >> users, and it does NOT provide a clear description of the architecture >> and of the functionalities. >> But it is NOT so complex. As soon as I finished reading this guide, >> it was easy for me to perform reverse engineering and extract the >> effective architecture (SOA with internal interfaces) and the implied >> functionalities (which are to be improved). > > Indeed it appears that you try to impose gLite specifics (like a logging > & bookkeeping service or proxy certificates on the transport level) as > requirements. This would severly limit the BES specification effort, and > will not be accepted (I hope) by other stakeholders. > >> In the text, I have prepended a few words to explain that. > > Basically you imply that the "architecture and functionalities" of the > gLite execution system (together with the PGI work) is somehow the > guideline to be followed, which I fully disagree with. > >>> >>> 2.4 Collaboration with other services >>> >>> While this is important for interoperability, it is unimportant >>> for the specification of a BES. The BES spec should NOT try to specify >>> all the interaction with the rest of the world. This is the task of a >>> "grid architecture specification" like OGSA. >> My document is NOT targeted only to the specification of the BES >> Client interface, but to the clear and consistent description of BES >> context and functional + operational requirements which are really >> necessary for interoperability. >> As far as I know, OGSA does NOT take into account GLUE 2.0 yet. >> Therefore, an up to date 'grid architecture specification' is absolutely >> necessary. > > Glue2 is just an information model, not necessarily a perfect one nor > the only one. However, I agree an information model has to be adopted > for BES and any associated information systems. > >> If OGSA members consider chapters 2.3 and 2.4 of my document as a >> 'grid architecture specification' which updates and improves OGSA, I >> thank them. If they consider that this 'grid architecture >> specification' does NOT comply with OGSA and competes with it, then I >> assert that it obsoletes OGSA. > > I can't really say, the OGSA group stopped its work a long time ago and > it's a long time that I looked at the documents. > >>> >>> Specifically, the interactions with security, monitoring, accounting and >>> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory >>> part of a BES specification. >> FAILURE of practical operations is often caused by LACK of early care >> about operational concerns during specification phase. > > Agreed. > >> As GIN-GC has >> proven and documented, this is even more true for interoperability on >> the field (as opposed to theoretical interoperability at the WSDL level). >> I confirm that care about operational concerns is REQUIRED for real >> operations and for practical interoperability on the field. Although >> operational concerns are NOT part of the BES Client interface, they are >> REQUIRED for the overall specifications of BES in its context. >> In the text, I have stressed that the document DOES take into account >> operational concerns. >> >>> >>> 4. BES non-functional requirements >>> >>> 4.1.2 Traceability - should be SHOULD not MUST >> This is an operational concern : Would you really take the risk that >> the whole EGI becomes a spambot or a scambot ? > > Isn't it already, powered by gLite and used by the wlcg botnet (just > kidding of course) :-) > >> No traceability --> No post mortem analysis after attack --> Large >> infection --> Panic --> Abrupt and very long shutdown of all services. >> I fully confirm that traceability is a MUST. > > It is an internal detail which any good implementation will provide. > If BES-A is much easier and more secure to operate than BES-B, admins > can choose which to install. > >>> >>> 4.1.3 Security >>> - how can a specification "implement" a policy? You probably >>> meant "BES implementations SHOULD ..." >> The text now is 'BES specifications MUST fully take into account the >> Security Policies ...' > > Still no understanding here... let's take traceability > The relevant EGI policy > says > > "[...] software deployed in the Grid MUST include the > ability to produce sufficient and relevant logging [...] > The level of the logging MUST be configured by all service providers, > including but not limited to the Sites, to produce the required > information which MUST be retained for a minimum of 90 days." > > For example all UNICORE services can be made to comply with this > by configuration of the logging library we use (Apache Log4j), and by > not deleting log files for 90 days. > So this is a feature of the implementation and the administrator in > charge, not the specification. Thus, your sentence should read "BES > implementations SHOULD ..." (It is MUST of course only if they want to > be deployed in EGI) > One does not try to specify implementation details, at least not in any > specification I've ever seen (e.g. does the HTTP specification say > anything about server logging or accepted CAs?). > > > >> [...] >>> >>> 4.2 >>> all of this is out of scope. For example the UNICORE service >>> container hosts a number of services including an execution service. >>> Probably you mean that the execution service SPECIFICATION should be >>> limited to the execution service and MUST NOT specify accounting, >>> security etc. >> 'Well defined and narrow scope' is a general engineering requirement. >> It is fundamental concern crossing requirements, design, >> specifications, fabrication, tests, operations, user experience and >> product maintenance for all types of products, even outside software >> engineering. > > exactly. > >> I confirm that 'Well defined and narrow scope' is absolutely REQUIRED >> for sound software design, implementation, deployment and maintainability. >> From your comment, I assume that the Execution Service of UNICORE >> does have a 'Well defined and narrow scope', does have precise >> interfaces with other UNICORE services, and minimizes overlaps with them. > > of course. And UNICORE does not include a L&B service :-) > > >> [...] >>> 6.1 >>> >>> * "SSL certificates MUST be signed by a CA..." this is an operational >>> decision, and has nothing to do with the BES spec. >>> For example, a site may run an inhouse deployment of BES using an >>> in-house CA. This requirement should be deleted. >> This operational concern is REQUIRED for practical interoperability >> on the field. I have prepended : >> * Authentication of Servers : The Execution Service SHOULD permit >> Clients to authenticate it. If an Execution Service authenticates >> itself to clients, it MUST permit Clients to really perform this >> authentication. > > This sentence makes no sense to me, sorry. > Maybe "Server and client SHOULD communicate via a secure channel > (SSL/TLS)". Even this may not be true in the future, though it is for > all(?) the Grid systems currently. > >>> >>> 6.3 >>> >>> * "For Client authentication, the Execution Service MUST accept all >>> following authentication methods: Full X509, RFC-3820-compliant X509 >>> Proxy" >>> >>> This requirement is invalid. I agree that it would be nice to be able to >>> specify authentication methods, but it is impossible. For example >>> Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface >>> over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be >>> valid authentication methods in some circumstances. >> There are 2 separate requirements : >> - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy >> - 1 'MAY' for all other ones. >>> >>> Furthermore, making proxies a MUST implies that nonstandard >>> authentication libraries instead of TLS/SSL must be used, making the BES >>> implementation insecure. For some implementors (like UNICORE) proxies on >>> the transport level are very much a no-go. >> I had clearly specified RFC-3820-compliant X509 Proxy, which ARE >> standard. >> Your critics are valid for GSI proxies, which I have taken care NOT >> to mention. > > by "standard" I did not mean that it is an RFC, but software support. > As opposed to standard SSL/TLS, proxies are almost not supported by > industry standard tools, for example Apache httpd or the Java JDK. > One has to rely on custom code, which is notoriously buggy and error > prone. > > Since one important non-functional requirement (for me at least) is to > be able use standard (off-the-shelf) open source software, having to > support proxies is a big limitation. > >>> 6.4. >>> >>> "This authorization mechanism MUST be consistent across all instances of >>> the Execution Service" >>> >>> This violates the autonomy of a site. Site administrators often wish to >>> stay in control of their resources, and do not accept external >>> authorisation decision points. And anyway, who cares? Since the AuthZ >>> mechanism is internal to the BES, it cannot be specified in the >>> BES spec as such. >> Interoperability requires a federation of independent administrative >> domain to agree on common functionalities, interfaces and operations. >> This DOES sometime violate the autonomy of each individual site. >> The requirement is NOT that the AUTHZ decision point is external to >> any site, but that all participating site MUST accept to install inside >> their site an instance of a commonly validated software implementing the >> decision point. > > No. Each site may choose their own authz decision point, IMO. > >> The AUTHZ mechanism MUST NOT be internal to the BES : > > Maybe I was not clear. The authz mechanism is invisible for outside > parties (like clients). It can be an external component, an internal > component, whatever, it is up to the BES implementor and the site admin. > >> For example, >> in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package >> is described as 'The security subsystem of UNICORE/X', and is NOT >> internal to 'de.fzj.unicore.uas.impl.job'. > > In UNICORE site admins can choose what attribute sources and XACML > decision points they want to use, but the clients (including other > services) do not need to know this. That is what I meant by "internal". > >>> >>> 6.5 >>> >>> These are reqiurements on the security layer (or framework) and should >>> not be used as requirements on BES. >> These security requirements DO have impacts on the BES Client >> interface and on the Job Description document. >> In the text, I have made it clear. > > indeed while preparing the EMI-ES specification, we came to the > following conclusions > 1) when using proxies for delegation, it is necessary to map each data > staging item to a delegated credential (you can check the EMI-ES job > description for details) > 2) the delegation operations are separate from the job management > operations, so they do not necessarily have to be part of the BES client > interface. > > Also, there are existing implementations (UNICORE and Genesis come to my > mind) that do not need this at all, because they do delegation without > proxies. > > So I disagree, 6.5 mostly describes features of the particular security > framework that is used. > >>> >>> 8 BES requirements related to "Application Repositories" >>> >>> While I agree that BES should understand the notion of an "Application" >>> (see e.g. JSDL ApplicationName), I don't agree that the BES should >>> use these for Scheduling. Rather, this is the job of a broker. >> The text is now : >> * Resource selection : The Execution Service MUST use, among others, >> these references to 'Installed Applications' in order to select the most >> adequate computing resource for the Job. >>> >>> 9 BES requirements applying to Accounting >>> >>> As a "MUST", these are out of scope, and should be made "SHOULD". >> No Accounting --> No precise reporting to funding agencies --> No >> funding --> Abrupt and very long shutdown of all services. >> I fully confirm that Accounting is a MUST. >> > > ... operational > >>> >>> 10 Logging/Bookkeeping >>> >>> Same as 9. >> Same as 'Traceability' : >> This is an operational concern : Would you really take the risk that >> the whole EGI becomes a spambot or a scambot ? >> No Logging and Bookkeeping --> No post mortem analysis after attack >> --> Large infection --> Panic --> Abrupt and very long shutdown of all >> services. >> I fully confirm that Logging and Bookkeeping is a MUST. >> > > an operational MUST maybe for some infrastructures, not all. > > E.g. a typical HPC site has its own accounting, its own logging systems > independent of the (Grid) software used to submit jobs to it. > >>> >>> 12 Jobs >>> >>> 12.1 Types of job >>> >>> Support for parallel jobs: it should be "MUST" :-) >> The text is now : >> - The concept of 'Single Job' includes Jobs running >> massively-parallel processes using MPI on one large-scale HPC System. >> The Execution Service MUST understand instructions for usage of MPI >> inside the Job Description document. The Execution Service SHOULD >> transmit these instructions to the Batch System, or return an explicit >> error message if not supported. > > OK > > > > Best regards, > Bernd. > > > -- > Dr. Bernd Schuller > Federated Systems and Data > Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc > Phone: +49 246161-8736 (fax -8556) > > > > > ------------------------------------------------------------------------------------------------ > ------------------------------------------------------------------------------------------------ > Forschungszentrum Juelich GmbH > 52425 Juelich > Sitz der Gesellschaft: Juelich > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher > Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, > Prof. Dr. Sebastian M. Schmidt > ------------------------------------------------------------------------------------------------ > ------------------------------------------------------------------------------------------------ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110916/2ffe9969/attachment-0001.bin From b.schuller at fz-juelich.de Fri Sep 16 08:26:30 2011 From: b.schuller at fz-juelich.de (Bernd Schuller) Date: Fri, 16 Sep 2011 15:26:30 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <4E7333DE.5040809@lal.in2p3.fr> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <000401cc6e15$42840a60$c78c1f20$@riedel@fz-juelich.de> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> <1316119475.24230.60.camel@zam994> <4E7333DE.5040809@lal.in2p3.fr> Message-ID: <1316179590.25767.49.camel@zam994> hi Etienne, On Fri, 2011-09-16 at 13:32 +0200, Etienne URBAH wrote: [...] > > RFC-3820-compliant X509 Proxies > ------------------------------- > The RFC-3820-compliant X509 proxies are fully supported by the jLite > library written in Java by Oleg SUKHOROSLOV and available at > http://code.google.com/p/jlite/ > You must be joking. I was talking about open source software like Apache httpd and Java JDK. jLite just wraps the cog-globus libraries and adds some gLite access APIs. No, thanks. > > Dependency of BES on other grid software components / operational issues > ------------------------------------------------------------------------ > It is very good that we know agree on GLUE 2.0 as base for the > Information System. Otherwise, we could NOT agree on the way to express > references to grid entities in the Job Description document. > > Your comments about chapter 6.5 confirm that the specifications of the > BES Client interface DEPENDS on whether BES supports X509 proxies for > delegation of Security credentials or NOT. > At least this was the EMI-ES v1.0 conclusion, which need not be the final word. The only dependency (in EMI-ES) is the specification which delegated credential is to be used for which data staging item. Delegation can be performed FULLY TRANSPARENT to the BES (for example on a message level as in UNICORE), and the BES interface specification is not dependent on it at all. > Since delegation of Security credentials is a MUST for BES, my > conclusion is that we MUST agree on SECURITY issues (even if some are > operational issues) BEFORE trying to write down BES requirements. > > In the same way, we know that Clients need to perform complex queries on > Jobs. The BES Client interface DEPENDS on whether such queries are > accepted by BES itself, of by a separate Logging and Bookkeeping > service. So, I think that we have to agree on the existence or absence > of a separate Logging and Bookkeeping service BEFORE trying to write > down BES requirements about queries. The BES client interface does not necessarily need to allow to perform complex queries, these should be part of a separate interface. > As a summary : > - Some BES requirements are quite independent from other grid > components, and we can discuss on them immediately. > - But some other BES requirements are DEPENDENT from foundational grid > components or operational issues, in particular Information System, > Security, Logging and Bookkeeping, ... > - Therefore, we have to agree on these other grid components or > operational issues FIRST. > > This is a critical issue, and I propose that we discuss on it at OGF33. Unfortunately I won't be in Lyon, only via phone. Summarising, my main points for the BES interface specification : * specifications must be narrowly scoped and composable (which is the JSDL/BES model anyway). * do not try to specify specific authentication methods. Do not try to specify specific delegation methods. Security is a cross cutting concern and should be dealt with separately. * do not assume a special environment where a BES instance will run. Interactions with other services (except for data access) are optional and should be treated as such. * leave operational aspects to operations. Recommendations for BES implementors may be given, of course. Best regards, Bernd. > > I will answer to your other comments later. > > Best regards. > > ----------------------------------------------------- > Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS > Bat 200 91898 ORSAY France > Tel: +33 1 64 46 84 87 Skype: etienne.urbah > Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr > ----------------------------------------------------- > > > On Thu, 15/09/2011 22:44, Bernd Schuller wrote: > > Hi Etienne, > > > > thanks for the clarifications. So indeed your document is aimed at both: > > > > 1) providing requirements for the actual BES specification ("client > > interface" in your terminology) > > 2) the operation and deployment issues that have to be solved for > > interoperability on an infrastructure level (say EGI and EDGI). > > > > It would be very beneficial for further progress if these two distinct > > concerns could be separated, at least CLEARLY marked in your document. > > > > I have added some more comments inline. > > > > On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: > > > >> Concerning the document named 'Requirements for an improved Basic > >> Execution Service (BES)' and available at > >> http://forge.gridforum.org/sf/go/doc16306 : > >> > >> THANK YOU VERY MUCH for having taken the time to read this document, and > >> for having taken the time to provide comments : > >> > >> Such comments are very useful for the improvement of documents, permit > >> convergence and prepare later agreement. > >> [...] > >> On Thu, 15/09/2011 13:00, Bernd Schuller wrote: > >>> > >>> [...] > >> > >>> 1.4 Methodology > >>> > >>> * ref 4: glite user guide -> out of scope, gLite is too complex and > >>> too specific in its architecture (if there is such a thing) to be useful > >>> as a base for BES > >> Yes, this is a very large user guide for specific usage of gLite by > >> users, and it does NOT provide a clear description of the architecture > >> and of the functionalities. > >> But it is NOT so complex. As soon as I finished reading this guide, > >> it was easy for me to perform reverse engineering and extract the > >> effective architecture (SOA with internal interfaces) and the implied > >> functionalities (which are to be improved). > > > > Indeed it appears that you try to impose gLite specifics (like a logging > > & bookkeeping service or proxy certificates on the transport level) as > > requirements. This would severly limit the BES specification effort, and > > will not be accepted (I hope) by other stakeholders. > > > >> In the text, I have prepended a few words to explain that. > > > > Basically you imply that the "architecture and functionalities" of the > > gLite execution system (together with the PGI work) is somehow the > > guideline to be followed, which I fully disagree with. > > > >>> > >>> 2.4 Collaboration with other services > >>> > >>> While this is important for interoperability, it is unimportant > >>> for the specification of a BES. The BES spec should NOT try to specify > >>> all the interaction with the rest of the world. This is the task of a > >>> "grid architecture specification" like OGSA. > >> My document is NOT targeted only to the specification of the BES > >> Client interface, but to the clear and consistent description of BES > >> context and functional + operational requirements which are really > >> necessary for interoperability. > >> As far as I know, OGSA does NOT take into account GLUE 2.0 yet. > >> Therefore, an up to date 'grid architecture specification' is absolutely > >> necessary. > > > > Glue2 is just an information model, not necessarily a perfect one nor > > the only one. However, I agree an information model has to be adopted > > for BES and any associated information systems. > > > >> If OGSA members consider chapters 2.3 and 2.4 of my document as a > >> 'grid architecture specification' which updates and improves OGSA, I > >> thank them. If they consider that this 'grid architecture > >> specification' does NOT comply with OGSA and competes with it, then I > >> assert that it obsoletes OGSA. > > > > I can't really say, the OGSA group stopped its work a long time ago and > > it's a long time that I looked at the documents. > > > >>> > >>> Specifically, the interactions with security, monitoring, accounting and > >>> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory > >>> part of a BES specification. > >> FAILURE of practical operations is often caused by LACK of early care > >> about operational concerns during specification phase. > > > > Agreed. > > > >> As GIN-GC has > >> proven and documented, this is even more true for interoperability on > >> the field (as opposed to theoretical interoperability at the WSDL level). > >> I confirm that care about operational concerns is REQUIRED for real > >> operations and for practical interoperability on the field. Although > >> operational concerns are NOT part of the BES Client interface, they are > >> REQUIRED for the overall specifications of BES in its context. > >> In the text, I have stressed that the document DOES take into account > >> operational concerns. > >> > >>> > >>> 4. BES non-functional requirements > >>> > >>> 4.1.2 Traceability - should be SHOULD not MUST > >> This is an operational concern : Would you really take the risk that > >> the whole EGI becomes a spambot or a scambot ? > > > > Isn't it already, powered by gLite and used by the wlcg botnet (just > > kidding of course) :-) > > > >> No traceability --> No post mortem analysis after attack --> Large > >> infection --> Panic --> Abrupt and very long shutdown of all services. > >> I fully confirm that traceability is a MUST. > > > > It is an internal detail which any good implementation will provide. > > If BES-A is much easier and more secure to operate than BES-B, admins > > can choose which to install. > > > >>> > >>> 4.1.3 Security > >>> - how can a specification "implement" a policy? You probably > >>> meant "BES implementations SHOULD ..." > >> The text now is 'BES specifications MUST fully take into account the > >> Security Policies ...' > > > > Still no understanding here... let's take traceability > > The relevant EGI policy > > says > > > > "[...] software deployed in the Grid MUST include the > > ability to produce sufficient and relevant logging [...] > > The level of the logging MUST be configured by all service providers, > > including but not limited to the Sites, to produce the required > > information which MUST be retained for a minimum of 90 days." > > > > For example all UNICORE services can be made to comply with this > > by configuration of the logging library we use (Apache Log4j), and by > > not deleting log files for 90 days. > > So this is a feature of the implementation and the administrator in > > charge, not the specification. Thus, your sentence should read "BES > > implementations SHOULD ..." (It is MUST of course only if they want to > > be deployed in EGI) > > One does not try to specify implementation details, at least not in any > > specification I've ever seen (e.g. does the HTTP specification say > > anything about server logging or accepted CAs?). > > > > > > > >> [...] > >>> > >>> 4.2 > >>> all of this is out of scope. For example the UNICORE service > >>> container hosts a number of services including an execution service. > >>> Probably you mean that the execution service SPECIFICATION should be > >>> limited to the execution service and MUST NOT specify accounting, > >>> security etc. > >> 'Well defined and narrow scope' is a general engineering requirement. > >> It is fundamental concern crossing requirements, design, > >> specifications, fabrication, tests, operations, user experience and > >> product maintenance for all types of products, even outside software > >> engineering. > > > > exactly. > > > >> I confirm that 'Well defined and narrow scope' is absolutely REQUIRED > >> for sound software design, implementation, deployment and maintainability. > >> From your comment, I assume that the Execution Service of UNICORE > >> does have a 'Well defined and narrow scope', does have precise > >> interfaces with other UNICORE services, and minimizes overlaps with them. > > > > of course. And UNICORE does not include a L&B service :-) > > > > > >> [...] > >>> 6.1 > >>> > >>> * "SSL certificates MUST be signed by a CA..." this is an operational > >>> decision, and has nothing to do with the BES spec. > >>> For example, a site may run an inhouse deployment of BES using an > >>> in-house CA. This requirement should be deleted. > >> This operational concern is REQUIRED for practical interoperability > >> on the field. I have prepended : > >> * Authentication of Servers : The Execution Service SHOULD permit > >> Clients to authenticate it. If an Execution Service authenticates > >> itself to clients, it MUST permit Clients to really perform this > >> authentication. > > > > This sentence makes no sense to me, sorry. > > Maybe "Server and client SHOULD communicate via a secure channel > > (SSL/TLS)". Even this may not be true in the future, though it is for > > all(?) the Grid systems currently. > > > >>> > >>> 6.3 > >>> > >>> * "For Client authentication, the Execution Service MUST accept all > >>> following authentication methods: Full X509, RFC-3820-compliant X509 > >>> Proxy" > >>> > >>> This requirement is invalid. I agree that it would be nice to be able to > >>> specify authentication methods, but it is impossible. For example > >>> Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface > >>> over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be > >>> valid authentication methods in some circumstances. > >> There are 2 separate requirements : > >> - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy > >> - 1 'MAY' for all other ones. > >>> > >>> Furthermore, making proxies a MUST implies that nonstandard > >>> authentication libraries instead of TLS/SSL must be used, making the BES > >>> implementation insecure. For some implementors (like UNICORE) proxies on > >>> the transport level are very much a no-go. > >> I had clearly specified RFC-3820-compliant X509 Proxy, which ARE > >> standard. > >> Your critics are valid for GSI proxies, which I have taken care NOT > >> to mention. > > > > by "standard" I did not mean that it is an RFC, but software support. > > As opposed to standard SSL/TLS, proxies are almost not supported by > > industry standard tools, for example Apache httpd or the Java JDK. > > One has to rely on custom code, which is notoriously buggy and error > > prone. > > > > Since one important non-functional requirement (for me at least) is to > > be able use standard (off-the-shelf) open source software, having to > > support proxies is a big limitation. > > > >>> 6.4. > >>> > >>> "This authorization mechanism MUST be consistent across all instances of > >>> the Execution Service" > >>> > >>> This violates the autonomy of a site. Site administrators often wish to > >>> stay in control of their resources, and do not accept external > >>> authorisation decision points. And anyway, who cares? Since the AuthZ > >>> mechanism is internal to the BES, it cannot be specified in the > >>> BES spec as such. > >> Interoperability requires a federation of independent administrative > >> domain to agree on common functionalities, interfaces and operations. > >> This DOES sometime violate the autonomy of each individual site. > >> The requirement is NOT that the AUTHZ decision point is external to > >> any site, but that all participating site MUST accept to install inside > >> their site an instance of a commonly validated software implementing the > >> decision point. > > > > No. Each site may choose their own authz decision point, IMO. > > > >> The AUTHZ mechanism MUST NOT be internal to the BES : > > > > Maybe I was not clear. The authz mechanism is invisible for outside > > parties (like clients). It can be an external component, an internal > > component, whatever, it is up to the BES implementor and the site admin. > > > >> For example, > >> in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package > >> is described as 'The security subsystem of UNICORE/X', and is NOT > >> internal to 'de.fzj.unicore.uas.impl.job'. > > > > In UNICORE site admins can choose what attribute sources and XACML > > decision points they want to use, but the clients (including other > > services) do not need to know this. That is what I meant by "internal". > > > >>> > >>> 6.5 > >>> > >>> These are reqiurements on the security layer (or framework) and should > >>> not be used as requirements on BES. > >> These security requirements DO have impacts on the BES Client > >> interface and on the Job Description document. > >> In the text, I have made it clear. > > > > indeed while preparing the EMI-ES specification, we came to the > > following conclusions > > 1) when using proxies for delegation, it is necessary to map each data > > staging item to a delegated credential (you can check the EMI-ES job > > description for details) > > 2) the delegation operations are separate from the job management > > operations, so they do not necessarily have to be part of the BES client > > interface. > > > > Also, there are existing implementations (UNICORE and Genesis come to my > > mind) that do not need this at all, because they do delegation without > > proxies. > > > > So I disagree, 6.5 mostly describes features of the particular security > > framework that is used. > > > >>> > >>> 8 BES requirements related to "Application Repositories" > >>> > >>> While I agree that BES should understand the notion of an "Application" > >>> (see e.g. JSDL ApplicationName), I don't agree that the BES should > >>> use these for Scheduling. Rather, this is the job of a broker. > >> The text is now : > >> * Resource selection : The Execution Service MUST use, among others, > >> these references to 'Installed Applications' in order to select the most > >> adequate computing resource for the Job. > >>> > >>> 9 BES requirements applying to Accounting > >>> > >>> As a "MUST", these are out of scope, and should be made "SHOULD". > >> No Accounting --> No precise reporting to funding agencies --> No > >> funding --> Abrupt and very long shutdown of all services. > >> I fully confirm that Accounting is a MUST. > >> > > > > ... operational > > > >>> > >>> 10 Logging/Bookkeeping > >>> > >>> Same as 9. > >> Same as 'Traceability' : > >> This is an operational concern : Would you really take the risk that > >> the whole EGI becomes a spambot or a scambot ? > >> No Logging and Bookkeeping --> No post mortem analysis after attack > >> --> Large infection --> Panic --> Abrupt and very long shutdown of all > >> services. > >> I fully confirm that Logging and Bookkeeping is a MUST. > >> > > > > an operational MUST maybe for some infrastructures, not all. > > > > E.g. a typical HPC site has its own accounting, its own logging systems > > independent of the (Grid) software used to submit jobs to it. > > > >>> > >>> 12 Jobs > >>> > >>> 12.1 Types of job > >>> > >>> Support for parallel jobs: it should be "MUST" :-) > >> The text is now : > >> - The concept of 'Single Job' includes Jobs running > >> massively-parallel processes using MPI on one large-scale HPC System. > >> The Execution Service MUST understand instructions for usage of MPI > >> inside the Job Description document. The Execution Service SHOULD > >> transmit these instructions to the Batch System, or return an explicit > >> error message if not supported. > > > > OK > > > > > > > > Best regards, > > Bernd. > > > > > > -- > > Dr. Bernd Schuller > > Federated Systems and Data > > Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc > > Phone: +49 246161-8736 (fax -8556) > > > > > > > > > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > > Forschungszentrum Juelich GmbH > > 52425 Juelich > > Sitz der Gesellschaft: Juelich > > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > > Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher > > Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), > > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, > > Prof. Dr. Sebastian M. Schmidt > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > -- Dr. Bernd Schuller Federated Systems and Data Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc Phone: +49 246161-8736 (fax -8556) From andre at merzky.net Fri Sep 16 08:36:26 2011 From: andre at merzky.net (Andre Merzky) Date: Fri, 16 Sep 2011 15:36:26 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <1316179590.25767.49.camel@zam994> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> <1316119475.24230.60.camel@zam994> <4E7333DE.5040809@lal.in2p3.fr> <1316179590.25767.49.camel@zam994> Message-ID: +1 on all points. Cheers, Andre. 2011/9/16 Bernd Schuller : > hi Etienne, > > On Fri, 2011-09-16 at 13:32 +0200, Etienne URBAH wrote: > [...] >> >> RFC-3820-compliant X509 Proxies >> ------------------------------- >> The RFC-3820-compliant X509 proxies are fully supported by the jLite >> library written in Java by Oleg SUKHOROSLOV and available at >> http://code.google.com/p/jlite/ >> > > You must be joking. I was talking about open source software like Apache > httpd and Java JDK. jLite just wraps the cog-globus libraries and adds > some gLite access APIs. No, thanks. > >> >> Dependency of BES on other grid software components / operational issues >> ------------------------------------------------------------------------ >> It is very good that we know agree on GLUE 2.0 as base for the >> Information System. ?Otherwise, we could NOT agree on the way to express >> references to grid entities in the Job Description document. >> >> Your comments about chapter 6.5 confirm that the specifications of the >> BES Client interface DEPENDS on whether BES supports X509 proxies for >> delegation of Security credentials or NOT. >> > > At least this was the EMI-ES v1.0 conclusion, which need not be the > final word. The only dependency (in EMI-ES) is the specification which > delegated credential is to be used for which data staging item. > > Delegation can be performed FULLY TRANSPARENT to the BES (for example on > a message level as in UNICORE), and the BES interface specification is > not dependent on it at all. > >> Since delegation of Security credentials is a MUST for BES, my >> conclusion is that we MUST agree on SECURITY issues (even if some are >> operational issues) BEFORE trying to write down BES requirements. >> >> In the same way, we know that Clients need to perform complex queries on >> Jobs. The BES Client interface DEPENDS on whether such queries are >> accepted by BES itself, of by a separate Logging and Bookkeeping >> service. ?So, I think that we have to agree on the existence or absence >> of a separate Logging and Bookkeeping service BEFORE trying to write >> down BES requirements about queries. > > > The BES client interface does not necessarily need to allow to perform > complex queries, these should be part of a separate interface. > > >> As a summary : >> - ?Some BES requirements are quite independent from other grid >> components, and we can discuss on them immediately. >> - ?But some other BES requirements are DEPENDENT from foundational grid >> components or operational issues, in particular Information System, >> Security, Logging and Bookkeeping, ... >> - ?Therefore, we have to agree on these other grid components or >> operational issues FIRST. >> >> This is a critical issue, and I propose that we discuss on it at OGF33. > > Unfortunately I won't be in Lyon, only via phone. > > Summarising, my main points for the BES interface specification : > > ?* specifications must be narrowly scoped and composable (which is the > JSDL/BES model anyway). > > ?* do not try to specify specific authentication methods. Do not try to > specify specific delegation methods. Security is a cross cutting concern > and should be dealt with separately. > > ?* do not assume a special environment where a BES instance will run. > Interactions with other services (except for data access) are optional > and should be treated as such. > > ?* leave operational aspects to operations. Recommendations for BES > implementors may be given, of course. > > > Best regards, > Bernd. > >> >> I will answer to your other comments later. >> >> Best regards. >> >> ----------------------------------------------------- >> Etienne URBAH ? ? ? ? LAL, Univ Paris-Sud, IN2P3/CNRS >> ? ? ? ? ? ? ? ? ? ? ? ?Bat 200 ? 91898 ORSAY ? ?France >> Tel: +33 1 64 46 84 87 ? ? ?Skype: etienne.urbah >> Mob: +33 6 22 30 53 27 ? ? ?mailto:urbah at lal.in2p3.fr >> ----------------------------------------------------- >> >> >> On Thu, 15/09/2011 22:44, Bernd Schuller wrote: >> > Hi Etienne, >> > >> > thanks for the clarifications. So indeed your document is aimed at both: >> > >> > 1) providing requirements for the actual BES specification ("client >> > interface" in your terminology) >> > 2) the operation and deployment issues that have to be solved for >> > interoperability on an infrastructure level (say EGI and EDGI). >> > >> > It would be very beneficial for further progress if these two distinct >> > concerns could be separated, at least CLEARLY marked in your document. >> > >> > I have added some more comments inline. >> > >> > On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: >> > >> >> Concerning the document named 'Requirements for an improved Basic >> >> Execution Service (BES)' and available at >> >> http://forge.gridforum.org/sf/go/doc16306 : >> >> >> >> THANK YOU VERY MUCH for having taken the time to read this document, and >> >> for having taken the time to provide comments : >> >> >> >> Such comments are very useful for the improvement of documents, permit >> >> convergence and prepare later agreement. >> >> [...] >> >> On Thu, 15/09/2011 13:00, Bernd Schuller wrote: >> >>> >> >>> [...] >> >> >> >>> 1.4 Methodology >> >>> >> >>> ? ?* ref 4: glite user guide -> ? out of scope, gLite is too complex and >> >>> too specific in its architecture (if there is such a thing) to be useful >> >>> as a base for BES >> >> ? ? Yes, this is a very large user guide for specific usage of gLite by >> >> users, and it does NOT provide a clear description of the architecture >> >> and of the functionalities. >> >> ? ? But it is NOT so complex. ?As soon as I finished reading this guide, >> >> it was easy for me to perform reverse engineering and extract the >> >> effective architecture (SOA with internal interfaces) and the implied >> >> functionalities (which are to be improved). >> > >> > Indeed it appears that you try to impose gLite specifics (like a logging >> > & ?bookkeeping service or proxy certificates on the transport level) as >> > requirements. This would severly limit the BES specification effort, and >> > will not be accepted (I hope) by other stakeholders. >> > >> >> ? ? In the text, I have prepended a few words to explain that. >> > >> > Basically you imply that the "architecture and functionalities" of the >> > gLite execution system (together with the PGI work) is somehow the >> > guideline to be followed, which I fully disagree with. >> > >> >>> >> >>> 2.4 Collaboration with other services >> >>> >> >>> While this is important for interoperability, it is unimportant >> >>> for the specification of a BES. The BES spec should NOT try to specify >> >>> all the interaction with the rest of the world. This is the task of a >> >>> "grid architecture specification" like OGSA. >> >> ? ? My document is NOT targeted only to the specification of the BES >> >> Client interface, but to the clear and consistent description of BES >> >> context and functional + operational requirements which are really >> >> necessary for interoperability. >> >> ? ? As far as I know, OGSA does NOT take into account GLUE 2.0 yet. >> >> Therefore, an up to date 'grid architecture specification' is absolutely >> >> necessary. >> > >> > Glue2 is just an information model, not necessarily a perfect one nor >> > the only one. However, I agree an information model has to be adopted >> > for BES and any associated information systems. >> > >> >> ? ? If OGSA members consider chapters 2.3 and 2.4 of my document as a >> >> 'grid architecture specification' which updates and improves OGSA, I >> >> thank them. ?If they consider that this 'grid architecture >> >> specification' does NOT comply with OGSA and competes with it, then I >> >> assert that it obsoletes OGSA. >> > >> > I can't really say, the OGSA group stopped its work a long time ago and >> > it's a long time that I looked at the documents. >> > >> >>> >> >>> Specifically, the interactions with security, monitoring, accounting and >> >>> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory >> >>> part of a BES specification. >> >> ? ? FAILURE of practical operations is often caused by LACK of early care >> >> about operational concerns during specification phase. >> > >> > Agreed. >> > >> >> ? As GIN-GC has >> >> proven and documented, this is even more true for interoperability on >> >> the field (as opposed to theoretical interoperability at the WSDL level). >> >> ? ? I confirm that care about operational concerns is REQUIRED for real >> >> operations and for practical interoperability on the field. ?Although >> >> operational concerns are NOT part of the BES Client interface, they are >> >> REQUIRED for the overall specifications of BES in its context. >> >> ? ? In the text, I have stressed that the document DOES take into account >> >> operational concerns. >> >> >> >>> >> >>> 4. BES non-functional requirements >> >>> >> >>> 4.1.2 Traceability - should be SHOULD not MUST >> >> ? ? This is an operational concern : ?Would you really take the risk that >> >> the whole EGI becomes a spambot or a scambot ? >> > >> > Isn't it already, powered by gLite and used by the wlcg botnet (just >> > kidding of course) :-) >> > >> >> ? ? No traceability --> ?No post mortem analysis after attack --> ?Large >> >> infection --> ?Panic --> ?Abrupt and very long shutdown of all services. >> >> ? ? I fully confirm that traceability is a MUST. >> > >> > It is an internal detail which any good implementation will provide. >> > If BES-A is much easier and more secure to operate than BES-B, admins >> > can choose which to install. >> > >> >>> >> >>> 4.1.3 Security >> >>> ? ?- how can a specification "implement" a policy? You probably >> >>> ? ? ?meant "BES implementations SHOULD ..." >> >> ? ? The text now is 'BES specifications MUST fully take into account the >> >> Security Policies ...' >> > >> > Still no understanding here... let's take traceability >> > The relevant EGI policy >> > ?says >> > >> > "[...] software deployed in the Grid MUST include the >> > ability to produce sufficient and relevant logging [...] >> > The level of the logging MUST be configured by all service providers, >> > including but not limited to the Sites, to produce the required >> > information which MUST be retained for a minimum of 90 days." >> > >> > For example all UNICORE services can be made to comply with this >> > by configuration of the logging library we use (Apache Log4j), and by >> > not deleting log files for 90 days. >> > So this is a feature of the implementation and the administrator in >> > charge, not the specification. Thus, your sentence should read "BES >> > implementations SHOULD ..." (It is MUST of course only if they want to >> > be deployed in EGI) >> > One does not try to specify implementation details, at least not in any >> > specification I've ever seen (e.g. does the HTTP specification say >> > anything about server logging or accepted CAs?). >> > >> > >> > >> >> [...] >> >>> >> >>> 4.2 >> >>> ? ?all of this is out of scope. For example the UNICORE service >> >>> container hosts a number of services including an execution service. >> >>> Probably you mean that the execution service SPECIFICATION should be >> >>> limited to the execution service and MUST NOT specify accounting, >> >>> security etc. >> >> ? ? 'Well defined and narrow scope' is a general engineering requirement. >> >> ? ?It is fundamental concern crossing requirements, design, >> >> specifications, fabrication, tests, operations, user experience and >> >> product maintenance for all types of products, even outside software >> >> engineering. >> > >> > exactly. >> > >> >> ? ? I confirm that 'Well defined and narrow scope' is absolutely REQUIRED >> >> for sound software design, implementation, deployment and maintainability. >> >> ? ? From your comment, I assume that the Execution Service of UNICORE >> >> does have a 'Well defined and narrow scope', does have precise >> >> interfaces with other UNICORE services, and minimizes overlaps with them. >> > >> > of course. And UNICORE does not include a L&B service :-) >> > >> > >> >> [...] >> >>> 6.1 >> >>> >> >>> ? ?* "SSL certificates MUST be signed by a CA..." this is an operational >> >>> decision, and has nothing to do with the BES spec. >> >>> For example, a site may run an inhouse deployment of BES using an >> >>> in-house CA. This requirement should be deleted. >> >> ? ? This operational concern is REQUIRED for practical interoperability >> >> on the field. ?I have prepended : >> >> ? ? * Authentication of Servers : ?The Execution Service SHOULD permit >> >> Clients to authenticate it. ?If an Execution Service authenticates >> >> itself to clients, it MUST permit Clients to really perform this >> >> authentication. >> > >> > This sentence makes no sense to me, sorry. >> > Maybe "Server and client SHOULD communicate via a secure channel >> > (SSL/TLS)". Even this may not be true in the future, though it is for >> > all(?) the Grid systems currently. >> > >> >>> >> >>> 6.3 >> >>> >> >>> * "For Client authentication, the Execution Service MUST accept all >> >>> following authentication methods: ?Full X509, ?RFC-3820-compliant X509 >> >>> Proxy" >> >>> >> >>> This requirement is invalid. I agree that it would be nice to be able to >> >>> specify authentication methods, but it is impossible. For example >> >>> Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface >> >>> over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be >> >>> valid authentication methods in some circumstances. >> >> ? ? There are 2 separate requirements : >> >> ? ? - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy >> >> ? ? - 1 'MAY' ?for all other ones. >> >>> >> >>> Furthermore, making proxies a MUST implies that nonstandard >> >>> authentication libraries instead of TLS/SSL must be used, making the BES >> >>> implementation insecure. For some implementors (like UNICORE) proxies on >> >>> the transport level are very much a no-go. >> >> ? ? I had clearly specified RFC-3820-compliant X509 Proxy, which ARE >> >> standard. >> >> ? ? Your critics are valid for GSI proxies, which I have taken care NOT >> >> to mention. >> > >> > by "standard" I did not mean that it is an RFC, but software support. >> > As opposed to standard SSL/TLS, proxies are almost not supported by >> > industry standard tools, for example Apache httpd or the Java JDK. >> > One has to rely on custom code, which is notoriously buggy and error >> > prone. >> > >> > Since one important non-functional requirement (for me at least) is to >> > be able use standard (off-the-shelf) open source software, having to >> > support proxies is a big limitation. >> > >> >>> 6.4. >> >>> >> >>> "This authorization mechanism MUST be consistent across all instances of >> >>> the Execution Service" >> >>> >> >>> This violates the autonomy of a site. Site administrators often wish to >> >>> stay in control of their resources, and do not accept external >> >>> authorisation decision points. And anyway, who cares? Since the AuthZ >> >>> mechanism is internal to the BES, it cannot be specified in the >> >>> BES spec as such. >> >> ? ? Interoperability requires a federation of independent administrative >> >> domain to agree on common functionalities, interfaces and operations. >> >> ? ? This DOES sometime violate the autonomy of each individual site. >> >> ? ? The requirement is NOT that the AUTHZ decision point is external to >> >> any site, but that all participating site MUST accept to install inside >> >> their site an instance of a commonly validated software implementing the >> >> decision point. >> > >> > No. Each site may choose their own authz decision point, IMO. >> > >> >> ? ? The AUTHZ mechanism MUST NOT be internal to the BES : >> > >> > Maybe I was not clear. The authz mechanism is invisible for outside >> > parties (like clients). It can be an external component, an internal >> > component, whatever, it is up to the BES implementor and the site admin. >> > >> >> For example, >> >> in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package >> >> is described as 'The security subsystem of UNICORE/X', and is NOT >> >> internal to 'de.fzj.unicore.uas.impl.job'. >> > >> > In UNICORE site admins can choose what attribute sources and XACML >> > decision points they want to use, but the clients (including other >> > services) do not need to know this. That is what I meant by "internal". >> > >> >>> >> >>> 6.5 >> >>> >> >>> These are reqiurements on the security layer (or framework) and should >> >>> not be used as requirements on BES. >> >> ? ? These security requirements DO have impacts on the BES Client >> >> interface and on the Job Description document. >> >> ? ? In the text, I have made it clear. >> > >> > indeed while preparing the EMI-ES specification, we came to the >> > following conclusions >> > 1) when using proxies for delegation, it is necessary to map each data >> > staging item to a delegated credential (you can check the EMI-ES job >> > description for details) >> > 2) the delegation operations are separate from the job management >> > operations, so they do not necessarily have to be part of the BES client >> > interface. >> > >> > Also, there are existing implementations (UNICORE and Genesis come to my >> > mind) that do not need this at all, because they do delegation without >> > proxies. >> > >> > So I disagree, 6.5 mostly describes features of the particular security >> > framework that is used. >> > >> >>> >> >>> 8 BES requirements related to "Application Repositories" >> >>> >> >>> While I agree that BES should understand the notion of an "Application" >> >>> (see e.g. JSDL ApplicationName), I don't agree that the BES should >> >>> use these for Scheduling. Rather, this is the job of a broker. >> >> ? ? The text is now : >> >> ? ? * Resource selection : ?The Execution Service MUST use, among others, >> >> these references to 'Installed Applications' in order to select the most >> >> adequate computing resource for the Job. >> >>> >> >>> 9 BES requirements applying to Accounting >> >>> >> >>> As a "MUST", these are out of scope, and should be made "SHOULD". >> >> ? ? No Accounting --> ?No precise reporting to funding agencies --> ?No >> >> funding --> ?Abrupt and very long shutdown of all services. >> >> ? ? I fully confirm that Accounting is a MUST. >> >> >> > >> > ... operational >> > >> >>> >> >>> 10 Logging/Bookkeeping >> >>> >> >>> Same as 9. >> >> ? ? Same as 'Traceability' : >> >> ? ? This is an operational concern : ?Would you really take the risk that >> >> the whole EGI becomes a spambot or a scambot ? >> >> ? ? No Logging and Bookkeeping --> ?No post mortem analysis after attack >> >> --> ?Large infection --> ?Panic --> ?Abrupt and very long shutdown of all >> >> services. >> >> ? ? I fully confirm that Logging and Bookkeeping is a MUST. >> >> >> > >> > an operational MUST maybe for some infrastructures, not all. >> > >> > E.g. a typical HPC site has its own accounting, its own logging systems >> > independent of the (Grid) software used to submit jobs to it. >> > >> >>> >> >>> 12 Jobs >> >>> >> >>> 12.1 Types of job >> >>> >> >>> Support for parallel jobs: it should be "MUST" :-) >> >> ? ? The text is now : >> >> ? ? - The concept of 'Single Job' includes Jobs running >> >> massively-parallel processes using MPI on one large-scale HPC System. >> >> The Execution Service MUST understand instructions for usage of MPI >> >> inside the Job Description document. ?The Execution Service SHOULD >> >> transmit these instructions to the Batch System, or return an explicit >> >> error message if not supported. >> > >> > OK >> > >> > >> > >> > Best regards, >> > Bernd. >> > >> > >> > -- >> > Dr. Bernd Schuller >> > Federated Systems and Data >> > Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >> > Phone: +49 246161-8736 (fax -8556) >> > >> > >> > >> > >> > ------------------------------------------------------------------------------------------------ >> > ------------------------------------------------------------------------------------------------ >> > Forschungszentrum Juelich GmbH >> > 52425 Juelich >> > Sitz der Gesellschaft: Juelich >> > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >> > Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher >> > Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), >> > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >> > Prof. Dr. Sebastian M. Schmidt >> > ------------------------------------------------------------------------------------------------ >> > ------------------------------------------------------------------------------------------------ >> > > -- > Dr. Bernd Schuller > Federated Systems and Data > Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc > Phone: +49 246161-8736 (fax -8556) > > > > -- Nothing is ever easy... From smartin at mcs.anl.gov Mon Sep 19 10:57:56 2011 From: smartin at mcs.anl.gov (Stuart Martin) Date: Mon, 19 Sep 2011 10:57:56 -0500 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> <1316119475.24230.60.camel@zam994> <4E7333DE.5040809@lal.in2p3.fr> <1316179590.25767.49.camel@zam994> Message-ID: <7BC2FE4B-CC94-42BA-88E2-15C78AD45A2E@mcs.anl.gov> I just wanted to point out that RFC-3820 compliant X.509 proxies are supported in OpenSSL. Several project (e.g., LIGO) use OpenSSL with proxy certs with the Apache server. Also, jGlobus 2 has support for RFC-3820 compliant X.509 proxies. Included is an implementation of a java SSL TrustManager that has proxy processing. -Stu On Sep 16, 2011, at Sep 16, 8:36 AM, Andre Merzky wrote: > +1 on all points. > > Cheers, Andre. > > > 2011/9/16 Bernd Schuller : >> hi Etienne, >> >> On Fri, 2011-09-16 at 13:32 +0200, Etienne URBAH wrote: >> [...] >>> >>> RFC-3820-compliant X509 Proxies >>> ------------------------------- >>> The RFC-3820-compliant X509 proxies are fully supported by the jLite >>> library written in Java by Oleg SUKHOROSLOV and available at >>> http://code.google.com/p/jlite/ >>> >> >> You must be joking. I was talking about open source software like Apache >> httpd and Java JDK. jLite just wraps the cog-globus libraries and adds >> some gLite access APIs. No, thanks. >> >>> >>> Dependency of BES on other grid software components / operational issues >>> ------------------------------------------------------------------------ >>> It is very good that we know agree on GLUE 2.0 as base for the >>> Information System. Otherwise, we could NOT agree on the way to express >>> references to grid entities in the Job Description document. >>> >>> Your comments about chapter 6.5 confirm that the specifications of the >>> BES Client interface DEPENDS on whether BES supports X509 proxies for >>> delegation of Security credentials or NOT. >>> >> >> At least this was the EMI-ES v1.0 conclusion, which need not be the >> final word. The only dependency (in EMI-ES) is the specification which >> delegated credential is to be used for which data staging item. >> >> Delegation can be performed FULLY TRANSPARENT to the BES (for example on >> a message level as in UNICORE), and the BES interface specification is >> not dependent on it at all. >> >>> Since delegation of Security credentials is a MUST for BES, my >>> conclusion is that we MUST agree on SECURITY issues (even if some are >>> operational issues) BEFORE trying to write down BES requirements. >>> >>> In the same way, we know that Clients need to perform complex queries on >>> Jobs. The BES Client interface DEPENDS on whether such queries are >>> accepted by BES itself, of by a separate Logging and Bookkeeping >>> service. So, I think that we have to agree on the existence or absence >>> of a separate Logging and Bookkeeping service BEFORE trying to write >>> down BES requirements about queries. >> >> >> The BES client interface does not necessarily need to allow to perform >> complex queries, these should be part of a separate interface. >> >> >>> As a summary : >>> - Some BES requirements are quite independent from other grid >>> components, and we can discuss on them immediately. >>> - But some other BES requirements are DEPENDENT from foundational grid >>> components or operational issues, in particular Information System, >>> Security, Logging and Bookkeeping, ... >>> - Therefore, we have to agree on these other grid components or >>> operational issues FIRST. >>> >>> This is a critical issue, and I propose that we discuss on it at OGF33. >> >> Unfortunately I won't be in Lyon, only via phone. >> >> Summarising, my main points for the BES interface specification : >> >> * specifications must be narrowly scoped and composable (which is the >> JSDL/BES model anyway). >> >> * do not try to specify specific authentication methods. Do not try to >> specify specific delegation methods. Security is a cross cutting concern >> and should be dealt with separately. >> >> * do not assume a special environment where a BES instance will run. >> Interactions with other services (except for data access) are optional >> and should be treated as such. >> >> * leave operational aspects to operations. Recommendations for BES >> implementors may be given, of course. >> >> >> Best regards, >> Bernd. >> >>> >>> I will answer to your other comments later. >>> >>> Best regards. >>> >>> ----------------------------------------------------- >>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >>> Bat 200 91898 ORSAY France >>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >>> ----------------------------------------------------- >>> >>> >>> On Thu, 15/09/2011 22:44, Bernd Schuller wrote: >>>> Hi Etienne, >>>> >>>> thanks for the clarifications. So indeed your document is aimed at both: >>>> >>>> 1) providing requirements for the actual BES specification ("client >>>> interface" in your terminology) >>>> 2) the operation and deployment issues that have to be solved for >>>> interoperability on an infrastructure level (say EGI and EDGI). >>>> >>>> It would be very beneficial for further progress if these two distinct >>>> concerns could be separated, at least CLEARLY marked in your document. >>>> >>>> I have added some more comments inline. >>>> >>>> On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: >>>> >>>>> Concerning the document named 'Requirements for an improved Basic >>>>> Execution Service (BES)' and available at >>>>> http://forge.gridforum.org/sf/go/doc16306 : >>>>> >>>>> THANK YOU VERY MUCH for having taken the time to read this document, and >>>>> for having taken the time to provide comments : >>>>> >>>>> Such comments are very useful for the improvement of documents, permit >>>>> convergence and prepare later agreement. >>>>> [...] >>>>> On Thu, 15/09/2011 13:00, Bernd Schuller wrote: >>>>>> >>>>>> [...] >>>>> >>>>>> 1.4 Methodology >>>>>> >>>>>> * ref 4: glite user guide -> out of scope, gLite is too complex and >>>>>> too specific in its architecture (if there is such a thing) to be useful >>>>>> as a base for BES >>>>> Yes, this is a very large user guide for specific usage of gLite by >>>>> users, and it does NOT provide a clear description of the architecture >>>>> and of the functionalities. >>>>> But it is NOT so complex. As soon as I finished reading this guide, >>>>> it was easy for me to perform reverse engineering and extract the >>>>> effective architecture (SOA with internal interfaces) and the implied >>>>> functionalities (which are to be improved). >>>> >>>> Indeed it appears that you try to impose gLite specifics (like a logging >>>> & bookkeeping service or proxy certificates on the transport level) as >>>> requirements. This would severly limit the BES specification effort, and >>>> will not be accepted (I hope) by other stakeholders. >>>> >>>>> In the text, I have prepended a few words to explain that. >>>> >>>> Basically you imply that the "architecture and functionalities" of the >>>> gLite execution system (together with the PGI work) is somehow the >>>> guideline to be followed, which I fully disagree with. >>>> >>>>>> >>>>>> 2.4 Collaboration with other services >>>>>> >>>>>> While this is important for interoperability, it is unimportant >>>>>> for the specification of a BES. The BES spec should NOT try to specify >>>>>> all the interaction with the rest of the world. This is the task of a >>>>>> "grid architecture specification" like OGSA. >>>>> My document is NOT targeted only to the specification of the BES >>>>> Client interface, but to the clear and consistent description of BES >>>>> context and functional + operational requirements which are really >>>>> necessary for interoperability. >>>>> As far as I know, OGSA does NOT take into account GLUE 2.0 yet. >>>>> Therefore, an up to date 'grid architecture specification' is absolutely >>>>> necessary. >>>> >>>> Glue2 is just an information model, not necessarily a perfect one nor >>>> the only one. However, I agree an information model has to be adopted >>>> for BES and any associated information systems. >>>> >>>>> If OGSA members consider chapters 2.3 and 2.4 of my document as a >>>>> 'grid architecture specification' which updates and improves OGSA, I >>>>> thank them. If they consider that this 'grid architecture >>>>> specification' does NOT comply with OGSA and competes with it, then I >>>>> assert that it obsoletes OGSA. >>>> >>>> I can't really say, the OGSA group stopped its work a long time ago and >>>> it's a long time that I looked at the documents. >>>> >>>>>> >>>>>> Specifically, the interactions with security, monitoring, accounting and >>>>>> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory >>>>>> part of a BES specification. >>>>> FAILURE of practical operations is often caused by LACK of early care >>>>> about operational concerns during specification phase. >>>> >>>> Agreed. >>>> >>>>> As GIN-GC has >>>>> proven and documented, this is even more true for interoperability on >>>>> the field (as opposed to theoretical interoperability at the WSDL level). >>>>> I confirm that care about operational concerns is REQUIRED for real >>>>> operations and for practical interoperability on the field. Although >>>>> operational concerns are NOT part of the BES Client interface, they are >>>>> REQUIRED for the overall specifications of BES in its context. >>>>> In the text, I have stressed that the document DOES take into account >>>>> operational concerns. >>>>> >>>>>> >>>>>> 4. BES non-functional requirements >>>>>> >>>>>> 4.1.2 Traceability - should be SHOULD not MUST >>>>> This is an operational concern : Would you really take the risk that >>>>> the whole EGI becomes a spambot or a scambot ? >>>> >>>> Isn't it already, powered by gLite and used by the wlcg botnet (just >>>> kidding of course) :-) >>>> >>>>> No traceability --> No post mortem analysis after attack --> Large >>>>> infection --> Panic --> Abrupt and very long shutdown of all services. >>>>> I fully confirm that traceability is a MUST. >>>> >>>> It is an internal detail which any good implementation will provide. >>>> If BES-A is much easier and more secure to operate than BES-B, admins >>>> can choose which to install. >>>> >>>>>> >>>>>> 4.1.3 Security >>>>>> - how can a specification "implement" a policy? You probably >>>>>> meant "BES implementations SHOULD ..." >>>>> The text now is 'BES specifications MUST fully take into account the >>>>> Security Policies ...' >>>> >>>> Still no understanding here... let's take traceability >>>> The relevant EGI policy >>>> says >>>> >>>> "[...] software deployed in the Grid MUST include the >>>> ability to produce sufficient and relevant logging [...] >>>> The level of the logging MUST be configured by all service providers, >>>> including but not limited to the Sites, to produce the required >>>> information which MUST be retained for a minimum of 90 days." >>>> >>>> For example all UNICORE services can be made to comply with this >>>> by configuration of the logging library we use (Apache Log4j), and by >>>> not deleting log files for 90 days. >>>> So this is a feature of the implementation and the administrator in >>>> charge, not the specification. Thus, your sentence should read "BES >>>> implementations SHOULD ..." (It is MUST of course only if they want to >>>> be deployed in EGI) >>>> One does not try to specify implementation details, at least not in any >>>> specification I've ever seen (e.g. does the HTTP specification say >>>> anything about server logging or accepted CAs?). >>>> >>>> >>>> >>>>> [...] >>>>>> >>>>>> 4.2 >>>>>> all of this is out of scope. For example the UNICORE service >>>>>> container hosts a number of services including an execution service. >>>>>> Probably you mean that the execution service SPECIFICATION should be >>>>>> limited to the execution service and MUST NOT specify accounting, >>>>>> security etc. >>>>> 'Well defined and narrow scope' is a general engineering requirement. >>>>> It is fundamental concern crossing requirements, design, >>>>> specifications, fabrication, tests, operations, user experience and >>>>> product maintenance for all types of products, even outside software >>>>> engineering. >>>> >>>> exactly. >>>> >>>>> I confirm that 'Well defined and narrow scope' is absolutely REQUIRED >>>>> for sound software design, implementation, deployment and maintainability. >>>>> From your comment, I assume that the Execution Service of UNICORE >>>>> does have a 'Well defined and narrow scope', does have precise >>>>> interfaces with other UNICORE services, and minimizes overlaps with them. >>>> >>>> of course. And UNICORE does not include a L&B service :-) >>>> >>>> >>>>> [...] >>>>>> 6.1 >>>>>> >>>>>> * "SSL certificates MUST be signed by a CA..." this is an operational >>>>>> decision, and has nothing to do with the BES spec. >>>>>> For example, a site may run an inhouse deployment of BES using an >>>>>> in-house CA. This requirement should be deleted. >>>>> This operational concern is REQUIRED for practical interoperability >>>>> on the field. I have prepended : >>>>> * Authentication of Servers : The Execution Service SHOULD permit >>>>> Clients to authenticate it. If an Execution Service authenticates >>>>> itself to clients, it MUST permit Clients to really perform this >>>>> authentication. >>>> >>>> This sentence makes no sense to me, sorry. >>>> Maybe "Server and client SHOULD communicate via a secure channel >>>> (SSL/TLS)". Even this may not be true in the future, though it is for >>>> all(?) the Grid systems currently. >>>> >>>>>> >>>>>> 6.3 >>>>>> >>>>>> * "For Client authentication, the Execution Service MUST accept all >>>>>> following authentication methods: Full X509, RFC-3820-compliant X509 >>>>>> Proxy" >>>>>> >>>>>> This requirement is invalid. I agree that it would be nice to be able to >>>>>> specify authentication methods, but it is impossible. For example >>>>>> Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface >>>>>> over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be >>>>>> valid authentication methods in some circumstances. >>>>> There are 2 separate requirements : >>>>> - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy >>>>> - 1 'MAY' for all other ones. >>>>>> >>>>>> Furthermore, making proxies a MUST implies that nonstandard >>>>>> authentication libraries instead of TLS/SSL must be used, making the BES >>>>>> implementation insecure. For some implementors (like UNICORE) proxies on >>>>>> the transport level are very much a no-go. >>>>> I had clearly specified RFC-3820-compliant X509 Proxy, which ARE >>>>> standard. >>>>> Your critics are valid for GSI proxies, which I have taken care NOT >>>>> to mention. >>>> >>>> by "standard" I did not mean that it is an RFC, but software support. >>>> As opposed to standard SSL/TLS, proxies are almost not supported by >>>> industry standard tools, for example Apache httpd or the Java JDK. >>>> One has to rely on custom code, which is notoriously buggy and error >>>> prone. >>>> >>>> Since one important non-functional requirement (for me at least) is to >>>> be able use standard (off-the-shelf) open source software, having to >>>> support proxies is a big limitation. >>>> >>>>>> 6.4. >>>>>> >>>>>> "This authorization mechanism MUST be consistent across all instances of >>>>>> the Execution Service" >>>>>> >>>>>> This violates the autonomy of a site. Site administrators often wish to >>>>>> stay in control of their resources, and do not accept external >>>>>> authorisation decision points. And anyway, who cares? Since the AuthZ >>>>>> mechanism is internal to the BES, it cannot be specified in the >>>>>> BES spec as such. >>>>> Interoperability requires a federation of independent administrative >>>>> domain to agree on common functionalities, interfaces and operations. >>>>> This DOES sometime violate the autonomy of each individual site. >>>>> The requirement is NOT that the AUTHZ decision point is external to >>>>> any site, but that all participating site MUST accept to install inside >>>>> their site an instance of a commonly validated software implementing the >>>>> decision point. >>>> >>>> No. Each site may choose their own authz decision point, IMO. >>>> >>>>> The AUTHZ mechanism MUST NOT be internal to the BES : >>>> >>>> Maybe I was not clear. The authz mechanism is invisible for outside >>>> parties (like clients). It can be an external component, an internal >>>> component, whatever, it is up to the BES implementor and the site admin. >>>> >>>>> For example, >>>>> in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package >>>>> is described as 'The security subsystem of UNICORE/X', and is NOT >>>>> internal to 'de.fzj.unicore.uas.impl.job'. >>>> >>>> In UNICORE site admins can choose what attribute sources and XACML >>>> decision points they want to use, but the clients (including other >>>> services) do not need to know this. That is what I meant by "internal". >>>> >>>>>> >>>>>> 6.5 >>>>>> >>>>>> These are reqiurements on the security layer (or framework) and should >>>>>> not be used as requirements on BES. >>>>> These security requirements DO have impacts on the BES Client >>>>> interface and on the Job Description document. >>>>> In the text, I have made it clear. >>>> >>>> indeed while preparing the EMI-ES specification, we came to the >>>> following conclusions >>>> 1) when using proxies for delegation, it is necessary to map each data >>>> staging item to a delegated credential (you can check the EMI-ES job >>>> description for details) >>>> 2) the delegation operations are separate from the job management >>>> operations, so they do not necessarily have to be part of the BES client >>>> interface. >>>> >>>> Also, there are existing implementations (UNICORE and Genesis come to my >>>> mind) that do not need this at all, because they do delegation without >>>> proxies. >>>> >>>> So I disagree, 6.5 mostly describes features of the particular security >>>> framework that is used. >>>> >>>>>> >>>>>> 8 BES requirements related to "Application Repositories" >>>>>> >>>>>> While I agree that BES should understand the notion of an "Application" >>>>>> (see e.g. JSDL ApplicationName), I don't agree that the BES should >>>>>> use these for Scheduling. Rather, this is the job of a broker. >>>>> The text is now : >>>>> * Resource selection : The Execution Service MUST use, among others, >>>>> these references to 'Installed Applications' in order to select the most >>>>> adequate computing resource for the Job. >>>>>> >>>>>> 9 BES requirements applying to Accounting >>>>>> >>>>>> As a "MUST", these are out of scope, and should be made "SHOULD". >>>>> No Accounting --> No precise reporting to funding agencies --> No >>>>> funding --> Abrupt and very long shutdown of all services. >>>>> I fully confirm that Accounting is a MUST. >>>>> >>>> >>>> ... operational >>>> >>>>>> >>>>>> 10 Logging/Bookkeeping >>>>>> >>>>>> Same as 9. >>>>> Same as 'Traceability' : >>>>> This is an operational concern : Would you really take the risk that >>>>> the whole EGI becomes a spambot or a scambot ? >>>>> No Logging and Bookkeeping --> No post mortem analysis after attack >>>>> --> Large infection --> Panic --> Abrupt and very long shutdown of all >>>>> services. >>>>> I fully confirm that Logging and Bookkeeping is a MUST. >>>>> >>>> >>>> an operational MUST maybe for some infrastructures, not all. >>>> >>>> E.g. a typical HPC site has its own accounting, its own logging systems >>>> independent of the (Grid) software used to submit jobs to it. >>>> >>>>>> >>>>>> 12 Jobs >>>>>> >>>>>> 12.1 Types of job >>>>>> >>>>>> Support for parallel jobs: it should be "MUST" :-) >>>>> The text is now : >>>>> - The concept of 'Single Job' includes Jobs running >>>>> massively-parallel processes using MPI on one large-scale HPC System. >>>>> The Execution Service MUST understand instructions for usage of MPI >>>>> inside the Job Description document. The Execution Service SHOULD >>>>> transmit these instructions to the Batch System, or return an explicit >>>>> error message if not supported. >>>> >>>> OK >>>> >>>> >>>> >>>> Best regards, >>>> Bernd. >>>> >>>> >>>> -- >>>> Dr. Bernd Schuller >>>> Federated Systems and Data >>>> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >>>> Phone: +49 246161-8736 (fax -8556) >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------------------------ >>>> ------------------------------------------------------------------------------------------------ >>>> Forschungszentrum Juelich GmbH >>>> 52425 Juelich >>>> Sitz der Gesellschaft: Juelich >>>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >>>> Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher >>>> Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), >>>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >>>> Prof. Dr. Sebastian M. Schmidt >>>> ------------------------------------------------------------------------------------------------ >>>> ------------------------------------------------------------------------------------------------ >>> >> >> -- >> Dr. Bernd Schuller >> Federated Systems and Data >> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >> Phone: +49 246161-8736 (fax -8556) >> >> >> >> > > > > -- > Nothing is ever easy... > -- > ogsa-bes-wg mailing list > ogsa-bes-wg at ogf.org > http://www.ogf.org/mailman/listinfo/ogsa-bes-wg From mamonski at man.poznan.pl Tue Sep 20 01:22:58 2011 From: mamonski at man.poznan.pl (=?UTF-8?Q?Mariusz_Mamo=C5=84ski?=) Date: Tue, 20 Sep 2011 08:22:58 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <7BC2FE4B-CC94-42BA-88E2-15C78AD45A2E@mcs.anl.gov> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> <1316119475.24230.60.camel@zam994> <4E7333DE.5040809@lal.in2p3.fr> <1316179590.25767.49.camel@zam994> <7BC2FE4B-CC94-42BA-88E2-15C78AD45A2E@mcs.anl.gov> Message-ID: Dear all, I finally manage to go thru this document. I agree with Bernhard that the BES spec should not request anything about the internal service architecture, in particularly requesting that the BES service operation be dependent on a set of other services, as stated for "performance reasons" is, in my opinion, a false assumption. External callouts to other services may actually decrease service performance. In our BES implementation (see: http://www.ogf.org/documents/GFD.179.pdf) we decided to handle authentication/authorization and staging on the dynamically loadable modules basics. The module implementation MAY do some call to the external service but in most cases the work would done locally (for performance and maintenance reasons) e.g. reading plain gird map file. Other things that i dislike: 1. "A ?Monitoring Framework? permitting local and grid operational staff to monitor running services and to be quickly alerted of incidents with sound information permitting quick incident management. So, the Execution Service MUST publish as much as it can to the ?Monitoring Framework? according to the Backend Interface provided by the ?Monitoring Framework?." Does it imply the PUSH model. In my opinion monitoring is usually done by PULLING (periodic tests). How the service would have a chance to report by itself that it was killed with SIGKILL? ;-) 2. "Delegation of user authorizations (SAML or X509 attributes) to the Batch System really executing the Job." - not fully understand what does it mean. Majority of the DRMS does not know anything about the SAML or X509 certificates. Things that i like: 1. "Backward compatibility : BES specifications SHOULD be an evolution of previous specifications : The scope and orientation of a spec should not change dramatically to the previous one, for example conceptual backward compatibility is a must. Unstable OGF standards could cause bad community reputation. Stable OGF standards ease OGF marketing efforts (Wiki NF10, Id 152)." A set of authentication and data transfer protocol must be required, otherwise we will end up with not interoperable implementations. But this can be done via set of profiles (e.g. X.509 proxy + gridftp profile) minor typo (page 3): "?Advanced Reservation?" -> "?Advance Reservation? Cheers, On 19 September 2011 17:57, Stuart Martin wrote: > I just wanted to point out that RFC-3820 compliant X.509 proxies are supported in OpenSSL. Several project (e.g., LIGO) use OpenSSL with proxy certs with the Apache server. > > Also, jGlobus 2 has support for RFC-3820 compliant X.509 proxies. ?Included is an implementation of a java SSL TrustManager that has proxy processing. > > -Stu > > On Sep 16, 2011, at Sep 16, 8:36 AM, Andre Merzky wrote: > >> +1 on all points. >> >> Cheers, Andre. >> >> >> 2011/9/16 Bernd Schuller : >>> hi Etienne, >>> >>> On Fri, 2011-09-16 at 13:32 +0200, Etienne URBAH wrote: >>> [...] >>>> >>>> RFC-3820-compliant X509 Proxies >>>> ------------------------------- >>>> The RFC-3820-compliant X509 proxies are fully supported by the jLite >>>> library written in Java by Oleg SUKHOROSLOV and available at >>>> http://code.google.com/p/jlite/ >>>> >>> >>> You must be joking. I was talking about open source software like Apache >>> httpd and Java JDK. jLite just wraps the cog-globus libraries and adds >>> some gLite access APIs. No, thanks. >>> >>>> >>>> Dependency of BES on other grid software components / operational issues >>>> ------------------------------------------------------------------------ >>>> It is very good that we know agree on GLUE 2.0 as base for the >>>> Information System. ?Otherwise, we could NOT agree on the way to express >>>> references to grid entities in the Job Description document. >>>> >>>> Your comments about chapter 6.5 confirm that the specifications of the >>>> BES Client interface DEPENDS on whether BES supports X509 proxies for >>>> delegation of Security credentials or NOT. >>>> >>> >>> At least this was the EMI-ES v1.0 conclusion, which need not be the >>> final word. The only dependency (in EMI-ES) is the specification which >>> delegated credential is to be used for which data staging item. >>> >>> Delegation can be performed FULLY TRANSPARENT to the BES (for example on >>> a message level as in UNICORE), and the BES interface specification is >>> not dependent on it at all. >>> >>>> Since delegation of Security credentials is a MUST for BES, my >>>> conclusion is that we MUST agree on SECURITY issues (even if some are >>>> operational issues) BEFORE trying to write down BES requirements. >>>> >>>> In the same way, we know that Clients need to perform complex queries on >>>> Jobs. The BES Client interface DEPENDS on whether such queries are >>>> accepted by BES itself, of by a separate Logging and Bookkeeping >>>> service. ?So, I think that we have to agree on the existence or absence >>>> of a separate Logging and Bookkeeping service BEFORE trying to write >>>> down BES requirements about queries. >>> >>> >>> The BES client interface does not necessarily need to allow to perform >>> complex queries, these should be part of a separate interface. >>> >>> >>>> As a summary : >>>> - ?Some BES requirements are quite independent from other grid >>>> components, and we can discuss on them immediately. >>>> - ?But some other BES requirements are DEPENDENT from foundational grid >>>> components or operational issues, in particular Information System, >>>> Security, Logging and Bookkeeping, ... >>>> - ?Therefore, we have to agree on these other grid components or >>>> operational issues FIRST. >>>> >>>> This is a critical issue, and I propose that we discuss on it at OGF33. >>> >>> Unfortunately I won't be in Lyon, only via phone. >>> >>> Summarising, my main points for the BES interface specification : >>> >>> ?* specifications must be narrowly scoped and composable (which is the >>> JSDL/BES model anyway). >>> >>> ?* do not try to specify specific authentication methods. Do not try to >>> specify specific delegation methods. Security is a cross cutting concern >>> and should be dealt with separately. >>> >>> ?* do not assume a special environment where a BES instance will run. >>> Interactions with other services (except for data access) are optional >>> and should be treated as such. >>> >>> ?* leave operational aspects to operations. Recommendations for BES >>> implementors may be given, of course. >>> >>> >>> Best regards, >>> Bernd. >>> >>>> >>>> I will answer to your other comments later. >>>> >>>> Best regards. >>>> >>>> ----------------------------------------------------- >>>> Etienne URBAH ? ? ? ? LAL, Univ Paris-Sud, IN2P3/CNRS >>>> ? ? ? ? ? ? ? ? ? ? ? ?Bat 200 ? 91898 ORSAY ? ?France >>>> Tel: +33 1 64 46 84 87 ? ? ?Skype: etienne.urbah >>>> Mob: +33 6 22 30 53 27 ? ? ?mailto:urbah at lal.in2p3.fr >>>> ----------------------------------------------------- >>>> >>>> >>>> On Thu, 15/09/2011 22:44, Bernd Schuller wrote: >>>>> Hi Etienne, >>>>> >>>>> thanks for the clarifications. So indeed your document is aimed at both: >>>>> >>>>> 1) providing requirements for the actual BES specification ("client >>>>> interface" in your terminology) >>>>> 2) the operation and deployment issues that have to be solved for >>>>> interoperability on an infrastructure level (say EGI and EDGI). >>>>> >>>>> It would be very beneficial for further progress if these two distinct >>>>> concerns could be separated, at least CLEARLY marked in your document. >>>>> >>>>> I have added some more comments inline. >>>>> >>>>> On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: >>>>> >>>>>> Concerning the document named 'Requirements for an improved Basic >>>>>> Execution Service (BES)' and available at >>>>>> http://forge.gridforum.org/sf/go/doc16306 : >>>>>> >>>>>> THANK YOU VERY MUCH for having taken the time to read this document, and >>>>>> for having taken the time to provide comments : >>>>>> >>>>>> Such comments are very useful for the improvement of documents, permit >>>>>> convergence and prepare later agreement. >>>>>> [...] >>>>>> On Thu, 15/09/2011 13:00, Bernd Schuller wrote: >>>>>>> >>>>>>> [...] >>>>>> >>>>>>> 1.4 Methodology >>>>>>> >>>>>>> ? ?* ref 4: glite user guide -> ? out of scope, gLite is too complex and >>>>>>> too specific in its architecture (if there is such a thing) to be useful >>>>>>> as a base for BES >>>>>> ? ? Yes, this is a very large user guide for specific usage of gLite by >>>>>> users, and it does NOT provide a clear description of the architecture >>>>>> and of the functionalities. >>>>>> ? ? But it is NOT so complex. ?As soon as I finished reading this guide, >>>>>> it was easy for me to perform reverse engineering and extract the >>>>>> effective architecture (SOA with internal interfaces) and the implied >>>>>> functionalities (which are to be improved). >>>>> >>>>> Indeed it appears that you try to impose gLite specifics (like a logging >>>>> & ?bookkeeping service or proxy certificates on the transport level) as >>>>> requirements. This would severly limit the BES specification effort, and >>>>> will not be accepted (I hope) by other stakeholders. >>>>> >>>>>> ? ? In the text, I have prepended a few words to explain that. >>>>> >>>>> Basically you imply that the "architecture and functionalities" of the >>>>> gLite execution system (together with the PGI work) is somehow the >>>>> guideline to be followed, which I fully disagree with. >>>>> >>>>>>> >>>>>>> 2.4 Collaboration with other services >>>>>>> >>>>>>> While this is important for interoperability, it is unimportant >>>>>>> for the specification of a BES. The BES spec should NOT try to specify >>>>>>> all the interaction with the rest of the world. This is the task of a >>>>>>> "grid architecture specification" like OGSA. >>>>>> ? ? My document is NOT targeted only to the specification of the BES >>>>>> Client interface, but to the clear and consistent description of BES >>>>>> context and functional + operational requirements which are really >>>>>> necessary for interoperability. >>>>>> ? ? As far as I know, OGSA does NOT take into account GLUE 2.0 yet. >>>>>> Therefore, an up to date 'grid architecture specification' is absolutely >>>>>> necessary. >>>>> >>>>> Glue2 is just an information model, not necessarily a perfect one nor >>>>> the only one. However, I agree an information model has to be adopted >>>>> for BES and any associated information systems. >>>>> >>>>>> ? ? If OGSA members consider chapters 2.3 and 2.4 of my document as a >>>>>> 'grid architecture specification' which updates and improves OGSA, I >>>>>> thank them. ?If they consider that this 'grid architecture >>>>>> specification' does NOT comply with OGSA and competes with it, then I >>>>>> assert that it obsoletes OGSA. >>>>> >>>>> I can't really say, the OGSA group stopped its work a long time ago and >>>>> it's a long time that I looked at the documents. >>>>> >>>>>>> >>>>>>> Specifically, the interactions with security, monitoring, accounting and >>>>>>> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory >>>>>>> part of a BES specification. >>>>>> ? ? FAILURE of practical operations is often caused by LACK of early care >>>>>> about operational concerns during specification phase. >>>>> >>>>> Agreed. >>>>> >>>>>> ? As GIN-GC has >>>>>> proven and documented, this is even more true for interoperability on >>>>>> the field (as opposed to theoretical interoperability at the WSDL level). >>>>>> ? ? I confirm that care about operational concerns is REQUIRED for real >>>>>> operations and for practical interoperability on the field. ?Although >>>>>> operational concerns are NOT part of the BES Client interface, they are >>>>>> REQUIRED for the overall specifications of BES in its context. >>>>>> ? ? In the text, I have stressed that the document DOES take into account >>>>>> operational concerns. >>>>>> >>>>>>> >>>>>>> 4. BES non-functional requirements >>>>>>> >>>>>>> 4.1.2 Traceability - should be SHOULD not MUST >>>>>> ? ? This is an operational concern : ?Would you really take the risk that >>>>>> the whole EGI becomes a spambot or a scambot ? >>>>> >>>>> Isn't it already, powered by gLite and used by the wlcg botnet (just >>>>> kidding of course) :-) >>>>> >>>>>> ? ? No traceability --> ?No post mortem analysis after attack --> ?Large >>>>>> infection --> ?Panic --> ?Abrupt and very long shutdown of all services. >>>>>> ? ? I fully confirm that traceability is a MUST. >>>>> >>>>> It is an internal detail which any good implementation will provide. >>>>> If BES-A is much easier and more secure to operate than BES-B, admins >>>>> can choose which to install. >>>>> >>>>>>> >>>>>>> 4.1.3 Security >>>>>>> ? ?- how can a specification "implement" a policy? You probably >>>>>>> ? ? ?meant "BES implementations SHOULD ..." >>>>>> ? ? The text now is 'BES specifications MUST fully take into account the >>>>>> Security Policies ...' >>>>> >>>>> Still no understanding here... let's take traceability >>>>> The relevant EGI policy >>>>> ?says >>>>> >>>>> "[...] software deployed in the Grid MUST include the >>>>> ability to produce sufficient and relevant logging [...] >>>>> The level of the logging MUST be configured by all service providers, >>>>> including but not limited to the Sites, to produce the required >>>>> information which MUST be retained for a minimum of 90 days." >>>>> >>>>> For example all UNICORE services can be made to comply with this >>>>> by configuration of the logging library we use (Apache Log4j), and by >>>>> not deleting log files for 90 days. >>>>> So this is a feature of the implementation and the administrator in >>>>> charge, not the specification. Thus, your sentence should read "BES >>>>> implementations SHOULD ..." (It is MUST of course only if they want to >>>>> be deployed in EGI) >>>>> One does not try to specify implementation details, at least not in any >>>>> specification I've ever seen (e.g. does the HTTP specification say >>>>> anything about server logging or accepted CAs?). >>>>> >>>>> >>>>> >>>>>> [...] >>>>>>> >>>>>>> 4.2 >>>>>>> ? ?all of this is out of scope. For example the UNICORE service >>>>>>> container hosts a number of services including an execution service. >>>>>>> Probably you mean that the execution service SPECIFICATION should be >>>>>>> limited to the execution service and MUST NOT specify accounting, >>>>>>> security etc. >>>>>> ? ? 'Well defined and narrow scope' is a general engineering requirement. >>>>>> ? ?It is fundamental concern crossing requirements, design, >>>>>> specifications, fabrication, tests, operations, user experience and >>>>>> product maintenance for all types of products, even outside software >>>>>> engineering. >>>>> >>>>> exactly. >>>>> >>>>>> ? ? I confirm that 'Well defined and narrow scope' is absolutely REQUIRED >>>>>> for sound software design, implementation, deployment and maintainability. >>>>>> ? ? From your comment, I assume that the Execution Service of UNICORE >>>>>> does have a 'Well defined and narrow scope', does have precise >>>>>> interfaces with other UNICORE services, and minimizes overlaps with them. >>>>> >>>>> of course. And UNICORE does not include a L&B service :-) >>>>> >>>>> >>>>>> [...] >>>>>>> 6.1 >>>>>>> >>>>>>> ? ?* "SSL certificates MUST be signed by a CA..." this is an operational >>>>>>> decision, and has nothing to do with the BES spec. >>>>>>> For example, a site may run an inhouse deployment of BES using an >>>>>>> in-house CA. This requirement should be deleted. >>>>>> ? ? This operational concern is REQUIRED for practical interoperability >>>>>> on the field. ?I have prepended : >>>>>> ? ? * Authentication of Servers : ?The Execution Service SHOULD permit >>>>>> Clients to authenticate it. ?If an Execution Service authenticates >>>>>> itself to clients, it MUST permit Clients to really perform this >>>>>> authentication. >>>>> >>>>> This sentence makes no sense to me, sorry. >>>>> Maybe "Server and client SHOULD communicate via a secure channel >>>>> (SSL/TLS)". Even this may not be true in the future, though it is for >>>>> all(?) the Grid systems currently. >>>>> >>>>>>> >>>>>>> 6.3 >>>>>>> >>>>>>> * "For Client authentication, the Execution Service MUST accept all >>>>>>> following authentication methods: ?Full X509, ?RFC-3820-compliant X509 >>>>>>> Proxy" >>>>>>> >>>>>>> This requirement is invalid. I agree that it would be nice to be able to >>>>>>> specify authentication methods, but it is impossible. For example >>>>>>> Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface >>>>>>> over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be >>>>>>> valid authentication methods in some circumstances. >>>>>> ? ? There are 2 separate requirements : >>>>>> ? ? - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy >>>>>> ? ? - 1 'MAY' ?for all other ones. >>>>>>> >>>>>>> Furthermore, making proxies a MUST implies that nonstandard >>>>>>> authentication libraries instead of TLS/SSL must be used, making the BES >>>>>>> implementation insecure. For some implementors (like UNICORE) proxies on >>>>>>> the transport level are very much a no-go. >>>>>> ? ? I had clearly specified RFC-3820-compliant X509 Proxy, which ARE >>>>>> standard. >>>>>> ? ? Your critics are valid for GSI proxies, which I have taken care NOT >>>>>> to mention. >>>>> >>>>> by "standard" I did not mean that it is an RFC, but software support. >>>>> As opposed to standard SSL/TLS, proxies are almost not supported by >>>>> industry standard tools, for example Apache httpd or the Java JDK. >>>>> One has to rely on custom code, which is notoriously buggy and error >>>>> prone. >>>>> >>>>> Since one important non-functional requirement (for me at least) is to >>>>> be able use standard (off-the-shelf) open source software, having to >>>>> support proxies is a big limitation. >>>>> >>>>>>> 6.4. >>>>>>> >>>>>>> "This authorization mechanism MUST be consistent across all instances of >>>>>>> the Execution Service" >>>>>>> >>>>>>> This violates the autonomy of a site. Site administrators often wish to >>>>>>> stay in control of their resources, and do not accept external >>>>>>> authorisation decision points. And anyway, who cares? Since the AuthZ >>>>>>> mechanism is internal to the BES, it cannot be specified in the >>>>>>> BES spec as such. >>>>>> ? ? Interoperability requires a federation of independent administrative >>>>>> domain to agree on common functionalities, interfaces and operations. >>>>>> ? ? This DOES sometime violate the autonomy of each individual site. >>>>>> ? ? The requirement is NOT that the AUTHZ decision point is external to >>>>>> any site, but that all participating site MUST accept to install inside >>>>>> their site an instance of a commonly validated software implementing the >>>>>> decision point. >>>>> >>>>> No. Each site may choose their own authz decision point, IMO. >>>>> >>>>>> ? ? The AUTHZ mechanism MUST NOT be internal to the BES : >>>>> >>>>> Maybe I was not clear. The authz mechanism is invisible for outside >>>>> parties (like clients). It can be an external component, an internal >>>>> component, whatever, it is up to the BES implementor and the site admin. >>>>> >>>>>> For example, >>>>>> in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package >>>>>> is described as 'The security subsystem of UNICORE/X', and is NOT >>>>>> internal to 'de.fzj.unicore.uas.impl.job'. >>>>> >>>>> In UNICORE site admins can choose what attribute sources and XACML >>>>> decision points they want to use, but the clients (including other >>>>> services) do not need to know this. That is what I meant by "internal". >>>>> >>>>>>> >>>>>>> 6.5 >>>>>>> >>>>>>> These are reqiurements on the security layer (or framework) and should >>>>>>> not be used as requirements on BES. >>>>>> ? ? These security requirements DO have impacts on the BES Client >>>>>> interface and on the Job Description document. >>>>>> ? ? In the text, I have made it clear. >>>>> >>>>> indeed while preparing the EMI-ES specification, we came to the >>>>> following conclusions >>>>> 1) when using proxies for delegation, it is necessary to map each data >>>>> staging item to a delegated credential (you can check the EMI-ES job >>>>> description for details) >>>>> 2) the delegation operations are separate from the job management >>>>> operations, so they do not necessarily have to be part of the BES client >>>>> interface. >>>>> >>>>> Also, there are existing implementations (UNICORE and Genesis come to my >>>>> mind) that do not need this at all, because they do delegation without >>>>> proxies. >>>>> >>>>> So I disagree, 6.5 mostly describes features of the particular security >>>>> framework that is used. >>>>> >>>>>>> >>>>>>> 8 BES requirements related to "Application Repositories" >>>>>>> >>>>>>> While I agree that BES should understand the notion of an "Application" >>>>>>> (see e.g. JSDL ApplicationName), I don't agree that the BES should >>>>>>> use these for Scheduling. Rather, this is the job of a broker. >>>>>> ? ? The text is now : >>>>>> ? ? * Resource selection : ?The Execution Service MUST use, among others, >>>>>> these references to 'Installed Applications' in order to select the most >>>>>> adequate computing resource for the Job. >>>>>>> >>>>>>> 9 BES requirements applying to Accounting >>>>>>> >>>>>>> As a "MUST", these are out of scope, and should be made "SHOULD". >>>>>> ? ? No Accounting --> ?No precise reporting to funding agencies --> ?No >>>>>> funding --> ?Abrupt and very long shutdown of all services. >>>>>> ? ? I fully confirm that Accounting is a MUST. >>>>>> >>>>> >>>>> ... operational >>>>> >>>>>>> >>>>>>> 10 Logging/Bookkeeping >>>>>>> >>>>>>> Same as 9. >>>>>> ? ? Same as 'Traceability' : >>>>>> ? ? This is an operational concern : ?Would you really take the risk that >>>>>> the whole EGI becomes a spambot or a scambot ? >>>>>> ? ? No Logging and Bookkeeping --> ?No post mortem analysis after attack >>>>>> --> ?Large infection --> ?Panic --> ?Abrupt and very long shutdown of all >>>>>> services. >>>>>> ? ? I fully confirm that Logging and Bookkeeping is a MUST. >>>>>> >>>>> >>>>> an operational MUST maybe for some infrastructures, not all. >>>>> >>>>> E.g. a typical HPC site has its own accounting, its own logging systems >>>>> independent of the (Grid) software used to submit jobs to it. >>>>> >>>>>>> >>>>>>> 12 Jobs >>>>>>> >>>>>>> 12.1 Types of job >>>>>>> >>>>>>> Support for parallel jobs: it should be "MUST" :-) >>>>>> ? ? The text is now : >>>>>> ? ? - The concept of 'Single Job' includes Jobs running >>>>>> massively-parallel processes using MPI on one large-scale HPC System. >>>>>> The Execution Service MUST understand instructions for usage of MPI >>>>>> inside the Job Description document. ?The Execution Service SHOULD >>>>>> transmit these instructions to the Batch System, or return an explicit >>>>>> error message if not supported. >>>>> >>>>> OK >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> Bernd. >>>>> >>>>> >>>>> -- >>>>> Dr. Bernd Schuller >>>>> Federated Systems and Data >>>>> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >>>>> Phone: +49 246161-8736 (fax -8556) >>>>> >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------------------------ >>>>> ------------------------------------------------------------------------------------------------ >>>>> Forschungszentrum Juelich GmbH >>>>> 52425 Juelich >>>>> Sitz der Gesellschaft: Juelich >>>>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >>>>> Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher >>>>> Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), >>>>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >>>>> Prof. Dr. Sebastian M. Schmidt >>>>> ------------------------------------------------------------------------------------------------ >>>>> ------------------------------------------------------------------------------------------------ >>>> >>> >>> -- >>> Dr. Bernd Schuller >>> Federated Systems and Data >>> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >>> Phone: +49 246161-8736 (fax -8556) >>> >>> >>> >>> >> >> >> >> -- >> Nothing is ever easy... >> -- >> ?ogsa-bes-wg mailing list >> ?ogsa-bes-wg at ogf.org >> ?http://www.ogf.org/mailman/listinfo/ogsa-bes-wg > > -- > ?ogsa-bes-wg mailing list > ?ogsa-bes-wg at ogf.org > ?http://www.ogf.org/mailman/listinfo/ogsa-bes-wg > -- Mariusz From m.riedel at fz-juelich.de Tue Sep 20 02:40:11 2011 From: m.riedel at fz-juelich.de (Morris Riedel) Date: Tue, 20 Sep 2011 09:40:11 +0200 Subject: [OGSA-BES-WG] REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in Message-ID: <000801cc7768$87e38200$97aa8600$@riedel@fz-juelich.de> All, after re-scheduling, we have the first of our sessions today [1]: Tuesday, September 20 2:00 pm - 3:30 pm PGI-OGSA-BES-JSDL Joint Session (90 mins) Andrew Grimshaw, Morris Riedel Location: Rhone 4 A high participation is appreciated especially since several of us have several parallel commitments in other sessions. Important will be three things (from my perspective): (a) Focus: baby steps together - concept by concept (e.g. manual data-staging, pre-post processing, scalable-time, etc.), we should not try to standardize all requirements at once, there could be BES 1.1, BES1.2, BES1.3, etc. (b) Working sessions: Skip introductions and general architecture discussions; focus on specification work concept by concept (XSD level?!) (c) Next steps: experience tells us that we will not manage to achieve very much in the time available at the OGF event, also some progress will be made - but how we organize the next months: milestones, weekly phone calls, chairs?!, specification work with deadlines, etc. Of course, I'm open to suggestions. Take care, Morris [1] http://www.ogf.org/gf/event_schedule/index.php?id=2392 ------------------------------------------------------------ Morris Riedel Deputy Division Leader Federated Systems and Data (FSD) J?lich Supercomputing Centre (JSC) Info: http://www.fz-juelich.de/SharedDocs/Personen/IAS/JSC/EN/staff/riedel_m.html -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3550 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110920/a5fa47b4/attachment.bin From urbah at lal.in2p3.fr Tue Sep 20 04:46:12 2011 From: urbah at lal.in2p3.fr (Etienne URBAH) Date: Tue, 20 Sep 2011 11:46:12 +0200 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> <1316119475.24230.60.camel@zam994> <4E7333DE.5040809@lal.in2p3.fr> <1316179590.25767.49.camel@zam994> <7BC2FE4B-CC94-42BA-88E2-15C78AD45A2E@mcs.anl.gov> Message-ID: <4E7860E4.1060207@lal.in2p3.fr> Mariusz, Concerning the document named 'Requirements for an improved Basic Execution Service (BES)' and available at http://forge.gridforum.org/sf/go/doc16306 : Thank you very much for your comments. Dependency of BES on other grid services ---------------------------------------- I do NOT define a service as a separate 'running process', but as a separate software component, with well defined scope and interfaces. I do NOT stress dependency of BES on other grid services for 'performance reasons' : On execution, it is always faster to have 1 monolithic piece of code. I stress this dependency because : - In many grid infrastructures, these other grid services already exist. - Modularity and definition of backend interfaces eases software maintenance and interoperation with other grid infrastructure using these backend interfaces (for example UR for accounting records). Implementation of AUTHN and AUTHZ as dynamically loadable modules is absolutely OK. Monitoring ---------- Often, monitoring is implemented using simultaneously 2 completely different methods : - Sensors / Probes installed inside the Service and PUSHING detailed information to a 'Grid Monitoring Framework', for example Ganglia or Nagios. The Service is often considered 'killed' as soon as the 'Grid Monitoring Framework' has NOT received any information from Sensors / Probes after 2 or 3 sampling periods. - Periodic tests sent by a dedicated 'Test Client' to the Service through its Client interface in order to PULL information about the overall availability of the Service for Clients. For BES, these tests would be 'test jobs'. The Service is often considered 'killed' as soon as the 'Test Client' can NOT even establish a successful TCP connection to the Service. I highly recommend the simultaneous usage of BOTH methods. Delegation of user authorizations --------------------------------- As you have written, delegated user credentials are mostly NOT understood and used by the Batch System itself. But they are often required by the 'Job wrapper' or the Payload itself to access remote files referenced by the Job. Best regards. ----------------------------------------------------- Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS Bat 200 91898 ORSAY France Tel: +33 1 64 46 84 87 Skype: etienne.urbah Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr ----------------------------------------------------- On Tue, 20/09/2011 08:22, Mariusz Mamo?ski wrote: > Dear all, > > I finally manage to go thru this document. I agree with Bernhard that > the BES spec should not request anything about the internal service > architecture, in particularly requesting that the BES service > operation be dependent on a set of other services, as stated for > "performance reasons" is, in my opinion, a false assumption. External > callouts to other services may actually decrease service performance. > In our BES implementation (see: > http://www.ogf.org/documents/GFD.179.pdf) we decided to handle > authentication/authorization and staging on the dynamically loadable > modules basics. The module implementation MAY do some call to the > external service but in most cases the work would done locally (for > performance and maintenance reasons) e.g. reading plain gird map file. > > Other things that i dislike: > > 1. "A ???Monitoring Framework??? permitting local and grid operational > staff to monitor running services and to be quickly alerted of > incidents with sound information permitting quick incident management. > So, the Execution Service MUST publish as much as it can to the > ???Monitoring Framework??? according to the Backend Interface provided by > the ???Monitoring Framework???." > > Does it imply the PUSH model. In my opinion monitoring is usually done > by PULLING (periodic tests). How the service would have a chance to > report by itself that it was killed with SIGKILL? ;-) > > 2. "Delegation of user authorizations (SAML or X509 attributes) to the > Batch System really executing the Job." - not fully understand what > does it mean. Majority of the DRMS does not know anything about the > SAML or X509 certificates. > > Things that i like: > > 1. "Backward compatibility : BES specifications SHOULD be an evolution > of previous specifications : The scope and orientation of a spec > should not change dramatically to the previous one, for example > conceptual backward compatibility is a must. Unstable OGF standards > could cause bad community reputation. Stable OGF standards ease OGF > marketing efforts (Wiki NF10, Id 152)." > > > A set of authentication and data transfer protocol must be required, > otherwise we will end up with not interoperable implementations. But > this can be done via set of profiles (e.g. X.509 proxy + gridftp > profile) > > > minor typo (page 3): "???Advanced Reservation???" -> "???Advance Reservation??? > > > > Cheers, > > On 19 September 2011 17:57, Stuart Martin wrote: >> I just wanted to point out that RFC-3820 compliant X.509 proxies are supported in OpenSSL. Several project (e.g., LIGO) use OpenSSL with proxy certs with the Apache server. >> >> Also, jGlobus 2 has support for RFC-3820 compliant X.509 proxies. ? Included is an implementation of a java SSL TrustManager that has proxy processing. >> >> -Stu >> >> On Sep 16, 2011, at Sep 16, 8:36 AM, Andre Merzky wrote: >> >>> +1 on all points. >>> >>> Cheers, Andre. >>> >>> >>> 2011/9/16 Bernd Schuller: >>>> hi Etienne, >>>> >>>> On Fri, 2011-09-16 at 13:32 +0200, Etienne URBAH wrote: >>>> [...] >>>>> >>>>> RFC-3820-compliant X509 Proxies >>>>> ------------------------------- >>>>> The RFC-3820-compliant X509 proxies are fully supported by the jLite >>>>> library written in Java by Oleg SUKHOROSLOV and available at >>>>> http://code.google.com/p/jlite/ >>>>> >>>> >>>> You must be joking. I was talking about open source software like Apache >>>> httpd and Java JDK. jLite just wraps the cog-globus libraries and adds >>>> some gLite access APIs. No, thanks. >>>> >>>>> >>>>> Dependency of BES on other grid software components / operational issues >>>>> ------------------------------------------------------------------------ >>>>> It is very good that we know agree on GLUE 2.0 as base for the >>>>> Information System. ? Otherwise, we could NOT agree on the way to express >>>>> references to grid entities in the Job Description document. >>>>> >>>>> Your comments about chapter 6.5 confirm that the specifications of the >>>>> BES Client interface DEPENDS on whether BES supports X509 proxies for >>>>> delegation of Security credentials or NOT. >>>>> >>>> >>>> At least this was the EMI-ES v1.0 conclusion, which need not be the >>>> final word. The only dependency (in EMI-ES) is the specification which >>>> delegated credential is to be used for which data staging item. >>>> >>>> Delegation can be performed FULLY TRANSPARENT to the BES (for example on >>>> a message level as in UNICORE), and the BES interface specification is >>>> not dependent on it at all. >>>> >>>>> Since delegation of Security credentials is a MUST for BES, my >>>>> conclusion is that we MUST agree on SECURITY issues (even if some are >>>>> operational issues) BEFORE trying to write down BES requirements. >>>>> >>>>> In the same way, we know that Clients need to perform complex queries on >>>>> Jobs. The BES Client interface DEPENDS on whether such queries are >>>>> accepted by BES itself, of by a separate Logging and Bookkeeping >>>>> service. ? So, I think that we have to agree on the existence or absence >>>>> of a separate Logging and Bookkeeping service BEFORE trying to write >>>>> down BES requirements about queries. >>>> >>>> >>>> The BES client interface does not necessarily need to allow to perform >>>> complex queries, these should be part of a separate interface. >>>> >>>> >>>>> As a summary : >>>>> - ? Some BES requirements are quite independent from other grid >>>>> components, and we can discuss on them immediately. >>>>> - ? But some other BES requirements are DEPENDENT from foundational grid >>>>> components or operational issues, in particular Information System, >>>>> Security, Logging and Bookkeeping, ... >>>>> - ? Therefore, we have to agree on these other grid components or >>>>> operational issues FIRST. >>>>> >>>>> This is a critical issue, and I propose that we discuss on it at OGF33. >>>> >>>> Unfortunately I won't be in Lyon, only via phone. >>>> >>>> Summarising, my main points for the BES interface specification : >>>> >>>> ? * specifications must be narrowly scoped and composable (which is the >>>> JSDL/BES model anyway). >>>> >>>> ? * do not try to specify specific authentication methods. Do not try to >>>> specify specific delegation methods. Security is a cross cutting concern >>>> and should be dealt with separately. >>>> >>>> ? * do not assume a special environment where a BES instance will run. >>>> Interactions with other services (except for data access) are optional >>>> and should be treated as such. >>>> >>>> ? * leave operational aspects to operations. Recommendations for BES >>>> implementors may be given, of course. >>>> >>>> >>>> Best regards, >>>> Bernd. >>>> >>>>> >>>>> I will answer to your other comments later. >>>>> >>>>> Best regards. >>>>> >>>>> ----------------------------------------------------- >>>>> Etienne URBAH ? ? ? ? LAL, Univ Paris-Sud, IN2P3/CNRS >>>>> ? ? ? ? ? ? ? ? ? ? ? ? Bat 200 ? 91898 ORSAY ? ? France >>>>> Tel: +33 1 64 46 84 87 ? ? ? Skype: etienne.urbah >>>>> Mob: +33 6 22 30 53 27 ? ? ? mailto:urbah at lal.in2p3.fr >>>>> ----------------------------------------------------- >>>>> >>>>> >>>>> On Thu, 15/09/2011 22:44, Bernd Schuller wrote: >>>>>> Hi Etienne, >>>>>> >>>>>> thanks for the clarifications. So indeed your document is aimed at both: >>>>>> >>>>>> 1) providing requirements for the actual BES specification ("client >>>>>> interface" in your terminology) >>>>>> 2) the operation and deployment issues that have to be solved for >>>>>> interoperability on an infrastructure level (say EGI and EDGI). >>>>>> >>>>>> It would be very beneficial for further progress if these two distinct >>>>>> concerns could be separated, at least CLEARLY marked in your document. >>>>>> >>>>>> I have added some more comments inline. >>>>>> >>>>>> On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: >>>>>> >>>>>>> Concerning the document named 'Requirements for an improved Basic >>>>>>> Execution Service (BES)' and available at >>>>>>> http://forge.gridforum.org/sf/go/doc16306 : >>>>>>> >>>>>>> THANK YOU VERY MUCH for having taken the time to read this document, and >>>>>>> for having taken the time to provide comments : >>>>>>> >>>>>>> Such comments are very useful for the improvement of documents, permit >>>>>>> convergence and prepare later agreement. >>>>>>> [...] >>>>>>> On Thu, 15/09/2011 13:00, Bernd Schuller wrote: >>>>>>>> >>>>>>>> [...] >>>>>>> >>>>>>>> 1.4 Methodology >>>>>>>> >>>>>>>> ? ? * ref 4: glite user guide -> ? out of scope, gLite is too complex and >>>>>>>> too specific in its architecture (if there is such a thing) to be useful >>>>>>>> as a base for BES >>>>>>> ? ? Yes, this is a very large user guide for specific usage of gLite by >>>>>>> users, and it does NOT provide a clear description of the architecture >>>>>>> and of the functionalities. >>>>>>> ? ? But it is NOT so complex. ? As soon as I finished reading this guide, >>>>>>> it was easy for me to perform reverse engineering and extract the >>>>>>> effective architecture (SOA with internal interfaces) and the implied >>>>>>> functionalities (which are to be improved). >>>>>> >>>>>> Indeed it appears that you try to impose gLite specifics (like a logging >>>>>> & ? bookkeeping service or proxy certificates on the transport level) as >>>>>> requirements. This would severly limit the BES specification effort, and >>>>>> will not be accepted (I hope) by other stakeholders. >>>>>> >>>>>>> ? ? In the text, I have prepended a few words to explain that. >>>>>> >>>>>> Basically you imply that the "architecture and functionalities" of the >>>>>> gLite execution system (together with the PGI work) is somehow the >>>>>> guideline to be followed, which I fully disagree with. >>>>>> >>>>>>>> >>>>>>>> 2.4 Collaboration with other services >>>>>>>> >>>>>>>> While this is important for interoperability, it is unimportant >>>>>>>> for the specification of a BES. The BES spec should NOT try to specify >>>>>>>> all the interaction with the rest of the world. This is the task of a >>>>>>>> "grid architecture specification" like OGSA. >>>>>>> ? ? My document is NOT targeted only to the specification of the BES >>>>>>> Client interface, but to the clear and consistent description of BES >>>>>>> context and functional + operational requirements which are really >>>>>>> necessary for interoperability. >>>>>>> ? ? As far as I know, OGSA does NOT take into account GLUE 2.0 yet. >>>>>>> Therefore, an up to date 'grid architecture specification' is absolutely >>>>>>> necessary. >>>>>> >>>>>> Glue2 is just an information model, not necessarily a perfect one nor >>>>>> the only one. However, I agree an information model has to be adopted >>>>>> for BES and any associated information systems. >>>>>> >>>>>>> ? ? If OGSA members consider chapters 2.3 and 2.4 of my document as a >>>>>>> 'grid architecture specification' which updates and improves OGSA, I >>>>>>> thank them. ? If they consider that this 'grid architecture >>>>>>> specification' does NOT comply with OGSA and competes with it, then I >>>>>>> assert that it obsoletes OGSA. >>>>>> >>>>>> I can't really say, the OGSA group stopped its work a long time ago and >>>>>> it's a long time that I looked at the documents. >>>>>> >>>>>>>> >>>>>>>> Specifically, the interactions with security, monitoring, accounting and >>>>>>>> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory >>>>>>>> part of a BES specification. >>>>>>> ? ? FAILURE of practical operations is often caused by LACK of early care >>>>>>> about operational concerns during specification phase. >>>>>> >>>>>> Agreed. >>>>>> >>>>>>> ? As GIN-GC has >>>>>>> proven and documented, this is even more true for interoperability on >>>>>>> the field (as opposed to theoretical interoperability at the WSDL level). >>>>>>> ? ? I confirm that care about operational concerns is REQUIRED for real >>>>>>> operations and for practical interoperability on the field. ? Although >>>>>>> operational concerns are NOT part of the BES Client interface, they are >>>>>>> REQUIRED for the overall specifications of BES in its context. >>>>>>> ? ? In the text, I have stressed that the document DOES take into account >>>>>>> operational concerns. >>>>>>> >>>>>>>> >>>>>>>> 4. BES non-functional requirements >>>>>>>> >>>>>>>> 4.1.2 Traceability - should be SHOULD not MUST >>>>>>> ? ? This is an operational concern : ? Would you really take the risk that >>>>>>> the whole EGI becomes a spambot or a scambot ? >>>>>> >>>>>> Isn't it already, powered by gLite and used by the wlcg botnet (just >>>>>> kidding of course) :-) >>>>>> >>>>>>> ? ? No traceability --> ? No post mortem analysis after attack --> ? Large >>>>>>> infection --> ? Panic --> ? Abrupt and very long shutdown of all services. >>>>>>> ? ? I fully confirm that traceability is a MUST. >>>>>> >>>>>> It is an internal detail which any good implementation will provide. >>>>>> If BES-A is much easier and more secure to operate than BES-B, admins >>>>>> can choose which to install. >>>>>> >>>>>>>> >>>>>>>> 4.1.3 Security >>>>>>>> ? ? - how can a specification "implement" a policy? You probably >>>>>>>> ? ? ? meant "BES implementations SHOULD ..." >>>>>>> ? ? The text now is 'BES specifications MUST fully take into account the >>>>>>> Security Policies ...' >>>>>> >>>>>> Still no understanding here... let's take traceability >>>>>> The relevant EGI policy >>>>>> ? says >>>>>> >>>>>> "[...] software deployed in the Grid MUST include the >>>>>> ability to produce sufficient and relevant logging [...] >>>>>> The level of the logging MUST be configured by all service providers, >>>>>> including but not limited to the Sites, to produce the required >>>>>> information which MUST be retained for a minimum of 90 days." >>>>>> >>>>>> For example all UNICORE services can be made to comply with this >>>>>> by configuration of the logging library we use (Apache Log4j), and by >>>>>> not deleting log files for 90 days. >>>>>> So this is a feature of the implementation and the administrator in >>>>>> charge, not the specification. Thus, your sentence should read "BES >>>>>> implementations SHOULD ..." (It is MUST of course only if they want to >>>>>> be deployed in EGI) >>>>>> One does not try to specify implementation details, at least not in any >>>>>> specification I've ever seen (e.g. does the HTTP specification say >>>>>> anything about server logging or accepted CAs?). >>>>>> >>>>>> >>>>>> >>>>>>> [...] >>>>>>>> >>>>>>>> 4.2 >>>>>>>> ? ? all of this is out of scope. For example the UNICORE service >>>>>>>> container hosts a number of services including an execution service. >>>>>>>> Probably you mean that the execution service SPECIFICATION should be >>>>>>>> limited to the execution service and MUST NOT specify accounting, >>>>>>>> security etc. >>>>>>> ? ? 'Well defined and narrow scope' is a general engineering requirement. >>>>>>> ? ? It is fundamental concern crossing requirements, design, >>>>>>> specifications, fabrication, tests, operations, user experience and >>>>>>> product maintenance for all types of products, even outside software >>>>>>> engineering. >>>>>> >>>>>> exactly. >>>>>> >>>>>>> ? ? I confirm that 'Well defined and narrow scope' is absolutely REQUIRED >>>>>>> for sound software design, implementation, deployment and maintainability. >>>>>>> ? ? From your comment, I assume that the Execution Service of UNICORE >>>>>>> does have a 'Well defined and narrow scope', does have precise >>>>>>> interfaces with other UNICORE services, and minimizes overlaps with them. >>>>>> >>>>>> of course. And UNICORE does not include a L&B service :-) >>>>>> >>>>>> >>>>>>> [...] >>>>>>>> 6.1 >>>>>>>> >>>>>>>> ? ? * "SSL certificates MUST be signed by a CA..." this is an operational >>>>>>>> decision, and has nothing to do with the BES spec. >>>>>>>> For example, a site may run an inhouse deployment of BES using an >>>>>>>> in-house CA. This requirement should be deleted. >>>>>>> ? ? This operational concern is REQUIRED for practical interoperability >>>>>>> on the field. ? I have prepended : >>>>>>> ? ? * Authentication of Servers : ? The Execution Service SHOULD permit >>>>>>> Clients to authenticate it. ? If an Execution Service authenticates >>>>>>> itself to clients, it MUST permit Clients to really perform this >>>>>>> authentication. >>>>>> >>>>>> This sentence makes no sense to me, sorry. >>>>>> Maybe "Server and client SHOULD communicate via a secure channel >>>>>> (SSL/TLS)". Even this may not be true in the future, though it is for >>>>>> all(?) the Grid systems currently. >>>>>> >>>>>>>> >>>>>>>> 6.3 >>>>>>>> >>>>>>>> * "For Client authentication, the Execution Service MUST accept all >>>>>>>> following authentication methods: ? Full X509, ? RFC-3820-compliant X509 >>>>>>>> Proxy" >>>>>>>> >>>>>>>> This requirement is invalid. I agree that it would be nice to be able to >>>>>>>> specify authentication methods, but it is impossible. For example >>>>>>>> Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface >>>>>>>> over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be >>>>>>>> valid authentication methods in some circumstances. >>>>>>> ? ? There are 2 separate requirements : >>>>>>> ? ? - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy >>>>>>> ? ? - 1 'MAY' ? for all other ones. >>>>>>>> >>>>>>>> Furthermore, making proxies a MUST implies that nonstandard >>>>>>>> authentication libraries instead of TLS/SSL must be used, making the BES >>>>>>>> implementation insecure. For some implementors (like UNICORE) proxies on >>>>>>>> the transport level are very much a no-go. >>>>>>> ? ? I had clearly specified RFC-3820-compliant X509 Proxy, which ARE >>>>>>> standard. >>>>>>> ? ? Your critics are valid for GSI proxies, which I have taken care NOT >>>>>>> to mention. >>>>>> >>>>>> by "standard" I did not mean that it is an RFC, but software support. >>>>>> As opposed to standard SSL/TLS, proxies are almost not supported by >>>>>> industry standard tools, for example Apache httpd or the Java JDK. >>>>>> One has to rely on custom code, which is notoriously buggy and error >>>>>> prone. >>>>>> >>>>>> Since one important non-functional requirement (for me at least) is to >>>>>> be able use standard (off-the-shelf) open source software, having to >>>>>> support proxies is a big limitation. >>>>>> >>>>>>>> 6.4. >>>>>>>> >>>>>>>> "This authorization mechanism MUST be consistent across all instances of >>>>>>>> the Execution Service" >>>>>>>> >>>>>>>> This violates the autonomy of a site. Site administrators often wish to >>>>>>>> stay in control of their resources, and do not accept external >>>>>>>> authorisation decision points. And anyway, who cares? Since the AuthZ >>>>>>>> mechanism is internal to the BES, it cannot be specified in the >>>>>>>> BES spec as such. >>>>>>> ? ? Interoperability requires a federation of independent administrative >>>>>>> domain to agree on common functionalities, interfaces and operations. >>>>>>> ? ? This DOES sometime violate the autonomy of each individual site. >>>>>>> ? ? The requirement is NOT that the AUTHZ decision point is external to >>>>>>> any site, but that all participating site MUST accept to install inside >>>>>>> their site an instance of a commonly validated software implementing the >>>>>>> decision point. >>>>>> >>>>>> No. Each site may choose their own authz decision point, IMO. >>>>>> >>>>>>> ? ? The AUTHZ mechanism MUST NOT be internal to the BES : >>>>>> >>>>>> Maybe I was not clear. The authz mechanism is invisible for outside >>>>>> parties (like clients). It can be an external component, an internal >>>>>> component, whatever, it is up to the BES implementor and the site admin. >>>>>> >>>>>>> For example, >>>>>>> in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package >>>>>>> is described as 'The security subsystem of UNICORE/X', and is NOT >>>>>>> internal to 'de.fzj.unicore.uas.impl.job'. >>>>>> >>>>>> In UNICORE site admins can choose what attribute sources and XACML >>>>>> decision points they want to use, but the clients (including other >>>>>> services) do not need to know this. That is what I meant by "internal". >>>>>> >>>>>>>> >>>>>>>> 6.5 >>>>>>>> >>>>>>>> These are reqiurements on the security layer (or framework) and should >>>>>>>> not be used as requirements on BES. >>>>>>> ? ? These security requirements DO have impacts on the BES Client >>>>>>> interface and on the Job Description document. >>>>>>> ? ? In the text, I have made it clear. >>>>>> >>>>>> indeed while preparing the EMI-ES specification, we came to the >>>>>> following conclusions >>>>>> 1) when using proxies for delegation, it is necessary to map each data >>>>>> staging item to a delegated credential (you can check the EMI-ES job >>>>>> description for details) >>>>>> 2) the delegation operations are separate from the job management >>>>>> operations, so they do not necessarily have to be part of the BES client >>>>>> interface. >>>>>> >>>>>> Also, there are existing implementations (UNICORE and Genesis come to my >>>>>> mind) that do not need this at all, because they do delegation without >>>>>> proxies. >>>>>> >>>>>> So I disagree, 6.5 mostly describes features of the particular security >>>>>> framework that is used. >>>>>> >>>>>>>> >>>>>>>> 8 BES requirements related to "Application Repositories" >>>>>>>> >>>>>>>> While I agree that BES should understand the notion of an "Application" >>>>>>>> (see e.g. JSDL ApplicationName), I don't agree that the BES should >>>>>>>> use these for Scheduling. Rather, this is the job of a broker. >>>>>>> ? ? The text is now : >>>>>>> ? ? * Resource selection : ? The Execution Service MUST use, among others, >>>>>>> these references to 'Installed Applications' in order to select the most >>>>>>> adequate computing resource for the Job. >>>>>>>> >>>>>>>> 9 BES requirements applying to Accounting >>>>>>>> >>>>>>>> As a "MUST", these are out of scope, and should be made "SHOULD". >>>>>>> ? ? No Accounting --> ? No precise reporting to funding agencies --> ? No >>>>>>> funding --> ? Abrupt and very long shutdown of all services. >>>>>>> ? ? I fully confirm that Accounting is a MUST. >>>>>>> >>>>>> >>>>>> ... operational >>>>>> >>>>>>>> >>>>>>>> 10 Logging/Bookkeeping >>>>>>>> >>>>>>>> Same as 9. >>>>>>> ? ? Same as 'Traceability' : >>>>>>> ? ? This is an operational concern : ? Would you really take the risk that >>>>>>> the whole EGI becomes a spambot or a scambot ? >>>>>>> ? ? No Logging and Bookkeeping --> ? No post mortem analysis after attack >>>>>>> --> ? Large infection --> ? Panic --> ? Abrupt and very long shutdown of all >>>>>>> services. >>>>>>> ? ? I fully confirm that Logging and Bookkeeping is a MUST. >>>>>>> >>>>>> >>>>>> an operational MUST maybe for some infrastructures, not all. >>>>>> >>>>>> E.g. a typical HPC site has its own accounting, its own logging systems >>>>>> independent of the (Grid) software used to submit jobs to it. >>>>>> >>>>>>>> >>>>>>>> 12 Jobs >>>>>>>> >>>>>>>> 12.1 Types of job >>>>>>>> >>>>>>>> Support for parallel jobs: it should be "MUST" :-) >>>>>>> ? ? The text is now : >>>>>>> ? ? - The concept of 'Single Job' includes Jobs running >>>>>>> massively-parallel processes using MPI on one large-scale HPC System. >>>>>>> The Execution Service MUST understand instructions for usage of MPI >>>>>>> inside the Job Description document. ? The Execution Service SHOULD >>>>>>> transmit these instructions to the Batch System, or return an explicit >>>>>>> error message if not supported. >>>>>> >>>>>> OK >>>>>> >>>>>> >>>>>> >>>>>> Best regards, >>>>>> Bernd. >>>>>> >>>>>> >>>>>> -- >>>>>> Dr. Bernd Schuller >>>>>> Federated Systems and Data >>>>>> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >>>>>> Phone: +49 246161-8736 (fax -8556) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------------------------------ >>>>>> ------------------------------------------------------------------------------------------------ >>>>>> Forschungszentrum Juelich GmbH >>>>>> 52425 Juelich >>>>>> Sitz der Gesellschaft: Juelich >>>>>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >>>>>> Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher >>>>>> Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), >>>>>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >>>>>> Prof. Dr. Sebastian M. Schmidt >>>>>> ------------------------------------------------------------------------------------------------ >>>>>> ------------------------------------------------------------------------------------------------ >>>>> >>>> >>>> -- >>>> Dr. Bernd Schuller >>>> Federated Systems and Data >>>> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >>>> Phone: +49 246161-8736 (fax -8556) >>>> >>>> >>>> >>>> >>> >>> >>> >>> -- >>> Nothing is ever easy... >>> -- >>> ? ogsa-bes-wg mailing list >>> ? ogsa-bes-wg at ogf.org >>> ? http://www.ogf.org/mailman/listinfo/ogsa-bes-wg >> >> -- >> ? ogsa-bes-wg mailing list >> ? ogsa-bes-wg at ogf.org >> ? http://www.ogf.org/mailman/listinfo/ogsa-bes-wg >> > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3882 bytes Desc: S/MIME Cryptographic Signature Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110920/f109e489/attachment-0001.bin From alexander.papaspyrou at tu-dortmund.de Tue Sep 20 04:50:34 2011 From: alexander.papaspyrou at tu-dortmund.de (alexander.papaspyrou at tu-dortmund.de) Date: Tue, 20 Sep 2011 09:50:34 +0000 Subject: [OGSA-BES-WG] OGF OGSA-BES - Requirements for an improved Basic Execution Service In-Reply-To: <7BC2FE4B-CC94-42BA-88E2-15C78AD45A2E@mcs.anl.gov> References: <4E5FD1DE.6000404@lal.in2p3.fr> <4E67AAE7.9070606@lal.in2p3.fr> <4E6A7C69.9010308@lal.in2p3.fr> <4E6FA620.8020307@lal.in2p3.fr> <1316084455.1787.18.camel@zam994> <4E723564.70407@lal.in2p3.fr> <1316119475.24230.60.camel@zam994> <4E7333DE.5040809@lal.in2p3.fr> <1316179590.25767.49.camel@zam994> <7BC2FE4B-CC94-42BA-88E2-15C78AD45A2E@mcs.anl.gov> Message-ID: +1. Actually, some major effort has been put into the jGlobus 2 code to make this independent from Globus, and rather tightly integrate with the standard APIs. Still, I don't see why all this is such a big deal? It makes sense to support RFC 3820, as it makes sense to support SAML, since both are somewhat widespread in the community. Whether this needs to be a MUST in the spec, I frankly don't know. -Alexander Am 19.09.2011 um 17:57 schrieb Stuart Martin: > I just wanted to point out that RFC-3820 compliant X.509 proxies are supported in OpenSSL. Several project (e.g., LIGO) use OpenSSL with proxy certs with the Apache server. > > Also, jGlobus 2 has support for RFC-3820 compliant X.509 proxies. Included is an implementation of a java SSL TrustManager that has proxy processing. > > -Stu > > On Sep 16, 2011, at Sep 16, 8:36 AM, Andre Merzky wrote: > >> +1 on all points. >> >> Cheers, Andre. >> >> >> 2011/9/16 Bernd Schuller : >>> hi Etienne, >>> >>> On Fri, 2011-09-16 at 13:32 +0200, Etienne URBAH wrote: >>> [...] >>>> >>>> RFC-3820-compliant X509 Proxies >>>> ------------------------------- >>>> The RFC-3820-compliant X509 proxies are fully supported by the jLite >>>> library written in Java by Oleg SUKHOROSLOV and available at >>>> http://code.google.com/p/jlite/ >>>> >>> >>> You must be joking. I was talking about open source software like Apache >>> httpd and Java JDK. jLite just wraps the cog-globus libraries and adds >>> some gLite access APIs. No, thanks. >>> >>>> >>>> Dependency of BES on other grid software components / operational issues >>>> ------------------------------------------------------------------------ >>>> It is very good that we know agree on GLUE 2.0 as base for the >>>> Information System. Otherwise, we could NOT agree on the way to express >>>> references to grid entities in the Job Description document. >>>> >>>> Your comments about chapter 6.5 confirm that the specifications of the >>>> BES Client interface DEPENDS on whether BES supports X509 proxies for >>>> delegation of Security credentials or NOT. >>>> >>> >>> At least this was the EMI-ES v1.0 conclusion, which need not be the >>> final word. The only dependency (in EMI-ES) is the specification which >>> delegated credential is to be used for which data staging item. >>> >>> Delegation can be performed FULLY TRANSPARENT to the BES (for example on >>> a message level as in UNICORE), and the BES interface specification is >>> not dependent on it at all. >>> >>>> Since delegation of Security credentials is a MUST for BES, my >>>> conclusion is that we MUST agree on SECURITY issues (even if some are >>>> operational issues) BEFORE trying to write down BES requirements. >>>> >>>> In the same way, we know that Clients need to perform complex queries on >>>> Jobs. The BES Client interface DEPENDS on whether such queries are >>>> accepted by BES itself, of by a separate Logging and Bookkeeping >>>> service. So, I think that we have to agree on the existence or absence >>>> of a separate Logging and Bookkeeping service BEFORE trying to write >>>> down BES requirements about queries. >>> >>> >>> The BES client interface does not necessarily need to allow to perform >>> complex queries, these should be part of a separate interface. >>> >>> >>>> As a summary : >>>> - Some BES requirements are quite independent from other grid >>>> components, and we can discuss on them immediately. >>>> - But some other BES requirements are DEPENDENT from foundational grid >>>> components or operational issues, in particular Information System, >>>> Security, Logging and Bookkeeping, ... >>>> - Therefore, we have to agree on these other grid components or >>>> operational issues FIRST. >>>> >>>> This is a critical issue, and I propose that we discuss on it at OGF33. >>> >>> Unfortunately I won't be in Lyon, only via phone. >>> >>> Summarising, my main points for the BES interface specification : >>> >>> * specifications must be narrowly scoped and composable (which is the >>> JSDL/BES model anyway). >>> >>> * do not try to specify specific authentication methods. Do not try to >>> specify specific delegation methods. Security is a cross cutting concern >>> and should be dealt with separately. >>> >>> * do not assume a special environment where a BES instance will run. >>> Interactions with other services (except for data access) are optional >>> and should be treated as such. >>> >>> * leave operational aspects to operations. Recommendations for BES >>> implementors may be given, of course. >>> >>> >>> Best regards, >>> Bernd. >>> >>>> >>>> I will answer to your other comments later. >>>> >>>> Best regards. >>>> >>>> ----------------------------------------------------- >>>> Etienne URBAH LAL, Univ Paris-Sud, IN2P3/CNRS >>>> Bat 200 91898 ORSAY France >>>> Tel: +33 1 64 46 84 87 Skype: etienne.urbah >>>> Mob: +33 6 22 30 53 27 mailto:urbah at lal.in2p3.fr >>>> ----------------------------------------------------- >>>> >>>> >>>> On Thu, 15/09/2011 22:44, Bernd Schuller wrote: >>>>> Hi Etienne, >>>>> >>>>> thanks for the clarifications. So indeed your document is aimed at both: >>>>> >>>>> 1) providing requirements for the actual BES specification ("client >>>>> interface" in your terminology) >>>>> 2) the operation and deployment issues that have to be solved for >>>>> interoperability on an infrastructure level (say EGI and EDGI). >>>>> >>>>> It would be very beneficial for further progress if these two distinct >>>>> concerns could be separated, at least CLEARLY marked in your document. >>>>> >>>>> I have added some more comments inline. >>>>> >>>>> On Thu, 2011-09-15 at 19:27 +0200, Etienne URBAH wrote: >>>>> >>>>>> Concerning the document named 'Requirements for an improved Basic >>>>>> Execution Service (BES)' and available at >>>>>> http://forge.gridforum.org/sf/go/doc16306 : >>>>>> >>>>>> THANK YOU VERY MUCH for having taken the time to read this document, and >>>>>> for having taken the time to provide comments : >>>>>> >>>>>> Such comments are very useful for the improvement of documents, permit >>>>>> convergence and prepare later agreement. >>>>>> [...] >>>>>> On Thu, 15/09/2011 13:00, Bernd Schuller wrote: >>>>>>> >>>>>>> [...] >>>>>> >>>>>>> 1.4 Methodology >>>>>>> >>>>>>> * ref 4: glite user guide -> out of scope, gLite is too complex and >>>>>>> too specific in its architecture (if there is such a thing) to be useful >>>>>>> as a base for BES >>>>>> Yes, this is a very large user guide for specific usage of gLite by >>>>>> users, and it does NOT provide a clear description of the architecture >>>>>> and of the functionalities. >>>>>> But it is NOT so complex. As soon as I finished reading this guide, >>>>>> it was easy for me to perform reverse engineering and extract the >>>>>> effective architecture (SOA with internal interfaces) and the implied >>>>>> functionalities (which are to be improved). >>>>> >>>>> Indeed it appears that you try to impose gLite specifics (like a logging >>>>> & bookkeeping service or proxy certificates on the transport level) as >>>>> requirements. This would severly limit the BES specification effort, and >>>>> will not be accepted (I hope) by other stakeholders. >>>>> >>>>>> In the text, I have prepended a few words to explain that. >>>>> >>>>> Basically you imply that the "architecture and functionalities" of the >>>>> gLite execution system (together with the PGI work) is somehow the >>>>> guideline to be followed, which I fully disagree with. >>>>> >>>>>>> >>>>>>> 2.4 Collaboration with other services >>>>>>> >>>>>>> While this is important for interoperability, it is unimportant >>>>>>> for the specification of a BES. The BES spec should NOT try to specify >>>>>>> all the interaction with the rest of the world. This is the task of a >>>>>>> "grid architecture specification" like OGSA. >>>>>> My document is NOT targeted only to the specification of the BES >>>>>> Client interface, but to the clear and consistent description of BES >>>>>> context and functional + operational requirements which are really >>>>>> necessary for interoperability. >>>>>> As far as I know, OGSA does NOT take into account GLUE 2.0 yet. >>>>>> Therefore, an up to date 'grid architecture specification' is absolutely >>>>>> necessary. >>>>> >>>>> Glue2 is just an information model, not necessarily a perfect one nor >>>>> the only one. However, I agree an information model has to be adopted >>>>> for BES and any associated information systems. >>>>> >>>>>> If OGSA members consider chapters 2.3 and 2.4 of my document as a >>>>>> 'grid architecture specification' which updates and improves OGSA, I >>>>>> thank them. If they consider that this 'grid architecture >>>>>> specification' does NOT comply with OGSA and competes with it, then I >>>>>> assert that it obsoletes OGSA. >>>>> >>>>> I can't really say, the OGSA group stopped its work a long time ago and >>>>> it's a long time that I looked at the documents. >>>>> >>>>>>> >>>>>>> Specifically, the interactions with security, monitoring, accounting and >>>>>>> logging framework are OPERATIONAL concerns that MUST NOT be a mandatory >>>>>>> part of a BES specification. >>>>>> FAILURE of practical operations is often caused by LACK of early care >>>>>> about operational concerns during specification phase. >>>>> >>>>> Agreed. >>>>> >>>>>> As GIN-GC has >>>>>> proven and documented, this is even more true for interoperability on >>>>>> the field (as opposed to theoretical interoperability at the WSDL level). >>>>>> I confirm that care about operational concerns is REQUIRED for real >>>>>> operations and for practical interoperability on the field. Although >>>>>> operational concerns are NOT part of the BES Client interface, they are >>>>>> REQUIRED for the overall specifications of BES in its context. >>>>>> In the text, I have stressed that the document DOES take into account >>>>>> operational concerns. >>>>>> >>>>>>> >>>>>>> 4. BES non-functional requirements >>>>>>> >>>>>>> 4.1.2 Traceability - should be SHOULD not MUST >>>>>> This is an operational concern : Would you really take the risk that >>>>>> the whole EGI becomes a spambot or a scambot ? >>>>> >>>>> Isn't it already, powered by gLite and used by the wlcg botnet (just >>>>> kidding of course) :-) >>>>> >>>>>> No traceability --> No post mortem analysis after attack --> Large >>>>>> infection --> Panic --> Abrupt and very long shutdown of all services. >>>>>> I fully confirm that traceability is a MUST. >>>>> >>>>> It is an internal detail which any good implementation will provide. >>>>> If BES-A is much easier and more secure to operate than BES-B, admins >>>>> can choose which to install. >>>>> >>>>>>> >>>>>>> 4.1.3 Security >>>>>>> - how can a specification "implement" a policy? You probably >>>>>>> meant "BES implementations SHOULD ..." >>>>>> The text now is 'BES specifications MUST fully take into account the >>>>>> Security Policies ...' >>>>> >>>>> Still no understanding here... let's take traceability >>>>> The relevant EGI policy >>>>> says >>>>> >>>>> "[...] software deployed in the Grid MUST include the >>>>> ability to produce sufficient and relevant logging [...] >>>>> The level of the logging MUST be configured by all service providers, >>>>> including but not limited to the Sites, to produce the required >>>>> information which MUST be retained for a minimum of 90 days." >>>>> >>>>> For example all UNICORE services can be made to comply with this >>>>> by configuration of the logging library we use (Apache Log4j), and by >>>>> not deleting log files for 90 days. >>>>> So this is a feature of the implementation and the administrator in >>>>> charge, not the specification. Thus, your sentence should read "BES >>>>> implementations SHOULD ..." (It is MUST of course only if they want to >>>>> be deployed in EGI) >>>>> One does not try to specify implementation details, at least not in any >>>>> specification I've ever seen (e.g. does the HTTP specification say >>>>> anything about server logging or accepted CAs?). >>>>> >>>>> >>>>> >>>>>> [...] >>>>>>> >>>>>>> 4.2 >>>>>>> all of this is out of scope. For example the UNICORE service >>>>>>> container hosts a number of services including an execution service. >>>>>>> Probably you mean that the execution service SPECIFICATION should be >>>>>>> limited to the execution service and MUST NOT specify accounting, >>>>>>> security etc. >>>>>> 'Well defined and narrow scope' is a general engineering requirement. >>>>>> It is fundamental concern crossing requirements, design, >>>>>> specifications, fabrication, tests, operations, user experience and >>>>>> product maintenance for all types of products, even outside software >>>>>> engineering. >>>>> >>>>> exactly. >>>>> >>>>>> I confirm that 'Well defined and narrow scope' is absolutely REQUIRED >>>>>> for sound software design, implementation, deployment and maintainability. >>>>>> From your comment, I assume that the Execution Service of UNICORE >>>>>> does have a 'Well defined and narrow scope', does have precise >>>>>> interfaces with other UNICORE services, and minimizes overlaps with them. >>>>> >>>>> of course. And UNICORE does not include a L&B service :-) >>>>> >>>>> >>>>>> [...] >>>>>>> 6.1 >>>>>>> >>>>>>> * "SSL certificates MUST be signed by a CA..." this is an operational >>>>>>> decision, and has nothing to do with the BES spec. >>>>>>> For example, a site may run an inhouse deployment of BES using an >>>>>>> in-house CA. This requirement should be deleted. >>>>>> This operational concern is REQUIRED for practical interoperability >>>>>> on the field. I have prepended : >>>>>> * Authentication of Servers : The Execution Service SHOULD permit >>>>>> Clients to authenticate it. If an Execution Service authenticates >>>>>> itself to clients, it MUST permit Clients to really perform this >>>>>> authentication. >>>>> >>>>> This sentence makes no sense to me, sorry. >>>>> Maybe "Server and client SHOULD communicate via a secure channel >>>>> (SSL/TLS)". Even this may not be true in the future, though it is for >>>>> all(?) the Grid systems currently. >>>>> >>>>>>> >>>>>>> 6.3 >>>>>>> >>>>>>> * "For Client authentication, the Execution Service MUST accept all >>>>>>> following authentication methods: Full X509, RFC-3820-compliant X509 >>>>>>> Proxy" >>>>>>> >>>>>>> This requirement is invalid. I agree that it would be nice to be able to >>>>>>> specify authentication methods, but it is impossible. For example >>>>>>> Shibboleth, Username/password, OpenID, OAuth (e.g. for a REST interface >>>>>>> over plain HTTP), or even NOTHING (e.g. in an inhouse grid) all can be >>>>>>> valid authentication methods in some circumstances. >>>>>> There are 2 separate requirements : >>>>>> - 1 'MUST' for Full X509 and RFC-3820-compliant X509 Proxy >>>>>> - 1 'MAY' for all other ones. >>>>>>> >>>>>>> Furthermore, making proxies a MUST implies that nonstandard >>>>>>> authentication libraries instead of TLS/SSL must be used, making the BES >>>>>>> implementation insecure. For some implementors (like UNICORE) proxies on >>>>>>> the transport level are very much a no-go. >>>>>> I had clearly specified RFC-3820-compliant X509 Proxy, which ARE >>>>>> standard. >>>>>> Your critics are valid for GSI proxies, which I have taken care NOT >>>>>> to mention. >>>>> >>>>> by "standard" I did not mean that it is an RFC, but software support. >>>>> As opposed to standard SSL/TLS, proxies are almost not supported by >>>>> industry standard tools, for example Apache httpd or the Java JDK. >>>>> One has to rely on custom code, which is notoriously buggy and error >>>>> prone. >>>>> >>>>> Since one important non-functional requirement (for me at least) is to >>>>> be able use standard (off-the-shelf) open source software, having to >>>>> support proxies is a big limitation. >>>>> >>>>>>> 6.4. >>>>>>> >>>>>>> "This authorization mechanism MUST be consistent across all instances of >>>>>>> the Execution Service" >>>>>>> >>>>>>> This violates the autonomy of a site. Site administrators often wish to >>>>>>> stay in control of their resources, and do not accept external >>>>>>> authorisation decision points. And anyway, who cares? Since the AuthZ >>>>>>> mechanism is internal to the BES, it cannot be specified in the >>>>>>> BES spec as such. >>>>>> Interoperability requires a federation of independent administrative >>>>>> domain to agree on common functionalities, interfaces and operations. >>>>>> This DOES sometime violate the autonomy of each individual site. >>>>>> The requirement is NOT that the AUTHZ decision point is external to >>>>>> any site, but that all participating site MUST accept to install inside >>>>>> their site an instance of a commonly validated software implementing the >>>>>> decision point. >>>>> >>>>> No. Each site may choose their own authz decision point, IMO. >>>>> >>>>>> The AUTHZ mechanism MUST NOT be internal to the BES : >>>>> >>>>> Maybe I was not clear. The authz mechanism is invisible for outside >>>>> parties (like clients). It can be an external component, an internal >>>>> component, whatever, it is up to the BES implementor and the site admin. >>>>> >>>>>> For example, >>>>>> in UNICORE atomic services, the 'de.fzj.unicore.uas.security' package >>>>>> is described as 'The security subsystem of UNICORE/X', and is NOT >>>>>> internal to 'de.fzj.unicore.uas.impl.job'. >>>>> >>>>> In UNICORE site admins can choose what attribute sources and XACML >>>>> decision points they want to use, but the clients (including other >>>>> services) do not need to know this. That is what I meant by "internal". >>>>> >>>>>>> >>>>>>> 6.5 >>>>>>> >>>>>>> These are reqiurements on the security layer (or framework) and should >>>>>>> not be used as requirements on BES. >>>>>> These security requirements DO have impacts on the BES Client >>>>>> interface and on the Job Description document. >>>>>> In the text, I have made it clear. >>>>> >>>>> indeed while preparing the EMI-ES specification, we came to the >>>>> following conclusions >>>>> 1) when using proxies for delegation, it is necessary to map each data >>>>> staging item to a delegated credential (you can check the EMI-ES job >>>>> description for details) >>>>> 2) the delegation operations are separate from the job management >>>>> operations, so they do not necessarily have to be part of the BES client >>>>> interface. >>>>> >>>>> Also, there are existing implementations (UNICORE and Genesis come to my >>>>> mind) that do not need this at all, because they do delegation without >>>>> proxies. >>>>> >>>>> So I disagree, 6.5 mostly describes features of the particular security >>>>> framework that is used. >>>>> >>>>>>> >>>>>>> 8 BES requirements related to "Application Repositories" >>>>>>> >>>>>>> While I agree that BES should understand the notion of an "Application" >>>>>>> (see e.g. JSDL ApplicationName), I don't agree that the BES should >>>>>>> use these for Scheduling. Rather, this is the job of a broker. >>>>>> The text is now : >>>>>> * Resource selection : The Execution Service MUST use, among others, >>>>>> these references to 'Installed Applications' in order to select the most >>>>>> adequate computing resource for the Job. >>>>>>> >>>>>>> 9 BES requirements applying to Accounting >>>>>>> >>>>>>> As a "MUST", these are out of scope, and should be made "SHOULD". >>>>>> No Accounting --> No precise reporting to funding agencies --> No >>>>>> funding --> Abrupt and very long shutdown of all services. >>>>>> I fully confirm that Accounting is a MUST. >>>>>> >>>>> >>>>> ... operational >>>>> >>>>>>> >>>>>>> 10 Logging/Bookkeeping >>>>>>> >>>>>>> Same as 9. >>>>>> Same as 'Traceability' : >>>>>> This is an operational concern : Would you really take the risk that >>>>>> the whole EGI becomes a spambot or a scambot ? >>>>>> No Logging and Bookkeeping --> No post mortem analysis after attack >>>>>> --> Large infection --> Panic --> Abrupt and very long shutdown of all >>>>>> services. >>>>>> I fully confirm that Logging and Bookkeeping is a MUST. >>>>>> >>>>> >>>>> an operational MUST maybe for some infrastructures, not all. >>>>> >>>>> E.g. a typical HPC site has its own accounting, its own logging systems >>>>> independent of the (Grid) software used to submit jobs to it. >>>>> >>>>>>> >>>>>>> 12 Jobs >>>>>>> >>>>>>> 12.1 Types of job >>>>>>> >>>>>>> Support for parallel jobs: it should be "MUST" :-) >>>>>> The text is now : >>>>>> - The concept of 'Single Job' includes Jobs running >>>>>> massively-parallel processes using MPI on one large-scale HPC System. >>>>>> The Execution Service MUST understand instructions for usage of MPI >>>>>> inside the Job Description document. The Execution Service SHOULD >>>>>> transmit these instructions to the Batch System, or return an explicit >>>>>> error message if not supported. >>>>> >>>>> OK >>>>> >>>>> >>>>> >>>>> Best regards, >>>>> Bernd. >>>>> >>>>> >>>>> -- >>>>> Dr. Bernd Schuller >>>>> Federated Systems and Data >>>>> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >>>>> Phone: +49 246161-8736 (fax -8556) >>>>> >>>>> >>>>> >>>>> >>>>> ------------------------------------------------------------------------------------------------ >>>>> ------------------------------------------------------------------------------------------------ >>>>> Forschungszentrum Juelich GmbH >>>>> 52425 Juelich >>>>> Sitz der Gesellschaft: Juelich >>>>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 >>>>> Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher >>>>> Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), >>>>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, >>>>> Prof. Dr. Sebastian M. Schmidt >>>>> ------------------------------------------------------------------------------------------------ >>>>> ------------------------------------------------------------------------------------------------ >>>> >>> >>> -- >>> Dr. Bernd Schuller >>> Federated Systems and Data >>> Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc >>> Phone: +49 246161-8736 (fax -8556) >>> >>> >>> >>> >> >> >> >> -- >> Nothing is ever easy... >> -- >> ogsa-bes-wg mailing list >> ogsa-bes-wg at ogf.org >> http://www.ogf.org/mailman/listinfo/ogsa-bes-wg > > -- > ogsa-bes-wg mailing list > ogsa-bes-wg at ogf.org > http://www.ogf.org/mailman/listinfo/ogsa-bes-wg -- Alexander Papaspyrou alexander.papaspyrou at tu-dortmund.de -------------- next part -------------- A non-text attachment was scrubbed... Name: Alexander Papaspyrou.vcf Type: text/directory Size: 498 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110920/3f95ba41/attachment-0002.bin -------------- next part -------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4678 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110920/3f95ba41/attachment-0003.bin From andre at merzky.net Tue Sep 20 05:08:24 2011 From: andre at merzky.net (Andre Merzky) Date: Tue, 20 Sep 2011 12:08:24 +0200 Subject: [OGSA-BES-WG] [gin] REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in In-Reply-To: <4e7848f2.0d52cd0a.5884.1e81SMTPIN_ADDED@mx.google.com> References: <4e7848f2.0d52cd0a.5884.1e81SMTPIN_ADDED@mx.google.com> Message-ID: On Tue, Sep 20, 2011 at 9:40 AM, Morris Riedel wrote: > > (a) > Focus: baby steps together - concept by concept (e.g. manual data-staging, pre-post processing, scalable-time, etc.), we should not try to standardize all requirements at once, there could be BES 1.1, BES1.2, BES1.3, etc. +1 > (c) > Next steps: experience tells us that we will not manage to achieve very much in the time available at the OGF event, also some progress will be made - but how we organize the next months: milestones, weekly phone calls, chairs?!, specification work with deadlines, etc. how about a F2F before next OGF? Cheers, Andre. -- Nothing is ever easy... From m.riedel at fz-juelich.de Tue Sep 20 05:33:08 2011 From: m.riedel at fz-juelich.de (Morris Riedel) Date: Tue, 20 Sep 2011 12:33:08 +0200 Subject: [OGSA-BES-WG] [gin] REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in In-Reply-To: References: <4e7848f2.0d52cd0a.5884.1e81SMTPIN_ADDED@mx.google.com> Message-ID: <001001cc7780$b11da110$1358e330$@riedel@fz-juelich.de> Hi, good idea - EMI can invite folks to Brussels (rooms, Internet, available) and it should be reasonable 'central' for all and good to reach... We might discuss, Morris >-- -----Urspr?ngliche Nachricht----- >-- Von: andremerzky at gmail.com [mailto:andremerzky at gmail.com] Im Auftrag von Andre Merzky >-- Gesendet: Dienstag, 20. September 2011 12:08 >-- An: Riedel, Morris >-- Cc: pgi-wg at ogf.org; ogsa-bes-wg at ogf.org; jsdl-wg at ogf.org; gin at ogf.org >-- Betreff: Re: [gin] REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in >-- >-- On Tue, Sep 20, 2011 at 9:40 AM, Morris Riedel wrote: >-- > >-- > (a) >-- > Focus: baby steps together - concept by concept (e.g. manual data-staging, pre-post processing, >-- scalable-time, etc.), we should not try to standardize all requirements at once, there could be BES 1.1, >-- BES1.2, BES1.3, etc. >-- >-- +1 >-- >-- >-- > (c) >-- > Next steps: experience tells us that we will not manage to achieve very much in the time available at >-- the OGF event, also some progress will be made - but how we organize the next months: milestones, weekly >-- phone calls, chairs?!, specification work with deadlines, etc. >-- >-- how about a F2F before next OGF? >-- >-- Cheers, Andre. >-- >-- -- >-- Nothing is ever easy... -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3550 bytes Desc: not available Url : http://www.ogf.org/pipermail/ogsa-bes-wg/attachments/20110920/ed42a623/attachment.bin From grimshaw at virginia.edu Tue Sep 20 06:44:14 2011 From: grimshaw at virginia.edu (Andrew Grimshaw) Date: Tue, 20 Sep 2011 07:44:14 -0400 Subject: [OGSA-BES-WG] [jsdl-wg] [gin] REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in In-Reply-To: References: <4e7848f2.0d52cd0a.5884.1e81SMTPIN_ADDED@mx.google.com> Message-ID: <003501cc778a$a0caf6f0$e260e4d0$@edu> +1 - and profiles rather than specs when it makes sense. A -----Original Message----- From: jsdl-wg-bounces at ogf.org [mailto:jsdl-wg-bounces at ogf.org] On Behalf Of Andre Merzky Sent: Tuesday, September 20, 2011 6:08 AM To: Morris Riedel Cc: pgi-wg at ogf.org; jsdl-wg at ogf.org; gin at ogf.org; ogsa-bes-wg at ogf.org Subject: Re: [jsdl-wg] [gin] REMINDER: PGI-OGSA-BES-JSDL Joint Session Today at 2pm in On Tue, Sep 20, 2011 at 9:40 AM, Morris Riedel wrote: > > (a) > Focus: baby steps together - concept by concept (e.g. manual data-staging, pre-post processing, scalable-time, etc.), we should not try to standardize all requirements at once, there could be BES 1.1, BES1.2, BES1.3, etc. +1 > (c) > Next steps: experience tells us that we will not manage to achieve very much in the time available at the OGF event, also some progress will be made - but how we organize the next months: milestones, weekly phone calls, chairs?!, specification work with deadlines, etc. how about a F2F before next OGF? Cheers, Andre. -- Nothing is ever easy... -- jsdl-wg mailing list jsdl-wg at ogf.org http://www.ogf.org/mailman/listinfo/jsdl-wg