Enabling Grids for E-sciencE
Passing Requirement information to the gLite CE via BLAHPD.

Francesco Prelz - INFN Milano
HEPix, April 4th 2006
INFSO-RI-508833


Contents
Enabling Grids for E-sciencE
  • This is just a follow-on to what was discussed at the October 2005 HEPix. Most of the reference/background information can be found there.
    • Many more details will be given in the next talk.
  • Simplifying options that were made for the current prototype.
  • Work ahead.

INFSO-RI-508833


Progress made
Enabling Grids for E-sciencE
  • Out of the two approaches suggested, the DIY approach was chosen at the last meeting.
  • A generic way to extract "interesting" attributes out of the Requirements expression was found (in the ClassAd "flatten" function).
  • These were forwarded to the gLite CE (included in gLite 3.0).
  • The job submission component (org.glite.ce.blahp) was adapted to invoke a "site" script callout (this part is committed for gLite 3.1 - just *one* RPM on the CE).
  • A test "site" callout was provided by the CERN LSF team, as we'll see in the next talk.
  • We'll also see the results of the first tests.

INFSO-RI-508833


Current simplifying assumptions
Enabling Grids for E-sciencE
  • Lots of pressure against too general schemes...
  • Individual Glue schema attributes are considered.
  • Requirements on selected attributes (configured on the Resource Broker, a.k.a. WMS node) are extracted from the job Requirements and forwarded to the CE.
  • Binary operators (>, <, >=, <=, =) are identified in the resulting tree, and used to define shell variables for the local callout:
    GlueHostMainMemoryRAMSize_Min=800
    GlueHostOperatingSystemName=SLC3
    etc...
    • All >'s are effectively translated to >=, and all <'s are translated to <=.
  • In addition, the VirtualOrganisation attribute is set to the user VO.

INFSO-RI-508833


In a bit more detail
Enabling Grids for E-sciencE
  • Job JDL:
    Requirements = RegExp("\.mi\.infn\.it",other.GlueSubClusterUniqueID) && other.GlueCEPolicyMaxCPUTime > 100 ;
  • "Flattened" requirement:
    REMOTE_REMOTE_CeRequirements = "true && ( true && ( true && other.GlueCEPolicyMaxCPUTime > 100 ) )"
  • Shell variables for local callout:
    GlueCEPolicyMaxCPUTime_Min=100
    VirtualOrganisation=EGEE

INFSO-RI-508833


Work still to do
Enabling Grids for E-sciencE
  • Evaluate the effects on match-making and CE information provisioning of the desired changes on the fly of the batch system queue.
  • Assess the validity of the current simplifying assumptions.
  • Finalise the list of attributes to map over.
  • Figure out reasonable templates for other batch systems.
  • Testing and scaling-up.

INFSO-RI-508833