Re: AW: AW: [geclipse-dev] Storage vs. Service

[Ariel Garcia <garcia@xxxxxxxxxx>]

Hi

> > We should try to keep them more or less in sync, otherwise our code
> > gets completely messy!!!
>
> Nope, they were never really in sync. Every structure has its dedicated
> purpose (see above) and is therefore organised in a special way. Parts
> of the structures are synced, but this is rather a side effect than a
> concept.

wait, _IF_ the GLUE schema is to provide the information  and model 
general "Grid Infrastructures"  then it shouldn't  be tooooooo different 
(at least regarding grid resources... jobs, etc are another film) from 
what we have in our model... but yes, i know, nothing is so simple ;-)

> I agree to the "more or less" ;-) As long as it makes sense (well I
> think it makes sense in any case) AND is user-friendly (which does
> definitely not hold for the data access services since I - as a dumb
> user - would prefer to have all "mountables" in one folder) I agree to

well... are you sure? ;-)
As Pawel already pointed out, in EGEE there are hundreds of SRMs, and a 
handfull of LFCs... yes we can have subfolders, but ideally i would tend 
to a relatively flat structure with ans little "artificial" 
classifications as possible, like the "Services" one which doesn't really 
add extra information for the user. 

> > It is then of course responsabillity of
> > the <middleware>.info plugins to arrange the different
> > servers/services/resources into the information tree (glue schema in
> > principle)
>
> Sorry, but here you are wrong. It is definitely the core's
> responsibility to group resources in the info/VO tree! If it would be
> the middleware's responsibility we would loose the common interface to
> all middlewares!

no, that is why i said in the <middleware>.info plugins
The MW itself decides what kind of service a given element in the 
infosystem is (ie, only the glite implementation can know that 
   GlueServiceType: org.glite.wms.WMProxy
is actually a "Job submission service", and that both 
   GlueServiceType: lcg-file-catalog
   GlueServiceType: lcg-local-file-catalog
are exactly the same functionality-wise!)

Core can say
   GlueCE -> IGridComputing
   GlueSE -> IGridStorage
   GlueService -> IGridService
but that would be all if the infoservice doesn't provide a _MW_independent_ 
service tag...  the ones that glite publishes in the infosystem are 
completely non-standarized (see the three ones in my example above!)

This is what we currently have in "core" and it is _not_ MW-independent, 
see the methods
     GlueService#isSupported()   and
     GlueService#processCreamService() 
in the .info plugin if you don't believe me! ;-)
(cream is glite stuff!... well perhaps when BES support comes...)

All this 
   if type = "some-mw-dependent-string" 
stuff _must_ be done by the <MW>.info plugins. That is to say, the 
<MW>.info  plugin has to encode the "knowledge" if a specific MW service 
is supported and how, ie, how it does "plug" into geclipse


> > where the Type is simply taken from Glue?  (ie, the glite.info plugin
> > could
> > take care of reading the GlueServiceType published by the service, and
> > convert it into a human readable category,
> >
> >   GlueServiceType: org.glite.wms.WMProxy -> Type="Job
> > SubmissionServices" GlueServiceType: lcg-local-file-catalog       ->
> > Type="File Catalogs"
>
> This is completely middleware-dependent! So no way!

no, because the proposal is that the <_MW_>.info plugin takes care of 
that!!  Currently the MW-independency _IS_  broken in core (e.g.info), see 
the GlueService#isSupported() method!!

> Please do not think too much about the gLite users. gLite is only our
> first exemplary implementation. And furthermore as I again saw at the
> user forum our users are most often not really familiar with gLite. They
> do know nothing about a SRM service. But on the other hand they love the
> mount action. So why not providing shortcut-access to the data access
> services by showing them as mountable storages. Once we decided to call
> it storage rather than SE in order to make it as general as possible. I
> know that LFC is a catalogue, but the ordinary user does recognise it as
> storage (which is in fact somehow the purpose why LFC was developed, to
> give the user easy access to storage by providing "virtual storage
> nodes/folders", right?!)

yes, you are right!
Actually.... the user shouldn't need to mount anything either, but when he 
creates the GridProject with a given VO, the infosystem could be used to 
automatically mount "the" File-Catalog... ;-)
or even a menu pops up asking it we should mount 
lfc1/lfc2/.../srm1/srm2/... for him ;-)

> We did something for GRIA which is even more ugly, making the data
> service both a service AND storage ;-) But it works, looks nice and is
> user-friendly :)

that should be always the case in the future, see Glue 2.0 ;-)

> Which is why we need it in storage ;-)

well... after this lengthy email and discussion... i think i don't care 
anymore then hahahaha

No, jokes aside, if we put it in storage it has to be in a subfolder (think 
Pawel's n-hundreds SRM's in the dteam VO)
  VO -> Storage -> { SRM, File-Catalogs } -> individual services

if it is in Services it would probably have to be 

  VO -> Services -> { JobSub.S, File-Catalogs, *} -> individual services
  VO -> Storage  -> all SRMs flat here

so it wouldn't be a big difference either. Only the direct association 
  "Storage" <-> EFS-mountable
would be missing

> We want to build a middleware independent model, right? That means to
> have an abstraction layer that gives us the possibility to map
> middleware functionalities to our model. We furthermore promised to

sure, as long as there _is_ functionality to map ;-)
I would fully agree to have  
   IGridFileCatalog  if we would have some specific method(s) for it 
(registerFile, getReplicaURI,...), just like we have
   IGridJobSubmissionService  with  submitJob()


> The original meaning of IGridResource was that Grid resources are
> non-local entities. Since a job is running on the Grid it is non-local.
> I know, it does not make too much sense and I personally have no string
> opinion on making it a Grid resource or not.

ok

> within IGridResource. The ResourceType could be a class of concatenated
> subtypes like for instance
>
> storage
> storage.srm
> computing
> computing.cream
> service.info
> service.job.submission
> service.job.status
> service.data.srm

yes! +1 for that

> And maybe we should then think about what really is an IGridResource.

"resource" and "service" can be interpreted in many ways, as Ken just 
pointed out...  but this email is way too long already... sorry ;-P

Cheers, A