RE: [smila-dev] BinaryStorage service concept

[Thomas Menzel <tmenzel@xxxxxxx>]

hi,

i opend bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=251546
for tracking the feedback on the concept rather than editing the discussion page, as I find this cumbersome.

I also left my first remarks...

Kind regards
Thomas Menzel @ brox IT-Solutions GmbH


-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Marius Cimpean
Sent: Montag, 20. Oktober 2008 12:55
To: Smila project developer mailing list
Subject: [smila-dev] BinaryStorage service concept

Hi all,

The Binary storage concept page is available -  please have a look and 
submit your remarks. We can discuss the binary storage based on this page

http://wiki.eclipse.org/SMILA/Project_Concepts/Binary_Storage

Best Regards,
Marius


----- Original Message ----- 
From: "Thomas Menzel" <tmenzel@xxxxxxx>
To: "Smila project developer mailing list" <smila-dev@xxxxxxxxxxx>
Sent: Monday, October 13, 2008 1:16 PM
Subject: RE: [smila-dev] Re: Problems with BinStorage


hi jürgen,

very nice, if u had not written this, i would have. thx

from what I understood, VFS supports already diff. storage implementations 
for diff. *mountpoints*. so, with VFS it's already possible to have a DB, 
distributed FS, local FS at the same time, albeit in diff. namespaces.

and yes, I think it is not part of the client to say directly, I want DB 
storage or local, it just would have to know the NS it is config'ed for and 
hence all this moves into the config/admin realm. he then has to decide what 
is best in what situation.

also: having followed the discussion so far, I think it is OK to say that at 
this time it makes more sense to write the BinStorage API such that it fits 
for the current use case, which is that the client doesn't need control of 
folders (or am I wrong here?).

an interface for FS-like distributed storage we can add later IMO. and I 
also have come to think that this could actually go into a new API that is 
different or just complementary to the BinStorage API.

Kind regards
Thomas Menzel @ brox IT-Solutions GmbH


PS: should we open a bug for this discussion as well? or do u prefer keeping 
this on the dev list?

-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] 
On Behalf Of Juergen.Schumacher@xxxxxxxxxxx
Sent: Montag, 13. Oktober 2008 12:02
To: smila-dev@xxxxxxxxxxx
Subject: RE: [smila-dev] Re: Problems with BinStorage

Hi,

It's not really clear to me, what this discussion is about currently (API? 
backend?), so I'll try to sum up my point of view on the complete package, 
and hopefully we'll get some answers from it (-;

The purpose of BinStorage is to store millions (or billions) of (possibly 
large) binary objects and make them available to a relatively large number 
of record processors running distributed in a "cluster" of computers. 
Currently I do not know what kind of backend might be the best to match 
these requirements, I rather think that it should be exchangeable in order 
to be able to match different requirements: In one application it could be 
simply a shared file system, because performance is sufficient and 
administration is easy, in other applications distributed file systems might 
be necessary, or some database technology or whatever. I don't think that we 
can decide this now for good. Thus the API must be as generic as possible to 
be implementable on any kind of backend.
And the details of the storage backend must not be of relevance to a client: 
One blackboard just writes the attachments of a record to bin storage, 
another one retrieves it again. They should not have to care about where to 
put the objects in a hierarchy for good performance, it's the task of 
BinStorage to create such hierarchies internally if a flat storage is not 
sufficient.

On the other side, I'm not against having a BinStorage API that enables 
different kinds of clients to use different "namespaces" in BinStorage to 
separate their data. These namespaces could be hierarchical: a "file system" 
metaphor behind the API might be helpful for developers using BinStorage, 
because most users are accustomed to it, even if the actual storage has 
nothing to with a file system.  On the other hand this might enable the 
configurator/administrator of BinStorage to distribute partitions of data to 
different storage mediums (just dreaming now ;-) for better performance. But 
BinStorage must not rely only on structures provided by the clients, but 
must organize the stored data for optimal performance even if a client does 
not provide any structure at all.

Hopefully this has not increased the confusion even more (-;

Yours,
Juergen.
_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev
_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev



_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev