Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [smila-dev] binstorage redesign
hm, i thought that binstorage was aimed to be just storage for binary objects and not virtual filesystem; is there any requirement about virtual filesystem? Regarding performance aspect, binstorage should decide internally how to keep files most effectively and it should be able to organize them into hierarchy if needed/distribute in cluster etc. 

Thomas Menzel wrote:
well, my thinking is that we provide a virtual file system as a binary storage. 
hence a client (not now, but it is thought of that the bin storage is an infrastructure item that may offer his service to other clients as well) may want/need to organize his data hierarchically, as if in normal file system.

if that is not needed for now in BS then we can limit the impl. currently to just offer a hashset-style storage but the API should support hierarchical (ie tree) structures and an FS is the most natural IMO.

we may also name the child just flat bin storage and later do a hierarchical bin storage if u like that better.
there is also most likely a performance aspect to full flat vs. hierarchical, as the lookup will be faster in a tree structure. that is not noticeable now with few items but when u have millions it will. also: when distributing the data in a cluster, nodes in a tree are a very welcome entity that can be mapped directly a hardware nodes. 

all these are details for the (near) future but need to taken into consideration.

Kind regards
Thomas Menzel @ brox IT-Solutions GmbH


-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Ivan Churkin
Sent: Dienstag, 30. September 2008 09:04
To: Smila project developer mailing list
Subject: Re: [smila-dev] binstorage redesign

I completely agree with.
Imho, binstorage API should be very simple. I can imagine that some customer will want to use database for storing binary data. As I understand there is only one problem - some text miner engine have to get real file access to extract data. 

--
Ivan



Dmitry Hazin wrote:
Why do you think that that binstorage should mimic the file system? What is the reason for that? The simplest will be just manage files in binstorage by Id (ie by name), and binstorage should decide how to store files physically, in file system or using any other way. So I don't see any the reason why binstorage should mimic filesystem, please explain your point. 
_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev


_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev
------------------------------------------------------------------------ 

_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev

Back to the top