[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[eclipse-incubator-e4-dev][resources] hadoop DFS

Hi Folks,

I've mentioned replication for distributed file systems in several previous notes, but I haven't said much about it. Rather than talk a lot about it myself, I thought I would point to an apache project called 'Hadoop Distributed FS' (HDFS) as it's based upon replication at the file block level...and is similar in approach to the Google File System (GFS).

One thought I've had (but not persued yet), is that perhaps an EFS implementation (with synchronous API) could be written that uses Hadoop DFS to implement EFS above. Although I'm not sure whether Hadoop can/could support all the operations that EFS has currently (e.g. notification)...not to mention other EFS API additions that are being considered for e4.

Scott

Hadoop Home Page: http://hadoop.apache.org/core/
Hadoop File System: http://wiki.apache.org/hadoop/DFS
HDFS arch and design: http://hadoop.apache.org/core/docs/current/hdfs_design.html