[erlang-questions] <DKIM> simple virtual file system in Erlang?

Joe Armstrong erlang@REDACTED
Mon Mar 27 21:16:08 CEST 2017


As regards your comments of performance - I would be extremely cautious about
predicting performance before any code is written and measured. This is the
one area where predictions can be wildly incorrect.

Any operating system worth the name caches large parts of the working set of the
file system in memory which makes predicting performance very difficult.

Even using a RAM file buffer might not help since the OS is itself
caching file blocks
in RAM - I'd go for the simplest and most beautiful design and then measure.

Cheers

/Joe

On Mon, Mar 27, 2017 at 11:07 AM, Marco Molteni
<marco.molteni@REDACTED> wrote:
> Hello all,
>
> thanks for the helpful answers, especially the idea of a two-level lookup, path to hash and hash to blob. This would also naturally take care of data deduplication. On the other hand, it would require to keep a reverse index to map a given checksum to all the paths pointing to it. Storing only the path, without notion of intermediate node, would also make very time consuming doing simple operations such as listing the contents of a given directory, I would have to lookup _all_ the paths... So the idea is nice, but it requires refinements :-)
>
> Now time to think a bit more about it.
>
> marco
>
>> On 26 Mar 2017, at 14:45, Marco Molteni <marco.molteni@REDACTED> wrote:
>>
>> Hello colleagues,
>>
>> I would like to start from the presentation "Build an FTP Server with Ranch in 30 Minutes" [1] and really build an FTP server, so at a certain point I need to hit the filesystem.
>>
>> Since the idea is still as a presentation (as opposed to building a production FTP server) I don't want to touch the real filesystem of the host, I want to use a as simple as possible virtual filesystem. It could be backed by DETS or Mnesia for example, or it could stay only in memory.
>>
>> On the other hand, it must still behave as a filesystem, that is, it must be hierarchical, so a direct mapping to a key/value store would not be enough. In the spirit of simplicity, I don't need any concept of read/write permission, I simply need a sort of graph with two types: inner nodes are directories, leaf nodes are files or empty directories.
>>
>> I am thinking to use DETS and somehow introduce a very simple intermediate layer that would offer the impression to be in a graph (each non-leaf node a directory) and map it to the DETS key/value API.
>>
>> Any suggestions?
>>
>> thanks
>> marco
>>
>> [1] https://ninenines.eu/articles/ranch-ftp/
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions



More information about the erlang-questions mailing list