[erlang-questions] Storing images

Tue May 20 23:13:23 CEST 2014

Thank you, Joe,

I get carried away with all the choices. The simplicity and logic of this approach is comforting.

All the best,


-----Original Message-----
From: "Joe Armstrong" <erlang@REDACTED>
Sent: Tuesday, May 20, 2014 4:31pm
To: "Lloyd R. Prentice" <lloyd@REDACTED>
Cc: "Erlang-Questions Questions" <erlang-questions@REDACTED>
Subject: Re: [erlang-questions] Storing images

I think you need to provide more data.
   - how many images ("unknown" thousands is not very precise) - how big is
the meta data?

I note you say thousands not millions. Say unknown = 100 then you have 100K

100K images @ 100K per image = 10^10 Bytes = 10 GBytes (= tiny)

Assume the meta data is small say 100Bytes/image then the
meta data is 10^5 * 100 = 10^7 Bytes = 10 MB (= tiny)

If you have a machine with 1GB of memory this is 1% of total memory.


   Make sure the meta data is small, ie no tags in it and possible
compressed and store the lot in RAM
   you could use an ets table or a custom binary.

You can store a heck of a lot of meta data in a GB and a heck of a lot of
images in a TB
so I'd keep the meta data in RAM and the image data on disk.

If you intend to use a SSD you could keep everything on disk

Start with the simplest possible correct solution and measure. You'll be
amazed what you can do with ets tables and
a GB of RAM



On Mon, May 19, 2014 at 5:52 AM, Lloyd R. Prentice <lloyd@REDACTED>wrote:

> Hello,
> Suppose I anticipate unknown thousands of records, each containing
> meta-data associated with an image. Each image averages, say, 100KB.
> Retrieval is by key and queries on the meta data. Retrieval predominates
> over insert. Reliability and speed of access are high on the requirements
> list.
> I'm considering Riak. But...
> Questions:
> - Would it be best practice to store the each image in its associated
> record or would it be better to store the images separately in, say,
> directories or in a separate distributed file system such as leoFS?
> - Is there a better solution altogether?
> Many thanks,
> Sent from my iPad
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

More information about the erlang-questions mailing list