[erlang-questions] Storing images
Tue May 20 23:13:23 CEST 2014
Thank you, Joe,
I get carried away with all the choices. The simplicity and logic of this approach is comforting.
All the best,
From: "Joe Armstrong" <erlang@REDACTED>
Sent: Tuesday, May 20, 2014 4:31pm
To: "Lloyd R. Prentice" <lloyd@REDACTED>
Cc: "Erlang-Questions Questions" <erlang-questions@REDACTED>
Subject: Re: [erlang-questions] Storing images
I think you need to provide more data.
- how many images ("unknown" thousands is not very precise) - how big is
the meta data?
I note you say thousands not millions. Say unknown = 100 then you have 100K
100K images @ 100K per image = 10^10 Bytes = 10 GBytes (= tiny)
Assume the meta data is small say 100Bytes/image then the
meta data is 10^5 * 100 = 10^7 Bytes = 10 MB (= tiny)
If you have a machine with 1GB of memory this is 1% of total memory.
Make sure the meta data is small, ie no tags in it and possible
compressed and store the lot in RAM
you could use an ets table or a custom binary.
You can store a heck of a lot of meta data in a GB and a heck of a lot of
images in a TB
so I'd keep the meta data in RAM and the image data on disk.
If you intend to use a SSD you could keep everything on disk
Start with the simplest possible correct solution and measure. You'll be
amazed what you can do with ets tables and
a GB of RAM
On Mon, May 19, 2014 at 5:52 AM, Lloyd R. Prentice <lloyd@REDACTED>wrote:
> Suppose I anticipate unknown thousands of records, each containing
> meta-data associated with an image. Each image averages, say, 100KB.
> Retrieval is by key and queries on the meta data. Retrieval predominates
> over insert. Reliability and speed of access are high on the requirements
> I'm considering Riak. But...
> - Would it be best practice to store the each image in its associated
> record or would it be better to store the images separately in, say,
> directories or in a separate distributed file system such as leoFS?
> - Is there a better solution altogether?
> Many thanks,
> Sent from my iPad
> erlang-questions mailing list
More information about the erlang-questions