[erlang-questions] Storing images
Tue May 20 22:31:50 CEST 2014
I think you need to provide more data.
- how many images ("unknown" thousands is not very precise) - how big is
the meta data?
I note you say thousands not millions. Say unknown = 100 then you have 100K
100K images @ 100K per image = 10^10 Bytes = 10 GBytes (= tiny)
Assume the meta data is small say 100Bytes/image then the
meta data is 10^5 * 100 = 10^7 Bytes = 10 MB (= tiny)
If you have a machine with 1GB of memory this is 1% of total memory.
Make sure the meta data is small, ie no tags in it and possible
compressed and store the lot in RAM
you could use an ets table or a custom binary.
You can store a heck of a lot of meta data in a GB and a heck of a lot of
images in a TB
so I'd keep the meta data in RAM and the image data on disk.
If you intend to use a SSD you could keep everything on disk
Start with the simplest possible correct solution and measure. You'll be
amazed what you can do with ets tables and
a GB of RAM
On Mon, May 19, 2014 at 5:52 AM, Lloyd R. Prentice <lloyd@REDACTED>wrote:
> Suppose I anticipate unknown thousands of records, each containing
> meta-data associated with an image. Each image averages, say, 100KB.
> Retrieval is by key and queries on the meta data. Retrieval predominates
> over insert. Reliability and speed of access are high on the requirements
> I'm considering Riak. But...
> - Would it be best practice to store the each image in its associated
> record or would it be better to store the images separately in, say,
> directories or in a separate distributed file system such as leoFS?
> - Is there a better solution altogether?
> Many thanks,
> Sent from my iPad
> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions