[erlang-questions] RE: DETS Problems

[Evans, Matthew]

It was actually worse than I thought.

It seems (initial glance) that DETS doesn't like the large elements I was inserting.

The table had 3 records of 3-5K in size, and about 700 of 300 bytes in size.

It was constantly failing to find the large elements, if I looked up a small element first.

Moving the large elements to a separate table seems to fix the problem.

I'm going to do more analysis of this, we also use SSD/flash as the storage device, maybe that is causing problems.

________________________________
From: Ahmed Omar [mailto:spawn.think@REDACTED]
Sent: Tuesday, November 02, 2010 3:46 PM
To: Evans, Matthew
Cc: erlang-questions
Subject: Re: [erlang-questions] RE: DETS Problems

Sorry to understand more, can you provide the options you are using for the table?
what do u do with ets:to_dets/2 exactly?
please also note that :
" The Dets table is emptied before the objects are inserted"
http://www.erlang.org/doc/man/ets.html#to_dets-2


On Tue, Nov 2, 2010 at 7:25 PM, rgowka1 <rgowka1@REDACTED<mailto:rgowka1@REDACTED>> wrote:
"Dets tables must be opened before they can be updated or read, and
when finished they must be properly closed. If a table has not been
properly closed, Dets will automatically repair the table. This can
take a substantial time if the table is large. A Dets table is closed
when the process which opened the table terminates. If several Erlang
processes (users) open the same Dets table, they will share the table.
The table is properly closed when all users have either terminated or
closed the table. Dets tables are not properly closed if the Erlang
runtime system is terminated abnormally."

I did play around with dets with the same sized records as you.
According to the above blurb and my observations, the corruption is
usually not closing the dets in all the processes..


On Tue, Nov 2, 2010 at 2:10 PM, Evans, Matthew <mevans@REDACTED<mailto:mevans@REDACTED>> wrote:
> I should add, that I do ets:insert/2 followed by a ets:to_dets/2.
>
> Some of the records are about 4K in size.
>
> -----Original Message-----
> From: erlang-questions@REDACTED<mailto:erlang-questions@REDACTED> [mailto:erlang-questions@REDACTED<mailto:erlang-questions@REDACTED>] On Behalf Of Evans, Matthew
> Sent: Tuesday, November 02, 2010 1:48 PM
> To: erlang-questions
> Subject: [erlang-questions] DETS Problems
> Importance: High
>
> List,
>
> I'm using a dets table to save reasonably large records (1K or there abouts).
>
> The following sequence is applied to a table:
>
> dets:open_file/2/
> dets:insert/2 (a few times).
> ets:to_dets/2
> dets:close/1
>
>
> Sometime later I reopen the table with dets:open_file/1
>
> The table is then corrupted. Reads produce the following error:
>
> {error,{{bad_object,eval_work_list},
>        "/content/c2idx/5/95/10b12417-4629-404a-8361-68bd3de77d90.dat"}}
>
> What causes that error?
>
> Thanks
>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org<http://erlang.org> mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED<mailto:erlang-questions-unsubscribe@REDACTED>
>
>

________________________________________________________________
erlang-questions (at) erlang.org<http://erlang.org> mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED<mailto:erlang-questions-unsubscribe@REDACTED>



--
Best Regards,
- Ahmed Omar
http://nl.linkedin.com/in/adiaa
Follow me on twitter
@spawn_think<http://twitter.com/#!/spawn_think>