[erlang-questions] What database can I use with erlang to store 100M records and select single one in 0.1s

[lenz]

hi,
searching for solutions to this type of problem is a challenge and there are
many ways to solve it. give this site (http://highscalability.com) a
thorough read and you might find something.
again, i am with jan, try CouchDB and it might pay off to use a unix based
machine for stuff like that. i think you are trying to trick your OS to do
things it is not meant for. not going into flame wars about windows here but
a basic linux setup is really done in no time and gives you way more control
for an application that is "non standard" :-)

cheers
lenz

On Wed, Feb 18, 2009 at 10:26 AM, lenz <norbu09@REDACTED> wrote:

> hi,
> searching for solutions to this type of problem is a challenge and there
> are many ways to solve it. give this site (http://highscalability.com) a
> thorough read and you might find something.
> again, i am with jan, try CouchDB and it might pay off to use a unix based
> machine for stuff like that. i think you are trying to trick your OS to do
> things it is not meant for. not going into flame wars about windows here but
> a basic linux setup is really done in no time and gives you way more control
> for an application that is "non standard" :-)
>
> cheers
> lenz
>
> On Wed, Feb 18, 2009 at 7:42 AM, Joe Armstrong <erlang@REDACTED> wrote:
>
>> What are the types of the keys and records? Is there are systematic
>> structure in
>> the keys? are the records fixed or variable lengths? How about using a
>> SSD?
>>
>> /Joe Armstrong
>>
>>
>> 2009/2/17 Scott Zhang <getyourcontacts@REDACTED>:
>> > Hi.
>> >
>> > I opened a question about mnesia days ago. Now I am facing another
>> question.
>> >
>> > I am going to scrape large amount of data and put them into database.
>> > Previously, my program was written in .NET/C# and saved the records into
>> > SqlServer, after 10 days running, I have 63M records in database which
>> make
>> > SqlServer really slow, select single record need more than 1 minute
>> which is
>> > unacceptable.
>> >
>> > And days ago, I tried to save 7M records in mnesia with table type=
>> > disc_only_copies. I tested , select single one from that talbe need more
>> > than 20 seconds which is unacceptable too. ( I don't want to conclude
>> > mnesia's performance is worse than SqlServer.)
>> >
>> > Lenz suggested me to use couchDB, just checked, couchDB depends on too
>> much
>> > things and these depends make it hard to configure on window system.
>> >
>> > So here, does anyone has a good suggestion for a database can fit
>> 63M(this
>> > time may get 2billion) and select one node within 1 sec?
>> >
>> > Thanks.
>> >
>> > Regards.
>> >
>> > Scott
>> >
>> > _______________________________________________
>> > erlang-questions mailing list
>> > erlang-questions@REDACTED
>> > http://www.erlang.org/mailman/listinfo/erlang-questions
>> >
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
>
>
>
> --
> iWantMyName.com
> painless domain registration (finally)
>



-- 
iWantMyName.com
painless domain registration (finally)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20090218/42b9739c/attachment.htm>