[erlang-questions] [ANN] Silly benchmarking

[Fred Hebert]

A rather simple trick to simulate load is to have a few busy-looping
processes on the VM, and give it a higher priority than the rest. It
should eat time more than anything else and make it look like there's a
lot of contention for schedulers, with reduced execution time for what
you want to measure.

I've used it a few times, although I am not 100% sure it's super
representative o a real system being under load.

On 04/30, Garrett Smith wrote:
> You have a point. Though "load" would be pretty arbitrary -- I could
> see some load-xxx.escript files that did terrible things while the
> other scripts were run. But it would depend on what else is running on
> your system. And your system architecture. And your system hardware.
> Waiiit a minute, this just got real complicated!
> 
> So, don't make decisions unless you have reasonably complete
> information, whatever that means :)
> 
> On Tue, Apr 30, 2013 at 12:06 PM, Loïc Hoguin <essen@REDACTED> wrote:
> > I'm sure *you* know these results might not be true for systems under load,
> > but others might not. Not having both could cause more harm than good, and
> > would not be interesting in that sense.
> >
> >
> > On 04/30/2013 07:03 PM, Garrett Smith wrote:
> >>
> >> Not very interesting to you. Of course you can write whatever you
> >> like. For *me* I was curious about some relative performance
> >> characteristics. No religion here.
> >>
> >> On Tue, Apr 30, 2013 at 11:59 AM, Loïc Hoguin <essen@REDACTED> wrote:
> >>>
> >>> It's not very interesting unless the numbers are also available for
> >>> systems
> >>> under load.
> >>>
> >>>
> >>> On 04/30/2013 06:57 PM, Jeremy Ong wrote:
> >>>>
> >>>>
> >>>> I'd be very interested if we got a wiki going on one of these projects
> >>>> with community updated numbers of various benchmark runs.
> >>>>
> >>>> For example:
> >>>>
> >>>> keylists vs orddicts vs dicts
> >>>> fold vs recursion
> >>>> fibonacci
> >>>> all the other various standard benchmarks
> >>>>
> >>>> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith <g@REDACTED> wrote:
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog <mjtruog@REDACTED>
> >>>>> wrote:
> >>>>>>
> >>>>>>
> >>>>>> On 04/30/2013 06:44 AM, Garrett Smith wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> This is not an announcement of anything -- but [ANN] seems to flag
> >>>>>>> "something I can maybe use" which does apply in this case :)
> >>>>>>>
> >>>>>>> Occasionally I wonder, "what's faster"? It's not often, but it
> >>>>>>> happens.
> >>>>>>>
> >>>>>>> I've found the best way to answer this is to measure things.
> >>>>>>>
> >>>>>>> So I have this silly project:
> >>>>>>>
> >>>>>>> https://github.com/gar1t/erlang-bench
> >>>>>>>
> >>>>>>> It's not rigorous but it's simple and I can experiment quickly with
> >>>>>>> different implementations. My goal is just to get a sense of things
> >>>>>>> --
> >>>>>>> not to formally prove anything.
> >>>>>>>
> >>>>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it
> >>>>>>> may provide value as a distributed repository for what people are
> >>>>>>> interested in. As it's in github there's no ownership -- please feel
> >>>>>>> free to fork and use for your own concerns!
> >>>>>>>
> >>>>>> You might want to look at erlbench here
> >>>>>> https://github.com/okeuday/erlbench since it has the same basic
> >>>>>> purpose, and
> >>>>>> allows you to use different compilation methods now (through the
> >>>>>> makefile
> >>>>>> specifying an optimization level).  The erlbench project is also
> >>>>>> ad-hoc, but
> >>>>>> it has been enough to produce results in the past.
> >>>>>>
> >>>>>> The other option is trying to use basho_bench here
> >>>>>> https://github.com/basho/basho_bench, if you are testing key/value
> >>>>>> storage.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is
> >>>>> nothing more than escript files with a 10 line include file.
> >>>>>
> >>>>> I'm an extraordinarily lazy person :)
> >>>>>
> >>>>> Though seriously, thanks for the references. If I was more concerned
> >>>>> about benchmark integrity, those might be good options -- but this is
> >>>>> just a sniff test approach to satisfy my curiosity about various
> >>>>> topics.
> >>>>>
> >>>>> Garrett
> >>>>> _______________________________________________
> >>>>> erlang-questions mailing list
> >>>>> erlang-questions@REDACTED
> >>>>> http://erlang.org/mailman/listinfo/erlang-questions
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> erlang-questions mailing list
> >>>> erlang-questions@REDACTED
> >>>> http://erlang.org/mailman/listinfo/erlang-questions
> >>>>
> >>>
> >>>
> >>> --
> >>> Loďc Hoguin
> >>> Erlang Cowboy
> >>> Nine Nines
> >>> http://ninenines.eu
> >
> >
> >
> > --
> > Loïc Hoguin
> >
> > Erlang Cowboy
> > Nine Nines
> > http://ninenines.eu
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions