[erlang-questions] beam.smp startup time regression

[Tuncer Ayaz]

On Fri, Apr 4, 2014 at 2:59 PM, Björn-Egil Dahlberg <egil@REDACTED> wrote:
> Most the time you measure has to do with file handling and loading
> modules. I believe there is around 75 files (modules) that loads
> during startup.

Recalling the code from the last time (2011) I touched escript.erl and
erl_prim_loader, it's a safe assumption to make that we won't be able
to exclude anything of significance out of the 75 files for
implementing a slim startup mode for escript.

> You can strip the system if you want faster load time and perhaps
> archives would be another choice.

Assuming, it would speed up load time, any reason why the default
install does not use stdlib-1.19.4.ez, ssl-5.5.3.ez etc, besides "we
didn't have a need for it"?

Also, maybe the code increase due to Bjorn's line number
implementation is also responsible for some of the time. And, no, I
wouldn't want to miss it :).

Node startup should be profiled, and given Erlang/OTP's primary use
case, there might be fruit reachable with a short ladder.

> Also, async threads delivers some overhead, you can disable them
> (don't do it).

Going from zero to ten kernel threads (spun up by default) is not
cheap, and the lowest I could get with R16B03-1 +A0 was "real 0.342s".

> On 2014-04-04 14:45, Tuncer Ayaz wrote:
>>
>> The recent HiPE and half-word-emu threads made me recheck a subjective
>> impression I had about the steady increase of beam.smp startup time
>> over several OTP releases.
>>
>> I used to build with --enable-native-libs, but the already not quick
>> enough startup time of beam.smp got even longer, so in default nodes I
>> use neither HiPE nor --enable-native-libs.
>>
>> Unscientifically and quickly measured average time to run
>> $ time erl -eval 'halt(0).'
>> with different Erlang/OTP releases (no native-libs at all):
>>
>> # R13B03
>> real    0m0.261s # min 0.254 max 0.264
>> user    0m0.143s
>> sys     0m0.020s
>>
>> # R14B04
>> real    0m0.293s # min 0.289 max 0.301
>> user    0m0.167s
>> sys     0m0.033s
>>
>> # R15B03-1
>> real    0m0.318s # min 0.315 max 0.323
>> user    0m0.203s
>> sys     0m0.002s
>>
>> # R16B03-1
>> real    0m0.373s # min 0.354 max 0.381
>> user    0m0.257s
>> sys     0m0.053s
>>
>> To be precise, R16B03-1 is the only beam.smp with halfword-emu, but
>> the last time I measured this some time back, R16 without halfword-emu
>> on i386 was also considerably slower.
>>
>> I believe the R15 increase is due to Raimo's flushing fixes and that's
>> the right thing to do, but it steadily increases with each release and
>> especially so in R16.
>>
>> Personally, I think this is a regression and makes using escripts less
>> fun than it used to be.
>>
>> A proper measurement would build beam.smp with the same configure
>> flags and probably even use git-bisect to track down commits. I
>> don't have time for this right now, but decreasing startup time
>> could be a fun project :).