<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>Hello,</div><div><br></div><div>I believe you should trade-off between memory utilization and performance.</div><div>Yes, lists are faster then binary but requires more memory.</div><div><br></div><div>I have figure out that binary parsing might be efficient if you scan it by changing index and capture tokens instead of single byte.</div><div>, e.g.:</div><div><br></div><div><span class="nf" style="margin: 0px; padding: 0px; border: 0px; color: rgb(153, 0, 0); font-weight: bold; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">parse</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">(</span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">In</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">,</span><span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); display: inline !important; float: none; "> </span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">Pos</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">,</span><span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); display: inline !important; float: none; "> </span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">Len</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">,</span><span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); display: inline !important; float: none; "> </span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">Line</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">,</span><span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); display: inline !important; float: none; "> </span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">Fun</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">,</span><span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); display: inline !important; float: none; "> </span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">Acc0</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Courier, monospace; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: pre; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); ">) -></span></div><div><pre style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; font-size: 12px; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-variant: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div class="line" id="LC76" style="font-weight: normal; color: rgb(51, 51, 51); font-style: normal; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 10px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; "><span class="k" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">case</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">In</span> <span class="k" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">of</span></div><div class="line" id="LC77" style="font-weight: normal; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 10px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; "><font class="Apple-style-span" color="#333333"> </font><span class="Apple-style-span" style="color: rgb(153, 153, 136); font-style: italic; ">% end of field</span></div><pre style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; font-size: 12px; font-family: Consolas, 'Liberation Mono', Courier, monospace; font-variant: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div class="line" id="LC81" style="font-weight: normal; color: rgb(51, 51, 51); font-style: normal; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 10px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; "> <span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; "><<</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">_:</span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Pos</span><span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">/</span><span class="n" style="margin: 0px; padding: 0px; border: 0px; ">binary</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Tkn</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">:</span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Len</span><span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">/</span><span class="n" style="margin: 0px; padding: 0px; border: 0px; ">binary</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">?</span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">FIELD_BY</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="p" style="margin: 0px; padding: 0px; border: 0px; ">_</span><span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">/</span><span class="n" style="margin: 0px; padding: 0px; border: 0px; ">binary</span><span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">>></span> <span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">-></span></div><div class="line" id="LC82" style="font-weight: normal; color: rgb(51, 51, 51); font-style: normal; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 10px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; "> <span class="n" style="margin: 0px; padding: 0px; border: 0px; ">parse</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">(</span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">In</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Pos</span> <span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">+</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Len</span> <span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">+</span> <span class="mi" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 153, 153); ">1</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="mi" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 153, 153); ">0</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="p" style="margin: 0px; padding: 0px; border: 0px; ">[</span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Tkn</span> <span class="p" style="margin: 0px; padding: 0px; border: 0px; ">|</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Line</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">],</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Fun</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Acc0</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">);</span></div><div class="line" id="LC82" style="font-weight: normal; color: rgb(51, 51, 51); font-style: normal; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 10px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; "><span class="p" style="margin: 0px; padding: 0px; border: 0px; "> ...</span></div></pre><div style="font-weight: normal; "><pre style="margin: 0px; padding: 0px; border: 0px; font-size: 12px; font-family: Consolas, 'Liberation Mono', Courier, monospace; color: rgb(51, 51, 51); font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 16px; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div class="line" id="LC92" style="margin: 0px; padding: 0px 0px 0px 10px; border: 0px; "><span class="p" style="margin: 0px; padding: 0px; border: 0px; "> _</span> <span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">-></span></div><div class="line" id="LC93" style="margin: 0px; padding: 0px 0px 0px 10px; border: 0px; "> <span class="c" style="margin: 0px; padding: 0px; border: 0px; color: rgb(153, 153, 136); font-style: italic; ">% no match increase token</span></div><div class="line" id="LC94" style="margin: 0px; padding: 0px 0px 0px 10px; border: 0px; "> <span class="n" style="margin: 0px; padding: 0px; border: 0px; ">parse</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">(</span><span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">In</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Pos</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Len</span> <span class="o" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">+</span> <span class="mi" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 153, 153); ">1</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Line</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Fun</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">,</span> <span class="nv" style="margin: 0px; padding: 0px; border: 0px; color: rgb(0, 128, 128); ">Acc0</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">)</span></div><div class="line" id="LC95" style="margin: 0px; padding: 0px 0px 0px 10px; border: 0px; "><span class="k" style="margin: 0px; padding: 0px; border: 0px; font-weight: bold; ">end</span><span class="p" style="margin: 0px; padding: 0px; border: 0px; ">;</span></div><div><br></div></pre></div></pre></div><div>- Dmitry</div><br><div><div>On Oct 23, 2012, at 6:11 PM, Erik Pearson wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">I used a simple thing like:<div><br></div><div><div><div>test_iter(Mod, Fun, Args, Iters) -></div><div> test_iter(Mod, Fun, Args, now(), Iters, Iters).</div><div><br></div><div>test_iter(_Mod, _Fun, _Args, Start, Iters, 0) -></div>
<div> Iters/(timer:now_diff(now(), Start)/1000000);</div><div><br></div><div>test_iter(Mod, Fun, Args, Start, Iters, CountDown) -></div><div> erlang:apply(Mod, Fun, Args),</div><div> test_iter(Mod, Fun, Args, Start, Iters, CountDown-1).</div>
</div><div><br></div><div>And was just looking at total iterations per sec. I would just repeat this several times until I found a relatively stable reading. Sure, there was variation, but I'm looking to simulate the pattern I use in this library, which is iterating through and copying many small bits of text (json keys and values.)</div>
<div><br></div><div>Since there was such a large difference in overall performance (string being more than twice as fast), I didn't feel the need to be more precise before posing the question.</div><div><br></div><div>
E.g.<br><br></div><div><div>20> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).</div><div>232018.5614849188</div><div>21> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).</div>
<div>224870.69934787496</div><div>22> json:test_iter(json, test_bin_copy, [<<"hi, my name is erik">>], 1000).</div><div>226193.16896629723</div><div>23> </div><div>23> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).</div>
<div>650732.3993154295</div><div>24> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).</div><div>608076.4716970806</div><div>25> json:test_iter(json, test_string_copy, ["hi, my name is erik"], 100000).</div>
<div>567359.7912115968</div></div><div><br></div><div>Many of the follow up observations and questions have been stimulating, so I'm now interested as well in a more detailed analysis.</div><div><br></div><div>However, in the end what I'm looking at is the differences in performance between list and binary string processing under what I believe is idiomatic Erlang for such problems.</div>
<div><br></div><div>Thanks,</div><div><br></div><div>Erik.</div><br><div class="gmail_quote">On Tue, Oct 23, 2012 at 1:07 AM, Martynas Pumputis <span dir="ltr"><<a href="mailto:martynasp@gmail.com" target="_blank">martynasp@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Could you show the exact steps of your simulation? Binary version should be faster, because some extra memory allocation is avoided per each iteration and large binaries aren't being copied.<br>
<br>
Take a look at: <a href="http://www.erlang.org/doc/efficiency_guide/binaryhandling.html" target="_blank">http://www.erlang.org/doc/<u></u>efficiency_guide/<u></u>binaryhandling.html</a><br>
<br>
Martynas<div><div class="h5"><br>
<br>
On 10/23/2012 12:18 AM, Erik Pearson wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
Hi,<br>
I've read from advice given many years ago that processing binaries byte<br>
by byte (e.g. a recursive parser), performance is better using a list to<br>
accumulate the bytes, rather than using binary concatenation. So<br>
[B|Accum] rather than <<Accum/binary, B>>. There seems to be<br>
a consensus however, on the efficiency of Binaries compared to List<br>
strings.<br>
<br>
My own quick test, which was just to copy a list or binary element by<br>
element, showed much better performance for the list version. The test<br>
was basically to pass an arbitrary string or binary, and copy it some<br>
number of thousands of times, and output the complete copies per second.<br>
<br>
I tried list based accumulation for a binary, using binary destructuring<br>
in the function head, and that sped things up, but it was still slower<br>
than the equivalent list string copy.<br>
<br>
Are there any tips for binaries? Of is this not a good use case for<br>
binaries.<br>
<br>
test_bin_copy(Bin) -><br>
test_bin_copy(Bin, <<>>).<br>
test_bin_copy(<<>>, Accum) -><br>
Accum;<br>
test_bin_copy(<<Char, Rest/binary>>, Accum) -><br>
test_bin_copy(Rest, <<Accum/binary, Char>>).<br>
<br>
test_string_copy(Bin) -><br>
test_string_copy(Bin, []).<br>
test_string_copy([], Accum) -><br>
lists:reverse(Accum);<br>
test_string_copy([Char|Rest], Accum) -><br>
test_string_copy(Rest, [Char|Accum]).<br>
<br>
For what its worth this is part of a json module. The current practice<br>
in json libraries seems to favor binaries, so I assumed there were<br>
inherent performance advantages. I can imagine, e.g., that an empty<br>
binary would be stored as a modest sized buffer that would be appended<br>
in place until there was a need to expand it or copy (e.g. if an older<br>
version of it was being appended), and that operations on it would be<br>
fast compared to arbitrary consing (which is however highly optimized.)<br>
<br>
I think some of the favoritism for binaries in json libs is because it<br>
makes it easy to differentiate json strings (as erlang binaries) from<br>
json arrays (as erlang lists), but my implementation is using tagged<br>
tuples to contain each json value, so this is not a concern. Of course<br>
there are the memory concerns, but in my tests any memory concerns with<br>
list char size vs binary bytes is erased by the performance gains.<br>
<br>
I'm sure I've put my foot in my mouth at least once, but, anyway, advice<br>
appreciated.<br>
<br>
Thanks,<br>
Erik.<br>
<br>
<br></div></div>
______________________________<u></u>_________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/<u></u>listinfo/erlang-questions</a><br>
<br>
</blockquote>
<br>
</blockquote></div><br></div>
_______________________________________________<br>erlang-questions mailing list<br><a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>http://erlang.org/mailman/listinfo/erlang-questions<br></blockquote></div><br></body></html>