[erlang-questions] Rough thought on a P2P package distribution model for Erlang

[Thomas Lindgren]

Twitter uses p2p to deploy their services: 
http://engineering.twitter.com/2010/07/murder-fast-datacenter-code-deploys.html
https://github.com/lg/murder


Nice judo, turning a big problem (that myriad of hosts) into a big advantage.

Best,

Thomas



>________________________________
>From: jm <jeffm@REDACTED>
>To: Jesper Louis Andersen <jesper.louis.andersen@REDACTED>
>Cc: erlang-questions@REDACTED
>Sent: Friday, September 23, 2011 6:35 AM
>Subject: Re: [erlang-questions] Rough thought on a P2P package distribution model for Erlang
>
>You assume speed is the only reason to use P2P. I'm more concerned with maintaining up to date package repositories, redundancy, and security. MIrrors that use ftp/http/rsync/etc have a tendency  to fall out of date if not well maintained. These mirrors also require that up to date lists be maintained so the users (human or machine) can find active ones and how do you distribute those? CDNs do this through things like redirection and dynamic DNS records. The method that is used behind the scenes to distribute content would vary from provider to provider. P2P is just one possible technology (and that is all I'm claiming). Most likely, these providers use a mix of technologies or a hybrid approach for internal content distribution.
>
>For example,
>http://goanna.cs.rmit.edu.au/~xiaodong/mbc/Theses/jaison-minorThesis.pdf mentions P2P as one technology.
>
>and page 340 and page 341 of this paper has tables which list uses of P2P one sub table is devoted to "Content Publishing and Storage Systems"
>A Survey of Peer-to-Peer Content Distribution Technologies
>STEPHANOS ANDROUTSELLIS-THEOTOKIS AND DIOMIDIS SPINELLIS
>Athens University of Economics and Business
>
>Jeff.
>
>On 21/09/11 10:25 PM, Jesper Louis Andersen wrote:
>> Mainly because there is no need to do it.
>> 
>> Data distribution systems like BitTorrent has the specific distinct
>> advantage that it can transfer data at high bandwidths, even if the
>> initial source is highly bandwidth constrained in its upstream.
>> Upstream bandwidth scales with demand in a BT network. Hence, the
>> applicability of BitTorrent (and like) protocols hinges on a need to
>> thwart an upstream bandwidth constraint. Example: You are Facebook and
>> need a 1 gigabyte image distributed quickly to 50.000 nodes from a
>> single deploy machine.
>> 
>> But as soon as there is demand for a package distribution, and the
>> demand is high enough, mirrors form and mirrors have ample amounts of
>> upstream bandwidth available. Far more than what is needed. Thus, the
>> additional complexity of adding BitTorrent into the mix isn't needed.
>> 
>> Add that HTTP transport is well-known and simple. You can say it is a
>> locally extreme value which is currently good enough. The proof is
>> Content Delivery Networks (CDNs) which does not use BitTorrent to
>> distribute content.
>> 
>> 
>
>_______________________________________________
>erlang-questions mailing list
>erlang-questions@REDACTED
>http://erlang.org/mailman/listinfo/erlang-questions
>
>
>