[erlang-questions] Reliable file transfer?

[Mikael Pettersson]

On Wed, May 29, 2019 at 9:41 PM Lloyd R. Prentice <lloyd@REDACTED> wrote:
>
> Hello,
>
> I’m evaluating https://www.rsync.net/ as a possible backup/archiving service for Erlang dets and Mnesia data.
>
> rsync.net offers replicated data store on ZFS with frequent snapshots at an attractive price. Data can be transferred via rsync, SFTP, etc.
>
> My thought is to create a data management app that would automate the backup/archive/restore functions. But I don’t understand the data transfer options available in Erlang well enough to implement.
>
> The options I see are:
>
> — calling rsync via os:cmd/1 — seems simple enough but I’m breaking out of Erlang
> — ssh-sftp — just don’t understand how to use this library
> — File Transfer Service — docs are totally confusing????
>
> One would think that this is an every day plain vanilla kind of problem. But there’s scant guidance on the web.
>
> Can some kind soul point me in the right direction?

We're using os:cmd and rsync/scp/ssh to transfer our DB backups to
internal backup servers,  rsync in particular because it enables
incremental transfers of our leveldb snapshots.  Stuff is also pushed
to S3, but that's done outside of the main Erlang transaction servers.
(Erlang-based micro-services classify and redistribute event-based
data, while Unix cron jobs transfer plain files.)

I don't think we ever used the other options you mention.