[erlang-questions] [Zotonic-Dev] zotonic CI deployment best practice for 2019

Tue Jan 15 19:22:05 CET 2019

Hi Marc,

Really helpful!

You're the best.

Lloyd

-----Original Message-----
From: "'Marc Worrell' via Zotonic developers" <zotonic-developers@REDACTED>
Sent: Tuesday, January 15, 2019 3:39am
To: "'Marc Worrell' via Zotonic developers" <zotonic-developers@REDACTED>
Cc: "erlang-questions@REDACTED" <erlang-questions@REDACTED>
Subject: Re: [Zotonic-Dev] zotonic CI deployment best practice for 2019

Hi Lloyd (and rest of list),
For Zotonic I have seen two setups:
1. Only Zotonic (Erlang)  (single machine)
With this setup external port 80 is redirected (using iptables prerouting rules) to 127.0.0.1:8000.
And port 443 to 8433. Erlang is directly handling all (SSL) traffic.
Same is done with the SMTP port 25, which is usually mapped to 2525.
2. Using a proxy (one or more machines)
In this haproxy or nginx are used to terminate the SSL connections and proxy requests to Zotonic/Erlang.
The Zotonic node is either running locally or on some other host in local network.
In this setup it is also common to have a mail server running to forward incoming email to the Zotonic/Erlang node.
Setup 1 (direct) is the easiest for small single-server operations.
Setup 2 (proxy) is when you want to have multiple machines and a local network.
We have setup 1 running on a €50/month VPS, handling monthly traffic of 1.5+ TB (~2M monthly visitors).
Another company is running setup 1 on dedicated hardware with more than 1M hourly requests.
(That server is mostly idle…)
So for most (98%?) sites the simple setup is actually very realistic.
You just need to be sure that you have a good backup scheme, as it is a single machine.
I know of some people that are working on a “lukewarm” failover setup.
With the single VPS server solution we didn’t have any significant down time in the last years and
usually have 100% monthly uptime (according to pingdom).
So, for me, the single server solution works best.
Especially with a VPS where the hosting company moves the VPS in case of any hardware problems.
Cheers,
Marc

On 14 Jan 2019, at 19:47, [ lloyd@REDACTED ]( mailto:lloyd@REDACTED ) wrote:

Hi Marc,

Many thanks for your prompt response.

This addresses one part of the puzzle. I can't speak for Eddie, but as a total DevOps noob, a big-picture perspective would give me greater comfort as I move into my own deployments.

I'm guessing that there are different issues if we're considering on-premises vs. cloud deployment, so considering the two hosting scenarios...

What are the specific security, scaling, or other issues we need to attend to? And how do we implement them? For instance, if on-premises: Do we need a dmz to protect our LAN or is port-forwarding sufficient? Should we put a proxy server such as nginx or HAProxy in front of our app? How can we most effectively harden our servers and applications against exploits?

I understand much of this goes beyond the realm of Zotonic, Erlang, and Nitrogen. But when I step out into the web for answers I'm totally overwhelmed with confusing and oft contradictory info.

I'd love to create a living-document tutorial and checklist for, at least, the simplest deployments, but wouldn't know where to start.

Thanks again,

Lloyd

-----Original Message-----From: "'Marc Worrell' via Zotonic developers" <[ zotonic-developers@REDACTED ]( mailto:zotonic-developers@REDACTED )>Sent: Monday, January 14, 2019 11:53amTo: "'Marc Worrell' via Zotonic developers" <[ zotonic-developers@REDACTED ]( mailto:zotonic-developers@REDACTED )>Cc: "[ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED )" <[ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED )>Subject: Re: [Zotonic-Dev] zotonic CI deployment best practice for 2019
Hi Eddie and Lloyd,
“It depends” is indeed the correct answer...
Most Zotonic installations fall in two categories:
1. Deployment via a Docker container, which is updated before deployment
2. Deployment via git (manual or automatic)
Some follow (1),  I follow (2).
Nice thing of (2) is that it allows for hot code upgrades and quick turnaround of small patches.
When we are i a development cycle for new features we might have 10 to 20 deployments per day.
This goes very smooth, we even deploy minor updates to Zotonic and dependencies using hot code upgrades.
Zotonic itself can watch changes in the file system and dynamically load new files.

Most of the updates we deploy have only minor changes.
Think of changes to templates, css, or translations.
And minor Erlang changes (ie. not completely new apps or other dependencies).
That is why we can have updates-via-git without service interruption.
The orgs that deploy via Docker (option 1) - have typically less frequent updates and perform periodic major updates.
As the whole container is updated this also restarts the Zotonic server.
Of course it is also possible to deploy using the OTP release mechanism.
This is something that will be possible with the 1.0 (really soon now) of Zotonic, as that version is OTP compliant.
I personally never had the need to use OTP releases as the version control via git is good enough for us.
(Especially with rebar3 managing versions of dependencies.)
Cheers, Marc

On 14 Jan 2019, at 17:40, [ lloyd@REDACTED ]( mailto:lloyd@REDACTED ) wrote:

Hi Eddie,

I'm in a similar situation--- planning soon to release a Nitrogen web application but feeling profound apprehension since I don't have a clear understanding of risks and how to minimize them.

No doubt the flip answer is, "It depends..." Are we talking cloud hosting or on-premises? What are projected traffic patterns? Etc. Etc.

But "It depends..." doesn't help folks like us who lack experience or organizational support.

One can find considerable information on the web covering firewalls, proxy servers, load balancers, site hardening, etc. But, like you, I've found little in the Erlang corpus that provides sufficiently clear patterns and guidelines to assuage my Erlang release/production apprehensions.

It may well be a book-length topic or more, but even a thoughtful, thorough tutorial, or even a checklist, would be helpful.

Please do let me know what you come up with.

All the best,

Lloyd

-----Original Message-----From: [ layeddie@REDACTED ]( mailto:layeddie@REDACTED )Sent: Monday, January 14, 2019 9:28amTo: "Zotonic developers" <[ zotonic-developers@REDACTED ]( mailto:zotonic-developers@REDACTED )>Subject: [Zotonic-Dev] zotonic CI deployment best practice for 2019

Hi,
I have followed zotonic for the last couple of years and am now interested in setting up a production server for zotonic.
I have just spent some time searching zotonic users and zotonic developers for an up to date guide on how you would go about developing and deploying zotonic to a production environment.
most of the links I found range from between 2010 - 2012.
I have looked through the zotonic documentation as well.
I have also installed locally using the zotonic full docker container and the manual install.
I am looking for a guide for Git / continuous integration / continuous deployment guide and any best practice tips.
Is something like this available?
Many thanks
Eddie L
(Uk based)--  ---  You received this message because you are subscribed to the Google Groups "Zotonic developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [ zotonic-developers+unsubscribe@REDACTED ]( mailto:zotonic-developers+unsubscribe@REDACTED ). For more options, visit [ https://groups.google.com/d/optout ]( https://groups.google.com/d/optout ).--  ---  You received this message because you are subscribed to the Google Groups "Zotonic developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [ zotonic-developers+unsubscribe@REDACTED ]( mailto:zotonic-developers+unsubscribe@REDACTED ). For more options, visit [ https://groups.google.com/d/optout ]( https://groups.google.com/d/optout ).
--  ---  You received this message because you are subscribed to the Google Groups "Zotonic developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [ zotonic-developers+unsubscribe@REDACTED ]( mailto:zotonic-developers+unsubscribe@REDACTED ). For more options, visit [ https://groups.google.com/d/optout ]( https://groups.google.com/d/optout ).--  ---  You received this message because you are subscribed to the Google Groups "Zotonic developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [ zotonic-developers+unsubscribe@REDACTED ]( mailto:zotonic-developers+unsubscribe@REDACTED ). For more options, visit [ https://groups.google.com/d/optout ]( https://groups.google.com/d/optout ).
-- 

 --- 
 You received this message because you are subscribed to the Google Groups "Zotonic developers" group.
 To unsubscribe from this group and stop receiving emails from it, send an email to [ zotonic-developers+unsubscribe@REDACTED ]( mailto:zotonic-developers+unsubscribe@REDACTED ).
 For more options, visit [ https://groups.google.com/d/optout ]( https://groups.google.com/d/optout ).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20190115/109222f3/attachment.htm>