OpenCensus / OpenTelemetry

[Tristan Sloughter]

Yes, for exporters there is a standardized wire format, OTLP -- https://github.com/open-telemetry/opentelemetry-proto/tree/master/opentelemetry/proto/collector -- for metrics and traces (and eventually logs). The Erlang lib will have a number of other exporter and currently has, in addition to OTLP https://github.com/opentelemetry-beam/opentelemetry_exporter, there is currently Zipkin https://hex.pm/packages/opentelemetry_zipkin and Honeycomb https://hex.pm/packages/opentelemetry_honeycomb

There is a Stackdriver OpenCensus exporter for traces https://github.com/opencensus-beam/oc_google_reporter and I will put together one for OpenTelemetry soon enough.

For the collector/agent (https://opentelemetry.io/docs/collector/about/) you don't have to run it as an agent, it can be a a central service all your nodes push traces and metrics push to and then it forwards to whatever you want.

Using the collector is the best way to have support for many vendors/products because they'll all create a Go library that can be used by the collector but the Erlang/Elixir libs are up to us for pretty much every vendor :)

On Mon, May 25, 2020, at 10:48, Vance Shipley wrote:
> On Mon, May 25, 2020 at 11:09 PM Tristan Sloughter <t@REDACTED> wrote:
>> __
>> Hey Vance, the OpenCensus Erlang lib has support for metrics. The OpenTelemetry Erlang lib has very basic/limited support because the OpenTelemetry metrics spec has gone through fluctuation and uncertainty until last week. With that now fairly complete and stable I hope to get time in the near future (meaning weeks) to update the Otel metrics support.
> 
> That time frame probably works for me. I am looking to modernize load benchmarking in our CI/CD. When I look at the tooling the important features are: automatic distribution, metrics visualization and easy scenario authoring. It seems to me that distribution should be done with k8s and we'll use GKE so their monitoring gives us the pretty pictures, and finally we'll be writing tests (and servers) in Erlang. Now if we can funnel metrics into Google's Stackdriver we'll be golden.
> 
> If OpenTelemetry/OpenCensus solves my current (GCE) requirement and will be portable to any future targets it's perfect. Unfortunately I'm not seeing an Erlang library to Stackdriver pipeline yet however I've discovered the answer to the matrix problem (many language libraries : many exporters) is to converge on one "wire" format from the library to an agent collector on the node. That fully featured agent implemented in Go gets the full list of exporters. However it's early days for this it seems:
> https://github.com/open-telemetry/opentelemetry-collector/blob/master/docs/design.md
> 
>> So I would not advise using Otel if metrics are needed unless you are interested in helping contribute :)
> 
> We could do that although I can also probably make do with what metrics I can collect adhoc for now and improve our reports when this stuff is ready to test (pun intended).
> 
>> Erlang/Elixir is an official SIG of the OpenTelemetry project but I have rolled it into the Erlang Ecosystem Foundation's Observability Working Group for handling meetings -- if you are wanting to help with the metrics development, or even if you aren't but are still interested, you are welcome to join our WG meetings.
> 
> I will, I have subscribed to the calendar. 
> 
> -- 
>  -Vance
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200525/e2376453/attachment.htm>