[erlang-questions] how: logging messages to another node

[Jay Nelson]

Serge wrote:

 > Hmm, seems like this is what you want to do:
 >      rpc:call(Node, erlang, whereis, [error_logger]).

Doh!   I never use rpc: so it didn't occur to me at all that any  
function I use locally can be trivially invoked using rpc:call.  That  
is a good tool when coordinating activity between two nodes.

I guess that is the solution to finding an arbitrary process id on  
any node in a cluster.  Makes sense that all other calls can work  
this way, except monitor and demonitor -- no wonder I found ways to  
do it with those but not with anything else.


Ulf wrote:

 >  Why not just install a local handler in the error_logger
 >  on the diskless nodes, which forwards to the backend node?

This is what I am doing, but I want it to be somewhat robust.  The  
diskless node may be up when the backend is not, so I created a  
gen_server:

process logs error -->  error_handler -->  local gen_server -->   
backend node

The local gen_server saves a queue of messages when the backend is  
disconnected.  Once it reconnects I want to forward the messages that  
I've accumulated:

1) If I just change gen_leader on the diskless node, I can't  
necessarily "guarantee" that messages get to the backend

2) With serge's lama suggestion it gets closer I think, but I don't  
know if the gen_event:notify gets received by the backend (and I'm  
not sure if I want the local gen_server doing blocking calls to get  
acked from the backend, but I can ensure that all messages arriving  
to local gen_server are asynchronous)

What I've created so far is the following:

1) diskless gen_server which can:
      a) asynchronously receive messages from error_handler installed  
on local error_logger
      b) queue which allows head to be retrieved and sent async to  
backend
      c) delete of head once ack comes from backend
      d) b&c are easily changed to batch a block of messages

2) backend error_logger / error_handler that does proper message  
logging for my app


I would like to expand to having clustered backend and more reliable  
logging without knowing magic global names, etc.   After thinking on  
it last night while I was sleeping I am going to approach it as follows:

1) Quick hook up using serge's approach to get the process_id via  
rpc:call of whereis using the currently known single backend node name

2) Longer term, detect backend machines:
      a) Create a process which monitors a global_group of backends
      b) When a node connects, rpc a probe process to discover  
process ids, config of backend
      c) Send this information back to the monitor process
      d) Monitor then signals local message queue gen_server to start  
forwarding messages


The key features I want are:

1) Transparent local error_logger calls
2) Somewhat more reliable logging capture guarantees
3) Dynamic backend configuration and crash/recovery
4) Transparent local backend error_logger calls for backend problems  
which don't get confused with error logging already properly tagged  
and formatted on diskless node


Thanks for the tips, I at least have a plan of attack now.

jay