> Ops is maybe trying to help developers and increase security (no login on the boxes needed anymore to look at logs).
Can this be handled fine by having tail run as a daemon and forwarding all log entries to somewhere that the developers do have access?
> Better workflow for (certain kind of) alerts, which may then get more complicated.
This is likely to be highly specific as to what sort of alerts you want, but will tailing the log and running it through an awk script work? I assume you'll have to do something much like this with any alert management tool.
Tools like kabana like nice, but this is one of those areas where I think people might be going for the shiny solution (that looks great to management) instead of analyzing what they really need.
You can use whatever system you want (syslog can drop the logs in a single box, no need for horrible tail-based concoctions), as long as you mantain it and it's not Ops responsibility to fix your AWK scripts that look for events in logs from 3 different services for the same request and the same customer (and/or respond to the alerts those script generate at 2am after Bob forgot to update it to parse correctly a new log message - we have our own fires to fight already).
I'm not saying one should go straight to ELK. There are other ways, but at the end of the day you are going to implement a similar stack, guaranteed, and you are going to regret using freeform logging instead of a sensible structured format.
Can this be handled fine by having tail run as a daemon and forwarding all log entries to somewhere that the developers do have access?
> Better workflow for (certain kind of) alerts, which may then get more complicated.
This is likely to be highly specific as to what sort of alerts you want, but will tailing the log and running it through an awk script work? I assume you'll have to do something much like this with any alert management tool.
Tools like kabana like nice, but this is one of those areas where I think people might be going for the shiny solution (that looks great to management) instead of analyzing what they really need.