Centralized logging Using lasers and other cool stuff Sylvain - PowerPoint PPT Presentation

Centralized logging Using lasers and other cool stuff Sylvain Fankhauser / Bastian Widmer

Agenda ● The beginnings ● What Liip is doing now ● Goals ● Possible Solutions ● Our Solution ● Implementations (SF2, Drupal 6&7, other php apps) ● Demo

The early beginnings August 2011 - Adrian Schwaller "Liipers like developing and Liipers are good in developing. However when it's going to maintenance we often lack of quality."

The early beginnings August 2011 - Adrian Schwaller "It is not rarely the case that the client has to inform us about an impediment. So why not developing an application that help us doing maintenance - a win-win solution."

Stock-check: what are we doing now? ● sending mails with logs (LOTS OF MAILS!) ● checking manually (humans fail regularly!) ● waiting for the client to complain about issues on his website Not a good solution, right?

Second iteration Sylvain and Bastian - September 2012 We get a Year Goal to centralize logging

Goals ● introduce a standard platform ● log unexpected results and thrown errors ● early on detection of problems (DDOS, Spammers) ● be faster to notice the problem than our customers ● Create a PHP tool that could be easily plugged into any type of application (SF2, Drupal, Mahara, Moodle)

Solutions there are many let's look at them!

Splunk (splunk.com) + nice graphical interface + google.com for logfiles + easy to maintain + easy to setup + can do everything you dream about - costs a big amount of money $$$

Loggly (loggly.com) + Cloud Based + no maintenance work needed (SaaS) - Costs: Depending on our needs - not cheap - Cloud Based (Security concerns)

Graylog 2 (graylog2.org) + nice interface + not complicated to setup + active development - very young project and architectural changes (mongodb -> elastic search)

Kibana (kibana.org) kibana went into our focus after we've done the ecostar. It's quite nice but we did not test it during our ecostar. Bastian short tested it afterwards on his own: + simple setup + actively maintained + multimegafunctional by use of logstash frontend/backend : ruby datastorage : elasticsearch

Logstash, WHAT? Logstash is the swiss army knife regarding logging needs, it's a translator between many different formats. e.g. translates .log files on the fly into gelf messages that could be consumed by graylog check : logstash.net

le Setup

graylog - what you could do

graylog- what we do currently

Graylog Server (currently) ● Frontend - UDP Backend : Graylog - Java ● Data Storage : MongoDB

Graylog Server Changes v0.10 ● Frontend - UDP Backend :Graylog - Java ● Data Storage : Elastic Search ● optional : Logstash ● new : Support for Plugins ○ GECKOBOARD - ohai hackday! ○ sending XMPP Jabber Messages ○ writing Graphs with Graphite ○ GELF over HTTP (a log entry is just a curl request away)

Developers, developers, developers finally some php ;)

Use Graylog in your project ● Configure your application to send logs to Graylog ● Very easy to use: log stuff as usual, it's automatically sent to Graylog ● Under the hood, you'll use a GELF library ● Docs about integration in your project are available here: http://liip.to/graylog

Logging with Symfony >= 2.1 ● Add the GELF library to composer: "require": { ... "mlehner/gelf-php": "1.0.*" }

Logging with Symfony >= 2.1 ● Configure monolog: services: gelf_formatter: class: Monolog\Formatter\GelfMessageFormatter arguments: ['clientdomain.com'] monolog: handlers: gelf: type: gelf publisher: hostname: log.liip.ch port: 12201 formatter: gelf_formatter level: warn

Logging with Symfony >= 2.1 ● Then log stuff as usual: $logger = $this->get('logger'); $logger->warn('Something went wrong');

Logging with Symfony 2.0 ● The version of Monolog shipped with SF2 doesn't support GELF ● Installation is a bit painful, but easy with the guide @ http://liip.to/graylog

Logging with Drupal 7 ● The easy way: curl -L http://liip.to/graylogdrupal7 | sh ● Adds the required modules as submodules ● Enables them ● Adds them to your modules.enabled file

Logging with Drupal 7 ● The less-easy way: ○ Install the gelf-php library ○ Install the libraries and gelf modules ○ Use the gelf module hosted on my github account

Logging with Drupal 6 ● Again the easy way: curl -L http://liip.to/graylogdrupal6 | sh ● Does the same as the Drupal 7 solution ● Uses the 6.x modules branches ● The less-easy way is the same as for Drupal 7

Logging with other PHP apps ● Use the gelf-php library from here: https: //github.com/Graylog2/gelf-php ● Create the message: <?php require('GELFMessage.php'); require('GELFMessagePublisher.php'); $message = new GELFMessage(); $message->setShortMessage('ohai.'); $message->setFullMessage('i are a cats'); $message->setHost('clientdomain.com'); $message->setLevel(GELFMessage::CRITICAL); $message->setFile(__FILE__); $message->setLine(__LINE__); $message->setAdditional('the_grass', 'is green'); $message->setAdditional('skies', 'so blue');

Logging with other PHP apps ● Publish the message: <?php ... $publisher = new GELFMessagePublisher('log.liip.ch'); $publisher->publish($message);

DEMO!

Questions?

Thank You! Logging for a better world : http://log.liip.ch username : liip password : on our pw.liip.ch server (search for graylog) http://liip.to/graylog