Logging Best Practices

RDFS Best Practices

The logging library we are using is Easylogging++

Named Logging Levels

These are the "named" levels that EL++ provides. Below are the use cases for each level.

• INFO - progress information only
• DEBUG - information useful for developers (block locations, data transfer sizes)
• WARNING - unexpected events
• ERROR - recoverable errors
• FATAL - fatal errors

Verbose Logging Levels

EL++ has 10 levels (0 - 9) available. Below are the use cases for the levels that we will be using.

• 9: Reserved for performance-tracking

Verbose Usage

• -v set maximum verbosity (9)
• --v=<x> set verbosity level to x (0 - 9)

A Note About Config Files

Note that the logging file configuration requires a fully qualified path to the logging file, e.g. /home/vagrant/rdfs/build/etc

Turning Off Console Logging

Sometimes you would like to reduce the amount of clutter printed to your terminal. There are at least two ways to do this.

• Modify the relevant configuration file(s) in <project root>/config, e.g. <project root>/config/nn-log-conf.conf if you are firing up the namenode from the command line. Add TO_STANDARD_OUTPUT = false for the logging levels you want to mute. Adding the line under * GLOBAL: mutes all logging output from Easylogging++.
• Easylogging++ can be programmatically configured. Normally this would be done in the main function, after reading in the configuration file. It should also be possible to do this during the execution of useful stuff, i.e. changing logging configurations on-the-fly. Below are some examples, where conf is an object of type el::Configurations, which can be instantiated by something like el::Configurations conf("path to config file");.

  • conf.set(el::Level::Info, el::ConfigurationType::Enabled, "false"); el::Loggers::reconfigureAllLoggers(conf);

  • conf.set(el::Level::Info, el::ConfigurationType::ToStandardOutput, "false"); el::Loggers::reconfigureAllLoggers(conf);

  • conf.set(el::Level::Global, el::ConfigurationType::ToStandardOutput, "false"); el::Loggers::reconfigureAllLoggers(conf);

ZooKeeper

Turning Off Console Logging

This can be done by modifying ~/zookeeper/conf/log4j.properties, changing zookeeper.console.threshold to something high enough, such as ERROR.