Follow

Adjusting flume log size and retention number

Unlike other components, the log rotation and size limits for Flume must be adjusted in the config file and not by using Ambari

The process for adjusting these settings is as follows:

1. Navigate to /etc/flume/conf
2. Edit the log4j.properties file
3. Change the values as listed below to align with your desired settings:

# Stock log4j rolling file appender
# Default log rotation configuration
log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender
log4j.appender.LOGFILE.MaxFileSize=100MB
log4j.appender.LOGFILE.MaxBackupIndex=10
log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}
log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout
log4j.appender.LOGFILE.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n

4. Restart Flume

Keep in mind that the majority of the logs for the various Interset components are its dependencies are very chatty.  If these values are set too low, troubleshooting may prove to be challenging as the logs will rotate and will not be present for the time of issues if the rotation and size is set too low.  In these cases Interset may ask for the settings to be increased and the issue recreated for troubleshooting purposes.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments