Mastering Advanced Logback Configurations in Spring Boot

by

Introduction

In previous posts, we introduced logging fundamentals in Spring Boot and covered basic and intermediate configurations for console and file logging. As we delve deeper into the realm of logging, this post focuses on advanced Logback configurations to address complex logging requirements. We’ll explore configuring sophisticated rolling policies, filtering logs, and integrating with external log aggregation tools.

The logback.xml & logback-spring.xml

While you can configure logback in the application.properties or application.yaml file. However, to extensively the logging mechanism, you need to configure extra knobs in logback xml files. There are two valid file names you can put to src/resources of your project: logback.xml & logback-spring.xml

logback.xml example

Here is one example of a logback.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="30 seconds">

    <!-- Define the default log level and additivity for root logger -->
    <root level="INFO">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="FILE"/>
    </root>

    <!-- Console Appender -->
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} - %logger{36} - %level - %msg%n</pattern>
        </encoder>
    </appender>

    <!-- Rolling File Appender -->
    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>logs/application.log</file>
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n</pattern>
        </encoder>

        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- Daily rollover, with gzipped archives -->
            <fileNamePattern>logs/archived/application-%d{yyyy-MM-dd}.log.gz</fileNamePattern>
            <maxHistory>30</maxHistory>
            <totalSizeCap>1GB</totalSizeCap>
        </rollingPolicy>
    </appender>

    <!-- Example of setting a specific log level for a package -->
    <logger name="org.springframework.web" level="WARN">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="FILE"/>
    </logger>
    <logger name="com.datmt.logging" level="DEBUG">
        <appender-ref ref="STDOUT"/>
        <appender-ref ref="FILE"/>    
    </logger>

</configuration>

There are quite many sections in this file. Let’s learn what they are.

Sections in a logback.xml file

A logback.xml configuration file is divided into several key sections, each serving a specific purpose in controlling how logging is handled within an application using Logback. Below, I’ll explain the common sections and elements you’ll encounter in a logback.xml file.

Configuration Tag

  • <configuration>: This is the root element of a Logback configuration file. It can contain attributes like scan and scanPeriod, which instruct Logback to periodically check for changes in the configuration file and reload it automatically if any changes are detected.

Root Logger Section

  • <root>: The root logger is at the top of the logger hierarchy. It captures all log messages by default unless overridden by more specific loggers. Like other loggers, it has a level attribute and can reference one or more appenders.

Appender Section

  • <appender>: Appenders define where log messages are outputted. Logback supports various types of appenders for different output destinations, such as console (ConsoleAppender), files (FileAppender, RollingFileAppender), and even network sockets. An appender configuration includes:
    • name: A unique name for the appender.
    • class: The fully qualified class name of the appender type.
    • <encoder> or <layout>: Defines the format of the log message.
    • <rollingPolicy>: Only for RollingFileAppender, it defines how files are rolled over (by time, size, etc.).
    • <triggeringPolicy>: Also for RollingFileAppender, it specifies when a rollover should occur.
  • <encoder>: Contains a <pattern> element that defines the format of log messages. Patterns can include elements like %d for date, %level for log level, and %msg for the log message.
  • <fileNamePattern>: Within a <rollingPolicy>, this specifies the naming convention for rolled-over log files, often incorporating date or time information.
  • <maxHistory>, <maxFileSize>, <totalSizeCap>: These elements control the retention policy of log files, limiting how many log files are kept, the size of each log file, and the total size of all log files.

Logger Section

  • <logger>: Loggers are responsible for capturing logging messages from your application. Each logger is associated with a specific package or class. Logger configuration can include:
    • name: The name of the package or class to log. The root logger is special and captures log messages from all classes.
    • level: The minimum level of messages that the logger will handle (e.g., DEBUG, INFO, WARN, ERROR).
    • additivity: A boolean that determines if log messages are propagated to higher-level loggers. For most loggers, this is true by default, but it can be set to false to prevent duplication of log messages in multiple appenders.

Fine-Tuning Log Levels

Before jumping into advanced configurations, let’s briefly revisit log levels. Logback allows for granular control over log levels on a per-class or per-package basis. This capability is crucial for managing verbose logging in development or focusing on error tracking in production.

<logger name="com.datmt.logging" level="DEBUG"/>
<logger name="org.springframework.web" level="INFO"/>

Advanced Rolling File Strategies

Rolling files ensure that logs are manageable, preventing them from consuming excessive disk space. Let’s explore advanced rolling file configurations.

Time based Rolling

TimeBasedRollingPolicy is possibly the most popular rolling policy. It defines a rollover policy based on time, for example by day or by month. TimeBasedRollingPolicy assumes the responsibility for rollover as well as for the triggering of said rollover. Indeed, TimeBasedTriggeringPolicy implements both RollingPolicy and TriggeringPolicy interfaces.

        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- daily rollover -->
            <fileNamePattern>logFile.%d{yyyy-MM-dd HH:mm}.log</fileNamePattern>
      
            <!-- keep 30 days' worth of history capped at 3GB total size -->
            <maxHistory>30</maxHistory>
            <totalSizeCap>3GB</totalSizeCap>
          </rollingPolicy>

In this example, the log files are created for every minute:

Log file created for every minute

Time and Size Based Rolling

Combining time and size-based rolling policies enables logs to roll over after reaching a specific size or time threshold, whichever comes first.

<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>log/myapp.log</file>
    <encoder>
        <pattern>%date %level [%thread] %logger{10} - %msg%n</pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
        <fileNamePattern>log/archived/myapp-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
        <maxFileSize>10MB</maxFileSize>
        <maxHistory>30</maxHistory>
        <totalSizeCap>1GB</totalSizeCap>
    </rollingPolicy>
</appender>

This configuration archives logs daily or when they exceed 10MB, compressing them into ZIP files to save space.

Log Filtering

Filtering logs can suppress or highlight specific log entries based on criteria such as log levels, message patterns, or MDC (Mapped Diagnostic Context) values.

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <filter class="ch.qos.logback.classic.filter.LevelFilter">
        <level>INFO</level>
        <onMatch>DENY</onMatch>
        <onMismatch>NEUTRAL</onMismatch>
    </filter>
    <!-- Appender configuration continues -->
</appender>

This example uses a LevelFilter to deny (suppress) all INFO level logs, allowing more critical logs to stand out.

Conclusion

Advanced Logback configurations empower developers and operations teams to tailor logging precisely to their application’s needs and operational environment. By leveraging sophisticated rolling policies, filters, and external integrations, you can ensure that your logging strategy scales efficiently with your application.

Remember, the key to effective logging is not just capturing as much information as possible but doing so in a way that enhances observability, debuggability, and operational intelligence without overwhelming your resources.

Leave a Comment