Java – Pass dynamic values to the log4j2 xml configuration

Pass dynamic values to the log4j2 xml configuration… here is a solution to the problem.

Pass dynamic values to the log4j2 xml configuration

I’m trying to write a logging module using log4j2 that needs to write logs to console STDOUT in json.

To do this, I tried using PatternLayout as JSON as shown below.

I’m having some difficulty dynamically passing values from my code to the log4j2.xml config file to replace them at runtime when writing logs.

I tried replacing with StructuredDataMessage and MapMessages https://logging.apache.org/log4j/2.0/manual/lookups.html The value in Map mentioned in

I’ve also tried StrLookup and ContextMaplookup, but so far without success.

Below is my xml config

<?xml version="1.0" encoding="UTF-8"?>
    <Configuration status="WARN" name="App" packages="com.test.common.logging">
        <Properties>
            <Property name="filename">target/rolling1/rollingtest.log</Property>
            <Property name="maptype">$${map:type}</Property>

</Properties>

<ThresholdFilter level="debug"/>

<Appenders>
            <Console name="Console" target="SYSTEM_OUT">
              <PatternLayout pattern="%highlight{{'logged time': '%d{dd MMM yyyy HH:mm:ss}', 
            'LEVEL' : '%level',
            'CLASS' : '%c{-1}',
            'Module' : '[%t]',
            'message' : '%m',
            'error' : '%exception',
            'class' : '%C',
            'threadid' : '%tid',
            'threadname' : '%thread',
            'whatisthis' : '${filename}',
            'processid' : '%pid', 
            'logdir' : '$${sd:type}'
            'location' : '${log4j:configLocation}'
            'systemproperty' : '$${ctx:key-}'
            }}%n"/>
            </Console>
        </Appenders>

<Loggers>
            <Root level="trace">
                <AppenderRef ref="Console"/>
            </Root>
        </Loggers>

</Configuration>

Below is my code, I try to pass dynamic values using structured data messages, strlookup, and mapmessages

public class App 
{

public static void main( String[] args )
    {
        Logger logger = LogManager.getLogger(App.class);

      ConfigurationBuilder<BuiltConfiguration> builder
       = ConfigurationBuilderFactory.newConfigurationBuilder();
//      
      LayoutComponentBuilder standard 
        = builder.newLayout("PatternLayout");
      standard.
//      
        System.out.println( "Hello World!" );
        StructuredDataMessage message = new StructuredDataMessage("1", "name", "string");
        message.put("1", "nme");
        MapMessage mapm = new MapMessage(map)
        MapMessage map = new MapMessage();
        map.put("type", "value");
        map.put("key", "value");
        map.put("name", "arun");
        StrLookup strlook = new StrLookup() {

public String lookup(LogEvent event, String key) {
                 TODO Auto-generated method stub
                return null;
            }

public String lookup(String key) {
                 TODO Auto-generated method stub

return "value";
            }
        };
        ContextMapLookup lookup = new ContextMapLookup();
        System.out.println(lookup.lookup("key"));
        System.out.println(strlook.lookup("key"));

 MapLookup.setMainArguments(args);

System.setProperty("log_dir", App.class.getSimpleName());;
        logger.trace("trace log message");
        logger.debug("Debug log message");
        logger.info("Info log message");
        logger.error("Error log message");
        logger.fatal("Fatal log message");
        logger.info("Info log message[]");
        logger.error("null exception[]", new NullPointerException());

 Lay

}
}

My output:

Hello World!
null
value
[36m{'logged time': '24 Aug 2018 12:32:51',    'LEVEL' : 'DEBUG',   'CLASS' : 'test.common.logging.App',   'Module' : '[main]',   'message' : 'Debug log message',   'error' : '',   'class' :  'com.test.common.logging.App',   'threadid' : '1',   'threadname' : 'main',   'whatisthis' : 'target/rolling1/rollingtest.log',   'processid' : 'DEBUGid',    'logdir' : '${sd:type}'   'location' :  '/Users/parunkarthick/ferry-commons/common/logging/target/classes/log4j2.xml'   'systemproperty' : '${ctx:key}'   [m}
[32m{'logged time': '24 Aug 2018 12:32:51',    'LEVEL' : 'INFO',   'CLASS' : 'test.common.logging.App',   'Module' : '[main]',   'message' : 'Info log message',   'error' : '',   'class' :  'com.test.common.logging.App',   'threadid' : '1',   'threadname' : 'main',   'whatisthis' : 'target/rolling1/rollingtest.log',   'processid' : 'INFOid',    'logdir' : '${sd:type}'   'location' :  '/Users/parunkarthick/ferry-commons/common/logging/target/classes/log4j2.xml'   'systemproperty' : '${ctx:key}'   [m}
[1; 31m{'logged time': '24 Aug 2018 12:32:51',    'LEVEL' : 'ERROR',   'CLASS' : 'test.common.logging.App',   'Module' : '[main]',   'message' : 'Error log message',   'error' : '',   'class' :  'com.test.common.logging.App',   'threadid' : '1',   'threadname' : 'main',   'whatisthis' : 'target/rolling1/rollingtest.log',   'processid' : 'ERRORid',    'logdir' : '${sd:type}'   'location' :  '/Users/parunkarthick/ferry-commons/common/logging/target/classes/log4j2.xml'   'systemproperty' : '${ctx:key}'   [m}
[1; 31m{'logged time': '24 Aug 2018 12:32:51',    'LEVEL' : 'FATAL',   'CLASS' : 'test.common.logging.App',   'Module' : '[main]',   'message' : 'Fatal log message',   'error' : '',   'class' :  'com.test.common.logging.App',   'threadid' : '1',   'threadname' : 'main',   'whatisthis' : 'target/rolling1/rollingtest.log',   'processid' : 'FATALid',    'logdir' : '${sd:type}'   'location' :  '/Users/parunkarthick/ferry-commons/common/logging/target/classes/log4j2.xml'   'systemproperty' : '${ctx:key}'   [m}
[32m{'logged time': '24 Aug 2018 12:32:51',    'LEVEL' : 'INFO',   'CLASS' : 'test.common.logging.App',   'Module' : '[main]',   'message' : 'Info log message[]',   'error' : '',   'class' :  'com.test.common.logging.App',   'threadid' : '1',   'threadname' : 'main',   'whatisthis' : 'target/rolling1/rollingtest.log',   'processid' : 'INFOid',    'logdir' : '${sd:type}'   'location' :  '/Users/parunkarthick/ferry-commons/common/logging/target/classes/log4j2.xml'   'systemproperty' : '${ctx:key}'   [m}
[1; 31m{'logged time': '24 Aug 2018 12:32:51',    'LEVEL' : 'ERROR',   'CLASS' : 'test.common.logging.App',   'Module' : '[main]',   'message' : 'null exception[]',   'error' : ' java.lang.NullPointerException
    at com.test.common.logging.App.main(App.java:69)
',   'class' : 'com.test.common.logging.App',   'threadid' : '1',   'threadname' : 'main',   'whatisthis' : 'target/rolling1/rollingtest.log',   'processid' : 'ERRORid',    'logdir' : '${sd:type}'   ' location' : '/Users/parunkarthick/ferry-commons/common/logging/target/classes/log4j2.xml'   'systemproperty' : '${ctx:key}'   [m}

If you see that the lastvalue system property is not reflected as expected by substituting from the code value.

Solution

I think you have some fundamental misconceptions about log4j2. I think the best thing to do is to provide some sample code and explain the output instead of me trying to list all the issues I’m seeing in your code. I think when you see some working code, you’ll understand what went wrong.

For the purposes of this example, I’m simplifying your log4j2 configuration file by removing elements that seem valid and focusing on those that don’t. I changed the PatternLayout to the following:

<PatternLayout pattern="{'LEVEL' : '%level', 'typeFromStructMsg' : '${sd:type}', 'contextValue' : '${ctx:myContextKey}', 'nameFromMapMsg' : '${map:name}', ' mySysProperty' : '${sys:mySysProperty}'}%n" />

I also modified the app class you provided:

package example;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
import org.apache.logging.log4j.message.MapMessage;
import org.apache.logging.log4j.message.StringFormattedMessage;
import org.apache.logging.log4j.message.StringMapMessage;
import org.apache.logging.log4j.message.StructuredDataMessage;

public class App {

private static final Logger logger = LogManager.getLogger();

public static void main( String[] args )
    {
        ThreadContext.put("myContextKey", "myContextValue");

StructuredDataMessage structMsg = new StructuredDataMessage("1", "name", "string");

StringMapMessage mapMsg = new StringMapMessage();
        mapMsg.put("name", "arun");

System.setProperty("mySysProperty", "sys prop value");

logger.info(mapMsg);
        logger.warn(structMsg);

logger.error("Error log message");
    }
}

When the App class runs, the following console output is generated:

{'LEVEL' : 'INFO', 'typeFromStructMsg' : '${sd:type}', 'contextValue' : 'myContextValue', 'nameFromMapMsg' : 'arun', 'mySysProperty' : 'sys prop value'}
{'LEVEL' : 'WARN', 'typeFromStructMsg' : 'string', 'contextValue' : 'myContextValue', 'nameFromMapMsg' : '${map:name}', 'mySysProperty' : 'sys prop value'}
{'LEVEL' : 'ERROR', 'typeFromStructMsg' : '${sd:type}', 'contextValue' : 'myContextValue', 'nameFromMapMsg' : '${map:name}', 'mySysProperty' : 'sys prop value'}

Notice that in the first line of the output we see: ‘nameFromMapMsg’ : ‘

arun’ while in the other lines we see: ‘nameFromMapMsg‘ : '${map:name }'

The first line of output is generated by this line of code: logger.info(mapMsg); It passes a MapMessage instance named mapMsg< to the info method. Because the message is an instance of MapMessage and it contains a key named name, map lookup replaces ${map:name} with the value it looks for the name key in the message. That’s why only the first line of output shows ‘nameFromMapMsg': 'arun' – the other output lines are generated from messages that are not instances of MapMessage.

Again, notice how we see ‘typeFromStructMsg' : 'string' in the second line of output. This is because the log is generated from StructuredDataMessage, which is defined with type “string”:

StructuredDataMessage structMsg = new StructuredDataMessage("1", "name", "string");

In the other lines of the output, we didn’t pass StructuredDataMessage so in those lines we see ‘typeFromStructMsg' : '${sd:type}‘ log4j2 can’ find the value of that type.

Finally, notice how we see in all the output lines: ‘mySysProperty' : 'sys prop value'. This is because system property lookups do not depend on the type of message passed to the logger. This lookup always finds the value of the system property mySysProperty because we defined it:

System.setProperty("mySysProperty", "sys prop value");

And, as I said earlier, system properties are independent of messages (they are not stored in messages).

The same is true for ‘contextValue’: ‘myContextValue'ThreadContext is message-independent because we define a value for this key:

ThreadContext.put("myContextKey", "myContextValue");

Regardless of the type of message sent to the logger, the lookup always finds the value.

I hope this sample code helps illustrate how to use some lookups and how to design the architecture of log4j2. Good luck!

Related Problems and Solutions