Go to All Forums

Site24x7 and the recent Apache Log4j vulnerability

On December 09, 2021, a severe vulnerability (CVE- 2021-4422) was disclosed in the popular Java logging library Log4j 2 versions- 2.0 to 2.14.1, that results in remote code execution (RCE) by logging a certain string. You can find the details of this vulnerability here: https://logging.apache.org/log4j/2.x/security.html 

Though there were a few attempts, we didn't find any traces or evidence of a successful exploitation. As we also possess some third-party components that could be potentially vulnerable, we've completely patched the vulnerability as a mitigation measure. And we can vouch for the fact that no sign of an active exploit could be found throughout Site24x7. Also, the different binary or installable software/agents we support aren't prone to this vulnerability.

We'll keep analyzing the issue and will be posting the new updates in this thread. Please feel free to contact support@site24x7.com or security@zohocorp.com for further details or assistance; we're happy to help you.

  

Regards,

Vinoth

Site24x7 Red Team

Like (34) Reply
Replies (35)

Thanks, please keep us posted (couldnt up vote the issue as i am based in the EU and portal doesnt support logging in as an EU User)

 

Like (0) Reply

Would be nice if there was some info on main Site24x7 page with link to this annoucement.

Like (0) Reply

yes agree on this, this is official cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228

If you look on all major players they have this official statement on their main page

Site 24x7 should do this also..

Like (0) Reply

Thanks for the update - I note that Log4j and PostgreSQL are components of agents that are end of life. They are carrying vulnerabilities too - are these being patched too?

Like (0) Reply

Hi Jonathan,

As commented in Jason's reply, we are safe against the log4shell vulnerabilities. We will also migrate the log4j dependency to the latest version as recommended by Apache.

With respect to your query on PostgreSQL, our product team is already working on this migration and will post an update regarding this soon

Thanks,

Vinoth,

Site24x7 Red Team

Like (0) Reply

What about On Prem Pollers:

C:\Program Files (x86)\Site24x7OnPremisePoller\lib\jars log4j-1.2.17.jar
C:\Program Files (x86)\Site24x7OnPremisePoller\NetworkPlus\lib log4j-1.2.8.jar
C:\Program Files (x86)\Site24x7OnPremisePoller\NetworkPlus\lib log4j-boot.jar
Like (0) Reply

I was wondering this same exact question.

Like (0) Reply

Hi Jason,
Site24x7 OnPremise Poller is not affected by this vulnerability(CVE- 2021-4422), the log4j 1.x version bundled in Poller doesn't support the the JNDI lookup feature. The log4j1.x version is vulnerable only under certain configurations when JMSAppender is used. Site24x7 OnPremise Poller doesn't use JMSAppender and hence not affected by the log4shell vulnerability

We are aware of the other vulnerability present in the the log4j 1.x,

The vulnerability with Log4j1.x (CVE-2019-17571), is RCE using insecure deserialization in SocketServer. The scenario is, if the application is running a Log4j's SocketServer opens a port and listens for Log Events from the network, then it can be exploited. The SocketServer implementation to deserialize the data coming in from the network to Java Object without verification can trigger RCE.

But Our usage of log4j in On-premise Poller is limited to basic logging functionality, and doesn't use the SocketServer feature. Hence we are safe against this vulnerability also.

However as per the recommendations from Apache, we are also planning to migrate the log4j jars to the latest one. I'll update this thread once the change is released.

 

Thanks,

Site24x7 Red Team

Like (0) Reply

Thank you for the detailed information, we appreciate that!.

Like (0) Reply

www.zdnet.com/article/second-log4j-vulnerability-found-apache-log4j-2-16-0-released/

Site24x7 team...is this covered

Like (0) Reply

Hi,

We are aware of the vulnerability CVE 2021-45046, The patch involves the removal of vulnerable JNDILookup.class from all our usage. We can confirm that we are resilient against this vulnerability also.

Thanks & Regards,

Vinoth

Site24x7 Red Team

Like (0) Reply

Dear All,

As mentioned earlier in this thread, we have migrated the log4j from 1.2.17 to 2.17.0 in the latest Site24x7 Poller binaries. The release notes can be found here.

https://www.site24x7.com/help/on-premise-poller-release-notes.html#version-5.1.3

 

Thanks,

Vinoth

Site24x7 Red Team

Like (0) Reply

Many thanks for this. Do you have an update on when we will see PostgreSQL updated to a version which is not end of life?

Like (0) Reply

We have upgraded our onpremise pollers to the latest version, 5.1.3, and we still see the old log4j file at Site24x7OnPremisePoller\NetworkPlus\lib\log4j-1.2.8.jar. Per the release notes, this file should have been replaced with log4j-2.17.0.

This is also being picked up by our vulnerability scans.

Like (0) Reply

Is the networkplus module active?  Same thing happened with us, but our networkplus module is disabled.  I thought that got installed when you activate it.  Just an idea.

Like (0) Reply

I'm not sure, I never did anything to activate NetworkPlus, I'm not even sure what it is. Can I just delete the NetworkPlus folder? Site24x7OnPremisePoller\NetworkPlus\lib\log4j-1.2.8.jar

Upon further investigation, it does look like the new 2.17 file does exist at Site24x7OnPremisePoller\lib\jars\log4j-core-2.17.0.jar. Vuln scan is now saying a 2.17.1 has been released.

 

Like (0) Reply

Hi we also updated to version 5.1.3 and after a rescan we are still showing 

/Site24x7OnPremisePoller/NetworkPlus/lib/log4j-1.2.8.jar

/opt/Site24x7OnPremisePoller/NetworkPlus Java 1.8.0_102

 

We do use the Network Modules so we cant just remove the folder. Will there be further updates to fix this?

Like (0) Reply

Hi Guest, Josh, Dough, and Shaheen

Thank you for reaching out. Your questions appear to point to the same scenario.

When you install the On-Premise Poller, it automatically downloads the Network Module. The Network Module is used only to monitor the network devices. In this case, as a workaround, we recommend you delete the content inside the NetworkPlus folder. (Site24x7OnPremisePoller/NetworkPlus/).

Please ensure that you are not deleting the parent NetworkPlus folder as the Network Module will be re-downloaded even if it is deleted. Hence, delete only the contents (files and subfolders) inside the NetworkPlus folder.

Regarding the log4j security issue in the Network Module, we have removed the vulnerable classes (JMSAppender.class and SocketServer.class) from log4j-1.2.8.jar and have released the latest build.

For existing Network Module installations, please follow the below steps to apply the security fix:

1. Download the patch from the below link.

https://staticdownloads.site24x7.com/probe/log4j-1.2.8-security-fix.zip

2.Once the patch is downloaded, stop the Site24x7 On-Premise Poller and ensure that all the processes are stopped.

3. Extract the patch file in the Site24x7 On-Premise Poller installed directory (default: Site24x7OnPremisePoller/). You have to replace the existing file(s).

4. Start the On-Premise Poller service with Administrator/root privileges.

 

Regarding PostgresSQL upgrade, we have added the Network Module's PostgreSQL and JRE version upgrade to our roadmap, and I'll update this thread when it's released. Currently, we do not have an exact timeline for the release. 

Regards,

Divyasree

Like (0) Reply

When will the upgrade to 2.17.1 happen?

Like (0) Reply

Hi Divyasree,

 

I have downloaded the jar file that you had provided in the link - staticdownloads.site24x7.com/probe/log4j-1.2.8-security-fix.zip.

But our Tenable scan is still detecting this vulnerability - Apache Log4j 1.2 JMSAppender Remote Code Execution (CVE-2021-4104)

Can you advice how to fix this issue?

Rgds

KoonYam

Like (0) Reply

Hi Koon Yam,

The class "org/apache/log4j/net/JMSAppender.class" has been removed from the jar mentioned in the comment and JMSAppender Remote Code Execution is not possible without this class. I'm also attaching the screenshot for your reference.

Security Advisory for mitigation against this attack, https://access.redhat.com/security/cve/CVE-2021-4104

I'm not sure why Tenable scan is flagging this as a vulnerability. If you think the case is different in your environment or there are some details that can shared on this issue, please reply to this thread or send a support request and we will be happy to analyze and resolve the issue.

Also, as a general note I want to mention,  Applications using Log4j 1.x are only vulnerable to this attack when they use JNDI in their configuration, we don't use any such configuration in our Application.

 

Thanks,

Vinoth Manoharan

Like (0) Reply

Hi Vinoth,

Let me verify further. But what you had provided is only a workaround since it is still on the older log4j 1.x.

The Tenable scan also found another vulnerability on "Apache Log4j Unsupported Version Detection".

The recommendation is to upgrade to the latest version of log4j - Refer to logging.apache.org/log4j/2.x/security.html for the latest versions. 

Can you advise if there is a plan in the pipeline to upgrade the NetworkPlus log4j to the latest version?

Rgds,

KoonYam

Like (0) Reply

Hi KoonYam,

Yes, I agree that the recommendation from Apache is to migrate to the latest version of log4j. The work to migrate log4j 1.x in NetworkPlus to the latest log4j2.x jar is already in our roadmap and our product team is working on this; we are expecting this to be completed by the second quarter of this year.

 

Thanks,

Vinoth

Site24x7 Team

Like (0) Reply

Hello Vinoth,

 

you previously stated that Log4j 2.x would be implemented in a new version of On-Premise Poller during Q2?  

 

Can you provide an update on this work since we are now in Q3?  

 

We are still showing vulnerabilities:

Apache Log4j Unsupported Version Detection (156032)

Apache Log4j 1.x Multiple Vulnerabilities (156860)

 

Path : C:\Program Files (x86)\Site24x7OnPremisePoller\NetworkPlus\lib\log4j-1.2.8.jar
Installed version : 1.2.8

 

 

I understand your prior statements say Site24x7 code is not impacted by security flaws, but it is still EOL software.   Our security team is very interested to see these critical vulnerabilities remediated.

 

Thank you

Brad

Like (0) Reply

Hello Brad,

I can understand the problem in using an older version of log4j. I would like to inform you that our team is working on this, and the delay is because of the complexity of migrating the legacy code, and I can assure you that the changes will get completed within two months' time.

I would like to inform you that the older version of log4j is only used in NetworkPlus Module, and we have migrated the log4j to the latest version in Site24x7 On-Premise Poller (version 5.1.4). Release Notes Link.

If you are not using Site24x7 Network Monitoring, I suggest you update On-Premise Poller to the latest version and remove the NetworkPlus Folder under On-Premise Poller. Here are the steps to remove Network Module from On-Premise Poller:

1. Update On-Premise Poller to the latest version.

2. Stop the On-Premise Poller.

3. Verify there are no existing processes using the link
https://support.site24x7.com/portal/en/kb/articles/processes-in-on-premise-poller

4. Delete the contents of the folder "Network/" or "NetworkPlus/" in the On-Premise Poller installed directory.

5. Start the On-Premise Poller service.

 

Thanks, 

Vinoth

Like (0) Reply

Hi, 

Any updates on this?

Like (0) Reply

Hi Eric,

We are currently working on a new On-Premise Poller version which will have migration support. It is in the testing phase and will be released soon.

In the meanwhile, feel free to contact our support team at support@site24x7.com with the details, as we can help you with the manual upgrade of the On-Premise Poller.

Best,
Rama

Like (0) Reply

Hi Vinoth,

What is the current status on this matter? Do we need to apply any patches ourselves manually or are these being automatically pushed out to pollers and agents? 

Is it possible I could be provided with a version number in which the patch is contained so that I can check our monitoring?

Thanks,

Mason

Like (0) Reply

Hi Mason Richards,

The Poller and Site24x7 APM java Agent are using log4j 1.x which is not affected by the vulnerability. So a patch is not required. NO ACTION IS REQUIRED FROM YOUR END.

I'll add a few more details regarding the two vulnerabilities and why our agents are not affected

CVE-2021-4104 - applications using Log4j 1.x may be impacted if their configuration uses JNDI (Site24x7 doesn't use any such configurations or JMSAppender)

CVE-2019-17571 - This vulnerability occurs only if the application uses SocketServer to listen for network traffic log data and deserialize the same. (Site24x7 doesn't use SocketServer).

We use log4j for basic logging functionality.

However, because of the EOL status of the log4j version used in our software, we are planning to upgrade it to the latest recommended log4j version and release it as a new version rather than a patch.

To update you on the current status, we have started the works on updating the log4j version to the latest recommended one, and also we have to do a quality check to ensure all the components are working properly.

I'll update this thread, once the updated version is available.

Thanks & Regards,

Vinoth

Site24x7 Red Team

Like (0) Reply

Our vulnerability scans flagged the on-prem poller for 2021-4104 JMSAppender. You stated you all do not use JMSAppender but this config file (C:\Program Files (x86)\Site24x7OnPremisePoller\conf\log4j.properties) appears to show otherwise:

# Log config for GeneralReportCollector

log4j.logger.GeneralReportCollector=DEBUG, generalreportcollectorappender
log4j.additivity.GeneralReportCollector=false
log4j.appender.generalreportcollectorappender=org.apache.log4j.RollingFileAppender
log4j.appender.generalreportcollectorappender.MaxFileSize=5MB
log4j.appender.generalreportcollectorappender.MaxBackupIndex=10
log4j.appender.generalreportcollectorappender.File=logs/generalreportcollector.log
log4j.appender.generalreportcollectorappender.layout=org.apache.log4j.PatternLayout
log4j.appender.generalreportcollectorappender.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n


# Log config for ImmediateReportCollector

log4j.logger.ImmediateReportCollector=DEBUG, immediatereportcollectorappender
log4j.additivity.ImmediateReportCollector=false
log4j.appender.immediatereportcollectorappender=org.apache.log4j.RollingFileAppender
log4j.appender.immediatereportcollectorappender.MaxFileSize=5MB
log4j.appender.immediatereportcollectorappender.MaxBackupIndex=10
log4j.appender.immediatereportcollectorappender.File=logs/immediatereportcollector.log
log4j.appender.immediatereportcollectorappender.layout=org.apache.log4j.PatternLayout
log4j.appender.immediatereportcollectorappender.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n

# Log config for ReportSender

log4j.logger.ReportSender=DEBUG, reportsenderappender
log4j.additivity.ReportSender=false
log4j.appender.reportsenderappender=org.apache.log4j.RollingFileAppender
log4j.appender.reportsenderappender.MaxFileSize=5MB
log4j.appender.reportsenderappender.MaxBackupIndex=10
log4j.appender.reportsenderappender.File=logs/reportsender.log
log4j.appender.reportsenderappender.layout=org.apache.log4j.PatternLayout
log4j.appender.reportsenderappender.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n

# Log config for GeneralReportSender

log4j.logger.GeneralReportSender=DEBUG, generalreportsenderappender
log4j.additivity.GeneralReportSender=false
log4j.appender.generalreportsenderappender=org.apache.log4j.RollingFileAppender
log4j.appender.generalreportsenderappender.MaxFileSize=5MB
log4j.appender.generalreportsenderappender.MaxBackupIndex=10
log4j.appender.generalreportsenderappender.File=logs/generalreportsender.log
log4j.appender.generalreportsenderappender.layout=org.apache.log4j.PatternLayout
log4j.appender.generalreportsenderappender.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n

# Log config for ImmediateReportSender

log4j.logger.ImmediateReportSender=DEBUG, immediatereportsenderappender
log4j.additivity.ImmediateReportSender=false
log4j.appender.immediatereportsenderappender=org.apache.log4j.RollingFileAppender
log4j.appender.immediatereportsenderappender.MaxFileSize=5MB
log4j.appender.immediatereportsenderappender.MaxBackupIndex=10
log4j.appender.immediatereportsenderappender.File=logs/immediatereportsender.log
log4j.appender.immediatereportsenderappender.layout=org.apache.log4j.PatternLayout
log4j.appender.immediatereportsenderappender.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n

# Log config for FailedReportSender

log4j.logger.FailedReportSender=DEBUG, failedreportsenderappender
log4j.additivity.FailedReportSender=false
log4j.appender.failedreportsenderappender=org.apache.log4j.RollingFileAppender
log4j.appender.failedreportsenderappender.MaxFileSize=5MB
log4j.appender.failedreportsenderappender.MaxBackupIndex=10
log4j.appender.failedreportsenderappender.File=logs/failedreportsender.log
log4j.appender.failedreportsenderappender.layout=org.apache.log4j.PatternLayout
log4j.appender.failedreportsenderappender.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n

#Log config for VMwareMasterDataCollector

log4j.logger.VMwareMasterDataCollector=DEBUG, vmwaremasterdatacollectorappender
log4j.additivity.VMwareMasterDataCollector=false
log4j.appender.vmwaremasterdatacollectorappender=org.apache.log4j.RollingFileAppender
log4j.appender.vmwaremasterdatacollectorappender.MaxFileSize=5MB
log4j.appender.vmwaremasterdatacollectorappender.MaxBackupIndex=10
log4j.appender.vmwaremasterdatacollectorappender.File=logs/vmwaremasterdatacollector.log
log4j.appender.vmwaremasterdatacollectorappender.layout=org.apache.log4j.PatternLayout
log4j.appender.vmwaremasterdatacollectorappender.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n


# initialize root logger with level INFO for stdout and fout
log4j.rootLogger=INFO,fout
log4j.logger.com.endeca=INFO
log4j.logger.com.endeca.itl.web.metrics=INFO

log4j.appender.fout=org.apache.log4j.RollingFileAppender
log4j.appender.fout.MaxFileSize=5MB
log4j.appender.fout.MaxBackupIndex=10
log4j.appender.fout.File=logs/pollerlog.log
log4j.appender.fout.layout=org.apache.log4j.PatternLayout
log4j.appender.fout.layout.ConversionPattern=%d{ISO8601}\t%p\t%c\t[%t]\t%m%n

Like (0) Reply

Hi,

The mentioned vulnerability is tracked under CVE-2021-4104. The vulnerability is affecting the JMSAppender.class but only under certain vulnerable configuration.

If you look at the shared log4j.properties, we use only RollingFileAppender and not the JMSAppender.  This vulnerability affect applications which are configured to use JMSAppender, which is not the default configuration.

So we can assure you the above configuration is safe and doesn't use JMSAppender.

Thanks,

Vinoth

Site24x7 Red Team

 

 

Like (0) Reply

Hi Vinoth,

    We have On-Premise Poller version 5.1.4. But we still see log4j version as 1.2.8

     site24x7/Site24x7OnPremisePoller/NetworkPlus/lib/log4j-1.2.8.jar

   As per release notes, this should have version 2.x

   This is being picked up by our scanner as vulnerable.

   Do you have any fix for this ?

 

Regards,

Lakshmi

Like (0) Reply

Hi Lakshmi Priya,

The version of log4j-1.2.8.jar used in the Site24x7 Network Plus module is protected against this security issue, we have removed all the vulnerable classes from the log4j-1.2.8.jar. Hence the exploitation is not possible and a manual security fix/patching is not required from your end.

You can find the details of the mitigation measure we have taken in this comment, As per the recommendation from this link

The migration to the latest version of log4j is also underway and we will soon release a version with the latest version of log4j.

 

Thanks,

Vinoth

Like (0) Reply

Hi Vinoth,

Has the Log4j 1.2.8 been upgraded in the NetworkPlus module with the release of 5.3.1 or are we still waiting for a update?

 

Thanks 

Like (0) Reply

Hi Shaheen,

New installations of the On-Premise Poller NetworkPlus module will include the latest jar. However, the existing On-Premise Poller NetworkPlus modules still have the older versions of the jar and do not have an upgrade option yet. We are working on migration support for our existing On-Premise Pollers. It is in the testing phase and will be released soon.

Meanwhile, feel free to contact our support team at support@site24x7.com with the details. Our team will be able to help you with the manual upgrade of the On-Premise Poller.

Best,
Rama

Like (0) Reply

Was this post helpful?