Graylog Access Logs - I'm lost

Before you post: Your responses to these questions will help the community help you. Please complete this template if you’re asking a support question.
Don’t forget to select tags to help index your topic!

1. Describe your incident:

I need the ability to log access attempts made against the graylog web interface for ISO cert purposes. By default, it does not seem Graylog does this. I’ve attempted the solution found here.. Unfortunately, I’ve had no luck logging the information or getting the local server.log file into graylog.

2. Describe your environment:

  • OS Information:
    Ubuntu 22.04 in a Linux container

  • Package Version:

  • Service logs, configurations, and environment variables:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration packages="org.graylog2.log4j" shutdownHook="disable">
    <Appenders>
        <RollingFile name="rolling-file" fileName="/var/log/graylog-server/server.log" filePattern="/var/log/graylog-server/server.log.%i.gz">
            <PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} %-5p [%c{1}] %m%n"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="50MB"/>
            </Policies>
            <DefaultRolloverStrategy max="10" fileIndex="min"/>
        </RollingFile>

        <!-- Internal Graylog log appender. Please do not disable. This makes internal log messages available via REST calls. -->
        <Memory name="graylog-internal-logs" bufferSize="500"/>
    </Appenders>
    <Loggers>
        <!-- RestAccessLogFilter -->
        <Logger name="org.graylog2.rest.accesslog" level="debug" additivity="false">
                <AppenderRef ref="rolling-file" level="debug"/>
                <AppenderRef ref="STDOUT" level="info"/>
        </Logger>
        <!-- Application Loggers -->
        <Logger name="org.graylog2" level="info"/>
        <Logger name="com.github.joschi.jadconfig" level="warn"/>
        <!-- Prevent DEBUG message about Lucene Expressions not found. -->
        <Logger name="org.elasticsearch.script" level="warn"/>
        <!-- Disable messages from the version check -->
        <Logger name="org.graylog2.periodical.VersionCheckThread" level="off"/>
        <!-- Silence chatty natty -->
        <Logger name="com.joestelmach.natty.Parser" level="warn"/>
        <!-- Silence Kafka log chatter -->
        <Logger name="org.graylog.shaded.kafka09.log.Log" level="warn"/>
        <Logger name="org.graylog.shaded.kafka09.log.OffsetIndex" level="warn"/>
        <Logger name="org.apache.kafka.clients.consumer.ConsumerConfig" level="warn"/>
        <!-- Silence useless session validation messages -->
        <Logger name="org.apache.shiro.session.mgt.AbstractValidatingSessionManager" level="warn"/>
        <Root level="warn">
            <AppenderRef ref="rolling-file"/>
            <AppenderRef ref="graylog-internal-logs"/>
        </Root>
    </Loggers>
</Configuration>

^Relevant portion in Bold

.Conf file in rsyslog.d subdirectory

#/etc/rsyslog.conf
$ModLoad imfile
$InputFileName /var/log/graylog-server/server.log
$InputFileTag graylog-server
$InputFileStateFile stat-graylog-server
$InputFileSeverity info
$InputFileFacility local3
$InputRunFileMonitor
local3.* action(type="omfwd" target="192.168.128.245" port="56234" protocol="udp" action.resumeRetryCount="100" queue.type="linkedlist" queue.size="10000" template="RSYSLOG_SyslogProtocol23Format")

Input even says messages are coming in:

But when you click on “Show received messages” there is nothing displayed

3. What steps have you already taken to try and solve the problem?
Hours of google-foo

4. How can the community help?
Tell me what I am doing wrong? Is there a better way?

Helpful Posting Tips: Tips for Posting Questions that Get Answers [Hold down CTRL and link on link to open tips documents in a separate tab]

Hi,
you could setup Graylog Sidecar and point it to the log file.
Docs: Graylog Sidecar
This should work.

Sascha

Hey @accidentaladmin, sorry you’re having trouble. My instructions are fairly old. It is pretty ridiculous that this functionality isn’t available in the product already. For what it’s worth, I have never touched the configuration again and it is still working. I just tested and got an alert as expected.

You mention that when you show received messages nothing is displayed. Is it possible that the time zone on your host is not correct? If so, then when you show received messages expand the search window to be much larger, several hours ahead or behind of ‘now’, to see if they have been indexed well into the future or the past.

1 Like

Appreciated the responses!
Unfortunately, the result is the same whether I use Sidecar or @ttsandrew’s solution.

image

And yes, it is ridiculous that this functionality isn’t built in.

Hey @accidentaladmin
Ive been doing this for a while also, since GL version 2.4. somethings have change like there use to be the user name in Access logs but now it UUID

Before

2023-01-03 16:55:08,999 DEBUG: org.graylog2.rest.accesslog - my_ipaddress greg.smith [-] “GET api/plugins/org.graylog.plugins.license/licenses/status/for-subject?subject=/license/enterprise/illuminate” Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 404 -1

Now

2023-01-03 16:55:08,999 DEBUG: org.graylog2.rest.accesslog - my_ipaddress 5e224e7683d72eff75055199 [-] “GET api/plugins/org.graylog.plugins.license/licenses/status/for-subject?subject=/license/enterprise/illuminate” Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 404 -1

If this is what you want and create a widget like this…

That was made from the file called restaccess.log
using this doc’s here

You will need a lookup table for the UUID’s and attach it to the input that graylog send those logs to.

I would love to do something like that and the link you referenced is actually the one I used but I just can’t get the data to show irrespective of whether I use sidecar or rsyslog. I guess there is a larger issue. Why does Graylog say I am getting data but does not produce any messages?

Hello @accidentaladmin

Yeah this will be long demo.

Ok so here is what I have.

CentOS 7 , GL 4.3, Mongo 4.4 Opensearch 1.3

log4j2.xml file

[root@graylog server]# cat log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration packages="org.graylog2.log4j" shutdownHook="disable">
    <Appenders>
        <RollingFile name="rolling-file" fileName="/var/log/graylog-server/server.log" filePattern="/var/log/graylog-server/server.log.                                                                            %i.gz">
            <PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX} %-5p [%c{1}] %m%n"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="50MB"/>
            </Policies>
            <DefaultRolloverStrategy max="10" fileIndex="min"/>
        </RollingFile>

        <!-- Internal Graylog log appender. Please do not disable. This makes internal log messages available via REST calls. -->
        <Memory name="graylog-internal-logs" bufferSize="500"/>

        <!-- Simple appender that writes access log to specified file -->
        <File name="RestAccessLog" fileName="/var/log/graylog-server/restaccess.log" append="true">
            <PatternLayout pattern="%d %-5p: %c - %m%n"/>
        </File>

    </Appenders>
    <Loggers>
         <!-- RestAccessLogFilter -->
        <Logger name="org.graylog2.rest.accesslog" level="debug" additivity="false">
                <AppenderRef ref="RestAccessLog" level="debug"/>
                <AppenderRef ref="AUDITLOG"/>
        </Logger>
        <!-- Application Loggers -->
        <Logger name="org.graylog2" level="info"/>
        <Logger name="com.github.joschi.jadconfig" level="warn"/>
        <!-- Prevent DEBUG message about Lucene Expressions not found. -->
        <Logger name="org.elasticsearch.script" level="warn"/>
        <!-- Disable messages from the version check -->
        <Logger name="org.graylog2.periodical.VersionCheckThread" level="off"/>
        <!-- Silence chatty natty -->
        <Logger name="com.joestelmach.natty.Parser" level="warn"/>
        <!-- Silence Kafka log chatter -->
        <Logger name="org.graylog.shaded.kafka09.log.Log" level="warn"/>
        <Logger name="org.graylog.shaded.kafka09.log.OffsetIndex" level="warn"/>
        <Logger name="org.apache.kafka.clients.consumer.ConsumerConfig" level="warn"/>
        <!-- Silence useless session validation messages -->
        <Logger name="org.apache.shiro.session.mgt.AbstractValidatingSessionManager" level="warn"/>
        <Root level="warn">
            <AppenderRef ref="rolling-file"/>
            <AppenderRef ref="graylog-internal-logs"/>
        </Root>
    </Loggers>
</Configuration>

Restart Graylog sevice and check /var/log/graylog-server/restaccess.log

Using Nxlog, doesnt matter what you use so long as your grabbing data from the rest file and sending it to Graylog, I perfer Nxlog or filebeat. In this example im using nxlog.

nxlog_config
<Extension _syslog>
    Module      xm_syslog
</Extension>
<Extension _json>
    Module      xm_json
</Extension>
<Extension _gelf>
    Module      xm_gelf
</Extension>

<Input access>
    Module       im_file
    FILE         "/var/log/graylog-server/restaccess.log"
    SavePos       TRUE
    ReadFromLast  TRUE
    PollInterval  1
    #Exec  $Message = $raw_event;
 </Input>

<Output out>
    Module      om_udp
    Host        192.168.1.100
    Port        51411
     Exec $ShortMessage = $raw_event;
</Output>

<Route>
    Path        access => out
</Route>

Data adaptor

Cache

INPUT syslog UDP make it globel to start with. I use my host ipaddress in nxlog becuz Im actually doing different iptables configuration /w network connections in my lab.

NOTE this can be done with Pipeline, but in this example Im using Regex extractor /w Converter

Last… This is the part that Graylog places a random UUID on user logged in. to Match those UUID I had ONLY the users that are allowed in our invironment to login and then I captured those UUID’s and made my CSV file.

Example This probably should be done prior to making your Lookup table , Ooop’s

[root@graylog graylog]# cat lookup_users.csv
"hash","somename"
"5e224e7683d72eff75055199","greg"
"5c6749e483d72e66f4815937","graylog-sidecar"
"5d6ed64583d72ed32385e17f","grant"
"/6035b016a0f28f5e0b86cad5","linux_dashboard"
"62ad4ecd777f8d5ed9cd6774","bruno"
[root@graylog graylog]#

NOTE: you can also go into MongoDb and grab all the users ID’ d like so…

MongoDb
root# mongo
> use graylog
> show collections;
sidecar_collectors
sidecar_configuration_variables
sidecar_configurations
sidecars
streamrules
streams
system.profile
system_messages
team_sync_backend_configs
teams
traffic
users
view_sharings
views
>
>db.users.find({},{"full_name":1,"ObjectId":1}); <--- this would be key

OR grap all the metadata from Mongodb/Graylog on users and place it in a file for reference

mongoexport  -u mongo_admin -p  primalFear  --collection=users --db=graylog --out=/var/log/streams/users.json 

Done.

For free you have to put the work into it, but you can pay or use the Operation license if you under 2Gb a day it free.

But wait!! Theres more :laughing:

Here are some pipes I have for this project

rule "Graylog Web Access User Convert"
when
    contains(to_string($message.message),"UID=")
then
    // create a local to make it slightly less intensive to work with
    let robin = to_string($message.message);
   
    let batman = replace(robin, "1914600003", "steveno");
    let batman = replace(robin, "100100110010011001001",    "tad.sherrill");
    let batman = replace(robin, "987324e32874ff32892b2829", "steven.o");

   // changed the number of digits count to be required between 6 and 10   {6,10}
    let batman = regex_replace("(?<=UID\\=)(?>\\d{6,10})",robin,"USER_NOT_FOUND",false);

    set_field("message",batman);
end

The newest Pipe .

This is a 2 stage on using the lookup table, this would replace the extractor from above.

[Stage 0]
rule "Grab UUID of the username"
when 
has_field ("message")
then
 let user = regex("(.[0\\-9a\\-fA\\-F]{24})","to_string($message.message))");
 set_field("graylog","*");
end

[Stage 1]
rule"user-lookup"
when
 has_field("graylog")
then
let user_id = lookup_value("active user",$message.graylog);
set_field("someone",user_id);
end

Hope that helps

This is amazing and thank you, so much! You make an excellent point re: Operations. Between you me and the doorpost, my employer needs to make the investment (we ingest ~15GB daily :face_vomiting:). I AM the IT department and am responsible for so, so much. Mastering Graylog (plus the underlying resources, e.g. Mongo and Elastic) is just not possible. For instance, my ElasticSearch is throwing up warnings I couldn’t hope to have time to resolve right now.

/copypastasysadminrant

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.