Input received, won't display/search

Graylog Central (peer support)
1

This is doubtless something really dumb on my part…

I have a fortigate FW sending logs to graylog server (v 4.0.7, Elastic search 7.10.2-1, mongodb 4.2.14, all on Centos 7.9.2009) Utilizing fortigate6.4_graylog4 content pack.

Looking at the “input” I see:

Throughput / Metrics

1 minute average rate: 3 msg/s
Network IO: 732.0B 0B (total: 1.3MiB )
Empty messages discarded: 0

However, if click on show received messages I get:
While retrieving data for this widget, the following error(s) occurred:

  • Elasticsearch exception [type=index_not_found_exception, reason=no such index ].

Hmmm,
Trying “Message Extractors → Add Extractor → Get Started → Load Message” (where I should be able to see a message that has come in) I get:
ERROR
Input did not return a recent message

I’m sure I have just configured something wrong but I don’t know what.

Suggestions would be appreciated

2

Hello and Welcome

I’m not 100% sure but you may need to set your action.auto_create_index parameter in elasticsearch.yml file. What does the ES log file show?

3

Nothing interesting in the logs that I can see…
also, the install instructions for Centos specifically said to do this for elasticsearch:
action.auto_create_index: false
tail -50 gc.log
[2021-05-18T07:43:52.294+0000][25023][gc,age ] GC(60) - age 11: 1821792 bytes, 33597336 total
[2021-05-18T07:43:52.294+0000][25023][gc,age ] GC(60) - age 12: 1757712 bytes, 35355048 total
[2021-05-18T07:43:52.295+0000][25023][gc,phases ] GC(60) Pre Evacuate Collection Set: 0.2ms
[2021-05-18T07:43:52.295+0000][25023][gc,phases ] GC(60) Merge Heap Roots: 0.2ms
[2021-05-18T07:43:52.295+0000][25023][gc,phases ] GC(60) Evacuate Collection Set: 7.8ms
[2021-05-18T07:43:52.295+0000][25023][gc,phases ] GC(60) Post Evacuate Collection Set: 1.2ms
[2021-05-18T07:43:52.295+0000][25023][gc,phases ] GC(60) Other: 0.5ms
[2021-05-18T07:43:52.295+0000][25023][gc,heap ] GC(60) Eden regions: 469->0(469)
[2021-05-18T07:43:52.295+0000][25023][gc,heap ] GC(60) Survivor regions: 42->38(64)
[2021-05-18T07:43:52.295+0000][25023][gc,heap ] GC(60) Old regions: 173->179
[2021-05-18T07:43:52.295+0000][25023][gc,heap ] GC(60) Archive regions: 2->2
[2021-05-18T07:43:52.295+0000][25023][gc,heap ] GC(60) Humongous regions: 51->42
[2021-05-18T07:43:52.295+0000][25023][gc,metaspace] GC(60) Metaspace: 79080K(81924K)->79080K(81924K) NonClass: 69481K(71576K)->69481K(71576K) Class: 9599K(10348K)->9599K(10348K)
[2021-05-18T07:43:52.295+0000][25023][gc ] GC(60) Pause Young (Normal) (G1 Evacuation Pause) 735M->258M(1024M) 10.037ms
[2021-05-18T07:43:52.295+0000][25023][gc,cpu ] GC(60) User=0.10s Sys=0.01s Real=0.01s
[2021-05-18T07:43:52.295+0000][25023][safepoint ] Safepoint “G1CollectForAllocation”, Time since last: 588944379099 ns, Reaching safepoint: 171605 ns, At safepoint: 10187808 ns, Total: 10359413 ns
[2021-05-18T07:44:25.319+0000][25023][safepoint ] Safepoint “Cleanup”, Time since last: 33003492101 ns, Reaching safepoint: 20580614 ns, At safepoint: 8448 ns, Total: 20589062 ns
[2021-05-18T07:51:21.366+0000][25023][safepoint ] Safepoint “Cleanup”, Time since last: 416047271838 ns, Reaching safepoint: 209964 ns, At safepoint: 10335 ns, Total: 220299 ns
[2021-05-18T07:52:03.372+0000][25023][safepoint ] Safepoint “Cleanup”, Time since last: 42004969900 ns, Reaching safepoint: 311527 ns, At safepoint: 16094 ns, Total: 327621 ns
[2021-05-18T07:54:23.571+0000][25023][gc,start ] GC(61) Pause Young (Normal) (G1 Evacuation Pause)
[2021-05-18T07:54:23.571+0000][25023][gc,task ] GC(61) Using 13 workers of 13 for evacuation
[2021-05-18T07:54:23.571+0000][25023][gc,age ] GC(61) Desired survivor size 33554432 bytes, new threshold 11 (max threshold 15)
[2021-05-18T07:54:23.580+0000][25023][gc,age ] GC(61) Age table with threshold 11 (max threshold 15)
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 1: 11256696 bytes, 11256696 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 2: 2810256 bytes, 14066952 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 3: 4797520 bytes, 18864472 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 4: 1658256 bytes, 20522728 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 5: 2222944 bytes, 22745672 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 6: 1738256 bytes, 24483928 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 7: 2212392 bytes, 26696320 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 8: 1791976 bytes, 28488296 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 9: 1997120 bytes, 30485416 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 10: 1631240 bytes, 32116656 total
[2021-05-18T07:54:23.581+0000][25023][gc,age ] GC(61) - age 11: 1700048 bytes, 33816704 total
[2021-05-18T07:54:23.581+0000][25023][gc,phases ] GC(61) Pre Evacuate Collection Set: 0.2ms
[2021-05-18T07:54:23.581+0000][25023][gc,phases ] GC(61) Merge Heap Roots: 0.2ms
[2021-05-18T07:54:23.581+0000][25023][gc,phases ] GC(61) Evacuate Collection Set: 7.2ms
[2021-05-18T07:54:23.581+0000][25023][gc,phases ] GC(61) Post Evacuate Collection Set: 1.8ms
[2021-05-18T07:54:23.581+0000][25023][gc,phases ] GC(61) Other: 0.5ms
[2021-05-18T07:54:23.581+0000][25023][gc,heap ] GC(61) Eden regions: 469->0(463)
[2021-05-18T07:54:23.581+0000][25023][gc,heap ] GC(61) Survivor regions: 38->36(64)
[2021-05-18T07:54:23.581+0000][25023][gc,heap ] GC(61) Old regions: 179->182
[2021-05-18T07:54:23.581+0000][25023][gc,heap ] GC(61) Archive regions: 2->2
[2021-05-18T07:54:23.581+0000][25023][gc,heap ] GC(61) Humongous regions: 66->49
[2021-05-18T07:54:23.581+0000][25023][gc,metaspace] GC(61) Metaspace: 79084K(81924K)->79084K(81924K) NonClass: 69484K(71576K)->69484K(71576K) Class: 9599K(10348K)->9599K(10348K)
[2021-05-18T07:54:23.581+0000][25023][gc ] GC(61) Pause Young (Normal) (G1 Evacuation Pause) 751M->267M(1024M) 9.961ms
[2021-05-18T07:54:23.581+0000][25023][gc,cpu ] GC(61) User=0.10s Sys=0.00s Real=0.01s
[2021-05-18T07:54:23.581+0000][25023][safepoint ] Safepoint “G1CollectForAllocation”, Time since last: 140198985439 ns, Reaching safepoint: 221064 ns, At safepoint: 10104406 ns, Total: 10325470 ns
[2021-05-18T07:55:04.326+0000][25023][safepoint ] Safepoint “Cleanup”, Time since last: 40744813286 ns, Reaching safepoint: 430729 ns, At safepoint: 24881 ns, Total: 455610 ns
[2021-05-18T07:58:33.351+0000][25023][safepoint ] Safepoint “Cleanup”, Time since last: 209024880638 ns, Reaching safepoint: 237673 ns, At safepoint: 14319 ns, Total: 251992 ns

tail graylog.log
[2021-05-18T03:28:57,294][INFO ][o.e.c.m.MetadataMappingService] [joshua.judahnet.net] [graylog_0/Eq6K4hmbQ5uJOTDDvAK9VA] update_mapping [message]

4

Hi there,

The version for Graylog 3.x

"
This Content Pack contains the following items:

  1. Input - Fortigate input
  2. Extractors - All fields as outlined by Fortinet documentation have a corresponding regex extractor
  3. Streams - Streams have been setup to align with the log views available on a FortiAnalyzer
  4. Dashboard - Limited 24 Hour summary dashboard
    "

Did you change anything in the content pack settings? Since is Using “Streams” settings you may need to create an Index for the Stream or use the correct index name? normaly is create one stream for input.

Thks

5

I am using graylog version 4.0.7 and the fortigate6.4_graylog4 says it supports v4, I have made no changes to the content pack settings as it says it supports the version of Graylog and the version of the OS on the Fortigate I am using

6

I’ve imported content pack to my test installation and works fine, so something bad happened in your installation. How did you setup graylog? Did you follow official docs?

7

Yes I did:
https://docs.graylog.org/en/4.0/pages/installation/os/centos.html

How is your fortigate configured ?
per the content pack mine looks like this:
config log syslogd2 setting
set status enable
set server “199.33.251.50”
set port 1514
set priority low
end
config log syslogd2 filter
set severity warning
end

8

So i did yum remove’s on all the packages, then deleted all the directories etc left behind as well as the cruft left in the password/shadow files…

Did a complete re-install using the same directions and it works. I have no idea why, but it’s working !

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.