# Graylog and mail server **URL:** \ **Category:** Graylog Central (peer support)\ **Created:** [March 30, 2022, 4:04pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262 "2022-03-30T16:04:40Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [March 30, 2022, 4:04pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/1 "2022-03-30T16:04:41Z")
How to set graylog to read mai lservers log. i read mail log on graylog via filebeat. logs appear as normal log not a mail messages. what should i do/missing? ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [March 31, 2022, 12:16am UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/2 "2022-03-31T00:16:59Z")
Hello @Sharzad Not sure could you show more information on this issue? ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [March 31, 2022, 6:23am UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/3 "2022-03-31T06:23:05Z")
I have log from mail server and when i using filebeat to read this, it’s a lot of info not showing because of log format (i think) . so my question is how or which type of file input using for log for mail. ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [March 31, 2022, 9:20pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/4 "2022-03-31T21:20:29Z")
> [@Sharzad](#): > > I have log from mail server and when i using filebeat to read this, Sorry, I don’t see any logs that you posted. You could use \a couple different inputs. This depends on how your mail logs are formatted. The only Input I know of at this point that you could use is Raw/Plaintext. I’m quit sure that it would be useful. ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 1, 2022, 6:39am UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/5 "2022-04-01T06:39:08Z")
how do i combined this with filebeat?I mounted log file in /data/maillog/ then i need to use filebeat orwhat? ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 1, 2022, 9:31pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/6 "2022-04-01T21:31:52Z")
Hello, > [@Sharzad](#): > > how do i combined this with filebeat? If your using FileBeat as a log shipper to send MAIL logs to Graylog Server then you would use BEAT Input and configure it accordantly. Hope that helps ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 1, 2022, 11:31pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/7 "2022-04-01T23:31:51Z")
I For got to add an example of a FileBeat configuration. ```auto # Needed for Graylog fields_under_root: true fields.collector_node_id: ${sidecar.nodeName} fields.gl2_source_collector: ${sidecar.nodeId} filebeat.inputs: - input_type: log paths: - /var/log/*.log - /data/maillog/*.log <-- Example of what you might want. - /var/log/maillog <-- This configuration is on my server. type: log output.logstash: hosts: ["192.168.1.1:5044"] path: data: /var/lib/graylog-sidecar/collectors/filebeat/data logs: /var/lib/graylog-sidecar/collectors/filebeat/log ``` Hope that is a better example for you. ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 3, 2022, 11:39am UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/8 "2022-04-03T11:39:25Z")
I use the filebeat. i know the configuration of filebeat. it’s not the issue. My issue is when i add it as filebeat, the format is not what i can use it. there is nothing on messages. problem is i can’t see messages in graylog. but if i search som text (i know it in messages fx. regards …) then i have output. i mean it’s look like reading log file is not correct. ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 4, 2022, 2:20pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/9 "2022-04-04T14:20:53Z")
fx. there is almost nothing under item messages in graylog ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/0/0f49f337445918d11d47c39fc92e65737a1b574a.png) but in log file i have ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/e/e4cb17fc9b3e9c79f89c2a19f0211d449a827ac2.png) ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 4, 2022, 10:43pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/10 "2022-04-04T22:43:28Z")
Hello, The screen shot above looks like a letter to someone and also the File your sending to Graylog is not properly formatted for elasticsearch. This might be the issue but I’m just guessing at this point. If you send a _letter text file_ over to graylog server you may want to use **Raw/Plaintext INPUT** to see if it works, Not sure how to configure FileBeat to do that . Next I see the file path. `/var/lib/elasticsearch/log/nic.log` Not sure why you have a mail server in your elasticsearch library directory. I see the name of the file is nic.log??? It dos not resemble anything for Mail server log. perhaps NIC meaning it a interface log file?? Just a guess. ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 5, 2022, 6:59am UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/11 "2022-04-05T06:59:46Z")
I mount (read-only) mail log to graylog server and used filebeat (i move one to filebeat space) . if i use raw/plaintext, how to tell graylog where my log file are? ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 5, 2022, 9:19pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/12 "2022-04-05T21:19:02Z")
Hello @Sharzad > [@Sharzad](#): > > I mount (read-only) mail log to graylog server and used FileBeat (I move one to > filebeat space) Is it possible to show what you did? > [@Sharzad](#): > > if i use raw/plaintext, how to tell graylog where my log file are? I believe using FileBeat you need to use a BEAT input. Unfortunately, I’m not sure how you setup your environment. ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 5, 2022, 10:02pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/13 "2022-04-05T22:02:18Z")
please forget about the filebeat and beat input. how to configure raw/plaintext in graylog, to read a maillog? ---
**Author:** ![gsmith](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/gsmith/32/1222_2.png) [@gsmith](https://community.graylog.org/u/gsmith)\ **Post date:** [April 5, 2022, 10:31pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/14 "2022-04-05T22:31:13Z")
> [@Sharzad](#): > > how to configure raw/plaintext in graylog, to read a maillog? What are you using for a log shipper? This would depend on what input you can use. ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 6, 2022, 6:43am UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/15 "2022-04-06T06:43:45Z")
I nfs mount maillog files. is it matter? i have a email logfiles and i want to read in to graylog. ---
**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\ **Post date:** [April 6, 2022, 2:38pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/16 "2022-04-06T14:38:36Z")
If you are using filebeat going to Graylog beats input, you can modify the sidecar configuration to accept multi line messages such as the one you have shown above. There are some Elasticsearch documentation on doing that [here](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html). Reading through your post it is very difficult to answer the question. You haven’t posted much about your environment (text formatted with \ forum tool is much better than a screen shot wherever possible.) the more information you give about your environment, what you have tried etc. the easier it is to help you to a solution. Here are [tips on asking questions](https://community.graylog.org/t/tips-for-posting-questions-that-get-answers/21828) on the forum that also has some of the general diagnostic commands ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 6, 2022, 3:09pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/17 "2022-04-06T15:09:18Z")
ok no problem here is my env: ```auto Graylog 4.2.6 MongoDB v4.0.27 Elasticsearch 7.10.2 ``` and elasticsearch, mogodb and graylog installation ```auto dpkg -l | grep -E ".*(elasticsearch|graylog|mongo).*" ii elasticsearch-oss 7.10.2 amd64 Distributed RESTful search engine built for the cloud ii graylog-4.2-repository 1-4 all Package to install Graylog 4.2 GPG key and repository ii graylog-server 4.2.6-1 all Graylog server ii graylog-sidecar 1.1.0-1 amd64 Graylog collector sidecar ii mongodb-org 4.0.28 amd64 MongoDB open source document-oriented database system (metapackage) ii mongodb-org-mongos 4.0.28 amd64 MongoDB sharded cluster query router ii mongodb-org-server 4.0.28 amd64 MongoDB database server ii mongodb-org-shell 4.0.28 amd64 MongoDB shell client ii mongodb-org-tools 4.0.28 amd64 MongoDB tools ``` my conf file in graylog is: ```auto is_master = true node_id_file = /etc/graylog/server/node-id password_secret = my-pass root_password_sha2 = mypass root_timezone = UTC bin_dir = /usr/share/graylog-server/bin data_dir = /var/lib/graylog-server plugin_dir = /usr/share/graylog-server/plugin http_bind_address = ip:9000 http_bind_address = ipv6:9000 http_publish_uri = http://localhost.org rotation_strategy = count elasticsearch_max_docs_per_index = 200000000 elasticsearch_max_number_of_indices = 20 retention_strategy = delete elasticsearch_shards = 4 elasticsearch_replicas = 0 elasticsearch_index_prefix = graylog allow_leading_wildcard_searches = false allow_highlighting = false elasticsearch_analyzer = standard output_batch_size = 1000 output_flush_interval = 1 output_fault_count_threshold = 5 output_fault_penalty_seconds = 30 processbuffer_processors = 5 outputbuffer_processors = 3 processor_wait_strategy = blocking ring_size = 65536 inputbuffer_ring_size = 65536 inputbuffer_processors = 2 inputbuffer_wait_strategy = blocking message_journal_enabled = true message_journal_dir = /var/lib/graylog-server/journal lb_recognition_period_seconds = 3 mongodb_uri = mongodb://localhost/graylog mongodb_max_connections = 1000 mongodb_threads_allowed_to_block_multiplier = 5 gc_warning_threshold = 30s proxied_requests_thread_pool_size = 32 ``` ```auto cat /etc/elasticsearch/elasticsearch.yml | egrep -v "^\s*(#|$)" path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch cluster.name: graylog ``` ```auto cat /etc/graylog/sidecar/sidecar.yml | egrep -v "^\s*(#|$)" server_url: "http://myhostname/api/" server_api_token: "" tls_skip_verify: true ``` ```auto curl -XGET http://localhost:9200/_cluster/health?pretty=true { "cluster_name" : "graylog", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 68, "active_shards" : 68, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 16, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 80.95238095238095 ``` ```auto curl -XGET http://localhost:9200/_cluster/allocation/explain?pretty { "index" : "graylog_3", "shard" : 1, "primary" : false, "current_state" : "unassigned", "unassigned_info" : { "reason" : "CLUSTER_RECOVERED", "at" : "2022-04-01T11:25:52.012Z", "last_allocation_status" : "no_attempt" }, "can_allocate" : "no", "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes", "node_allocation_decisions" : [ { "node_id" : "pPiimofqSdSMFsgOuQjKiA", "node_name" : "hostname.domain", "transport_address" : "127.0.0.1:9300", "node_decision" : "no", "deciders" : [ { "decider" : "same_shard", "decision" : "NO", "explanation" : "a copy of this shard is already allocated to this node [[graylog_3][1], node[pPiimofqSdMMFagOuQjKiA], [P], s[STARTED], a[id=5qpoh5zUR8aMyzov78J_Bw]]" } ] } ] } ``` ```auto curl -XGET http://localhost:9200/_cat/indices?pretty green open gl-events_1 vce0mjFtTiC8a1iFkUs6TA 4 0 0 0 832b 832b green open gl-events_0 -M7yVuuQRw2UL_YvOqHKbQ 4 0 0 0 832b 832b green open gl-events_5 HKHnyWpfRC2r_d-z0bFu6Q 4 0 66546000 0 11.5gb 11.5gb green open gl-events_4 BC_0S_SXRwuF9IIGfatUjA 4 0 315045628 0 48gb 48gb green open gl-events_3 vr4jp5Q3SCWpCHv2FwTJCA 4 0 0 0 832b 832b green open gl-events_2 Lm4DCmEeSy6SDMAjLND5Mw 4 0 0 0 832b 832b green open gl-system-events_2 rVN8CxrPRt2Zm0-FSxcXbQ 4 0 0 0 832b 832b yellow open graylog_1 x2w9kfLWRaCGZXO859L2HQ 4 1 20549492 0 6gb 6gb green open gl-system-events_3 cHAwlO0gQPG5D1XojgmAnw 4 0 0 0 832b 832b green open graylog_0 mYBO3Vu5Q-icyJ98ijX2RA 4 0 20707250 0 6.2gb 6.2gb green open gl-system-events_0 bsivTyMySPCrTQA0dlV2VQ 4 0 0 0 832b 832b green open gl-system-events_1 Yn0Uo6XfQLSW8mF5H5_YZA 4 0 0 0 832b 832b yellow open graylog_4 EH_eODqGRAWD4s0e3sQ5tQ 4 1 4941486 0 1.5gb 1.5gb yellow open graylog_3 K8mu_z9KT1eb2mVblCe6xQ 4 1 20597728 0 5.9gb 5.9gb green open gl-system-events_4 0w46xyqfRsShPOA0RF9V5Q 4 0 0 0 832b 832b yellow open graylog_2 2ilDajsjSNyBn7caMjJDVw 4 1 20688957 0 5.3gb 5.3gb green open gl-system-events_5 w1OpxMXzQt6SGwInLCfrqQ 4 0 0 0 832b 832b ``` ---
**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\ **Post date:** [April 6, 2022, 5:42pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/18 "2022-04-06T17:42:17Z")
OK - That’s the general data. Although I appreciate it, you didn’t need to follow the entire post… 😝 What I needed you to glean from that is how to ask you initial question in the future… what is usually needed is targeted information… I should have been more specific. Also - PLEASE use the formatting tool `` when you are posting code by highlighting the code then clicking on the `` tool ![image](https://us1.discourse-cdn.com/flex016/uploads/graylog/original/2X/0/05131eebc35f850aec894b437e97a64e16760362.png) as example, I will reformat your last post with it… it makes things MUCH easier to decipher. It appears you have a filebeat input and you are receiving data from the filebeat sidecar but it is not in the format that you want? If the format you want is multi-line messages like the e-mail example , the elasticsearch link I posted previously is a good start to research how to pull in multi-line messages. You just have to put together a filebeat configuration in Graylog using the elastic file multiline parameters. Give it a try putting in the parameters from the elasticsearch link, make sure to check the sidecare and filebeat logs on the client, they usually tell you exactly what the issue is (if any) if you run into further issues, post your sidecar configuration (using `` !!!) and comment on things you have tried and describe where your issue is as specifically as possible and with as much relevant information you can. ---
**Author:** ![tmacgbay](https://sea2.discourse-cdn.com/flex016/user_avatar/community.graylog.org/tmacgbay/32/3878_2.png) [@tmacgbay](https://community.graylog.org/u/tmacgbay)\ **Post date:** [April 6, 2022, 5:55pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/19 "2022-04-06T17:55:03Z")
On a side note, you have some issues in your Elasticsearch that you should address: ```auto yellow open graylog_1 x2w9kfLWRaCGZXO859L2HQ 4 1 20549492 0 6gb 6gb ... yellow open graylog_4 EH_eODqGRAWD4s0e3sQ5tQ 4 1 4941486 0 1.5gb 1.5gb yellow open graylog_3 K8mu_z9KT1eb2mVblCe6xQ 4 1 20597728 0 5.9gb 5.9gb ... yellow open graylog_2 2ilDajsjSNyBn7caMjJDVw 4 1 20688957 0 5.3gb 5.3gb ``` This is showing on viewing allocation: ```auto "can_allocate" : "no", "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes", "node_allocation_decisions" : [ ``` If you google [elasticsearch “cannot allocate because allocation is not permitted to any of the nodes”](https://www.google.com/search?q=elasticsearch+%22cannot+allocate+because+allocation+is+not+permitted+to+any+of+the+nodes%22&rlz=1C1GCEU_enUS960US960&oq=elasticsearch+%22cannot+allocate+because+allocation+is+not+permitted+to+any+of+the+nodes%22&aqs=chrome..69i57j0i30j0i5i30j0i8i30.8241j0j7&sourceid=chrome&ie=UTF-8) you will find the [first link](https://stackoverflow.com/questions/65317998/elasticsearch-cannot-allocate-because-allocation-is-not-permitted-to-any-of-th) shows that you have replicas that are unallocated… (the light blue area that shows “`4 1`”… 4 shards, 1 replica) likely because you are running one server for Graylog. There is a command you can run to set replicas to zero - **if** - you don’t want/need them that I have handy - it looks something like this: `curl -XPUT -H 'Content-Type: application/json' http://:9200//_settings?pretty -d '{"number_of_replicas":0}'` ---
**Author:** ![Sharzad](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Sharzad](https://community.graylog.org/u/Sharzad)\ **Post date:** [April 6, 2022, 9:47pm UTC](https://community.graylog.org/t/graylog-and-mail-server/23262/20 "2022-04-06T21:47:05Z")
yes i have filebeat from one server, format and everything is ok from this input. what i want is i have maillog from other server when i mount nfs to graylog server. i don’t want to use filebeat for this input, because format is not match. i want to use raw/plaintext. how to create it i mean how to say where logs place are? [Next page](https://community.graylog.org/t/graylog-and-mail-server/23262.md?page=2)