New user woes and whining

Graylog Central (peer support)
1

Morning all,

#whoami: Old nerd, 54, Reg’d Linux User #247167 (since 1995), network and network security, homelabber, etc.

I’m currently a Grafana/Loki user and I generally like most of it but the performance. Getting my firewall dashboard loaded for anything more than 6 hours backwards is pretty much impossible. It will be either be super slow or Loki will fall over.
Recently, I have seen a video on YouTube about Graylog from Awesome Open Source and thought, I give it a try. I installed a single node on an RK3588 ARM64 box with SSDs and am piping my OpenWRT home edge router’s logs into it. Performance looks pretty nice so far. My dashboard is much snappier than with Grafana/Loki, even for 24 hours back and it doesn’t seem to skip log lines (which Loki does when aggregating, e.g. count over a bigger amount of lines).
I hate the RAM demands, but I guess, that is owed to it being Java…

Some things I don’t get though, e.g. why is OIDC an Enterprise feature but LDAP/AD is included in the open version??? I’d say, a homelabber or small business is much more likely to have some OIDC like e.g. Authentik running than some M$ crap or something wildly overpowered such as LDAP. That’s almost a K.O.-criteria.

I also find the API documentation fairly complicated. I guess, it’s aimed at professional full-time developers who are familiar with all the terms and such. Searching through the forum posts at least gives me the impression that I’m not alone with pondering on how to get a simple search going.

On that note… I’m currently publishing 2 blocklists to Github: IPs which tried to attack my SIP pbx (fraudulent call and auth attempts) and IPs which tried to attack my Postfix MTA (auth attempts). Both are realized by a shell script querying logs from my Loki instance.
If anybody has a link to a “Graylog API log queries for non-developers” -style tutorial, that would be greatly appreciated!

Have a great weekend!
Stefan

2

Howdy and welcome!

Regarding your quesitons:

why is OIDC an Enterprise feature but LDAP/AD is included in the open version???

At the end of the day there needs to be a way for the Developers to get paid. For better or worse, the standard is typically to allow LDAP and to charge for more enterprise focused needs, such as Group Management and 2FA/MFA. I’m not saying i agree/disagree with this but Graylog is not unique in how it approaches this. As to why LDAP? Its an open standard and you can more or less swap out LDAP with Active Directory in terms of an authentication source.

API documentation fairly complicated

I’m afraid you are right. Technically i wouldn’t call what we have “documentation” but more so an API reference. I will say, the OpenAPI/Swagger browser accessible from any graylog node at /api/api-browser/ is really powerful and great for determining what API endpoint you want to use and to test it.

If anybody has a link to a “Graylog API log queries for non-developers” -style tutorial, that would be greatly appreciated!

I would check out both the /api/search/messages and /api/search/aggregate api endpoints depending on what your needs are:

/api/search/messages

Provides a simple api endpoint to return messages in json format.

Example:

GET /api/search/messages?query=_exists_%3Atimestamp&timerange=300&fields=timestamp%2Csource%2Cmessage

See the api browser for supported Parameters.

image
image856×503 59.3 KB

/api/search/aggregate

Provides a simple api to return an aggregation. In graylog speak, an aggregation is a grouping of data by fields. For example, in the Graylog UI, if you click on a field and to “Show Top Values” it outputs a table that shows each unique occurrence of all values of this field and how many messages use each unique field value. This is especially useful for returning only unique values.

Example:

GET /api/search/aggregate?query=_exists_%3Atimestamp&timerange=300&groups=source

See the api browser for supported Parameters.

image
image859×486 31.8 KB

Quick blurb about API authentication

The Graylog API uses HTTP Basic authentication, meaning it will accept the username/password of any user that has the roles/privleges needed by the applicable API endopint.

While you can use a graylog username/password, I strongly recommend to use a token.

This allows you to decouple your authentication credentials from your actual user and makes revoking/reissuing much simpler.

Lastly, this is a bit counterintuitive, but when using an api token with HTTP basic auth, the token value is used as the username, and the literaly word token is used as the password.

Example using curl:

curl -v -u "okgzlvzupcjwixktqdgzblenbqh64ebccna8y2qgxoqyf9kn6ghp:token"  "<graylog_uri>/api/search/aggregate?query=_exists_%3Atimestamp&timerange=300&groups=source"

Let me know if you have any questions!

3

Hey Drew, thanks for your reply and especially the extensive help to get started with the API! I’ll start playing with it - let’s see how far I get :smiley: .

Yeah, I get that the money needs to come from somewhere.
But let me make my case :slight_smile:
I switched to Authentik for SSO/OAUTH/OIDC at home 9 or so months ago. I run a lot at home. Nextcloud, mail (Postfix), media server, you know, the usual stuff for selfhosters/homelabbers plus some fancy stuff. Users are family and close friends. At some point, the account management for all those services plus the management layers (Portainer, etc, etc.) gets a little tedious, especially when most of your “users” are not technically inclined. As pretty much everything in my home rack supports OAUTH/OIDC - even the Jellyfin media server - I decided to use that. First I was thinking using Nextcloud as OAUTH backend, which doesn’t require much extra config - but in the end, after watching another Awesome Open Source video, I looked into Authentik and decided to use that because it has additional integrations such as Radius (yay, same account for my switches and access points!). During this process, I was really surprised how much OS software supports OAUTH/OIDC directly or through community plugins. And from the Authentik community I got the strong vibe that there’s plenty selfhosters and homelabbers there.
So in my perception, OAUTH/OIDC is a really popular thing for us “small” people while from my experience, I associate LDAP with really enterprise Enterprise stuff, you know, MS AD, huge companies, and so on. Hence my wondering. Maybe you could take that as feedback or so to the guys who make the decisions? :smiley:

Have a great weekend!
Stefan

4

Trusted Header Authentication may be of interest and/or use to you.

It basically allows you to use a proxy to handle the authentication so you can do pretty much anything from a simple HTTP Basic username/password to more complex things like having a web app auth with an external service.

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.