I try to start development of a new Graylog plugin for OpenID Connect authentication/authorization but fail to get my development environment running properly.
Building Graylog from the current master (commit e8ee0fb1b08d3c26f637c021e27cc0206246e20c) fails:
[INFO] --------------------< org.graylog2:graylog2-server >--------------------
[INFO] Building Graylog 3.2.0-SNAPSHOT [3/7]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ graylog2-server ---
[INFO] Executing: /bin/sh -c cd '/home/jan/projects/graylog/graylog-project-repos/graylog2-server/graylog2-server' && 'git' 'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: /home/jan/projects/graylog/graylog-project-repos/graylog2-server/graylog2-server
[INFO] Storing buildNumber: e8ee0fb1b08d3c26f637c021e27cc0206246e20c at timestamp: 1570352232575
[INFO] Storing buildScmBranch: master
[INFO]
[INFO] --- antlr4-maven-plugin:4.7.1:antlr4 (antlr) @ graylog2-server ---
[INFO] No grammars to process
[INFO] ANTLR 4: Processing source directory /home/jan/projects/graylog/graylog-project-repos/graylog2-server/graylog2-server/src/main/antlr4
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @ graylog2-server ---
[INFO]
[INFO] --- frontend-maven-plugin:1.6:install-node-and-yarn (install node and yarn) @ graylog2-server ---
[INFO] Node v10.13.0 is already installed.
[INFO] Installing Yarn version v1.12.3
[INFO] Unpacking /home/jan/.m2/repository/com/github/eirslett/yarn/1.12.3/yarn-1.12.3.tar.gz into /home/jan/projects/graylog/graylog-project-repos/graylog2-server/graylog2-web-interface/node/yarn
[INFO] Installed Yarn locally.
[INFO]
[INFO] --- frontend-maven-plugin:1.6:yarn (yarn install) @ graylog2-server ---
[INFO] Running 'yarn install' in /home/jan/projects/graylog/graylog-project-repos/graylog2-server/graylog2-web-interface
[INFO] yarn install v1.12.3
[INFO] [1/4] Resolving packages...
[ERROR] warning Resolution field "upath@1.0.5" is incompatible with requested version "upath@^1.1.1"
[INFO] success Already up-to-date.
[INFO] Done in 0.94s.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Graylog Parent POM 3.2.0-SNAPSHOT:
[INFO]
[INFO] Graylog Parent POM ................................. SUCCESS [ 0.003 s]
[INFO] Graylog Project Parent POM ......................... SUCCESS [ 0.064 s]
[INFO] Graylog ............................................ FAILURE [ 2.809 s]
[INFO] integration-tests .................................. SKIPPED
[INFO] Graylog Plugin Parent POM .......................... SKIPPED
[INFO] Graylog Plugin with Web support Parent POM ......... SKIPPED
[INFO] graylog-plugin-archetype ........................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.112 s
[INFO] Finished at: 2019-10-06T10:57:14+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.6:yarn (yarn install) on project graylog2-server: Failed to run task: 'yarn install' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 134 (Exit value: 134) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :graylog2-server
I use graylog-project-cli exec "mvn compile -B" to compile the graylog sources. I’m on Debian GNU/Linux 10 if that matters.
Do I do something wrong or are the npm/yarn dependencies in a broken state currently?
@merouaneagar thanks for the link. I came to similar conclusions how to get a build environment before but tried to get it running from scratch, just to avoid own mistakes.
I followed the guide and got the same compilation errors mentioned above when running the Initial Build step of the README file. I continued and my freshly created plugin failed to build because of missing dependencies that have not been built from the graylog2-server tree in the initial build step:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.445 s
[INFO] Finished at: 2019-10-07T18:23:39+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project graylog-plugin-oidc-auth: Could not resolve dependencies for project info.dittberner:graylog-plugin-oidc-auth:jar:1.0.0-SNAPSHOT: Could not find artifact org.graylog2:graylog2-server:jar:tests:3.2.0-SNAPSHOT in sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
unfortunatelly the Travis CI builds for Graylog2-server (master branch) and the graylog-plugin-auth-sso are failing too, so I assume this is not just a problem for me.
I would love to help you getting your dev setup up and running. First thing, I think it would be a better idea to develop against the 3.1.2 tag of the server repo instead of master. Things are changing constantly on master and might break your build.
This alone would probably solve the issue you are seeing already, as the 3.1.2 artefacts are on sonatype. If you really want to develop against master, you have to run maven in the graylog-project directory, so it is able to resolve the snapshot dependencies from its module tree instead of looking them up on sonatype.
I hope this helps!
P.S.: We know that we really need to update our docs when it comes to the scripts/bootstrap-plugin script that we removed. Sorry for that!
how do you recover version 3.1.2
because I do not arrive with the following command:
…/graylog-project.linux bootstrap https://github.com/Graylog2/graylog-project.git --manifest=“manifests/3.1.2.json”
git clone https://github.com/Graylog2/graylog-project.git graylog-project
Cloning into ‘graylog-project’…
git checkout master
Already on ‘master’
Your branch is up to date with ‘origin/master’.
Unable to read manifest: open /home/agar/Documents/projects/graylog/3.1.2/graylog-project/manifests/3.1.2.json: no such file or directory
cat 3.1.2.json
{
"modules": [
{
"repository": "git@github.com:Graylog2/graylog2-server.git",
"revision": "3.1.2",
"server": true,
"submodules": [
{
"path": "graylog2-server"
},
{
"path": "graylog2-web-interface"
}
]
},
{
"repository": "git@github.com:Graylog2/graylog-plugin-collector.git",
"revision": "3.1.2",
"assemblies": ["graylog"]
},
{
"repository": "git@github.com:Graylog2/graylog-plugin-aws.git",
"revision": "3.1.2",
"assemblies": ["graylog"]
},
{
"repository": "git@github.com:Graylog2/graylog-plugin-threatintel.git",
"revision": "3.1.2",
"assemblies": ["graylog"]
}
]
}
../graylog-project.linux bootstrap https://github.com/Graylog2/graylog-project.git --manifest="3.1.2.json"
git clone https://github.com/Graylog2/graylog-project.git graylog-project
Cloning into 'graylog-project'...
git checkout master
Already on 'master'
Your branch is up to date with 'origin/master'.
Unable to read manifest: open /home/agar/Documents/projects/graylog/3.1.2/graylog-project/3.1.2.json: no such file or directory
agar@P67RGRE ~/Documents/projects/graylog/3.1.2 $ rm -rf graylog-project/
agar@P67RGRE ~/Documents/projects/graylog/3.1.2 $ ../graylog-project.linux bootstrap https://github.com/Graylog2/graylog-project.git --manifest="../3.1.2.json"
git clone https://github.com/Graylog2/graylog-project.git graylog-project
Cloning into 'graylog-project'...
git checkout master
Already on 'master'
Your branch is up to date with 'origin/master'.
Repository: https://github.com/Graylog2/graylog2-server.git
Cloning https://github.com/Graylog2/graylog2-server.git into /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog2-server
git clone https://github.com/Graylog2/graylog2-server.git /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog2-server
Cloning into '/home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog2-server'...
Checkout revision: 3.1.2
git branch 3.1.2 origin/3.1.2
git checkout 3.1.2
Note: checking out '3.1.2'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 9e96b08ff [graylog2-server] prepare release 3.1.2
Repository: https://github.com/Graylog2/graylog-plugin-collector.git
Cloning https://github.com/Graylog2/graylog-plugin-collector.git into /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-collector
git clone https://github.com/Graylog2/graylog-plugin-collector.git /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-collector
Cloning into '/home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-collector'...
Checkout revision: 3.1.2
git branch 3.1.2 origin/3.1.2
git checkout 3.1.2
Note: checking out '3.1.2'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at b4ecf46 [graylog-plugin-collector] prepare release 3.1.2
Repository: https://github.com/Graylog2/graylog-plugin-aws.git
Cloning https://github.com/Graylog2/graylog-plugin-aws.git into /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-aws
git clone https://github.com/Graylog2/graylog-plugin-aws.git /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-aws
Cloning into '/home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-aws'...
Checkout revision: 3.1.2
git branch 3.1.2 origin/3.1.2
git checkout 3.1.2
Note: checking out '3.1.2'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 25ceaf1 [graylog-plugin-aws] prepare release 3.1.2
Repository: https://github.com/Graylog2/graylog-plugin-threatintel.git
Cloning https://github.com/Graylog2/graylog-plugin-threatintel.git into /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-threatintel
git clone https://github.com/Graylog2/graylog-plugin-threatintel.git /home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-threatintel
Cloning into '/home/agar/Documents/projects/graylog/3.1.2/graylog-project-repos/graylog-plugin-threatintel'...
Checkout revision: 3.1.2
git branch 3.1.2 origin/3.1.2
git checkout 3.1.2
Note: checking out '3.1.2'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 2953ede [graylog-plugin-threatintel] prepare release 3.1.2
Generating pom.xml file from template pom.xml.tmpl
Generating runner/pom.xml file from template runner/pom.xml.tmpl
Generating src/main/assembly/server-tarball.xml file from template src/main/assembly/server-tarball.xml.tmpl
Writing manifest state to .graylog-project-manifest-state
I can only guess, but I assume it is a CORS error. Can you try connecting to http://localhost:8080 (vs. http://127.0.0.1:8080) in your browser? it is a fine difference that only matters to CORS checks in your browser.
