All dashboards lost after upgrade from 3.1.x to 3.2.1

Graylog Central (peer support)
1

Hi!
After upgrade to 3.2 I’ve lost all (~50) dashboards with hundreds of widgets.
Following meaningful error is thrown:

2020-02-11T10:36:40.194+02:00 ERROR [ConfigurationManagementPeriodical] Error while running migration <V20191125144500_MigrateDashboardsToViews{2019-11-25T14:45:00Z}>
java.lang.RuntimeException: Unable to parse function: avg(time-taken)
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.viewwidgets.Series.toSeriesSpec(Series.java:60) ~[graylog.jar:?]
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.AggregationWidget.lambda$toSearchTypes$2(AggregationWidget.java:82) ~[graylog.jar:?]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_212]
at java.util.Collections$2.tryAdvance(Collections.java:4717) ~[?:1.8.0_212]
at java.util.Collections$2.forEachRemaining(Collections.java:4725) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_212]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_212]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_212]
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.AggregationWidget.toSearchTypes(AggregationWidget.java:82) ~[graylog.jar:?]
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.V20191125144500_MigrateDashboardsToViews.createSearchType(V20191125144500_MigrateDashboardsToViews.java:186) ~[graylog.jar:?]
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.V20191125144500_MigrateDashboardsToViews.lambda$migrateDashboard$5(V20191125144500_MigrateDashboardsToViews.java:154) ~[graylog.jar:?]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_212]
at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_212]
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_212]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_212]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_212]
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.V20191125144500_MigrateDashboardsToViews.migrateDashboard(V20191125144500_MigrateDashboardsToViews.java:155) ~[graylog.jar:?]
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.V20191125144500_MigrateDashboardsToViews.lambda$upgrade$0(V20191125144500_MigrateDashboardsToViews.java:90) ~[graylog.jar:?]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_212]
at java.util.ArrayList.forEach(ArrayList.java:1257) ~[?:1.8.0_212]
at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) ~[?:1.8.0_212]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_212]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_212]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_212]
at org.graylog.plugins.views.migrations.V20191125144500_MigrateDashboardsToViewsSupport.V20191125144500_MigrateDashboardsToViews.upgrade(V20191125144500_MigrateDashboardsToViews.java:91) ~[graylog.jar:?]
at org.graylog2.periodical.ConfigurationManagementPeriodical.doRun(ConfigurationManagementPeriodical.java:43) [graylog.jar:?]
at org.graylog2.plugin.periodical.Periodical.run(Periodical.java:77) [graylog.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

After master node restart the same error message is thrown. Is there something i can do to resolve this?
Thanks!

2

Dashboards are obviously there (in a mongodb collection) but migration script could not migrate them. I’ve tried to export/import mongodb dashboard collection to 3.1 test environment I have for a later creation of content pack on that 3.1 version and uploading this content pack on 3.2 version afterwards as a workaround but this also failed (i could import mongodb data, create content pack on version 3.1, could upload it on 3.2 but it is failing to install).
I had one custom content pack with dashboards which was uploaded but not installed previously and this pack installed successfully so basically version 3.1 content packs are at some level compatible with 3.2 but still I could not install successfully content pack from that temporary 3.1 test environment into 3.2 version even with only 1 dashboard and 3 simple widgets.
Can someone assist me on how to migrate those as it will take many days to recreate them manually.

3

same error here after migration

ERROR [ConfigurationManagementPeriodical] Error while running migration <V20191125144500_MigrateDashboardsToViews{2019-11-25T14:45:00Z}>

from graylog-3.1-repository_latest.deb to graylog-3.2-repository_latest.deb

please advise

4

he @ericMont

you did not share enough from your error. But you might run into the issue mentioned here:

5

Just found this bug. Checked and i don’t have dashes though i have underscores in field names of series. Is this bug relevant to my problem or it is a whole different story?

6

Hello, we have updated Graylog with 3.1.x to 3.2.1 and all our dashboards have disappeared

we get an error in the log:
2020-02-12T11: 53: 18.952+01: 00 error [MongoDB Processing Status Recorder Service] failed to save the processing state
2020-02-12T12: 08: 55.870+01: 00 error [periodic configuration management] error when performing migration <V20191125144500_MigrateDashboardstoviews{2019-11-25T14: 45: 00Z}>

Graylog-server 3.2.1
MongoDB 3.6.17
Elasticsearch in 5.6.16

7

he @rsa
please add this information to the issue - because that might be related and the devs can check for that too

8

he @ec10

sorry that you run into problems - but with the provided information it is not possible to identify the issue. It might be related to your MongoDB version. But that is just guessing. Because it might be something else.

9

Cause of my problem indeed is as described in bug (https://github.com/Graylog2/graylog2-server/issues/7439). As this affected only few percents of all widgets (I’ve hundreds of widgets and only some 20 included dash in a field name) I exported dashboard collection from mongodb, deleted those widgets there, cleared/truncated dashboard collection and imported back collection with affected widgets removed. Migration script succeeded. Will add those ~20 manually later.

1 Like
10

This will be fixed in 3.2.2. Underscores should always have worked, but dashes and @ haven’t. See https://github.com/Graylog2/graylog2-server/pull/7448

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.