Data adapter failure with multiple value key

when i create a CSV data adapter, system report errors as:


and there error reports in server.log as following

2018-03-14T20:37:01.063+08:00 ERROR [LookupDataAdapter] Couldn't refresh data adapter <city-name/5aa90e9d28a47a0b22cac59e/@27528fde>
java.lang.IllegalArgumentException: Multiple entries with same key: 0023=ǭ�� and 0023=����
	at ~[graylog.jar:?]
	at ~[graylog.jar:?]
	at ~[graylog.jar:?]
	at$ ~[graylog.jar:?]
	at org.graylog2.lookup.adapters.CSVFileDataAdapter.parseCSVFile( ~[graylog.jar:?]
	at org.graylog2.lookup.adapters.CSVFileDataAdapter.doRefresh( ~[graylog.jar:?]
	at org.graylog2.plugin.lookup.LookupDataAdapter.refresh( ~[graylog.jar:?]
	at org.graylog2.lookup.LookupDataAdapterRefreshService.lambda$schedule$0( ~[graylog.jar:?]
	at java.util.concurrent.Executors$ [?:1.8.0_161]
	at java.util.concurrent.FutureTask.runAndReset( [?:1.8.0_161]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301( [?:1.8.0_161]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor.runWorker( [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor$ [?:1.8.0_161]
	at [?:1.8.0_161]

I am using lastest graylogs release:

# rpm -qa|grep graylog-server

any idea on this?

Given that there aren’t duplicate entries in the key column of your CSV file, this looks like a bug.

Could you please provide the CSV file causing this bug and create an issue at

Also, can you say which character set has been used to create the CSV file? The CSV data adapter assumes UTF-8 by default.

Does the mutiple value lookup table come with a key with multiple attributes so multiple values, or just repeat the same key with other values? my case being:

code1      name1
code1      name2

those chars are in UTF-8.

This would work if was your key column. The values in the key column need to be unique across the complete file.

thanks. so my table wrong with duplicate keys.

i realized that you should have mentioned also the display issue of those chinese words.

i upload the file in UTF-8 without BOM. but the result for the lookup doesn’t display correctly. let me open a issue for this.

It was proven that the wrong display issue due to the charset encoding issue. when i upload the CSV file with UTF-8 , keep push the lookup button, sometimes display correctly, most of time display wrongly. when reload the CSV file with UTF-8 without BOM, the display always correctly.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.