We have a need to enrich the incoming data.
I’d like to know if I can pull data from another external source as part of an enrichment stage in the pipeline.
I haven’t identified a core feature for this, but may have missed it.
It may be that I can pull the external data into a cache object and store in Mongo for example, then use as part of my enrichment. The case would either be to add an extra field as part of a standard lookup, or possibly act on the correlation between a stream field and the lookup.
Any ideas or direction would really be appreciated.