Solr Indexing and Analysis Tricks, Erik Hatcher, LucidWorks
Why Stack Overflow recommends it
Stack Overflow users link it when answering questions about storing analyzed tokens, custom index-time processing, or extracting entities in Solr, since it shows update processors doing exactly that.
Recommended by 3 people across 3 different months, 2013 to 2021.
What it is
A 45-minute conference talk by Erik Hatcher of LucidWorks at Lucene/Solr Revolution. It surveys Solr's analysis components and then shows how to use the update processing pipeline, including the ScriptUpdateProcessor, to enrich documents at index time, building on his 'Poor Man's Entity Extraction' blog post.
What people said
Recommends Persimmonium · 2013-12 · Stack Overflow · on “Store and export raw tokens in a solr index”
Let's see, I think what you want is to store, alongside the original content of some field, the field value but after it goes through some analysis chain, right? You would think copyFields would help, but they don't as if you store them, the original field value is stored. You need to use an updateProcessor. Look at this talk [this video] Erik Hatcher gave, minutes 7:30 to 20:00 aprox, and you will see exactly this case explained very well, with examples and all. Once you have that stored in the index, you can return it and do anything you like.
Recommends Adisesha · 2015-02 · Stack Overflow · on “Is it possible to do this type of search in Java”
You can use tika for text extraction. If there is a fixed pattern, you can extract information using regex or xpath queries. Other solution is to use Solr as shown in this video [this video] .You don't need solr but watch the video to get idea.
Recommends Norbert Bodnar · 2021-01 · Stack Overflow · on “Can I get the Field value in String into custom TokenFilter in Apache Solr?”
So I think I found the answer, or actually two answers. One would be to write my client application in a way, that incoming requests are first processed - the field value is sent to the external API and the response is stored into some global variable, which can then be accessed from the custom TokenFilters. Another one would be to use custom UpdateRequestProcessors, which allow us to modify the content of the incoming document, calling the external API and again saving the response so it's somehow globally accessible from custom TokenFilters. Here [this video] Erik Hatcher talks about the use of the ScriptUpdateProcessor, which I believe can be used in my case too. Hope this helps to anyone stum
Cites Kevin · 2016-01 · Stack Overflow · on “Solr - Copy tokenized multi valued field to a string”
I decided to add a field (cancerTerms) and add in the terms there. But I am not doing this outside of Solr. I am using the analysis chain and passing it through a ScriptUpdateProcessor. Here I can take the results of the analysis chain and store the results to the document (as as strField). Then I facet on this field (cancerTerms). This actually gives me the correct results, it does not give me issues with the not and not necessarily or any other similar issue. Also I am not storing the the analysis chain field (I was previously). It makes no sense to store this because it was a copy field (apparently copy fields only copy the source text then pipe it to the analyzer, and cannot be chained).