I want to figure out how to use my custom plugin for Solr.

I found the following relevant article:

/@akshayt030/developing-custom-filter-plugin-in-lucene-solr-8-63cb099036cc

After reading the instructions, I wrote my plugin.

I use the following libraries:

<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.4</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-core</artifactId>
        <version>5.0.0</version>
    </dependency>

    <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-queryparser</artifactId>
        <version>5.0.0</version>
    </dependency>

    <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-analyzers-common</artifactId>
        <version>5.0.0</version>
    </dependency>

   <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-test-framework</artifactId>
        <version>5.0.0</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Filter code:

import java.io.IOException;
import java.util.Arrays;

import .apache.lucene.analysis.TokenFilter;
import .apache.lucene.analysis.TokenStream;
import .apache.lucene.analysis.tokenattributes.CharTermAttribute;
import .apache.lucene.analysis.tokenattributes.OffsetAttribute;
import .apache.lucene.analysis.tokenattributes.PayloadAttribute;
import .apache.lucene.analysis.tokenattributes.PositionIncrementAttribute;
import .apache.lucene.analysis.tokenattributes.TypeAttribute;

import .apache.logging.log4j.LogManager;
import .apache.logging.log4j.Logger;

public final class DebugTokenFilter extends TokenFilter {

    private static final Logger logger = LogManager.getLogger(DebugTokenFilter.class);

    private final CharTermAttribute charTerm = getAttribute(CharTermAttribute.class);
    private final OffsetAttribute offset = getAttribute(OffsetAttribute.class);
    private final PositionIncrementAttribute posInc = getAttribute(PositionIncrementAttribute.class);
    private final TypeAttribute type = getAttribute(TypeAttribute.class);
    private final PayloadAttribute payload = getAttribute(PayloadAttribute.class);

    private final String name;

    public DebugTokenFilter(TokenStream input, String name) {
        super(input);
        this.name = name;
    }

    @Override
    public boolean incrementToken() throws IOException {

        boolean hasNext = input.incrementToken();

        if (hasNext) {

            String term =  new String(charTerm.buffer(), 0, charTerm.length());
            logger.info("{}, term = {}, type = {}, payload =, offset = , length = , increment = ",
                    name,
                    term,
                    type.type().toString(),
                    getPayloadString(),
                    String.valueOf(offset.startOffset()),
                    String.valueOf(offset.endOffset() - offset.startOffset()),
                    String.valueOf(posInc.getPositionIncrement()));
        }

        return hasNext;
    }

    private String getPayloadString() {
        if (payload == null || payload.getPayload() == null) {
            return "(no payload)";
        }
        return Arrays.toString(payload.getPayload().bytes);
    }
}

The factory code:

import java.util.Map;

import .apache.lucene.analysis.TokenStream;
import .apache.lucene.analysis.util.TokenFilterFactory;

public class DebugTokenFilterFactory extends TokenFilterFactory {

    private final String name;

    public DebugTokenFilterFactory(Map<String, String> args) {
        super(args);
        this.name = args.containsKey("name") ? args.get("name") : "debug";
    }

    @Override
    public TokenStream create(TokenStream input) {
        return new DebugTokenFilter(input, name);
    }
}

I use solr version 9.7.0 to check the functionality. Compiled jar I put to the following lib:

${SOLR_HOME}\lib\custom

and included in solrconfig.xml file

    <lib dir="${solr.install.dir:../../../..}/custom_lib" regex=".*\.jar" />

I included my filter to text_general field type in management_schema.xml

  <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100" multiValued="true">
    <analyzer type="index">
      <tokenizer name="standard"/>
      <filter name="stop" ignoreCase="true" words="stopwords.txt"/>
      <filter name="lowercase"/>
      <filter class="abc.example.ReverseFilterFactory”/>
   </analyzer>
    <analyzer type="query">
      <tokenizer name="standard"/>
      <filter name="stop" ignoreCase="true" words="stopwords.txt"/>
      <filter expand="true" ignoreCase="true" synonyms="synonyms.txt" name="synonymGraph"/>
      <filter name="lowercase"/>
    </analyzer> 
  </fieldType>

After Solr restarted I saw an initialization error and none core was loaded.

[![enter image description here][1]][1]

There aren't any error in logs:

5-03-24 15:55:39.328 INFO  (main) [c: s: r: x: t:] o.e.j.s.Server jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git: 5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 11+28
2025-03-24 15:55:39.675 WARN  (main) [c: s: r: x: t:] o.e.j.u.DeprecationWarning Using @Deprecated Class .eclipse.jetty.servlet.listener.ELContextCleaner
2025-03-24 15:55:39.691 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Using logger factory .apache.logging.slf4j.Log4jLoggerFactory
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider  ___      _       Welcome to Apache Solr™ version 9.7.0
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider / __| ___| |_ _   Starting in standalone mode on port 8983
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_|  Install dir: C:\software\solr-9.7.0
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider |___/\___/_|_|    Start time: 2025-03-24T15:55:39.707195900Z
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Solr started with "-XX:+CrashOnOutOfMemoryError" that will crash on any OutOfMemoryError exception. The cause of the OOME will be logged in the crash file at the following path: C:\software\solr-9.7.0\server\logs\jvm_crash_12756.log
2025-03-24 15:55:39.729 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Solr Home: C:\software\solr-9.7.0\server\solr (source: system property: solr.solr.home)
2025-03-24 15:55:39.744 INFO  (main) [c: s: r: x: t:] o.a.s.c.SolrXmlConfig Loading solr.xml from C:\software\solr-9.7.0\server\solr\solr.xml
2025-03-24 15:55:39.776 INFO  (main) [c: s: r: x: t:] o.a.s.c.SolrResourceLoader Added 3 libs to classloader, from paths: [/C:/software/solr-9.7.0/lib, /C:/software/solr-9.7.0/lib/custom]

What could I have missed for the filter to work correctly?

I want to figure out how to use my custom plugin for Solr.

I found the following relevant article:

https://medium/@akshayt030/developing-custom-filter-plugin-in-lucene-solr-8-63cb099036cc

After reading the instructions, I wrote my plugin.

I use the following libraries:

<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.4</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-core</artifactId>
        <version>5.0.0</version>
    </dependency>

    <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-queryparser</artifactId>
        <version>5.0.0</version>
    </dependency>

    <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-analyzers-common</artifactId>
        <version>5.0.0</version>
    </dependency>

   <dependency>
        <groupId>.apache.lucene</groupId>
        <artifactId>lucene-test-framework</artifactId>
        <version>5.0.0</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Filter code:

import java.io.IOException;
import java.util.Arrays;

import .apache.lucene.analysis.TokenFilter;
import .apache.lucene.analysis.TokenStream;
import .apache.lucene.analysis.tokenattributes.CharTermAttribute;
import .apache.lucene.analysis.tokenattributes.OffsetAttribute;
import .apache.lucene.analysis.tokenattributes.PayloadAttribute;
import .apache.lucene.analysis.tokenattributes.PositionIncrementAttribute;
import .apache.lucene.analysis.tokenattributes.TypeAttribute;

import .apache.logging.log4j.LogManager;
import .apache.logging.log4j.Logger;

public final class DebugTokenFilter extends TokenFilter {

    private static final Logger logger = LogManager.getLogger(DebugTokenFilter.class);

    private final CharTermAttribute charTerm = getAttribute(CharTermAttribute.class);
    private final OffsetAttribute offset = getAttribute(OffsetAttribute.class);
    private final PositionIncrementAttribute posInc = getAttribute(PositionIncrementAttribute.class);
    private final TypeAttribute type = getAttribute(TypeAttribute.class);
    private final PayloadAttribute payload = getAttribute(PayloadAttribute.class);

    private final String name;

    public DebugTokenFilter(TokenStream input, String name) {
        super(input);
        this.name = name;
    }

    @Override
    public boolean incrementToken() throws IOException {

        boolean hasNext = input.incrementToken();

        if (hasNext) {

            String term =  new String(charTerm.buffer(), 0, charTerm.length());
            logger.info("{}, term = {}, type = {}, payload =, offset = , length = , increment = ",
                    name,
                    term,
                    type.type().toString(),
                    getPayloadString(),
                    String.valueOf(offset.startOffset()),
                    String.valueOf(offset.endOffset() - offset.startOffset()),
                    String.valueOf(posInc.getPositionIncrement()));
        }

        return hasNext;
    }

    private String getPayloadString() {
        if (payload == null || payload.getPayload() == null) {
            return "(no payload)";
        }
        return Arrays.toString(payload.getPayload().bytes);
    }
}

The factory code:

import java.util.Map;

import .apache.lucene.analysis.TokenStream;
import .apache.lucene.analysis.util.TokenFilterFactory;

public class DebugTokenFilterFactory extends TokenFilterFactory {

    private final String name;

    public DebugTokenFilterFactory(Map<String, String> args) {
        super(args);
        this.name = args.containsKey("name") ? args.get("name") : "debug";
    }

    @Override
    public TokenStream create(TokenStream input) {
        return new DebugTokenFilter(input, name);
    }
}

I use solr version 9.7.0 to check the functionality. Compiled jar I put to the following lib:

${SOLR_HOME}\lib\custom

and included in solrconfig.xml file

    <lib dir="${solr.install.dir:../../../..}/custom_lib" regex=".*\.jar" />

I included my filter to text_general field type in management_schema.xml

  <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100" multiValued="true">
    <analyzer type="index">
      <tokenizer name="standard"/>
      <filter name="stop" ignoreCase="true" words="stopwords.txt"/>
      <filter name="lowercase"/>
      <filter class="abc.example.ReverseFilterFactory”/>
   </analyzer>
    <analyzer type="query">
      <tokenizer name="standard"/>
      <filter name="stop" ignoreCase="true" words="stopwords.txt"/>
      <filter expand="true" ignoreCase="true" synonyms="synonyms.txt" name="synonymGraph"/>
      <filter name="lowercase"/>
    </analyzer> 
  </fieldType>

After Solr restarted I saw an initialization error and none core was loaded.

[![enter image description here][1]][1]

There aren't any error in logs:

5-03-24 15:55:39.328 INFO  (main) [c: s: r: x: t:] o.e.j.s.Server jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git: 5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 11+28
2025-03-24 15:55:39.675 WARN  (main) [c: s: r: x: t:] o.e.j.u.DeprecationWarning Using @Deprecated Class .eclipse.jetty.servlet.listener.ELContextCleaner
2025-03-24 15:55:39.691 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Using logger factory .apache.logging.slf4j.Log4jLoggerFactory
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider  ___      _       Welcome to Apache Solr™ version 9.7.0
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider / __| ___| |_ _   Starting in standalone mode on port 8983
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_|  Install dir: C:\software\solr-9.7.0
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider |___/\___/_|_|    Start time: 2025-03-24T15:55:39.707195900Z
2025-03-24 15:55:39.707 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Solr started with "-XX:+CrashOnOutOfMemoryError" that will crash on any OutOfMemoryError exception. The cause of the OOME will be logged in the crash file at the following path: C:\software\solr-9.7.0\server\logs\jvm_crash_12756.log
2025-03-24 15:55:39.729 INFO  (main) [c: s: r: x: t:] o.a.s.s.CoreContainerProvider Solr Home: C:\software\solr-9.7.0\server\solr (source: system property: solr.solr.home)
2025-03-24 15:55:39.744 INFO  (main) [c: s: r: x: t:] o.a.s.c.SolrXmlConfig Loading solr.xml from C:\software\solr-9.7.0\server\solr\solr.xml
2025-03-24 15:55:39.776 INFO  (main) [c: s: r: x: t:] o.a.s.c.SolrResourceLoader Added 3 libs to classloader, from paths: [/C:/software/solr-9.7.0/lib, /C:/software/solr-9.7.0/lib/custom]

What could I have missed for the filter to work correctly?

• java
• search
• solr
• lucene

1 Answer 1

I made a mistake when connecting the library

I put my jar file in the custom_lib folder in the Solr root directory.

Include the jar file by defining the below line in solrconfig.xml conf file:

 <lib dir="${solr.install.dir:../../../..}/custom_lib/" regex=".*\.jar" />

The completed project with the installed plugin can be viewed at the link:

https://github/EvadS/my-first-solr-plugin