我正在學習Hibernate Search 6.1.3.Final作為Lucene 8.11.1后端和Spring Boot 2.6.6. 我正在嘗試搜索產品名稱、條形碼和制造商。目前，我正在做一個集成測驗，看看當幾個產品具有相似名稱時會發生什么：
@Test
void shouldFindSimilarTobaccosByQuery() {
var tobaccoGreen = TobaccoBuilder.builder()
.name("TobaCcO GreEN")
.build();
var tobaccoRed = TobaccoBuilder.builder()
.name("TobaCcO ReD")
.build();
var tobaccoGreenhouse = TobaccoBuilder.builder()
.name("TobaCcO GreENhouse")
.build();
tobaccoRepository.saveAll(List.of(tobaccoGreen, tobaccoRed, tobaccoGreenhouse));
webTestClient
.get().uri("/tobaccos?query=green")
.exchange()
.expectStatus().isOk()
.expectBodyList(Tobacco.class)
.value(tobaccos -> assertThat(tobaccos)
.hasSize(2)
.contains(tobaccoGreen, tobaccoGreenhouse)
);
}
正如您在測驗中看到的那樣，我希望獲得具有相似名稱的兩種煙草：tobaccoGreen并tobaccoGreenhouse使用greenas 查詢搜索條件。物體如下：
@Data
@Entity
@Indexed
@NoArgsConstructor
@AllArgsConstructor
@Builder(toBuilder = true)
@EqualsAndHashCode(of = "id")
@EntityListeners(AuditingEntityListener.class)
public class Tobacco {
@Id
@GeneratedValue
private UUID id;
@NotBlank
@KeywordField
private String barcode;
@NotBlank
@FullTextField(analyzer = "name")
private String name;
@NotBlank
@FullTextField(analyzer = "name")
private String manufacturer;
@CreatedDate
private Instant createdAt;
@LastModifiedDate
private Instant updatedAt;
}
我遵循了檔案并為名稱配置了一個分析器：
@Component("luceneTobaccoAnalysisConfigurer")
public class LuceneTobaccoAnalysisConfigurer implements LuceneAnalysisConfigurer {
@Override
public void configure(LuceneAnalysisConfigurationContext context) {
context.analyzer("name").custom()
.tokenizer("standard")
.tokenFilter("lowercase")
.tokenFilter("asciiFolding");
}
}
并使用帶有模糊選項的簡單查詢：
@Component
@AllArgsConstructor
public class IndexSearchTobaccoRepository {
private final EntityManager entityManager;
public List<Tobacco> find(String query) {
return Search.session(entityManager)
.search(Tobacco.class)
.where(f -> f.match()
.fields("barcode", "name", "manufacturer")
.matching(query)
.fuzzy()
)
.fetch(10)
.hits();
}
}
測驗顯示只能找到tobaccoGreen而不能找到tobaccoGreenhouse，我不明白為什么，如何搜索相似的產品名稱（或條形碼，制造商）？
uj5u.com熱心網友回復：
在我回答你的問題之前，我想指出呼叫.fetch(10).hits()是次優的，尤其是在使用默認排序時（就像你一樣）：
return Search.session(entityManager)
.search(Tobacco.class)
.where(f -> f.match()
.fields("barcode", "name", "manufacturer")
.matching(query)
.fuzzy()
)
.fetch(10)
.hits();
如果您.fetchHits(10)直接呼叫，Lucene 將能夠跳過部分搜索（計算總命中數的部分），并且在大型索引中，這可能會帶來相當大的性能提升。所以，改為這樣做：
return Search.session(entityManager)
.search(Tobacco.class)
.where(f -> f.match()
.fields("barcode", "name", "manufacturer")
.matching(query)
.fuzzy()
)
.fetchHits(10);
現在，實際答案：
通過搜索查詢來解決這個問題
.fuzzy()不是魔術，它不會只匹配你認為應該匹配的任何東西:) 它的作用有一個特定的定義，這不是你想要的。
要獲得您想要的行為，您可以使用它而不是您當前的謂詞：
.where(f -> f.simpleQueryString()
.fields("barcode", "name", "manufacturer")
.matching("green*")
)
您失去了模糊性，但您可以執行前綴查詢，這將給出您想要的結果（green*將匹配greenhouse）。
但是，前綴查詢是明確的：用戶必須*在“green”之后添加才能匹配“所有以 green 開頭的單詞”。
這導致我們...
通過分析器解決這個問題
If you want this "prefix matching" behavior to be automatic, without the need to add * in the query, then what you need is a different analyzer.
Your current analyzer breaks down indexed text using space as a separator (more or less; it's a bit more complex but that's the idea). But you apparently want it to break down "greenhouse" into "green" and "house"; that's the only way a query with the word "green" would match the word "greenhouse".
To do that, you can use an analyzer similar to yours, but with an additional "edge_ngram" filter, to generate additional indexed tokens for every prefix string of your existing tokens.
Add another analyzer to your configurer:
@Component("luceneTobaccoAnalysisConfigurer")
public class LuceneTobaccoAnalysisConfigurer implements LuceneAnalysisConfigurer {
@Override
public void configure(LuceneAnalysisConfigurationContext context) {
context.analyzer("name").custom()
.tokenizer("standard")
.tokenFilter("lowercase")
.tokenFilter("asciiFolding");
// THIS PART IS NEW
context.analyzer("name_prefix").custom()
.tokenizer("standard")
.tokenFilter("lowercase")
.tokenFilter("asciiFolding")
.tokenFilter("edgeNGram")
// Handling prefixes from 2 to 7 characters.
// Prefixes of 1 character or more than 7 will
// not be matched.
// You can extend the range, but this will take more
// space in the index for little gain.
.param( "minGramSize", "2" )
.param( "maxGramSize", "7" );
}
}
And change your mapping to use the name analyzer when querying, but the name_prefix analyzer when indexing:
@Data
@Entity
@Indexed
@NoArgsConstructor
@AllArgsConstructor
@Builder(toBuilder = true)
@EqualsAndHashCode(of = "id")
@EntityListeners(AuditingEntityListener.class)
public class Tobacco {
@Id
@GeneratedValue
private UUID id;
@NotBlank
@KeywordField
private String barcode;
@NotBlank
// CHANGE THIS
@FullTextField(analyzer = "name_prefix", searchAnalyzer = "name")
private String name;
@NotBlank
// CHANGE THIS
@FullTextField(analyzer = "name_prefix", searchAnalyzer = "name")
private String manufacturer;
@CreatedDate
private Instant createdAt;
@LastModifiedDate
private Instant updatedAt;
}
Now reindex your data.
Now your query "green" will also match "TobaCcO GreENhouse", because "GreENhouse" was indexed as ["greenhouse", "gr", "gre", "gree", "green", "greenh", "greenho"].
Variations
edgeNGram filter on distinct fields
Instead of changing the analyzer of your current fields, you could add new fields for the same Java properties, but using the new analyzer with the edgeNGram filter:
@Data
@Entity
@Indexed
@NoArgsConstructor
@AllArgsConstructor
@Builder(toBuilder = true)
@EqualsAndHashCode(of = "id")
@EntityListeners(AuditingEntityListener.class)
public class Tobacco {
@Id
@GeneratedValue
private UUID id;
@NotBlank
@KeywordField
private String barcode;
@NotBlank
@FullTextField(analyzer = "name")
// ADD THIS
@FullTextField(name = "name_prefix", analyzer = "name_prefix", searchAnalyzer = "name")
private String name;
@NotBlank
@FullTextField(analyzer = "name")
// ADD THIS
@FullTextField(name = "manufacturer_prefix", analyzer = "name_prefix", searchAnalyzer = "name")
private String manufacturer;
@CreatedDate
private Instant createdAt;
@LastModifiedDate
private Instant updatedAt;
}
Then you can target these fields as well as the normal ones in your query:
@Component
@AllArgsConstructor
public class IndexSearchTobaccoRepository {
private final EntityManager entityManager;
public List<Tobacco> find(String query) {
return Search.session(entityManager)
.search(Tobacco.class)
.where(f -> f.match()
.fields("barcode", "name", "manufacturer").boost(2.0f)
.fields("name_prefix", "manufacturer_prefix")
.matching(query)
.fuzzy()
)
.fetchHits(10);
}
}
As you can see, I added a boost to the fields that don't use prefix. This is the main advantage of this variant over the one I explained higher up: matches on actual words (not prefixes) will be deemed more important, yielding a better score and thus pulling documents to the top of the result list if you use a relevance sort (which is the default sort).
Handling only compound words instead of all words
I won't detail it here, but there's another approach if all you want is to handle compound words ("greenhouse" => "green" "house", "superman" => "super" "man", etc.). You can use the "dictionaryCompoundWord" filter. This is less powerful, but will generate less noise in your index (fewer meaningless tokens) and thus could lead to better relevance sorts.
Another downside is that you need to provide the filter with a dictionary that contains all words that could possibly be "compounded".
For more information, see the source and javadoc of class org.apache.lucene.analysis.compound.DictionaryCompoundWordTokenFilterFactory, or the documentation of the equivalent filter in Elasticsearch.
轉載請註明出處，本文鏈接：https://www.uj5u.com/qiye/457265.html