Elasticsearch 入門實戰(5)--Java API Client 使用-有解無憂

本文主要介紹 Elasticsearch Java API Client 的使用，相關的環境及軟體資訊如下：CentOS 7.6.1810、Java 1.8.0_321(客戶端用)、Elasticsearch 8.2.2，

1、Java API Client 的特點

Strongly typed requests and responses for all Elasticsearch APIs.
Blocking and asynchronous versions of all APIs.
Use of fluent builders and functional patterns to allow writing concise yet readable code when creating complex nested structures.
Seamless integration of application classes by using an object mapper such as Jackson or any JSON-B implementation.
Delegates protocol handling to an http client such as the Java Low Level REST Client that takes care of all transport-level concerns: HTTP connection pooling, retries, node discovery, and so on.

2、引入依賴

<dependency>
    <groupId>co.elastic.clients</groupId>
    <artifactId>elasticsearch-java</artifactId>
    <version>8.2.2</version>
</dependency>

3、使用

Elasticsearch Java API Client 通過 API 的方式來組裝請求資料，避免直接撰寫 JSON 字串；請求資料的詳細說明可參考：Elasticsearch 入門實戰(3)--REST API 使用，

3.1、連接及關閉

Java API Client 底層依賴 Java Low Level REST Client，需先創建 Low Level REST Client，

private ElasticsearchTransport transport;
private ElasticsearchClient client;

@Before
public void before() {
    RestClient restClient = RestClient.builder(
            new HttpHost("10.49.196.10", 9200),
            new HttpHost("10.49.196.11", 9200),
            new HttpHost("10.49.196.12", 9200)).build();
    ObjectMapper objectMapper = new ObjectMapper();
    transport = new RestClientTransport(restClient, new JacksonJsonpMapper(objectMapper));
    client = new ElasticsearchClient(transport);
}

@After
public void after() throws IOException {
    client.shutdown();
}

3.2、索引

3.2.1、創建索引

@Test
public void createIndex() throws IOException {
    CreateIndexResponse response = client.indices().create(builder -> builder
            .settings(indexSettingsBuilder -> indexSettingsBuilder.numberOfReplicas("1").numberOfShards("2"))
            .mappings(typeMappingBuilder -> typeMappingBuilder
                    .properties("age", propertyBuilder -> propertyBuilder.integer(integerNumberPropertyBuilder -> integerNumberPropertyBuilder))
                    .properties("name", propertyBuilder -> propertyBuilder.keyword(keywordPropertyBuilder -> keywordPropertyBuilder))
                    .properties("poems", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word")))
                    .properties("about", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word")))
                    .properties("success", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word")))
            )
            .index(INDEX_NAME));
    logger.info("acknowledged={}", response.acknowledged());
}

3.2.2、修改 _mapping 資訊

欄位可以新增，已有的欄位只能修改欄位的 search_analyzer 屬性，

@Test
public void modifyIndex() throws IOException {
    PutMappingResponse response = client.indices().putMapping(typeMappingBuilder -> typeMappingBuilder
            .index(INDEX_NAME)
            .properties("age", propertyBuilder -> propertyBuilder.integer(integerNumberPropertyBuilder -> integerNumberPropertyBuilder))
            .properties("name", propertyBuilder -> propertyBuilder.keyword(keywordPropertyBuilder -> keywordPropertyBuilder))
            .properties("poems", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart")))
    );
    logger.info("acknowledged={}", response.acknowledged());
}

3.2.3、洗掉索引

@Test
public void deleteIndex() throws IOException {
    DeleteIndexResponse response = client.indices().delete(builder -> builder.index(INDEX_NAME));
    logger.info("acknowledged={}", response.acknowledged());
}

3.2.4、查詢索引串列

@Test
public void getIndex() throws IOException {
    //使用 * 也可以
    GetIndexResponse response = client.indices().get(builder -> builder.index("_all"));
    logger.info(response.result().toString());
}

3.2.5、查詢索引詳情

@Test
public void getIndexDetail() throws IOException {
    GetIndexResponse response = client.indices().get(builder -> builder.index(INDEX_NAME));
    logger.info(response.result().toString());
}

3.3、檔案

3.3.1、創建檔案

@Test
public void createDoc() throws IOException {
    Map<String, Object> doc = new HashMap<>();
    doc.put("age", 30);
    doc.put("name", "李白");
    doc.put("poems", "靜夜思");
    doc.put("about", "字太白");
    doc.put("success", "創造了古代浪漫主義文學高峰、歌行體和七絕達到后人難及的高度");

    CreateResponse response = client.create(builder -> builder.index(INDEX_NAME).id("1").document(doc));
    logger.info(response.toString());

    Poet poet = new Poet(31, "杜甫", "登高", "字子美", "唐代偉大的現實主義文學作家，唐詩思想藝術的集大成者");
    response = client.create(builder -> builder.index(INDEX_NAME).id("2").document(poet));
    logger.info(response.toString());
}

3.3.2、洗掉檔案

@Test
public void deleteDoc() throws IOException {
    DeleteResponse response = client.delete(builder -> builder.index(INDEX_NAME).id("1"));
    logger.info(response.toString());
}

3.3.3、修改檔案

修改檔案，只修改設定的欄位，

@Test
public void updateDoc() throws IOException {
    Map<String, Object> doc = new HashMap<>();
    doc.put("age", 33);
    doc.put("name", "李白2");

    UpdateResponse response = client.update(builder -> builder.index(INDEX_NAME).id("1").doc(doc), Map.class);
    logger.info(response.toString());

    Poet poet = new Poet();
    poet.setAge(40);
    poet.setName("杜甫2");
    response = client.update(builder -> builder.index(INDEX_NAME).id("2").doc(poet).docAsUpsert(true), Poet.class);
    logger.info(response.toString());
}

3.3.4、新增或修改檔案

新增或修改檔案，修改時所有的欄位都會覆寫(相當于先洗掉在新增)，

@Test
public void createOrUpdateDoc() throws IOException {
    Map<String, Object> doc = new HashMap<>();
    doc.put("age", 33);
    doc.put("name", "李白2");

    //只更新設定的欄位
    IndexResponse response = client.index(builder -> builder.index(INDEX_NAME).id("1").document(doc));
    logger.info(response.toString());

    Poet poet = new Poet();
    poet.setAge(40);
    poet.setName("杜甫2");
    response = client.index(builder -> builder.index(INDEX_NAME).id("2").document(poet));
    logger.info(response.toString());
}

3.3.5、批量操作

@Test
public void bulk() throws IOException {
    List<BulkOperation> list = new ArrayList<>();

    //批量新增
    for (int i = 0; i < 5; i++) {
        Map<String, Object> doc = new HashMap<>();
        doc.put("age", 30);
        doc.put("name", "李白" + i);
        doc.put("poems", "靜夜思");
        doc.put("about", "字太白");
        doc.put("success", "創造了古代浪漫主義文學高峰、歌行體和七絕達到后人難及的高度");
        String id = 10 + i + "";
        list.add(new BulkOperation.Builder().create(builder -> builder.index(INDEX_NAME).id(id).document(doc)).build());
    }
    for (int i = 0; i < 5; i++) {
        Poet poet = new Poet(31, "杜甫" + i, "登高", "字子美", "唐代偉大的現實主義文學作家，唐詩思想藝術的集大成者");
        String id = 20 + i + "";
        list.add(new BulkOperation.Builder().create(builder -> builder.index(INDEX_NAME).id(id).document(poet)).build());
    }

    //批量洗掉
    list.add(new BulkOperation.Builder().delete(builder -> builder.index(INDEX_NAME).id("1")).build());
    list.add(new BulkOperation.Builder().delete(builder -> builder.index(INDEX_NAME).id("2")).build());

    BulkResponse response = client.bulk(builder -> builder.index(INDEX_NAME).operations(list));
    logger.info(response.toString());
}

3.4、查詢

3.4.1、查詢所有檔案

@Test
public void getDocAll() throws IOException {
    SearchResponse<Map> response = client.search(builder -> builder.index(INDEX_NAME), Map.class);
    logger.info(response.toString());
}

3.4.2、查詢單個檔案

@Test
public void getDoc() throws IOException {
    GetResponse<Map> response = client.get(builder -> builder.index(INDEX_NAME).id("1"), Map.class);
    if (response.found()) {
        logger.info(response.source().toString());
    }

    GetResponse<Poet> response2 = client.get(builder -> builder.index(INDEX_NAME).id("2"), Poet.class);
    if (response2.found()) {
        logger.info(response2.source().toString());
    }
}

3.4.3、term/terms 查詢

term/terms查詢，對輸入內容不做分詞處理，

@Test
public void searchTerm() throws IOException {
    SearchResponse<Map> response = client.search(searchRequestBuilder -> searchRequestBuilder
            .index(INDEX_NAME)
            .query(queryBuilder -> queryBuilder
                    .term(termQueryBuilder -> termQueryBuilder
                            .field("name").value("李白")))
            .sort(sortOptionsBuilder -> sortOptionsBuilder
                    .field(fieldSortBuilder -> fieldSortBuilder
                            .field("name").order(SortOrder.Asc)))
            .source(sourceConfigBuilder -> sourceConfigBuilder
                    .filter(sourceFilterBuilder -> sourceFilterBuilder
                            .includes("age", "name")))
            .from(0)
            .size(10)
            , Map.class);
    logger.info(response.toString());

    List<FieldValue> words = new ArrayList<>();
    words.add(new FieldValue.Builder().stringValue("李白").build());
    words.add(new FieldValue.Builder().stringValue("杜甫").build());
    SearchResponse<Poet> response2 = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .terms(termsQueryBuilder -> termsQueryBuilder
                                    .field("name").terms(termsQueryFieldBuilder -> termsQueryFieldBuilder.value(words))))
                    .source(sourceConfigBuilder -> sourceConfigBuilder
                            .filter(sourceFilterBuilder -> sourceFilterBuilder
                                    .excludes("about")))
                    .from(0)
                    .size(10)
            , Poet.class);
    logger.info(response2.toString());
}

3.4.4、range(范圍) 查詢

@Test
public void searchRange() throws IOException {
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .range(rangeQueryBuilder -> rangeQueryBuilder
                                    .field("age").gte(JsonData.of("20")).lt(JsonData.of("40"))))
            , Poet.class);
    logger.info(response.toString());
}

3.4.5、全文查詢

3.4.5.1、match 查詢

match 查詢，對輸入內容先分詞再查詢，

@Test
public void searchMatch() throws IOException {
    SearchResponse<Map> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .match(matchQueryBuilder -> matchQueryBuilder
                                    .field("success").query("思想")))
            , Map.class);
    logger.info(response.toString());
}

3.4.5.2、multi_match 查詢

多個欄位進行匹配，

@Test
public void searchMultiMatch() throws IOException {
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .multiMatch(multiMatchQueryBuilder -> multiMatchQueryBuilder
                                    .fields("about", "success").query("思想")))
            , Poet.class);
    logger.info(response.toString());
}

3.4.5.3、match_phrase 查詢

匹配整個查詢字串，

@Test
public void searchMatchPhrase() throws IOException {
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .matchPhrase(matchPhraseQueryBuilder -> matchPhraseQueryBuilder.field("success").query("文學作家")))
            , Poet.class);
    logger.info(response.toString());
}

3.4.5.4、match_all 查詢

查詢所有檔案，

@Test
public void searchMatchAll() throws IOException {
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .matchAll(matchAllQueryBuilder -> matchAllQueryBuilder))
            , Poet.class);
    logger.info(response.toString());
}

3.4.5.5、query_string 查詢

query_string 可以同時實作前面幾種查詢方法，

@Test
public void searchQueryString() throws IOException {
    //類似 match
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                .defaultField("success").query("古典文學")))
            , Poet.class);
    logger.info(response.toString());

    //類似 mulit_match
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                    .fields("about", "success").query("古典文學")))
            , Poet.class);
    logger.info(response.toString());

    //類似 match_phrase
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                    .defaultField("success").query("\"文學作家\"")))
            , Poet.class);
    logger.info(response.toString());

    //帶運算子查詢，運算子兩邊的詞不再分詞
    //查詢同時包含 ”文學“ 和 ”偉大“ 的檔案
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                    .fields("success").query("文學 AND 偉大")))
            , Poet.class);
    logger.info(response.toString());

    //等同上一個查詢
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                    .fields("success").query("文學 偉大").defaultOperator(Operator.And)))
            , Poet.class);
    logger.info(response.toString());

    //查詢 name 或 success 欄位包含"文學"和"偉大"這兩個單詞，或者包含"李白"這個單詞的檔案，
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                    .fields("name","success").query("(文學 AND 偉大) OR 高度")))
            , Poet.class);
    logger.info(response.toString());
}

3.4.5.6、simple_query_string 查詢

類似 query_string，主要區別如下：

1、不支持AND OR NOT ，會當做字符處理；使用 + 代替 AND，| 代替OR，- 代替 NOT
2、會忽略錯誤的語法

@Test
public void searchSimpleQueryString() throws IOException {
    //查詢同時包含 ”文學“ 和 ”偉大“ 的檔案
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .simpleQueryString(simpleQueryStringQueryBuilder -> simpleQueryStringQueryBuilder
                                    .fields("success").query("文學 + 偉大")))
            , Poet.class);
    logger.info(response.toString());
}

3.4.6、模糊查詢

@Test
public void searchFuzzy() throws IOException {
    //全文查詢時使用模糊引數，先分詞再計算模糊選項，
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .match(matchQueryBuilder -> matchQueryBuilder
                                    .field("success").query("思考").fuzziness("1")))
            , Poet.class);
    logger.info(response.toString());

    //使用 fuzzy query，對輸入不分詞，直接計算模糊選項，
    SearchResponse<Poet> response2 = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .fuzzy(fuzzyQueryBuilder ->  fuzzyQueryBuilder
                                    .field("success").fuzziness("1").value("理想")))
            , Poet.class);
    logger.info(response2.toString());
}

3.4.7、組合查詢

@Test
public void searchBool() throws IOException {
    //查詢 success 包含 “思想” 且 age 在 [20-40] 之間的檔案
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
            .index(INDEX_NAME)
            .query(queryBuilder -> queryBuilder
                    .bool(boolQueryBuilder -> boolQueryBuilder
                            .must(queryBuilder2 -> queryBuilder2
                                    .match(matchQueryBuilder -> matchQueryBuilder
                                            .field("success").query("思想"))
                            )
                            .must(queryBuilder2 -> queryBuilder2
                                    .range(rangeQueryBuilder -> rangeQueryBuilder
                                            .field("age").gte(JsonData.of("20")).lt(JsonData.of("40")))
                            )
                    )
            )
    , Poet.class);
    logger.info(response.toString());

    //過濾出 success 包含 “思想” 且 age 在 [20-40] 之間的檔案，不計算得分
    SearchResponse<Poet> response2 = client.search(searchRequestBuilder -> searchRequestBuilder
            .index(INDEX_NAME)
            .query(queryBuilder -> queryBuilder
                    .bool(boolQueryBuilder -> boolQueryBuilder
                            .filter(queryBuilder2 -> queryBuilder2
                                    .match(matchQueryBuilder -> matchQueryBuilder
                                            .field("success").query("思想"))
                            )
                            .filter(queryBuilder2 -> queryBuilder2
                                    .range(rangeQueryBuilder -> rangeQueryBuilder
                                            .field("age").gte(JsonData.of("20")).lt(JsonData.of("40")))
                            )
                    )
            )
    , Poet.class);
    logger.info(response2.toString());
}

3.4.8、聚合查詢

@Test
public void searchAggs() throws IOException {
    //求和
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .aggregations("age_sum", aggregationBuilder -> aggregationBuilder
                            .sum(sumAggregationBuilder -> sumAggregationBuilder
                                    .field("age")))
            , Poet.class);
    logger.info(response.toString());

    //類似 select count distinct(age) from poet-index
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .aggregations("age_count", aggregationBuilder -> aggregationBuilder
                            .cardinality(cardinalityAggregationBuilder -> cardinalityAggregationBuilder.field("age")))
            , Poet.class);
    logger.info(response.toString());

    //數量、最大、最小、平均、求和
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .aggregations("age_stats", aggregationBuilder -> aggregationBuilder
                            .stats(statsAggregationBuilder -> statsAggregationBuilder
                                    .field("age")))
            , Poet.class);
    logger.info(response.toString());

    //select name,count(*) from poet-index group by name
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .aggregations("name_terms", aggregationBuilder -> aggregationBuilder
                            .terms(termsAggregationBuilder -> termsAggregationBuilder
                                    .field("name")))
            , Poet.class);
    logger.info(response.toString());

    //select name,age,count(*) from poet-index group by name,age
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .aggregations("name_terms", aggregationBuilder -> aggregationBuilder
                            .terms(termsAggregationBuilder -> termsAggregationBuilder
                                    .field("name")
                            )
                            .aggregations("age_terms", aggregationBuilder2 -> aggregationBuilder2
                                    .terms(termsAggregationBuilder -> termsAggregationBuilder
                                            .field("age")
                                    ))
                    )
            , Poet.class);
    logger.info(response.toString());

    //類似 select avg(age) from poet-index where name='李白'
    response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .bool(boolQueryBuilder -> boolQueryBuilder
                                    .filter(queryBuilder2 -> queryBuilder2
                                            .term(termQueryBuilder -> termQueryBuilder
                                                    .field("name").value("李白")))))
                    .aggregations("ave_age", aggregationBuilder -> aggregationBuilder
                            .avg(averageAggregationBuilder -> averageAggregationBuilder.field("age")))
            , Poet.class);
    logger.info(response.toString());
}

3.4.9、推薦搜索

@Test
public void searchSuggest() throws IOException {
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .suggest(suggesterBuilder -> suggesterBuilder
                            .suggesters("success_suggest", fieldSuggesterBuilder -> fieldSuggesterBuilder
                                    .text("思考")
                                    .term(termSuggesterBuilder -> termSuggesterBuilder
                                            .field("success")
                                            .suggestMode(SuggestMode.Always)
                                            .minWordLength(2)
                                    )
                            )
                    )
            , Poet.class);
    logger.info(response.toString());
}

該測驗會報如下錯誤：

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.core.search.TermSuggest: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'START_ARRAY' instead of '[START_OBJECT, KEY_NAME]' (JSON path: suggest['term#success_suggest'][0].options) (line no=1, column no=247, offset=-1)

還有高手知道原因？

3.4.10、高亮顯示

@Test
public void searchHighlight() throws IOException {
    SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                .index(INDEX_NAME)
                .query(queryBuilder -> queryBuilder
                        .match(matchQueryBuilder -> matchQueryBuilder
                                .field("success").query("思想")))
                .highlight(highlightBuilder -> highlightBuilder
                        .preTags("<span color='red'>")
                        .postTags("</span>")
                        .fields("success", highlightFieldBuilder -> highlightFieldBuilder))
        , Poet.class);
    logger.info(response.toString());
}

3.4.11、SQL 查詢

@Test
public void searchSql() throws IOException {
    QueryResponse response = client.sql().query(builder -> builder
            .format("json").query("SELECT * FROM \"" + INDEX_NAME + "\" limit 3"));
    logger.info(response.toString());
}

該測驗會報如下錯誤：

co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.sql.QueryResponse: java.lang.UnsupportedOperationException (JSON path: rows[0][0]) (line no=1, column no=184, offset=-1)

還有高手知道原因？

3.5、完整代碼

package com.abc.demo.es;

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch._types.*;
import co.elastic.clients.elasticsearch._types.aggregations.AverageAggregation;
import co.elastic.clients.elasticsearch._types.aggregations.CardinalityAggregation;
import co.elastic.clients.elasticsearch._types.aggregations.SumAggregation;
import co.elastic.clients.elasticsearch._types.query_dsl.*;
import co.elastic.clients.elasticsearch.cat.IndicesResponse;
import co.elastic.clients.elasticsearch.core.*;
import co.elastic.clients.elasticsearch.core.bulk.BulkOperation;
import co.elastic.clients.elasticsearch.indices.*;
import co.elastic.clients.elasticsearch.sql.QueryResponse;
import co.elastic.clients.json.JsonData;
import co.elastic.clients.json.jackson.JacksonJsonpMapper;
import co.elastic.clients.transport.ElasticsearchTransport;
import co.elastic.clients.transport.rest_client.RestClientTransport;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class ElasticsearchJavaCase {
    private static final Logger logger = LoggerFactory.getLogger(ElasticsearchJavaCase.class.getName());

    private static final String INDEX_NAME = "poet-index";

    private ElasticsearchTransport transport;
    private ElasticsearchClient client;

    @Before
    public void before() {
        RestClient restClient = RestClient.builder(
                new HttpHost("10.49.196.10", 9200),
                new HttpHost("10.49.196.11", 9200),
                new HttpHost("10.49.196.12", 9200)).build();
        ObjectMapper objectMapper = new ObjectMapper();
        transport = new RestClientTransport(restClient, new JacksonJsonpMapper(objectMapper));
        client = new ElasticsearchClient(transport);
    }

    @After
    public void after() throws IOException {
        transport.close();
    }

    /**
     * 創建索引
     */
    @Test
    public void createIndex() throws IOException {
        CreateIndexResponse response = client.indices().create(builder -> builder
                .settings(indexSettingsBuilder -> indexSettingsBuilder.numberOfReplicas("1").numberOfShards("2"))
                .mappings(typeMappingBuilder -> typeMappingBuilder
                        .properties("age", propertyBuilder -> propertyBuilder.integer(integerNumberPropertyBuilder -> integerNumberPropertyBuilder))
                        .properties("name", propertyBuilder -> propertyBuilder.keyword(keywordPropertyBuilder -> keywordPropertyBuilder))
                        .properties("poems", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word")))
                        .properties("about", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word")))
                        .properties("success", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_max_word")))
                )
                .index(INDEX_NAME));
        logger.info("acknowledged={}", response.acknowledged());
    }

    /**
     * 修改索引的_mapping資訊
     * 欄位可以新增，已有的欄位只能修改欄位的search_analyzer屬性
     */
    @Test
    public void modifyIndex() throws IOException {
        PutMappingResponse response = client.indices().putMapping(typeMappingBuilder -> typeMappingBuilder
                .index(INDEX_NAME)
                .properties("age", propertyBuilder -> propertyBuilder.integer(integerNumberPropertyBuilder -> integerNumberPropertyBuilder))
                .properties("name", propertyBuilder -> propertyBuilder.keyword(keywordPropertyBuilder -> keywordPropertyBuilder))
                .properties("poems", propertyBuilder -> propertyBuilder.text(textPropertyBuilder -> textPropertyBuilder.analyzer("ik_max_word").searchAnalyzer("ik_smart")))
        );
        logger.info("acknowledged={}", response.acknowledged());
    }

    /**
     * 洗掉索引
     */
    @Test
    public void deleteIndex() throws IOException {
        DeleteIndexResponse response = client.indices().delete(builder -> builder.index(INDEX_NAME));
        logger.info("acknowledged={}", response.acknowledged());
    }

    /**
     * 查詢索引串列
     */
    @Test
    public void getIndex() throws IOException {
        //使用 * 也可以
        GetIndexResponse response = client.indices().get(builder -> builder.index("_all"));
        logger.info(response.result().toString());
    }

    /**
     * 查詢索引詳情
     */
    @Test
    public void getIndexDetail() throws IOException {
        GetIndexResponse response = client.indices().get(builder -> builder.index(INDEX_NAME));
        logger.info(response.result().toString());
    }

    /**
     * 創建檔案
     */
    @Test
    public void createDoc() throws IOException {
        Map<String, Object> doc = new HashMap<>();
        doc.put("age", 30);
        doc.put("name", "李白");
        doc.put("poems", "靜夜思");
        doc.put("about", "字太白");
        doc.put("success", "創造了古代浪漫主義文學高峰、歌行體和七絕達到后人難及的高度");

        CreateResponse response = client.create(builder -> builder.index(INDEX_NAME).id("1").document(doc));
        logger.info(response.toString());

        Poet poet = new Poet(31, "杜甫", "登高", "字子美", "唐代偉大的現實主義文學作家，唐詩思想藝術的集大成者");
        response = client.create(builder -> builder.index(INDEX_NAME).id("2").document(poet));
        logger.info(response.toString());
    }

    /**
     * 洗掉檔案
     */
    @Test
    public void deleteDoc() throws IOException {
        DeleteResponse response = client.delete(builder -> builder.index(INDEX_NAME).id("1"));
        logger.info(response.toString());
    }

    /**
     * 修改檔案，只修改設定的欄位
     */
    @Test
    public void updateDoc() throws IOException {
        Map<String, Object> doc = new HashMap<>();
        doc.put("age", 33);
        doc.put("name", "李白2");

        UpdateResponse response = client.update(builder -> builder.index(INDEX_NAME).id("1").doc(doc), Map.class);
        logger.info(response.toString());

        Poet poet = new Poet();
        poet.setAge(40);
        poet.setName("杜甫2");
        response = client.update(builder -> builder.index(INDEX_NAME).id("2").doc(poet).docAsUpsert(true), Poet.class);
        logger.info(response.toString());
    }

    /**
     * 新增或修改檔案，修改時所有的欄位都會覆寫(相當于先洗掉在新增)
     */
    @Test
    public void createOrUpdateDoc() throws IOException {
        Map<String, Object> doc = new HashMap<>();
        doc.put("age", 33);
        doc.put("name", "李白2");

        //只更新設定的欄位
        IndexResponse response = client.index(builder -> builder.index(INDEX_NAME).id("1").document(doc));
        logger.info(response.toString());

        Poet poet = new Poet();
        poet.setAge(40);
        poet.setName("杜甫2");
        response = client.index(builder -> builder.index(INDEX_NAME).id("2").document(poet));
        logger.info(response.toString());
    }


    /**
     * 批量操作
     */
    @Test
    public void bulk() throws IOException {
        List<BulkOperation> list = new ArrayList<>();

        //批量新增
        for (int i = 0; i < 5; i++) {
            Map<String, Object> doc = new HashMap<>();
            doc.put("age", 30);
            doc.put("name", "李白" + i);
            doc.put("poems", "靜夜思");
            doc.put("about", "字太白");
            doc.put("success", "創造了古代浪漫主義文學高峰、歌行體和七絕達到后人難及的高度");
            String id = 10 + i + "";
            list.add(new BulkOperation.Builder().create(builder -> builder.index(INDEX_NAME).id(id).document(doc)).build());
        }
        for (int i = 0; i < 5; i++) {
            Poet poet = new Poet(31, "杜甫" + i, "登高", "字子美", "唐代偉大的現實主義文學作家，唐詩思想藝術的集大成者");
            String id = 20 + i + "";
            list.add(new BulkOperation.Builder().create(builder -> builder.index(INDEX_NAME).id(id).document(poet)).build());
        }

        //批量洗掉
        list.add(new BulkOperation.Builder().delete(builder -> builder.index(INDEX_NAME).id("1")).build());
        list.add(new BulkOperation.Builder().delete(builder -> builder.index(INDEX_NAME).id("2")).build());

        BulkResponse response = client.bulk(builder -> builder.index(INDEX_NAME).operations(list));
        logger.info(response.toString());
    }

    /**
     * 查詢索有檔案
     */
    @Test
    public void getDocAll() throws IOException {
        SearchResponse<Map> response = client.search(builder -> builder.index(INDEX_NAME), Map.class);
        logger.info(response.toString());
    }

    /**
     * 查詢單個檔案
     */
    @Test
    public void getDoc() throws IOException {
        GetResponse<Map> response = client.get(builder -> builder.index(INDEX_NAME).id("1"), Map.class);
        if (response.found()) {
            logger.info(response.source().toString());
        }

        GetResponse<Poet> response2 = client.get(builder -> builder.index(INDEX_NAME).id("2"), Poet.class);
        if (response2.found()) {
            logger.info(response2.source().toString());
        }
    }

    /**
     * term/terms查詢,對輸入內容不做分詞處理
     */
    @Test
    public void searchTerm() throws IOException {
        SearchResponse<Map> response = client.search(searchRequestBuilder -> searchRequestBuilder
                .index(INDEX_NAME)
                .query(queryBuilder -> queryBuilder
                        .term(termQueryBuilder -> termQueryBuilder
                                .field("name").value("李白")))
                .sort(sortOptionsBuilder -> sortOptionsBuilder
                        .field(fieldSortBuilder -> fieldSortBuilder
                                .field("name").order(SortOrder.Asc)))
                .source(sourceConfigBuilder -> sourceConfigBuilder
                        .filter(sourceFilterBuilder -> sourceFilterBuilder
                                .includes("age", "name")))
                .from(0)
                .size(10)
                , Map.class);
        logger.info(response.toString());

        List<FieldValue> words = new ArrayList<>();
        words.add(new FieldValue.Builder().stringValue("李白").build());
        words.add(new FieldValue.Builder().stringValue("杜甫").build());
        SearchResponse<Poet> response2 = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .terms(termsQueryBuilder -> termsQueryBuilder
                                        .field("name").terms(termsQueryFieldBuilder -> termsQueryFieldBuilder.value(words))))
                        .source(sourceConfigBuilder -> sourceConfigBuilder
                                .filter(sourceFilterBuilder -> sourceFilterBuilder
                                        .excludes("about")))
                        .from(0)
                        .size(10)
                , Poet.class);
        logger.info(response2.toString());
    }

    /**
     * range查詢,范圍查詢
     */
    @Test
    public void searchRange() throws IOException {
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .range(rangeQueryBuilder -> rangeQueryBuilder
                                        .field("age").gte(JsonData.of("20")).lt(JsonData.of("40"))))
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * match查詢，對輸入內容先分詞再查詢
     */
    @Test
    public void searchMatch() throws IOException {
        SearchResponse<Map> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .match(matchQueryBuilder -> matchQueryBuilder
                                        .field("success").query("思想")))
                , Map.class);
        logger.info(response.toString());
    }

    /**
     * multi_match查詢,-
     */
    @Test
    public void searchMultiMatch() throws IOException {
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .multiMatch(multiMatchQueryBuilder -> multiMatchQueryBuilder
                                        .fields("about", "success").query("思想")))
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * match_phrase 查詢,匹配整個查詢字串
     */
    @Test
    public void searchMatchPhrase() throws IOException {
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .matchPhrase(matchPhraseQueryBuilder -> matchPhraseQueryBuilder.field("success").query("文學作家")))
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * match_all 查詢,查詢所有
     */
    @Test
    public void searchMatchAll() throws IOException {
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .matchAll(matchAllQueryBuilder -> matchAllQueryBuilder))
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * query_string 查詢
     */
    @Test
    public void searchQueryString() throws IOException {
        //類似 match
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                    .defaultField("success").query("古典文學")))
                , Poet.class);
        logger.info(response.toString());

        //類似 mulit_match
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                        .fields("about", "success").query("古典文學")))
                , Poet.class);
        logger.info(response.toString());

        //類似 match_phrase
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                        .defaultField("success").query("\"文學作家\"")))
                , Poet.class);
        logger.info(response.toString());

        //帶運算子查詢，運算子兩邊的詞不再分詞
        //查詢同時包含 ”文學“ 和 ”偉大“ 的檔案
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                        .fields("success").query("文學 AND 偉大")))
                , Poet.class);
        logger.info(response.toString());

        //等同上一個查詢
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                        .fields("success").query("文學 偉大").defaultOperator(Operator.And)))
                , Poet.class);
        logger.info(response.toString());

        //查詢 name 或 success 欄位包含"文學"和"偉大"這兩個單詞，或者包含"李白"這個單詞的檔案，
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .queryString(queryStringQueryBuilder -> queryStringQueryBuilder
                                        .fields("name","success").query("(文學 AND 偉大) OR 高度")))
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * simple_query_string 查詢,和query_string類似
     * 不支持AND OR NOT，會當做字串處理
     * 使用 +替代AND,|替代OR,-替代NOT
     */
    @Test
    public void searchSimpleQueryString() throws IOException {
        //查詢同時包含 ”文學“ 和 ”偉大“ 的檔案
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .simpleQueryString(simpleQueryStringQueryBuilder -> simpleQueryStringQueryBuilder
                                        .fields("success").query("文學 + 偉大")))
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * 模糊查詢
     */
    @Test
    public void searchFuzzy() throws IOException {
        //全文查詢時使用模糊引數，先分詞再計算模糊選項，
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .match(matchQueryBuilder -> matchQueryBuilder
                                        .field("success").query("思考").fuzziness("1")))
                , Poet.class);
        logger.info(response.toString());

        //使用 fuzzy query，對輸入不分詞，直接計算模糊選項，
        SearchResponse<Poet> response2 = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .fuzzy(fuzzyQueryBuilder ->  fuzzyQueryBuilder
                                        .field("success").fuzziness("1").value("理想")))
                , Poet.class);
        logger.info(response2.toString());
    }

    /**
     * bool查詢,組合查詢
     */
    @Test
    public void searchBool() throws IOException {
        //查詢 success 包含 “思想” 且 age 在 [20-40] 之間的檔案
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                .index(INDEX_NAME)
                .query(queryBuilder -> queryBuilder
                        .bool(boolQueryBuilder -> boolQueryBuilder
                                .must(queryBuilder2 -> queryBuilder2
                                        .match(matchQueryBuilder -> matchQueryBuilder
                                                .field("success").query("思想"))
                                )
                                .must(queryBuilder2 -> queryBuilder2
                                        .range(rangeQueryBuilder -> rangeQueryBuilder
                                                .field("age").gte(JsonData.of("20")).lt(JsonData.of("40")))
                                )
                        )
                )
        , Poet.class);
        logger.info(response.toString());

        //過濾出 success 包含 “思想” 且 age 在 [20-40] 之間的檔案，不計算得分
        SearchResponse<Poet> response2 = client.search(searchRequestBuilder -> searchRequestBuilder
                .index(INDEX_NAME)
                .query(queryBuilder -> queryBuilder
                        .bool(boolQueryBuilder -> boolQueryBuilder
                                .filter(queryBuilder2 -> queryBuilder2
                                        .match(matchQueryBuilder -> matchQueryBuilder
                                                .field("success").query("思想"))
                                )
                                .filter(queryBuilder2 -> queryBuilder2
                                        .range(rangeQueryBuilder -> rangeQueryBuilder
                                                .field("age").gte(JsonData.of("20")).lt(JsonData.of("40")))
                                )
                        )
                )
        , Poet.class);
        logger.info(response2.toString());
    }

    /**
     * aggs查詢,聚合查詢
     */
    @Test
    public void searchAggs() throws IOException {
        //求和
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .aggregations("age_sum", aggregationBuilder -> aggregationBuilder
                                .sum(sumAggregationBuilder -> sumAggregationBuilder
                                        .field("age")))
                , Poet.class);
        logger.info(response.toString());

        //類似 select count distinct(age) from poet-index
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .aggregations("age_count", aggregationBuilder -> aggregationBuilder
                                .cardinality(cardinalityAggregationBuilder -> cardinalityAggregationBuilder.field("age")))
                , Poet.class);
        logger.info(response.toString());

        //數量、最大、最小、平均、求和
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .aggregations("age_stats", aggregationBuilder -> aggregationBuilder
                                .stats(statsAggregationBuilder -> statsAggregationBuilder
                                        .field("age")))
                , Poet.class);
        logger.info(response.toString());

        //select name,count(*) from poet-index group by name
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .aggregations("name_terms", aggregationBuilder -> aggregationBuilder
                                .terms(termsAggregationBuilder -> termsAggregationBuilder
                                        .field("name")))
                , Poet.class);
        logger.info(response.toString());

        //select name,age,count(*) from poet-index group by name,age
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .aggregations("name_terms", aggregationBuilder -> aggregationBuilder
                                .terms(termsAggregationBuilder -> termsAggregationBuilder
                                        .field("name")
                                )
                                .aggregations("age_terms", aggregationBuilder2 -> aggregationBuilder2
                                        .terms(termsAggregationBuilder -> termsAggregationBuilder
                                                .field("age")
                                        ))
                        )
                , Poet.class);
        logger.info(response.toString());

        //類似 select avg(age) from poet-index where name='李白'
        response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .query(queryBuilder -> queryBuilder
                                .bool(boolQueryBuilder -> boolQueryBuilder
                                        .filter(queryBuilder2 -> queryBuilder2
                                                .term(termQueryBuilder -> termQueryBuilder
                                                        .field("name").value("李白")))))
                        .aggregations("ave_age", aggregationBuilder -> aggregationBuilder
                                .avg(averageAggregationBuilder -> averageAggregationBuilder.field("age")))
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * suggest查詢,推薦搜索, 報錯
     */
    @Test
    public void searchSuggest() throws IOException {
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                        .index(INDEX_NAME)
                        .suggest(suggesterBuilder -> suggesterBuilder
                                .suggesters("success_suggest", fieldSuggesterBuilder -> fieldSuggesterBuilder
                                        .text("思考")
                                        .term(termSuggesterBuilder -> termSuggesterBuilder
                                                .field("success")
                                                .suggestMode(SuggestMode.Always)
                                                .minWordLength(2)
                                        )
                                )
                        )
                , Poet.class);
        logger.info(response.toString());
    }

    /**
     * 高亮顯示
     */
    @Test
    public void searchHighlight() throws IOException {
        SearchResponse<Poet> response = client.search(searchRequestBuilder -> searchRequestBuilder
                    .index(INDEX_NAME)
                    .query(queryBuilder -> queryBuilder
                            .match(matchQueryBuilder -> matchQueryBuilder
                                    .field("success").query("思想")))
                    .highlight(highlightBuilder -> highlightBuilder
                            .preTags("<span color='red'>")
                            .postTags("</span>")
                            .fields("success", highlightFieldBuilder -> highlightFieldBuilder))
            , Poet.class);
        logger.info(response.toString());
    }

    /**
     * sql查詢，報錯
     */
    @Test
    public void searchSql() throws IOException {
        QueryResponse response = client.sql().query(builder -> builder
                .format("json").query("SELECT * FROM \"" + INDEX_NAME + "\" limit 1"));
        logger.info(response.toString());
    }


    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    static class Poet {
        private Integer age;
        private String name;
        private String poems;
        private String about;
        /**成就*/
        private String success;
    }
}

ElasticsearchJavaCase.java

詳細的 Elasticsearch Java API Client 使用說明，請參考官網檔案：https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/index.html，

轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/500094.html

標籤：其他

上一篇：day03

下一篇：C++多型性