ElasticSearch學習筆記

1. ElasticSearch概述
2. ES與Solr的差別
- 2.1. Solr簡介
- 2.2. Lucene簡介
- 2.3. ES VS Solr
3. ElasticSearch 安裝
4. Kibana安裝
5. ES核心概念
6. IK分詞器
7. Restful風格說明
8. 關于檔案的基本操作
9. 集成SpringBoot
10. 實戰：模擬全文搜索-京東搜索

1. ElasticSearch概述

在這里插入圖片描述

2. ES與Solr的差別

2.1. Solr簡介

在這里插入圖片描述

2.2. Lucene簡介

在這里插入圖片描述

2.3. ES VS Solr

在這里插入圖片描述

3. ElasticSearch 安裝

在這里插入圖片描述

官網

ElasticSearch: https://mirrors.huaweicloud.com/elasticsearch/?C=N&O=D
logstash: https://mirrors.huaweicloud.com/logstash/?C=N&O=D
kibana: https://mirrors.huaweicloud.com/kibana/?C=N&O=D

在這里插入圖片描述

測驗訪問

在這里插入圖片描述

這個，沒有測驗，等后期回來再看！head的插件

4. Kibana安裝

開箱即用

組態檔

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://192.168.1.30:9201"]
kibana.index: ".kibana"
i18n.locale: "zh-CN"			# 中文漢化

訪問測驗

在這里插入圖片描述

5. ES核心概念

索引
欄位型別（mapping）
檔案（document）

在這里插入圖片描述

6. IK分詞器

下載鏈接
解壓放入到es對應的plugins下即可
重啟觀察ES，發現ik插件被加載了

在這里插入圖片描述

也可以通過bin目錄下elasticsearch-plugin list 查看已經加載的插件
使用kibana測驗
- ik_smart: 最少切分
- ik_max_word為最細粒度劃分！窮盡詞庫的可能，字典！
ik分詞器增加自己的配置！

在這里插入圖片描述

重啟ES 和 Kibana

在這里插入圖片描述

7. Restful風格說明

在這里插入圖片描述

基礎測驗

創建一個索引！

PUT /索引名/~型別名~/檔案id
{請求體}

# PUT 創建命令  test1 索引 type1 型別 1 id
PUT test1/type1/1
{
  "name": "xiaofan",
  "age": 28
}

# 回傳結果
# 警告資訊： 不支持在檔案索引請求中的指定型別
# 而是使用無型別的斷點(/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id}).
{
  "_index" : "test1",	# 索引
  "_type" : "type1",	# 型別（已經廢棄）
  "_id" : "1",			# id
  "_version" : 1,		# 版本
  "result" : "created",	# 操作型別
  "_shards" : {			# 分片資訊
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

在這里插入圖片描述

指定欄位的型別（創建規則）

在這里插入圖片描述

獲取具體的索引規則

# GET test2

{
  "test2" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "integer"
        },
        "birthday" : {
          "type" : "date"
        },
        "name" : {
          "type" : "text"
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1599708623941",
        "number_of_shards" : "1",
        "number_of_replicas" : "1",
        "uuid" : "ANWnhwArSMSl8k8iipgH1Q",
        "version" : {
          "created" : "7080099"
        },
        "provided_name" : "test2"
      }
    }
  }
}

# 查看默認的規則
PUT /test3/_doc/1
{
  "name": "狂神說Java",
  "age": 28,
  "birthday": "1997-01-05"
}

# GET test3

{
  "test3" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "long"
        },
        "birthday" : {
          "type" : "date"
        },
        "name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1599708906181",
        "number_of_shards" : "1",
        "number_of_replicas" : "1",
        "uuid" : "LzPLCDgeQn6tdKo3xBBpbw",
        "version" : {
          "created" : "7080099"
        },
        "provided_name" : "test3"
      }
    }
  }
}

在這里插入圖片描述

修改索引 POST

# 只會修改指定項，其他內容保證不變
POST /test3/_doc/1/_update
{
  "doc": {
    "name":"暴徒狂神"
  }
}

# GET test3/_doc/1

{
  "_index" : "test3",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 2,
  "_seq_no" : 1,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "暴徒狂神",
    "age" : 28,
    "birthday" : "1997-01-05"
  }
}

在這里插入圖片描述

8. 關于檔案的基本操作

基本操作（簡單的查詢）

put /kuangshen/user/1
{
  "name": "狂神說",
  "age": 23,
  "desc": "一頓操作猛如虎，一看工資2500",
  "tags": ["碼農", "技術宅", "直男"]
}

put /kuangshen/user/2
{
  "name": "張三",
  "age": 28,
  "desc": "法外狂徒",
  "tags": ["旅游", "渣男", "交友"]
}

put /kuangshen/user/3
{
  "name": "李四",
  "age": 30,
  "desc": "不知道怎么描述",
  "tags": ["旅游", "靚女", "唱歌"]
}

GET kuangshen/user/1


GET kuangshen/user/_search?q=name:狂神

復雜操作(排序、分頁、高亮、模糊查詢、標準查詢！)

在這里插入圖片描述

# 模糊查詢
GET kuangshen/user/_search
{
  "query": {
    "match": {
      "name": "狂神"
    }
  }
}

# 對查詢結果進行欄位過濾
GET kuangshen/user/_search
{
  "query": {
    "match": {
      "name": "狂神"
    }
  },
  "_source": ["name", "desc"]
}

# 排序
GET kuangshen/user/_search
{
  "query": {
    "match": {
      "name": "狂神"
    }
  },
  "sort":[{
    "age": "asc"
  }]
}

# 分頁
GET kuangshen/user/_search
{
  "query": {
    "match": {
      "name": "狂神"
    }
  },
  "sort":[{
    "age": "asc"
  }], 
  "from": 0,
  "size": 2
}

布林值條件查詢

# 多條件查詢 must 相當于and
GET kuangshen/user/_search
{
  "query": {
    "bool": {
      "must": [
        {"match": {
          "name": "狂神"
        }},
        {"match": {
          "age": 23
        }}
      ]
    }
  }
}

# 多條件查詢 should 相當于or
GET kuangshen/user/_search
{
  "query": {
    "bool": {
      "should": [
        {"match": {
          "name": "狂神說"
        }},
        {"match": {
          "age": 25
        }}
      ]
    }
  }
}

# 多條件查詢 must_not 相當于 not
GET kuangshen/user/_search
{
  "query": {
    "bool": {
      "must_not": [
        {"match": {
          "age": 25
        }}
      ]
    }
  }
}


# 過濾查詢1 age > 24
GET kuangshen/user/_search
{
  "query": {
    "bool": {
      "must": [
        {"match": {
          "name": "狂神"
        }}
      ],
      "filter": [
        {"range": {
          "age": {
            "gt": 24
          }
        }}
      ]
    }
  }
}

# 過濾器2  22<age<30 
GET kuangshen/user/_search
{
  "query": {
    "bool": {
      "must": [
        {"match": {
          "name": "狂神"
        }}
      ],
      "filter": [
        {"range": {
          "age": {
            "lt": 30,
            "gt": 22
          }
        }}
      ]
    }
  }
}

多條件查詢

GET kuangshen/user/_search
{
  "query": {
    "match": {
      "tags": "技術 男"
    }
  }
}

在這里插入圖片描述

keyword型別不會被分詞器決議

term: 精確匹配

# 定義型別
PUT xiaofan_test_db
{
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "desc": {
        "type": "keyword"
      }
    }
  }
}


PUT /xiaofan_test_db/_doc/1
{
  "name": "小范說Java Name",
  "desc": "小范說Java Desc"
}

PUT /xiaofan_test_db/_doc/2
{
  "name": "小范說Java Name",
  "desc": "小范說Java Desc 2"
}

# 按照keyword型別精準匹配
GET xiaofan_test_db/_search
{
  "query": {
    "term": {
      "desc": "小范說Java Desc"
    }
  }
}
# 結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 0.6931471,
    "hits" : [
      {
        "_index" : "test_db",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 0.6931471,
        "_source" : {
          "name" : "小范說Java Name",
          "desc" : "小范說Java Desc"
        }
      }
    ]
  }
}

# 按照text型別匹配
GET xiaofan_test_db/_search
{
  "query": {
    "term": {
      "name": "小"
    }
  }
}

# 結果：
{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 0.18232156,
    "hits" : [
      {
        "_index" : "test_db",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 0.18232156,
        "_source" : {
          "name" : "小范說Java Name",
          "desc" : "小范說Java Desc"
        }
      },
      {
        "_index" : "test_db",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 0.18232156,
        "_source" : {
          "name" : "小范說Java Name",
          "desc" : "小范說Java Desc 2"
        }
      }
    ]
  }
}

多個值匹配精確查詢

PUT /test_db/_doc/3
{
  "t1": "22",
  "t2": "2020-09-10"
}

PUT /test_db/_doc/4
{
  "t1": "33",
  "t2": "2020-09-11"
}

GET test_db/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "term": {
            "t1": "22"
          }
        },
         {
          "term": {
            "t1": "33"
          }
        }
      ]
    }
  }
}

高亮查詢

GET kuangshen/user/_search
{
  "query": {
    "match": {
      "name": "狂神"
    }
  },
  "highlight": {
    "pre_tags": "<p class='key' style='color:red'>",
    "post_tags": "</p>", 
    "fields": {
      "name": {}
    }
  }
}

# 結果顯示：
#! Deprecation: [types removal] Specifying types in search requests is deprecated.
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.3862942,
    "hits" : [
      {
        "_index" : "kuangshen",
        "_type" : "user",
        "_id" : "1",
        "_score" : 1.3862942,
        "_source" : {
          "name" : "狂神說",
          "age" : 23,
          "desc" : "一頓操作猛如虎，一看工資2500",
          "tags" : [
            "碼農",
            "技術宅",
            "直男"
          ]
        },
        "highlight" : {
          "name" : [
            "<p class='key' style='color:red'>狂</p><p class='key' style='color:red'>神</p>說"
          ]
        }
      },
      {
        "_index" : "kuangshen",
        "_type" : "user",
        "_id" : "4",
        "_score" : 1.0892314,
        "_source" : {
          "name" : "狂神說前端",
          "age" : 25,
          "desc" : "大王叫我來巡山",
          "tags" : [
            "碼農1",
            "技術宅1",
            "直男1"
          ]
        },
        "highlight" : {
          "name" : [
            "<p class='key' style='color:red'>狂</p><p class='key' style='color:red'>神</p>說前端"
          ]
        }
      }
    ]
  }
}

9. 集成SpringBoot

官網
添加依賴

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

自定義配置

package com.xiaofan.config;

import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ElasticSearchClientConfig {

    @Bean
    public RestHighLevelClient restHighLevelClient() {
        RestHighLevelClient client = new RestHighLevelClient(
            RestClient.builder(
                new HttpHost("192.168.1.30", 9201, "http")
            )
        );

        return client;
    }


}

撰寫測驗類

package com.xiaofan;

import com.alibaba.fastjson.JSON;
import com.xiaofan.pojo.User;
import org.apache.http.entity.ContentType;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.index.query.MatchAllQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.TermQueryBuilder;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.fetch.subphase.FetchSourceContext;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.test.context.SpringBootTest;

import java.io.IOException;
import java.util.ArrayList;
import java.util.concurrent.TimeUnit;

@SpringBootTest
class EsApiApplicationTests {

	public static final String INDEX = "xiaofan_test_index";

	@Autowired
	@Qualifier(value = "restHighLevelClient")
	private RestHighLevelClient client;

	// 創建索引
	@Test
	void testCreateIndex() throws IOException {
		// 1. 創建索引請求
		CreateIndexRequest request = new CreateIndexRequest(INDEX);
		// 2. 客戶端執行請求， IndicesClient，請求后獲得回應
		CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);
		System.out.println(createIndexResponse);
	}

	// 測驗索引存在
	@Test
	void testExistsIndex() throws IOException {
		GetIndexRequest request = new GetIndexRequest(INDEX);
		boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);
		System.out.println(exists);
	}

	// 洗掉索引
	@Test
	void testDeleteIndex() throws IOException {
		DeleteIndexRequest request = new DeleteIndexRequest(INDEX);
		AcknowledgedResponse acknowledgedResponse = client.indices().delete(request, RequestOptions.DEFAULT);
		System.out.println(acknowledgedResponse.isAcknowledged());
	}

	// 添加檔案
	@Test
	void testAddDocument() throws IOException {
		User user = new User("狂神說", 28);
		IndexRequest request = new IndexRequest(INDEX);
		// 規則 PUT /index/_doc/1
		request.id("1");
		request.timeout(TimeValue.timeValueSeconds(1));
		// 將資料放入請求 json
		request.source(JSON.toJSONString(user), XContentType.JSON);
		IndexResponse response = client.index(request, RequestOptions.DEFAULT);
		System.out.println(response.toString());
		System.out.println(response.status());
	}

	// 獲取檔案 判斷是否存在 GET /index/_doc/1
	@Test
	void testIsExists() throws IOException {
		GetRequest request = new GetRequest(INDEX, "1");
		// 不獲取回傳的 _source 的背景關系了
		request.fetchSourceContext(new FetchSourceContext(false));
		request.storedFields("_none_");

		boolean exists = client.exists(request, RequestOptions.DEFAULT);
		System.out.println(exists);
	}

	// 獲取檔案

	/**
	 * 回傳結果：
	 * {"age":28,"name":"狂神說"}
	 * {"_index":"xiaofan_test_index","_type":"_doc","_id":"1","_version":1,"_seq_no":0,"_primary_term":1,"found":true,"_source":{"age":28,"name":"狂神說"}}
	 */
	@Test
	void testGetDocument() throws IOException {
		GetRequest request = new GetRequest(INDEX, "1");
		GetResponse response = client.get(request, RequestOptions.DEFAULT);
		System.out.println(response.getSourceAsString());
		System.out.println(response);
	}

	// 更新檔案
	@Test
	void testUpdateDocument() throws IOException {
		UpdateRequest request = new UpdateRequest(INDEX, "1");
		request.timeout("1s");

		User user = new User("小范說Java", 18);
		request.doc(JSON.toJSONString(user), XContentType.JSON);

		UpdateResponse updateResponse = client.update(request, RequestOptions.DEFAULT);
		System.out.println(updateResponse);
	}

	// 洗掉檔案
	@Test
	void testDeleteDocument() throws IOException {
		DeleteRequest request = new DeleteRequest(INDEX, "1");
		request.timeout("1s");

		DeleteResponse deleteResponse = client.delete(request, RequestOptions.DEFAULT);
		System.out.println(deleteResponse);

	}

	// 批量插入資料（修改，洗掉類似操作）
	@Test
	void testBulkRequest() throws IOException {
		BulkRequest request = new BulkRequest();
		request.timeout("10s");

		ArrayList<User> users = new ArrayList<>();
		users.add(new User("kuangshen1", 21));
		users.add(new User("kuangshen2", 22));
		users.add(new User("kuangshen3", 23));
		users.add(new User("xiaofan1", 18));
		users.add(new User("xiaofan2", 19));

		// 批處理請求， 修改，洗掉，只要在這里修改相應的請求就可以
		for (int i = 0; i < users.size(); i++) {
			request.add(new IndexRequest(INDEX)
					.id(String.valueOf(i + 1))
					.source(JSON.toJSONString(users.get(i)), XContentType.JSON));
		}

		BulkResponse bulkResponse = client.bulk(request, RequestOptions.DEFAULT);
		//是否失敗，回傳false表示成功
		System.out.println(bulkResponse.hasFailures());
	}

	// 查詢檔案
	@Test
	void testSearch() throws IOException {
		SearchRequest searchRequest = new SearchRequest(INDEX);
		// 構建搜索條件
		SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();

		// 查詢條件， 可以使用QueryBuilders工具類實作
		// QueryBuilders.termQuery 精確
		// QueryBuilders.matchLLQuery() 匹配所有
		TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", "kuangshen1");
		// MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();
		sourceBuilder.query(termQueryBuilder);
		sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));

		searchRequest.source(sourceBuilder);

		SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
		System.out.println(JSON.toJSON(searchResponse.getHits()));
		System.out.println("======================================");
		for (SearchHit documentFields : searchResponse.getHits().getHits()) {
			System.out.println(documentFields.getSourceAsMap());
		}

	}

}

10. 實戰：模擬全文搜索-京東搜索

github鏈接：https://github.com/fanjianhai/SpringBoot-ElasticSearch
搭建springboot專案，添加依賴，修改es版本

<properties>
    <java.version>1.8</java.version>
    <elasticsearch.version>7.2.1</elasticsearch.version>
</properties>

<dependencies>
    <dependency>
        <groupId>com.alibaba</groupId>
        <artifactId>fastjson</artifactId>
        <version>1.2.73</version>
    </dependency>
    <dependency>
        <groupId>org.jsoup</groupId>
        <artifactId>jsoup</artifactId>
        <version>1.13.1</version>
    </dependency>
</dependencies>

整體效果

在這里插入圖片描述

轉載請註明出處，本文鏈接：https://www.uj5u.com/ruanti/48144.html

標籤：其他

上一篇：hadoop | 服務器分布式安裝hadoop,hbase,zookeeper,phoenix,spark | 集群啟動順序

下一篇：Linux中網路連接不上解決方案大全

4. ElasticSearch學習筆記

ElasticSearch學習筆記

1. ElasticSearch概述

2. ES與Solr的差別

2.1. Solr簡介

2.2. Lucene簡介

2.3. ES VS Solr

3. ElasticSearch 安裝

4. Kibana安裝

5. ES核心概念

6. IK分詞器

7. Restful風格說明

8. 關于檔案的基本操作

9. 集成SpringBoot

10. 實戰：模擬全文搜索-京東搜索