文章目錄
- 一、Goreplay 介紹
- 二、Golang環境安裝
- 三、Goreplay 安裝
- 四、Goreplay 使用示例
- 1、準備 RESTful API 環境
- 2、捕獲服務器一流量保存到本地檔案
- 3、將流量從檔案回放到服務器二
- 4、實時將流量請求列印到終端
- 5、復制流量實時轉發到服務器二
- 6、壓力測驗
- 五、總結
一、Goreplay 介紹
- Goreplay 是用 Golang 寫的一個 HTTP 實時流量復制工具,功能更強大,支持流量的放大、縮小,頻率限制,還支持把請求記錄到檔案,方便回放和分析,也支持和 ElasticSearch 集成,將流量存入 ES 進行實時分析,
- GoReplay 不是代理,而是監聽網路介面上的流量,不需要更改生產基礎架構,而是在與服務相同的計算機上運行 GoReplay 守護程式,
- 特點:簡單易用
- 地址:https://github.com/buger/goreplay
- stars:14.1k
與 TCPCopy 相比它的架構更簡單,只有一個 gor 組件,如下:


只需要在生產服務器上啟動一個 gor 行程,它負責所有的作業包括監聽、過濾和轉發,
它的設計遵循 Unix 設計哲學:一切都是由管道組成的,各種輸入將資料復用為輸出,
二、Golang環境安裝
一 、首先安裝Golang及相關依賴環境
在官網上下載安裝包或者go的一些中文網站上下載安裝包
- https://golang.org/dl/
- https://studygolang.com/dl
我下載的是: go1.15.5.linux-amd64.tar.gz
二、解壓到/usr/local目錄下
tar -C /usr/local -zxvf go1.14.4.linux-amd64.tar.gz
三、配置環境變數
# 打開
vim /etc/profile
# 添加
export GOROOT=/usr/local/go
export PATH=$PATH:$GOROOT/bin
# 編譯生效
source /etc/profile
?
四、驗證
go env
[root@vm ~]# go version
go version go1.15.5 linux/amd64
三、Goreplay 安裝
從https://github.com/buger/gor/releases 下載最新的 Gor 二進制檔案(提供 Windows、Linux x64 和 Mac OS 的預編譯二進制檔案),也可以自己編譯 ,

下載二進制包:
[root@vm-1 ~]# curl -L -O https://github.com/buger/goreplay/releases/download/v1.3.1/gor_1.3.1_x64.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 626 100 626 0 0 741 0 --:--:-- --:--:-- --:--:-- 741
100 10.5M 100 10.5M 0 0 3258k 0 0:00:03 0:00:03 --:--:-- 5951k
[root@vm-1 ~]# tar xvzf gor_1.3.1_x64.tar.gz
gor
?
解壓縮包后,您可以從當前目錄運行 Gor,或者您可能希望將二進制檔案復制到您的 PATH(對于 Linux 和 Mac OS,它可以是 /usr/local/bin),
四、Goreplay 使用示例

1、準備 RESTful API 環境
介面設計:
| 請求型別 | 請求路徑 | 功能 |
|---|---|---|
| Get | /person | 查詢所有人 |
| Post | /person/two | 事務測驗 |
打包程式分別放到兩臺服務器 run 起來,
?
服務器一:
[root@vm-1 ~]# java -jar spirng-boot-demo-0.0.1-SNAPSHOT.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.8.RELEASE)
2021-08-17 14:19:10.467 WARN 17537 --- [ main] o.s.boot.StartupInfoLogger : InetAddress.getLocalHost().getHostName() took 10014 milliseconds to respond. Please verify your network configuration.
2021-08-17 14:19:20.494 INFO 17537 --- [ main] com.zuozewei.SpirngbootdemoApplication : Starting SpirngbootdemoApplication v0.0.1-SNAPSHOT on vm-jmeter with PID 17537 (/root/spirng-boot-demo-0.0.1-SNAPSHOT.jar started by root in /root)
2021-08-17 14:19:20.495 INFO 17537 --- [ main] com.zuozewei.SpirngbootdemoApplication : The following profiles are active: a
2021-08-17 14:19:22.225 INFO 17537 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-08-17 14:19:22.355 INFO 17537 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 111ms. Found 1 JPA repository interfaces.
2021-08-17 14:19:23.642 INFO 17537 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8082 (http)
2021-08-17 14:19:23.667 INFO 17537 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2021-08-17 14:19:23.667 INFO 17537 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.41]
2021-08-17 14:19:23.804 INFO 17537 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2021-08-17 14:19:23.805 INFO 17537 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 3179 ms
2021-08-17 14:19:24.314 INFO 17537 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-08-17 14:19:24.479 INFO 17537 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.4.27.Final
2021-08-17 14:19:25.002 INFO 17537 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-08-17 14:19:25.228 INFO 17537 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2021-08-17 14:19:25.635 INFO 17537 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2021-08-17 14:19:25.706 INFO 17537 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MySQL57Dialect
2021-08-17 14:19:27.086 INFO 17537 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-08-17 14:19:27.112 INFO 17537 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-08-17 14:19:27.990 WARN 17537 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2021-08-17 14:19:28.261 INFO 17537 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2021-08-17 14:19:28.749 INFO 17537 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8082 (http) with context path ''
2021-08-17 14:19:28.770 INFO 17537 --- [ main] com.zuozewei.SpirngbootdemoApplication : Started SpirngbootdemoApplication in 39.553 seconds (JVM running for 40.431)
?
服務器二:
[root@vm-2 ~]# java -jar spirng-boot-demo-0.0.1-SNAPSHOT.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.8.RELEASE)
2021-08-17 14:18:34.814 INFO 12775 --- [ main] com.zuozewei.SpirngbootdemoApplication : Starting SpirngbootdemoApplication v0.0.1-SNAPSHOT on vm-hsyjy-znkzb-vbi-1.novalocal with PID 12775 (/root/spirng-boot-demo-0.0.1-SNAPSHOT.jar started by root in /root)
2021-08-17 14:18:34.820 INFO 12775 --- [ main] com.zuozewei.SpirngbootdemoApplication : The following profiles are active: a
2021-08-17 14:18:36.293 INFO 12775 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2021-08-17 14:18:36.419 INFO 12775 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 107ms. Found 1 JPA repository interfaces.
2021-08-17 14:18:37.694 INFO 12775 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8082 (http)
2021-08-17 14:18:37.716 INFO 12775 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2021-08-17 14:18:37.717 INFO 12775 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.41]
2021-08-17 14:18:37.841 INFO 12775 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2021-08-17 14:18:37.841 INFO 12775 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2886 ms
2021-08-17 14:18:38.305 INFO 12775 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2021-08-17 14:18:38.444 INFO 12775 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.4.27.Final
2021-08-17 14:18:38.864 INFO 12775 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.2.Final}
2021-08-17 14:18:39.119 INFO 12775 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2021-08-17 14:18:39.556 INFO 12775 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2021-08-17 14:18:39.648 INFO 12775 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MySQL57Dialect
2021-08-17 14:18:41.052 INFO 12775 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2021-08-17 14:18:41.086 INFO 12775 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2021-08-17 14:18:42.042 WARN 12775 --- [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2021-08-17 14:18:42.330 INFO 12775 --- [ main] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2021-08-17 14:18:42.768 INFO 12775 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8082 (http) with context path ''
2021-08-17 14:18:42.786 INFO 12775 --- [ main] com.zuozewei.SpirngbootdemoApplication : Started SpirngbootdemoApplication in 8.988 seconds (JVM running for 9.939)
2、捕獲服務器一流量保存到本地檔案
?
在終端運行這個命令:
?
#將埠 8082 流量保存到本地的檔案
sudo ./gor --input-raw :8082 --output-file=requests.gor
[root@vm-1 ~]# sudo ./gor --input-raw :8082 --output-file=requests.gor
Interface: eth0 . BPF Filter: ((tcp dst port 8082) and (dst host 172.16.106.149 or dst host fe80::f599:6e65:3dc9:b84a))
Interface: lo . BPF Filter: ((tcp dst port 8082) and (dst host 127.0.0.1 or dst host ::1))
Interface: virbr0 . BPF Filter: ((tcp dst port 8082) and (dst host 192.168.124.1))
2021/08/17 14:26:41 [PPID 17654 and PID 17655] Version:1.3.0
注意:
使用–output-file-append 選項可以將捕獲流量保存為一個單獨的檔案
使用sudo并要求輸入密碼:為了分析網路,Gor 需要只有超級用戶才能使用的權限,但是,可以為非 root 用戶配置運行Gor ,
在 Postman 上向服務器一發送請求:

我們查看服務一日志:
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: insert into person (age, name, id) values (?, ?, ?)
Hibernate: insert into person (age, name, id) values (?, ?, ?)
可以看到服務已經正常處理了,
?
我們停掉在服務器一上運行的 Gor,可以看到已經生成了錄制的請求檔案:
[root@vm-1 ~]# ls -l | grep "requests"
-rw-r----- 1 root root 300 8月 17 14:33 requests_0.gor
?
我們看下請求檔案的內容:
[root@vm-1 ~]# cat requests_0.gor
1 04481f92ac106afb42ce55ca 1629182282437724065 0
GET /person HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: ab405d20-ef93-4ffb-b2f6-a16cf1205e7c
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
?
3、將流量從檔案回放到服務器二
?
現在是時候將原始流量回放到服務二中了,如果是多個請求并且它們將按照與記錄的順序和時間完全相同的順序重放,
?
–output-http 提供第二個服務的 URL
./gor --input-file requests_0.gor --output-http="http://172.16.106.237:8082"
[root@vm-1 ~]./gor --input-file requests_0.gor --output-http="http://172.16.106.237:8082"
2021/08/17 14:42:55 [PPID 12356 and PID 17914] Version:1.3.0
[DEBUG][elapsed 985.792μs]: [INPUT-FILE] FileInput: end of file 'requests_0.gor'
我們看到服務二的日志:
[root@vm-2 ~]# java -jar spirng-boot-demo-0.0.1-SNAPSHOT.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.8.RELEASE)
2021-08-17 14:18:34.814 INFO 12775 --- [ main] com.zuozewei.SpirngbootdemoApplication : Starting SpirngbootdemoApplication v0.0.1-SNAPSHOT on vm-hsyjy-znkzb-vbi-1.novalocal with PID 12775 (/root/spirng-boot-demo-0.0.1-SNAPSHOT.jar started by root in /root)
......
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_
日志說明服務正常回應,
?
?
4、實時將流量請求列印到終端
通過命令:
sudo ./gor --input-raw :8082 --output-stdout
此命令其記錄到標準控制臺輸出,請注意,默認情況下 GoReplay 不跟蹤回應,您可以使用 --output-http-track-response 選項啟用它們,
?
[root@vm-1 ~]sudo ./gor --input-raw :8082 --output-stdout
Interface: eth0 . BPF Filter: ((tcp dst port 8082) and (dst host 172.16.106.149 or dst host fe80::f599:6e65:3dc9:b84a))
Interface: lo . BPF Filter: ((tcp dst port 8082) and (dst host 127.0.0.1 or dst host ::1))
Interface: virbr0 . BPF Filter: ((tcp dst port 8082) and (dst host 192.168.124.1))
2021/08/17 14:55:45 [PPID 18060 and PID 18061] Version:1.3.0
1 22ee1f92ac106afb594d205a 1629183362175214036 0
POST /person/two HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: 4922ce23-5706-42c5-8a54-ade5d5aee785
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 0
5、復制流量實時轉發到服務器二
運行命令:
?
sudo ./gor --input-raw :8082 --output-http="http://172.16.106.237:8082"
[root@vm-1 ~]# sudo ./gor --input-raw :8082 --output-http="http://172.16.106.237:8082"
Interface: eth0 . BPF Filter: ((tcp dst port 8082) and (dst host 172.16.106.149 or dst host fe80::f599:6e65:3dc9:b84a))
Interface: lo . BPF Filter: ((tcp dst port 8082) and (dst host 127.0.0.1 or dst host ::1))
Interface: virbr0 . BPF Filter: ((tcp dst port 8082) and (dst host 192.168.124.1))
2021/08/17 14:59:49 [PPID 18118 and PID 18119] Version:1.3.0
在服務器二查看日志,.發現流量已經進來了:
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_
Hibernate: select person0_.id as id1_0_, person0_.age as age2_0_, person0_.name as name3_0_ from person person0_ where person0_.age=?
6、壓力測驗
goreplay支持將捕獲到的生產實際請求流量減少或者放大重播以用于測驗環境的壓力測驗.壓力測驗一般針對 Input 流量減少或者放大,
?
錄制的請求檔案:
[root@vm-1 ~]# cat requests.gor
1 040f1f92ac106afb7e8d5679 1629183084044026882 0
POST /person/two HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: a8600dd1-28c1-4825-b6a2-68c42659942f
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 0
???
[root@vm-jmeter ~]# cat requests.gor
1 040f1f92ac106afb7e8d5679 1629183084044026882 0
POST /person/two HTTP/1.1
User-Agent: PostmanRuntime/7.28.2
Accept: */*
Cache-Control: no-cache
Postman-Token: a8600dd1-28c1-4825-b6a2-68c42659942f
Host: 172.16.106.149:8082
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 0
我們可以看到主要是兩個請求,
運行以下命令,將流量從檔案回放到服務器二,并放大兩倍:
[root@vm-1 ~]./gor --input-file "requests.gor|200%" --output-http="http://172.16.106.237:8082"
2021/08/17 15:03:58 [PPID 12356 and PID 18187] Version:1.3.0
[DEBUG][elapsed 1.361742ms]: [INPUT-FILE] FileInput: end of file 'requests.gor'
我們查看下服務二日志的處理情況:
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: select next_val as id_val from hibernate_sequence for update
Hibernate: update hibernate_sequence set next_val= ? where next_val=?
Hibernate: insert into person (age, name, id) values (?, ?, ?)
Hibernate: insert into person (age, name, id) values (?, ?, ?)
我們可以看到服務處理了四個請求,?
注意,當然也支持請求流量 10%,20% 等縮小,
?
五、總結
今天簡單介紹 Gor的常見玩法,事實上遠不止上述提到的這些,更多的玩法有時間再寫,
?
?
檔案原始碼:
- https://github.com/zuozewei/blog-example/tree/master/Performance-testing/04-full-link/springboot-demo
?
轉載請註明出處,本文鏈接:https://www.uj5u.com/qita/294924.html
標籤:其他
