搭建storm本地python開發環境

需要使用python進行storm的開發，這里記錄本地開發環境搭建,程序中使用的資源gitee中下載

centos7 普通用戶
python 3.5.1
jdk 1.8.0
storm 1.1.0
lein+streamparse

本地用戶faron, 根目錄/home/faron

這次使用普通用戶安裝

python 3.5.1安裝

這里我之前使用pyenv已經部署好 /home/faron目錄下的pthon環境為3.5.1, 這里可自行搜索

jdk 1.8.0安裝

安裝包下載

在～/.bashrc文間尾部追加java環境變數

# java
export JAVA_HOME=//home/faron/lein/java_storm/jdk1.8.0_161
export JRE_HOME=$JAVA_HOME/jre
# export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

激活環境變數
```
source ~/.bashrc
```
校驗java環境
```
java versoin
```
終端顯示

java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

strom 1.1.0安裝

安裝包下載

在～/.bashrc文間尾部追加storm環境變數

# storm
export STORM_HOME=/home/faron/lein/java_storm/apache-storm-1.1.0
export PATH=$STORM_HOME/bin:$PATH

激活環境變數
```
source ~/.bashrc
```
校驗storm環境
```
storm version
```
終端顯示

Running: //home/faron/lein/java_storm/jdk1.8.0_161/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/home/faron/lein/java_storm/apache-storm-1.1.0 -Dstorm.log.dir=/home/faron/lein/java_storm/apache-storm-1.1.0/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /home/faron/lein/java_storm/apache-storm-1.1.0/lib/storm-core-1.1.0.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/kryo-3.0.3.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/reflectasm-1.10.1.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/asm-5.0.3.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/minlog-1.3.0.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/objenesis-2.1.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/clojure-1.7.0.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/ring-cors-0.1.5.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/disruptor-3.3.2.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/log4j-api-2.8.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/log4j-core-2.8.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/log4j-slf4j-impl-2.8.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/slf4j-api-1.7.21.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/log4j-over-slf4j-1.6.6.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/servlet-api-2.5.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/lib/storm-rename-hack-1.1.0.jar:/home/faron/lein/java_storm/apache-storm-1.1.0/conf org.apache.storm.utils. VersionInfo
Storm 1.1.0
URL https://git-wip-us.apache.org/repos/asf/storm.git -r e40d213de7067f7d3aa4d4992b81890d8ed6ff31
Branch (no branch)
Compiled by ptgoetz on 2017-03-21T17:04Z
From source with checksum 4ddc442e8b804654454ab26fbec6348

lein+streamparse安裝

lein安裝

安裝程序中可能會出現lein腳本無法下載,leiningen-2.9.3-standalone.jar無法下載的情況
```
makdir ~/bin
cd ~/bin
cp ~/lein/lein ./
chmod +x ~/bin/lein
# 運行lein安裝腳本
lein
```
終端顯示(下載速度可能會有點慢)

WARNING: You have $CLASSPATH set, probably by accident.
It is strongly recommended to unset this before proceeding.
Downloading Leiningen to /home/faron/.lein/self-installs/leiningen-2.9.3-standalone.jar now...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 637 100 637 0 0 393 0 0:00:01 0:00:01 --:--:-- 393
27 13.9M 27 3942k 0 0 34711 0 0:07:02 0:01:56 0:05:06 26468

如果leiningen-2.9.3-standalone.jar下載失敗
```
cp ~/lein/java_storm/leiningen-2.9.3-standalone.zip ~/.lein/self-installs/
```
lein環境驗證
```
lein version
# >>>終端顯示
# WARNING: You have $CLASSPATH set, probably by accident.
# It is strongly recommended to unset this before proceeding.
# Leiningen 2.9.3 on Java 1.8.0_161 Java HotSpot(TM) 64-Bit Server VM
```
streamparse安裝
- 安裝
```
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --upgrade pip
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple streamparse
```
- 啟動wordcount
```
mkdir ~/storm_workplace
cd ~/storm_workplace
sparse quickstart wordcount
cd wordcount
sparse run
```
- 例外中心包無法下載
  
  Could not transfer artifact org.apache.storm:storm-core:jar:1.1.0 from/to central (https://repo1.maven.org/maven2/): Connect to repo1.maven.org:443 [repo1.maven.org/151.101.40.209] failed: 拒絕連接 (Connection refused)
  Could not find artifact org.apache.storm:storm-core:jar:1.1.0 in clojars (https://repo.clojars.org/)
  Could not transfer artifact org.clojure:clojure:jar:1.7.0 from/to central (https://repo1.maven.org/maven2/): Connect to repo1.maven.org:443 [repo1.maven.org/151.101.40.209] failed: 拒絕連接 (Connection refused)
  Could not find artifact org.clojure:clojure:jar:1.7.0 in clojars (https://repo.clojars.org/)
  This could be due to a typo in :dependencies, file system permissions, or network issues.
  If you are behind a proxy, try setting the 'http_proxy' environment variable.
```
# 僅限storm1.1.0，因為我的repository里面是storm-core:1.1.0本地環境
tar -zxvf ~/lein/java_storm/repository.tar.gz -C  ~/.m2/
# 再次運行
sparse run
```

轉載請註明出處，本文鏈接：https://www.uj5u.com/houduan/146286.html

標籤：Python

上一篇：關于python反射的getattr,我終于想通了！

下一篇：B站視頻下載批量拼接合成工具

storm本地python開發環境搭建

搭建storm本地python開發環境

本地用戶faron, 根目錄/home/faron

python 3.5.1安裝

jdk 1.8.0安裝

strom 1.1.0安裝

lein+streamparse安裝