一、簡介
MogileFS是一個開源的分布式檔案存盤系統，由LiveJournal旗下的Danga Interactive公司開發；它主要由三部分組成，第一部分是server端，server端主要包括mogilefsd和mogstored兩個應用程式，mogilefsd實作的是tracker，它通過資料庫來保存元資料資訊，包括站點domain、class、host等；mogstored是存盤節點(store node)，它其實是個WebDAV服務，默認監聽在7500埠，接受客戶端的檔案存盤請求，在MogileFS安裝完后，要運行mogadm工具將所有的store node注冊到mogilefsd的資料庫里，mogilefsd會對這些節點進行管理和監控，第二部是mogilefs的utils,這部分主要是一些工具，比如mogadm、mogupload、mogfileinfo 等等；第三部分是客戶端API，mogilefs支持眾多編程語言的客戶端API介面，使用對應的客戶端API介面，我們可以撰寫mogilefs的客戶端，從而實作來管理mogilefs上的檔案；
二、Mogilefs架構圖
提示：tracker的主要作用是代理前端用戶的請求到資料庫查詢相關檔案的元資訊，同時它還要監控注冊到tracker的storagenode的健康狀態，一旦后端storage掛掉，它會立刻把對應的storage節點上的device從集群的中洗掉，上線時在加入到集群；客戶端通過向tracker發送請求檔案的資訊，tracker通過去資料庫中找到對應的檔案所在storage節點的ip地址、埠、設備id以及檔案fid，然后通過http鏈接的形式從tracker回傳給客戶端，客戶端拿到這tracker的回傳的檔案地址，然后在去對應的storage請求檔案，從而在客戶端就訪問到對應的檔案內容資訊；客戶端和tracker，storage之間通信都是http協議，從本質上講storage就是一個web服務器；所以對于客戶端我們可以使用web服務器的反向代理，也可以是呼叫mogilefsAPI寫的客戶端程式；
三、Mogilefs安裝部署
環境說明
| 名稱 | 角色 | ip地址 | 埠 |
| node01 | tracker+mysql+storage | 192.168.0.41 | 7001/7500/7501/3306 |
| node02 | tracker+storage | 192.168.0.42 | 7001/7500/7501 |
| node03 | storage | 192.168.0.43 | 7500/7501 |
各server時間同步
提示：如果有必要，可以配置各節點用hosts檔案決議；
安裝環境依賴包
[root@node01 ~]# yum install perl-Net-Netmask perl-IO-stringy perl-Sys-Syslog perl-IO-AIO -y Loaded plugins: fastestmirror base | 3.6 kB 00:00:00 epel | 4.7 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Resolving Dependencies --> Running transaction check ---> Package perl-IO-AIO.x86_64 0:4.20-1.el7 will be installed --> Processing Dependency: perl(common::sense) for package: perl-IO-AIO-4.20-1.el7.x86_64 ---> Package perl-IO-stringy.noarch 0:2.110-22.el7 will be installed ---> Package perl-Net-Netmask.noarch 0:1.9015-13.el7 will be installed ---> Package perl-Sys-Syslog.x86_64 0:0.33-3.el7 will be installed --> Running transaction check ---> Package perl-common-sense.noarch 0:3.6-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ======================================================================================================================================= Package Arch Version Repository Size ======================================================================================================================================= Installing: perl-IO-AIO x86_64 4.20-1.el7 epel 117 k perl-IO-stringy noarch 2.110-22.el7 base 71 k perl-Net-Netmask noarch 1.9015-13.el7 epel 25 k perl-Sys-Syslog x86_64 0.33-3.el7 base 42 k Installing for dependencies: perl-common-sense noarch 3.6-4.el7 epel 28 k Transaction Summary ======================================================================================================================================= Install 4 Packages (+1 Dependent package) Total download size: 283 k Installed size: 636 k Downloading packages: (1/5): perl-Sys-Syslog-0.33-3.el7.x86_64.rpm | 42 kB 00:00:00 (2/5): perl-Net-Netmask-1.9015-13.el7.noarch.rpm | 25 kB 00:00:00 (3/5): perl-IO-AIO-4.20-1.el7.x86_64.rpm | 117 kB 00:00:00 (4/5): perl-common-sense-3.6-4.el7.noarch.rpm | 28 kB 00:00:00 (5/5): perl-IO-stringy-2.110-22.el7.noarch.rpm | 71 kB 00:00:00 --------------------------------------------------------------------------------------------------------------------------------------- Total 642 kB/s | 283 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : perl-common-sense-3.6-4.el7.noarch 1/5 Installing : perl-IO-AIO-4.20-1.el7.x86_64 2/5 Installing : perl-Net-Netmask-1.9015-13.el7.noarch 3/5 Installing : perl-Sys-Syslog-0.33-3.el7.x86_64 4/5 Installing : perl-IO-stringy-2.110-22.el7.noarch 5/5 Verifying : perl-IO-AIO-4.20-1.el7.x86_64 1/5 Verifying : perl-common-sense-3.6-4.el7.noarch 2/5 Verifying : perl-IO-stringy-2.110-22.el7.noarch 3/5 Verifying : perl-Sys-Syslog-0.33-3.el7.x86_64 4/5 Verifying : perl-Net-Netmask-1.9015-13.el7.noarch 5/5 Installed: perl-IO-AIO.x86_64 0:4.20-1.el7 perl-IO-stringy.noarch 0:2.110-22.el7 perl-Net-Netmask.noarch 0:1.9015-13.el7 perl-Sys-Syslog.x86_64 0:0.33-3.el7 Dependency Installed: perl-common-sense.noarch 0:3.6-4.el7 Complete! [root@node01 ~]#
提示：三個節點都需要安裝環境依賴的包；
上傳rpm包到服務器并安裝
[root@node01 ~]# rz rz waiting to receive. zmodem trl+C ? 100% 1 KB 1 KB/s 00:00:01 0 Errors.el7.centos.noarch.rpm... 100% 173 KB 173 KB/s 00:00:01 0 Errorsentos.noarch.rpm... 100% 26 KB 26 KB/s 00:00:01 0 Errorsentos.noarch.rpm... 100% 75 KB 75 KB/s 00:00:01 0 Errorsh.rpm... 100% 27 KB 27 KB/s 00:00:01 0 Errors.rpm... 100% 29 KB 29 KB/s 00:00:01 0 Errors.noarch.rpm... 100% 262 KB 262 KB/s 00:00:01 0 Errors [root@node01 ~]# ls MogileFS-Server-2.46-2.el7.centos.noarch.rpm perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm perl-Perlbal-1.78-1.el6.noarch.rpm MogileFS-Utils-2.19-1.el7.centos.noarch.rpm [root@node01 ~]# yum install ./*.rpm -y Loaded plugins: fastestmirror Examining ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-2.46-2.el7.centos.noarch Marking ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm to be installed Examining ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch Marking ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm to be installed Examining ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogstored-2.46-2.el7.centos.noarch Marking ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm to be installed Examining ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm: MogileFS-Utils-2.19-1.el7.centos.noarch Marking ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm to be installed Examining ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm: perl-Danga-Socket-1.61-1.el6.rf.noarch Marking ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm to be installed Examining ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm: perl-MogileFS-Client-1.14-1.el7.centos.noarch Marking ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm to be installed Examining ./perl-Perlbal-1.78-1.el6.noarch.rpm: perl-Perlbal-1.78-1.el6.noarch Marking ./perl-Perlbal-1.78-1.el6.noarch.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package MogileFS-Server.noarch 0:2.46-2.el7.centos will be installed ---> Package MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos will be installed --> Processing Dependency: perl(DBI) >= 1.44 for package: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com --> Processing Dependency: perl(DBD::mysql) >= 3 for package: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch ---> Package MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos will be installed ……省略部分內容…… Installed: MogileFS-Server.noarch 0:2.46-2.el7.centos MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos MogileFS-Utils.noarch 0:2.19-1.el7.centos perl-Danga-Socket.noarch 0:1.61-1.el6.rf perl-MogileFS-Client.noarch 0:1.14-1.el7.centos perl-Perlbal.noarch 0:1.78-1.el6 Dependency Installed: mailcap.noarch 0:2.1.41-2.el7 perl-BSD-Resource.x86_64 0:1.29.07-1.el7 perl-Business-ISBN.noarch 0:2.06-2.el7 perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7 perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7 perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7 perl-DBD-MySQL.x86_64 0:4.023-6.el7 perl-DBI.x86_64 0:1.627-4.el7 perl-Data-Dumper.x86_64 0:2.145-3.el7 perl-Digest.noarch 0:1.17-245.el7 perl-Digest-MD5.x86_64 0:2.52-3.el7 perl-Encode-Locale.noarch 0:1.03-5.el7 perl-File-Listing.noarch 0:6.04-7.el7 perl-HTML-Parser.x86_64 0:3.71-4.el7 perl-HTML-Tagset.noarch 0:3.20-15.el7 perl-HTTP-Cookies.noarch 0:6.01-5.el7 perl-HTTP-Daemon.noarch 0:6.01-8.el7 perl-HTTP-Date.noarch 0:6.02-8.el7 perl-HTTP-Message.noarch 0:6.06-6.el7 perl-HTTP-Negotiate.noarch 0:6.01-5.el7 perl-IO-Compress.noarch 0:2.061-2.el7 perl-IO-HTML.noarch 0:1.00-2.el7 perl-IO-Socket-IP.noarch 0:0.21-5.el7 perl-IO-Socket-SSL.noarch 0:1.94-7.el7 perl-LWP-MediaTypes.noarch 0:6.02-2.el7 perl-Mozilla-CA.noarch 0:20130114-5.el7 perl-Net-Daemon.noarch 0:0.48-5.el7 perl-Net-HTTP.noarch 0:6.06-2.el7 perl-Net-LibIDN.x86_64 0:0.12-15.el7 perl-Net-SSLeay.x86_64 0:1.55-6.el7 perl-PlRPC.noarch 0:0.2020-14.el7 perl-Sys-Syscall.noarch 0:0.25-10.el7 perl-TimeDate.noarch 1:2.30-2.el7 perl-URI.noarch 0:1.60-9.el7 perl-WWW-RobotRules.noarch 0:6.02-5.el7 perl-libwww-perl.noarch 0:6.05-2.el7 Complete! [root@node01 ~]#
提示：安裝環境依賴和以上rpm包，首先要配置好epel源；因為有些依賴包，它來自epel源；相關包我已經上傳至我的網盤http://www.qiuhom.com/d/383f34f57cf847beb296/；
安裝mariadb-server
配置yum源
[root@node01 ~]# cat /etc/yum.repos.d/mairadb.repo [mariadb] name=mariadb repo baseurl=https://mirrors.tuna.tsinghua.edu.cn/mariadb//mariadb-10.1.46/yum/centos/7/x86_64/ gpgcheck=0 [root@node01 ~]#
安裝mariadb-server
[root@node01 ~]# yum install MariaDB-server -y Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Resolving Dependencies --> Running transaction check ---> Package MariaDB-server.x86_64 0:10.1.46-1.el7.centos will be installed --> Processing Dependency: MariaDB-client for package: MariaDB-server-10.1.46-1.el7.centos.x86_64 --> Processing Dependency: rsync for package: MariaDB-server-10.1.46-1.el7.centos.x86_64 --> Processing Dependency: MariaDB-common for package: MariaDB-server-10.1.46-1.el7.centos.x86_64 --> Processing Dependency: galera for package: MariaDB-server-10.1.46-1.el7.centos.x86_64 --> Processing Dependency: libjemalloc.so.1()(64bit) for package: MariaDB-server-10.1.46-1.el7.centos.x86_64 --> Running transaction check ……省略部分內容…… Installed: MariaDB-server.x86_64 0:10.1.46-1.el7.centos MariaDB-shared.x86_64 0:10.1.46-1.el7.centos Dependency Installed: MariaDB-client.x86_64 0:10.1.46-1.el7.centos MariaDB-common.x86_64 0:10.1.46-1.el7.centos boost-program-options.x86_64 0:1.53.0-28.el7 galera.x86_64 0:25.3.29-1.rhel7.el7.centos jemalloc.x86_64 0:3.6.0-1.el7 rsync.x86_64 0:3.1.2-10.el7 Replaced: mariadb-libs.x86_64 1:5.5.56-2.el7 Complete! [root@node01 ~]#
提示：mariadb-server包名是大寫的MariaDB-server；
配置mariadb忽略名稱決議
設定mariadb開機自動啟動，并啟動mariadb
添加用于mogilefs訪問資料庫的賬號
初始化資料庫
[root@node01 ~]# mogdbsetup --help
Usage: mogdbsetup [opts]
Options:
Default Description
============ ===========================================
--verbose <off> Be verbose about what's happening.
--dbhost= localhost hostname or IP to database server.
--dbport= dbd default port number to database server.
--dbname= mogilefs database name to create/upgrade.
--dbrootuser= root Database administrator username. Only needed
for initial setup, not subsequent upgrades.
--dbrootpass= <blank> Database administrator password. Only needed
for initial setup, not subsequent upgrades.
--dbuser= mogile Regular database user to create and/or use
for MogileFS database. This is what the
mogilefsd trackers connect as.
--dbpass= <blank> You should change this, especially if your
database servers are accessible to other users
on the network. But they shouldn't be
if you're running MogileFS, because MogileFS
assumes your network is closed.
--type= MySQL Which MogileFS::Store implementation to use.
Available: MySQL, Postgres
--yes Run without questions.
[root@node01 ~]# mogdbsetup --dbhost=192.168.0.41 --dbport=3306 --dbname=mogilefs --dbrootpass=admin --dbuser=mogilefs --dbpass=mogilefs
This will attempt to setup or upgrade your MogileFS database.
It won't destroy existing data.
Run with --help for more information. Run with --yes to shut up these prompts.
Continue? [N/y]: y
Create/Upgrade database name 'mogilefs'? [Y/n]: y
Grant all privileges to user 'mogilefs', connecting from anywhere, to the mogilefs database 'mogilefs'? [Y/n]: n
Stopped.
[root@node01 ~]# mogdbsetup --dbhost=192.168.0.41 --dbport=3306 --dbname=mogilefs --dbrootpass=admin --dbuser=mogilefs --dbpass=mogilefs
This will attempt to setup or upgrade your MogileFS database.
It won't destroy existing data.
Run with --help for more information. Run with --yes to shut up these prompts.
Continue? [N/y]: y
[root@node01 ~]#
提示：如果不想讓mogdbsetup在資料庫中創建一個任何地址都可訪問的mogilefs的用戶，在最后的選項中使用n即可，然后在重復執行上述命令即可；如果不想互動式輸入y，也可以直接在命令后面用--yes；
驗證：在資料庫中查看對應的庫是否都已創建，對應庫中是否都有表？
[root@node01 ~]# mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 7 Server version: 10.1.46-MariaDB MariaDB Server Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mogilefs | | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.01 sec) MariaDB [(none)]> use mogilefs Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed MariaDB [mogilefs]> show tables; +----------------------+ | Tables_in_mogilefs | +----------------------+ | checksum | | class | | device | | domain | | file | | file_on | | file_on_corrupt | | file_to_delete | | file_to_delete2 | | file_to_delete_later | | file_to_queue | | file_to_replicate | | fsck_log | | host | | server_settings | | tempfile | | unreachable_fids | +----------------------+ 17 rows in set (0.00 sec) MariaDB [mogilefs]>
到此資料庫就準備好了
配置mogilefsd連接資料庫
提示：我們只需要修改紅框中的內容即可；db_dsn用于指定連接資料庫的驅動，以及資料庫庫名和主機地址；db_user用于指定連接資料庫所使用的用戶，db_pass用于指定連接資料庫的密碼；listen用于指定mogilefsd監聽的ip地址和埠；后面的配置就是和啟動的行程數量的配置，這個可以根據業務的規模來設定；
啟動mogilefsd
提示：在centos7上啟動mogilefsd時，會報錯，我們可以不用管這個錯誤，實際上mogilefsd是啟動成功的，只要看到對應的埠和行程起來，表示mogilefsd啟動就沒有問題；
配置node01上的mogstored
[root@node01 ~]# cat /etc/mogilefs/mogstored.conf maxconns = 10000 httplisten = 0.0.0.0:7500 mgmtlisten = 0.0.0.0:7501 docroot = /data/mogilefs [root@node01 ~]# mkdir /data/mogilefs/ -pv mkdir: created directory ‘/data’ mkdir: created directory ‘/data/mogilefs/’ [root@node01 ~]# chown -R mogilefs.mogilefs /data/mogilefs/ [root@node01 ~]# ll -d /data/mogilefs/ drwxr-xr-x 2 mogilefs mogilefs 6 Sep 16 13:37 /data/mogilefs/ [root@node01 ~]#
啟動mogstored
提示：在centos7上啟動mogstored也會同啟動mogilefsd一樣報錯，這個主要是因為systemd的緣故，在centos6不會報錯；到此，node01上的mariadb、mogilefsd和mogstored都已經安裝配置好了；
復制安裝包到node02和node03
在node02上安裝
[root@node02 ~]# yum install perl-Net-Netmask perl-IO-stringy perl-Sys-Syslog perl-IO-AIO ./*.rpm -y Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Package perl-Net-Netmask-1.9015-13.el7.noarch already installed and latest version Package perl-IO-stringy-2.110-22.el7.noarch already installed and latest version Package perl-Sys-Syslog-0.33-3.el7.x86_64 already installed and latest version Package perl-IO-AIO-4.20-1.el7.x86_64 already installed and latest version Examining ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-2.46-2.el7.centos.noarch Marking ./MogileFS-Server-2.46-2.el7.centos.noarch.rpm to be installed Examining ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch Marking ./MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm to be installed Examining ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogstored-2.46-2.el7.centos.noarch Marking ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm to be installed Examining ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm: MogileFS-Utils-2.19-1.el7.centos.noarch Marking ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm to be installed Examining ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm: perl-Danga-Socket-1.61-1.el6.rf.noarch Marking ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm to be installed Examining ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm: perl-MogileFS-Client-1.14-1.el7.centos.noarch Marking ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm to be installed Examining ./perl-Perlbal-1.78-1.el6.noarch.rpm: perl-Perlbal-1.78-1.el6.noarch Marking ./perl-Perlbal-1.78-1.el6.noarch.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package MogileFS-Server.noarch 0:2.46-2.el7.centos will be installed ---> Package MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos will be installed --> Processing Dependency: perl(DBI) >= 1.44 for package: MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch ……省略部分內容…… Installed: MogileFS-Server.noarch 0:2.46-2.el7.centos MogileFS-Server-mogilefsd.noarch 0:2.46-2.el7.centos MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos MogileFS-Utils.noarch 0:2.19-1.el7.centos perl-Danga-Socket.noarch 0:1.61-1.el6.rf perl-MogileFS-Client.noarch 0:1.14-1.el7.centos perl-Perlbal.noarch 0:1.78-1.el6 Dependency Installed: mailcap.noarch 0:2.1.41-2.el7 perl-BSD-Resource.x86_64 0:1.29.07-1.el7 perl-Business-ISBN.noarch 0:2.06-2.el7 perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7 perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7 perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7 perl-DBD-MySQL.x86_64 0:4.023-6.el7 perl-DBI.x86_64 0:1.627-4.el7 perl-Data-Dumper.x86_64 0:2.145-3.el7 perl-Digest.noarch 0:1.17-245.el7 perl-Digest-MD5.x86_64 0:2.52-3.el7 perl-Encode-Locale.noarch 0:1.03-5.el7 perl-File-Listing.noarch 0:6.04-7.el7 perl-HTML-Parser.x86_64 0:3.71-4.el7 perl-HTML-Tagset.noarch 0:3.20-15.el7 perl-HTTP-Cookies.noarch 0:6.01-5.el7 perl-HTTP-Daemon.noarch 0:6.01-8.el7 perl-HTTP-Date.noarch 0:6.02-8.el7 perl-HTTP-Message.noarch 0:6.06-6.el7 perl-HTTP-Negotiate.noarch 0:6.01-5.el7 perl-IO-Compress.noarch 0:2.061-2.el7 perl-IO-HTML.noarch 0:1.00-2.el7 perl-IO-Socket-IP.noarch 0:0.21-5.el7 perl-IO-Socket-SSL.noarch 0:1.94-7.el7 perl-LWP-MediaTypes.noarch 0:6.02-2.el7 perl-Mozilla-CA.noarch 0:20130114-5.el7 perl-Net-Daemon.noarch 0:0.48-5.el7 perl-Net-HTTP.noarch 0:6.06-2.el7 perl-Net-LibIDN.x86_64 0:0.12-15.el7 perl-Net-SSLeay.x86_64 0:1.55-6.el7 perl-PlRPC.noarch 0:0.2020-14.el7 perl-Sys-Syscall.noarch 0:0.25-10.el7 perl-TimeDate.noarch 1:2.30-2.el7 perl-URI.noarch 0:1.60-9.el7 perl-WWW-RobotRules.noarch 0:6.02-5.el7 perl-libwww-perl.noarch 0:6.05-2.el7 Complete! [root@node02 ~]#
把node01上的mogilefsd的組態檔復制到node02上，然后啟動mogilefsd
把node01上的mogstored組態檔復制到node02上，然后在node02上創建對應的目錄并修改屬主和屬組為mogilefs，然后啟動mogstored
到此node02上的mogilefsd和mogstored就安裝配置啟動好了；
在node03上安裝配置mogstored
[root@node03 ~]# ls MogileFS-Server-2.46-2.el7.centos.noarch.rpm perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm MogileFS-Server-mogilefsd-2.46-2.el7.centos.noarch.rpm perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm perl-Perlbal-1.78-1.el6.noarch.rpm MogileFS-Utils-2.19-1.el7.centos.noarch.rpm [root@node03 ~]# yum install ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm ./perl-*.rpm perl-Net-Netmask perl-IO-stringy perl-Sys-Syslog perl-IO-AIO -y Loaded plugins: fastestmirror Examining ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm: MogileFS-Server-mogstored-2.46-2.el7.centos.noarch Marking ./MogileFS-Server-mogstored-2.46-2.el7.centos.noarch.rpm to be installed Examining ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm: MogileFS-Utils-2.19-1.el7.centos.noarch Marking ./MogileFS-Utils-2.19-1.el7.centos.noarch.rpm to be installed Examining ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm: perl-Danga-Socket-1.61-1.el6.rf.noarch Marking ./perl-Danga-Socket-1.61-1.el6.rf.noarch.rpm to be installed Examining ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm: perl-MogileFS-Client-1.14-1.el7.centos.noarch Marking ./perl-MogileFS-Client-1.14-1.el7.centos.noarch.rpm to be installed Examining ./perl-Perlbal-1.78-1.el6.noarch.rpm: perl-Perlbal-1.78-1.el6.noarch Marking ./perl-Perlbal-1.78-1.el6.noarch.rpm to be installed Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Package perl-Net-Netmask-1.9015-13.el7.noarch already installed and latest version Package perl-IO-stringy-2.110-22.el7.noarch already installed and latest version Package perl-Sys-Syslog-0.33-3.el7.x86_64 already installed and latest version Package perl-IO-AIO-4.20-1.el7.x86_64 already installed and latest version Resolving Dependencies --> Running transaction check ---> Package MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos will be installed ---> Package MogileFS-Utils.noarch 0:2.19-1.el7.centos will be installed --> Processing Dependency: perl(Compress::Zlib) for package: MogileFS-Utils-2.19-1.el7.centos.noarch ……省略部分內容…… Installed: MogileFS-Server-mogstored.noarch 0:2.46-2.el7.centos MogileFS-Utils.noarch 0:2.19-1.el7.centos perl-Danga-Socket.noarch 0:1.61-1.el6.rf perl-MogileFS-Client.noarch 0:1.14-1.el7.centos perl-Perlbal.noarch 0:1.78-1.el6 Dependency Installed: mailcap.noarch 0:2.1.41-2.el7 perl-BSD-Resource.x86_64 0:1.29.07-1.el7 perl-Business-ISBN.noarch 0:2.06-2.el7 perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7 perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7 perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7 perl-Data-Dumper.x86_64 0:2.145-3.el7 perl-Digest.noarch 0:1.17-245.el7 perl-Digest-MD5.x86_64 0:2.52-3.el7 perl-Encode-Locale.noarch 0:1.03-5.el7 perl-File-Listing.noarch 0:6.04-7.el7 perl-HTML-Parser.x86_64 0:3.71-4.el7 perl-HTML-Tagset.noarch 0:3.20-15.el7 perl-HTTP-Cookies.noarch 0:6.01-5.el7 perl-HTTP-Daemon.noarch 0:6.01-8.el7 perl-HTTP-Date.noarch 0:6.02-8.el7 perl-HTTP-Message.noarch 0:6.06-6.el7 perl-HTTP-Negotiate.noarch 0:6.01-5.el7 perl-IO-Compress.noarch 0:2.061-2.el7 perl-IO-HTML.noarch 0:1.00-2.el7 perl-IO-Socket-IP.noarch 0:0.21-5.el7 perl-IO-Socket-SSL.noarch 0:1.94-7.el7 perl-LWP-MediaTypes.noarch 0:6.02-2.el7 perl-Mozilla-CA.noarch 0:20130114-5.el7 perl-Net-HTTP.noarch 0:6.06-2.el7 perl-Net-LibIDN.x86_64 0:0.12-15.el7 perl-Net-SSLeay.x86_64 0:1.55-6.el7 perl-Sys-Syscall.noarch 0:0.25-10.el7 perl-TimeDate.noarch 1:2.30-2.el7 perl-URI.noarch 0:1.60-9.el7 perl-WWW-RobotRules.noarch 0:6.02-5.el7 perl-libwww-perl.noarch 0:6.05-2.el7 Complete! [root@node03 ~]#
復制node01上的mogstored組態檔到node03上，然后在node03上創建對應的目錄并修改屬主和屬組為mogilefs，然后啟動mogstored
提示：在node03上沒有安裝mogilefs-server這個包，所以在對應系統上沒有mogilefs這個賬號，我們需要手動創建這個賬號；
啟動mogstored
到此三個節點的環境配置，以及服務都已經正常啟動了；
四、使用mogadm管理mogilefs
把三個storage節點注冊到tracker上
提示：如果集群各節點沒有做hosts主機名決議,那么對應的的節點就需要寫清楚ip地址即可，如果埠不是7500，我們還需要用--port去指定埠；
檢查集群
提示：在tracker節點監聽在本機所有地址的7001，執行上述命令不需要加--trackers選項來指明tracker是誰，它默認會去找本機的7001；否則執行上述命令需要加上--trackers來指定tracker的地址和埠；
在非tracker節點執行檢查集群的命令
創建devices
分別在3個mogstored節點的docroot說指定的目錄下創建以dev+數字的目錄，并將屬主和屬組修改成mogilefs
[root@node03 ~]# mkdir /data/mogilefs/dev3 [root@node03 ~]# chown -R mogilefs.mogilefs /data/mogilefs/dev3 [root@node03 ~]# ll -d /data/mogilefs/dev3 drwxr-xr-x 3 mogilefs mogilefs 37 Sep 16 21:02 /data/mogilefs/dev3 [root@node03 ~]#
添加device
查看device
提示：如果在指定節點的docroot所指定的目錄下沒有創建dev+設備編號的目錄，上面是看不到對應設備的空間大小的；
創建domain
創建class
好了，到此我們就可以上傳檔案了
上傳檔案，把/etc/passwd上傳到files域中，使用html class
提示：mogupload這個工具是mogilefs的一個上傳檔案的工具，其中--domain用于指定域，就是把檔案放到那個名稱空間中；--class用于指定使用那個class策略來同步檔案，默認不指定使用default；--key用于指定客戶端訪問檔案的路徑；--file用于指定要上傳的檔案路徑；
查看上傳檔案的資訊
提示：查看上傳檔案的資訊需要用mogfileinfo這個工具，我們只需要指定trancker，domain以及檔案的key就可以看到，該檔案的class，devcount,domain,fid,key,length,以及訪問http地址；這里需要說明一下，我們剛才在創建files域中的html這個class 我們指定了mindevcount=3，表示只要使用這個class上傳檔案，檔案至少有3個副本；而我們現在看到的只有一個，這是因為在centos7上，Sys-syscall這個包的版本過高，導致mogilefs例外；
降級Sys-Syscall，解決centos7上安裝mogilefs 后端mogstored 同步問題
安裝perl編譯環境
[root@node01 ~]# yum -y install make gcc unzip perl-DBD-MySQL perl perl-CPAN perl-YAML perl-Time-HiRes Loaded plugins: fastestmirror base | 3.6 kB 00:00:00 epel | 4.7 kB 00:00:00 extras | 2.9 kB 00:00:00 mariadb | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 (1/2): epel/x86_64/updateinfo | 1.0 MB 00:00:00 (2/2): epel/x86_64/primary_db | 6.9 MB 00:00:00 Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Package gcc-4.8.5-39.el7.x86_64 already installed and latest version Package unzip-6.0-21.el7.x86_64 already installed and latest version Package perl-DBD-MySQL-4.023-6.el7.x86_64 already installed and latest version Package 4:perl-5.16.3-295.el7.x86_64 already installed and latest version Package 4:perl-Time-HiRes-1.9725-3.el7.x86_64 already installed and latest version Resolving Dependencies --> Running transaction check ---> Package make.x86_64 1:3.82-23.el7 will be updated ---> Package make.x86_64 1:3.82-24.el7 will be an update ---> Package perl-CPAN.noarch 0:1.9800-295.el7 will be installed --> Processing Dependency: perl(local::lib) for package: perl-CPAN-1.9800-295.el7.noarch ……省略部分內容…… Installed: perl-CPAN.noarch 0:1.9800-295.el7 perl-YAML.noarch 0:0.84-5.el7 Dependency Installed: gdbm-devel.x86_64 0:1.10-8.el7 libdb-devel.x86_64 0:5.3.21-25.el7 perl-Digest-SHA.x86_64 1:5.85-4.el7 perl-ExtUtils-Install.noarch 0:1.58-295.el7 perl-ExtUtils-MakeMaker.noarch 0:6.68-3.el7 perl-ExtUtils-Manifest.noarch 0:1.61-244.el7 perl-ExtUtils-ParseXS.noarch 1:3.18-3.el7 perl-Test-Harness.noarch 0:3.28-3.el7 perl-devel.x86_64 4:5.16.3-295.el7 perl-local-lib.noarch 0:1.008010-4.el7 pyparsing.noarch 0:1.5.6-9.el7 systemtap-sdt-devel.x86_64 0:4.0-11.el7 Updated: make.x86_64 1:3.82-24.el7 Dependency Updated: libdb.x86_64 0:5.3.21-25.el7 libdb-utils.x86_64 0:5.3.21-25.el7 Complete! [root@node01 ~]#
提示：在mogstored節點上都要安裝好perl環境依賴包；然后按照下面的步驟重新編譯安裝Sys-Syscall；然后重啟mogstored即可生效；
下載低版本的Sys-Syscall原始碼包，并解壓，進入到原始碼目錄
編譯安裝Sys-Syscall
驗證
重啟mogstored，mogilefsd
提示：所有storage節點都有重新編譯安裝Sys-Syscall，然后重啟mogstored；對應的tracker節點也需要重啟mogilefsd；
上傳檔案看看對應的副本數量是否還是只有一個？
提示：可以看到對應的devcount和我們之前指定的class上的mindevcount就一樣了；
驗證：訪問指定上述任一一個地址，看看訪問到的檔案內容是否是我們上傳到檔案內容？
提示：可以看到我們上傳到檔案內容在tracker給我們回傳的地址上能夠訪問到，并且里面的內容也是一樣的；
驗證：停掉node02上的mogstored，看看對應的tracker是否能夠及時發現對應的storage不在線了呢？
提示：可以看到當node02上的mogstored掛掉以后，在檢查集群，tracker就能及時的發現對應的設備不在線了，這個有點類似nginx對后端web服務器做健康狀態監測的感覺；也就說tracker在監視著注冊在tracker上的storage節點；
到此，mogilefs在centos7上的安裝部署，以及使用mogadm工具管理集群就完成了，對于mogilefs通常我們上傳檔案是通過業務程式呼叫api實作，mogupload只是一個測驗工具；對于查看檔案，通常用戶也不會使用mogfieinfo回傳的鏈接去查看檔案，通常會使用一個web反向代理（nginx）去代理用戶的訪問；比如用戶要查看/images/test.jpg，而在tracker回傳的對應資源的鏈接中，一定不是/images/test.jpg，這個時候我們就需要用到web反向代理，讓其反向代理能夠幫助我們去tracker取到對應uri的資源鏈接，從而幫助我們去storage節點獲取到對應的檔案，然后再回傳給用戶；最后還是建議，如果用到mogilefs，最好跑在centos6上，centos7系統環境較高，有些依賴可能成問題；在centos6上，我測驗過，完全沒有上述的問題；我們只需要安裝perl-IO-AIO這一個依賴包就可以解決mogstored啟動問題；centos6上需要用到的rpm包可以去我的網盤下載http://www.qiuhom.com/d/47e8f7ac56a445f1bfc6/；
轉載請註明出處，本文鏈接：https://www.uj5u.com/caozuo/60918.html
標籤：Linux
上一篇：Linux下用戶的創建與洗掉