目录

实验环境:

实验步骤:

一、安装配置storm

1.解压

2.改名:

3.配置环境变量:

4.配置storm.yaml

 5.分发给slave节点

6.启动:

7.web测试

二、Maven安装:

1.解压:(每个人的压缩包名字可能不一样)

2.改名:

3.配置环境变量:

4.Maven检验:

5.配置Maven配置文件

三、运行样例:


实验环境:

Ubuntu 18.04系统3台

Storm 0.9.6(一定要是此版本才有WordCountTopology的例子)

Zookeeper 3.7.1

Hadoop 3.1.3

实验步骤:

一、安装配置storm

1.解压

tar -zxf ./apache-storm-0.9.6.tar.gz -C /usr/local

2.改名:

mv apache-storm-0.9.6 storm

3.配置环境变量:

vim /etc/profile

添加 

export STORM_HOME=/usr/local/storm
export PATH=$PATH:$STORM_HOME/bin

 刷新环境变量:

source /etc/profile

4.配置storm.yaml

cd /usr/local/storm/conf

vim storm.yaml

把storm、master、slave1、slave2和nimubus前面的#去掉,然后做图中修改

或者不修改,不删#,直接添加:

storm.zookeeper.servers:
    - "master"
    - "slave1"
    - "slave2"
nimbus.host: "master"
#配置storm数据存储路径
storm.local.dir: "/tmp/storp"
#配置supervisor:两个slave,对应两个worker进程
supervisor.slots.ports :
    - 6700
    - 6701
#配置节点健康检测
storm.health.check.dir: "healthchecks"
storm.health.check.timeout.ms : 5000
#配置storm UI(为了避免与spark的8080瑞口冲突,改为8089)
ui.port: 8089

 5.分发给slave节点

scp -r /usr/local/storm root@slave1:/usr/local/
scp -r /usr/local/storm root@slave2:/usr/local/

6.启动:

先开启hdfs和

start-dfs.sh
start-yarn.sh

启动zookeeper集群需要在每台服务器上都运行命令:

zkServer.sh start

 查看进程编号:

storm nimbus &

查看进程  

jps

Storm ui &

7.web测试

Master:8089

Storm supervisor & 

jps

二、Maven安装:

1.解压:(每个人的压缩包名字可能不一样)

tar -zxf ./apache-maven-3.3.9-bin.tar.gz -C /usr/local

2.改名:

mv ./apache-maven-3.3.9 maven

3.配置环境变量:

vim /etc/profile

添加:

export PATH=/usr/local/maven/bin:$PATH
export MAVEN_HOME=/usr/local/maven
export PATH=${PATH}:${MAVEN_HOME}/bin

刷新环境变量 :
 

source /etc/profile

4.Maven检验:

mvn -v

5.配置Maven配置文件

cd /usr/local/maven/conf
cp ./maven-settings.xml /usr/local/maven/conf/settings.xml

vim settings.xml

添加:

这是阿里镜像源,可以防止部分错误:

<mirror>
         <id>alimaven</id>
        <mirrorOf>central</mirrorOf>
        <name>aliyun maven</name>
        <url>https://maven.aliyun.com/repository/central</url>
</mirror>

<mirror>
        <id>alimaven</id>
        <name>aliyun maven</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
        <mirrorOf>central</mirrorOf>
 </mirror>

这是配置文件:

        <profile>
			<id>jdk-1.8</id>

			<activation>
				<jdk>1.8</jdk>
			</activation>

			<repositories>
				<repository>
					<id>nexus</id>
					<name>local private nexus</name>
					<url>http://maven.oschina.net/content/groups/public/</url>
					<releases>
						<enabled>true</enabled>
					</releases>
					<snapshots>
						<enabled>false</enabled>
					</snapshots>
				</repository>
				<repository>
				    <id>osc_thirdparty</id>
				    <url>http://maven.oschina.net/content/repositories/thirdparty/</url>
				</repository>
				<repository>
				    <id>clojars</id>
				    <url>http://clojars.org/repo/</url>
				</repository>
			</repositories>
			<pluginRepositories>
				<pluginRepository>
					<id>nexus</id>
					<name>local private nexus</name>
					<url>http://maven.oschina.net/content/groups/public/</url>
					<releases>
						<enabled>true</enabled>
					</releases>
					<snapshots>
						<enabled>false</enabled>
					</snapshots>
				</pluginRepository>
			</pluginRepositories>
		</profile>

三、运行样例:

mvn compile exec:java -Dstorm.topology=storm.starter.WordCountTopology

或者使用有简洁的结果的命令:

mvn compile exec:java -Dstorm.topology=storm.starter.WordCountTopology | grep 'Thread-[0-9]*-count'

打包代码:

mvn package

 再运行:

/usr/local/storm/bin/storm jar ./target/storm-starter-0.9.6.jar storm.starter.WordCountTopology | grep 'Thread-[0-9]*-count'

/usr/local/storm/bin/storm jar ./target/storm-starter-0.9.6.jar storm.starter.WordCountTopology | grep 'Thread-[0-9]*-count'

如果出现报错: 

将镜像源改为阿里:

然后将样例解压到.m2文件,查看

tar -zxf ./storm-starter-0.9.6-repository.tar.gz -C ~/.m2/

配置完成 

Logo

旨在为数千万中国开发者提供一个无缝且高效的云端环境,以支持学习、使用和贡献开源项目。

更多推荐