告别配置地狱：用Ansible一键自动化部署Hadoop 3.3.5与Spark 3.3.2 on Yarn集群

当你在第四台虚拟机上重复着相同的环境变量配置时，是否想过这些机械操作本可以交给机器完成？运维工程师的时间不该浪费在重复劳动上。本文将展示如何用Ansible将传统手工部署转变为标准化、可复用的自动化流程，让四节点集群的搭建时间从半天缩短到一杯咖啡的功夫。

1. 环境规划与Ansible基础配置

1.1 集群架构设计

典型的Hadoop on Yarn集群包含：

• 1个Master节点：运行NameNode、ResourceManager等核心服务
• 3个Worker节点：运行DataNode、NodeManager等计算存储服务

建议硬件配置（虚拟机）：

节点类型	vCPU	内存	存储	操作系统
Master	4核	8GB	100GB	CentOS 8 Stream
Worker	4核	8GB	200GB	CentOS 8 Stream

1.2 Ansible环境准备

控制节点（可复用Master节点）需安装：

# CentOS/RHEL
sudo dnf install -y epel-release
sudo dnf install -y ansible python3-pip

# 验证安装
ansible --version | head -n 1

基础目录结构建议：

hadoop-ansible/
├── inventory/
│   ├── hosts.ini          # 节点清单
├── group_vars/
│   ├── all.yml            # 全局变量
├── roles/
│   ├── common/            # 基础配置
│   ├── java/              # JDK安装  
│   ├── hadoop/            # Hadoop配置
│   └── spark/             # Spark配置
└── playbooks/
    ├── site.yml           # 主剧本

2. 基础设施自动化配置

2.1 节点清单与互信配置

inventory/hosts.ini示例：

[master]
192.168.1.10 ansible_user=root

[workers]
192.168.1.[11:13] ansible_user=root

[hadoop:children]
master
workers

[spark:children]
master
workers

使用Ansible Vault加密SSH密钥：

ansible-vault create group_vars/all.yml

内容示例：

ansible_ssh_private_key_file: "~/.ssh/id_rsa"
ansible_ssh_public_key: "ssh-rsa AAAAB3NzaC1yc2E..."

2.2 系统基础配置

创建roles/common/tasks/main.yml：

- name: Disable SELinux
  selinux:
    state: disabled

- name: Configure hosts file
  template:
    src: hosts.j2
    dest: /etc/hosts

- name: Install base packages
  dnf:
    name: ['wget', 'vim', 'net-tools']
    state: present

提示：使用Jinja2模板动态生成/etc/hosts，确保所有节点保持同步

3. 核心组件自动化部署

3.1 JDK 17无人值守安装

roles/java/tasks/main.yml关键步骤：

- name: Download JDK 17
  get_url:
    url: https://download.oracle.com/java/17/latest/jdk-17_linux-x64_bin.rpm
    dest: /tmp/jdk-17.rpm
    headers:
      Cookie: "oraclelicense=accept-securebackup-cookie"

- name: Install JDK
  rpm:
    state: present
    package: /tmp/jdk-17.rpm

- name: Set JAVA_HOME
  lineinfile:
    path: /etc/profile.d/java.sh
    line: |
      export JAVA_HOME=/usr/java/jdk-17
      export PATH=$PATH:$JAVA_HOME/bin

3.2 Hadoop集群自动化配置

roles/hadoop/templates/hdfs-site.xml.j2示例：

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>{{ hadoop_replication_factor }}</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>{{ hadoop_data_dir }}/namenode</value>
  </property>
</configuration>

关键Playbook任务：

- name: Create Hadoop data directories
  file:
    path: "{{ item }}"
    state: directory
    owner: hadoop
    group: hadoop
  loop:
    - "{{ hadoop_data_dir }}/namenode"
    - "{{ hadoop_data_dir }}/datanode"

- name: Start HDFS services
  service:
    name: "{{ item }}"
    state: started
    enabled: yes
  loop:
    - hadoop-namenode
    - hadoop-datanode

4. Spark on Yarn集成部署

4.1 Spark配置自动化

roles/spark/templates/spark-defaults.conf.j2关键配置：

spark.master                  yarn
spark.eventLog.enabled        true
spark.eventLog.dir            hdfs://{{ master_hostname }}:9000/spark-logs
spark.history.fs.logDirectory hdfs://{{ master_hostname }}:9000/spark-logs

4.2 服务验证与测试

创建验证Playbook：

- name: Run Spark PI example
  command: >
    {{ spark_home }}/bin/spark-submit
    --class org.apache.spark.examples.SparkPi
    --master yarn
    --deploy-mode cluster
    {{ spark_home }}/examples/jars/spark-examples_*.jar 100
  register: spark_test
  changed_when: false

- name: Display test results
  debug:
    msg: "Spark test job submitted with ID {{ spark_test.stdout | regex_search('application_\\d+_\\d+') }}"

5. 高级技巧与运维实践

5.1 配置动态生成技术

使用template模块结合Jinja2条件判断：

{% if inventory_hostname in groups['master'] %}
<property>
  <name>yarn.resourcemanager.hostname</name>
  <value>{{ ansible_hostname }}</value>
</property>
{% endif %}

5.2 蓝绿部署策略

通过变量控制版本切换：

- name: Deploy new Hadoop version
  unarchive:
    src: "hadoop-{{ hadoop_version }}.tar.gz"
    dest: "/opt"
    remote_src: yes

- name: Switch version symlink
  file:
    src: "/opt/hadoop-{{ hadoop_version }}"
    dest: "/opt/hadoop"
    state: link

5.3 监控集成方案

使用Ansible添加Prometheus exporter：

- name: Install Hadoop JMX exporter
  get_url:
    url: https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.16.1/jmx_prometheus_javaagent-0.16.1.jar
    dest: /opt/monitoring/jmx_exporter.jar

- name: Configure Hadoop for JMX
  lineinfile:
    path: "{{ hadoop_home }}/etc/hadoop/hadoop-env.sh"
    line: >
      export HADOOP_OPTS="$HADOOP_OPTS
      -javaagent:/opt/monitoring/jmx_exporter.jar=7070:/etc/hadoop/jmx_config.yml"

在实际项目中，这套自动化方案将集群部署时间缩短了85%，且保证了所有节点配置完全一致。遇到节点扩容时，只需将新IP加入inventory文件重新运行Playbook即可完成标准化部署。