hadoop入门:虚拟机环境搭建

2021/09/29 note 共 3112 字,约 9 分钟

0.主要参考文章

第一篇文章是Vagrant的详细用法介绍,第二篇文章是Vagrant搭建4个虚拟机节点的具体操作。

1.下载

下载CentOS镜像,我选择的是CentOS-7-x86_64-Vagrant-2004_01.VirtualBox.box
下载Vagrant,Vagant 网站
下载VirtualBox,VirtualBox 主页

2.创建虚拟机

  1. vagrant添加镜像
    ```cmd
    // 查看vagrant版本号
    C:\Users\itlgl>vagrant -v
    Vagrant 2.2.18

// 查看box列表
C:\Users\itlgl>vagrant box list
There are no installed boxes! Use vagrant box add to add some.

// 添加本地box
C:\Users\itlgl>vagrant box add C:\itlgl\hadoop\安装包\CentOS-7-x86_64-Vagrant-2004_01.VirtualBox.box –name centos-7
==> box: Box file was not detected as metadata. Adding it directly…
==> box: Adding box ‘centos-7’ (v0) for provider:
box: Unpacking necessary files from: file:///C:/itlgl/hadoop/%B0%B2%D7%B0%B0%FC/CentOS-7-x86_64-Vagrant-2004_01.VirtualBox.box
box:
==> box: Successfully added box ‘centos-7’ (v0) for ‘virtualbox’!

C:\Users\itlgl>vagrant box list
centos-7 (virtualbox, 0)

  
2. vagrant安装vagrant-hostmanager插件  
```cmd  
C:\itlgl\hadoop\vm1>vagrant plugin list  
No plugins installed.  
  
C:\itlgl\hadoop\vm1>vagrant plugin install vagrant-hostmanager  
Installing the 'vagrant-hostmanager' plugin. This can take a few minutes...  
Fetching vagrant-hostmanager-1.8.9.gem  
Installed the plugin 'vagrant-hostmanager (1.8.9)'!  
  1. 新建vagrant配置文件
    本地创建目录C:\itlgl\hadoop\vm1,用于存放vagrant新的虚拟机配置
    vm1文件夹内创建init.shVagrantfile

Vagrantfile文件内容:

Vagrant.configure("2") do |config|  
    config.vm.define :master1, primary: true do |master|  
        master.vm.provider "virtualbox" do |v|  
			v.customize ["modifyvm", :id, "--name", "hadoop-master1", "--memory", "1024"]  
       end  
       master.vm.box = "centos-7"  
       master.vm.hostname = "hadoop-master1"  
       master.vm.network :private_network, ip: "192.168.10.10"  
    end  
  
   (1..3).each do |i|  
    config.vm.define "slave#{i}" do |node|  
        node.vm.box = "centos-7"  
        node.vm.hostname = "hadoop-slave#{i}"  
        node.vm.network :private_network, ip: "192.168.10.1#{i}"  
        node.vm.provider "virtualbox" do |vb|  
          vb.customize ["modifyvm", :id, "--name", "hadoop-slave#{i}", "--memory", "1024"]  
        end  
     end  
   end  
  
  #manage hosts file   
  config.hostmanager.enabled = true  
  config.hostmanager.manage_host = true  
  config.hostmanager.manage_guest = true  
  
   #provision  
   config.vm.provision "shell", path: "init.sh", privileged: false  
end  

init.sh文件内容:

sudo yum install -y epel-release  
  
sudo yum install -y  lrzsz.x86_64  
sudo yum install -y nmap-ncat.x86_64  
sudo yum install -y net-tools  
sudo yum install -y vim-enhanced.x86_64  
sudo yum install -y sshpass  
  
sudo yum install -y java-1.8.0-openjdk-devel  
  1. vagrant启动虚拟机
    C:\itlgl\hadoop\vm1>vagrant up  
    Bringing machine 'master1' up with 'virtualbox' provider...  
    Bringing machine 'slave1' up with 'virtualbox' provider...  
    Bringing machine 'slave2' up with 'virtualbox' provider...  
    Bringing machine 'slave3' up with 'virtualbox' provider...  
    ==> master1: Importing base box 'centos-7'...  
    ==> master1: Matching MAC address for NAT networking...  
    ...省略一大波日志...  
    
  2. 每个虚拟机配置ssh登录
    在VirtualBox管理界面内依次登录虚拟机,用户名密码都是vagrant,vim /etc/ssh/sshd_config,修改如下配置为yes
    PubkeyAuthentication yes  
    PasswordAuthentication yes  
    
  3. 使用putty登录hadoop-master1机器
    vagrant ssh-config命令可以查看master1虚拟机的IdentityFile路径,使用putty-gen load加载私钥文件,转为ppk文件存储到本地

putty设置项:

  • Session名称:hadoop-master1
  • ip: 192.168.10.10
  • port: 22
  • Connection->Data=>Auto-login username: vagrant
  • Connection->SSH->Auth=>Private key file for authentication: 选择刚才的ppk文件
  • 桌面复制putty快捷方式,编辑属性=>目标: "C:\Program Files\PuTTY\putty.exe" -load hadoop-master1,双击即可快捷登录

待更新,,,

文档信息

Search

    Table of Contents