Tutorial for Latest Ambari(2.7.1)

Posted on In , Views: Symbols count in article: 4.6k Reading time ≈ 3 mins.

ambari-2.7-12

Before I start to install Ambari 2.7.1, I wrote some key information that I need to use later.

Key Information

hostname ip operating system RAM disk space cores of CPU package
master01.ambari.com 192.168.110.210 CentOS-7-x86_64-Minimal-1611.iso 20g 100g 16 ambari-server
slave01.ambari.com 192.168.110.211 CentOS-7-x86_64-Minimal-1611.iso 20g 100g 16 ambari-agent
slave02.ambari.com 192.168.110.212 CentOS-7-x86_64-Minimal-1611.iso 20g 100g 16 ambari-agent

VM Setting and CentOS7 Installation

VM Setting

Upload the IOS file

Before we start to create VMs and install CentOS7, we ought to upload CentOS7 installation file (using minimal version) to Datacenter in VMware Vsphere.

  1. Choose a specific data node in your Datacenter, then upload files.

    10

Create VMs

  1. Visit and login vsphere Web Client.

    1

  2. Create a folder for collecting all nodes.

    2

  3. Create a VM node in a specific folder that created in the last step.

    3

  4. Input the name of VM

    4

  5. Choose computing resources.

    5

  6. Choose the storage node.

    6

  7. Choose the compatibility of your VM.

    7

  8. Choose the version of your VM.

    8

  9. Input the number of your CPU cores, the memory size, the hard-drive size and the IOS file path of your CentOS7.

    9

CentOS7 Installation

  1. Power on the VM.

    11

  2. Choose the language, then click the button “Continue”.

    12

  3. Start to update setting(changing date&time, specifying installation destination, turning on the network).

    13

  4. Specify the installation destination.

    14

    15

  5. Turn on the network or update the configuration of the network.

    16

  6. Finally, update the password of the user root.

    17

File Preparation

We can download the requisite installation file from the official website below.

1
2
3
4
5
6
7
8
#Ambari
https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/ambari_repositories.html

#HDP
https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/hdp_30_repositories.html

#HDF
https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.2.0/release-notes/content/hdf_repository_locations.html

  1. Ambari installation file.

    ambari-2.7.1.0-centos7.tar.gz.tar

  2. HDP installation file.

    HDP-3.0.1.0-centos7-rpm.tar.gz.tar

    HDP-UTILS-1.1.0.22-centos7.tar.gz.tar

  3. HDF installation file.

    hdf-ambari-mpack-3.2.0.0-520.tar.gz.tar

    HDF-3.2.0.0-centos7-rpm.tar.gz.tar

System Preparation

Before we start to install Ambari 2.7.1 cluster, some important steps we need to complete.

  1. Install some basic tools.

    1
    2
    yum -y install unzip
    yum -y install wget

  2. Change the Hostname on all cluster hosts.

    1. Edit the Host File on all cluster hosts.

      1
      vi /etc/hosts

      Then append the content below to the Host File (Do NOT delete the original contents).

      1
      2
      3
      192.168.110.210 master01.ambari.com master01
      192.168.110.211 slave01.ambari.com slave01
      192.168.110.212 slave02.ambari.com slave02

    2. Set the Hostname on all cluster hosts.

      master01.ambari.com:

      1
      hostname master01.ambari.com

      slave01.ambari.com:

      1
      hostname slave01.ambari.com

      slave02.ambari.com:

      1
      hostname slave02.ambari.com

    3. Edit the Network Configuration File on all cluster hosts.

      1
      vi /etc/sysconfig/network

      Then append the contents below to the file.

      1
      2
      NETWORKING=yes
      HOSTNAME=<fully.qualified.domain.name>

    4. Test the hostname on all cluster hosts.

      1
      2
      hostname
      hostname -f

      If the result above is incorrect, you need to repair it until the result is what you expect it.

  3. Set up Password-less SSH

    To have Ambari Server automatically install Ambari Agent on all your cluster hosts, you must set up a password-less SSH connection between the Ambari Server host and all other hosts in the cluster.

    1. Generate public and private SSH keys on the Ambari Server host.

      1
      ssh-keygen

    2. Using the ssh command to copy a master public key file to slave nodes.

      1
      2
      ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave01.ambari.com
      ssh-copy-id -i ~/.ssh/id_rsa.pub root@slave02.ambari.com

    3. Test connections between the Ambari Server and the other cluster hosts.

      1
      2
      3
      ssh root@master01.ambari.com
      ssh root@slave01.ambari.com
      ssh root@slave02.ambari.com

      It is successful when you do not need to input any passwords.

  4. Enable NTP on all cluster hosts.

    1
    2
    yum install -y ntp
    systemctl enable ntpd

  5. Disable SELinux on all cluster hosts.

    1
    vi /etc/selinux/config

    Then change the value of SELINUX.

    1
    SELINUX=disabled

  6. Disable iptables on all cluster hosts.

    For Ambari to communicate during installation, certain ports must be open and available. So, the easiest way to do this is to temporarily disable iptables.

    1
    2
    systemctl disable firewalld
    systemctl stop firewalld

  7. Install Oracle JDK8 (Not JRE) and add JCE extension package on all cluster hosts.

    You can also install JDK8 including JCE automatically on the step that runs command “amber-server setup”

    1. Download the Oracle JDK8 installation file from Oracle.

    2. Download the JCE installation file.

      1
      http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html

    3. Install JDK8 firstly on all cluster hosts. 

      1
      yum install [your jdk8 yum installation file]

    4. Unzip JCE file to “$JAVA_HOME/jre/lib/security/“ on all cluster hosts.

      1
      unzip -o -j -q jce_policy-8.zip -d /usr/java/jdk1.8.0_201-amd64/jre/lib/security

    5. Reboot.

    6. JDK HOME

      1
      /usr/java/jdk1.8.0_201-amd64/

  8. Install Mysql Server on the Ambari Server host.

    1. Download and add the repository, then update.

      1
      2
      3
      wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
      rpm -ivh mysql-community-release-el7-5.noarch.rpm
      yum update

    2. Install mysql-server and start the service.

      1
      2
      yum install mysql-server
      systemctl start mysqld

    3. Change the password of the user ROOT (the default password is blank).

      1
      mysql -uroot -p
      1
      2
      set password for 'root'@'localhost' = password('root');
      set password for 'root'@'master01.ambari.com' = password('root');

    4. Create the database “ambari” and the user “ambari”.

      1
      2
      3
      4
      5
      6
      7
      create database ambari;

      create user 'ambari'@'localhost' identified by 'ambari';
      create user 'ambari'@'master01.ambari.com' identified by 'ambari';

      grant all on ambari.* to 'ambari'@'localhost';
      grant all on ambari.* to 'ambari'@'master01.ambari.com';

  9. Download mysql-connector-java.jar and put this jar to a specific folder on the Ambari Server host.

    1
    2
    3
    4
    5
    6
    7
    8
    wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar

    mkdir /usr/share/java

    cp /root/mysql-connector-java-5.1.47.jar /usr/share/java/

    #JDBC Driver Path
    /usr/share/java/mysql-connector-java-5.1.47.jar

  10. Setting up a local repository with NO Internet access on the Ambari Server host.

    1. Install and start httpd.

      1
      2
      yum -y install httpd
      systemctl start httpd

    2. Untar installation file to the web server folder.

      1
      2
      3
      4
      5
      6
      7
      mkdir /var/www/html/hdf
      mkdir /var/www/html/hdp

      tar -xvf /root/ambari-2.7.1.0-centos7.tar.gz.tar -C /var/www/html
      tar -xvf /root/HDP-3.0.1.0-centos7-rpm.tar.gz.tar -C /var/www/html/hdp
      tar -xvf /root/HDP-UTILS-1.1.0.22-centos7.tar.gz.tar -C /var/www/html/hdp
      tar -xvf /root/HDF-3.2.0.0-centos7-rpm.tar.gz.tar -C /var/www/html/hdf

    3. Confirm that you can browse to the newly created local repository.

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      #Ambari BaseURL
      http://192.168.110.210/ambari/centos7/2.7.1.0-169/

      #HDP BaseURL
      http://192.168.110.210/hdp/HDP/centos7/3.0.1.0-187/

      #HDP-UTIL BaseURL
      http://192.168.110.210/hdp/HDP-UTILS/centos7/1.1.0.22/

      #HDF BaseURL
      http://192.168.110.210/hdf/HDF/centos7/3.2.0.0-520/

    4. Downloading and Editing the Ambari Repository Configuration File to Use the Local Repository.

      1
      2
      3
      wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.1.0/ambari.repo -O /etc/yum.repos.d/ambari.repo

      vi /etc/yum.repos.d/ambari.repo

      Replace the Ambari baseurl to your local repository.

      1
      2
      3
      4
      5
      6
      7
      8
      9
      #VERSION_NUMBER=2.7.1.0-169
      [ambari-2.7.1.0]
      #json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
      name=ambari Version - ambari-2.7.1.0
      baseurl=http://192.168.110.210/ambari/centos7/2.7.1.0-169/
      gpgcheck=1
      gpgkey=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.1.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
      enabled=1
      priority=1

  11. Install and set up the Ambari server on the Ambari Server host.

    1
    yum install ambari-server

    Then, start to set up ambari-server.

    1
    ambari-server setup

    18

  12. Run the SQL script.

    1
    2
    3
    4
    5
    mysql -uambari -p

    use ambari;

    source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

  13. Start Ambari-server.

    1
    ambari-server start

19

Installing Ambari Cluster

  1. Browse Ambari WebUI, then use account admin to log in.

    ambari-2.7-1

  2. Launch Install Wizard.

    ambari-2.7-2

  3. Step 0: Name your cluster.

    ambari-2.7-3

  4. Step 1: Edit BaseURL to your local repository.

    ambari-2.7-4

  5. Step 2: Input a list of hosts using the hostname and input your ssh private key of the Ambari server host.

ambari-2.7-5

  1. Step 3: Confirm Hosts.

    ambari-2.7-6

  2. Step 4: Choose Services.

    ambari-2.7-7

  3. Step 5: Assign Masters.

    ambari-2.7-8

  4. Step 6: Assign Slaves and Clients.

    ambari-2.7-9

  5. Step 7-1: Input usernames and passwords for your services.

    ambari-2.7-10-1

  6. Step 7-2: Create Databases and users for services, then input information for login.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    #Ranger
    create database ranger;
    create user 'ranger'@'localhost' identified by 'ranger';
    create user 'ranger'@'master01.ambari.com' identified by 'ranger';
    grant all on ranger.* to 'ranger'@'localhost';
    grant all on ranger.* to 'ranger'@'master01.ambari.com';

    #RangerKMS
    create database rangerkms;
    create user 'rangerkms'@'localhost' identified by 'rangerkms';
    create user 'rangerkms'@'master01.ambari.com' identified by 'rangerkms';
    grant all on rangerkms.* to 'rangerkms'@'localhost';
    grant all on rangerkms.* to 'rangerkms'@'master01.ambari.com';

    #Oozie
    create database oozie;
    create user 'oozie'@'localhost' identified by 'oozie';
    create user 'oozie'@'master01.ambari.com' identified by 'oozie';
    grant all on oozie.* to 'oozie'@'localhost';
    grant all on oozie.* to 'oozie'@'master01.ambari.com';

    #Hive
    create database hive;
    create user 'hive'@'localhost' identified by 'hive';
    create user 'hive'@'master01.ambari.com' identified by 'hive';
    grant all on hive.* to 'hive'@'localhost';
    grant all on hive.* to 'hive'@'master01.ambari.com';

    ambari-2.7-10-2

  7. Step 7-3: Follow other steps to complete.

    ambari-2.7-10-3

    ambari-2.7-10-4

ambari-2.7-10-5

  1. Step 8: Review

    ambari-2.7-11

  2. Step 9: Install, Start, and Test.

  3. Summary.

  4. Completion

    ambari-2.7-12

Installing HDF on Ambari 2.7.1 + HDP 3.0.1

Regarding the HDF installation, I will write another post to describe specifically.