HDP (Hortonworks Data Platform) installation using Apache Ambari.
In this article , We will learn how to install HDP (Hortonworks Data Platform) using Apache Ambari. Before HDP installation we need to complete Ambari server installation.
1) Monitor the Ambari-server log
It is recommended to have live amabri-server log to see detailed error if you hit any issues during the HDP installation. So open ambari-server log file and monitor the same.
The picture below shows live ambari-server log.
2) Install Wizard
Login into Ambari GUI using credentials (default :admin/admin) and click on Launch install wizard button to start HDP installation.
3) HDP version
Select the HDP version you want to install. Amabri also shows what are the different components we get in a HDP version.
Currently HDP2.6 is the latest and we are trying to install HDP 2.5 here.
Click Next once HDP version is selected.
4)Name the cluster
Mention the name you want to use for your cluster. I have given myhdp25 as the name to my cluster.
5) Install options
In this step , We will register nodes to be added to the cluster. We will mention hostnames one per line. We can also use patterns, click on pattern expressions to know more.
We are adding single node named secured01 to the cluster.
/etc/hosts file on every node should contain whatever hostnames you add here.
We can register nodes in two ways.
Automatic registration :
Ambari will auto-install ambari-agents on all nodes to register them with ambari-server. We need to specify private key of Ambari server host.
Check private key file path below.
Manual registration :
In manual registration , we are responsible for installing and configuring ambari-agents to make them report to ambari server.
We are going with automatic registration. click next after entering hostnames and private key.
All hostnames mentioned should be FullyQualifiedDomainNames (FQDN) names. In other words hostnames should be output of hostname -f command.
Ambari will throw warning to check if hostnames are FQDNs, if you are confident click OK.
6)Confirm hosts
In this step , Ambari will check all nodes to see if they meet prerequisites for hadoop installation and also installs and configures ambari-agents.
Ambari will display warnings in the bottom if any of nodes do not meet prerequisites. Click on the click to see warning link to know more about warnings.
The picture below shows ntpd process is not started on a node.
Go to node and start the ntpd process . If ntp is not installed ,first install (yum install ntp) and then start the ntp.
After fixing warnings click on rerun checks to confirm warning are gone.
The picture below shows no warnings now.
7) Choose services
In this step, we will choose services required for our cluster. We are selecting HDFS, Mapreduce2 and YARN.
Ambari will also add services those are mandatory.
8) Assign Masters.
In this step , We will choose nodes for master components. For example , we will choose a node/host to install Namenode.
9) Assign slave and clients
In this step, we will just select select slave components and clients to be installed.
10) Customize Services
In this step , We can modify configurations of the services. Ambari will show us red colored numbers if any inputs are required from us.
For example : ambari-metrics and Smartsense require password for admin user. mention admin password for both and click next.
Ambari will also recommend configuration values as warnings. You can also ignore them and clik proceed anyway or hit cancel to fix them.
11) Review
Ambari gives us last opportunity to review our HDP installation. At this step , We can go back and modify things.
In this step, Ambari shows installation details like hosts,repositories and services.
Click deploy to continue installation.
12) Install , Start and Test
This step performs three operations .
Installation of services
Starting the services
Performing service checks to see if services are in working state.
It takes some time to complete this step if services are being downloaded from the public repositories.
13) Summary
Ambari will provide us the final summary about our HDP installation. It will also list warnings and errors if any.
The picture below shows one warning and one error.
Click on the complete to go to your new HDP cluster.
14)
The picture below shows New HDP cluster installed with HDFS, Mapreduce2, YARN, Zookeeper, Metrics and Smartsense services.
This is how we install HDP (Hortonqorks Data Platform) using Apache Ambari on single node. This is the same process we can follow for multi node cluster. Few steps like install options and Assign masters will change.
Let me know if you have any questions.
In this article , We will learn how to install HDP (Hortonworks Data Platform) using Apache Ambari. Before HDP installation we need to complete Ambari server installation.
1) Monitor the Ambari-server log
It is recommended to have live amabri-server log to see detailed error if you hit any issues during the HDP installation. So open ambari-server log file and monitor the same.
The picture below shows live ambari-server log.
2) Install Wizard
Login into Ambari GUI using credentials (default :admin/admin) and click on Launch install wizard button to start HDP installation.
3) HDP version
Select the HDP version you want to install. Amabri also shows what are the different components we get in a HDP version.
Currently HDP2.6 is the latest and we are trying to install HDP 2.5 here.
Click Next once HDP version is selected.
4)Name the cluster
Mention the name you want to use for your cluster. I have given myhdp25 as the name to my cluster.
5) Install options
In this step , We will register nodes to be added to the cluster. We will mention hostnames one per line. We can also use patterns, click on pattern expressions to know more.
We are adding single node named secured01 to the cluster.
/etc/hosts file on every node should contain whatever hostnames you add here.
We can register nodes in two ways.
Automatic registration :
Ambari will auto-install ambari-agents on all nodes to register them with ambari-server. We need to specify private key of Ambari server host.
Check private key file path below.
Manual registration :
In manual registration , we are responsible for installing and configuring ambari-agents to make them report to ambari server.
We are going with automatic registration. click next after entering hostnames and private key.
All hostnames mentioned should be FullyQualifiedDomainNames (FQDN) names. In other words hostnames should be output of hostname -f command.
Ambari will throw warning to check if hostnames are FQDNs, if you are confident click OK.
6)Confirm hosts
In this step , Ambari will check all nodes to see if they meet prerequisites for hadoop installation and also installs and configures ambari-agents.
Ambari will display warnings in the bottom if any of nodes do not meet prerequisites. Click on the click to see warning link to know more about warnings.
The picture below shows ntpd process is not started on a node.
Go to node and start the ntpd process . If ntp is not installed ,first install (yum install ntp) and then start the ntp.
After fixing warnings click on rerun checks to confirm warning are gone.
The picture below shows no warnings now.
7) Choose services
In this step, we will choose services required for our cluster. We are selecting HDFS, Mapreduce2 and YARN.
Ambari will also add services those are mandatory.
8) Assign Masters.
In this step , We will choose nodes for master components. For example , we will choose a node/host to install Namenode.
9) Assign slave and clients
In this step, we will just select select slave components and clients to be installed.
10) Customize Services
In this step , We can modify configurations of the services. Ambari will show us red colored numbers if any inputs are required from us.
For example : ambari-metrics and Smartsense require password for admin user. mention admin password for both and click next.
Ambari will also recommend configuration values as warnings. You can also ignore them and clik proceed anyway or hit cancel to fix them.
11) Review
Ambari gives us last opportunity to review our HDP installation. At this step , We can go back and modify things.
In this step, Ambari shows installation details like hosts,repositories and services.
Click deploy to continue installation.
12) Install , Start and Test
This step performs three operations .
Installation of services
Starting the services
Performing service checks to see if services are in working state.
It takes some time to complete this step if services are being downloaded from the public repositories.
13) Summary
Ambari will provide us the final summary about our HDP installation. It will also list warnings and errors if any.
The picture below shows one warning and one error.
Click on the complete to go to your new HDP cluster.
14)
The picture below shows New HDP cluster installed with HDFS, Mapreduce2, YARN, Zookeeper, Metrics and Smartsense services.
This is how we install HDP (Hortonqorks Data Platform) using Apache Ambari on single node. This is the same process we can follow for multi node cluster. Few steps like install options and Assign masters will change.
Let me know if you have any questions.
No comments:
Post a Comment