To trouble shoot hadoop issues we need to have debug logs to get more low level errors.by default debug logs are not enabled in Hortonworks data platform and in plain hadoop also.
In this post,we will discuss how to enable debug logs in HDP and plain hadoop.
1. Modify /var/lib/ambari-server/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/templates/hadoop-env.sh.j2
In HDP,we need to add below line to hadoop-env.sh.j2 to enable debug logs on HDFS services.
If you are using Plain hadoop you can directly add above line to /etc/hadoop/conf/hadoop-env.sh file in all nodes and restart HDFS deamons on all nodes.
If you are using Hortonworks Data Platform ,You need to follow below steps.
2. Restart Ambari agents on all nodes
We need to restart ambari agent on all master nodes,data nodes and edge nodes if any. We need to run below command for restarting.
service ambari-agent restart.
Or we can stop and start ambari agent.
service ambari-agent stop
service ambari-agent start
3. Restart Ambari server
We also need to restart ambari server on first master.We can run below command.
service ambari-server restart
Or we can stop and start ambari server.
service ambari-server stop
service ambari-server start
4. Restart HDFS services
In Ambari UI ,Click on HDFS , click service actions drop down and select restart all option.It will ask for confirmation once you confirm, it will restart all HDFS daemons.
Once debug logs are enabled you can check them in name node logs and data node logs.Debug logs consume lot of space you need to disable them once you collect required logs. to disable remove added export command and restart above mentioned services.