查看datanode的logs--http://slave:50075/logs/yarn-ubuntu-nodemanager-ip-172-31-0-241.log
查找Error关键词,可以看到下列错误信息
2017-05-19 02:04:06,480 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /home/ubuntu/Download/hadoop-2.8.0/tmp/nm-local-dir error, used space above threshold of 90.0%, removing from list of valid directories
2017-05-19 02:04:06,480 WARN org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /home/ubuntu/Download/hadoop-2.8.0/logs/userlogs error, used space above threshold of 90.0%, removing from list of valid directories
2017-05-19 02:04:06,481 INFO org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Disk(s) failed: 1/1 local-dirs are bad: /home/ubuntu/Download/hadoop-2.8.0/tmp/nm-local-dir; 1/1 log-dirs are bad: /home/ubuntu/Download/hadoop-2.8.0/logs/userlogs
2017-05-19 02:04:06,481 ERROR org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService: Most of the disks failed. 1/1 local-dirs are bad: /home/ubuntu/Download/hadoop-2.8.0/tmp/nm-local-dir; 1/1 log-dirs are bad: /home/ubuntu/Download/hadoop-2.8.0/logs/userlogs
解决方法: 在yarn-site.xml中配置:
<property>
<name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name>
<value>95.0</value>
</property>
然后重启yarn!
sbin/yarn-daemon.sh stop nodemanager
sbin/yarn-daemon.sh start nodemanager