You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Good afternoon, I encountered the fact that hadoop-yarn-nodemanager daemons do not start on datanodes.
All installation steps up to this step are fine, but here is the problem with this particular daemon.
How can this problem be solved?
The environment on which I run the playbook:
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy
TASK [hadoop : start services] ************************************************************************************************************************************************************************************
changed: [hadoop-datanode06-dev] => (item=hadoop-hdfs-datanode)
changed: [hadoop-datanode04-dev] => (item=hadoop-hdfs-datanode)
changed: [hadoop-datanode05-dev] => (item=hadoop-hdfs-datanode)
failed: [hadoop-datanode05-dev] (item=hadoop-yarn-nodemanager) => {"ansible_loop_var": "item", "changed": false, "item": "hadoop-yarn-nodemanager", "msg": "Unable to start service hadoop-yarn-nodemanager: Job for hadoop-yarn-nodemanager.service failed because the control process exited with error code. See \"systemctl status hadoop-yarn-nodemanager.service\" and \"journalctl -xe\" for details.\n"}
failed: [hadoop-datanode06-dev] (item=hadoop-yarn-nodemanager) => {"ansible_loop_var": "item", "changed": false, "item": "hadoop-yarn-nodemanager", "msg": "Unable to start service hadoop-yarn-nodemanager: Job for hadoop-yarn-nodemanager.service failed because the control process exited with error code. See \"systemctl status hadoop-yarn-nodemanager.service\" and \"journalctl -xe\" for details.\n"}
failed: [hadoop-datanode04-dev] (item=hadoop-yarn-nodemanager) => {"ansible_loop_var": "item", "changed": false, "item": "hadoop-yarn-nodemanager", "msg": "Unable to start service hadoop-yarn-nodemanager: Job for hadoop-yarn-nodemanager.service failed because the control process exited with error code. See \"systemctl status hadoop-yarn-nodemanager.service\" and \"journalctl -xe\" for details.\n"}
First error on data nodes:
23/08/31 06:18:08 ERROR containermanager.AuxServices: Failed to initialize mapreduce_shuffle
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler not fo
und
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2726)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.createAuxServiceFromConfiguration(AuxServices.java:204)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.createAuxService(AuxServices.java:297)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.initAuxService(AuxServices.java:452)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:758)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:327)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:494)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:962)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1042)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2693)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2718)
... 13 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2597)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2691)
... 14 more
Second error on datanodes:
23/08/31 06:18:08 ERROR nodemanager.NodeManager: Error starting NodeManager
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2726)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.createAuxServiceFromConfiguration(AuxServices.java:204)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.createAuxService(AuxServices.java:297)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.initAuxService(AuxServices.java:452)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:758)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:327)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:109)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:494)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:962)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1042)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2693)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2718)
... 13 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2597)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2691)
... 14 more
The text was updated successfully, but these errors were encountered:
Good afternoon, I encountered the fact that hadoop-yarn-nodemanager daemons do not start on datanodes.
All installation steps up to this step are fine, but here is the problem with this particular daemon.
How can this problem be solved?
The environment on which I run the playbook:
Ansible version:
Servers environment:
hdfs version which installed the playbook:
Launch playbook:
ansible-playbook -i inventory/yandex/dev/hosts-hadoop-cloudera playbooks/hadoop-cloudera.yml
Error ansible task:
First error on data nodes:
Second error on datanodes:
The text was updated successfully, but these errors were encountered: