202 questions
- Bountied 0
- Unanswered
- Frequent
- Score
- Trending
- Week
- Month
- Unanswered (my tags)
1
vote
1
answer
45
views
Failed to find datanode (scope="" excludedScope="/rack0")
When I build a hadoop cluster(version 3.3.6) by docker swarm. I have 3 machines, and 1 for namenode, all for datanode. After all starts, I checked everything, namenode is healthy, datanode is healthy, ...
0
votes
1
answer
26
views
Impact of HDFS replication factor on namenode memory
Does increasing replication factor increases namenode memory usage in HDFS?
This link states replication factor do not have impact on namenode memory usage but another link states otherwise.
Dan's user avatar
- 1,474
0
votes
1
answer
43
views
Shutting down NameNode datanode resource manger etc immediately on running hadoop
C:\hadoop-3.3.6\sbin>.\start-dfs.cmd
after running above command command prompt opens for datanode and namenode but it immediately shutdown what should i do
java.io.IOException: All specified ...
1
vote
0
answers
58
views
Hadoop namenode container unhealthy
I am following a guided course on Apache Airflow, I am building images but finding my Hadoop namenode container unhealthy. When checked docker logs I found this:
Re-format filesystem in Storage ...
2
votes
2
answers
559
views
Cannot make hadoop HDFS data persist with docker
I have a namenode and a datanode that are created using this docker-compos.yaml file
version: "3"
services:
namenode:
image: apache/hadoop:3
hostname: 192.168.105.139
...
0
votes
2
answers
345
views
hdfs reformat name node failed by error Unable to check if JNs are ready for formatting
I want to start fresh hdfs cluster, I start zookeeper and yarn with command :
./zkServer.sh start && start-yarn.sh
after above command I started journalnode on each 5 node with command:
hdfs --...
0
votes
1
answer
161
views
How can I resolve the missing blocks inconsistency issue between 2 name nodes
After performing decommission and recommission tasks, I encountered an issue with missing blocks, resulting in an inconsistency problem. In Ambari, I received a "Namenode Health" alert with ...
1
vote
2
answers
4k
views
writing to hdfs error: Not enough replicas was chosen. Reason: {NO_REQUIRED_STORAGE_TYPE=1}
I encountered the problem similar to this:
https://stackoverflow.com/questions/52809233/failed-to-place-enough-replicas-expected-size-is-1-but-only-0-storage-types-can
my name node logs in $...
0
votes
0
answers
94
views
Namenode is not showing in JPS
Namenode is not working
When I run the command start-all.sh it says running namenode[localhost] But when I run JPS command namenode is not showing. Everytime I have to run hadoop namenode -format in ...
0
votes
1
answer
110
views
How to have a cluster of 3 datanodes that work at the same time?
I run one datanode with: ./bin/hdfs datanode -conf ./etc/hadoop/datanode1.xml only one work
when i try run two: "datanode is running as process. Stop it first and ensure /tmp/hadoop-user-datanode....
0
votes
1
answer
104
views
HDFS unreachable outside of network
How do I properly reach my hdfs in the cloud from my laptop with hdfs dfs command?
I set up a HDFS on EC2 following this guide and I've set up my (AWS) security group to allow all inbound and outbound ...
0
votes
1
answer
1k
views
How can I solve this problem : hadoop namenode -format gives error?
I setup Hadoop HA Cluster.
I try to format hadoop as below code:
hdfs --loglevel DEBUG namenode -format -force
then when I see the log ( it is not DEBUG mode and this is another problem which i ve ...
-1
votes
1
answer
22
views
Namenode can't running
I built a single hadoop,when I tried ./start-dfs.sh
enter image description here
namenode didn't work
logs↓
enter image description here
I have tried to delete tmp and initialize nanenode format hdfs ...
0
votes
1
answer
200
views
Re-format filesystem error starting Hadoop services on Mac
iMac 2020 Intel, MacOS Monterey 12.6, Java 1.8, Hadoop 3.3.4 as at 9-Feb-23
I am getting this error when starting Hadoop with this command:
$HADOOP_HOME/sbin/start-all.sh
Irrespective of response as ...
0
votes
1
answer
958
views
NameNode Format error "failure to login for principal: X from keytab Y: Unable to obtain password from user" with Kerberos in a Hadoop cluster
I've been setting up Kerberos with my Hadoop cluster on Ubuntu 20.04.1 LTS and when I try to reformat the namenode in command line after changing all config files and setting everything up (including ...