The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

This page contains historical information. It may be outdated or unreliable.

This process no longer needs to run on the snapshot hosts. (Nov 1 2010)

Hadoop is being setup on snapshot1, snapshot2, snapshot3 to test XML Snapshots.

Setting up a node cluster

wget http://www.trieuvan.com/apache/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz
apt-get install sun-java6-bin

core-site.xml

<configuration>
 <property>
    <name>fs.default.name</name>
    <value>hdfs://208.80.152.139:9000</value>
  </property>
</configuration>

hdfs-site.xml

<configuration>
  <property>
    <name>dfs.name.dir</name>
    <value>/var/hdfs/name</value>
    <final>true</final>
   </property>
  <property>
    <name>dfs.data.dir</name>
    <value>/var/hdfs/data</value>
    <final>true</final>
   </property>p
  <property>
    <name>fs.checkpoint.dir</name>
    <value>/var/hdfs/namesecondary</value>
    <final>true</final>
   </property>
</configuration>

Slaves

Add slave servers to conf/slaves

hadoop-env.sh

export JAVA_HOME=/usr/lib/jvm/java-6-sun/

Post Config

mkdir /var/hdfs/name
mkdir /var/hdfs/data
chown -R hadoop.hadoop /var/hdfs

On namenode:

mkdir /var/hdfs/namesecondary

Logs

namenode - tells you who's registered connections
datanode - block activity

SVN

private-wmf/hadoop/conf