Enter Job ID:

 

Job Found

Program output is shown as following:

1386015326
eucarc = default
eucaImageEmi = emi-A8F63C29
instanceType = c1.medium
pubkey = default
numOfNodes = 1
applicationType = HadoopKmeans
Job start Time = 1386015326557

[INFO] Getting cloud mode configuration parameters
[INFO] getting chef cloud parameters.
/home/taklwu/stephen.pem
/var/www/salsadpi/apps/hadoopKmeans.jar
[INFO] Amount of running instances(s): 1
Job ID = b30ffe84-94be-4642-ab8a-f71c715073bb
[INFO] Cloud mode start VM(s) with using chef
[INFO] configured with recipe: recipe[hadoopCloud]
1 VM(s) are starting.................................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
......... All VMs have been started
[INFO] Node 0 information i-E84F3E91
[INFO] Cloud VM startup plus Chef preparation time = 284.687 Seconds
[INFO] Copying ssh information
[INFO] Setting VM Environment.
[INFO] Connecting to 149.165.158.215
sudo hostname i-E84F3E91
[INFO] uploading data from /tmp/b30ffe84-94be-4642-ab8a-f71c715073bb_hosts to VM's/etc/hosts
[INFO] uploading data from /home/taklwu/stephen.pem to VM's/root/.ssh/id_rsa
chmod 600 /root/.ssh/id_rsa
[INFO] uploading data from salsaDPI.jar to VM's/root/salsaDPI.jar
mkdir -p /tmp/b30ffe84-94be-4642-ab8a-f71c715073bb
[INFO] Uploading Binary and Input
[INFO] /var/www/salsadpi/apps/hadoopKmeans.jar
[INFO] copying data from local client to VMs
[INFO] uploading data from /var/www/salsadpi/apps/hadoopKmeans.jar to VM's/tmp/b30ffe84-94be-4642-ab8a-f71c715073bb
[INFO] Cloud Program input preparation time = 8.428 Seconds
End of Copying ssh information
Application Type = HADOOP
Running Hadoop application
cd /tmp/taklwu/; rm -rf data local name;
get HADOOP_HOME and JAVA_HOME
[REMOTE Msg] /root/hadoop-0.20.203.0
[REMOTE Msg] /tmp/jdk1.6.0_33/
/tmp/b30ffe84-94be-4642-ab8a-f71c715073bb_ips /root/hadoop-0.20.203.0/nodes
[INFO] uploading data from /tmp/b30ffe84-94be-4642-ab8a-f71c715073bb_ips to VM's/root/hadoop-0.20.203.0/nodes
cd /root/hadoop-0.20.203.0; . ./MultiNodesOneClickStartUp.sh /tmp/jdk1.6.0_33/ nodes
[REMOTE Msg] *********************************************************************
Hadoop 0.20.203.0 Multinodes OneClick Startup.
*********************************************************************
HADOOP_HOME = /root/hadoop-0.20.203.0
Master IP/Hostname = 10.154.70.152
Slaves IP/Hostname = 10.154.70.152

Username = root
Port 9000 is avaliable
Port 9001 is avaliable
Port 9002 is avaliable
Port 9003 is avaliable
set Master IP/Hostname
Set JAVA_HOME = /tmp/jdk1.6.0_33/
copy conf/hadoop-env.sh to work node 10.154.70.152
copy conf/core-site.xml to work node 10.154.70.152
copy conf/hdfs-site.xml to work node 10.154.70.152
[REMOTE Msg] copy conf/mapred-site.xml to work node 10.154.70.152
no jobtracker to stop
10.154.70.152: no tasktracker to stop
no namenode to stop
[REMOTE Msg] 10.154.70.152: no datanode to stop
10.154.70.152: no secondarynamenode to stop
Remove all files under /tmp/root in 10.154.70.152
Remove all files under /tmp/root in 10.154.70.152
[REMOTE Msg] bin/hadoop namenode -format
[REMOTE Msg] bin/start-all.sh
[REMOTE Msg] starting namenode, logging to /root/hadoop-0.20.203.0/bin/../logs/hadoop-root-namenode-i-E84F3E91.out
[REMOTE Msg] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
[REMOTE Msg] 10.154.70.152: starting datanode, logging to /root/hadoop-0.20.203.0/bin/../logs/hadoop-root-datanode-i-E84F3E91.out
[REMOTE Msg] 10.154.70.152: starting secondarynamenode, logging to /root/hadoop-0.20.203.0/bin/../logs/hadoop-root-secondarynamenode-i-E84F3E91.out
[REMOTE Msg] starting jobtracker, logging to /root/hadoop-0.20.203.0/bin/../logs/hadoop-root-jobtracker-i-E84F3E91.out
[REMOTE Msg] 10.154.70.152: starting tasktracker, logging to /root/hadoop-0.20.203.0/bin/../logs/hadoop-root-tasktracker-i-E84F3E91.out
checking Hadoop status, retrying after 4 second......
[REMOTE Msg] *************************************
Hadoop has been started successfully.
*************************************
Please use lynx 10.154.70.152:9001 to see HDFS status
Please use lynx 10.154.70.152:9003 to see MapReduce Daemon status
mkdir -p ~/b30ffe84-94be-4642-ab8a-f71c715073bb
cd /root/hadoop-0.20.203.0; bin/hadoop jar /tmp/b30ffe84-94be-4642-ab8a-f71c715073bb/hadoopKmeans.jar 500 10 8 3 b30ffe84-94be-4642-ab8a-f71c715073bb > ~/b30ffe84-94be-4642-ab8a-f71c715073bb/b30ffe84-94be-4642-ab8a-f71c715073bb.txt
/var/www/salsadpi/./salsaDPI_output/b30ffe84-94be-4642-ab8a-f71c715073bb/output/
Creates the directory : true
[INFO] Download file/dir name ..
[INFO] Download file/dir name .
[INFO] Download file/dir name b30ffe84-94be-4642-ab8a-f71c715073bb.txt
cd /root/hadoop-0.20.203.0; bin/stop-all.sh
[REMOTE Msg] stopping jobtracker
10.154.70.152: stopping tasktracker
stopping namenode
10.154.70.152: stopping datanode
[REMOTE Msg] 10.154.70.152: stopping secondarynamenode
[INFO] Hadoop or Twistert startup time plus program execution time = 217.133 Seconds
[INFO] VM termination and Chef deregistering.....
[INFO] VM termination and Chef deregistering
[INFO] VM termination and Chef deregister time = 7.65 Seconds

[INFO] -------------------------------------------
[INFO] Job b30ffe84-94be-4642-ab8a-f71c715073bb has been finished
[INFO] Please check program output at local path /var/www/salsadpi/salsaDPI_output/b30ffe84-94be-4642-ab8a-f71c715073bb/output/
[INFO] -------------------------------------------

Output Dir = salsaDPI_output/b30ffe84-94be-4642-ab8a-f71c715073bb/output/

Job Execution Time = 518.123
Job Finished Time = 1386015844680
Job Finished