Enter Job ID:

 

Job Found

Program output is shown as following:

1386569460
eucarc = default
eucaImageEmi = fe974b15-490e-4082-95a3-3f3e35a83dcb
instanceType = 3
pubkey = default
numOfNodes = 1
applicationType = 2220_sconcomp_snap-social-slashdot0811.modified.galib_VM
Job start Time = 1386569460766

[INFO] Getting cloud mode configuration parameters
[INFO] getting chef cloud parameters.
/home/taklwu/taklwu_openstack.pem
/var/www/salsadpi/apps/hadoopWordCount.jar
/var/www/salsadpi/input/hadoopWordCountInput.txt
[INFO] Amount of running instances(s): 1
Job ID = b0b6f6dd-4763-4a30-8acb-7ddafa4193a8
[INFO] Cloud mode start VM(s) with using chef
[INFO] configured with recipe: recipe[hadoopCloud]
[INFO] Knife commands: knife openstack server create -a -I fe974b15-490e-4082-95a3-3f3e35a83dcb -f 3 -A taklwu -K Y2IzM2E4ZTBlNzFlMGUyNDQ4Yjk5ZGMw --openstack-api-endpoint http://149.165.146.50:5000/v2.0/tokens -S taklwu_openstack -x root -i /home/taklwu/taklwu_openstack.pem --insecure -r recipe[hadoopCloud]
1 VM(s) are starting.................................
..............................
..............................
..............................
.................. All VMs have been started
[INFO] Node 0 information f112af0b-3100-4b5e-ad34-544f2be26f31
[INFO] Total started VM = 1
[INFO] Cloud VM startup plus Chef preparation time = 142.725 Seconds
[INFO] Copying ssh information
[INFO] Setting VM Environment.
[INFO] Connecting to 149.165.158.2
sudo hostname os-87247498881823
[INFO] uploading data from /tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8_hosts to VM's/etc/hosts
[INFO] uploading data from /home/taklwu/taklwu_openstack.pem to VM's/root/.ssh/id_rsa
chmod 600 /root/.ssh/id_rsa
[INFO] uploading data from salsaDPI.jar to VM's/root/salsaDPI.jar
mkdir -p /tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8
[INFO] Uploading Binary and Input
[INFO] /var/www/salsadpi/apps/hadoopWordCount.jar /var/www/salsadpi/input/hadoopWordCountInput.txt
[INFO] copying data from local client to VMs
[INFO] uploading data from /var/www/salsadpi/apps/hadoopWordCount.jar to VM's/tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8
[INFO] copying data from local client to VMs
[INFO] uploading data from /var/www/salsadpi/input/hadoopWordCountInput.txt to VM's/tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8
[INFO] Cloud Program input preparation time = 21.606 Seconds
End of Copying ssh information
[INFO] starting with recipes service recipe[hadoopCloud]
cd /tmp/taklwu/; rm -rf data local name;
get HADOOP_HOME and JAVA_HOME
[REMOTE Msg] /tmp/hadoop-0.20.203.0
[REMOTE Msg] /tmp/jdk1.6.0_33/
/tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8_ips /tmp/hadoop-0.20.203.0/nodes
[INFO] uploading data from /tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8_ips to VM's/tmp/hadoop-0.20.203.0/nodes
cd /tmp/hadoop-0.20.203.0; . ./MultiNodesOneClickStartUp.sh /tmp/jdk1.6.0_33/ nodes
[REMOTE Msg] *********************************************************************
Hadoop 0.20.203.0 Multinodes OneClick Startup.
*********************************************************************
[REMOTE Msg] HADOOP_HOME = /tmp/hadoop-0.20.203.0
Master IP/Hostname = 10.1.2.4
Slaves IP/Hostname = 10.1.2.4

Username = root
[REMOTE Msg] Port 9000 is avaliable
Port 9001 is avaliable
Port 9002 is avaliable
Port 9003 is avaliable
set Master IP/Hostname
[REMOTE Msg] Set JAVA_HOME = /tmp/jdk1.6.0_33/
copy conf/hadoop-env.sh to work node 10.1.2.4
[REMOTE Msg] copy conf/core-site.xml to work node 10.1.2.4
[REMOTE Msg] copy conf/hdfs-site.xml to work node 10.1.2.4
copy conf/mapred-site.xml to work node 10.1.2.4
[REMOTE Msg] no jobtracker to stop
[REMOTE Msg] 10.1.2.4: no tasktracker to stop
no namenode to stop
10.1.2.4: no datanode to stop
[REMOTE Msg] 10.1.2.4: no secondarynamenode to stop
Remove all files under /tmp/root in 10.1.2.4
Remove all files under /tmp/root in 10.1.2.4
[REMOTE Msg] bin/hadoop namenode -format
[REMOTE Msg] bin/start-all.sh
starting namenode, logging to /tmp/hadoop-0.20.203.0/bin/../logs/hadoop-root-namenode-os-87247498881823.out
[REMOTE Msg] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
[REMOTE Msg] 10.1.2.4: starting datanode, logging to /tmp/hadoop-0.20.203.0/bin/../logs/hadoop-root-datanode-os-87247498881823.out
10.1.2.4: WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
[REMOTE Msg] 10.1.2.4: starting secondarynamenode, logging to /tmp/hadoop-0.20.203.0/bin/../logs/hadoop-root-secondarynamenode-os-87247498881823.out
10.1.2.4: WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
starting jobtracker, logging to /tmp/hadoop-0.20.203.0/bin/../logs/hadoop-root-jobtracker-os-87247498881823.out
[REMOTE Msg] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
[REMOTE Msg] 10.1.2.4: starting tasktracker, logging to /tmp/hadoop-0.20.203.0/bin/../logs/hadoop-root-tasktracker-os-87247498881823.out
10.1.2.4: WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
*************************************
Hadoop has been started successfully.
*************************************
Please use lynx 10.1.2.4:9001 to see HDFS status
Please use lynx 10.1.2.4:9003 to see MapReduce Daemon status
Application Type = HADOOP
Running Hadoop application
[REMOTE Msg] /tmp/hadoop-0.20.203.0
cd /tmp; mkdir -p /tmp/root_dpi/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8
mv /tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8/hadoopWordCountInput.txt /tmp/root_dpi/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8/; cd /tmp/hadoop-0.20.203.0; bin/hadoop dfs -put /tmp/root_dpi/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8 b0b6f6dd-4763-4a30-8acb-7ddafa4193a8
[REMOTE Msg] /tmp/hadoop-0.20.203.0
mkdir -p ~/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8
cd /tmp/hadoop-0.20.203.0; bin/hadoop jar /tmp/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8/hadoopWordCount.jar b0b6f6dd-4763-4a30-8acb-7ddafa4193a8 output
cd /tmp/hadoop-0.20.203.0; bin/hadoop dfs -copyToLocal output/* ~/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8/
/var/www/salsadpi/./salsaDPI_output/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8/output/
Creates the directory : true
[INFO] Download file/dir name part-r-00000
[INFO] Download file/dir name _logs
[INFO] Download file/dir name .
[INFO] Download file/dir name _SUCCESS
[INFO] Download file/dir name ..
cd /tmp/hadoop-0.20.203.0; bin/stop-all.sh
[REMOTE Msg] stopping jobtracker
10.1.2.4: stopping tasktracker
stopping namenode
[REMOTE Msg] 10.1.2.4: stopping datanode
[REMOTE Msg] 10.1.2.4: stopping secondarynamenode
[INFO] Hadoop or Twistert startup time plus program execution time = 46.335 Seconds
[INFO] VM termination and Chef deregistering.....
knife openstack server delete f112af0b-3100-4b5e-ad34-544f2be26f31 -A taklwu -K Y2IzM2E4ZTBlNzFlMGUyNDQ4Yjk5ZGMw --openstack-api-endpoint http://149.165.146.50:5000/v2.0/tokens --insecure -P -y
knife node delete os-87247498881823 -y
knife client delete os-87247498881823 -y
[INFO] VM termination and Chef deregister time = 6.215 Seconds
MasterNode Hostname=f112af0b-3100-4b5e-ad34-544f2be26f31
MasterNode Public IP=149.165.158.2
MasterNode Private IP=10.1.2.4
MasterNode Instance Name=os-87247498881823

[INFO] -------------------------------------------
[INFO] Job b0b6f6dd-4763-4a30-8acb-7ddafa4193a8 has been finished
[INFO] Please check program output at local path /var/www/salsadpi/salsaDPI_output/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8/output/
[INFO] -------------------------------------------

Output Dir = salsaDPI_output/b0b6f6dd-4763-4a30-8acb-7ddafa4193a8/output/

Job Execution Time = 249.772
Job Finished Time = 1386569710538
Job Finished