Enter Job ID:

 

Job Found

Program output is shown as following:

1386015884
eucarc = default
eucaImageEmi = 8b058f02-acb1-45a2-927b-98d59ad6c571
instanceType = 2
pubkey = default
numOfNodes = 2
applicationType = TwisterWordCount
Job start Time = 1386015885131

[INFO] Getting cloud mode configuration parameters
[INFO] getting chef cloud parameters.
/home/taklwu/taklwu-sierraKey.pem
/var/www/salsadpi/apps/Twister-WordCount-0.9.jar
/var/www/salsadpi/input/twisterWordCountInput.tar.gz
[INFO] Amount of running instances(s): 2
Job ID = a55705d1-58b3-4dbd-aa29-51b78b45a912
[INFO] Cloud mode start VM(s) with using chef
[INFO] configured with recipe: recipe[twisterCloud]
2 VM(s) are starting.................................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
All VMs have been started
[INFO] Node 0 information d2da8241-8379-4684-8a7f-d5002156da3d
[INFO] Node 1 information cfb6ff75-047f-45cc-92d1-ef598493e787
[INFO] Cloud VM startup plus Chef preparation time = 306.594 Seconds
[INFO] Copying ssh information
[INFO] Setting VM Environment.
[INFO] Connecting to 198.202.121.21
sudo hostname d2da8241-8379-4684-8a7f-d5002156da3d
[INFO] uploading data from /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912_hosts to VM's/etc/hosts
[INFO] uploading data from /home/taklwu/taklwu-sierraKey.pem to VM's/root/.ssh/id_rsa
chmod 600 /root/.ssh/id_rsa
[INFO] uploading data from salsaDPI.jar to VM's/root/salsaDPI.jar
mkdir -p /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912
[INFO] Uploading Binary and Input
[INFO] /var/www/salsadpi/apps/Twister-WordCount-0.9.jar /var/www/salsadpi/input/twisterWordCountInput.tar.gz
[INFO] copying data from local client to VMs
[INFO] uploading data from /var/www/salsadpi/apps/Twister-WordCount-0.9.jar to VM's/tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912
[INFO] copying data from local client to VMs
[INFO] uploading data from /var/www/salsadpi/input/twisterWordCountInput.tar.gz to VM's/tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912
[INFO] Connecting to 198.202.121.22
sudo hostname cfb6ff75-047f-45cc-92d1-ef598493e787
[INFO] uploading data from /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912_hosts to VM's/etc/hosts
[INFO] uploading data from /home/taklwu/taklwu-sierraKey.pem to VM's/root/.ssh/id_rsa
chmod 600 /root/.ssh/id_rsa
[INFO] uploading data from salsaDPI.jar to VM's/root/salsaDPI.jar
mkdir -p /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912
[INFO] Uploading Binary and Input
[INFO] /var/www/salsadpi/apps/Twister-WordCount-0.9.jar /var/www/salsadpi/input/twisterWordCountInput.tar.gz
[INFO] copying data from local client to VMs
[INFO] uploading data from /var/www/salsadpi/apps/Twister-WordCount-0.9.jar to VM's/tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912
[INFO] copying data from local client to VMs
[INFO] uploading data from /var/www/salsadpi/input/twisterWordCountInput.tar.gz to VM's/tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912
[INFO] Cloud Program input preparation time = 37.963 Seconds
End of Copying ssh information
Application Type = TWISTER
Running Twister application, Type: TWISTER
prefix wc_data
get TWISTER_HOME and JAVA_HOME
[REMOTE Msg] /tmp/twister-0.9
/tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912_ips /tmp/twister-0.9/bin/nodes
[INFO] uploading data from /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912_ips to VM's/tmp/twister-0.9//bin/nodes
cd /tmp/twister-0.9/bin/; ./TwisterPowerMakeUp.sh
[REMOTE Msg] use normal MultiNode Setup
no special processing to nodes
ActiveMQ uri=failover:(tcp://10.35.23.119:61616)
nodes_file=/tmp/twister-0.9/bin/nodes
daemons_per_node=1
workers_per_daemon=1
app_dir=/tmp/twister-0.9/apps
[REMOTE Msg] 10.35.23.124:/tmp/twister-0.9/data created.
10.35.23.119:/tmp/twister-0.9/data created.
data_dir=/tmp/twister-0.9/data
Change max memory to 1335 MB
[REMOTE Msg] copied to 10.35.23.124:/tmp/twister-0.9
copied to 10.35.23.119:/tmp/twister-0.9
Auto configuration is done.
cd /tmp/twister-0.9/bin/; echo 'uri = failover\:(tcp\://10.35.23.119\:61616)' > amq.properties
cd /tmp/twister-0.9/bin/; scp amq.properties 10.35.23.124:/tmp/twister-0.9//bin/;
cd /tmp/twister-0.9/bin/; scp stimr.sh 10.35.23.124:/tmp/twister-0.9//bin/;
[REMOTE Msg] /tmp/apache-activemq-5.4.2
cd /tmp/apache-activemq-5.4.2/bin/; ./activemq start
[REMOTE Msg] INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /root/.activemqrc)

INFO: Invoke the following command to create a configuration file
./activemq setup [ /etc/default/activemq | /root/.activemqrc ]

INFO: Using java '/tmp/jdk1.6.0_33//bin/java'
INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
INFO: pidfile created : '/tmp/apache-activemq-5.4.2/data/activemq.pid' (pid '4357')
cd /tmp/twister-0.9/bin/; nohup ./start_twister.sh > nohup.txt &
[INFO] ssh exeAndQuit connection run;
[INFO] Twister Started
cd /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912;tar -zxvf twisterWordCountInput.tar.gz; cd /tmp/twister-0.9/bin ; ./twister.sh mkdir a55705d1-58b3-4dbd-aa29-51b78b45a912; ./twister.sh put /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912/data a55705d1-58b3-4dbd-aa29-51b78b45a912 wc_data 1 1;
[REMOTE Msg] data/
data/wc_data_2.txt
data/wc_data_3.txt
[REMOTE Msg] data/wc_data_0.txt
[REMOTE Msg] data/wc_data_1.txt
[REMOTE Msg] 10.35.23.124:/tmp/twister-0.9/data/a55705d1-58b3-4dbd-aa29-51b78b45a912 created.
[REMOTE Msg] 10.35.23.119:/tmp/twister-0.9/data/a55705d1-58b3-4dbd-aa29-51b78b45a912 created.
Number of files to copy = 4
Number of nodes = 2
Destintion Directory =/tmp/twister-0.9/data/a55705d1-58b3-4dbd-aa29-51b78b45a912
INPUT :Copying /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912/data/wc_data_3.txt to 10.35.23.124:/tmp/twister-0.9/data/a55705d1-58b3-4dbd-aa29-51b78b45a912
[REMOTE Msg] INPUT :Copying /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912/data/wc_data_2.txt to 10.35.23.124:/tmp/twister-0.9/data/a55705d1-58b3-4dbd-aa29-51b78b45a912
[REMOTE Msg] INPUT :Copying /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912/data/wc_data_1.txt to 10.35.23.119:/tmp/twister-0.9/data/a55705d1-58b3-4dbd-aa29-51b78b45a912
[REMOTE Msg] INPUT :Copying /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912/data/wc_data_0.txt to 10.35.23.119:/tmp/twister-0.9/data/a55705d1-58b3-4dbd-aa29-51b78b45a912
[REMOTE Msg] Total Data Copy Time = 11.578 Seconds.
mv /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912/Twister-WordCount-0.9.jar /tmp/twister-0.9/apps/ ;
ssh 10.35.23.124 'mv /tmp/a55705d1-58b3-4dbd-aa29-51b78b45a912/Twister-WordCount-0.9.jar /tmp/twister-0.9//apps/; '
mkdir -p ~/a55705d1-58b3-4dbd-aa29-51b78b45a912
cd /tmp/twister-0.9/bin/; ./create_partition_file.sh a55705d1-58b3-4dbd-aa29-51b78b45a912/ wc_data partition.pf; cp partition.pf /tmp/twister-0.9/samples/wordcount/bin
[REMOTE Msg] Partition file created.
cd /tmp/twister-0.9/samples/wordcount/bin; ./run_wc.sh partition.pf ~/a55705d1-58b3-4dbd-aa29-51b78b45a912/wc.out 4 1
[REMOTE Msg] JobID: word-count-map-reduceacb38099-5b90-11e3-86cc-9390baa7ddd5
[REMOTE Msg] 0 [main] INFO cgl.imr.client.TwisterDriver - Configure Mappers through the partition file, please wait....
[REMOTE Msg] 92 [main] INFO cgl.imr.client.TwisterDriver - Configuring Mappers through the partition file is completed.
[REMOTE Msg] 23802 [main] INFO cgl.imr.client.TwisterDriver - MapReduce computation termintated gracefully.
------------------------------------------------------
Word Count took 26.587 seconds.
------------------------------------------------------
23811 [Thread-0] DEBUG cgl.imr.client.ShutdownHook - Shutting down completed.
/var/www/salsadpi/./salsaDPI_output/a55705d1-58b3-4dbd-aa29-51b78b45a912/output/
Creates the directory : true
[INFO] Download file/dir name .
[INFO] Download file/dir name wc.out
[INFO] Download file/dir name ..
cd /tmp/twister-0.9/bin/; ./stop_twister.sh
[REMOTE Msg] 10.35.23.124
0 [main] DEBUG cgl.imr.script.StopDaemons - Stopping daemon on port number: 12500
7 [main] DEBUG cgl.imr.script.StopDaemons - Finished sending the term signal to daemons in host: 10.35.23.124
[REMOTE Msg] 10.35.23.119
[REMOTE Msg] 0 [main] DEBUG cgl.imr.script.StopDaemons - Stopping daemon on port number: 12501
9 [main] DEBUG cgl.imr.script.StopDaemons - Finished sending the term signal to daemons in host: 10.35.23.119
cd /tmp/apache-activemq-5.4.2/bin/; ./activemq stop
[REMOTE Msg] INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /root/.activemqrc)

INFO: Invoke the following command to create a configuration file
./activemq setup [ /etc/default/activemq | /root/.activemqrc ]

INFO: Using java '/tmp/jdk1.6.0_33//bin/java'
INFO: Waiting at least 30 seconds for regular process termination of pid '4357' :
Java Runtime: Sun Microsystems Inc. 1.6.0_33 /tmp/jdk1.6.0_33/jre
Heap sizes: current=31040k free=30694k max=496960k
JVM args: -Dactivemq.classpath=/tmp/apache-activemq-5.4.2/conf; -Dactivemq.home=/tmp/apache-activemq-5.4.2 -Dactivemq.base=/tmp/apache-activemq-5.4.2
ACTIVEMQ_HOME: /tmp/apache-activemq-5.4.2
ACTIVEMQ_BASE: /tmp/apache-activemq-5.4.2
[REMOTE Msg] Connecting to pid: 4357
Stopping broker: localhost
[REMOTE Msg] .[REMOTE Msg] . FINISHED
[INFO] Hadoop or Twistert startup time plus program execution time = 77.285 Seconds
[INFO] VM termination and Chef deregistering.....
[INFO] VM termination and Chef deregistering
knife openstack server delete d2da8241-8379-4684-8a7f-d5002156da3d -A taklwu -K qSEYyEqxDaTn --openstack-api-endpoint https://s77r.idp.sdsc.futuregrid.org:5000/v2.0/tokens --insecure -P -y
knife node delete os-2561143929060724 -y
knife client delete os-2561143929060724 -y
knife openstack server delete cfb6ff75-047f-45cc-92d1-ef598493e787 -A taklwu -K qSEYyEqxDaTn --openstack-api-endpoint https://s77r.idp.sdsc.futuregrid.org:5000/v2.0/tokens --insecure -P -y
knife node delete os-8412495299375363 -y
knife client delete os-8412495299375363 -y
[INFO] VM termination and Chef deregister time = 16.952 Seconds

[INFO] -------------------------------------------
[INFO] Job a55705d1-58b3-4dbd-aa29-51b78b45a912 has been finished
[INFO] Please check program output at local path /var/www/salsadpi/salsaDPI_output/a55705d1-58b3-4dbd-aa29-51b78b45a912/output/
[INFO] -------------------------------------------

Output Dir = salsaDPI_output/a55705d1-58b3-4dbd-aa29-51b78b45a912/output/

Job Execution Time = 439.01
Job Finished Time = 1386016324141
Job Finished