Enter Job ID:

 

Job Found

Program output is shown as following:

1386015345
eucarc = default
eucaImageEmi = emi-A8F63C29
instanceType = c1.medium
pubkey = default
numOfNodes = 1
applicationType = TwisterWordCountWithUrlSupport
Job start Time = 1386015345907

[INFO] Getting cloud mode configuration parameters
[INFO] getting chef cloud parameters.
/home/taklwu/stephen.pem
[INFO] Amount of running instances(s): 1
Job ID = c3afa2af-9040-4fa7-9462-689ad63c7dbe
[INFO] Cloud mode start VM(s) with using chef
[INFO] configured with recipe: recipe[twisterCloud]
1 VM(s) are starting.................................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
..............................
...... All VMs have been started
[INFO] Node 0 information i-F5DE4163
[INFO] Cloud VM startup plus Chef preparation time = 281.064 Seconds
[INFO] Copying ssh information
[INFO] Setting VM Environment.
[INFO] Connecting to 149.165.158.217
sudo hostname i-F5DE4163
[INFO] uploading data from /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe_hosts to VM's/etc/hosts
[INFO] uploading data from /home/taklwu/stephen.pem to VM's/root/.ssh/id_rsa
chmod 600 /root/.ssh/id_rsa
[INFO] uploading data from salsaDPI.jar to VM's/root/salsaDPI.jar
mkdir -p /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe
[INFO] Uploading Binary and Input
[INFO] http://salsahpc.indiana.edu/salsadpi/apps/Twister-WordCount-0.9.jar http://salsahpc.indiana.edu/salsadpi/input/twisterWordCountInput.tar.gz
[INFO] downloading data from web to VMs
[INFO] downloading data from web to VMs
[INFO] Cloud Program input preparation time = 9.365 Seconds
End of Copying ssh information
Application Type = TWISTER
Running Twister application, Type: TWISTER
prefix wc_data
get TWISTER_HOME and JAVA_HOME
[REMOTE Msg] /root/twister-0.9/
/tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe_ips /root/twister-0.9//bin/nodes
[INFO] uploading data from /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe_ips to VM's/root/twister-0.9///bin/nodes
cd /root/twister-0.9//bin/; ./TwisterPowerMakeUp.sh
[REMOTE Msg] use StandAlone Setup.
no special processing to nodes
ActiveMQ uri=failover:(tcp://10.154.71.119:61616)
nodes_file=/root/twister-0.9/bin/nodes
daemons_per_node=1
workers_per_daemon=1
app_dir=/root/twister-0.9/apps
10.154.71.119:/root/twister-0.9/data created.
data_dir=/root/twister-0.9/data
[REMOTE Msg] Change max memory to 1374 MB
no configuration replication required
Auto configuration is done.
cd /root/twister-0.9//bin/; echo 'uri = failover\:(tcp\://10.154.71.119\:61616)' > amq.properties
[REMOTE Msg] /root/apache-activemq-5.4.2
cd /root/apache-activemq-5.4.2/bin/; ./activemq start
[REMOTE Msg] INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /root/.activemqrc)

INFO: Invoke the following command to create a configuration file
./activemq setup [ /etc/default/activemq | /root/.activemqrc ]

INFO: Using java '/tmp/jdk1.6.0_33//bin/java'
INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
INFO: pidfile created : '/root/apache-activemq-5.4.2/data/activemq.pid' (pid '4362')
cd /root/twister-0.9//bin/; nohup ./start_twister.sh > nohup.txt &
[INFO] ssh exeAndQuit connection run;
[INFO] Twister Started
cd /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe;tar -zxvf twisterWordCountInput.tar.gz; cd /root/twister-0.9//bin ; ./twister.sh mkdir c3afa2af-9040-4fa7-9462-689ad63c7dbe; ./twister.sh put /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe/data c3afa2af-9040-4fa7-9462-689ad63c7dbe wc_data 1 1;
[REMOTE Msg] data/
data/wc_data_2.txt
[REMOTE Msg] data/wc_data_3.txt
[REMOTE Msg] data/wc_data_0.txt
[REMOTE Msg] data/wc_data_1.txt
[REMOTE Msg] 10.154.71.119:/root/twister-0.9/data/c3afa2af-9040-4fa7-9462-689ad63c7dbe created.
Number of files to copy = 4
Number of nodes = 1
Destintion Directory =/root/twister-0.9/data/c3afa2af-9040-4fa7-9462-689ad63c7dbe
INPUT :Copying /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe/data/wc_data_2.txt to 10.154.71.119:/root/twister-0.9/data/c3afa2af-9040-4fa7-9462-689ad63c7dbe
[REMOTE Msg] INPUT :Copying /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe/data/wc_data_3.txt to 10.154.71.119:/root/twister-0.9/data/c3afa2af-9040-4fa7-9462-689ad63c7dbe
[REMOTE Msg] INPUT :Copying /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe/data/wc_data_1.txt to 10.154.71.119:/root/twister-0.9/data/c3afa2af-9040-4fa7-9462-689ad63c7dbe
[REMOTE Msg] INPUT :Copying /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe/data/wc_data_0.txt to 10.154.71.119:/root/twister-0.9/data/c3afa2af-9040-4fa7-9462-689ad63c7dbe
[REMOTE Msg] Total Data Copy Time = 12.324 Seconds.
mv /tmp/c3afa2af-9040-4fa7-9462-689ad63c7dbe/Twister-WordCount-0.9.jar /root/twister-0.9//apps/ ;
mkdir -p ~/c3afa2af-9040-4fa7-9462-689ad63c7dbe
cd /root/twister-0.9//bin/; ./create_partition_file.sh c3afa2af-9040-4fa7-9462-689ad63c7dbe/ wc_data partition.pf; cp partition.pf /root/twister-0.9//samples/wordcount/bin
[REMOTE Msg] Partition file created.
cd /root/twister-0.9//samples/wordcount/bin; ./run_wc.sh partition.pf ~/c3afa2af-9040-4fa7-9462-689ad63c7dbe/wc.out 4 1
[REMOTE Msg] JobID: word-count-map-reduce43a822eb-5b8f-11e3-85fe-1bb593fb73bb
[REMOTE Msg] 0 [main] INFO cgl.imr.client.TwisterDriver - Configure Mappers through the partition file, please wait....
78 [main] INFO cgl.imr.client.TwisterDriver - Configuring Mappers through the partition file is completed.
[REMOTE Msg] 48396 [main] INFO cgl.imr.client.TwisterDriver - MapReduce computation termintated gracefully.
------------------------------------------------------
Word Count took 49.738 seconds.
------------------------------------------------------
48403 [Thread-0] DEBUG cgl.imr.client.ShutdownHook - Shutting down completed.
/var/www/salsadpi/./salsaDPI_output/c3afa2af-9040-4fa7-9462-689ad63c7dbe/output/
Creates the directory : true
[INFO] Download file/dir name ..
[INFO] Download file/dir name wc.out
[INFO] Download file/dir name .
cd /root/twister-0.9//bin/; ./stop_twister.sh
[REMOTE Msg] 10.154.71.119
0 [main] DEBUG cgl.imr.script.StopDaemons - Stopping daemon on port number: 12500
5 [main] DEBUG cgl.imr.script.StopDaemons - Finished sending the term signal to daemons in host: 10.154.71.119
cd /root/apache-activemq-5.4.2/bin/; ./activemq stop
[REMOTE Msg] INFO: Using default configuration
(you can configure options in one of these file: /etc/default/activemq /root/.activemqrc)

INFO: Invoke the following command to create a configuration file
./activemq setup [ /etc/default/activemq | /root/.activemqrc ]

INFO: Using java '/tmp/jdk1.6.0_33//bin/java'
INFO: Waiting at least 30 seconds for regular process termination of pid '4362' :
Java Runtime: Sun Microsystems Inc. 1.6.0_33 /tmp/jdk1.6.0_33/jre
Heap sizes: current=31936k free=31582k max=510784k
JVM args: -Dactivemq.classpath=/root/apache-activemq-5.4.2/conf; -Dactivemq.home=/root/apache-activemq-5.4.2 -Dactivemq.base=/root/apache-activemq-5.4.2
ACTIVEMQ_HOME: /root/apache-activemq-5.4.2
ACTIVEMQ_BASE: /root/apache-activemq-5.4.2
Connecting to pid: 4362
[REMOTE Msg] Stopping broker: localhost
[REMOTE Msg] .[REMOTE Msg] . FINISHED
[INFO] Hadoop or Twistert startup time plus program execution time = 82.547 Seconds
[INFO] VM termination and Chef deregistering.....
[INFO] VM termination and Chef deregistering
[INFO] VM termination and Chef deregister time = 6.584 Seconds

[INFO] -------------------------------------------
[INFO] Job c3afa2af-9040-4fa7-9462-689ad63c7dbe has been finished
[INFO] Please check program output at local path /var/www/salsadpi/salsaDPI_output/c3afa2af-9040-4fa7-9462-689ad63c7dbe/output/
[INFO] -------------------------------------------

Output Dir = salsaDPI_output/c3afa2af-9040-4fa7-9462-689ad63c7dbe/output/

Job Execution Time = 381.695
Job Finished Time = 1386015727602
Job Finished