FutureGrid Eucalyptus preparetion

 

SALSA Group
PTI Indiana University
July 13th 2012

1. Introduction

Before we schedule any SalsaDPI job run on FutureGrid India academic cloud (Eucalyptus), we need to go through several steps to setup our Eucalyptus correctly.

2. Setup FutureGrid Eucalyptus environment

First, every FutureGrid user should have generated their own ssh public/private keypair and have uploaded the public key to FutureGrid portal. Now, we use ssh private to login to FutureGrid India headnode (india.futuregrid.org) and setup the FutureGrid Eucalyptus environment.

Convert Windows generated ssh private key to OPENSSH format (only for Windows user)

NOTED THAT if you generated your FutureGrid headnode ssh login key under Windows, you will need to convert it to a OPENSSH format private. Other Linux/MAC user dont need this key conversion.

We are using pageant.exe http://the.earth.li/~sgtatham/putty/latest/x86/puttygen.exe in the following step.

Copy the FutureGrid Headnode ssh private file to the VirtualBox Shared Folder

You will copy the FutureGrid Headnode ssh private file from the Host OS machine with using the Shared Folder function support by the VirtuialBox Applications (refer to the Install and use the prepackaged Virtualbox VM Image page).

root@ubuntu:~# ls -l /media/shared/

# assuming you have copied FutureGrid Headnode ssh private file to the shared folder
-rw------- 1 johnny johnny 983 2012-07-14 18:04 fg_private_key.pem

root@ubuntu:~# cp /media/shared/fg_private_key.pem ~/

Login to FutureGrid India headnode i136.

Assuming the project number is fg241, and the login username is johnny, and your FutueGrid ssh login private key is "~/fg_private_key.pem" (NOTE THAT replace these to your own username and private key file name):

# login to FutureGrid India Headnode i136 
root@ubuntu:~$ ssh -i ~/fg_private_key.pem johnny@india.futuregrid.org

torque/2.5.5 version 2.5.5 loaded
moab version 5.4.0 loaded
[johnny@i136 ~]$

Load and Use euca2ools, create a security ssh keyparis, security groups

Then, we load euca2ools and the Eucalyptus environment file eucarc from the project directory ~/.futuregrid/eucalyptus/fg300/eucarc

[johnny@i136 ~]$ module load euca2ools
euca2ools version 2.0.2 loaded
[johnny@i136 ~]$ cd .futuregrid/eucalyptus/fg241/
[johnny@i136 fg241]$ ls -l

-rw-r--r-- 1 johnny users 5102 May 14 22:58 euca3-johnny-india-fg241.zip

[johnny@i136 fg241]$ unzip euca3-johnny-india-fg241.zip
[johnny@i136 fg241]$ source ~/.futuregrid/eucalyptus/fg241/eucarc

# Check if the euca2ool is working

[johnny@i136 fg241]$ euca-describe-availability-zones
AVAILABILITYZONE euca3india 149.165.146.135 arn:euca:eucalyptus:euca3india:cluster:euca3indiaCC/

Afterward, we setup the VM ssh keypair that is used to login VM. Also, create a euca secuirty group to allow ssh connection from internet. NOTED THAT, you must change "userkey" to your username, OTHERWISE it will cause errors which fails to login to VM with the generated key:

# create a VM ssh keypairs, e.g. userkey and userkey.pem, you have to replace userkey to your username
# otherwise, it will cause errors which fails to login to VM with the generated key.

[johnny@i136 ~]$ cd ~
[johnny@i136 ~]$ mkdir fg241
[johnny@i136 ~]$ cd fg241
[johnny@i136 fg241]$ euca-add-keypair userkey > userkey.pem
[johnny@i136 fg241]$ chmod 0600 userkey.pem


# check if the keypair has been setup correctly

[johnny@i136 fg241]$ euca-describe-keypairs
KEYPAIR userkey 82:c3:6f:81:31:0d:88:1b:6b:e0:58:21:42:c6:7d:34:ef:2e:ac:ca
[johnny@i136 fg241]$ ls -l johnny.pem
-rw------- 1 johnny users 1751 Jul 14 17:54 userkey.pem


# add euca security groups to allow ssh connection

[johnny@i136 fg241]$ euca-authorize -P tcp -p 22 -s 0.0.0.0/0 default
[johnny@i136 fg241]$ euca-authorize -P icmp -t -1:-1 -s 0.0.0.0/0 default
[johnny@i136 fg241]$ euca-describe-groups
GROUP 150119463462 default default group
PERMISSION 150119463462 default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0
PERMISSION 150119463462 default ALLOWS icmp -1 -1 FROM CIDR 0.0.0.0/0

Then you will need to logout from FutureGrid India Headnode:

[johnny@i136 fg241]$ exit


root@ubuntu:~$

3. Copy eucarc file and ssh VM private key file

Once we have above setup, we will use sftp to copy two files, eucarc and ssh private key (e.g. johnny), back to local working desktop/laptop.

NOTE that you will need to replace "~/fg_private_key.pem" to the path of your own FutureGrid ssh private key inside the VirtualBox machine.

root@ubuntu:~$ sftp -o IdentityFile=~/fg_private_key.pem johnny@india.futuregrid.org
sftp> get .futuregrid/eucalyptus/fg241/eucarc
sftp> get fg241/userkey.pem
sftp> exit
root@ubuntu:~$ ls -l
-rw------- 1 johnny johnny 983 2012-07-14 18:04 eucarc
-rw------- 1 johnny johnny 1751 2012-07-14 18:04 userkey.pem

Next, you could see the Hands-on 2 Run User-defined Hadoop/Twister Applications on FutureGrid Eucalyptus

FAQ

Please see FAQ if you have any problem.

Reference