Hadoop MapReduce Cookbook Errata
Following are the list of errors that have been found on our book so far. We highly appreciate if you can share with us any errors you have found in the book, so that we can add them to the errata.
Page 36 : Chapter 2 - Setting HDFS block size recipe
2. To specify the HDFS block size for specific file paths, you can specify the block size when uploading the file from the command line as follows:>bin/hadoop fs -Ddfs.blocksize=134217728 -put data.in /user/fooThe above command should be as follows.
>bin/hadoop fs -D dfs.block.size=134217728 -put data.in /user/foo
Please accept our apologies about any inconvenience caused due to the above errors.
Page 36 : Chapter 2 - Setting HDFS block size recipe
2. To specify the HDFS block size for specific file paths, you can specify the block size when uploading the file from the command line as follows:>bin/hadoop fs -Ddfs.blocksize=134217728 -put data.in /user/fooThe above command should be as follows.
>bin/hadoop fs -D dfs.block.size=134217728 -put data.in /user/foo
Please accept our apologies about any inconvenience caused due to the above errors.
Categories: Member's blogs
Running MPI.NET Applications with Mono in Ubuntu
Sometime back I played around Mono to get some of our parallel applications running on Ubuntu. These applications were C# based and used MPI.NET.
The following blog post is a great starting point with all the details you'll need. So I'll skip the steps, except to point out couple of caveats you need to consider.
http://blog.biophysengr.net/2011/11/compiling-mpinet-under-ubuntu-oneiric.html
The following blog post is a great starting point with all the details you'll need. So I'll skip the steps, except to point out couple of caveats you need to consider.
http://blog.biophysengr.net/2011/11/compiling-mpinet-under-ubuntu-oneiric.html
- automake versions above 1.9 will give you an error when building MPI.NET. May be you can change the make script to work with them, but I found it easy to just install automake1.9 to solve it.
- You'll need to add /usr/local/lib to your LD_LIBRARY_PATH. Essentially what you need to do is add this path to /etc/ld.so.conf and run ldconfig as root. See documenation from Mono on this at http://www.mono-project.com/DllNotFoundException
- Make sure to do chmod +x to your dlls
Categories: Member's blogs
Repair Mr.Coffee (IDS77) Thermal Fuse
If you haven't bought this product yet, then STOP don't buy it!!
If you, however, have bought it and broke it in the first run then continue.
Mine simply stopped working right on the day I bought it, in fact this is the second Mr. Coffee grinders I bought that day. May be I was trying to grind too much, but as a consumer device I'd expect it to "auto shut off" if it's too hot, rather burn itself.
The good news is, it only burns a thermal fuse, which fairly easy to replace if you get under the hood. Once you remove the grinder cup you can see the following,
Just remove the three plastic hole cover knobs, which you can simply pull out using tweezers or by lifting one side with a sharp point like that of a knife. Once removed these knobs look as shown below.
Get a small flat head screw driver and remove the three screws. Then you can simply take out the motor compartment. You'll need to pop the button panel to find a bolt holding the circuit board. You'll be able to figure out hopefully.
Once the motor is out you can see the thermal fuse wrapped inside the yellow tape covering the motor winding as shown. I cut the pins of the fuse and it's shown left to the motor.
Now it's time to find a replacement. The original fuse that's there in this one comes from China and here's a link I could find on it. A close up picture is given below.
I found several options in ebay, but either they had low ampere rating or too high functioning temperature. Also, it'd take more than a week to arrive. In the end, I decided to go with an alternative one from RadioShack, which in fact is cheaper (~$1.40) than options from ebay.
If you need some instructions on how to solder these look at this. This one has a high ampere rating, but from what I read having a higher ampere rating than the one used does no harm. It's the temperature that's important.
Once soldered, the unit is alive again. This time I will not grind coffee continuously though :)
If you, however, have bought it and broke it in the first run then continue.
Mine simply stopped working right on the day I bought it, in fact this is the second Mr. Coffee grinders I bought that day. May be I was trying to grind too much, but as a consumer device I'd expect it to "auto shut off" if it's too hot, rather burn itself.
The good news is, it only burns a thermal fuse, which fairly easy to replace if you get under the hood. Once you remove the grinder cup you can see the following,
Just remove the three plastic hole cover knobs, which you can simply pull out using tweezers or by lifting one side with a sharp point like that of a knife. Once removed these knobs look as shown below.
Get a small flat head screw driver and remove the three screws. Then you can simply take out the motor compartment. You'll need to pop the button panel to find a bolt holding the circuit board. You'll be able to figure out hopefully.
Once the motor is out you can see the thermal fuse wrapped inside the yellow tape covering the motor winding as shown. I cut the pins of the fuse and it's shown left to the motor.
Now it's time to find a replacement. The original fuse that's there in this one comes from China and here's a link I could find on it. A close up picture is given below.
I found several options in ebay, but either they had low ampere rating or too high functioning temperature. Also, it'd take more than a week to arrive. In the end, I decided to go with an alternative one from RadioShack, which in fact is cheaper (~$1.40) than options from ebay.
If you need some instructions on how to solder these look at this. This one has a high ampere rating, but from what I read having a higher ampere rating than the one used does no harm. It's the temperature that's important.
Once soldered, the unit is alive again. This time I will not grind coffee continuously though :)
Categories: Member's blogs
Home Made Meditation Benches
My friend, Lahiru, suggested to make few meditation benches as a donation to Indiana Buddhist Temple few weeks back. I had no idea what a meditation bench was, but luckily he had a basic one with him. After few modifications to the design, here’s what we ended up with.
If you like to make one at home too, here are the details.
If you like to make one at home too, here are the details.
- Materials
- Wood – 1 x 8 (i.e. 3/4” x 7+1/4”) pine boards http://www.menards.com/main/building-materials/lumber-boards/boards/select-boards/1-x-8-x-8-select-pine-board/p-1934090-c-13118.htm
- Hinges – 3” x 3” door hinges, 1/4” round corner http://www.menards.com/main/tools-hardware/builders-hardware/door-window/residential-hinges/3x3-door-hinge/p-1475757-c-9687.htm
- Used 1/2” #10 Phillips flat head screws http://www.menards.com/main/building-materials/drywall/screws/wood/10-x-1-2-phillips-flat-wood-100pcs-pkg/p-1638963-c-8940.htm
- Leg sliders – Slipstick 3/4” Slider Foot http://www.amazon.com/Slipstick-CB190-4-Inch-Slider-16-Pack/dp/B000G9WK44/ref=sr_1_2?ie=UTF8&id=1365832735&sr=8-2&keywords=slipstick+slider
- Felt pads (not shown in picture) - http://www.amazon.com/Waxman-Self-Stick-Round-Brown-8-Inch/dp/B000VYN7CS/ref=sr_1_4?s=hi&ie=UTF8&qid=1365832847&sr=1-4&keywords=felt+pads
- De-waxed Shellac – Bulls Eye Seal Coat http://www.menards.com/main/interior-wood-care/specialty-wood-finishes/bulls-eye-seal-coat-universal-sealer/p-1963335-c-13129.htm
- Steps
- Cutting wood – a hand saw is enough actually, but I used a jig saw to make it bit faster and to cut the semi circles on legs. Dimensions are given below For legs, cut from the dashed lines, which are 1/2” from each edge. The 5.25 and 6.75 are lengths along the outside edges though. Alternatively, you could determine the angle from these two values and then calculate the the lengths for the actual cut piece. The jig I made to draw semi circles and flush legs to the size.
- Fixing hinges – I used a router to make the mortises so that legs would close flush with the main board.You could omit this if this is not a concern.
- NOTE. I have flipped one side of the hinge (by removing its pin) to get them fixed as shown above. The reason was that the hinges I used won’t close flushed. See diagrams below.
- I didn’t like the protruding wedge shape (right most image above), hence the flipping (see here to know how to remove hinge pin) and mortising.
- Now, with some custom mortise routing I could get the legs to close flushed (above right). I made few jigs to make it easy to batch these as below.
- Finishing – 120 grit sanding followed by de-waxed shellac with 220 sanding between coats.
About the semi circle, I just used one of my wife’s planting pots to draw it. So I didn’t measure its diameter. You can use whatever shape or size you like.
Categories: Member's blogs
Notes on Windows PowerShell
- Is bit slow and creepy ;) but better than command line
- Running as administrator helps you overcome many of the access denied situations
- PowerShell script files are simple text files with .ps1 extension
- To run a script file,
- Open a PowerShell instance (preferably as administrator)
- Type the path to .ps1 file and hit enter
- If you get an error due to execution-policy is restricted, try set-executionpolicy remotesigned
- The above will ask for your permission. To avoid that and force use set-executionpolicy remotesigned -force
- If you want this to be done across a Windows HPC cluster using clusrun command$nodes="node1,node2,..,noden"
clusrun /nodes:$nodes powershell set-executionpolicy remotesigned -force
- To run a script file,
- To start a new process through a script - http://technet.microsoft.com/en-us/library/hh849848.aspxstart-process command "options and args"
- If you want it to be on the same window and wait for it to complete usestart-process command "options and args" -NoNewWindow –Wait
- Invoke a command on a remote machine - http://technet.microsoft.com/en-us/library/hh849719.aspx Invoke-Command -ComputerName $node -ScriptBlock {Start-Service MyService}
- Local variables $x=10
- Command line parameters $param0=$args[0]
- Environment (User/System) variables $javaBin=$env:JAVA_HOME + "\bin"
- A better alternative – number 2 indicates system variable. Use number 1 for user variable $javaHome=[Environment]::GetEnvironmentVariable("JAVA_HOME",2)
- Similarly to set a value – again number 2 is for system level. [Environment]::SetEnvironmentVariable("Path","$tmp",2)
Handling variables
$path="C:\users\dd"
$param1=$args[1]
- String operations (few)
- Concatenation $name="John"
- Substring $name.Substring(1) // "ohn"
- StartsWith $name.StartsWith("hello") // true
- IndexOf $name.IndexOf("ohn") // 1
- Length $name.Length // 4
- Join paths $javaHome=$env:JAVA_HOME // e.g. "C:\Program Files\Java\jdk1.7.0_10"
- Loops – a good tutorial at http://www.powershellpro.com/powershell-tutorial-introduction/logic-using-loops/
- For loop
- -le is <=
- -lt is <
- -gt is >
- -eq is == in usual programming language notation
- Foreach loop
- .NET runtime directory $([System.Runtime.InteropServices.RuntimeEnvironment]::GetRuntimeDirectory())
$x="hello " + $name + "! How are you ?"
$name.Substring(1,2) // "oh"
$javaBinString=join-path -path $javaHome "bin" // e.g. "C:\Program Files\Java\jdk1.7.0_10\bin"
//body
}
foreach ($jar in $jars) {
$jar=$jar.DirectoryName + "\" + $jar.Name
$cp=$jar+";"+$cp
}
Categories: Member's blogs
Study of Biological Sequence Structure: Clustering and Visualization
My presentation in our SALSA HPC weekly meeting.
http://salsahpc.blogspot.com/2013/02/study-of-biological-sequence-structure.html
http://salsahpc.blogspot.com/2013/02/study-of-biological-sequence-structure.html
Categories: Member's blogs
