Home

User login

Hadoop MapReduce Cookbook Errata

Thilina's Blog - Fri, 2013-05-17 20:01
Following are the list of errors that have been found on our book so far. We highly appreciate if you can share with us any errors you have found in the book, so that we can add them to the errata.

Page 36 : Chapter 2 - Setting HDFS block size recipe
2. To specify the HDFS block size for specific file paths, you can specify the block size when uploading the file from the command line as follows:>bin/hadoop fs -Ddfs.blocksize=134217728 -put data.in /user/fooThe above command  should be as follows.
>bin/hadoop fs -D dfs.block.size=134217728 -put data.in /user/foo

Please accept our apologies about any inconvenience caused due to the above errors.
Categories: Member's blogs

Running MPI.NET Applications with Mono in Ubuntu

Saliya's Blog - Thu, 2013-04-18 02:45
Sometime back I played around Mono to get some of our parallel applications running on Ubuntu. These applications were C# based and used MPI.NET.

The following blog post is a great starting point with all the details you'll need. So I'll skip the steps, except to point out couple of caveats you need to consider.

http://blog.biophysengr.net/2011/11/compiling-mpinet-under-ubuntu-oneiric.html


  1. automake versions above 1.9 will give you an error when building MPI.NET. May be you can change the make script to work with them, but I found it easy to just install automake1.9 to solve it.
  2. You'll need to add /usr/local/lib to your LD_LIBRARY_PATH. Essentially what you need to do is add this path to /etc/ld.so.conf and run ldconfig as root. See documenation from Mono on this at http://www.mono-project.com/DllNotFoundException
  3. Make sure to do chmod +x to your dlls
Categories: Member's blogs

Repair Mr.Coffee (IDS77) Thermal Fuse

Saliya's Blog - Wed, 2013-04-17 12:24
If you haven't bought this product yet, then STOP don't buy it!!

If you, however, have bought it and broke it in the first run then continue.

Mine simply stopped working right on the day I bought it, in fact this is the second Mr. Coffee grinders I bought that day. May be I was trying to grind too much, but as a consumer device I'd expect it to "auto shut off" if it's too hot, rather burn itself.

The good news is, it only burns a thermal fuse, which fairly easy to replace if you get under the hood. Once you remove the grinder cup you can see the following,

Just remove the three plastic hole cover knobs, which you can simply pull out using tweezers or by lifting one side with a sharp point like that of a knife. Once removed these knobs look as shown below.
Get a small flat head screw driver and remove the three screws. Then you can simply take out the motor compartment. You'll need to pop the button panel to find a bolt holding the circuit board. You'll be able to figure out hopefully.
Once the motor is out you can see the thermal fuse wrapped inside the yellow tape covering the motor winding as shown. I cut the pins of the fuse and it's shown left to the motor. 
Now it's time to find a replacement. The original fuse that's there in this one comes from China and here's a link I could find on it. A close up picture is given below.
I found several options in ebay, but either they had low ampere rating or too high functioning temperature. Also, it'd take more than a week to arrive. In the end, I decided to go with an alternative one from RadioShack, which in fact is cheaper (~$1.40) than options from ebay.
If you need some instructions on how to solder these look at this. This one has a high ampere rating, but from what I read having a higher ampere rating than the one used does no harm. It's the temperature that's important.
Once soldered, the unit is alive again. This time I will not grind coffee continuously though :)




Categories: Member's blogs

Home Made Meditation Benches

Saliya's Blog - Sat, 2013-04-13 04:32
My friend, Lahiru, suggested  to make few meditation benches as a donation to Indiana Buddhist Temple few weeks back. I had no idea what a meditation bench was, but luckily he had a basic one with him. After few modifications to the design, here’s what we ended up with.

If you like to make one at home too, here are the details.
I think this covers most of the stuff you need to complete this project and any questions are welcome in comments. Happy bench making !!
Categories: Member's blogs

Notes on Windows PowerShell

Saliya's Blog - Tue, 2013-02-19 18:44
  • Is bit slow and creepy ;) but better than command line
  • Running as administrator helps you overcome many of the access denied situations
  • PowerShell script files are simple text files with .ps1 extension
    • To run a script file,
      • Open a PowerShell instance (preferably as administrator) 
      • Type the path to .ps1 file and hit enter
    • If you get an error due to execution-policy is restricted, try set-executionpolicy remotesigned
    • The above will ask for your permission. To avoid that and force use set-executionpolicy remotesigned -force
    • If you want this to be done across a Windows HPC cluster using clusrun command$nodes="node1,node2,..,noden"
      clusrun /nodes:$nodes powershell set-executionpolicy remotesigned -force
  • To start a new process through a script - http://technet.microsoft.com/en-us/library/hh849848.aspxstart-process command "options and args"

    • If you want it to be on the same window and wait for it to complete usestart-process command "options and args" -NoNewWindow –Wait
  • Invoke a command on a remote machine - http://technet.microsoft.com/en-us/library/hh849719.aspx
  • Invoke-Command -ComputerName $node -ScriptBlock {Start-Service MyService}
    Handling variables

    • Local variables
    • $x=10
      $path="C:\users\dd"
    • Command line parameters
    • $param0=$args[0]
      $param1=$args[1]
    • Environment (User/System) variables
    • $javaBin=$env:JAVA_HOME + "\bin"

      • A better alternative – number 2 indicates system variable. Use number 1 for user variable
      • $javaHome=[Environment]::GetEnvironmentVariable("JAVA_HOME",2)
      • Similarly to set a value – again number 2 is for system level.
      • [Environment]::SetEnvironmentVariable("Path","$tmp",2)

  • String operations (few)

    • Concatenation
    • $name="John"
      $x="hello " + $name + "! How are you ?"
    • Substring
    • $name.Substring(1) // "ohn"
      $name.Substring(1,2) // "oh"
    • StartsWith
    • $name.StartsWith("hello") // true
    • IndexOf
    • $name.IndexOf("ohn") // 1
    • Length
    • $name.Length // 4
    • Join paths
    • $javaHome=$env:JAVA_HOME // e.g. "C:\Program Files\Java\jdk1.7.0_10"
      $javaBinString=join-path -path $javaHome "bin" // e.g. "C:\Program Files\Java\jdk1.7.0_10\bin"

  • Loops – a good tutorial at http://www.powershellpro.com/powershell-tutorial-introduction/logic-using-loops/

    • For loop
    for($i=1;$i -le 10; $i++) {
    //body
    }
      • -le is <=
      • -lt is <
      • -gt is >
      • -eq is == in usual programming language notation

    • Foreach loop
    $jars=ls ($env:TWISTER_HOME + "\lib") *.jar
    foreach ($jar in $jars) {
    $jar=$jar.DirectoryName + "\" + $jar.Name
    $cp=$jar+";"+$cp
    }
  • .NET runtime directory
  • $([System.Runtime.InteropServices.RuntimeEnvironment]::GetRuntimeDirectory())
This is pretty much I found useful for my work. Feel free to suggest any.
Categories: Member's blogs

Wed, 1969-12-31 20:00