Hacking just for Fun: using Bookmarklets

So there are a handful of webtools using Bookmarklets for their services. The first i know was del.icio.us for saving a Webpage to your del.icio.us bookmarks. Another famous service is Instapaper (it uses internally read it later pocket, but that is another Story). I have a special service in mind, i want to create using a Bookmarklets, before i start, i played around with the Bookmarklet from Instapaper.

The Magic is just a normale HTML A Tag, in whith some javascript is embedded:

javascript:function%20iprl5(){var%20d=document,z=d.createElement('scr'+'ipt'),b=d.body,l=d.location;try{if(!b)throw(0);d.title='(Saving...)%20'+d.title;z.setAttribute('src',l.protocol+'//www.instapaper.com/j/foobar?u='+encodeURIComponent(l.href)+'&t='+(new%20Date().getTime()));b.appendChild(z);}catch(e){alert('Please%20wait%20until%20the%20page%20has%20loaded.');}}iprl5();void(0)

If we unwrap that script we got

function iprl5(){
    var d=document,
    z=d.createElement('scr'+'ipt'),
    b=d.body,
    l=d.location;
    try{
        if(!b)
            throw(0);
        d.title='(Saving...) '+d.title;
        z.setAttribute('src',l.protocol+'//www.instapaper.com/j/foobar?u='+encodeURIComponent(l.href)+'&t='+(new Date().getTime()));
        b.appendChild(z);
    }catch(e){
        alert('Please wait until the page has loaded.');
    }
}
iprl5();
void(0)

The command basically just creates a script tag in the active DOM-Tree and preloads some Javascript-File. The File is then executed by the Browsers JS Runtime.
I started a litte Demo Project. You find it here on GitHub. It is based on Play! 1.2.4 (Installation Guide here).

ATM there are just two JS Templates. The first one (app/views/Application/bookmarklet.js) just contains the source for the Script-Tag itself. The second (app/views/Application/input.js) will be loaded then after the Javascript behinde that link is called.

Hacking just for Fun: Get Mails from IMAP with Java

I have the feeling, that i might need this someday :-).

Collecting Mails from an IMAP Server with Java is pretty easy:

package de.javastream.imapcollector;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.Properties;
import javax.mail.Folder;
import javax.mail.Message;
import javax.mail.Multipart;
import javax.mail.Part;
import javax.mail.Session;
import javax.mail.Store;
import javax.mail.internet.MimeBodyPart;
import javax.swing.JOptionPane;

public class App {

    public static void getMail(String host, String user, String passwd) throws Exception {
        Session session = Session.getDefaultInstance(new Properties());
        Store store = session.getStore("imap");
        store.connect(host, user, passwd);
        Folder folder = store.getFolder("INBOX");
        folder.open(Folder.READ_ONLY);
        // the funny part is... you can count all the messages before you download them.
        Message[] messages = folder.getMessages();
        System.out.println("there are "+messages.length+" in your INBOX.");
        for (int i = 0; i <; messages.length; i++) {
            Message m = messages[i];
            System.out.println("Nachricht: " + i);
            System.out.println("From: " + m.getFrom()[0]);
            System.out.println("Subject: " + m.getSubject());
            // some Messages are multipart messages (e.g. text with an attachement)
            if (m.getContentType().equals("multipart")) {
                Multipart mp = (Multipart) m.getContent();
                for (int j = 0; j <; mp.getCount(); j++) {
                    Part part = mp.getBodyPart(j);
                    String disposition = part.getDisposition();
                    if (disposition == null) {
                        MimeBodyPart mimePart = (MimeBodyPart) part;
                        if (mimePart.isMimeType("text/plain")) {
                            BufferedReader in = new BufferedReader(
                                    new InputStreamReader(mimePart.getInputStream()));
                            for (String line; (line = in.readLine()) != null;) {
                                System.out.println(line);
                            }
                        }
                    }
                }
            } else {
                System.out.println(m.getContent());
            }
        }
        System.out.println("done :-)");
        folder.close(false);
        store.close();
    }

    public static void main(String[] args) throws Exception {
        // show some simple Dialogs to get server, username and password
        String server = JOptionPane.showInputDialog("enter your Mail Server e.g. mail.example.com");
        String user = JOptionPane.showInputDialog("enter your login e.g. user");
        String password = JOptionPane.showInputDialog("enter your mail password");
        getMail(server, user, password);
    }
}

Becource you will need the java mail lib as a dependency i just created a small maven project to add them.
Just unzip it and run

mvn exec:java

Download Maven Project for ImapCollector

Update:
A Colleague of mine asks for the imports of that Code. I just added them to the sources. You have to include the javamail lib from Sun Oracle as a dependency to run the code.

Hacking just for Fun: Raid5 in Java

I was just curious how easy it might be to write a RAID5 compatible Outputstream in Java? Just a few Lines. For sure it is not the most elegante solution. Especially if you see the nice possibility to integrate one Outputstream within another… so maybe two Raid5s into one Raid0 Stream? (would be RAID50) then. Reading is missing ;-).

/**
 * Raid5Stream
 * 31.03.2012
 * @author Philipp Haussleiter
 *
 */
package de.javastream.jraid;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;

public class Raid5Stream extends OutputStream {

    public final static int PARITY_MARKER = 1;
    public final static int DATA_MARKER = 0;
    private File disks[];
    private FileOutputStream streams[];
    private int mode = 0;
    private final static int MAX_MODE = 2;
    private int[] buffer = new int[2];

    public Raid5Stream(File disks[]) throws IOException {
        super();
        if (disks.length != 3) {
            throw new RuntimeException("we need a disk count of x times 3");
        }
        int i = 0;
        this.disks = new File[disks.length];
        this.streams = new FileOutputStream[disks.length];
        for (File f : disks) {
            if (!f.exists() && !f.createNewFile()) {
                throw new RuntimeException(f.getAbsolutePath() + " does not exists and cannot be created!");
            } else {
                System.out.println("using " + f.getAbsolutePath() + "\n");
                this.disks[i] = f;
                this.streams[i] = new FileOutputStream(f);
            }
            i++;
        }
        writerMarker();
    }

    private void writerMarker() throws IOException {
        for (int i = 0; i < this.streams.length; i++) {
            if (i % 3 == 0) {
                this.streams[i].write(PARITY_MARKER);
            } else {
                this.streams[i].write(DATA_MARKER);
            }
        }
    }

    @Override
    public void write(int i) throws IOException {
        switch (mode) {
            case MAX_MODE:
                this.streams[mode].write(buffer[0] ^ buffer[1]);
                mode = 0;
            default:
                this.buffer[mode] = i;
                this.streams[mode].write(i);
                mode++;
        }
    }
}

Using it is easy as:

...
public class App {

    public static void main(String[] args) throws IOException {
        File raid5_disk1 = new File("raid5.disk1");
        File raid5_disk2 = new File("raid5.disk2");
        File raid5_disk3 = new File("raid5.disk3");
        long count = 0;
        Raid5Stream raid5Stream = new Raid5Stream(new File[]{raid5_disk1, raid5_disk2, raid5_disk3});
        FileInputStream in = new FileInputStream("/dev/random");
        BufferedInputStream bis = new BufferedInputStream(in);
        BufferedOutputStream raid5bos = new BufferedOutputStream(raid5Stream);
        while(count < 1024*1024*2){
            raid5bos.write(bis.read());
            count++;
        }
        raid5bos.close();
        bis.close();
    }
}

Good sources for more reading are the Wikipedia Articles about RAID and XOR.

Run local/remote terminal commands with java using ssh

Sometimes you need to use some CLI-Tools before you want to create or search for a native JNI Binding.
So there is a common way, using the Java Process-Class. But then you might meet two problems i had to face in the past during several problems:

  1. There are (a really small) number of CLI-Tools, that giving no constant output over the STD-OUT (the standard output the Process-Class uses for output)
  2. There is no “elegant” way to implement a process call into your project.

To solve this Problem I created a basic HelperClass, that calls the System over SSH (with the Convenience to work remote and the side-effect to always get STD-compatible output).
I am primarely using it for a fun project SAM i started some months ago to try to create a Management-Tool for Unices and Windows with a very low client-side footprint.

The first Class is used to capsulate the basic SSH Calls:

 
// some imports... 
 public class SystemHelper {
    private Runtime r;
    private String sshPrefix = "";

    // call with $user and 127.0.0.1 to run local command.     
    public SystemHelper(String user, String ip) {
        r = Runtime.getRuntime();
        sshPrefix = " ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no " + user + "@" + ip;
    }

    public void runCommand(String command, ProcessParser pp) {
        try {
            Logger.info("running: " + this.sshPrefix + " " + command);
            Process p = r.exec(this.sshPrefix + " " + command);
            InputStream in = p.getInputStream();
            BufferedInputStream buf = new BufferedInputStream(in);
            InputStreamReader inread = new InputStreamReader(buf);
            BufferedReader bufferedreader = new BufferedReader(inread);
            pp.parse(bufferedreader);
            try {
                if (p.waitFor() != 0) {
                    Logger.info("exit value = " + p.exitValue());
                }
            } catch (InterruptedException e) {
                System.err.println(e);
            } finally {
                // Close the InputStream                 
                bufferedreader.close();
                inread.close();
                buf.close();
                in.close();
            }
        } catch (IOException ex) {
            Logger.error(ex.getLocalizedMessage());
        }
    }
}

ProcessParser is an interface that defines the methode parse, accepting a BufferedReader for parsing the output of the Process. Unfortunately there is no timeout ATM to kill a hanging SSH-Call.

 public interface ProcessParser { 
     public void parse(BufferedReader bufferedreader); 
 } 

The most basic (Output-)Parser looks like this:

 
    public String getPublicSSHKey() {
        SimpeOutputPP so = new SimpeOutputPP();
        String command = "cat ~/.ssh/id_rsa.pub";
        runCommand(command, so);
        if (!so.getOutput().isEmpty()) {
            return so.getOutput().get(0);
        }
        return "";
    }

This returns just the public SSH-Key of the current user. I implemented some more parsers for the output of apt (dpkg), rpm and pacman. You can find them in the github project here.

Counter Update

I just finished my latest improvements to the legacy version of my counter script.
I just added the lookup for ISPs and added dynamic scaling for the axis legend.
I will now going forward to change the whole system to a more sophisticated software, e.g. using a Datawarehouse approach. The first version of the Data-Model is finished.

So from the old version (that was just some plain tables, flowing around)

 

 

 

 

 

 

 

 

 

 

I created a new Model with more Tables, connected to each other. Basically i devided the Model into a basic Fact (Count) and some Dimensions (for every Value):

 

 

 

 

 

 

 

 

 

 

With my current values (around 22k Facts), i have already to limit the facts that are queryed from the Datebase. I wrote a short script to migrate all old Datasets to the new Data-Model.

Running a Rails 3 Application in a Sub-URI Enviroment

Sometimes you need to run your Rails (3) Application on a Sub-URI (e.g. examle.com/prod, example.com/dev). In my current Project there was a Problem with that Configuration and the Rails url-helpers (link_for, url_for, usw.) becourse the app wasn’t aware of the necessary prefix (in our example “/dev”, “/prod”).

There is always the possibility to set the apps basic URL within your app-Context. Together with setting the prefix in a System Enviroment it was possible to achieve the wanted app behaviour.
How to configure a Rails3 App with Passenger in a Sub-URI Configuration please have a look in the Phusion Passenger users guide here or in another blog post here.

The Basic Apache Configuration looks somehow like this:

<VirtualHost *:80>
    ServerName      example.com
    ServerAdmin     admin@example.com
    DocumentRoot    /var/www
    # we added two file hardlinks from our apps public folder to /var/www/dev,  /var/www/prod
        <Directory /var/www/prod>
        ...
        SetEnv RAILS_RELATIVE_URL_ROOT /prod
        ...
    </Directory>
        <Directory /var/www/dev>
        ...
        SetEnv RAILS_RELATIVE_URL_ROOT /dev
        ...
    </Directory>
</VirtualHost>

The last part is to add a Before-Hook to the ApplicationController for updating the apps default_url if RAILS_RELATIVE_URL_ROOT is set:

class ApplicationController < ActionController::Base
  protect_from_forgery

  before_filter :action_set_url_options

  def action_set_url_options
    if ENV['RAILS_RELATIVE_URL_ROOT']
      @host = request.host+":"+request.port.to_s+"/"+ENV['RAILS_RELATIVE_URL_ROOT']
    else
      @host = request.host+":"+request.port.to_s
    end
    Rails.application.routes.default_url_options = { :host => @host}
  end
end

Testing Play! Applications with HTTP Basic Auth

Um eine Play!-Anwendung zu testen, welche HTTP-Basic-Auth verlangt ist es notwendig, die Standard-Datei ApplicationTest.java anzupassen:
Verändert werden muss die Test-Methode testThatIndexPageWorks():

Aus

    @Test
    public void testThatIndexPageWorks() {
        Response response = GET("/");
        assertIsOk(response);
        assertContentType("text/html", response);
        assertCharset(play.Play.defaultWebEncoding, response);
    }

Wird:

    @Test
    public void testThatIndexPageWorks() {
        Request request = FunctionalTest.newRequest();
        request.user = "test";
        request.password = "test";
        request.url = "/";
        Response response = GET(request, "/");
        assertIsOk(response);
        assertContentType("text/html", response);
        assertCharset("utf-8", response);
    }

Wobei User = test und Passwort = test.
Bei allen weiteren Test-Methoden verfährt man analog, oder erstellt sich eine Factory-Methode für den Request.

Play! Applications und der App-Context

Es ist möglich, eine Play!-Anwendung sehr einfach in eine WAR-Struktur zu übertragen und in einen Application-Server zu deployen.
Dies ist recht gut unter Deployment options in der Play!-Dokumentation recht gut erklärt. Was hier allerdings verschwiegen wird ist, wie man den notwendigen Context beim Routing konfiguriert. (Der Context ist der Pfad der Anwendung, welcher standardmäßig vom Namen des WAR-Archivs abgeleitet wird, oder per Descriptor konfiguriert wird) – heißt das Archiv testapp.war ist der Context /testapp.

Innerhalb einer Play!-App muss der Context sowohl in der Config Datei, als auch im Routing definiert werde:

application.conf

...
context=testapp
...

Danach lässt sich in der routes Datei der Context in die Routen konfigurieren:

# Routes
# This file defines all application routes (Higher priority routes first)
# ~~~~

%{ context = play.configuration.getProperty('context', '') }%

# Home page
GET     ${context}                                        Application.index
GET     ${context}/                                       Application.index

# Map static resources from the /app/public folder to the /public path
GET     ${context}/public/                                staticDir:public
GET     ${context}/Users/like/{uid}                       Users.like
GET     ${context}/Users/{uid}/show                       Users.show
...