Author: Philipp

iPhoto crashes on Lion 10.7 after the update to 9.3

PhilippLeave a comment

After the latest iPhoto update to Version 9.3 i experienced a reproducable crash of iPhoto.

This was the error i found in the Crah-Report.

Library not loaded: /System/Library/PrivateFrameworks/CoreMediaIOServices.framework/Versions/A/CoreMediaIOServices

After some time i found some solution in the apple support forums.
You just need to restore the folder /System/Library/PrivateFrameworks/CoreMediaIOServices.framework from your latest backup.

I really wounder what happened to apples QA. I mean thats the second update (after the thunderbolt 1.2 update), that affect other software or the system stability itself.

Fixing Redirects of a Play! App behind an Apache2 SSL Proxy

PhilippLeave a comment

So you just finished your first Play! App. You want to run that thing behind an Apache2 as a HTTPS Proxy, because you do not want, that your User-Credentials are read as clear text.

So a very basic Apache Configuration looks like this:

    <IfModule mod_ssl.c>

        Listen 443
        SSLRandomSeed startup builtin
        SSLRandomSeed connect builtin

        <VirtualHost _default_:443>

            SSLEngine on
            ServerName example.com
            ServerAdmin admin@example.com
            ErrorLog /var/log/apache2/ssl_error_log

            SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
            SSLCertificateFile /etc/apache2/ssl/example/newcert.pem
            SSLCertificateKeyFile /etc/apache2/ssl/example/webserver.nopass.key
            SSLCACertificateFile /etc/apache2/ssl/demoCA/cacert.pem
            SSLCARevocationPath /etc/apache2/ssl/certs/demoCA/crl

            ProxyPass               /play            http://127.0.0.1:9000/play
            ProxyPassReverse        /play            http://127.0.0.1:9000/play

        </VirtualHost>

    </IfModule>

I did already explained how to run a Play! Application within a Application Context. Here our Context is just “play”, but you can set it to something else. You can also change and add seperate instances with different ports (over 9000!!!).

You should alter two settings in your conf/application.conf

    # you need to add this
    context=/play

    # you need to uncomment this to prevent Play! from serving aside your Apache2 Proxy
    http.address=127.0.0.1

   # you may uncomment and change this port number for chaning it
   # http.port=9000

Time for a Test Run. At a first glance it seems to work pretty nice. But as soon as you want to use the nifty routing redirects from Play!, the whole system breaks, because Play! still things, it runs in plain http on port 9000. To solve this, you need to change to things:

  1. Make Apache2 to send a specific header, that the Request was send through a Proxy
  2. Make Play! to fix the redirect to the correct URL

The first part is pretty easy. Just add

    RequestHeader set X_FORWARDED_PROTO 'https'

within your VirtualHost Tag – you may need to enable the headers module first to make that work.

The second part is a little bit more difficult. You have to add a before filter to your application controller:

    public class Application extends Controller {

        @Before
        private static void checkSSL() {
            if (request.headers.get("x_forwarded_proto") != null
                    && "https".equals(request.headers.get("x_forwarded_proto").value())) {
                request.secure = true;
                request.port = 443;
            }
            if (request.headers.get("x-forwarded-server") != null) {
                request.domain = request.headers.get("x-forwarded-server").value();
            }
        }
        ...
    }

You can find the sources on GitHub.
B.t.w. the same problem also might appears in Rails Apps, i might write about this later on.

Hacking just for Fun: using Bookmarklets

Philipp1 Comment

So there are a handful of webtools using Bookmarklets for their services. The first i know was del.icio.us for saving a Webpage to your del.icio.us bookmarks. Another famous service is Instapaper (it uses internally read it later pocket, but that is another Story). I have a special service in mind, i want to create using a Bookmarklets, before i start, i played around with the Bookmarklet from Instapaper.

The Magic is just a normale HTML A Tag, in whith some javascript is embedded:

javascript:function%20iprl5(){var%20d=document,z=d.createElement('scr'+'ipt'),b=d.body,l=d.location;try{if(!b)throw(0);d.title='(Saving...)%20'+d.title;z.setAttribute('src',l.protocol+'//www.instapaper.com/j/foobar?u='+encodeURIComponent(l.href)+'&t='+(new%20Date().getTime()));b.appendChild(z);}catch(e){alert('Please%20wait%20until%20the%20page%20has%20loaded.');}}iprl5();void(0)

If we unwrap that script we got

function iprl5(){
    var d=document,
    z=d.createElement('scr'+'ipt'),
    b=d.body,
    l=d.location;
    try{
        if(!b)
            throw(0);
        d.title='(Saving...) '+d.title;
        z.setAttribute('src',l.protocol+'//www.instapaper.com/j/foobar?u='+encodeURIComponent(l.href)+'&t='+(new Date().getTime()));
        b.appendChild(z);
    }catch(e){
        alert('Please wait until the page has loaded.');
    }
}
iprl5();
void(0)

The command basically just creates a script tag in the active DOM-Tree and preloads some Javascript-File. The File is then executed by the Browsers JS Runtime.
I started a litte Demo Project. You find it here on GitHub. It is based on Play! 1.2.4 (Installation Guide here).

ATM there are just two JS Templates. The first one (app/views/Application/bookmarklet.js) just contains the source for the Script-Tag itself. The second (app/views/Application/input.js) will be loaded then after the Javascript behinde that link is called.

Hacking just for Fun: Get Mails from IMAP with Java

PhilippLeave a comment

I have the feeling, that i might need this someday :-).

Collecting Mails from an IMAP Server with Java is pretty easy:

package de.javastream.imapcollector;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.Properties;
import javax.mail.Folder;
import javax.mail.Message;
import javax.mail.Multipart;
import javax.mail.Part;
import javax.mail.Session;
import javax.mail.Store;
import javax.mail.internet.MimeBodyPart;
import javax.swing.JOptionPane;

public class App {

    public static void getMail(String host, String user, String passwd) throws Exception {
        Session session = Session.getDefaultInstance(new Properties());
        Store store = session.getStore("imap");
        store.connect(host, user, passwd);
        Folder folder = store.getFolder("INBOX");
        folder.open(Folder.READ_ONLY);
        // the funny part is... you can count all the messages before you download them.
        Message[] messages = folder.getMessages();
        System.out.println("there are "+messages.length+" in your INBOX.");
        for (int i = 0; i <; messages.length; i++) {
            Message m = messages[i];
            System.out.println("Nachricht: " + i);
            System.out.println("From: " + m.getFrom()[0]);
            System.out.println("Subject: " + m.getSubject());
            // some Messages are multipart messages (e.g. text with an attachement)
            if (m.getContentType().equals("multipart")) {
                Multipart mp = (Multipart) m.getContent();
                for (int j = 0; j <; mp.getCount(); j++) {
                    Part part = mp.getBodyPart(j);
                    String disposition = part.getDisposition();
                    if (disposition == null) {
                        MimeBodyPart mimePart = (MimeBodyPart) part;
                        if (mimePart.isMimeType("text/plain")) {
                            BufferedReader in = new BufferedReader(
                                    new InputStreamReader(mimePart.getInputStream()));
                            for (String line; (line = in.readLine()) != null;) {
                                System.out.println(line);
                            }
                        }
                    }
                }
            } else {
                System.out.println(m.getContent());
            }
        }
        System.out.println("done :-)");
        folder.close(false);
        store.close();
    }

    public static void main(String[] args) throws Exception {
        // show some simple Dialogs to get server, username and password
        String server = JOptionPane.showInputDialog("enter your Mail Server e.g. mail.example.com");
        String user = JOptionPane.showInputDialog("enter your login e.g. user");
        String password = JOptionPane.showInputDialog("enter your mail password");
        getMail(server, user, password);
    }
}

Becource you will need the java mail lib as a dependency i just created a small maven project to add them.
Just unzip it and run

mvn exec:java

Download Maven Project for ImapCollector

Update:
A Colleague of mine asks for the imports of that Code. I just added them to the sources. You have to include the javamail lib from Sun Oracle as a dependency to run the code.

Hacking just for Fun: Raid5 in Java

PhilippLeave a comment

I was just curious how easy it might be to write a RAID5 compatible Outputstream in Java? Just a few Lines. For sure it is not the most elegante solution. Especially if you see the nice possibility to integrate one Outputstream within another… so maybe two Raid5s into one Raid0 Stream? (would be RAID50) then. Reading is missing ;-).

/**
 * Raid5Stream
 * 31.03.2012
 * @author Philipp Haussleiter
 *
 */
package de.javastream.jraid;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;

public class Raid5Stream extends OutputStream {

    public final static int PARITY_MARKER = 1;
    public final static int DATA_MARKER = 0;
    private File disks[];
    private FileOutputStream streams[];
    private int mode = 0;
    private final static int MAX_MODE = 2;
    private int[] buffer = new int[2];

    public Raid5Stream(File disks[]) throws IOException {
        super();
        if (disks.length != 3) {
            throw new RuntimeException("we need a disk count of x times 3");
        }
        int i = 0;
        this.disks = new File[disks.length];
        this.streams = new FileOutputStream[disks.length];
        for (File f : disks) {
            if (!f.exists() && !f.createNewFile()) {
                throw new RuntimeException(f.getAbsolutePath() + " does not exists and cannot be created!");
            } else {
                System.out.println("using " + f.getAbsolutePath() + "\n");
                this.disks[i] = f;
                this.streams[i] = new FileOutputStream(f);
            }
            i++;
        }
        writerMarker();
    }

    private void writerMarker() throws IOException {
        for (int i = 0; i < this.streams.length; i++) {
            if (i % 3 == 0) {
                this.streams[i].write(PARITY_MARKER);
            } else {
                this.streams[i].write(DATA_MARKER);
            }
        }
    }

    @Override
    public void write(int i) throws IOException {
        switch (mode) {
            case MAX_MODE:
                this.streams[mode].write(buffer[0] ^ buffer[1]);
                mode = 0;
            default:
                this.buffer[mode] = i;
                this.streams[mode].write(i);
                mode++;
        }
    }
}

Using it is easy as:

...
public class App {

    public static void main(String[] args) throws IOException {
        File raid5_disk1 = new File("raid5.disk1");
        File raid5_disk2 = new File("raid5.disk2");
        File raid5_disk3 = new File("raid5.disk3");
        long count = 0;
        Raid5Stream raid5Stream = new Raid5Stream(new File[]{raid5_disk1, raid5_disk2, raid5_disk3});
        FileInputStream in = new FileInputStream("/dev/random");
        BufferedInputStream bis = new BufferedInputStream(in);
        BufferedOutputStream raid5bos = new BufferedOutputStream(raid5Stream);
        while(count < 1024*1024*2){
            raid5bos.write(bis.read());
            count++;
        }
        raid5bos.close();
        bis.close();
    }
}

Good sources for more reading are the Wikipedia Articles about RAID and XOR.

SVG – the lost son of formats

PhilippLeave a comment

I was on the Chemnitz Linux Days last weekend. Besides great other talks a saw “Free your slides – Vortragsfolien im Browser anzeigen” from Sirko Kemter. He talked about a tiny tool called jessyink for creating Slides out of inkscape SVGs. I did a small SVG-Demo some months ago using RaphaelJS and a draft about this project exists since then. So instead of trying to complete this draft and published it, i decided to ask what area of SVG might be of interest here. Basic SVG elements? Animation? It might need some time, but i re-discovered what a great format SVG is and really want to create some posts for that.

 

Run local/remote terminal commands with java using ssh

Philipp3 Comments

Sometimes you need to use some CLI-Tools before you want to create or search for a native JNI Binding.
So there is a common way, using the Java Process-Class. But then you might meet two problems i had to face in the past during several problems:

  1. There are (a really small) number of CLI-Tools, that giving no constant output over the STD-OUT (the standard output the Process-Class uses for output)
  2. There is no “elegant” way to implement a process call into your project.

To solve this Problem I created a basic HelperClass, that calls the System over SSH (with the Convenience to work remote and the side-effect to always get STD-compatible output).
I am primarely using it for a fun project SAM i started some months ago to try to create a Management-Tool for Unices and Windows with a very low client-side footprint.

The first Class is used to capsulate the basic SSH Calls:

 
// some imports... 
 public class SystemHelper {
    private Runtime r;
    private String sshPrefix = "";

    // call with $user and 127.0.0.1 to run local command.     
    public SystemHelper(String user, String ip) {
        r = Runtime.getRuntime();
        sshPrefix = " ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no " + user + "@" + ip;
    }

    public void runCommand(String command, ProcessParser pp) {
        try {
            Logger.info("running: " + this.sshPrefix + " " + command);
            Process p = r.exec(this.sshPrefix + " " + command);
            InputStream in = p.getInputStream();
            BufferedInputStream buf = new BufferedInputStream(in);
            InputStreamReader inread = new InputStreamReader(buf);
            BufferedReader bufferedreader = new BufferedReader(inread);
            pp.parse(bufferedreader);
            try {
                if (p.waitFor() != 0) {
                    Logger.info("exit value = " + p.exitValue());
                }
            } catch (InterruptedException e) {
                System.err.println(e);
            } finally {
                // Close the InputStream                 
                bufferedreader.close();
                inread.close();
                buf.close();
                in.close();
            }
        } catch (IOException ex) {
            Logger.error(ex.getLocalizedMessage());
        }
    }
}

ProcessParser is an interface that defines the methode parse, accepting a BufferedReader for parsing the output of the Process. Unfortunately there is no timeout ATM to kill a hanging SSH-Call.

 public interface ProcessParser { 
     public void parse(BufferedReader bufferedreader); 
 }

The most basic (Output-)Parser looks like this:

 
    public String getPublicSSHKey() {
        SimpeOutputPP so = new SimpeOutputPP();
        String command = "cat ~/.ssh/id_rsa.pub";
        runCommand(command, so);
        if (!so.getOutput().isEmpty()) {
            return so.getOutput().get(0);
        }
        return "";
    }

This returns just the public SSH-Key of the current user. I implemented some more parsers for the output of apt (dpkg), rpm and pacman. You can find them in the github project here.

Blog Design update

PhilippLeave a comment

After endless years of using the old classic WordPress Theme, i started to creating my own Theming. So it might be a little bit edgy round the corners during the next days :-). Thank you to falkh for giving me a basic start with the nice Post-Caption-Thingys :-).

Counter Update

PhilippLeave a comment

I just finished my latest improvements to the legacy version of my counter script.
I just added the lookup for ISPs and added dynamic scaling for the axis legend.
I will now going forward to change the whole system to a more sophisticated software, e.g. using a Datawarehouse approach. The first version of the Data-Model is finished.

So from the old version (that was just some plain tables, flowing around)

 

 

 

 

 

 

 

 

 

 

I created a new Model with more Tables, connected to each other. Basically i devided the Model into a basic Fact (Count) and some Dimensions (for every Value):

 

 

 

 

 

 

 

 

 

 

With my current values (around 22k Facts), i have already to limit the facts that are queryed from the Datebase. I wrote a short script to migrate all old Datasets to the new Data-Model.

Joyent SmartOS VM MYSQL Setup

Philipp4 Comments

I just played around with the new SmartOS from Joyent ( Homepage ). I followed a basic tutorial from a Blog called opusmagnus to setup my basic SmartOS Machine on Virtualbox. You basically just have to insert the latest iso image and startup the VM (SmartOS is a system that runs from a live Medium – an DVD- or USB Image, so all your harddisk belongs to your VMs).

I will just summarize the basic steps to setup a basic VM for Mysql (you should also read the original post here).
After you installed your VM (setup Networking/ZFS Pool – you should use >3 virtual harddisks), you login into your new system and perfom the following steps:

Check for VM-Templates to install:

# dsadm avail
UUID                                 OS      PUBLISHED  URN                    
9dd7a770-59c7-11e1-a8f6-bfd6347ab0a7 smartos 2012-02-18 sdc:sdc:percona:1.3.8  
467ca742-4873-11e1-80ea-37290b38d2eb smartos 2012-02-14 sdc:sdc:smartos64:1.5.3
7ecb80f6-4872-11e1-badb-3f567348a4b1 smartos 2012-02-14 sdc:sdc:smartos:1.5.3  
1796eb3a-48d3-11e1-94db-3ba91709fad9 smartos 2012-01-27 sdc:sdc:riak:1.5.5     
86112bde-43c4-11e1-84df-8f7fd850d78d linux   2012-01-25 sdc:sdc:centos6:0.1.1  
...
5fef6eda-05f2-11e1-90fc-13dac5e4a347 smartos 2011-11-03 sdc:sdc:percona:1.2.2  
d91f80a6-03fe-11e1-8f84-df589c77d57b smartos 2011-11-01 sdc:sdc:percona:1.2.1  
...
9199134c-dd79-11e0-8b74-1b3601ba6206 smartos 2011-09-12 sdc:sdc:riak:1.4.1     
3fcf35d2-dd79-11e0-bdcd-b3c7ac8aeea6 smartos 2011-09-12 sdc:sdc:mysql:1.4.1    
...
7456f2b0-67ac-11e0-b5ec-832e6cf079d5 smartos 2011-04-15 sdc:sdc:nodejs:1.1.3   
febaa412-6417-11e0-bc56-535d219f2590 smartos 2011-04-11 sdc:sdc:smartos:1.3.12

I choose a percona VM (that is a VM with MySQL and some Backup Stuff pre-install – more here).

# dsadm import  a9380908-ea0e-11e0-aeee-4ba794c83c33
a9380908-ea0e-11e0-aeee-4ba794c83c33 doesnt exist. continuing with install
a9380908-ea0e-11e0-aeee-4ba794c83c33 successfully installed

Then you need to create a basic VM-Config file /tmp/percona-vm:

{
        "alias": "percona-vm",
        "brand": "joyent",
        "dataset_uuid": "a9380908-ea0e-11e0-aeee-4ba794c83c33",
        "dns_domain": "haussleiter.de",
        "quota": "10",
        "nics": [
                {
                        "nic_tag": "admin",
                        "ip": "192.168.178.42",
                        "netmask": "255.255.255.0",
                        "gateway": "192.168.178.1"
                }
        ]
}

You are now able to create your VM:

# vmadm create -f /tmp/percona-vm
Successfully created df4108a7-c3af-4372-b959-6066c70661e9

You can check if your new VM is running:

# vmadm list
UUID                                  TYPE  RAM      STATE             ALIAS
df4108a7-c3af-4372-b959-6066c70661e9  OS    256      running           percona-vm

# ping 192.168.178.42
192.168.178.42 is alive

You can login in your VM (that is a Zone to be precisely with the zlogin command):

# zlogin df4108a7-c3af-4372-b959-6066c70661e9
[Connected to zone 'df4108a7-c3af-4372-b959-6066c70661e9' pts/2]

Becourse i could not found any information about the predifined MySQL Password, i just change it using standard-Mylsq CMDs:
First you need to stop the Mysql Service:

# svcadm disable mysql:percona

Then you need to start Mysql again with skipping the tables for user-credentials.

# mysqld_safe --skip-grant-tables &

Enter Mysql CMD-Tools and change the root Password:

# mysql -uroot
mysql> use mysql;
mysql> update user set password=PASSWORD("NEW-ROOT-PASSWORD") where User='root';
mysql> flush privileges;
mysql> quit;

You need to shutdown your MySQL instance:

# prstat
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP       
...
 10921 root     4060K 3016K cpu2     1    0   0:00:00 0.0% prstat/1
 10914 root     3200K 2236K sleep    1    0   0:00:00 0.0% bash/1
 10913 root     3172K 2244K sleep    1    0   0:00:00 0.0% login/1
 10894 mysql 207M 37M sleep 1 0 0:00:00 0.0% mysqld/27
...
kill 10894

You now can start the MySQL Daemon again.

# svcadm enable mysql:percona

I hope i will have time to look into more features of SmartOS. It seems to be a great System to Virtualize a lot of differend Service. It also supports virtualizing Windows VMs with KVM.