If you want to try this, I’ve provided some small test code that uses the ‘guestbook’ demo project that comes with AppEngine.

The screenshot below shows what the ‘guestbook’ demo looks like.

And deploying a test Flex application with the Java project, we get the screenshot below.

First, build a new ‘Flex Project’ using the following code. The key configuration within this code is the ‘HTTPService’ that will call a Java servlet which in turn returns the XML of the guestbook postings.

<?xml version="1.0" encoding="UTF-8" ?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" xmlns:jw="components.*" applicationComplete="guestbook.send();">
<mx:Script>
import mx.rpc.Fault;
import mx.rpc.events.*;
import mx.controls.Alert;

public function faultHandler(event:FaultEvent):void
{  Alert.show(event.fault.faultString, "Error");  }

public function resultHandler(event:ResultEvent):void
{  }
</mx:Script>

<mx:HTTPService id="guestbook" url="/greetings"	method="GET" resultFormat="e4x"
  fault="faultHandler(event);" result="resultHandler(event);" />

<mx:Panel id="panel" title="Guest Book">
  <mx:DataGrid id="dataGrid" dataProvider="{guestbook.lastResult.post}" editable="false">
     <mx:columns>
       <mx:DataGridColumn id="nicknameColumn" width="100" dataField="nickname" headerText="Nickname" />
       <mx:DataGridColumn id="contentColumn" width="200" dataField="content" headerText="Content" />
       <mx:DataGridColumn id="dateColumn" width="100" dataField="date" headerText="Date" />
      </mx:columns>
     </mx:DataGrid>
</mx:Panel>
</mx:Application>

Within the ‘Guest Book’ project, create a new servlet ‘GreetingXMLServlet.java’.

Paste the following code that creates the XML file to be returned to the Flex application. This servlet queries the object datastore for all the guestbook postings.

package guestbook;
import java.util.List;
import javax.jdo.PersistenceManager;
import guestbook.Greeting;
import guestbook.PMF;
import java.io.IOException;
import javax.servlet.http.*;
@SuppressWarnings("serial")
public class GreetingsXMLServlet extends javax.servlet.http.HttpServlet
{
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException
{
resp.setContentType("text/xml");
PersistenceManager pm = PMF.get().getPersistenceManager();
String query = "select from " + Greeting.class.getName() + " order by date desc range 0,5";
List<Greeting> greetings = (List<Greeting>) pm.newQuery(query).execute();
if (!greetings.isEmpty())
{
resp.getWriter().println("<greetings>");
for (Greeting g : greetings)
{
resp.getWriter().println("
<post>");
if (g.getAuthor() == null)
resp.getWriter().println("<nickname>Anonymous</nickname>");
else
resp.getWriter().println("<nickname>" + g.getAuthor().getNickname() + "</nickname>");
resp.getWriter().println("<content>" + g.getContent() + "</content>");
resp.getWriter().println("<date>" + g.getDate() + "</date>");
resp.getWriter().println("</post>");
}
resp.getWriter().println("</greetings>");
}
pm.close();
}
} 

Paste the following into ‘war/WEB-INF/lib/web.xml’ to configure the servlet.

<servlet>
<servlet-name>greetings</servlet-name>
<servlet-class>guestbook.GreetingsXMLServlet</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>greetings</servlet-name>
<url-pattern>/greetings</url-pattern>
</servlet-mapping>

Finally, import the ‘Flex’ release files into the ‘war’ folder of the ‘Guest Book’. If you have both projects in Eclipse, drag and drop it!

The Files are now within the ‘Guest Book’ project, and deploy into AppEngine!

What version information can be gathered from the AppEngine? There’s a handy function getProperties() that will return all the system properties available to a Java app. The list that is available from the AppEngine JVM is restricted as outlined in the ‘Environment’ section of this page. So…. running the following line of code… System.getProperties().list(resp.getWriter());….what do we get?

– listing properties –
java.vendor=Sun Microsystems Inc.
java.specification.version=1.6
line.separator=
java.class.version=50.0
java.util.logging.config.file=WEB-INF/logging.properties
java.specification.name=Java Platform API Specification
java.vendor.url=http://java.sun.com/
java.vm.version=1.6.0_13
os.name=Linux
java.version=1.6.0_13
java.vm.specification.version=1.0
user.dir=/base/data/home/apps/versionapp/1.3327…
java.vm.specification.name=Java Virtual Machine Specification
java.specification.vendor=Sun Microsystems Inc.
java.vm.vendor=Sun Microsystems Inc.
file.separator=/
path.separator=:
java.vm.name=Java HotSpot(TM) Client VM
java.vm.specification.vendor=Sun Microsystems Inc.
file.encoding=ANSI_X3.4-1968

No real surprises, running Java version 6 – update 13 on Linux. Admittedly, this doesn’t tell the full story as the ‘readme’ file included with the App Engine SDK reveals more.

Google App Engine SDK for Java
Copyright (c) Google, Inc. 2009. All rights reserved.
Visit Google Code (http://code.google.com/appengine/).
This product includes software developed by:
- The Apache Software Foundation (http://www.apache.org/).
- The servlet specification classes: servlet-api-2.5.jar, jsp-api-2.1.jar, el-api-2.1.jar
- Apache Ant (http://ant.apache.org/)
- Jakarta Commons Logging (http://commons.apache.org/logging/)
- Jakarta Commons HttpClient (http://jakarta.apache.org/commons/httpclient/)
- Jakarta Commons Codec (http://commons.apache.org/codec/)
- Jakarta Commons EL (http://commons.apache.org/el/)
- Jakarta Jasper (http://tomcat.apache.org/svn.html)
- Jakarta Taglibs (http://jakarta.apache.org/taglibs/)
- The Apache Geronimo Project (http://geronimo.apache.org/)
- javax/mail with modifications
- javax/activation
- geronimo-servlet_2.5_spec-1.2.jar, geronimo-jsp_2.1_spec-1.0.1.jar, geronimo-el_1.0_spec-1.0.1.jar
- Mort Bay Consulting (http://www.mortbay.org)
- Jetty web server

We can see that Jetty web server and the Geronimo project are used.

Before anybody complains, that technically more than one line of code was used as there is no declaration function for the servlet… here’s the full one line!

package com.test; public class versionServlet extends javax.servlet.http.HttpServlet { public void doGet(javax.servlet.http.HttpServletRequest req, javax.servlet.http.HttpServletResponse resp) throws java.io.IOException { System.getProperties().list(resp.getWriter()); } }

Disclaimer!! – As with any metric testing, I need to state a disclaimer that a lot of factors can heavily influence the timing, such as other applications running locally, network performance, Java optimization, garbage collection, class loading, caching etc. etc. the list can go on and on. There isn’t an easy way that I’m aware of to get precise and fair timings across all environments. In particular as it’s a hosted environment Google could decide next week to upgrade the hardware thus giving a very different set of results!

I decided to create a bare minimum app that calls math functions that are available in Java and Python. Upload this app to AppEngine without any optimizations to the code or changing any of the default settings in AppEngine. I wanted the code in both apps to be as similar as possible, but another disclaimer Python and Java can’t be compared fully as they both have differences in their behavior. In particular when using the Math functions in Python it is dependent upon the OS that is hosting it. I.e. running ‘print math.pow(10,6)’ generates an error in the Google Python environment.

I decided to run three tests using Square Root, Power and Log. Run it 10 times and note the output. The local environment that I ran the tests is a single core Intel Pentium M 2Ghz, Win XP Service Pack 3 with 1Gb RAM.

What happened?

Before I go any further, the code that I used… yes, yes.. optimizations can be applied etc. but I wanted to use the most basic app out of the box.

Java:

import java.io.IOException;
import javax.servlet.http.*;
@SuppressWarnings("serial")
public class metric1Servlet extends HttpServlet
{
 public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException
 {
   resp.setContentType("text/plain");

  long start = System.nanoTime();
  int result = 0;
  for (int i = 0; i < Math.pow(10,5); i++)
  {  result += Math.sqrt(i);  }
  long duration = (System.nanoTime() - start) / 1000000;
  resp.getWriter().println("Test 1: " + duration + "ms");

  start = System.nanoTime();
  result = 0;
  for (int i = 0; i < Math.pow(10,5); i++)
  { result += Math.pow(Math.E, 2) }
  duration = (System.nanoTime() - start) / 1000000; <
  resp.getWriter().println("Test 2: " + duration + "ms"); 

  start = System.nanoTime();
  result = 0;
  for (int i = 0; i < Math.pow(10,5); i++) &
  {  result += Math.log(Math.random()); }
  duration = (System.nanoTime() - start) / 1000000;
  resp.getWriter().println("Test 3: " + duration + "ms");
 }
}

Python:

import time
import math
import random

print 'Content-Type: text/plain'
print ''

start = time.time()
result=0
for i in range(0,int(math.pow(10,5))):
result =+ math.sqrt(i)
duration = (time.time() - start)*1000
print "Test 1: ", duration ,"ms"

start = time.time()
result=0
for i in range(0,int(math.pow(10,5))):
result += math.pow(math.e, 2)
duration = (time.time() - start)*1000
print "Test 2: ", duration ,"ms"

start = time.time()
result=0
for i in range(0,int(math.pow(10,5))):
result += math.log(random.random())
duration = (time.time() - start)*1000
print "Test 3: ", duration ,"ms"

I should mention that the AppEngine has a request timeout to kill any page requests taking longer than 30 seconds. The tests had to be quite short thus most disappointingly couldn’t be used to calculate the ‘meaning of life’. Perhaps next time Google? Anyway, details about the request timer for Java can be found here under ‘Request Timer’.

The output:

Test 1:

Test 2:

Test 3:

The first thing from the results is that running it 10 times really isn’t enough… when I get the time, I need to run it on a larger scale.

From the data, it’s interesting to see that Java is fairly consistent overall and there is a performance boost running it on the Google environment which is what I would expect as my local machine isn’t exactly ‘state of the art’. Test 2 does show some discrepancy for running the Java locally but shows some sign of consistency towards the end.

Python is very different in that running it in the AppEngine environment doesn’t show anywhere near the consistency of Java. The chart with the averages below shows the differences clearer.

Could it be the timing of when I ran the tests, such that the Python servers have more workload and that the Java environment is on a limited rollout? Hard to say… also running Java locally is consistently slower whereas Python in the AppEngine doesn’t always seem to be faster. Are these results just a one off? I used a clean install of Python when testing locally thus I made no customizations to the Python environment or to the Python App Engine SDK. I think I’ve raised more questions to myself than answers…hmmm not what I was planning! I think the key thing that I take from these basic tests is that it has shown differences between the two environments and that I really need to do these again on a larger scale.

That lead to another question, is it worth developing everything in Java as it’s that much faster than Python? I wouldn’t say so, as the tests don’t really answer this question sufficiently as both languages have their own strengths and weaknesses. Also, if it is the case that Java is on a limited rollout, the results could be very different when the Java AppEngine is fully available to everyone.

A key point that I take from these tests, is that there are so many ‘factors’ involved when the application is being hosted, it’s got me thinking about how applications can be accurately stress tested and measured when the hardware environment could change any time along with the varying performance of the local internet connectivity. Somebody might already have the answer for this, if they have let me know!

I can sleep safely at night as I can see ‘unsubscribe’ is very popular. Maybe it’s time for me to start reading the junk and clicking ‘unsubscribe’ and buying watches from my junk mail folder.

For those who have the morbid curiosity to do the same for their own mailbox, Gmail currently only provides one documented interface for extracting emails and that’s via POP3/IMAP4. However, an alternative and quicker approach if you use ‘offline email’ is to read the local Gmail Gears sqlite database. Gears uses the database to store the offline data, thus it’s a case of querying the local database with a Sql statement and writing to a text file. The text file I pass through to the word-cloud generator.

The location of the database can vary depending upon the browser you are using and whether you are going through http or https. Look here if you want to find the specific location of the database. The code I used to extract the mails is quite short as below.

import java.sql.*;
import java.io.*;

public class ReadGMail
{
  public static void main(String[] args) throws Exception
  {
    Class.forName("org.sqlite.JDBC");
    Connection conn = DriverManager.getConnection("gmail.com-GoogleMail#database");

    FileWriter fstream = new FileWriter("gmail.txt");
    BufferedWriter out = new BufferedWriter(fstream);
    Statement stat = conn.createStatement();
    ResultSet rs = stat.executeQuery("SELECT c0Subject, c1Body  FROM MessagesFT_content;");
    while (rs.next())
    {
      out.write(rs.getString("c0Subject") + "\n");
      out.write(rs.getString("c1Body").replaceAll("\\<.*?>","") + "\n\n");
    }
    out.close();
    rs.close();
    conn.close();
  }
}

More examples can be found within the gallery. The code to generate the cloud images is available from the ‘Word-Cloud Generator’ project part of IBM alphaWorks. However, what I really wanted to do is to generate a ‘cloud’ from a Lotus Notes mailbox. As the tool only requires a text file, it’s just a case of exporting the subject and body of the emails into a text file. (Initially, I was doing it with DXL and applying a style sheet as it was faster, but DXL crashes too often for my liking!)

I scheduled a weekly java agent to read the inbox and within the agent I attached the archive ‘ibm-word-cloud.jar’ to allow the ‘generator’ to run under the Domino JVM. ‘Word Cloud’ requires a configuration file that tells it how to generate the image such as fonts, stop words etc. This file is very well commented and included within the word-cloud generator project download. It’s very interesting seeing mailboxes in a cloud format, thus I would recommend giving it a shot on your own mail file!

Below you’ll find an amended version of the java agent that I schedule weekly under R8 to send an image via email.

import java.io.*;
import java.io.File;
import lotus.domino.*;

public class JavaAgent extends AgentBase
{
    // Work Directory
    private static final String WORK_DIRECTORY="C:\\temp\\WordCloud\\";

    // Word Cloud configuration file
    private static final String CONFIG_FILE =WORK_DIRECTORY + "Configuration.txt";

    // Mail file Output
    private static final String MAIL_TEXT =WORK_DIRECTORY + "mail.txt";

    // Generated Image
   private static final String TEMP_FILENAME=WORK_DIRECTORY + "temp.png";

    // Size of Image
    private static final String IMAGE_WIDTH="800";
    private static final String IMAGE_HEIGHT="800";

    public void NotesMain()
    {
        try
        {
            Session session = getSession();
            AgentContext agentContext = session.getAgentContext();

            Database db = agentContext.getCurrentDatabase();

            // Intentionally processed inbox view only
            // Change the following to db.alldocuments if you want to do the whole database
            View view = db.getView("($Inbox)");

            // Write out the mails to text file.
            File outFile = new File(MAIL_TEXT);
            FileWriter fstream = new FileWriter(outFile);
            BufferedWriter out = new BufferedWriter(fstream);

            Document doc = view.getFirstDocument();
            while (doc != null)
            {
                if (doc.hasItem("Body"))
                {
                    Item body = doc.getFirstItem("Body");
                    if (body.getType()==Item.RICHTEXT)
                    {
                        RichTextItem rtbody = (RichTextItem)doc.getFirstItem("Body");
                        // Extract subject and body only
                        out.write(doc.getItemValueString("Subject"));
                        out.write(rtbody.getFormattedText(false, 0, 0) + "\n");
                    }
                }
            doc = view.getNextDocument(doc);
        }

        out.close();

        // Pass parameters to IBM Word Cloud
        String[] params= new String[10];
        params[0]="-c";
        params[1]=CONFIG_FILE;
        params[2]="-w";
        params[3]=IMAGE_WIDTH;
        params[4]="-h";
        params[5]=IMAGE_HEIGHT;
        params[6]="-i";
        params[7]=MAIL_TEXT;
        params[8]="-o";
        params[9]=TEMP_FILENAME;

        // There is an additional parameter '-s' for seed. See readme.html file included within IBM Word Cloud package
        com.ibm.wordcloud.Main.main(params);

        // Send the generated image via email
        Document memo = db.createDocument();
        memo.appendItemValue("Form", "Memo");
        memo.appendItemValue ("Subject", "Wordy output of mailfile");

        RichTextItem body = memo.createRichTextItem("Body");
        body.embedObject(EmbeddedObject.EMBED_ATTACHMENT, null, TEMP_FILENAME,null);
        memo.send("mailinggroup@example.com");

        // Don't need the image and text file anymore.
       if (outFile.exists())
         outFile.delete();

       File f= new File(TEMP_FILENAME);
       if (f.exists())
          f.delete();

    }
    catch(Exception e)
    {  e.printStackTrace();  }
    }
}

using System;
using CommunicatorAPI;
using System.Runtime.InteropServices; 

namespace CommunicatorAPIHelloWorld
{
 class Program
 {
  static void Main(string[] args)
  {
   CommunicatorAPISample client = new CommunicatorAPISample();
   Console.WriteLine("Login");
   client.Signin("username", "password");
   Console.ReadKey();

   Console.WriteLine("Start IM Conversation");
   client.startConversation("joe.bloggs@example.com");
   Console.ReadKey();

   Console.WriteLine("Show contacts");
   client.ShowContacts();
   Console.ReadKey();

   Console.WriteLine("Send Message");
   client.SendTextMessage("Another Message");
   Console.ReadKey();

   Console.WriteLine("Capture IM Window");
   client.WriteHistory();
   client.Signout();
  }
 }

class CommunicatorAPISample
{
 private long imWindowHandle; // handle to the current conversation window
 private bool connected = false;
 private CommunicatorAPI.Messenger communicator = null;
 private IMessengerConversationWndAdvanced imWindow = null;
 public CommunicatorAPISample()
 { }

 // A simple implementation of signing in.
 public void Signin(string account, string passwd)
 {
    if (connected)
     return;

   if (communicator == null)
   {
     // Create a Messenger object, if necessary
     communicator = new CommunicatorAPI.Messenger();

    // Register event handlers for OnSignin and Signout events
    communicator.OnSignin += new DMessengerEvents_OnSigninEventHandler(communicator_OnSignin);
    communicator.OnSignout += new DMessengerEvents_OnSignoutEventHandler(communicator_OnSignout);
    communicator.OnIMWindowCreated += new DMessengerEvents_OnIMWindowCreatedEventHandler(communicator_OnIMWindowCreated);
    communicator.OnIMWindowDestroyed += new DMessengerEvents_OnIMWindowDestroyedEventHandler(communicator_OnIMWindowDestroyed);
  }

  if (account == null)
    communicator.AutoSignin();
  else
    communicator.Signin(0, account, passwd);
}

 // Event handler for OnSignin event.
 void communicator_OnSignin(int hr)
 {
   if (hr != 0)
   {
     Console.WriteLine("Signin failed.");
     return;
   }
   connected = true;
 }

 void communicator_OnSignout()
 {
    connected = false;

    // Release the unmanaged resource.
   Marshal.ReleaseComObject(communicator);
   communicator = null;
 }

 // Register for IMWindowCreated event to receive the
 // conversation window object. Other window objects can
 // received via this event handling as well.
 void communicator_OnIMWindowCreated(object pIMWindow)
 {
   try
   {
     imWindow = (IMessengerConversationWndAdvanced)pIMWindow;
     if (((IMessengerConversationWndAdvanced)pIMWindow).HWND == imWindowHandle)
       { /* SendTextMessage("Hello World!"); Could send message here */ }
     else
       { imWindowHandle = imWindow.HWND; }
   }
   catch (COMException comException)
     { throw comException; }
   catch (Exception exception)
     { throw (exception); }
 }

 void communicator_OnIMWindowDestroyed(object pIMWindow)
 {
   try
    { Marshal.ReleaseComObject(pIMWindow); }
   catch (COMException comException)
    { throw comException; }
  catch (Exception exception)
    { throw (exception); }
 }

 public long startConversation(string account)
 {
   object[] sipUris = { account };
   CommunicatorAPI.IMessengerAdvanced msgrAdv = communicator as CommunicatorAPI.IMessengerAdvanced;

  if (msgrAdv != null)
  {
    try
    {
     object obj = msgrAdv.StartConversation(
        CONVERSATION_TYPE.CONVERSATION_TYPE_IM, // Type of conversation
        sipUris, // object array of signin names for having multiple conversations
        null,
        "Testing",
        "1″,
        null);

     imWindowHandle = long.Parse(obj.ToString());
    }
    catch (COMException ex)
    { Console.WriteLine(ex.ErrorCode); }
  }

  return imWindowHandle;
 }

  // Send text only if the window object matches the desired window handle
  public void SendTextMessage(string msg)
  {
    if (imWindow == null)
      return;

    if (imWindow.HWND == imWindowHandle)
      { imWindow.SendText(msg); }
  }

  public void ShowContacts()
  {
    // Display contacts to a console window(for illustration only).
    foreach (IMessengerContact contact in communicator.MyContacts as IMessengerContacts)
    {
      if (!contact.IsSelf)
       Console.WriteLine("{0} ({1})", contact.SigninName, contact.Status);
    }
  }

 public void WriteHistory()
 {
    if (imWindow != null)
   {
     try
      { Console.WriteLine(imWindow.History); }
    catch (COMException CE)
      { Console.WriteLine("COM Exception " + CE.ErrorCode.ToString()); }
   }
 }

 // A simple implementation of signing out.
 public void Signout()
 {
   if (!connected)
    return;

   if (communicator == null)
    return;

  communicator.Signout();
  }
 }
}


If you're after a WinForms sample, a nice VS project can be found here.

For some strange reason the ‘Next’ button wouldn’t appear until I forced a screen refresh. (I.e. Alt-Tab)

In a very simplified nutshell, the key thing to remember is that S/MIME involves sending an ‘encrypted attachment’ called ‘smime.p7m’. You can send this attachment over SMTP and receive it via POP3/IMAP4. When receiving the email, as long as your email client can recognize that there is an attachment ‘smime.p7m’, it will open and decrypt it automatically. (Assuming you have the right certificates installed!)

For instance, Microsoft Outlook has recognized that I have received an encrypted email via Gmail, shown below. I sent this encrypted email from one Gmail account via SMTP and received it in a different Gmail account via IMAP4.

As I have the valid certificates installed, opening the email allows me to read it. (Notice the padlock in the screenshot showing it is encrypted)

Also works with Thunderbird!

Outlook Express!

And even Lotus Notes via IMAP!

Ah, how about in the Gmail web interface? Gmail knows there is an attachment, but doesn’t do anything with it.

If you want to view it within the web interface, installing the Gmail S/Mime Firefox plugin for Firefox does the trick! This nifty plug-in will also allow you to create and send SMIME emails within the browser interface.