My approach to JSONP limitations


Kanban columnDuring the development of My Personal Kanban I stumbled across interesting problem. One of My Personal Kanban features that I was developing, was the possibility to upload Kanban to Google Cloud (Google App Engine application).

My Personal Kanban is designed to work off the local file system, without the need of Internet connection. It means that trying to send something into Interweb is going to hit modern Browser security settings.

The browser security feature prevents web request from being made to a site with a different domain. It also stops from making a request to anything other the same file if you have opened any web application from the local file system. This kind of behaviour is default across any modern web browser.

Fortunately, there is a special type of requests browser will allow happening and it’s JSONP. It’s a GET request with a callback parameter. Callback is a name of the function that the browser will call when it receives successful response from the web.

HTTP GET request limits

My Personal Kanban is written in JavaScript with Angular; server side on Google App Engine is a very simple Servlet written in Groovy.

When I finished my first implementation and started to test it locally with Google App Engine SDK it all looked good. However, upload stopped working when I tested with a real GAE deployment.

Quick research confirmed that different web server implementation might have different settings for Maximum Length of HTTP GET parameters. Those GET parameters are used to send JSONP request data to the web server. Google App Engine has different limit than GAE SDK (which uses Jetty).

I also discovered that the upload was working in one browser but stopped working in another. As it turns out, browsers have its own limits of length of HTTP GET request. It is literally the length of the URL you will put in your browsers address box.

My Personal Kanban is sending a bit of data. It’s not Megabytes, however still too much for the HTTP GET parameters.

Choices of workaround

I thought of writing My Personal Kanban as Chrome extensions. It would enable me to overcome the JSONP limitations however it would bind my application to work on Chrome only. So I ditched the idea.

I decided to chop data into small chunks and send it to the server. To minimize errors of the transfer I invented this Client-to-Server protocol:

  1. All calls during transmission to server are in order, so the data can be assembled in correct way.
  2. First a handshake is made announcing the beginning of transmission for specific user, with information of how many data chunks will be sent.
  3. Chunks of data are sent only if previous chunk was sent successfully.
  4. Finally MD5 hash of data is sent so the server can verify that what was received is correct.

On server side I decided to store chunks for a user in session which in case of Google App Engine is stored in Memcache backed by Data Store. What happens on the server:

  1. Server receives a handshake and creates new Array for data chunks (or removes previous one if transfer was not completed or not successful).
  2.  Server receives data chunks and places them in the Chunk Array stored in session.
  3. When server receives Kanban hash it concatenates the Array in order into String and validates against received hash. If the hash is valid it stores the Kanban in the Data Store.

Details and code samples

The uploaded Kanban is Encrypted with user key and kept on the Cloud encrypted to ensure data privacy. I’ve chosen Rabbit as encryption algorithm. As a side effect to encryption, transmitted data doesn’t need to be encoded and sanitized.
For encryption I’m using fantastic CryptoJS library. CryptoJS also includes MD5 hash implementation.

md5Hash : function(stringToHash){
	return CryptoJS.MD5(stringToHash).toString();

encrypt: function(stringToEncrypt, encryptionKey){
	var utfEncoded = CryptoJS.enc.Utf8.parse(stringToEncrypt);
	return CryptoJS.Rabbit.encrypt(utfEncoded, encryptionKey).toString();

decrypt: function(stringToDecrypt, encryptionKey){
	var notYetUtf8 = CryptoJS.Rabbit.decrypt(stringToDecrypt, encryptionKey);
	return CryptoJS.enc.Utf8.stringify(notYetUtf8);

Thanks to fantastic Angular Promise API it is very easy to implement ordered transmission of data.

var encryptetKanban = cryptoService.encrypt(kanban, this.settings.encryptionKey);
var kanbanInChunks = splitSlice(encryptetKanban, 1000);

var promise = sendStart(kanbanInChunks.length);
angular.forEach(kanbanInChunks, function(value, index){
	promise = promise.then(function(){
		return sendChunk(value, index + 1);

return promise.then(function(){
	return checkKanbanValidity(encryptetKanban);

My Personal Kanban is Open Source project and available for browsing at


The above approach made it possible to upload data to the server from any browser eliminating issues related to default browser’s security settings.
Unfortunately it comes with a cost of more HTTP requests and custom server side coding.

For small amount of text data it is good enough for me, perhaps it would be good enough for you. Let me know your thoughts.

Do Less to Do More – My Personal Kanban in the Clouds

Kanban columnAs you might already know I did a small application called My Personal Kanban, that serves as offline Kanban board. It’s HTML application which stores your Personal Kanban on your device. It free to use and Open Source.

I just released a new feature and service for My Personal Kanban that allows you to Upload and Download your Personal Kanban Boards to Cloud.

The service and functionality is in Beta phase but it would be great if you could give it a try and provide me with some feedback.

Heres a video of how the Cloud features work.

Here is the link to the previous post with Themes.

If you want to help me and contribute a Theme or some code, please do get in touch.


Mixture of technology at Google App Engine (GAE)

For some time now I am developing my little hobby project. It is called F1Dashboard and it contains informations related to Formula 1 (my favorite motor-sport).

F1 Dashboard
F1 Dashboard

Since the site is quite stable now I will share with you some technologies and ideas I used.


F1Dashboard is build and deployed on Google App Engine (GAE). I started this as an experiment and as a toy project just to learn something new.

GAE is not a new idea, it is another platform for cloud computing. Main difference from others is the Database model. GAE is using a thing called Data Store (Big Table), which is not a normal relational database that anyone used to. There is no schema defined, and querying is impossible without indexing it before. It is something like a massive bag that could store stuff in.

GAE is offered as Python and Java. The later was released long time after Python. It has limited version of JVM environment and a whole bunch of Google services to use. Lots of standard JDK types got repackaged by Google just to limit standard Java functionality, like spawning a Threads.

GAE offers standard Java servlets as web technology. This means that you could use some of the well established frameworks out there. I decided to create one myself of course 🙂 Something simple and not complicated.


I choose Groovy, just because I wonted to learn another dynamic language and I liked the way Groovy did closures.


The framework I did follows MVC pattern. It has single servlet entry. Every requested URL is parsed internally and resolved into Controller and Action. I’m using String Template as rendering engine for my views.

All database operations are entirely hidden behind Repositories, and I made it simple to switch from Objectify to Data Nucleus to anything else that comes.

I choose Objectify for its simplicity. If you are starting with GAE I would very much recommend it.


The design is simple and was done by fried of mine Mike from Rosslyn Digital.

I plugged in jQuery Tools for Overlay effect. It also has a nice plugin for picture gallery.

I also added automatic Twitter updates for every message that is updated on a site.


Site is automatically harvesting for new content from various of Formula 1 related sites. There is also content from Twitter, YouTube and Flickr. It is utilizing GAE cron jobs for that.

I’m trying to gather some ideas for what I can do with the site in near future. If you have an idea, drop me an email. You can also see latest updates on a site at

Cheers, Greg

F1 Dashboard released

In last few months I was working on a little personal project. When Google App Engine team announced that preview of Java version is available I decided to give it a try. When I was looking for a subject of my application, one of the ideas started to emerge more that others. I was looking for a web site that could aggregate all the information about Formula 1 racing. Because there was none available (or my Google search ended on first few results that were not satisfactory) I decided to make one.

Few weeks and cups of coffee latter is released.

F1 Dashboard screen shot
F1 Dashboard screen shot


So, in few words, F1 Dashboard is agregator of news, and media feeds from world of Formula 1. I decided not to collect content, just store information where to find it and short description.

News are harvested from a number of web sites. Twitter updates serves as another source of news.

Images are comming from Flickr, videos from YouTube.

All updates  are happening every few minutes and are 100% automated.


GAE supports Groovy as one of JVM programming languages. I decided to give it a try. I love Groovy, it is somewhere in between a friendly and known Java API and Ruby programming language.

It took me a moment to use to BigTable type of DataStore. It restrictes a way I used to work with databases normaly. Google provide entire environment for local development, whitch means I don’t have to upload to a cloud to see working result.

I created a little homegrown Model Views Presenter/Controller framework and used StringTemplate as rendering engine.

jQuery is the library of choice to handle all JavaScript.

Page styling was done by a friendly and kind designer from Circa82 Michael Austin (thanks buddy).

Come on guys, give it a try and tell me what you think 😉 Feedback greatlly appriciated.

Cheerios, Greg

Groovy with JDO on Google App Engine, enhancing with DataNucleus

I have some time this days and I’m trying to fly some clouds with Google App Engine (GAE), Now when they have released Java support I can try to use Groovy. Today I tried to implement some simple storage using JDO. GAE team supports some of its features via  DataNucleus. DataNucleus is using a post compilation hook to enhance any persistable classes.

Having Groovy in a lib folder of your war (GAE runs your application from this folder) I need to put asm library as well. Latest version of Groovy (1.6.1) has a dependency on asm library in version 2.2.3. On the other hand DataNucleus enhancer has dependency on asm library in version 3.1.

GAE Java SDK is shipped with Ant task for enhancing persistable classes. It also contains Ant macro definition file that makes it simple for usage. Unfortunately this macro includes all libraries from application lib folder. This is causing Enhancer to use asm library in a version that causes it to fail massively.

[enhance] SEVERE: An error occured for ClassEnhancer "ASM" when trying to call the method "
m.ASMClassEnhancer" on class "getClassNameForFileName" : org.objectweb.asm.ClassReader.accept(Lorg/objectweb/asm/ClassVi

Number of this error messages is as big as the number of classes in my application.

Solution is to exclude asm library from the macro that Google shipped. Easiest way is to edit, or copy to your local application folder and use it there. The line problematic line of code is in classpath element.

          <pathelement path="${}"/>
          <pathelement path="@{war}/WEB-INF/classes"/>
          <fileset dir="@{war}/WEB-INF/lib" includes="*.jar">

The easiest way to fix it is to exclude asm library in fileset element, just like this:

          <pathelement path="${}"/>
          <pathelement path="@{war}/WEB-INF/classes"/>
          <fileset dir="@{war}/WEB-INF/lib" includes="*.jar" excludes="asm*.jar">

Hope that helps anyone with similar problem.

Cheers, Greg