Controlling central heating with Arduino and Raspberry Pi


When Arduino and Raspberry Pi released first versions, I did buy one of each. Being a gadget man and … well, a man, I played for a bit with my new toys and left them in a drawer.

Time has passed and not much happened with any of them. As I became increasingly unhappy with the central heating controller in my current house, I decided to take the Ino and Pi out of the drawer and actually build central heating controller that I would be happy with.

The design phase

First requirement of the new heating control system was the least intrusive installation as possible.

The dial thermostat that I have at home, works as a simple switch: switching the heating on when temperature falls bellow pre-set; and switching heating off when temperature rise above another pre-set. I decided to use this simplicity in my design. All I needed to do is hook with one 230V cable into that thermostat (230V, hell yeah).

With entry point sorted, I went into the controller bit. Arduino board will control the Relay, which will simply switch heating on/off. For the temperature reading I chose the Digital Thermometer which offered more stable reading than the Analog one (included in Arduino Uno starter kit). I’ve also added LED to indicate when the heating was on.

Leveraging serial port used to program Arduino, I decided that the board will send temperature updates and receive setup command from Pi via USB connection. It will also be powered via the same. This also solved a problem of 2 separate power adapters for both Arduino and Raspberry Pi.

The build phase

I’ve been trying few things before arriving with the below solution.

Electronics

The Arduino circuit is very simple.

circuit-smaller

Three elements connected to Arduino with a couple of resistors, not much.

The LED is not necessary, it’s there to indicate when the heating is switched on.

The final prototype doesn’t look very attractive, but the lot is hidden under the furniture and the only elements sticking out are the temperature sensors and a bit of LED.

Final prototype of my controller

 

 

Software

I had to write three separate pieces of software:

Arduino

For the temperature sensor I included two extra libraries, the OneWire protocol library and a DallasTemperature sensor library. I use 0.5 centi-degree approximation of a temperature reading.

Temperature reads are sent via Serial Port on every loop. Arduino also expects the Float number on a Serial Port. The received number indicates the desired room temperature for Arduino.

To limit the sensor reading fluctuations, the control of relay changes after at least 10 successful consecutive reads of the same temperature from the sensor.

Raspberry Pi

The software that runs on Raspberry Pi does the following:

  • It waits on the temperature updates from Arduino, and stores the Updates in Memory (for the latest update) and in simple file based H2 database (for historical data),
  • It exposes REST API for UI to get the temperature information and receive the new settings,
  • It schedules temperature changes according to schedule stored in the JSON file.

I started the code in Python, but it was running slow. I did a simple comparison of execution time for Prime number algorithms, and Java 8 was beating Python. On a single core Raspberry Pi 1, it was a good incentive to change the platform. I chose Kotlin programming language as it was new to me and I wanted to learn it.

As a framework for Event driven application I choose Vert.x 3. For Serial Port communication, a bit dated RXTX library.

The UI / Phone controller app

The Web App I build works on computers as well as mobile devices. I did choose React as a UI framework with Material UI components. The lot is build with Webpack into a small set of html/js files.

The testing phase

The testing phase involved connecting everything together, starting it, and hoping that I will not sense the smell of burning electrics and experience no explosions. In other words, a standard scientific and engineering approach 🙂 .

I do run the setup for 4 weeks continuously and it not failed so far.

Summary

This is the first time I used my skills to build something that interacts with physical world. It gave me a great feeling of achievement and satisfaction. I know that I could buy something that looks much nicer and probably works better but I learned a lot during the process.

Raspberry Pi 3 was released while I was building my design and I switched to it. You can see it in the pictures. I also want to switch Arduino Uno prototyping board to Arduino Nano.

All the code and more detailed technical description is available for you to grab from my GitHub repository on https://github.com/greggigon/Home-Temperature-Controller .

My approach to JSONP limitations

Why JSONP?

Kanban columnDuring the development of My Personal Kanban I stumbled across interesting problem. One of My Personal Kanban features that I was developing, was the possibility to upload Kanban to Google Cloud (Google App Engine application).

My Personal Kanban is designed to work off the local file system, without the need of Internet connection. It means that trying to send something into Interweb is going to hit modern Browser security settings.

The browser security feature prevents web request from being made to a site with a different domain. It also stops from making a request to anything other the same file if you have opened any web application from the local file system. This kind of behaviour is default across any modern web browser.

Fortunately, there is a special type of requests browser will allow happening and it’s JSONP. It’s a GET request with a callback parameter. Callback is a name of the function that the browser will call when it receives successful response from the web.

HTTP GET request limits

My Personal Kanban is written in JavaScript with Angular; server side on Google App Engine is a very simple Servlet written in Groovy.

When I finished my first implementation and started to test it locally with Google App Engine SDK it all looked good. However, upload stopped working when I tested with a real GAE deployment.

Quick research confirmed that different web server implementation might have different settings for Maximum Length of HTTP GET parameters. Those GET parameters are used to send JSONP request data to the web server. Google App Engine has different limit than GAE SDK (which uses Jetty).

I also discovered that the upload was working in one browser but stopped working in another. As it turns out, browsers have its own limits of length of HTTP GET request. It is literally the length of the URL you will put in your browsers address box.

My Personal Kanban is sending a bit of data. It’s not Megabytes, however still too much for the HTTP GET parameters.

Choices of workaround

I thought of writing My Personal Kanban as Chrome extensions. It would enable me to overcome the JSONP limitations however it would bind my application to work on Chrome only. So I ditched the idea.

I decided to chop data into small chunks and send it to the server. To minimize errors of the transfer I invented this Client-to-Server protocol:

  1. All calls during transmission to server are in order, so the data can be assembled in correct way.
  2. First a handshake is made announcing the beginning of transmission for specific user, with information of how many data chunks will be sent.
  3. Chunks of data are sent only if previous chunk was sent successfully.
  4. Finally MD5 hash of data is sent so the server can verify that what was received is correct.

On server side I decided to store chunks for a user in session which in case of Google App Engine is stored in Memcache backed by Data Store. What happens on the server:

  1. Server receives a handshake and creates new Array for data chunks (or removes previous one if transfer was not completed or not successful).
  2.  Server receives data chunks and places them in the Chunk Array stored in session.
  3. When server receives Kanban hash it concatenates the Array in order into String and validates against received hash. If the hash is valid it stores the Kanban in the Data Store.

Details and code samples

The uploaded Kanban is Encrypted with user key and kept on the Cloud encrypted to ensure data privacy. I’ve chosen Rabbit as encryption algorithm. As a side effect to encryption, transmitted data doesn’t need to be encoded and sanitized.
For encryption I’m using fantastic CryptoJS library. CryptoJS also includes MD5 hash implementation.


md5Hash : function(stringToHash){
	return CryptoJS.MD5(stringToHash).toString();
},

encrypt: function(stringToEncrypt, encryptionKey){
	var utfEncoded = CryptoJS.enc.Utf8.parse(stringToEncrypt);
	return CryptoJS.Rabbit.encrypt(utfEncoded, encryptionKey).toString();
},

decrypt: function(stringToDecrypt, encryptionKey){
	var notYetUtf8 = CryptoJS.Rabbit.decrypt(stringToDecrypt, encryptionKey);
	return CryptoJS.enc.Utf8.stringify(notYetUtf8);
}

Thanks to fantastic Angular Promise API it is very easy to implement ordered transmission of data.

var encryptetKanban = cryptoService.encrypt(kanban, this.settings.encryptionKey);
var kanbanInChunks = splitSlice(encryptetKanban, 1000);

var promise = sendStart(kanbanInChunks.length);
angular.forEach(kanbanInChunks, function(value, index){
	promise = promise.then(function(){
		return sendChunk(value, index + 1);
	});
});

return promise.then(function(){
	return checkKanbanValidity(encryptetKanban);
});

My Personal Kanban is Open Source project and available for browsing at https://github.com/greggigon/my-personal-kanban/

Conclusion

The above approach made it possible to upload data to the server from any browser eliminating issues related to default browser’s security settings.
Unfortunately it comes with a cost of more HTTP requests and custom server side coding.

For small amount of text data it is good enough for me, perhaps it would be good enough for you. Let me know your thoughts.

Mixture of technology at Google App Engine (GAE)

For some time now I am developing my little hobby project. It is called F1Dashboard and it contains informations related to Formula 1 (my favorite motor-sport).

F1 Dashboard
F1 Dashboard

Since the site is quite stable now I will share with you some technologies and ideas I used.

Platform

F1Dashboard is build and deployed on Google App Engine (GAE). I started this as an experiment and as a toy project just to learn something new.

GAE is not a new idea, it is another platform for cloud computing. Main difference from others is the Database model. GAE is using a thing called Data Store (Big Table), which is not a normal relational database that anyone used to. There is no schema defined, and querying is impossible without indexing it before. It is something like a massive bag that could store stuff in.

GAE is offered as Python and Java. The later was released long time after Python. It has limited version of JVM environment and a whole bunch of Google services to use. Lots of standard JDK types got repackaged by Google just to limit standard Java functionality, like spawning a Threads.

GAE offers standard Java servlets as web technology. This means that you could use some of the well established frameworks out there. I decided to create one myself of course 🙂 Something simple and not complicated.

Language

I choose Groovy, just because I wonted to learn another dynamic language and I liked the way Groovy did closures.

Patterns

The framework I did follows MVC pattern. It has single servlet entry. Every requested URL is parsed internally and resolved into Controller and Action. I’m using String Template as rendering engine for my views.

All database operations are entirely hidden behind Repositories, and I made it simple to switch from Objectify to Data Nucleus to anything else that comes.

I choose Objectify for its simplicity. If you are starting with GAE I would very much recommend it.

UI

The design is simple and was done by fried of mine Mike from Rosslyn Digital.

I plugged in jQuery Tools for Overlay effect. It also has a nice plugin for picture gallery.

I also added automatic Twitter updates for every message that is updated on a site.

Updates

Site is automatically harvesting for new content from various of Formula 1 related sites. There is also content from Twitter, YouTube and Flickr. It is utilizing GAE cron jobs for that.

I’m trying to gather some ideas for what I can do with the site in near future. If you have an idea, drop me an email. You can also see latest updates on a site at http://blog.f1dashboard.com

Cheers, Greg

F1 Dashboard released

In last few months I was working on a little personal project. When Google App Engine team announced that preview of Java version is available I decided to give it a try. When I was looking for a subject of my application, one of the ideas started to emerge more that others. I was looking for a web site that could aggregate all the information about Formula 1 racing. Because there was none available (or my Google search ended on first few results that were not satisfactory) I decided to make one.

Few weeks and cups of coffee latter F1Dashboard.com is released.

F1 Dashboard screen shot
F1 Dashboard screen shot

Generalities

So, in few words, F1 Dashboard is agregator of news, and media feeds from world of Formula 1. I decided not to collect content, just store information where to find it and short description.

News are harvested from a number of web sites. Twitter updates serves as another source of news.

Images are comming from Flickr, videos from YouTube.

All updates  are happening every few minutes and are 100% automated.

Technicalities

GAE supports Groovy as one of JVM programming languages. I decided to give it a try. I love Groovy, it is somewhere in between a friendly and known Java API and Ruby programming language.

It took me a moment to use to BigTable type of DataStore. It restrictes a way I used to work with databases normaly. Google provide entire environment for local development, whitch means I don’t have to upload to a cloud to see working result.

I created a little homegrown Model Views Presenter/Controller framework and used StringTemplate as rendering engine.

jQuery is the library of choice to handle all JavaScript.

Page styling was done by a friendly and kind designer from Circa82 Michael Austin (thanks buddy).

Come on guys, give it a try and tell me what you think 😉 Feedback greatlly appriciated. http://www.f1dashboard.com

Cheerios, Greg

Prototypal nature of JavaScript

Hated by many, loved by others. JavaScript is undeniably one of those troubled languages. The fact that it’s implementation and support is not compatible across different web browser makes it even worse and possibly hated by web developers. I would like to look closer into one of the JavaScript features, prototyping.

Some facts

JavaScript is object oriented language. It is “so much” object oriented that even Functions are first-class objects. It could be a surprise for a programmer (like myself) who used to classical object oriented approach, used in Java or C#.

It means that Functions can have properties, another functions, objects etc. Functions could be use to change state of the object or make calculations. They could also be used to create another Functions based on original.

Classes and Prototypes

Classical object oriented language is using types of object, or classes to describe it. In Prototypes one object is describing other, it is almost comparable to cloning.

To make it easier to understand I’d like to use this example:

  1. Using classical approach if I asks someone to make me another Aston Martin, I would be asked to get technical specs for it so it could be build it.
  2. Using prototypal approach if I ask for Ferrari I will be asked to bring one so they can look at it and build it.

I can define my object and if I need another copy of it I just use prototype to get the same one. In JavaScript world it also means that once I got a bunch of objects that have the same prototype, if I add new property to it I will be able to use it in all of them.

All objects have one common prototype that is Object prototype. Prototype for functions is Function prototype. One useful object method to determine if some object’s property is its own or inherited from its prototype is hasOwnProperty method.

Few examples

To create new Object in JavaScript we don’t need to describe it, just create it:

var myObject = { fooProperty: ‘some value’, barProperty: 2 };

Now if we would like another copy of it we could type something like this:

var Tmp = function(){};
Tmp.prototype = myObject;
var newObject = new Tmp();

You might wonder why is this necessary, looks messy and is not clear. Simple construction :

var newObject = {};
newObject.prototype = myObject;

should work. The reason that it is not is because in first line you are creating the object. Once it’s created it’s got Object as prototype its prototype. Adding another prototype to it’s chain of prototypes will not automatically make it inherit all the properties of another prototype.

Adding new property to prototype could be done using this contruction:

var newObject = {};
newObject.prototype.foo = “some text”;
newObject.prototype.bar = function(){
// do something
}

Multiple prototypes, chain of prototypes

It is impossible to define multiple prototypes for an object.

var objectA = { foo: ‘blah’ };
var objectB = { bar: ‘blah two’ };

var Tmp = function(){};
Tmp.prototype = objectA;
Tmp.prototype = objectB;
var newObject = Tmp.new();

In here newObject will only inherit properties of a last assign prototype and it would be objectB.

If you want to inherit multiple prototypes you have to get Function to help you.

function A(){
    this.foo = ‘blah’;
};
function B(){
    this.bar = ‘another blah’;
};
B.prototype = new A();
var c = new B();

Little help from jQuery

jQuery has a method that helps us to extend one object with some properties from another. Here how you can use it:

function A(){
   this.foo = ‘some text’;
}
var anotherObject = { bar: 2 };
$.extend(A.prototype, anotherObject);

 

Cheers, Greg

Verify JavaScript with JSLint during build using Nant

image Project I’m working on has a lot of JavaScript. We are minimizing JavaScript files during the build and combining them into one file. Problem is when someone checks in some invalid JavaScript into repository that doesn’t minify properly. As a result you can end up with not working JavaScript at all. There is a tool called JSLint. It is created by Douglas Crockford who is also the author of a very good book about JavaScript: “JavaScript: The Good Pars”.

JSLint is a very strict JavaScript verifier. JS files corrected to a JSLint specification could be minified with confidence.

We made the decision that build should brake when JSLint detects invalid JavaScript file, so developer will need to fix it before the code checkin.

Prepare the tool base

As a first step we need to get all the tools that will execute JSLint validation against our codebase. You can grab fulljslint.js from the JSLint home page. There is no executable binary for JSLint, it is just a JavaScript file with set of rules and function to execute verification. Our working environment is Windows we can execute validation using cscript.exe command line tool that Windows has. There is a very useful script created by Jason Diamond that helps to execute JSLint validation. You can get it from from here: jslint.wsf. One last thing is a command line batch file that helps execute this tool. You can name it whatever you would like to. I named it jslint.bat. The content of it looks like this:

@cscript //nologo %~dp0\jslint.wsf %*

I put all those three files into build tree, in tools folder. You can check if all the files are valid and working properly by running this command from command line:

jslint.bat c:\path\to\your\js\file.js

Create a build target

Now that we have all the tools we can create a build target. We are using Nant as a build tool. I created Nant target and called it jslint. It looks like this.

<target name="jslint" description="validates JS files">

    <foreach item="File" property="jsfile">

        <in>

            <items>

                <include name="path\to\scripts\folder\**\*.js" />

            </items>

        </in>

        <do>

            <exec program="jslint.bat" commandline="${jsfile}" />

        </do>

    </foreach>

</target>

There is a lot of stuff that could be turned on/off during validation. If you type jslint.bat without any parameters it will give you a list of possible options. Those could as well be tweaked in the jslint.wsf file.

Hope this helps you as it did for us.

Some says that JSLint might hurt your feelings. I say, bring it on 🙂

Gregster