Build script and project level variables (Gradle)

Following on my Introduction to Gradle here is a simple tip on script level variables.

Variables

I found out today that declaring something on the top level of a project build file doesn’t mean that all the Gradle blocks will see it. The problematic one is buildscript {}. It happens as buildscript section is evaluated before any other part of the script is.

So, for example:

myVariable = “I was here”

allProject {
  println “It will be here $myVariable”
}

buildscript {
  println “But it won’t be here $myVariable”
}

Defining variables on top level could be useful as they could be treated as constants and reused throught the script (I use those for repository URLs for example)

The easiest and simplest solution would be to put buildscript{} as a first block and declare all the variables inside.

buildscript {
  myVariable = “I was here”
  println “Here $myVariable”
}

allProject {
  println “.. and here $myVariable”
}

Buildscript block doesn’t have to be at the top but I feel it’s more elegant and clear as all the constants are on the top of the file that way.

Introduction to Gradle

For last few weeks I’ve been moving rather large collection of Ant, Shell and other scripts into Gradle. In my opinion Gradle is great. It has two things that I value the most when it comes to tools:

  1. The build scripts are CODE. As scripts are CODE I can do everything I could do when I code. It gives me freedom from XML
  2. It has dependencies management powered by Ivy.

Scripts are Code

The build scripts are Groovy code. Groovy is dynamic language that runs on JVM (similar to JRuby in some sense).  This means that I can test my build scripts with unit tests. This also means that I can do loops and conditions easier without weird, artificial XML tags.

Dependency management by Ivy

Gradle team, instead of inventing new dependency management, actually decided to use Ivy. Simple and easy tool that does what it says on a tin, nothing more. More important it will not try to download the Internet every time it runs.

The beginnings

First of all, if you would like to follow me with this article, grab yourself installation of Gradle. It is self-contained zip, and the only thing that you would need is Java installed.  Verify your Gradle installation by typing gradle –v from your command line.

Gradle expects your build scripts to be in a project folder in a file called build.gradle. You can write your scripts in another file but every time you execute Gradle you will have to specify where your build script is: gradle –b file_name.gradle.

To execute a task defined in build.gradle file you need to type: gradle taskName.

To see the list of all available tasks in for project type gradle tasks from command line.

Write you some tasks

Gradle has a concept of a task. Task is combined of multiple actions. One can define a task as new and assign some actions to it, or use predefined Types of task (Copy, Exec, etc.).

Example of new task definition:

task doStuff << {
  // Hey hey, I’m doing stuff
  println ‘Doing stuff’
}

Example of task that copies files:

task copySomeThings(type: Copy) {
  from ‘source’
  into ‘destination’
}

There is a very subtle difference between those two ways of defining tasks, and I got caught many times by it. The first task, doStuff doesn’t have predefined functionality. Calling method task doStuff creates task with name doStuff. However, I need to define some actions. This is the role of operator << (shift left). What actually happens is:

  1. Task doStuff is created
  2. Action that is defined in closure (between two curlies, {}) is appended to the end of the actions list of this task.  First and only action that will happen will be printed out into STDOUT.

If I need another action to follow the thirst one I defined I could do:

doStuff << {
  // Wow, now I’m doing another action
  println ‘Weheeeey’
}

The definition of copy task however doesn’t contain << assignment. It does contain the closure though. This closure is passed into the task Copy to configure it. from and into are methods defined on task type Copy that can be used to specify where to pick up files from and where to drop them. Every predefined task has some parameters and methods that I can specify. Please refer to documentation  (link to Gradle task list) for details of task.

This is important

The most common mistake I kept on making was defining a custom task and forgetting the << operator. The task did nothing and I was spending countless hours investigating why.

task doStuff(){
  // this will not happen
}

Task dependencies

In Gradle I can define dependencies between tasks. This means that when I want to execute specific task and there is another one that needs to be executed before it, Gradle will execute it too. The way to define task dependencies:

task doStuff() << {
  // I’m doing stuff
}

task doingEvenMore(<strong>dependsOn: doStuff</strong>) << {
  // Doing more
}

You can also specify name of the task instead of the task reference. This allows for kind of a lazy evaluation of build script.

task doingEventMore(dependsOn: ‘doStuff’) << { }

If you need to add dependency for already defined task you can use dependsOn method of a task.

doingEvenMore.dependsOn someOtherTask

Or dependsOn property of a task:

doingEvenMore.dependsOn = [doStuff, someOtherTask]

There is a catch with the last method: it overrides all previously created dependencies.

Then, there was Java

As of now I was describing how to create tasks and declare dependencies between them. To actually use it on a project we will need some more action. That’s why it is time to introduce Gradle Java plugin.

apply plugin:’java’

Java plugin adds some task to your build. You can check them out by executing gradle tasks from command line. Tasks that can compile, test and even jar your project.

What we are interested in are dependencies that project might need and the way Gradle handles them. By dependencies I mean third party libraries like: log4j, junit, etc. You can see dependencies in your project by executing gradle dependencies from command line.

Dependencies

Java plugin adds few dependency configurations. I’m interested in compile time dependencies, as it is needed for my project to compile and run.

apply plugin:’java”

repositories {
  mavenCentral()
}

dependencies {
  compile group: ‘log4j’, name:’log4j’, version: ‘1.2.9’
}

Inside repositories block I used method mavenCentral(). This is method predefined by Java plugin providing access to Maven central repository. As I mentioned earlier Gradle uses Ivy to resolve dependencies. Ivy in turn works rather well with Maven repositories.

Lines above will add single log4j dependency to mine project. The cool thing about it is that it will also resolve all its dependencies.

We can also add multiple dependencies by doing:

dependencies{
  compile(
    [group:’log4j’, name:’log4j’, version:’1.2.9’],
    [group:’xfire’, name:’xfire’, version:’1.2.6]
  )
}

This will add two dependencies to my project.

There are few more dependencies (runtime, compile, testCompile, testRuntime, archive) added by Java plugin. Dependencies are nothing more than a list of libraries (files) included in a build lifecycle (compile, test etc).

Configurations

One can define its own dependency configuration by creating new entry in configurations block of a build script.

configurations {
  gregWasHere
}

dependencies{
  gregWasHere group: ‘log4j’, name:’log4j’, version: ‘1.2.9’
}

Configurations are nice and simple way of adding compiletime or runtime libraries to any build. For example I used it when executing external Java process that needs extra libraries in classpath.

task executeSomeJava(type: JavaExec){
  main = “com.some.main.Bla”
  args = [“1”, “2”]
  classpath = configurations.gregWasHere
}

Summary

Gradle is really great and if you are still stuck in the world of XML and hate it, give it a try. Few tips I written in this article, could be used as a good starting point. I also found that coding to task convention that Gradle uses, simplifies the build and release process.

Few good resources to read when starting with gradle:

Broken Maven repository and Gradle dependencies

Few days ago I got into trouble when building project using Gradle.

Created deployables (fat jars) looked fine when I build it on my local machine, however when I build on remote box (Linux) I was missing some dependencies. The build scripts were exactly the same. The only difference was: the remote box didn’t have local Maven repository. This shouldn’t cause any problems as remote repository should be the same.

Gradle, a better way to build

However 🙂 on the project that we are working, the remote repository is managed by team and it is in our internal network. It also doesn’t proxy to Maven repository.

Some of the artifacts that got uploaded to that repository were missing pom files. When Gradle (via Ivy) was resolving the dependencies it created pom file on the fly with no dependencies on other artifacts.

When I discovered the problem I uploaded missing pom files. It didn’t fix the problem though. As it turns out, Ivy cached the self created pom files in USER_HOME/.gradle/cache folder. Running gradle with -C rebuild option doesn’t solve the problem as this option only clears the cached Gradle scripts.

When I deleted the folder USER_HOME/.gradle/cache, Gradle got all the dependencies resolved properly!

I was using Gradle milestone 3.

I hope you won’t get stuck on the similar problem as I did. Maintain you repository or use public.

Greg

Difference between Innovation and Creativity

The subjects of my recent posts are “creativity” and “innovation“. But what does do creativity and innovation actually mean?Discussing both terms with my colleagues, we couldn’t not notice that we sort of arrived at the same definitions. We did recognise that there was definitely a difference, but somehow couldn’t point out what it was. I decided to consult the Internet 🙂

Creation

Innovation:

  • A new method, idea, product (Google)
  • The central meaning of innovation thus relates to renewal or improvement, with novelty being a consequence of this improvement. (Wikipedia)
  • Something newly introduced, such as a new method or device (dictionary.reference.com)
Creativity:
  • The action or process of bringing something into existence (Google)
  • Refers to the phenomenon whereby a person creates something new (a product, a solution, a work of art etc.) that has some kind of value (Wikipedia)
  • The act of producing or causing to exist (dictionary.reference.com)
Reading definition of both those terms it is rather clear that the difference is very subtle. However:
  • Every Innovation is a Creation of something new. Innovation is a Creative process.
  • Not every Creation leads to Innovation. Creating new processes/devices/art doesn’t mean that something was renewed or improved.
Thoughts?

Design release process from very beginning

Creating good software that meets customer expectations is 50% of success. The other 50% is getting it to the customer in a most quick, easy and smooth matter as possible. This process is called releasing software.

breaking chain

Release flavours

Releasing software comes in two flavors, one is a clean slate installation, second is version/maintenance upgrade. Where first one is rather simple the latter could be quite problematic. The more complex the application and environment it runs in, the more problematic it’ll be.  Releasing software to running cluster of application servers with data migration and without downtime could prove challenging for the toughest release gurus.

I always thought of release in a same way I receive or send parcel. When I have something to sell I organize it online, Delivery Company comes to pick it up and it’s on its way. I can then track it online to see the progress my parcel is making.

It is exactly how I would like any release process to be. When my package (new version of software) is ready I would like to press the button and have it on its way. All I should do is monitor if something went wrong and what if it did.

Automation

Automation of the process is a must if one wants to remove wasted time on same repeatable steps. It is quite often automation that gets neglected from the beginning of software life, until the point when release becomes unmanageable manual monster.

Automation should actually be planned from the very beginning. Whenever I start to work on a project I ask myself what do I need to do to get the software to end user as quick as it’s ready with as less possible effort. This is the beginning of the release automation adventure.

Releasing often and quickly adds into the whole iterative development process, combining the whole as one iterative delivery. Process reduces time that takes for the requirement to reach end user as a piece of functionality. That makes end user happy and gives way to feedback. Software development team can use feedback to adapt and evolve the software.

So, what now?

It is important to design and implement release process from the beginning. It is also important to automate it. It will make it possible to ship as soon as piece of work is done. I know of teams where releasing is happening on a daily basis.

Automate every step that requires human interaction. Look for candidates in places where the only manual thing is pressing a button or running a script.

If release process requires monitoring (e.g. waiting for application to stop before continuing release process), try to automate that as well.

I also found that usage of tools that support coding the process, not configuring it via XML, works much better and are more flexible and easy to use.

Little creative fingers

We all have heard about creativity but did we ever take a moment to think about it for a while. What it really is or what it means? How to be better at it?

Creativity

Definitions of creativity are actually rather simple. Wikipedia states that Creativity is the ability to improve, adding value. Google quotes Princeton University’s website saying that it’s “The ability to create”.

Creativity is a mental process, involving discovery of new ideas, or new view on the old ideas. The process is powered by our conscious and unconscious mind.

With today’s modern psychology and cognitive science, creativity is still rather unknown field. There are number of theories on the process, but all of them can’t explain it precisely.

Graham Wallas presented his theory about creative thinking in 1926. He is dividing creative process into following steps:

  1. Preparation – work on a problem that involves understanding it and exploring different views on it
  2. Incubation – where the problem is injected into unconscious mind
  3. Intimation – it happens when one get the feeling that idea is going to emerge soon
  4. Illumination – where the creative idea bursts into conscious mind
  5. Verification – when the idea is consciously verified, elaborated and applied

My own findings

I started with reading some book on the topic and browsing the Internet. Then I saw TED talk by creativity and innovation expert, Sir Ken Robinson. He inspired me to observe my 3 years old daughter when she was playing. They say that small persons mind is like a sponge. It soaks in all the new encounters.

Things I noticed

A lot of my daughter’s new toys emerged from an item of everyday use, used in a completely new way (laundry basket became a boat, toilet paper roll became a telescope, etc.). She has a great skill to take an item, turn it around, look at it from different angle and than decide what new toy it will be.

My daughter is very good girl and she is hardly ever destructive  (I only assume that boys are more talented in this area). She grabs things that are around her and puts them together. New creation is inspected and either accepted as new toy or discarded (sometimes put together in different way).

Kids have more simple/different way of looking at things. This is what makes them so creative. When “Little Prince” saw picture of a hat, he said it was a picture of giant snake that swallowed elephant (perhaps he was right).

Thinking about the way my daughter creates new toys I came to conclusion that creative adult thinking is very similar most of the times. We look at the things from a different angle, look at it upside down, take tools that we have never used to do different sometimes odd job (using shoe as a hammer).

Putting things together is less physical, more mental. It involves taking pieces of information, knowledge, experience, and combine them together to produce something new. A book, real life observations, notes and mind maps turned into this article.

Recall any music interview that you might hear. Musicians typically mention who is their inspiration. Normally they mention another musician/music style etc. You can recognize in their music, tunes from other musicians, same musical patterns, instruments, etc. This is being creative by combining all the bits together in a new way.

Creativity and software development

Above description of creative processes are sounding very familiar to what I’ve been doing everyday creating software. All my experience from working with other people, technical knowledge I have, domain knowledge I acquired and everything else I know influences the code that I write.

I found that knowledge of different languages and patterns that are present in those, helped me to bring new ideas, become more creative at what I was doing. C# experience helped me to introduce some new patterns to Java. Dynamic and functional languages gave different view on type safety and state. Tools that I typically used in one technology, gave new light to a better use of tools in other technology.

I also found that being brave enough to try new things, use of different tools, led me to a new discovery. For example, swapping build tool to brand new, helped defragment not fully automated release process.

Summary

Everyone can be creative however it doesn’t mean that it is simple. More information you have (including domain, technical knowledge and everyday life things) is helpful. More willing you are to try new and different things the better.

Try to stay open to information but be careful, as it is easy to get overflowed with it in the age of Internet. Be open to new things and listen to other’s ideas, don’t shut them down immediately.

Handfull of links (somehow references)

Books:

Internet:

Organisation change in disguise

 

There is a lot of tweets in the Twitterverse about changing organisations, people, processes. Not too long ago I was a consultant with “change everything” mind set. I was also surrounded by like minded people. We tried together in our crusade against bad, bad things in software development industry. I don’t feel we have achieved substantial success.

After many attempts I realised the biggest problem that we have never tackled properly. PEOPLE ARE AFRAID OF CHANGES. This rather simple psychological thing has no technical or economical background.

However, there is something I observed during my attempts at the place I work.

 

On the other hand I observed that adopting different approach in the place I work at the moment produces better results.

I started as usual with the same consulting spiel of “we need to change“. Even with the people who saw problems it was hard to convince them to change. I got a little bit demotivated at work.

Disguise

Coming back from holidays gave me new energy and new will to tackle problems. This time however I change the spiel from “we need to change” into “how about I improve this thing by doing that“. I found that in most cases there was very little resistance for improvement. Very often I tried to hide the change behind the innovation.

So, my little advise is: don’t change where/what you can’t. Disguise the change behind innovation and improvement. The change will happen by itself, the difference is: people won’t be scared of it. It also goes much better with management as they fear big bang changes the most.

Greg

Innovation, academia VS industry

Last week I was presenting the paper me and my collogues wrote. You can find it here: Enabling Testing, Design and Refactoring Practices in Remote Locations.  The paper was accepted for ICST 2011 conference workshop, RefTest. It is a short paper about problems and some solutions for practice transfer in distributed team.  However this is not what I would like to write about.

The workshops gave me something to think about. The whole conference was targeted at bridging industry and academia together. Unfortunately at the track I attended there was only two papers and presentation from the industry (one was mine).

I was hoping to hear some new innovative ideas that I would be able to take into my everyday work. After a whole day of workshop sessions I was left with no extra ideas. Plenty of smart academic people were presenting research that I found rather useless in everyday life.

I was also little disappointed after the presentations, as none were actually summarizing the research in a way useful for the industry.

I thought of all those blog articles, posts, tutorials, video presentations, etc. from industry people with the ideas and the way of using it. Of all the technical innovations, processes, practices that got discovered in last few years.

Does it mean that academia lost their leading role in software development industry? There is very little in press about new discoveries in the field of software development.

Thoughts?

Greg

 

Evernote as blog software

From Evernote:

Evernote as blog software

I was looking for a good software to use as offline writing tool for my blog. Something in the line of the great Windows Live Writer. Unfortunately some good candidates for Mac were not free.

I’m using Evernote to store notes of different kinds for my ideas and research. Idea just popped into my head that I could use one of the Evernote’s notepads to create blog entries. Each note could be a separate entry.

Evernote has editor that enables me to style it. The only thing that is missing is the “Post to blog button”. WordPress.com gives a possibility to post by email though. I will give it a try now.

If you see this post it means that it was a success 🙂

If by any chance you are a Evernote developer or could influence the features, you might consider my simple idea.

Cheers, Greg

Edited after this line.

As you can see Evernote adds additional header. Pity as it could be such a great way of hacking Evernote and WordPress.com together 🙂

Innovation at Google

Light bulbLast week at CQon London 2011, one of the day’s opening session was Patrick Copeland from Google, talking about the way they innovate. Thoughts that speaker was trying to get across:

  • Idea is only 1% of the successful innovation, 99% is constant repetition of try and fail cycle. Thomas Edison as an example of the cycle that lead to invention of a light bulb.
  • Every idea can be a winner or a loser. Twitter was never suppose to catch up and Windows CE was suppose to be the greatest mobile platform.
  • Surround yourself with innovators, not thinkers. Look for track record of innovation in portfolio/cv.  Innovators beat ideas.
  • Build fast, build the THING, and don’t worry about the RIGHT way of building it. Just get it out. Iterate fast. Dare to fail.
  • Create Pretotypes. Google’s Android application design kit contains a paper clip and a pencil. Draw your idea, application on a piece of paper. Put it into pocket and try to use it like a real thing.
  • More time you spend on the idea without releasing it, the more in love you’ll be with it. Therefore less likely to dump it.
  • Data is apolitical and factual. Use data not opinion to decide upon idea, if it’s worth doing, if the prototype/prototype/beta is getting more popular, etc.

Are you feeling like innovating something now? Go and do it 🙂