Solving the ‘Hero’ Paradox: Strategies for a Resilient Software Development Culture

In the intricate world of software development, one phenomenon occasionally takes centre stage – the emergence of a “Hero.” This individual swoops in amidst crisis, solving problems that no one else can. On the surface, they may seem like a team’s greatest asset, but a deeper dive often reveals hidden complexities. This article explores the phenomenon of the “Hero” in software development, its implications, and strategies to navigate it effectively.

Understanding the ‘Hero’ Phenomenon

The “Hero” in a software development team often arises due to various motivations. For some, the recognition and affirmation of being a problem-solving champion can be irresistible. Others might be driven by a profound sense of responsibility or the fear of the project spiralling into failure. For a select few, the sheer thrill of problem-solving under pressure is motivation enough.

Whatever the cause, it’s essential to realise that this dynamic, while sometimes beneficial in the short term, often holds hidden perils.

The Impact of the ‘Hero’ Issue on Teams

The “Hero” issue can significantly disrupt team dynamics. With one person shouldering most of the workload and responsibility, a dangerous imbalance can form, leading to increased risk and dependency on a single individual. It’s akin to putting all your eggs in one basket.

More subtly, this dynamic can stifle growth and learning for other team members, who may be relegated to bystander roles. Consider the case of DevOps engineers being the sole knowledge bearers of specific activities for the release, configuration, and setup of non-production environments. It can leave the team in a vulnerable state should the DevOps engineer be unavailable.

The ‘Hero’ Issue: Implications for the Organisation

While the impact of the ‘Hero’ issue on teams is substantial, its repercussions extend to the broader organisation. This single-point dependency creates significant risk, primarily in terms of business continuity.

In the event of the “Hero’s” unexpected absence, whether due to illness, attrition, or even a well-deserved vacation, business processes can grind to a halt. Without their unique knowledge and skills, teams might struggle to maintain the pace of operations, leading to delays, cost overruns, and customer dissatisfaction.

Moreover, the ‘Hero’ issue can inadvertently stifle innovation. If one person continuously resolves crises, the organisation may become over-reliant on their methods, stifling the emergence of fresh ideas and novel approaches. Over time, this can lead to stagnation and reduced competitiveness in the fast-paced world of software development.

Recognising and addressing these risks is paramount for an organisation’s long-term success and sustainability.

Addressing the ‘Hero’ Issue

The first step to mitigating the “Hero” issue lies in fostering a culture of shared responsibility. Encourage all team members to contribute their expertise and partake in problem-solving. However, creating this culture extends beyond just day-to-day activities.

A practical performance framework is an essential tool to combat the ‘Hero’ phenomenon. It’s important to define roles, expectations, and responsibilities clearly. This doesn’t just create transparency; it ensures that the performance of each team member is evaluated on a fair and comprehensive basis rather than the frequency of ‘heroic’ acts.

Next, establish robust knowledge transfer mechanisms. This can be through detailed documentation, pair programming, or regular team meetings to share insights and learnings. Project management strategies can also effectively ensure that workloads are evenly distributed and dependencies are minimised.

Lastly, promoting a culture of continual learning and upskilling within the team can help ensure everyone has the confidence and competence to tackle challenges, reducing the need for a “Hero” to swoop in.

Conclusion

Addressing the “Hero” issue is critical for fostering an inclusive, balanced, and cooperative team culture in software development. While it’s easy to laud the efforts of a singular hero, it’s essential to remember the immense power of collective strength, knowledge, and expertise. After all, software development is a team sport, and its true champions are those teams that work seamlessly together, each member contributing their unique strengths towards a shared vision of success.

The Power of Quality: Why Your Software Needs More Than Just Correctness

When developing software, many people focus on one aspect above all others: functional correctness. While this is an important aspect of software development, it’s not the only one. Two other vital elements should be considered: software quality and software testing.

So what is the difference between software quality and testing, and why are they so important? Let’s take a closer look.

Software Quality

Software quality refers to the overall quality of a piece of software, considering factors such as reliability, maintainability, and usability. In other words, it’s about more than just whether the software works correctly or not. It’s about how easy it is to use, how easy it is to maintain and update, and how reliable it is over time.

High software quality requires careful planning and attention to detail throughout software development. It includes everything from writing clean, well-organized code to designing intuitive user interfaces to rigorously testing the software to ensure it meets all its requirements.

Software Testing

On the other hand, software testing is focused explicitly on verifying that the software works correctly and meets all its requirements. The process involves designing and executing test cases that exercise the software in various ways to ensure it behaves as expected.

While software testing is integral to software development, it’s important to remember that it’s just one part of the process. There needs to be more than testing to ensure high-quality software. You must take a holistic approach that includes careful planning, design, coding, and testing to achieve that.

Why It Matters

So why is focusing on software quality rather than just functional correctness so important? The answer is simple: because software is rarely used in isolation. In most cases, it’s part of a more extensive system that includes other software components, hardware components, and human users. As a result, if the software is difficult to use, unreliable, or hard to maintain, it can negatively impact the entire system.

For example, imagine you’re using a piece of software that is difficult to use and prone to crashing. Even if the software technically meets all its functional requirements, it will still be frustrating and time-consuming to use. In addition, it can have a ripple effect throughout the entire system, slowing down other processes and potentially causing errors or delays.

By focusing on software quality and not just functional correctness, you can ensure that your software is reliable, easy to use, leading to a better overall user experience and more successful outcomes.

Summing up

While functional correctness is an essential aspect of software development, it’s not the only one. To truly ensure that your software meets the needs of your users, you need to focus on software quality and testing. By taking a holistic approach to software development, you can help ensure that your software is reliable, easy to use, and easy to maintain, leading to better outcomes for everyone involved.

The Price of Chaos: Balancing Standardisation and Creativity in Software Development

As a software engineer, I once worked for a large organisation that was experiencing continuous growth. However, I found myself in the midst of a chaotic environment where every team was using a different programming language and framework. While the situation seemed exciting initially, it soon became apparent that it was unsustainable. As the number of projects grew, the cost of maintaining all these different technologies began to spiral out of control.

As a solution, the company’s technical leaders decided to standardise on a single programming language and framework across all projects. The benefits were clear: reduced maintenance costs, improved collaboration and better knowledge sharing. It was a no-brainer, right?

Well, not quite. While standardisation can effectively reduce costs and improve delivery speed, it can also harm creativity and innovation. As software engineers, architects and technical leaders, we must know the potential trade-offs and take a measured approach when considering standardisation.

On the one hand, standardisation can help with cost reduction by reducing the number of technologies needing maintenance. But on the other hand, it can also speed up delivery by allowing teams to share knowledge and resources more efficiently. As a result, it can benefit large organisations with many teams working on different projects.

However, standardisation can also stifle creativity and innovation. When we limit ourselves to a single programming language and framework, we may miss out on new ideas and approaches that could lead to project breakthroughs. We may also lose talented engineers who prefer to work with different technologies. As technical leaders, it’s essential to balance standardisation and creativity.

So, what can we do to strike this balance? Here are a few tips:

Consider your organisation’s needs: Every organisation is unique, and there is no one-size-fits-all solution. Consider the size of your organisation, the number of projects you have, and the resources available when deciding whether or not to standardise.
Involve your team in the decision-making process: Your team members are the ones who will be working with the chosen technology daily, so involve them in the decision-making process. It will help ensure that the chosen technology is a good fit for your team and that they are motivated to work with it.
Allow for experimentation and innovation: Even if you choose to standardise, it’s essential to allow for experimentation and innovation. Encourage your team members to try new tools and approaches and be open to new ideas and suggestions.
Keep an eye on emerging technologies: The technology landscape is constantly changing, so keep an eye on emerging technologies that may be relevant to your organisation. Feel free to explore new technologies and incorporate them into your tech stack if they make sense.

In conclusion, standardisation can be an effective way to reduce costs and improve delivery speed, but it can also have a negative impact on creativity and innovation. As software engineers, architects and technical leaders, it’s essential to strike a balance between the two. By considering your organisation’s needs, involving your team in the decision-making process, allowing for experimentation and innovation, and keeping an eye on emerging technologies, you can find the right balance for your team and projects. And one day, you too will find yourself in a situation where you must choose between chaos and standardisation!

Easy way to use multiple GIT accounts with SSH keys from single computer

I’ve been using a single machine for multiple GIT accounts. I would often get stuck when trying to push/pull changes while using wrong SSH keys or creating commits with invalid username and email combinations. The last bit was incredibly annoying as I would have to review commit history and amend all invalid commit messages.

I do use the GIT command-line client from my Mac, authenticate over SSH with keys. I also use ZSH shell with Oh My ZSH and the excellent Powerlevel10k theme for ZSH.

Researching articles on the Internet, I didn’t find a solution that would work efficiently for me from the command line. As Oh My ZSH is easy to extend (as I found out after reading a bit of documentation), I created my solution.

Git profile switch in action

The switching functions, git_switch_to_personal and git_switch_to_work perform two tasks:

Switches the files in the .ssh configuration folder to the appropriate SSH keys
Sets up global git configuration with “user.email” and “user.name” parameters.

I also modified ZSH prompt to show me which profile I have set up currently. The same information can be obtained by executing git_what_profile command.

Following are what’s required to get the functionality sorted.

Switching SSH keys and GIT configuration

I created git_switch.zsh file in the ~/.oh-my-zsh/custom/ folder. When a new shell is started, the file is loaded, and new functions become available.

git_switch_to_work() {
  rm -rf ~/.ssh/personal ~/.ssh/work && rm -rf ~/.ssh/id_rsa ~/.ssh/id_rsa.pub
  cp ~/.ssh/id_rsa.work ~/.ssh/id_rsa && cp ~/.ssh/id_rsa.work.pub ~/.ssh/id_rsa.pub
  git config --global user.email "greg.gigon@work-email.com" && git config --global user.name "greggigon-work"
  touch ~/.ssh/work
}


git_switch_to_personal() {
  rm -rf ~/.ssh/personal ~/.ssh/work && rm -rf ~/.ssh/id_rsa ~/.ssh/id_rsa.pub
  cp ~/.ssh/id_rsa.personal ~/.ssh/id_rsa && cp ~/.ssh/id_rsa.personal.pub ~/.ssh/id_rsa.pub
  git config --global user.email "greg.gigon@personal-email.com" && git config --global user.name "greggigon"
  touch ~/.ssh/personal
}

git_what_profile() {
  if test -f ~/.ssh/personal; then
    echo personal
  else
    echo work
  fi
}

The functions assume that I have two pairs of SSH keys in the ~/.ssh/ folder, one with keys I use for authenticating with personal GIT account and second for authenticating with work GIT account.

The function removes the default SSH keys and copies the selected as defaults, id_rsa and id_rsa.pub. Also, functions create empty files, which indicate the active profile.

Adding GIT profile information to the ZSH prompt

To make permanent changes to the prompt, I had to edit ~/.p10k.zsh file.

First, I created the function with the following content:

function prompt_my_git_profile(){
    if test -f ~/.ssh/personal; then
      p10k segment -s PERSONAL -f green -t "personal"
    else
      p10k segment -s WORK -f red -t "work"
    fi
  }

The function checks for the existence of the files indicating the active profile. The file contains an example prompt function; a great to put the newly created function.

I also had to add the element to the prompt by modifying POWERLEVEL9K_LEFT_PROMPT_ELEMENTS variable (at the top of the file).

# The list of segments shown on the left. Fill it with the most important segments.

  typeset -g POWERLEVEL9K_LEFT_PROMPT_ELEMENTS=(
      # =========================[ Line #1 ]=========================
      os_icon                 # os identifier
      dir                     # current directory
      my_git_profile
      vcs                     # git status
      # =========================[ Line #2 ]=========================
      newline
      # prompt_char           # prompt symbol
  ) "

Note that the function is called prompt_my_git_profile but when adding it as one of the elements, I had to drop the prompt_ prefix and only use the my_git_profile in the name.

You can adapt the functions to work with more than two profiles.

Enjoy the simple hack.

How safety at work improves creativity and productivity

I finished 2019 with a short holiday. I used the time to reflect on the journey past year. In December, I was at the “AWS re:Invent” conference. Amongst many sessions, I particularly enjoyed those that focused on Software Engineering processes and practices. During one of the presentations, the speaker said a sentence which very much engraved itself into my mind:

“The most productive and creative work happens in environments where it’s safe to experiment. To get most out of your Engineers, create safety instead of restricting freedom by introducing draconian rules and policies.”

Working for many years in tightly controlled environments, I realised the importance of the above statements. I also realised the many dimensions of safety to which the speaker was referring.

Impact of tight rules and policies

Many organisations, specifically large organisations, introduce regulations and procedures to prevent undesired situations from happening. In my experience as a software engineer, I had to deal with tight controls of Internet access, permissions of my development environment, access to test environments, curated list of software and dependencies.

Those rules were not only frustrating and inconvenient for other developers and me, but were also preventing us from trying new things, improving existing solutions and slowing down the delivery process.

The frustration resulting from the blockers, caused me and my colleagues to find creative ways to work around the restrictions, by “hacking” processes. We wasted our creative efforts on the wrong outcomes.

The sad truth is that regardless of the rules and policies undesired situations happened. Back in the days, you could hear about security mishaps and performance issues on the BBC.

When I think about the quote from the beginning of the post, I know that fruits of my work might have been more creative and take much less time if I would have the freedom to experiment and progress safely.

Safety comes in different shapes and sizes.

As a software engineer, when I talk about safety, I immediately think about safe development environments where I will cause no harm to the production systems. However, there is another dimension to it: a safe environment in a company, where:

it’s fine to voice opinions and raise concerns
there is no prejudice, bias or harassment
no blame is assigned.

No amount of technical solutions will make a difference in your organisation unless you make people feel safe to express their thoughts honestly and openly.

How to create safety

road_block_driving_safety

As leaders and managers, we create a safe environment by being transparent and open. When our people realise they don’t have to worry about speaking their mind and challenging current approaches, they would have time to focus on existing work and bring ideas for improvements.

Safety is a feeling, and different people might feel it differently. It might take time, and a different approach before people in your team feel safe. Start by asking yourself, what would/is making you feel safe at work? Adopt it as your behaviour towards others and iterate on the approach. I find it useful to get feedback on my methods during 1-2-1s.

When it comes to technical solutions, the following are some ideas on safety for experimentation:

Managed developer desktops, with no access to the corporate network, with permissions to add additional software at will
Sandboxed dev environments with no connection to delivery pipelines
Separate Internet access without restrictions on the development resources
Production-like data for experiments

Following ideas are for safety in the delivery work, as a part of development practices :

Automated functional and non-functional testing
An automatic, no-human-touch deployment pipeline
Reproducible environments
Configuration as code

I know that the last few points are essential DevOps practices. It’s not coincidental as they are designed for safety due to the fast-paced nature and the elimination of human error.

Remember, keep your workplace “safe”!

Naming things is hard … and very important

Have you ever watched “The Croods“? When my daughter was younger, I watched it many times. The Croods are family of a caveman trying to survive another day in the dangerous, prehistoric world. There is no doubt in the entertaining value of the movie, but the thing that I remembered the most are the scenes where cavemen are attempting to name new “things”. I always wondered how did they come up with names for things, how come “wheel” is a wheel? It is difficult even to imagine the thought process. Why not try yourself? Think of a different name for a “wall”.

800px-English-English_and_English-Persian_dictionaries

Naming things is hard. It’s even harder when you have to do it regularly during the day. If you are somehow puzzled to what profession requires to continually come up with proper names for things throughout the day, let me put you out of your misery; it is Software Developer or Software Engineer.

When creating software, developers have to name: variables, functions, objects, modules, systems, classes, etc. All of them would have behaviour, which requires a proper name as well. Proper naming is crucial because:

it communicates meaning
it helps collaborate
reduces time and cost of maintenance

Working with a team means that developers have to collaborate and incorporate each-others code in their work. They need to understand the meaning of the code, intention and behaviour to incorporate it within their work correctly.

Naming things in Software

Today, computer programs can contain thousands if not millions of lines of code. Without code being transparent, it is hard to modify it, change it, improve it. The naming of different elements of code is a massive part of making code clean.

Each programming language has its own set of conventions and practices when it comes to naming. Following those standards provides a mental shortcut and reduces cognitive load during the development process.

Naming standards are also crucial for aspects of the development process itself. Thanks to the common understanding of words and phrases like “build” and “release”, or “story” and “feature”, we know how to work with each other.

Naming things in the Development Flow

When working with code, we collaborate using Version Control Systems. The one that is the most popular at the moment is Git. Few services are offering Git repository hosting, with GitHub being defacto golden standard in the industry.

branches

Git is quite flexible with the ways of using it; however, teams typically adopt one of the standard models. GitHub or Atlassian has a great set of posts explaining the different models. The model that seems the most suitable for the Enterprises that worked 90% of the time for the teams in companies I worked for seems to be GitFlow. The flow prescribes names for branches. Those names are relevant as of the points mentioned above in my post. Branch names help with the understanding of:

where to put code that is Work in Progress
where to integrated completed code
where to stage code ready for creating a release candidate
where to put code that is officially released and in production
where to put code that fixes issues and bugs

I do like to make it easy for people to do the right thing and hard or impossible to do the wrong thing. Hence I created a little GitHub Application, that looks after names of branches in the repository and raises an issue if the name is wrong or deletes the offending branch.

You can check it out at https://greggigon.com/brunchyyy and https://github.com/greggigon/brunchyyy.

Final words

Correct naming is essential in software development. Naming standards provide mental shortcuts, reducing cognitive load. They also provide a shared vocabulary for communicating and building blocks of understanding. Pay attention to naming and help your teams adopting standards.

I’m going to leave you with this funny short Clip from “The Croods” illustrating how hard it could be to name things correctly 🙂

9 years in The Bank – lessons learned

I worked at RBS for almost a decade. During my nine years at the bank, I got the opportunity to work with very talented and smart people, and I learned a lot. I want to use this opportunity to summarise some learnings and experiences that I encountered during that time.

Large Organisations are like Big Ships

Large Organisations, like traditional banks, are like big ships, hard to get going, require an enormous amount of energy to move and a long distance to stop. When it finally departs, it’s impossible for it to quickly and swiftly change the direction. The organization is on the constant lookout for problems far into the horizon, to be ready for a direction change maneuver. Finally, if unexpected happens, just like the Titanic, the ship will not be able to avoid the obstacle and will crash into it. Make a mental note as this will impede decisions and choices you’ll have to make.

white cruise ship

Smart people do dumb things – the fallacy of targets

I’ve seen people make irrational decisions, sacrificing Software quality and Customer Experience for the sake of delivering to deadlines. Deadlines quite often self-imposed, making no commercial reason. Those decisions were driven by localized targets, imposed by line managers. Values that the organization pursued, the good of customer was forgotten because the way performance was assessed, was against the targets.
The result was a poor quality software, delivered at a high cost, and at the end had to be scrapped and re-written. Beware of personal targets as those are often in direct opposition to organization vision and objectives.

Smart people do smart things – if you give them some autonomy

I’ve seen and been working in the hugely successful teams. Teams were focused, creative in solving problems and very productive. Motivation was high as well as team satisfaction. What was common for those teams was the autonomy in solving a clearly articulated problem they were given.
When you hire smart people, give them problems to solve, guide them in organization intricacies and help get the resources they need to solve problems but don’t tell them what to do (don’t micromanage).

Shared services for technology don’t work

Creating a team of specialists (for example DataPower developers), co-locating them in one place and offering their skills as services to delivery teams via the ticketing system is a recipe for disaster. Something that makes sense from the accounting point of view doesn’t work for agile teams that are building software. Again, localized targets for Shared Service teams are never aligned with the vision for product teams.

Beware of re-inventing the wheel

I’ve seen the countless amount of open-source frameworks and tools wrapped in a “thin” wrapper, offering the “same” functionality as the original. In reality, central platform teams started with good intention but hardly ever had the time, resources or expertise to deliver the promise. Avoid re-inventing the wheel, instead of wrapping try to integrate and use extension mechanisms.

gray multi spoke wheel leaning on wall

Management and Leadership are two different things

Leaders inspire people to take actions; managers help people to overcome obstacles and deal with distribution of work. It is possible to be a great leader but at the same time a terrible manager, and the other way around as well. During the nine years, I’ve not met a single Great Leader who was a great manager as well. I believe that in a large organization this happens due to a simple lack of time to perform both roles well. Beware of giving Leaders the responsibilities of managers. They might do a mediocre job at both instead of great at one.

Distributed work is hard

Distributing work on a single product creates communication overhead. For example, scaling one team into two teams creates the illusion of doubling productivity. In reality communication, teams synchronization and integration overhead will only yield a fraction of improvement. Introducing new teams, distributed across different location only amplifies the problem reducing improvements further. When scaling for improved productivity, first look into augmenting existing team and fixing all issues in the delivery process to achieve a smooth flow of work.

Working with vendors require much care

Agencies and Consultancies like to deliver software or services on their terms. The typical sales pitch will mention how beneficial it is for the customer and how productive they can be. The only problem is that quite often after the engagement is done, resulting work doesn’t integrate, is build with incompatible or problematic technologies or lacks on quality. When choosing vendors, make sure to have a set of requirements ready, including technologies to use, ways of working (e.g., integrate early and often), engineering practices and quality standards. If you hear questions related to the above list from the vendor, it’s a good sign, but don’t take anything for granted.

Scaling teams to early could backfire

This point is the expansion of the previous mark of Distributing work. If appropriate software architecture, engineering practices, and delivery pipeline are not in place, the resulting product will lack in quality and cohesive solution. Multiple teams will try to isolate itself via designing architecture and solutions that make it possible for them to work in parallel in relative isolation. This time it’s a Convey’s law applicable on the micro level. If you need to move faster, first try to optimize your delivery process before looking into scaling.

Technology is hardly ever a problem

Majority of issues that arise during software development have very little to do with technology and are associated with people relationship and communication problems. Bad communication leads to missed requirements and invalid assumptions. Make sure to validate all the assumptions early and keep revisiting information you gathered. When you have all the requirements, keep on going back to the Product Owner to confirm that what you are building is the right thing. More importantly, if it is still needed.

Finally

Above are just some of the lessons I learned. I will use this knowledge at the new organization I just started work. I think that the above might provoke some thinking for you as well as re-affirm the lessons learned for me.

Fixing DevOps

I recently posted a one-liner on LinkedIn, that attracted a great deal of interest and thought-provoking discussion.

If I was paid a £1 for every consultancy, company or private contractors who claim to come in and “fix DevOps” for us and then fail, I would be a very rich man 🙂

In light of the comments and queries, I decided to expand on what I mean by Fixing DevOps and failing at it. First, let me start by explaining what was the trigger to write the line.

Devops toolchain

One Cheeky Email

As a Head of Software Engineering I have been targeted by Sales representatives attempting to sell software products and software development services for quite some time now; a few days ago I received yet another email promising to Fix all the DevOps headaches we might have and change our company to become a DevOps Nirvana if only we would to bring them in.

I have been working in the financial sector for 9 years and witnessed a number of times promises that hardly ever been delivered on. I know that my industry colleagues have had similar experiences.

Thus, the above one-liner shared on LinkedIn, was born.

Some problems of large organizations

Historically, the organization I work for had nothing to do with technology. Banks offered financial services for centuries without the use of Software. Computer systems and software were adopted in the 60ties. The technology was used as an aid to business, means of making money easier and faster. Today banks would not exist without IT systems. There is more virtual money in the economy than tangible assets.

In many large organizations, technology is still looked at as an afterthought, the necessary evil that has to be dealt with in the most cost-effective way possible. Latest advances and innovation are hard to introduce. New technologies and processes are adopted at a much slower pace than technology focused organizations like Google or Amazon.

Large and complex organizations can’t exist without modern technology as well as technology makes no sense without their core business. The truth is, both sides have to work together but in reality, the way organizations are constructed prevents it from happening.

Technology is siloed into one organizational unit and business into another, each with its respective leader. Technology becomes a service organization for business. Local goals emerge, driven by local targets resulting in both organization pulling into different directions and the customer finding little to no improvement.

Let’s reiterate some of the DevOps principles at this point:

Focus on delivering value to a user
Thinking big picture – End-to-End product delivery, from inception to delivery and beyond
Never-ending feedback loop on the product, it’s quality and behavior in production
Cross-functional and autonomous teams
Ruthless automation of everything

#BuzzWords to the rescue

I observed the following pattern in the history of DevOps adoption with the involvement of technology leaders at different organizational levels.

A Leader hears a ‘buzzword’: DevOps. Next steps are:

some research into benefits,
multiple visits by DevOps consultancies, referring to case studies within a similar large organization,
a consultancy (or few) comes in to perform DevOps assessment,
a report is produced with information about organizational challenges,
recommendations on how to change and what tools to adopt

Tools become the focus area of proposed improvements as organizational changes are too problematic for A Leader. Consultancy begins the new engagement, ramping up resources and bringing in new tools. The process of “fixing the DevOps” in the organization starts.

A Leader chooses a small area of Organization to roll out new approached and tools. Neither the consultancy nor small area of Organization has enough remit nor possibility to influence any organizational changes, resulting in: consultancies automating a few basic processes, leaving behind a large backlog of future/unfinished work and a hefty bill.

Small, local improvements make little impact in the large organization. The experiment is deemed as a failure and adoption stops (until next Leader arrives or a good sales strategy from different consultancy).

Summary

Many DevOps consultancies are selling The Dream, utilizing template case studies based on the size of targeted organizations, rather than being tailored to individual requirements of said organizations. Challenges posed by organization structure in DevOps adoption are overlooked during sales negotiations. Resulting engagement creates an invalid perception of DevOps as not being fit for purpose, causing more damage than good.

The truth is that for any change to be successful, creating long-lasting effect the initiative has to come out from within, driven by ‘outside the box’ approach.

Controlling central heating with Arduino and Raspberry Pi

When Arduino and Raspberry Pi released first versions, I did buy one of each. Being a gadget man and … well, a man, I played for a bit with my new toys and left them in a drawer.

Time has passed and not much happened with any of them. As I became increasingly unhappy with the central heating controller in my current house, I decided to take the Ino and Pi out of the drawer and actually build central heating controller that I would be happy with.

The design phase

First requirement of the new heating control system was the least intrusive installation as possible.

The dial thermostat that I have at home, works as a simple switch: switching the heating on when temperature falls bellow pre-set; and switching heating off when temperature rise above another pre-set. I decided to use this simplicity in my design. All I needed to do is hook with one 230V cable into that thermostat (230V, hell yeah).

With entry point sorted, I went into the controller bit. Arduino board will control the Relay, which will simply switch heating on/off. For the temperature reading I chose the Digital Thermometer which offered more stable reading than the Analog one (included in Arduino Uno starter kit). I’ve also added LED to indicate when the heating was on.

Leveraging serial port used to program Arduino, I decided that the board will send temperature updates and receive setup command from Pi via USB connection. It will also be powered via the same. This also solved a problem of 2 separate power adapters for both Arduino and Raspberry Pi.

The build phase

I’ve been trying few things before arriving with the below solution.

Electronics

The Arduino circuit is very simple.

Three elements connected to Arduino with a couple of resistors, not much.

The LED is not necessary, it’s there to indicate when the heating is switched on.

The final prototype doesn’t look very attractive, but the lot is hidden under the furniture and the only elements sticking out are the temperature sensors and a bit of LED.

Final prototype of my controller

Software

I had to write three separate pieces of software:

Arduino code that reads current temperature and controls the Relay, https://github.com/greggigon/Home-Temperature-Controller/tree/master/arduino
Raspberry Pi code that communicates with Arduino and exposes REST API for the UI https://github.com/greggigon/Home-Temperature-Controller/tree/master/temperature-sensor-and-rest
UI Code, simple web app that runs on any device and communicates with Pi via REST API https://github.com/greggigon/Home-Temperature-Controller/tree/master/ui/mobile-web-ui

Arduino

For the temperature sensor I included two extra libraries, the OneWire protocol library and a DallasTemperature sensor library. I use 0.5 centi-degree approximation of a temperature reading.

Temperature reads are sent via Serial Port on every loop. Arduino also expects the Float number on a Serial Port. The received number indicates the desired room temperature for Arduino.

To limit the sensor reading fluctuations, the control of relay changes after at least 10 successful consecutive reads of the same temperature from the sensor.

Raspberry Pi

The software that runs on Raspberry Pi does the following:

It waits on the temperature updates from Arduino, and stores the Updates in Memory (for the latest update) and in simple file based H2 database (for historical data),
It exposes REST API for UI to get the temperature information and receive the new settings,
It schedules temperature changes according to schedule stored in the JSON file.

I started the code in Python, but it was running slow. I did a simple comparison of execution time for Prime number algorithms, and Java 8 was beating Python. On a single core Raspberry Pi 1, it was a good incentive to change the platform. I chose Kotlin programming language as it was new to me and I wanted to learn it.

As a framework for Event driven application I choose Vert.x 3. For Serial Port communication, a bit dated RXTX library.

The UI / Phone controller app

The Web App I build works on computers as well as mobile devices. I did choose React as a UI framework with Material UI components. The lot is build with Webpack into a small set of html/js files.

The testing phase

The testing phase involved connecting everything together, starting it, and hoping that I will not sense the smell of burning electrics and experience no explosions. In other words, a standard scientific and engineering approach 🙂 .

I do run the setup for 4 weeks continuously and it not failed so far.

Summary

This is the first time I used my skills to build something that interacts with physical world. It gave me a great feeling of achievement and satisfaction. I know that I could buy something that looks much nicer and probably works better but I learned a lot during the process.

Raspberry Pi 3 was released while I was building my design and I switched to it. You can see it in the pictures. I also want to switch Arduino Uno prototyping board to Arduino Nano.

All the code and more detailed technical description is available for you to grab from my GitHub repository on https://github.com/greggigon/Home-Temperature-Controller .

5 key learnings from the Innovation Spike

Innovation

Last week I took part in the innovation spike at RBS. This post is my personal retrospective of the spike, findings and lessons learned.

Outline:

What is an innovation spike?
Format of the spike
Technologies and tools used
Findings and lessons for the future

What is an innovation spike?

In RBS the innovation spike means an event where small team works in a startup mode, trying to deliver new product or a service by using new technologies. The reasons for the spike are to explore new technologies as well as prove technical possibility and produce proof of concept for business.

The prototype that we delivered was functional enough to spawn discussions around business model and to take a look at product from the user perspective. It was also small enough to simply throw away if business assumptions doesn’t hold or technology is not good enough.

It is possible there is a better name for this event, however “Innovation spike” sounds great and naming things is hard.

Format of the spike

Our small team had 6 developers, graphics designer and a product owner. As a team we had all the technical and non-technical skills required to work on every aspect of product development.

We did prepared a little bit for the spike. Our product owner had a pretty good idea of business that we would like to validate. We had a small infrastructure prepared with some basic tools and did read on the choice of technology for the spike.

The spike took place in The Bakery London, which is a great place for a startup gigs with fantastic people ready to jump on problem and help. We run it for 4 days (Tuesday to Friday) with one day of preparation (Monday) and demo to business on the last day.

Technologies and tools used

There was a number of candidate technologies we wanted to try during the spike. We ended up with the following:

Microsoft Azure for our infrastructure needs. Ubuntu VMs with Docker for spike tools and for our applications deployment
GitLab, Let’s – Chat, Taiga.IO, Jenkins, Docker, Docker registry with Front-end and ApacheDS as LDAP authentication was our development tools stack.
Meteor framework for all front-end and some backend development and Vert.x 3 with Java as a backend API services.
VirtualBox with Vagrant for consistent development VMs.
Atom, Visual Studio Code and IntelliJ as development IDEs

Some of the technologies were familiar to us, some were entirely new. We picked the above technologies as we wanted to give them a try and provide feedback to the rest of the bank.

At the end we didn’t use Taiga.IO at all as good old fashioned boards with stickies worked great.

Findings and lessons learned

The bellow findings represent my thoughts after holding conversations with others in the team.

1. Location is important

Being away from the office in a different location changes how you think. I mean it. Many things adds up towards it. Lack of dress code and non corporate style of office made everyone feel relaxed. Food and drinks took away the need of thinking about those things during the day. Little technical touches made us being more productive. Things like fast and easy Wireless Internet, big flatscreen connected to Apple TV so that everyone could share their screen when needed.

IMG_20150728_135347~2

2. Cross functional, co-located team is important

It was great for all of us to sit around one big table, where we could pair with each other, ask each other questions or simply showcase what we’ve just done. Business (or product) owner could provide constant feedback and get us on the right path. Spike team in turn could immediately report back what was technical doable or what would take more time than we had.

The range of different skills across our team meant that we could handle all aspects of delivery lifecycle: design, development, testing and release.

There was simply no time wasting and no communication issues.

3. Having a leader is important

Having a leader is important as we got constantly refocused on the goal and next piece of work to handle.

A leader is not there to manage! Role of a leader is to propose initial structure and adapt it as we go along. Innovation spike is no time and place for a project manager.

In our case, our Mighty Leader prepared everything for the spike, time, place and the right people. On the first day of spike he introduced initial structure. We had a session with product owner, than we jumped to a small design phase and we started coding. In average, once every 3 hours we stopped and synched on what we managed to do.

4. Coming prepared is important

As I mentioned earlier the innovation spike is about exploring new technologies. It’s a learning exercise. However, we did prepare a bit and here’s why. Having a good starting point saves time.

Not everyone have to prepare on every aspects of the spike. I took time to learn a bit about Azure Cloud and prepare some basic infrastructure for our team. Others did complete introduction and tutorials on Meteor framework. Someone else created Mockup APIs. We all learned from each other. Someone always had an answer to a problem others were facing. Being prepared is always good.

5. Simple and good enough is GOOD … ENOUGH

Simple and good enough is more than enough for the innovation spike because it’s easier and faster to make. At the end, the final product is something that could be simply thrown away or taken as a proof of concept for a bigger project.

We were constantly reminded about not paying to much attention to unnecessary details, but to focus on the business functionality and pushing boundary of the tested technology.

Summary

Innovation spike was great learning experience. Finding out about new technologies and business opportunities. It was also great fun to be on. The key to its success was location, cross-functional team and being prepared. Having a good and focused leader made us very productive and good enough approach caused us to deliver successful and working proof of concept in a very short time.