Most people are aware that automation is changing the technical landscape in all aspects of IT, and it isn’t just limited to development and operations teams. Testing teams have also been embracing this change to add more value and delivery speed to the products that they work on. Things at Movio move fast, and that doesn’t just include our Run Club. I learnt this quickly when I joined Movio a year ago they were in the process of Dockerizing the Movio Cinema product. Throughout this piece of work I realised certain areas, where increasing or upgrading the level of test automation would enhance our impact to the product in both value and speed.

Testing at Movio

We use anything but standard practice at Movio, which is an interesting change from the more traditional roles that I’ve had in the past. Testers are tasked with the challenge of being the ‘Champions of Testing’ within the company and are able to impart our knowledge, experience and tools to the rest of the development teams with the idea that “many hands, make light work”.

Our testing approach is lean, as is everything we do. Normally, one of the standard practices for a test team is to make sure a test case repository is set-up and then follow this up with a large amount of reports for management with each release shipped. We have decided not to follow this standard practice and therefore freed ourselves from ‘rinse and repeat’ reporting that would come with each release. Doing this has allowed us to focus much more on test automation where possible, or make sure that the tested changes outside of the framework are captured in JIRA - our project management software.

Front-end Frameworks

As part of our dedication to automating testing and its processes, we viewed the web front-end as a great place to the start. For Movio Cinema, the existing front-end test coverage looked like it could do with a freshen up, especially with the pending change from our existing front-end UI code base to a new React based UI. To help make this transition a much faster and easier process to achieve, we needed to provide a testing framework that could also be used by the development teams. We wanted to make sure that the framework we provided was in a language that didn’t frighten the development teams, and gave confidence when porting existing UI code. With that in mind, it was important that the framework was easy to use and provided all the necessary functions that the development teams would need.

After much discussion and a couple of strong coffees, we decided to get on the cool kids program and jump headlong into the JavaScript/Node world. Ok, so it wasn’t just so we could feel all ‘Silicon Valley Startup’, Movio has some excellent JavaScript developers that the Test Team would be able to call on if needed. With Movio Cinema being a web product it seemed like a great fit. On a side note, the chance to look into newly released ES6 (ECMA 2016) functionality was a little too good to pass up for me. Therefore, since this project was heavily weighted to 'End to End (E2E) Testing', I chose Selenium Webdriver as the engine to use.

NightwatchJS

From there it was on to finding a suitable JavaScript based Selenium framework to use. After driving down a few dead end streets in some pretty rough neighbourhoods, NightwatchJS was determined the winner - with it’s easy to understand implementation, usage documentation and well-rounded API documentation that made test create simple and fast. After using it for almost nine months we have found some approaches and methods that have worked well for us. As part of this blog post we are releasing an example repository here, for anyone interested in getting more details on NightwatchJS and some of the ways we've implemented it.

After getting NightwatchJS running for our web project, we noticed there were other projects at Movio that could possibly benefit from using a test framework, like NightwatchJS, instead of their custom automation frameworks when using Appium for their mobile testing. So this lead us to investigate an integration between NightwatchJS and Appium.

Appium

Hooking up NightwatchJS to run Appium was a little trickier than expected, but we persisted as it looked like it would pay off. There was little information floating around on the internet about this approach, which seemed a pity as it appeared like a great fit for us. After a ‘little’ bit of hair pulling and good old “trial, error, scream, rinse and repeat” we finally got there. We are currently in the early stages at the moment, but as some of our mobile testing expands I hope this approach will help make this process much easier.

Kubernetes

Once we finished those two testing projects we thought that we would try and push our luck one more time. As we’ve had Kubernetes in Production at Movio for the last 6 months, we are now looking at moving more of our infrastructure across to Kubernetes in the near future. We decided it was a good time to get ahead of the curve, and because it seemed like a fun idea, we set about  getting our 'NightwatchJS Testing Framework' up and running in Kubernetes. The first things we tackled were getting Firefox and Chrome running headless in our Docker container. Once that was done, we set about getting the container to run in Kubernetes. Two donuts and a cup of coffee later, it was up and running in Kubernetes. This gave us an easier way to deploy our container and try out testing changes on the fly. It also gives us some future options like test run results returning through our main logging infrastructure, which would save logging into the container manually to retrieve the results. It will also allow multiple parties to view results immediately.

Outtakes Matter

Sometimes in a test team it isn't the technology or processes that you can see that matter, it the ones that you can’t. At times we take a leaf out of the dev teams pair programming book and pair with a developer to discuss their code. This can be very beneficial as it can head off potential bugs or issues before the product even gets to the first round of testing. It also allows for great discussion on the requirements of a piece of code at the stage when it is the easiest to fix. So by the time it gets to testing we are confident on a lot of the code and can really focus on areas of importance, rather than spend a lot of time on the basics.

In summary

We have much more to cover in testing at Movio including:

• Plans to improve our use and integration of NightwatchJS.
• Adding REST endpoint(s) to the Nightwatch container to allow remote test runs instead of from a local installation.
• Setting up a JSON reporting option from NightwatchJS so that test results can be picked up by Kubernetes and then routed through to Kibana which allows more people to view test run results at once.

This blog is just a small taster of what being a Tester at Movio is like and what we get to do during an average day.