The Tribe Bug Hunt

As a software tester, debating whether something is a bug or not with a developer isn’t an uncommon occurrence. This is no bad thing. These discussions provoke new ideas and help us to make the best software we can.

It’s unusual, however, when it’s the developer insisting that something is a bug in that debate. It feels like my world has been turned upside down. To be honest, it freaked me out….

So, 10 developers, 3 product managers, 2 support staff, an application consultant, the chief architect and the director of research and development…

Welcome to the TSG Tribe bug hunt.

We briefed our group of unlikely bug-hunters on the areas of the software, how to log bugs and other key bits of information they may need, paired them up then set them to work on TSG’s new membership solution Tribe.

There were two sessions of testing, each two hours in length. One hunter drove the mouse and keyboard and the other dictated the actions, took notes and logged the bugs. We also insisted that they swap around at least once throughout the day. Variety is the spice of life, after all.

The QA Manager Luke roamed the room providing support and guidance like a (slightly) taller, less green Yoda, while Muggins here was assigned the task of reviewing each bug that was raised. Roughly one issue was logged a minute.

During the first hour I started to understand how a builder might feel watching me try to build a wall (I’m not what one might call ‘handy’). I got cranky, I snapped at people, I significantly reduced my chances of being invited to the next R&D curry night.

To be fair though, it didn’t take long for the bugs to be logged in the correct format, screenshots to be included and some even used something similar to grammar!

What we got from the bug hunt was, in effect, 72 hours of diverse testing with a wide and varied coverage. We had the people using the software that they themselves designed and built, and we also exposed Tribe to those who hadn’t worked on it at all.

It was a huge administrative task required from the Quality Assurance team, both during the hunt and after, to review, verify, prioritise and polish the bugs that had been raised. But it was well worth all the effort.

We had a list of prizes too. One for the most bugs found (unsurprisingly won by the senior Tribe developer), the best bug (purely a subjective call by Luke and I) and the booby prize. The booby prize was again a subjective prize that would be given to whoever did something we found to be particularly funny, daft or whatever we felt was deserving.

We weren’t let down. The booby prize was presented (with little competition) to the developer who logged a bug that had been raised and assigned to said developer for the previous six weeks leading up to the bug hunt. I won’t name names. Not because I don’t want to, but because Mat, the R&D Director, has specifically told me not to.

It’s a testament to the developers, and the testing we’d already done, that the majority of the issues raised were considered low priority or cosmetic. Only a couple issues found during the bug hunt were considered to be of high priority. But all of these issues being found, and since fixed, can make the difference between good software and great software.

As well as being a very productive day, it was also a very enjoyable one. We abandoned our usual pristine appearance in favour of jeans, t-shirts and other such trendy streetwear, ate heart-threateningly unhealthy fast food for lunch and all participated in some pizza and beer based socialising once the working day was done.

In a nutshell, the bug hunt both reassured us that Tribe is the high-quality software we aimed to create, as well as enable us to iron out any creases.

In an industry where quality can often be relegated to merely an afterthought, it demonstrates real commitment to our high standards that Mat, and TSG in general, invested both time and resources to allow us to organise such a day.

This was our first bug hunt, but it definitely won’t be our last.

Next time somebody else can review the bug reports though.