Not enough cake for everyone?

I left a comment in a LinkedIn group that people seemed to like so I'm putting it in a blog now.

The discussion was about testing time getting squeezed by the sponsor and PM as extension of that in many (non-agile) projects and what the Test Managers responsibilities are.

If you ask the team to prepare a cake for twelve but only buy ingredients for six you only have a few options. Either you reduce the number of people. Or everyone get's a slimmer piece. Or someone goes out and buys more ingredients. I'll let you work out the infamous Project Management Triangle for yourself..
And yes, there are more options like extending the dough with other materials, fake more cake by making it shallower but with a bigger diameter, etc which would be options that a clever and motivated team may come up with.

My point though is that it's not the Test Manager who takes on sole responsibility when the customer doesn't like the cake. If you shorten preparation time you'll find some lumps at the end, no question about it.

Quality of the software is a team effort with the sponsor / owner and by extension the PM setting the boundaries. It is up to the Test / Dev Manager  to then speak up and describe what the practical effects of these boundaries are but that's where it ends.

With a good team a lot can be done to find out what it is that is really required and how to get there. And hey, at the end there may be cake for everyone involved.

The discussion and learning about "it" is more important than the definition of "it".

On one of the LinkedIn Groups, Steve Green, who I value a lot both for his experience in exploratory testing as well as a person argued against the use of exit criteria as they are almost meaningless in his experience.

I agree but meaningless doesn't mean useless. Let's say that:

The discussion and learning about "it" is more important than the definition of "it".

While in many cases exit criteria are sacrificed to get a project out of the door I think that it would be a mistake to dismiss them. By trying to write down exit criteria and discussion within the team a lot can be learned about the way software is developed within a given company. When one struggles to come up with an exit criteria that may point to one or several problems.
Are stakeholders not clear? Are decisions made ad hoc? Is planning or estimating a problem?

A discussion about what exit criteria may be realistic and which ones aren't including the reasoning behind it will unearth a lot of important information for the project.

You may end up without exit criteria after all and that's OK. But the important bit is that information was gained and learning happened.

Of course "it" could be anything. Exit Criteria; a bug; a process; a workflow; an approach; a standard. As long as there's learning we have gained something that cannot be taken away.

Trials and Tribulations of a Test Manager (Part IV)

You may want to read Trials and Tribulations of a Test Manager (Part III) if you haven't done so already.


The stuff they didn’t tell you about when you got the job

Some of the most challenging and tragic but also some absolutely hilarious things happened in this category. Some were sort of harmless where I found out some time after I started that the Test Manager job meant that I’m also expected to take the role as the Environment and Configuration Manager for the test environments at least.

Some examples of the more serious situations were that I had people come to me telling me they’ve been diagnosed to be bipolar; partners with failing kidneys; alcoholics; parents in hospital with live threatening illnesses; current divorce proceedings and the list goes on. At times like these the people approaching me were usually quite vulnerable. So while knowing that the next project will surely be impacted by sometimes grave news what they want to hear is that I’m taking care of the business side and say “Don’t you worry, I’ll take care of the work side, how can I help you – do you need time off, flexible working, etc?”

I always gave people leeway in situations like these mostly because it’s the right thing to do and I would like to be greeted by understanding from my line manager myself. But at the end of the day there isn’t really an alternative. Under pressure people will put their personal interests above the company anyway. Sometimes without knowing, sometimes with full understanding they don’t really ask for permission for time off / flexible working hours / understanding/ etc. They are only letting me know that their work will be impacted for some time. They do the right thing and expect the same in return.

Once they get a positive response it usually makes their lives a bit easier with one thing less to worry about. I had several instances where people then put in double the amount of effort and working hours once the situation was resolved, far in excess of what they took in the first place. Sometimes HR needs to be involved, sometimes it was fine to deal with it myself depending on the situation.

I worked in multicultural teams and what one perceives to be a joke can be an insult to the next. Careful explaining and setting of boundaries of what’s allowed and where the no man’s land begins becomes very important here. Sexual harassment, invoking grievance policies and offering a trip to the nearest dark alley are all part of this area which I have been lucky enough to diffuse before anything too serious happened.

Something related which tended to be between hilarious and so far out there that it stretched my years of experience to the extreme are what I call the “Rammstein” jokers. (After the band – insult and make fun of everybody, no taboos but expect to get some feedback). How do you react if someone sends an email round to the team saying something along the lines of “Peter is OK but he smells a bit this morning, probably borrowed his wife’s perfume again. I worry more about John who got out his thing under the table again, talks to it a bit in a Gollum voice and then looks at ‘hoff pictures.” (Names and wording changed but that email happened...) In these cases it’s either a “Thank you for working for us, don’t bother logging off. Let me escort you out of the building.” Or seeing how the rest of the team handles it, have a quiet word with the author and then take it from there.

I hope you enjoyed the musings of my last few years as a Test Manager. It was an interesting experience and I like to manage teams. There’s always something new, exciting and sometimes unexpected to learn and I hope I gave something in return. 

Thanks for reading.

Trials and Tribulations of a Test Manager (Part III)

You may want to read Trials and Tribulations of a Test Manager (Part II) if you haven't done so already.

The Test Manager as a company approach

A Test Manager (or Development Manager or any other type of Manager with line management responsibilities and a certain degree of freedom and responsibility) usually has roles that reflect the whole company. This allegory works best for people who lead a team or department.

She would be the CEO, expected to prepare a vision for her team and provide guidance; the CTO to provide the technical vision and advise on which technology path is suitable to follow and which isn’t; the Finance section for keeping track of budgets; HR for hiring and firing people and providing guidance about policies in the company and so on.

So in very many ways a Manager is a company acting in a Micro-cosmos in their own little world. Of course the good ones make sure it fits with the Macro-cosmos of the outside world.

Not every Manager can be good at everything so what are you going to do about it? Delegate or get train yourself in the weak areas? This is very much a personal and context question which I can’t really answer here, it’s only there to remind us that the day to day tasks not necessarily encompass everything a Test Manager is responsible for and that some time and thought may need to go in these areas as we tend to forget about the things that we’re not so comfortable with.

One role that I haven’t mentioned above is that of a trainer. If I want to train, coach and mentor I should know a thing or two about the area in question. So everyday learning is a must in my opinion. This does not mean that I’m reading a testing book or internet articles every day although I try – often live interferes. Some days I learn a lot just by talking to colleagues, how they approach a particular problem and how their experiences play a part in that. Sometimes I learn how to do something, sometimes how not to. But it’s all useful, one way or another.

One thing that should be pretty clear is that you can’t manage what you don’t know. If you rely on reports about how the team is doing a lot is lost in translation. If the Test Manager and the Test team are located in the same building there shouldn’t be any excuse for the Test Manager NOT to sit with the team. That private office may look fancier and more comfortable but you could just as well sitting in another country. All the day to day problems, friendly banter and everything that makes up that team will be lost to the manager. In short, co-location with the team is a must, otherwise the Manager will not properly understand the problems that the team is facing. If the Manager doesn’t know where the rocks on the road to success are he can’t remove them.

It works the other way round as well. The team sees the problems and challenges and work that the Test Manager does. Overhearing discussions I had with dev managers, sys admins, etc has helped my team understand what’s expected from their manager so they could work toward helping towards these goals. When sitting with the team any small problems can be nipped in the bud before they become serious problems. Morale issues can be spotted early before they become serious problems. If there are talks that need to be had without the team that’s what meetings rooms are for. Letting people overhear all discussions also fosters an atmosphere of openness. It's saying “see I have nothing to hide” and we're all heading in the same direction.

Read on in the final chapter of Trials and Tribulations of a Test Manager (Part IV) coming soon.

Trials and Tribulations of a Test Manager (Part II)

You may want to read Trials and Tribulations of a Test Manager (Part I) if you haven't done so already.


The Shield, the Sword and the Horn of Met

In my experience one of the most important functions of any Team Leader is that they can shield their team and make sure all the rubbish and rotten tomatoes coming flying don’t hit them. The importance of that can’t be overemphasized from my point of view. If the Team Leader is seen to put his money where his mouth is in protecting the team from external distractions they will rally when called upon.

As an ex-Manager of mine once said “Get them to work for you, not for the company. Sooner or later the business will do something that will disappoint them. But if they work for you they will do their best because they don’t want to let you down, even if they’re unhappy with the company.”

I’ve been in both camps in my working live and can say it’s a lot easier to work at a place where this Shield is in place. It breeds mutual respect and is in very many ways the return to the tribal society.

If any threats to the team are swiftly intercepted and struck down it also sends a message to people outside the team– don’t mess with these guys. They’re a professional bunch and we don’t tolerate substandard behavior. Saying “no” for matters concerning the team sets boundaries for the behavior that the manager sees as acceptable. Precedence is created.

Something else I learned from an ex-Manager is that the team manager is not there to do the hands on work although it doesn’t hurt if they can jump in when necessary. The main goal for the team manager is to remove the rocks on the way and make sure that there is nothing impeding or slowing them down. Only if everyone in the team can work unobstructed should the team manager look at their own work tasks.

I can’t count how many times I got absolutely invaluable information about problems in the team or company in the pub. A week or two after joining I went out with the team for a drink and learned more there in two hours than in the weeks before. Things that people genuinely cared about or that upset them for one reason or other came to light in a more relaxed atmosphere than the workplace. Having some distance between the manager and the team is fine. When it comes to the crunch the manager sticks their head out and take a decision. But being seen as part of the team is a must in my book. If people can’t see the real me behind the manager facade, how are they going to trust that I stand up for their interests?

Leading by example is not only old news it was ancient when dinosaurs roamed the earth. What it does for the team is manifold. It sets the boundaries of what is acceptable and what isn’t. If the Test Manager comes late and leaves early that will soon be copied. Is a two line bug report acceptable? Depends on the company, maybe a bit more information would be helpful. So logging bug reports as you’d like to see them done is a good way to lead by example. These reports can become the standard that everyone should be working towards. Same with many other small things, emails, written and verbal reporting, etc. Setting precedencies and explaining what the reasoning behind it is and why it’s important will help form a professional team that works as the Test Manager expects.

I’m not promoting the lone hero approach, far from it. In order to get a team into shape you need people to help you. Someone who has seen the light and believes your approach is the right thing to do. The video Leadership Lessons from the dancing guy makes it clearer than I could do. It's been around for quite a while and is worth a look.

If you can get some momentum going with the help of a believer from within the group it will go a long way to building a successful team.

Read on in Trials and Tribulations of a Test Manager (Part III).

Trials and Tribulations of a Test Manager (Part I)

It’s been a while since I last wrote here but I was busy moving country and jobs again. I left Gibraltar behind and am now back in Germany in my home town but that’s a different story..

Let’s share the experience

I’ve been thinking about Test Management and Management in general a lot in the last couple of months having observed both the good, the bad and the downright ugly approaches. Stuart Reid in the Foreword to Essential Test Design from Torbjörn Ryber says “… I would argue that the majority of software testing problems stem from poor test management …”. I’m sure that context dictates what the majority would be and that it depends on how it’s measured but I agree with the sentiment – that test approach, process, technology and technique all have a role to play but that good or bad management will dictate for many a project if it succeeds or fails (This is also true for work done outside a project).

A couple of weeks ago I had chat with a colleague from another department about my test team.  He told me that he and others wished they could be part of that team – not because they wanted to test but because they wanted a part of that team spirit and ambience that comes with it. I was quite flustered – it was one of the nicest compliments I ever got as I put that team together. If people want to get in not because they like the work or pay but because it’s cool I must have done something right. I’ll try to put into words what I believe that is over the next few entries.

It’s all about connections man...

A Test Manager like everyone else in a company structure is a link in the chain of business responsibilities. There’s a link leading to your team and there’s a link towards Senior Management; and possibly several more to your peers (Having no identifiable peers is a sign something went wrong but that’s another story). If you consider yourself as a key ring to which many chains are attached that picture works best for what I’m trying to convey.

Each direction is a relationship with slightly different expectations and requirements at the end of it. My approach for each relationship in every direction in the past has been the same – read: show the same courtesy and listen to everyone about what their requirements are, no matter what their position.  Each person at the end of the relationship chain has their own needs and requirements that I may be involved with. Some will be the same across all levels, others differ wildly from person to person.

Some want help with their career, others want to be left alone as there’s enough going on in their lives already. Some want gentle pushing some object violently. Some can be rewarded with money, others with flexible work hours, visits to conferences. A pat on the back when something was well done has never hurt and is the strongest reward that people will not ask for.

It helps to have worked in similar jobs comparable to that of the person at the end of each chain. In other words, if you worked in support before it’s easier to understand the needs and requirements of a support role. Imagine walking a mile in their shoes. That’s a bit harder if you look up at the food chain but as the requirements of the CTO or other Senior Managers become clearer building a team that can satisfy their high level goals becomes a lot easier.

If you haven’t worked in a comparable job before sit down with the person in question and ask them to tell you about what the tasks you just gave them means for them. Is it a 5 minute job? Is it a long nightmare tasks because of complications that you haven’t foreseen? Talk to them and say that you genuinely want to know. Of course it has to be heartfelt or that one backfires... A few months ago I asked for something that I considered to be a half hour job, restructuring some of our virtual machines and renaming some URLs. Turned out it would’ve been a week or more with serious impacts on another team. I sat down with the person that I requested it from and subsequently now know more about networks and VMs. These kind of sit downs were great learning experiences for me. Next time I won’t make this particular unreasonable request as my knowledge has increased.

Read on in Trials and Tribulations of a Test Manager (Part II).

Performance Testing on a shoestring

The setting for this particular story is the run-up to the biggest yearly Horseracing event in the UK, the
Grand National in Aintree.

Even people usually not betting have a flutter heading to the nearest betting shop or hitting the net. Good news for an online gambling company, mixed news when it comes to the servers that need to cope with a lot more traffic than any other time of the year. Quite a few online gaming companies had problems on the day.

Our servers held and we only load tested one aspect of our system. Here's the story.

About two weeks before the race it was decided that we want our payment section tested after all. Right, plenty of time, better get started. We looked into JMeter, however there's limited knowledge in the team. So we asked some developers for help. After a lot of headscratching it was decided, yes, it can be done within about 3 weeks and 2 devs. So the idea of using JMeter was out of the window. A commercial solution was not feasable due to timescales and some other factors.

We decided, since it was to be something that had to be quick and was to be a one off (we'll worry about next year later), to do an in-house solution. While I was running around, gathering required information, investigating possible pitfalls, etc our automation expert (don't tell him, he will ask for more money) used AutoIT to code the load tool.
Yes, you read right. That's about the most unlikely tool for a performance test that I can think of but I employed most of my team members, I have trust in their abilities so time to put my money where my mouth is.

We knew that we wanted to get 8+ transactions a second to test that our system and our third party payment provider could take the heat. And we wanted that sustained over a period of time, say half an hour before the race is busiest so let's just run it for that time. That's the detailed requirements gathering out of the way...


I won't go into the details of the system for obvious reasons, rather explain how we went about it. We ran a transaction manually and found that it took between 10-25 seconds to complete, including filling out the form again for a second try. We also found that we could fit 8 browser windows on a monitor and show all necessary fields and buttons when setting the window size to 35%.

The idea was then to click "Submit" on all 8 browsers at the same time (or a millisecond apart). That worked fine. So, we could hit the system with 8 transactions. Once.
Ok, no problem, since we needed about 25 seconds for a round trip that was maintainable, we added another 5 seconds in case the system slowed down and then went to hunt down 30 PCs in the office (an experience in itself) to start them a second apart.

After having identified the machines we set all 30 to the same screen resolution (I didn't know how many different monitors we had until that day), deployed our performance tool to each machine and set up the machines. That involved setting the IE homepage to the desired URL, running the setup tool, clicking the necessary buttons and fields so that the correct X/Y coordinates were captured. The AutoIT application would then calculate the relative difference for the other 7 windows.

That was a lot of manual setup but we haven't had the time to code for more. Each machine got it's own ID (set up in the application and post it on the monitor) from 1 to 30 and we coded an editable start time into it. For example, machine with ID 1 would start at 08:30:01, ID 2 would start at 08:30:02, etc and then loop round starting again at 08:30:31.

Quite a bit of thinking and discussion has gone into this bit because we could have let each window just start again after each transaction was finished. Depending on the system response time transaction would have drifted apart though resulting in potentially a lot higher transaction number for some seconds and none at others. I was more concerned about the former as it could have been a theoretical 30*8=240 transactions at the same time which would very likely have created serious problems.

So we had our 8 transactions running for 30 seconds, looping around indefinetely until we stopped them. Setup was completed in the evening after a 13 hour+ day. We also did a small test run just to make sure it would work.

Before starting the next morning we had sys admins and several other people in place to monitor the various system components during the test and in case something went seriously wrong.

Execution threw up some problems, not least some network issues that weren't identified before. Also, some pages threw errors that the application couldn't recover from. Time was running out and people came into the office wanting their PCs to work on, so work was stopped. Two days later, after resolving the issues and having two more days to put some resilience into the AutoIT scripts so that recovery after errors was better we tried again.

This time it worked really well. Some machines had to be started again but overall we put the desired load on the system.

Lessons learnt:

• Regardless how much you ask people for information, there's always something that someone has forgot to mention that will ruin your day. Do a test run and plan for round 2.
• Don't use IE if you don't have a standard company build on all PCs. Use a browser that you install yourself and control so you don't run into configuration issues when you least need it.
• Ask people for help. Running it yourself will only stress you out. I found that once we explained the somewhat mad plan people were happy to assist
• Tell people if you're highjacking their PCs, you may not get a chance to revert all changes. That was done for most but the ones we forgot were understandably miffed that we changed their settings without warning.
• Get a demonstration of what will be monitored and what can be saved for analysis later. Assuming it will all be there can lead to disappointment and repeats of the test.
• Don't ask others to do the boring jobs. Being hands on helps the rest of the team to see that you're serious about making this work.
•  It's possible to get the job done, regardless if you're prepared, have the tools or the time. Determination, the belief that the job can be done and trust in others will get you a long way.

  If someone has similar experiences I'd like to hear about it.

Thanks for reading.

Do I want to write this?

I usually keep private and work net presence clearly separated. This one is different but I decided to post it as I couldnt' get the testing mindset out of my private life and it may help. It also explains why I haven't blogged or been active in the testing scene in the last couple of months. Not that I think that needs explaining.


Germany/Wuppertal, December 2012, Intensive Care Unit. I browse through the notes the nurses left for the last two days, noting the structure of it to be easily recognisable for the next ICU nurse. It's easy to see at a glance what drugs and treatments the patient got in the last 24 hours. It has to be, any failure here could be fatal.

Looking at the syringes next to my fathers bed I wonder what they all are and take note of the names. He's been in a coma since before Christmas. Reading the drug names comes easy, I worked as a pharmaceutical research scientist for over a decade. Memories of that come back. At home I find that Ketamine is for disassociating the body from pain and that the street price has fallen over the last couple of years. For some reason that stuck with me. I read up on resuscitation, survival rates and the side effects like personality changes and brain disorders. I read a lot and educated myself but sometimes I learn things that I really don't want to know.

Beds, syringes, forms, power, gas supplies, room layout, etc are all standardized so that any nurse or doctor can take over where the last one left off. In an ICU that's of vital importance. Each shift has a 1 hour handover/scrum to brief  the next shift of what happened. I wonder what would happen if we were to do 1 hour handovers each day.  In my current line of work it's not that important to know exactly where the last person left of. It's useful but no one dies if you miss a piece of information.

Seeing that everyone working in the ICU absolutely has to know everything about each patient/project was an eye opener. Of course that comes at a cost. But in that context that cost is worth paying.
So how much is it worth in our projects that everyone knows everything about the project? How many people in the project have no idea what their colleagues are working on? Is that OK or is that acceptable? How is the risk covered that information goes missing?

I learned a lot more in these weeks. How to recognise if people make mistakes and where the system fails; who puts in more effort than the rest; for some nurses the relatives play a bigger part, for others the patient is the only important thing. Most are somewhere in between. I reckon that's the Manager in me making these observations.
The people who I think of as "best" without defining what I mean exactly all have a passion for what they're doing. They're not only knowledgeable but are emotionally involved. I can say the same about the testing scene or probably any other craft that people are working in.

Of course the "learning" during this time wasn't purely to do with this mindset. Most of it was on the emotional side as can be expected. I learned quite a bit about what my approach to thinking and learning is compared to my parents and what is self-learned. But watching myself making these observations was a convincing sign that I'm working in the right job.

My whole family spent Christmas in Germany (unplanned and at very short notice; 6 hours from getting the call to leaving for the airport with my wife, 8 year old son and all Christmas presents) while my father was in coma all the time. He woke up in January and I flew back to Germany to spend some time with him. He died in February.

You won't be forgotten.

Why using custom pictures in XMind is useful

 I read this Blog from Albert Gareev, thought that's great but there's another way where you can use not only marker packs but also any pictures which is useful as described in my Mindmaps for managing test projects blog.

For example, I wrote the names of my team members in Word, Arial, 14pt, bold and took a picture of the name and now use them as markers. That's very useful when assigning projects to people so I can see at a glance who is working on which projects and where they may be stretched a bit thin.

In XMind this can be done by
navigating to your installation folder, for example
C:\Program Files\XMind\plugins\org.xmind.ui.resources_3.2.1.201011212218\markers

Drop your *.png or even *.jpg files in there.

Open "markerSheet.xml" in the text editor of your choice and add XML entries using the existing entries as a guideline. It's pretty straight forward.
Save and re-open XMind if you had it open before.

Now right click on an object and insert a Marker - voila, your new pictures as markers are now available. You are no longer restricted to pre-designed packs but can cook your own.

Bringing Information Back Into IT

I wrote a bit for the EuroStar blog roll some time ago called "Bringing Information Back Into IT".

Any comments please leave at this post as the EuroStar one is out of date now.