Dear tester! Others care about quality, too.

Dear tester,

I know that sometimes it feels like people you work with just want to mark the cards in JIRA as Done without proper testing. Sometimes they tell others “…once it passes the QA” or create tasks for you subjected “QA X” like it is being done just because “they have to”. The feeling of annoyance caused by urgency to complete the task immediately without reporting any bugs is inevitable because of the way they tell just to “pass the QA”. I did write of that before on why “pass the QA” makes me cringe, so I can feel your pain really well. Especially, that even after me trying to explain QA vs Testing vs Checking many times in my company and clarifying, the very same wording is still being used. I would like to share with you a story that happened to me which made me think that sometimes we exaggerate a little bit assuming that others do not care of quality as much as we do.

Today one of developers in my company came back with initial implementation and results to “QA”. The task was fairly simple – there is a lot of data generated by an algorithm and we should check it (I’m using here check consciously as it’s not really testing at this point): does it make sense, what patterns of fault we notice, does algo actually work? All of this should evaluate the quality of results produced by this new algorithm.

The wording of this task’s formulation and documentation with the data had QA mentioned around 5 times in various forms and I’m sure you are familiar with most of them: “data to be QAed”, “for the QAing” or my least favorite “pass the QA”. These terms do not feel too good as they are not correctly used and it may feel slightly insulting sometimes that your colleagues may not bother to even understand what you’re doing. However, you cannot teach all people to use the terms and it’s important to let it go sometimes. Remind yourself that we all have biases (and I do have a story on Managing your biases which made me slow down a little bit before judging). I decided not to exaggerate and think from that developer’s point of view: we both know what he wants as a result – the quality should be evaluated even if he is using the wrong terms.

Some colleagues may use the wrong terms and confuse testing/checking/QA, but don’t go and nit-pick on that. Words matter, but not everyone cares either how to name the rose: all you can do as an empathic quality specialist is to show people that you are open to explain to them, but only if they want to. 

This is not why I’m writing to you, though – this colleague of mine may have used the wrong wording, but letting go of that wasn’t the main takeaway I got.

When the colleague created the task description, it lacked one thing: any description of implementation details of algorithm. No documentation was yet created, no code mentioned, only thing provided was the generated data and vague explanation what should be done (compare columns and say if it’s okay or not using some human sense and research on each of options).

I really wanted to see implementation details: how else can I assess actual risks? Maybe there are areas and patterns that are design flaws and can be seen before even looking at the data generated. This developer tends to work alone as well, so there isn’t much of code review going on.

When I asked if there is any documentation on this algorithm, this was the response I got from the developer:
“Not yet, this is not ready for production yet. When it passes QA there will be a documentation page with all the changes that have come out of the QA process.”

This wasn’t something that I expected to be honest – I replied that to do the QA process we need to know the implementation details and this shouldn’t be made visible only when the algo goes to production. We shouldn’t check in the dark.

My reply has made this developer write to me personally and the words that were used by them again were a little bit rough I could say. The arguments on why the documentation wasn’t created were that “it is too big overhead” and then eventually “it seems that we disagree on the QA process here: for this task, there is no need for implementation details”. How would you react to this, my dear tester? Developer is claiming that as someone who is hired to test and give quality evaluations you shouldn’t look at implementation details at all.

As someone who recently encountered several design flaws in built products which caused issues and could have been spotted years ago, I felt ridiculed. Of course testers or QA (whatever way people want to call these specialists) should see implementation details. Is this developer really thinking that their design and implementation is perfect that we should look just at the results produced?

Issues can be spotted when getting to know algorithms and implementation: you may spot a logical error which causes certain bugs before you even look at the data obtained from running the algorithm

I stood my ground then, though. I tried to explain that I would love to see the implementation because it will help me to do the “QA processes” faster, more efficient and may display me some of issues before I actually look at the data. I want to be familiar with what it is actually doing.

And, to my surprise, it worked. This very same developer who was fighting that QA does not need any details on implementation shared with me the code they wrote to produce the results. It turns out that they thought I needed detailed documentation, but even code was enough which could easily be provided.

In the end, I realized that I could have given up. I could have closed myself up and exaggerated thinking that it’s only me who cares about proper quality judgement and people just assign tasks blindly without even considering that there may be issues in their logic of implementation. I could have felt hurt by the words used and impressions I got from this person, but in the end, even if we spoke in different terms, we both aim to finalize quality assesment (not to pass the QA, just understand if this implementation is good enough). I stood up for myself trying just to do my job better and I got help even if it took an extra step.

So, dear tester, believe that your colleagues are there to help you – you all want your products to be successful and of great quality. It is not only you, just sometimes others don’t know what you exactly need to do your tasks – open yourself up and ask for it. Only by sharing your needs and communicating you can make others understand your tasks better. 

 

 

 

Advertisements

Why phrase “pass the QA” makes me cringe

Today I heard someone say “pass the QA” and in this post I will share why I believe that we should cross out this phrase from all dictionaries where it is included because it is just wrong use of definitions.

Let’s break this phrase into two parts: QA and pass.

What is QA?
I am talking here in a sense of quality assurance. Okay, that sounds clear, however, what is quality assurance?

A lot of people mix up QA and testing on a daily basis. There have been various discussions about it and I mostly lean towards the point of Michael Bolton in his post Testers: Get Out of the Quality Assurance Business. It was an eye opener blog post for me: my first job as a tester even had it in the title “Software Quality Assurance Analyst”. Later on, I turned into QA Engineer. However, I am a tester.

Michael Bolton in that blog post gives so many valid points, it’s like a gold mine. It is one of my favorite ever posts about testing. It basically stresses that as a tester you cannot really assure quality. You just inspect it and help to improve it. You test.

QA is not a person or the department of certain type of professionals. It is a task of everyone in the company to work towards assuring quality. Tester may play a huge part in it, but the actual “action” assurers of quality usually are programmers because they actually are making changes to the quality level. And, let’s not forget the main part:

Assuring quality is an ongoing task/goal of the company.

What does it mean to pass?

In testing passing the test means that the test has passed based on its acceptance criteria.

Test may be built from multiple specific test cases or lead by charters. However, the defined acceptance criteria should be clear.

Passing of tests could be related to the common question in testing: how much is it enough to test? Sometimes the answer is not that obvious. There may be various scenarios, explorations to be made and a common standard should be discussed with product management team on what are the requirements and if edge cases should be addressed for the initial release/iteration.

Why don’t I like the phrase “pass the QA”?

After explaining both parts of this phrase, I can say that for me saying “pass the quality assurance” makes almost no sense.

Quality assuring is an ongoing task, so it is never going to end. You cannot pass the quality assurance as it is, but you can pass the test.

I do understand the intent of this phrase and why it was used: it was meant to say that testing will be completed with no show-stopper issues and will pass the acceptance criteria.

Let’s not underestimate the power of wording. Saying “pass the QA” can definitely be misleading. However, sad news are that this term is quite popular to describe the teams of testers. In this case, let’s spread the awareness of the differences between QA and testing – we all are doing QA in the company, but only testers do testing as their full time job (programmers do a fair deal of testing as well, but it is not their main responsibility usually).

 

 

 

My first pair testing session – Day 21 of 30 Days of Testing

PAIR TEST WITH SOMEONE

Some time ago I’ve read a great article by Katrina the Tester on Pair Testing. This topic since then would often pop up, but I never had a chance to actually try it out.

I am very grateful and lucky because I have super supportive colleagues when it comes to 30 days of testing challenge. Not only that they show a sincere interest and ask about the progress, but they also agree to get involved into some of the challenges. At the very start of July, I was talking to one of our great front-end developers and mentioned pair testing as one of the challenges. He immediately said “I can do it with you!”.

Ironically, today I did not have any specific task which we could pair test, so we thought of testing an existing feature.

In the very start it was a lot about seeing how he does testing. It is fairly different than my testing, so it was very beneficial to me. He showed how he configured dev environment locally for testing so he could fix any coming up issues as well immediately.

Then, we tried to specify on how this feature should be tested. This involved some UI tests verifying requirements, but not only that – we looked at monitoring, requests being sent. Usually, when I test, I concentrate on UI when the feature is mainly seen on the UI, but it was great to see how differently the very same feature is viewed by a developer who knows what events are being recorded and which requests should be sent for the feature to work. He as well showed me some of his unit tests for that feature.

To sum up, we asked when we can say that we have tested the feature completely. How much is enough? What should we do to be sure that it’s well tested?

This question is a challenging one. We decided that in addition to what we discussed, we should as well just try to be users for a while and try to use the product. Funny enough, trying a few of scenarios – we found a bug. In a tested released feature in production.

Here came in a very important point in testing: while testing one of the features you can have blind spots on some of inconsistencies on other existing features, so it’s best to stay alert and check for regressions which concentrate not only on the new feature.

Pair testing definitely exceeded all my expectations: I learned a lot new ways how to test, and, seeing the product with the help of different set of eyes even helped us to find a bug!

And, it was all about improvisation and sharing knowledge rather than dictating the rules.

128istqb

Illustration above taken from brilliant Cartoon Tester.

 

 

 

 

TestCast podcast “Testing is Dead”: Day 3 of 30 Days of Testing

LISTEN TO A TESTING PODCAST

I have never listened to a testing podcast before, so this was my first one ever -testing challenge is already yielding results in teaching me new things.

Not knowing where to start, I created a Google search with “testing podcasts”. One of the first results was a list of Software Testing Podcasts made by Ministry of Testing. I do trust this resource, so without further hesitation, I gave a go to #1 on the list – TestCast. The only problem with the list is that it’s pretty old – dating back to 2011.

I was looking for an interesting title for the first time podcaster and chose Testing is Dead. It is around 30 minutes discussion made by Bruce McLeod and Trish Khoo.

In the very start, they talk about some big names in the industry which claim that testing is dead. For example, Google or Facebook often would give examples in industry where testers are not present and developers do their own testing. Trish and Bruce gave some valid points that giant companies like that can give distorted view to other companies and it should be rather presented as their own way of looking at things and it does not mean that no testers approach would work for each company.

The way Google or Facebook can “reassure” their quality is users. They can be their first testers. If Google search does not work – a user may rather blame her own connection than the service itself. Nevertheless, Google can release their produt to one demographic only (let’s say a few thousands of users) and monitor the change. This approach may not work in other companies where pre-testing and quality is very important.

Later Trish and Bruce moved towards the fact that in many companies people don’t understand what testers are supposed to do. If management just wants a “safe net” because they feel that testing should be a part of the better quality, but they don’t really know what testing is, it may lead to test script writing, amount of work done estimated by checkmarks and actually, not testing, but checking.

To sum up my first podcast experience: it was a bit strange just to listen without seeing the presenters, it could have benefits of doing other task, but I am pretty bad at multitasking. Talking about the content, even if it’s back from 2011, but definitely we have same problems in 2016. Awareness of testing and what testers are supposed to do is a big problem and the approach to work with or without testers is just a choice. So far, testing is not dead and does not seem to be dying.

What I’m doing at work: Day 2 of 30 Days of Testing

TAKE A PHOTO OF SOMETHING YOU ARE DOING AT WORK

I had to cheat a bit on this one: it’s Saturday today, so I am not at work. I had to take a picture yesterday, however, I’ll share it now.

13570268_10209801873230058_84405356_o

This looks like a wonderful job for a woman – doesn’t it? All day looking at clothing. My boss sometimes remarks “You’re again shopping!”.

I am testing fit solutions and visual similarity algorithm. It is a pretty exciting task to do with many layers and challenges. This picture just reflects a common screen view you can see on my computer when you pass by.

Rebirth as an Omega Tester

Last time I wrote an entry in this blog was 2 years ago. Definitely, a lot has changed and it’s time for a rebirth.

Reasons why I got silent were simple: I got tired at my work, it became monotonous and lacking of challenges (later reasons became simpler: I forgot about the blog, it was not a habit anymore and I was pretty busy).

More than a year ago, I decided to look for a new job. With a bit of luck and accidents, I stumbled upon an ad for a first full-time software tester in a startup. As I have met all of the requirements and job sounded challenging enough, I just had to apply even if it was based in other country (where I have never been before).

Response was quick and inspiring: I got homework (I always think it’s a great sign of a company – you can prove yourself with actual task). After that, I had multiple Skype interviews and… quit my job (which is definitely a tough thing to do when it’s your first job), packed my bags and moved to a new land of opportunities.

I must tell you that this change was the happiest change in my life. Now it’s more than a year that I work as an only full-time tester in a promising and exciting startup! I have many stories to share, so, it’s about time to make a rebirth.

Changing from manual tester executing test scripts in a big international company to being a sole tester in a (sometimes chaotic) startup is a huge difference, but, oh boy, how worth it is! Advice for you:

If you ever wake up at least a couple of days in a week thinking “I don’t like my work and don’t want to go there” – it’s about time to change your work. New opportunities are around the corner waiting for you.

And, if you find yourself wondering, why did I call myself an Omega tester, there is a great article by James Bach: Omega Tester: Testing with a Team One. Definitely worth reading it!

How to Attract Interesting Bugs: Use your Talents and Be a Software Terrorist

The release was supposed to be tomorrow, and, today I found a really interesting bug. I think it is the bug that I’m the most proud of so far… or at least today. (:

I will tell what that bug was like in this post, but before that I must say how my colleagues reacted after getting an e-mail with developer’s commit and bug’s description: “What? Why did you do that? How did you manage to find that bug?”. And the answer is pretty simple: I try to make my testing nice and not monotonous…

Collect the best pictures: art, photography, travel, whatever you like and use it when you need to test something with Insert Picture, or, with image files in general. 

This seems so unimportant, right? Who cares what kind of picture you insert while testing. But, hey! You get tired of those default Windows pictures with tulips or penguins. And, let me tell you a secret – a picture which you like looking at can do wonders. Your imagination strikes with all this creativity/nice view in the image file. You suddenly feel happier and testing is more fun. To make it even more interesting, you could save quite many images you love with random names, so each time inserting an image you wouldn’t know what you are inserting exactly and get a bit surprised.

Practice your foreign language while testing. 

Modifying a file (usually it’s a text modification) may seem such a boring task – just type in ‘Aaaaa’ or any other ‘asafsfjdsf’. So dull, repetitive and what a mess it creates when you try to save a lot of files with random file names like that and you are never sure whether this random name is unique. Then again you need to change it, oh, boy, so annoying.
One way could be to think of interesting words, people to name your files, but…
Why not to practice your foreign language skills? Try to use testing as a chance to repeat newly learned words and just have fun while doing so. 🙂 It will definitely be more fun to write a funny word in a language you are studying instead of ‘Aaaa’. 🙂

Take inspiration from music.

If you lack of words in a foreign language, why not to listen to a radio of that language you are learning while testing? Then when cannot think of any word – just write down the one you hear in the song. In general, music can help to create a good vibe and mood for testing. However, this is not for everyone (for example, I cannot listen to it all the time, I need to get silence, but sometimes it really helps to refresh my mind).

Be a Software Terrorist. 

A what? It’s time to tell about my lovely bug which made our release shift even more and a developer send me a message saying “You’re a terrorist” (I am not a terrorist, just a software terrorist…). 🙂
Try things out, have fun and you will find bugs like these…

***

I turned on the software, inserted a lovely travel photo as a background picture, and, thought of a song I just heard on a Swedish radio (I am just a beginner, and, they say it’s really good to hear the language constantly, so I try to listen even if I understand only a few words). It had lyrics like: “wherever i turn in the world i’m standing here with empty hands”, so, I thought that this time my modification will be “empty hands”, but in Swedish. I wrote: “tomma hander” (without Swedish characters). Then I wanted to check some shortcuts and pressed Ctrl and got surprised. We test integration of products like that with a file system product of ours. And, there showed up the login to our file system. This is weird, I thought. Ctrl itself should not make any dialogs open. After log in there came integrated Open dialog… So, basically, it meant that if a user wants to save the new document with “tomma hander” and presses Ctrl+S he won’t be suggested to Save the file, but Open it from that file system product. This may be very annoying for a user working with large files and using shortcuts.
I would have never found it if I typed in ‘asdajdaskdj’… The bug itself was as follows:
every time typing the letter o in the document (either it alone, or, in the text) and then pressing Ctrl key would work as Ctrl+O and give log in to our software and integrated Open dialog. 

A secret of a happy and productive tester: be not only passionate about testing, but fuel it with other passions you have in life. 

 

In the land of bugs or why you shouldn’t blame yourself for missing a bug

2 months that I’m a software tester. I have learned a lot, but there is a lot of other things to be learned. 

In the previous post I was talking about the relationship between developers and testers. Now, I think it’s time to talk about problems with other testers, and, inner hurricanes of emotions. 

You won’t catch all bugs. It may sound sad, it may break your perfectionist’s heart, and, on some level, you realize that you will never be perfect at your job… However, this is the beauty of being a tester – there is no comfort zone. The learning is constant and sometimes it does hurt. Sometimes you do shout on yourself and regret not checking something. 

There are pluses and minuses of working in a team of testers with testing cycles. I must admit that some testers make me bitter by skipping quite important bugs. They just seem to ‘float on the surface’ of the test plan and don’t put too much of attention in actually finding bugs. But… it’s good to work in a team and get the same thing tested by several people. This prevents possible major bugs of being unnoticed. 

It breaks my heart when the test plan I did before and all its bugs got fixed, fails for some other tester. I feel like I did not notice something, or, did not put too much of importance to something. However, there are a few main things to remember why you shouldn’t be angry on yourself for letting the bug pass unnoticed:

New builds may cause trouble.
You shouldn’t blame yourself for not finding something that was not even alive in a previous build. Things happen, and, sometimes, by trying to fix something – we ruin something else. 

Importance of the bug is subjective. 
Of course, you should write down everything. However, at times the bug seems so tiny, that you consider it as a feature. You simply take it that it works this way. For example, you may think that in that application hyperlink does not get created automatically. Then after a while someone else tests the same thing and files the bug about it. It gets fixed. The feeling is not too good as you were aware of it, but thought that it’s not important (as there was an obvious workaround and manual creation). 

In conclusion, bugs are everywhere. There are lots of bugs, but you must know which ones are the most important and must be fixed for this release. And, I found this lovely quote in a book called Testing Computer Software by Cem Kaner, Jack Falk,and, Hung Q. Nguyen: 
Image

Responsibility in Software Testing

After people hear what I actually do, they somehow start thinking that maybe they should change their field and go into testing as it seems so easy:
Seriously, how come this job needs no pre-requirements, no degrees, anyone can do it, and, yet, it gets paid pretty well?

No. Not everyone can be a software tester. Not only you have to think constantly and be technology-friendly, but the main thing a tester has to deal with is… responsibility. 

I got this developer lunch buddy. We somehow synchronized our patterns and meet during the lunch break. Being all so curious, I asked him what he thinks of testers in general. Is it true that some developers look to testers from high above? Of course, he was not as emotional as a female and said that he does not feel any hard feelings on testers. However, to be honest, there are only a few good testers. Most of the testers just work… to work. Not to name the teams of countries (somehow he got bad experience with certain nationalities), but as he says “I don’t understand, how come for them everything passes, and, whatever I run – it does not work?”. 

If you want an easy life, you could just write pass on every single test case and act that everything works fine, or, as it worked in a previous release. Then you would just sit in the office, maybe playing solitaire or sitting on facebook, mark test plans as pass, and, live happily ever after. Not forever, though. Product gets released, users notice bugs, and… it comes back to you. 

Passing every test plan is somehow an extreme case. However, it can be also the case, that a tester bumps into just main and obvious bugs, or, sticks to the test plan if there is one. The scariest part is that:

Tester is responsible for a bug he did not notice. 

Even if a developer is the one who created the software, but… When it comes to testing and getting paid for that – everyone hopes for the best job possible. Then developers just take reported bugs one by one and fix them. Developers do not report new bugs as they trust testers. This means that all bugs that are left in the software first of all are because of lousy testers. 

When I started out, I would just blindly get stuck to the test plan. I was scared to play around. Last week, my manager wrote to me about a project I worked on my first week. It figures there was a test plan on 2014 version of software installed together with version of 2013. I did that smoke test and it passed, nothing new. And now support says that users say that integration for 2013 stops working when 2014 is installed! Manager is like “Did you check 2013? It was not written but I take it as kind of a common sense to do that.”. My answer was really simple “My common sense was non-existent at the very start. I did what was written, not something more.”. After a day he made me redo that test on 2013. It passed. It figures that I was lucky this time: users themselves did something wrong (later reported that it works again). I said to my manager that “oh well, magic happens”. And his reply was: “There is no magic in software.”.

After all this we come to a one more important thing. Managers just plan things, developers just fix things, and, well…

Testers need to have a big picture of the software and work for it to be bug-free themselves.

This means that a tester cannot just be stuck with a test plan. A tester needs to look around and think what he’s testing, what else could be somehow complicated. Even if test plans have a time limit, but… We work for a bug-free software and have to work hard to achieve it.  

One more thing is that only testers are like users pre-release. Usually, the bosses are so high above that they don’t even write requirements what they hope from the software. Developers don’t know themselves what bosses want. Manager just makes sure that we all are getting along. 

Testers have to decide how software is going to be. 

Of course, it comes with discussions with management and developers. However, we write bugs about changes that need to be made, we make suggestions, and, we get asked even by developers – how should it work. This is a difficult task at times as we would like just to work and not to think too much. However, creation of software is a long process. 

To sum everything up, there is no easy life in testing. If you want to be a good tester, sometimes you will spend hours trying to figure out something that may not be working. And, you cannot trust previous testers – maybe they are one of those always pass people who live comfortable lives. 

Bugs come in through open Windows

Today I’m in a deep sorrow. I lost those two my beloved weirdo bugs which have magical powers.

I think it’s time for me to tell you all their beauty and my silliness. I was testing integration with Microsoft Excel, and, our product has its own ribbon when add-in is installed. In that add-in there is special Insert Object button. That’s the place where bugs decided to live and prosper. After inserting an object which is also an Excel file a blank Excel document would open up, and, cause a lot of trouble: closing it you would close all the file and file’s window would become blank, closing that blank window would end up in Excel not responding. The beauty was that not with all files this would happen. It would just be sort of random!

The second bug was with another part of files who usually did not have the first bug effecting them. So, it seems that everything works great: object is inserted and no blank window appeared. Dream on. Dragging the object Excel’s blank window would open up and flash. Creepy, right?

As a noob software tester, I checked the previous release with Excel 2010, and, of course, there were no bugs like that. I got so happy  and filed them even if later on I was confused several times when bugs did not reproduce. 

Today software developer writes to me and breaks my heart (of course, he is young, smart, and, cute… what else guys like that can do than break hearts?). It figures out that non-integrated Excel cannot handle its own Excel files inserted as objects. So, at least one bug, is not our problem, but Excel’s. Another bug is sort of a “son” of the previous one. Our add-in even solves the problem somehow for some files. So, that blank page does not show up, but flashing appears. Our guess is that it’s just an effect of this “father” bug. 

Of course, I wouldn’t be me if I did not say to a developer that he made me sad, and, I feel like a mother who just figured out that her two kids are actually not hers.  Our scrum calls are more than formal, but today that developer could not hold himself and started laughing when telling about discussing the bugs with me. 

To sum up, the lesson I’ve learned today:
It’s not always your products fault, check the native application, because Bugs come in through open Windows, and, well, everything else that’s made by Microsoft.