Give someone positive feedback – Day 30 of 30 Days of Testing


I couldn’t imagine of a better way to end an amazing journey of 30 Days of Testing. After all days filled with various testing tasks the final touch is personal, positive and related to people around us.

One of previous challenges was rather work-based- to say something nice about product you tested. It is closely related to today’s challenge as you may say it to a developer who made the product, but today it’s time to say something personal!

In my company, every three months we have personal feedback sessions with our CTO and VP of Engineering. Not only that we get feedback about ourselves, but also we are encouraged to express feedback about everyone in the team (including the people who are in the room). Usually, even if there are some things to improve, positive feedback is a big driving force which calms you down and makes you realize that people value you.

However, I never said positive feedback on purpose to someone personally, so today I chose a programmer who I greatly value, but maybe do not express that enough, and wrote that to him.

The experience was pretty rewarding! I had some struggles to formulate the words and it took a while to click  “Send”, but I feel that he really deserved it and hope that it will make his day. It would make mine.

In the end, I believe that it would be nice to give (and receive!) positive feedback real-time, not on challenges or feedback sessions, but when you feel like it.



Off-by-one error hunt – Day 29 of 30 Days of Testing


I did not know where to start with this challenge: what is an out by one error?

After some googling, I decided to stick to off-by-one error and its definition from Wikipedia:
An off-by-one error is a logic error involving the discrete equivalent of a boundary condition. It often occurs in computer programming when an iterative loop iterates one time too many or too few. This problem could arise when a programmer makes mistakes such as using “is less than or equal to” where “is less than” should have been used in a comparison or fails to take into account that a sequence starts at zero rather than one (as with array indices in many languages).

I did discuss this with some friends and they confirmed that it must be the same as “an out by one error”. They gave some more practical examples as well:

  • an array has 10 elements and you want the fifth one, so you do arr[5], but since arrays might be indexed from 0, the fifth element is arr[4] and you’re off-by-one;
  • Vitruvius fencepost error: if you build a straight fence 30 meters long with posts spaced 3 meters apart, how many posts do you need? You may think that you need 10, but to make 10 sections, you need 11 posts.

I am super grateful for wonderful friends as one of them doubting that I will be able to find any of these errors as they are quite easy to check and rare in production, created my own broken counter! If you write a number to which this little program should count – it will start counting with 0 (intentional programmer’s error):

I did not want to give up this easily, so I invested some time checking other sites.I was thinking of where any kind of counting would be done, and, ended up checking e-commerce sites: their searches and price filtering (searching for < instead of =<).

I went to Vero Moda’s Day Dresses category. I decided to check if price filters are working as they say: if there is a price filter “10-20” I expect that it would include both 10 and 20. So, I checked first 10-20 filter and filtered results by High to Low price:
As you can see, there are no 20 pounds dresses. I may guess here that there is less than 20 filter (which user doesn’t know), not less or equal as you may assume. However, maybe site simply does not have dresses which are worth 20 pounds? So, I checked filter 20-30 and filtered prices from low to high to see if there are 20 pounds dresses:


And it turns out that there are dresses which are worth 20 pounds. This means that I found an off-by-one error:

In Vero Moda’s price range filter, upper bound is not included: it is < instead of =<.

This is definitely not a huge mistake, but may cost a lot: if user’s budget is up to 20 pounds after filtering to 10-20 price range she may not find what she wants, thus she may end up not buying anything. Maybe some of the dresses which were costing equally 20 dollars would have been most attractive to the user, but there was no chance to see them with the filter on.

30 Days of Testing is moving towards the end and I am so grateful that I need to google some of the definitions because I did encounter off-by-one errors before (especially with strict bounds), but never knew the exact name. Now I do and it feels great to have a term which describes a specific type of a problem!

Summarizing an issue in 130 (originally 89) characters – Day 28 of 30 Days of Testing


Most of testers have to do this every day: we don’t write tweets about bugs, but the title of it definitely has a character limit and should be as short as possible.

My issue summary is a real-life example I’ve encountered today. I wanted to buy train tickets in Lithuania. I went to the site for buying train tickets.

The very main page has a very basic UI for buying tickets: one way, round trip radio buttons; two selects for the cities you are traveling between, passengers number field and Search button.

I am not traveling alone, so I wanted to change passengers number to 2. This is where I encountered a problem that even if I changed it in opened selector – number of passengers did not change. So here is a short summary of this bug:

User cannot select more than 1 passenger buying an online ticket with Lithuanian Railways

Here you go. 89 characters summarizing the problem! And, well, I must include a screenshot to depict that there is no change in Passengers field after selecting more passengers:


Update: my issue summary just expanded! And, issue turned out to be the UX issue. Prashant just pointed out that there actually is Save Changes button! I did not see it because my computer’s screen resolution is smaller. Looking at the same site with bigger resolution site looks like this:
Screen Shot 2016-07-28 at 9.43.17 AM
Save the changes actually does work and changes passenger number. So, second shot to my issue summary:

User with resolution height less than 666, cannot see “Save the changes” button which prevents them from changing passenger number

Screen Shot 2016-07-28 at 9.49.06 AM

Now the character count is 130, but still challenge completed. 😉

Say nice things about what you test – Day 27 of 30 Days of Testing


Usually testers are the ones who report the bad news. This is a professional behavior in a way: we need to spot mistakes and let the team know about it as soon as possible so they would get fixed. The positive feedback often is left unspoken because we are concentrated on finding the negatives.

Being the lone tester, I work very closely with people whose work I have to test. Be it a product manager or a developer – I ask them questions, we collaborate and clarify the requirements and implementation. This allows me to feel closer to the team and as well get happy when something reaches final implementation. Sometimes when my verification is done I express positivity about what I tested: usually it is rather a “well done!” comment in JIRA, but there are also cases when we discuss the new feature’s “superpower” together with the programmer.

In general, my long-term aspiration is to soothe the pain of bad news with acts of appreciation and interest, so I try to say nice things about what I tested. Nevertheless, today I tried to do that even more!

I talked about the new feature with the programmer who has been working on it for a few months now. I really do believe that it’s a great investment in the long-run and I have noticed an amazing improvement in quality of his work. It was great to test a stable and pretty version of the product with no surprises except from a few cosmetic UI issues. As the result of this conversation, mood was lifted for both sides and the programmer shared his own nice thoughts about the same feature.

The greatest part of saying nice things about what you tested is that the programmer will get appreciation and even may explain more about the internals of the feature which helps you to test better.

Positivity is a win-win situation: helps to build a strong team with members who respect each other, share not only their problems, but wins as well. 



Non-tester, come to testing meetups! – Day 26 of 30 Days of Testing


In one of previous challenges I created a post on how to find testing events. Today’s challenge is half done knowing which event you’d like to invite people to.

So, to any of non-testers reading this, I would like to kindly invite you to come to Budapest’s QA meetup’s palinQA events! Upcoming one will happen in September.

Great part about this meetup is that each time we have some non-testers joining us. A lot of topics are very interesting even if you’re not a tester. A great example of this is the latest meetup we had on Making a QA Career. Not only that the audience had testers who wanted to know more about job opportunities, but also people who are interested in the QA career, or, even non-testers (who possibly won’t change their career) come to listen to what qualities should testers have and how it feels to start a career as a tester.

Join a testing meetup and come to get to know the breed of testers: how they feel, what challenges they face and it will definitely give some insight to their work. The more insight there is – the better team we can create.


My top 3 online spots for testing discussions – Day 25 of 30 Days of Testing


Some time ago I wouldn’t have guessed that there are so many forums and places to talk about testing. However, now I know a few and these are my 3 favorite online spots for testing discussions:

  1. uTest forums
    Recently I have become a more active uTester. It’s a great place to meet all kinds of testers: from beginners to advanced. Not only that it is a testing platform, but as well uTest has a strong community section and aims to be “twitter for testers“. There I can share knowledge, meet people who are interested in testing and are able to contribute to testing discussions.
  2. Software Testing Club
    I found Software Testing Club via Ministry of Testing. What I like a lot about it is that discussions are more “moderated” than in uTest. They are not as frequent, but at the same time more professional and it’s easier to find valuable content.
  3. Twitter/blogs
    This point is rather an obvious one – following testing professionals on Twitter you can read a lot of interesting testing discussions and be a part of it. Same goes for blogs.

Today I contributed to a testing discussion on uTest forums!

Getting to know more testers – Day 24 of 30 Days of Testing


30 Days of Testing definitely introduced me to multiple testing community members whom I did not connect with before. It has been very rewarding to feel the sense of community!

Nevertheless, today I spent some time connecting to possible speakers for Budapest QA meetup. Sometimes it can be very tough to find new speakers, so a lot of research has to be put in it. So, I have looked at some nearby testing meetups and connected to some of the organizers/speakers from there.

A wonderful thing was that I got very positive replies and now we will try to keep in touch to make sure to arrange a talk when they are here.

If you are interested in speaking at Budapest’s QA meetup and tend to come here sometimes – let me know, we would like to get to know you.


How I helped the account manager to test better – Day 23 of 30 Days of Testing


Quality of a product is a very important topic to anyone in the company, so ability to quickly check real-time status of major product’s KPIs is a great skill to have.

In my company, very often some of the checks are done by our account manager. Imagine a situation where a partner writes in with a problem with web-service which your company is delivering. Account manager is usually the first person to read this, check the problem and reply to the partner. In other scenarios, account manager may just need a quick check of certain partner service quality before making any kind of communication step with them. Have in mind that sometimes other colleagues are not able to help immediately, so account manager would clearly benefit from knowing how to execute some checks independently.

Our account manager is a great learner and has been very open to get to know more about the product, so we arranged a meeting to talk about my beloved monitoring tool – New Relic.

I have explained to him some basics of New Relic: where to find dashboards, how to see queries behind each dashboard, how to make your own queries. We created some queries together to check a few important questions and modified existing ones to get the grasp of how to build queries if you don’t know where to start.

Of course it will take some time for the account manager to get used to this new tool, but I’m sure that he got a good background now and will be able to check out some of important questions real-time.

It was very fun for me to share knowledge and we even found out together that one of device types for our web-service in the last 2 weeks was a car, specifying which car it was we learned a fun fact – our web service was used by Tesla car owners.


Ode to New Relic – Day 22 of 30 Days of Testing


I must admit that I love using analytics and monitoring in my testing. This is why my favorite tool is New Relic! It’s a software service which monitors web and mobile applications in real-time.

You may wonder why I call it a testing tool: I believe that monitoring and analytics are a huge part of regression testing. If numbers are still correct (or better), it can be a very valuable KPI to measure the system change. And that is not the only benefit in testing that New Relic can give.

I am going to list 3 reasons why I love using New Relic Insights (that’s the part that I tend to use the most):

  1. You can collect statistical significance and evidence of reported bugs.
    In most of the cases, I can check real-time data and see how many users were affected by a certain issue. For example, if found issue happens for a specific set of items, I can check the percentage of all requests which were made on those items. The bug will definitely need to be fixed if the impact is high.
  2. You can find out your user demographics and devices/browsers they use.
    It is very important to know who is your audience. If most of your users are based in India, it is likely that they use different devices than the users in the USA. This can help you to decide which devices and browsers you should test on and which location settings are important. For example, the following screenshot displays what are the most popular desktop and mobile browsers among the users of the product I test:
    Screen Shot 2016-07-18 at 5.46.25 PM
  3. By monitoring the system, you can find the issues easily.
    You can create all kinds of monitoring dashboards. Our product is a service, so knowing that service is functioning correctly means a lot. If there is a drop in one of its functions – we have a problem. For this reason, I have created a dashboard for myself including all important KPIs to me. Every day I check to see if system is in tact, if not – I query for more specific examples and investigate what could be the reason. Nevertheless, after releases, I always doublecheck the dashboards for any possible regressions. It helps me to make sure that system continues working as expected real-time.

New Relic Insights has given me more confidence in bug reporting because I can add a real-life value and importance to them. Also, it helps me to do quick regression checks and investigations which I was not able to do this easily pre-New Relic.



My first pair testing session – Day 21 of 30 Days of Testing


Some time ago I’ve read a great article by Katrina the Tester on Pair Testing. This topic since then would often pop up, but I never had a chance to actually try it out.

I am very grateful and lucky because I have super supportive colleagues when it comes to 30 days of testing challenge. Not only that they show a sincere interest and ask about the progress, but they also agree to get involved into some of the challenges. At the very start of July, I was talking to one of our great front-end developers and mentioned pair testing as one of the challenges. He immediately said “I can do it with you!”.

Ironically, today I did not have any specific task which we could pair test, so we thought of testing an existing feature.

In the very start it was a lot about seeing how he does testing. It is fairly different than my testing, so it was very beneficial to me. He showed how he configured dev environment locally for testing so he could fix any coming up issues as well immediately.

Then, we tried to specify on how this feature should be tested. This involved some UI tests verifying requirements, but not only that – we looked at monitoring, requests being sent. Usually, when I test, I concentrate on UI when the feature is mainly seen on the UI, but it was great to see how differently the very same feature is viewed by a developer who knows what events are being recorded and which requests should be sent for the feature to work. He as well showed me some of his unit tests for that feature.

To sum up, we asked when we can say that we have tested the feature completely. How much is enough? What should we do to be sure that it’s well tested?

This question is a challenging one. We decided that in addition to what we discussed, we should as well just try to be users for a while and try to use the product. Funny enough, trying a few of scenarios – we found a bug. In a tested released feature in production.

Here came in a very important point in testing: while testing one of the features you can have blind spots on some of inconsistencies on other existing features, so it’s best to stay alert and check for regressions which concentrate not only on the new feature.

Pair testing definitely exceeded all my expectations: I learned a lot new ways how to test, and, seeing the product with the help of different set of eyes even helped us to find a bug!

And, it was all about improvisation and sharing knowledge rather than dictating the rules.


Illustration above taken from brilliant Cartoon Tester.