Summarizing an issue in 130 (originally 89) characters – Day 28 of 30 Days of Testing

SUMMARISE AN ISSUE IN 140 CHARACTERS OR LESS

Most of testers have to do this every day: we don’t write tweets about bugs, but the title of it definitely has a character limit and should be as short as possible.

My issue summary is a real-life example I’ve encountered today. I wanted to buy train tickets in Lithuania. I went to the site for buying train tickets.

The very main page has a very basic UI for buying tickets: one way, round trip radio buttons; two selects for the cities you are traveling between, passengers number field and Search button.

I am not traveling alone, so I wanted to change passengers number to 2. This is where I encountered a problem that even if I changed it in opened selector – number of passengers did not change. So here is a short summary of this bug:

User cannot select more than 1 passenger buying an online ticket with Lithuanian Railways

Here you go. 89 characters summarizing the problem! And, well, I must include a screenshot to depict that there is no change in Passengers field after selecting more passengers:

traukinys

Update: my issue summary just expanded! And, issue turned out to be the UX issue. Prashant just pointed out that there actually is Save Changes button! I did not see it because my computer’s screen resolution is smaller. Looking at the same site with bigger resolution site looks like this:
Screen Shot 2016-07-28 at 9.43.17 AM
Save the changes actually does work and changes passenger number. So, second shot to my issue summary:

User with resolution height less than 666, cannot see “Save the changes” button which prevents them from changing passenger number

Screen Shot 2016-07-28 at 9.49.06 AM

Now the character count is 130, but still challenge completed. 😉

Say nice things about what you test – Day 27 of 30 Days of Testing

SAY SOMETHING NICE ABOUT THE THING YOU JUST TESTED

Usually testers are the ones who report the bad news. This is a professional behavior in a way: we need to spot mistakes and let the team know about it as soon as possible so they would get fixed. The positive feedback often is left unspoken because we are concentrated on finding the negatives.

Being the lone tester, I work very closely with people whose work I have to test. Be it a product manager or a developer – I ask them questions, we collaborate and clarify the requirements and implementation. This allows me to feel closer to the team and as well get happy when something reaches final implementation. Sometimes when my verification is done I express positivity about what I tested: usually it is rather a “well done!” comment in JIRA, but there are also cases when we discuss the new feature’s “superpower” together with the programmer.

In general, my long-term aspiration is to soothe the pain of bad news with acts of appreciation and interest, so I try to say nice things about what I tested. Nevertheless, today I tried to do that even more!

I talked about the new feature with the programmer who has been working on it for a few months now. I really do believe that it’s a great investment in the long-run and I have noticed an amazing improvement in quality of his work. It was great to test a stable and pretty version of the product with no surprises except from a few cosmetic UI issues. As the result of this conversation, mood was lifted for both sides and the programmer shared his own nice thoughts about the same feature.

The greatest part of saying nice things about what you tested is that the programmer will get appreciation and even may explain more about the internals of the feature which helps you to test better.

Positivity is a win-win situation: helps to build a strong team with members who respect each other, share not only their problems, but wins as well. 

 

 

Non-tester, come to testing meetups! – Day 26 of 30 Days of Testing

INVITE A NON-TESTER TO A TEST EVENT

In one of previous challenges I created a post on how to find testing events. Today’s challenge is half done knowing which event you’d like to invite people to.

So, to any of non-testers reading this, I would like to kindly invite you to come to Budapest’s QA meetup’s palinQA events! Upcoming one will happen in September.

Great part about this meetup is that each time we have some non-testers joining us. A lot of topics are very interesting even if you’re not a tester. A great example of this is the latest meetup we had on Making a QA Career. Not only that the audience had testers who wanted to know more about job opportunities, but also people who are interested in the QA career, or, even non-testers (who possibly won’t change their career) come to listen to what qualities should testers have and how it feels to start a career as a tester.

Join a testing meetup and come to get to know the breed of testers: how they feel, what challenges they face and it will definitely give some insight to their work. The more insight there is – the better team we can create.

50bugsevery

My top 3 online spots for testing discussions – Day 25 of 30 Days of Testing

CONTRIBUTE TO A TESTING DISCUSSION

Some time ago I wouldn’t have guessed that there are so many forums and places to talk about testing. However, now I know a few and these are my 3 favorite online spots for testing discussions:

  1. uTest forums
    Recently I have become a more active uTester. It’s a great place to meet all kinds of testers: from beginners to advanced. Not only that it is a testing platform, but as well uTest has a strong community section and aims to be “twitter for testers“. There I can share knowledge, meet people who are interested in testing and are able to contribute to testing discussions.
  2. Software Testing Club
    I found Software Testing Club via Ministry of Testing. What I like a lot about it is that discussions are more “moderated” than in uTest. They are not as frequent, but at the same time more professional and it’s easier to find valuable content.
  3. Twitter/blogs
    This point is rather an obvious one – following testing professionals on Twitter you can read a lot of interesting testing discussions and be a part of it. Same goes for blogs.

Today I contributed to a testing discussion on uTest forums!

Getting to know more testers – Day 24 of 30 Days of Testing

CONNECT WITH A TESTER WHO YOU HAVEN’T PREVIOUSLY CONNECTED WITH

30 Days of Testing definitely introduced me to multiple testing community members whom I did not connect with before. It has been very rewarding to feel the sense of community!

Nevertheless, today I spent some time connecting to possible speakers for Budapest QA meetup. Sometimes it can be very tough to find new speakers, so a lot of research has to be put in it. So, I have looked at some nearby testing meetups and connected to some of the organizers/speakers from there.

A wonderful thing was that I got very positive replies and now we will try to keep in touch to make sure to arrange a talk when they are here.

If you are interested in speaking at Budapest’s QA meetup and tend to come here sometimes – let me know, we would like to get to know you.

600_442114348

How I helped the account manager to test better – Day 23 of 30 Days of Testing

HELP SOMEONE TEST BETTER

Quality of a product is a very important topic to anyone in the company, so ability to quickly check real-time status of major product’s KPIs is a great skill to have.

In my company, very often some of the checks are done by our account manager. Imagine a situation where a partner writes in with a problem with web-service which your company is delivering. Account manager is usually the first person to read this, check the problem and reply to the partner. In other scenarios, account manager may just need a quick check of certain partner service quality before making any kind of communication step with them. Have in mind that sometimes other colleagues are not able to help immediately, so account manager would clearly benefit from knowing how to execute some checks independently.

Our account manager is a great learner and has been very open to get to know more about the product, so we arranged a meeting to talk about my beloved monitoring tool – New Relic.

I have explained to him some basics of New Relic: where to find dashboards, how to see queries behind each dashboard, how to make your own queries. We created some queries together to check a few important questions and modified existing ones to get the grasp of how to build queries if you don’t know where to start.

Of course it will take some time for the account manager to get used to this new tool, but I’m sure that he got a good background now and will be able to check out some of important questions real-time.

It was very fun for me to share knowledge and we even found out together that one of device types for our web-service in the last 2 weeks was a car, specifying which car it was we learned a fun fact – our web service was used by Tesla car owners.

 

Ode to New Relic – Day 22 of 30 Days of Testing

SHARE YOUR FAVORITE TESTING TOOL

I must admit that I love using analytics and monitoring in my testing. This is why my favorite tool is New Relic! It’s a software service which monitors web and mobile applications in real-time.

You may wonder why I call it a testing tool: I believe that monitoring and analytics are a huge part of regression testing. If numbers are still correct (or better), it can be a very valuable KPI to measure the system change. And that is not the only benefit in testing that New Relic can give.

I am going to list 3 reasons why I love using New Relic Insights (that’s the part that I tend to use the most):

  1. You can collect statistical significance and evidence of reported bugs.
    In most of the cases, I can check real-time data and see how many users were affected by a certain issue. For example, if found issue happens for a specific set of items, I can check the percentage of all requests which were made on those items. The bug will definitely need to be fixed if the impact is high.
  2. You can find out your user demographics and devices/browsers they use.
    It is very important to know who is your audience. If most of your users are based in India, it is likely that they use different devices than the users in the USA. This can help you to decide which devices and browsers you should test on and which location settings are important. For example, the following screenshot displays what are the most popular desktop and mobile browsers among the users of the product I test:
    Screen Shot 2016-07-18 at 5.46.25 PM
  3. By monitoring the system, you can find the issues easily.
    You can create all kinds of monitoring dashboards. Our product is a service, so knowing that service is functioning correctly means a lot. If there is a drop in one of its functions – we have a problem. For this reason, I have created a dashboard for myself including all important KPIs to me. Every day I check to see if system is in tact, if not – I query for more specific examples and investigate what could be the reason. Nevertheless, after releases, I always doublecheck the dashboards for any possible regressions. It helps me to make sure that system continues working as expected real-time.

New Relic Insights has given me more confidence in bug reporting because I can add a real-life value and importance to them. Also, it helps me to do quick regression checks and investigations which I was not able to do this easily pre-New Relic.

 

 

My first pair testing session – Day 21 of 30 Days of Testing

PAIR TEST WITH SOMEONE

Some time ago I’ve read a great article by Katrina the Tester on Pair Testing. This topic since then would often pop up, but I never had a chance to actually try it out.

I am very grateful and lucky because I have super supportive colleagues when it comes to 30 days of testing challenge. Not only that they show a sincere interest and ask about the progress, but they also agree to get involved into some of the challenges. At the very start of July, I was talking to one of our great front-end developers and mentioned pair testing as one of the challenges. He immediately said “I can do it with you!”.

Ironically, today I did not have any specific task which we could pair test, so we thought of testing an existing feature.

In the very start it was a lot about seeing how he does testing. It is fairly different than my testing, so it was very beneficial to me. He showed how he configured dev environment locally for testing so he could fix any coming up issues as well immediately.

Then, we tried to specify on how this feature should be tested. This involved some UI tests verifying requirements, but not only that – we looked at monitoring, requests being sent. Usually, when I test, I concentrate on UI when the feature is mainly seen on the UI, but it was great to see how differently the very same feature is viewed by a developer who knows what events are being recorded and which requests should be sent for the feature to work. He as well showed me some of his unit tests for that feature.

To sum up, we asked when we can say that we have tested the feature completely. How much is enough? What should we do to be sure that it’s well tested?

This question is a challenging one. We decided that in addition to what we discussed, we should as well just try to be users for a while and try to use the product. Funny enough, trying a few of scenarios – we found a bug. In a tested released feature in production.

Here came in a very important point in testing: while testing one of the features you can have blind spots on some of inconsistencies on other existing features, so it’s best to stay alert and check for regressions which concentrate not only on the new feature.

Pair testing definitely exceeded all my expectations: I learned a lot new ways how to test, and, seeing the product with the help of different set of eyes even helped us to find a bug!

And, it was all about improvisation and sharing knowledge rather than dictating the rules.

128istqb

Illustration above taken from brilliant Cartoon Tester.

 

 

 

 

Gruyere’s paradise for security testing – Day 20 of 30 Days of Testing

FIND A GOOD PLACE TO PERFORM SOME SECURITY TESTS

In many sources, security testing is called ethical hacking. It means that you create a controlled attack to evaluate the security status of your product.

To familiarize more with the concept of security testing, I read a very detailed tutorial. Then, after some googling, I found a wonderful place to learn more and actually perform security tests: Google’s Gruyere!

Gruyere is not only a great playground for security testing, but it has lessons and challenges to help you find vulnerabilities. If you get stuck, there are hints on how you should proceed.

I have read and tried out first few parts in Gruyere and had a great fun! It feels very nice to find security flaws. For example, I changed my account to be administrator account which enabled option Manage this server. Now I could edit any user.

manageserver

Security testing always looked like a very complex and difficult area to me. I have never performed any security tests before, so this seemed like one of the biggest challenges. However, after actually trying to perform some security tests, I can say that it’s actually not that difficult. It is hacking in a way, but it’s rather knowing the product you are testing really well and trying to think of the ways how to unleash the security flaws. I will definitely come back to this exciting topic and I am very grateful to Gruyere for being such a great place to learn and execute some security tests!

P.S.  Web Application Security Testing Cheat Sheet can come in handy for further learning.

Struggles and discoveries using te52 – Day 19 of 30 Days of Testing

FIND AND USE A NEW TOOL

Today I decided to try out tellurium 52. It’s a web-based simple automation tool. The main features of te52 are:

  • It’s free (this is always a plus!)
  • Tests are written in “Plain English”
  • Cross-browser testing
  • Scheduling

It sounded great, so I got my hands dirty.

In the start, it can be confusing on how to write a test in “Plain English” when web elements are way more complex than that. I write Selenium tests, so for me this more business-orientated way of writing tests was both something new and challenging.

Trying to write my own test, I found out that there is a Tellurium Chrome extension which works rather the same as Selenium IDE: you can record your browser activity and get a “pre-made” test. I have had my share of disappointments with Selenium IDE, so I did not expect much from a recording tool, but… it was way worse than I expected.

The extension would constantly crash and make the whole page which I was recording unresponsive. I would often blame the sites for being not te52-friendly and changed the sites for my test idea multiple times.  Screen Shot 2016-07-19 at 6.14.15 PM

After a lot of struggles with the extension and eventually inspecting web elements manually, I created a test for Urban Outfitters Quick View: open Quick View, assert that Add to Bag is visible and take a screenshot.

My final test “script” looked as follows:

For the url http://www.urbanoutfitters.com/uk/catalog/category.jsp?id=WOMENS-DAY-DRESSES-EU#/
When the mouse is moved over the first “Kimchi Blue Katrina Polka Dot Shirt Dress” image
When the first Quick Shop link is clicked
Once the “action-buttons ng-scope” div is present
Then the text “Add to Bag” is present
Then take a screenshot

As you can see, plain English definitely is very easy to understand and implementation details are hidden. Definitely, test could be even more obvious if my elements were chosen wiser wouldn’t be as specific (but I made them to be so to get it done!). The screenshot was successfully taken and saved displaying the Quick View:
screenshot-0001

Some final observations about the tool:

  • Idea is very good: you get to do quick checks online which are written in an easy to understand language and anyone could check them out.
  • Different browsers support is still very weak. To run tests online in the simple way you can do it only on Chrome, Firefox and IE11 (Tellurium grid). And, this is what I got trying to run it on Firefox:
    Screen Shot 2016-07-19 at 6.57.05 PM.png
  • Chrome extension is a pain to use: it crashes and freezes the whole page. You should be quite familiar with inspecting elements in order to write tests in te52 and cannot rely on the extension to help you.
  • Import from Selenium IDE works: I tried to import Selenium IDE recorded scenario and it worked fine. Did not dare to import real Selenium test.
  • Some web elements are not easy to capture. This is a constant struggle with web UI testing – it changes, it’s dynamic, implementations very much differ from site to site. However, some of elements which Selenium IDE grasped fine did not get recognized on te52.

In the end, I have had quite a few struggles with te52 and I think it still has room for improvement. This table demonstrates it well:

Screen Shot 2016-07-19 at 6.28.59 PM.png

I may use it in the future for scheduled automated smoke checks of UI as it allows to take screenshots easily, run tests on the regular basis and it all is for free. However, at this moment I don’t feel that I’m ready to bother with a lot of limitations that te52 still has.