SHARE YOUR FAVORITE TESTING TOOL
I must admit that I love using analytics and monitoring in my testing. This is why my favorite tool is New Relic! It’s a software service which monitors web and mobile applications in real-time.
You may wonder why I call it a testing tool: I believe that monitoring and analytics are a huge part of regression testing. If numbers are still correct (or better), it can be a very valuable KPI to measure the system change. And that is not the only benefit in testing that New Relic can give.
I am going to list 3 reasons why I love using New Relic Insights (that’s the part that I tend to use the most):
- You can collect statistical significance and evidence of reported bugs.
In most of the cases, I can check real-time data and see how many users were affected by a certain issue. For example, if found issue happens for a specific set of items, I can check the percentage of all requests which were made on those items. The bug will definitely need to be fixed if the impact is high.
- You can find out your user demographics and devices/browsers they use.
It is very important to know who is your audience. If most of your users are based in India, it is likely that they use different devices than the users in the USA. This can help you to decide which devices and browsers you should test on and which location settings are important. For example, the following screenshot displays what are the most popular desktop and mobile browsers among the users of the product I test:
- By monitoring the system, you can find the issues easily.
You can create all kinds of monitoring dashboards. Our product is a service, so knowing that service is functioning correctly means a lot. If there is a drop in one of its functions – we have a problem. For this reason, I have created a dashboard for myself including all important KPIs to me. Every day I check to see if system is in tact, if not – I query for more specific examples and investigate what could be the reason. Nevertheless, after releases, I always doublecheck the dashboards for any possible regressions. It helps me to make sure that system continues working as expected real-time.
New Relic Insights has given me more confidence in bug reporting because I can add a real-life value and importance to them. Also, it helps me to do quick regression checks and investigations which I was not able to do this easily pre-New Relic.