The contract we have with one of our current clients includes a weekly review of their website. It’s a fairly simple brochure websi
te so there are not a lot of pages or complex functionality. Their weekly “subscription” includes reviewing social media posts for spelling and grammar, testing all the links on the site, and performing various manual testing tasks including ad-hoc testing.
Is this a good investment by our client? Let’s consider the pros and cons.
Pro:
- Easy to budget. The cost variability of irregular testing periods is replaced by the predictability of a consistent and affordable monthly expense.
- Regular review. The site receives a professional review on a regular basis. There is a high level of confidence that any bugs introduced by any changes will not go undiscovered for a long period of time.
- Turn-key process. No need to ramp up and manage a test effort. No need to hire anyone for a test effort. No need to try to keep QA personnel that you just hired busy when the immediate project is over.
- Site familiarity. The testing partner becomes very familiar with the site. This is helpful in detecting anomalies and when testing the site after major changes. But this pro can also be a con…
Con:
- Site familiarity. Testing the same site again and again can lead to fatigue and make it more likely that a bug gets overlooked. This can be mitigated by rotating the testing team.
- Ongoing cost. Though the monthly cost is nominal, there is a line item for testing every It is even possible for the total annual expenditure for subscription testing to be greater than the cost of a more traditional approach.
- Manual effort. Much of the cost for the subscription testing is directed toward manual testing. Though automated scans of the site using off-the-shelf tools are performed each week, part of the test effort involves manual testing. For many applications and sites that have more complexity and breadth to them, such reliance on manual testing does not provide adequate coverage and is not as efficient (and ultimately as cost effective) as automation.
So when does this make sense? For this client, who is in the business of doing something that is NOT software development and whose website is relatively small, it is a good fit. They value the peace of mind that comes from regular review, and one day of testing per week is sufficient to provide adequate test coverage.
As the site under test gets larger with more complex functionality, however, this arrangement starts to burst at the seams. The difficulty of adequately covering such a site with just one (or even two) days of testing per week increases exponentially. The pressure of organizing the testing between multiple days and testers, handling blockers, regressing issues, accommodating delays and absences, communicating questions and details between devs and testers, etc, would add more stress than the peace of mind provided by regular testing would relieve. In this case, a more traditional approach such as an automated regression suite built into a CI/CD process would be warranted.
Conclusion: The ongoing, “subscription” testing described here isn’t for every client or for every project, but it has proven to work well for this particular client. As with anything else, software testing is not a one-size-fits-all product and companies should not be shy about advocating for what works best for their particular product and situation.
AI content: 0%