QualityAssuranceQualityEngineeringTestAutomation
The University of New South Wales (UNSW)’s CRM was designed to enhance constituent relationship management and support staff efficiency in a dynamic and fast-paced digital environment. Faced with high maintenance requirements for automation scripts and constant changes in system functionality, UNSW needed a partner with expertise in quality engineering and automation to ensure the system was performing optimally.
Planit played a critical role in transforming UNSW’s testing processes, delivering a tailored solution that improved reliability, expanded test coverage, and ensured seamless sprint releases while maintaining the highest standards of quality.
Delivered 161 automation scenarios, achieving 94% test coverage.
Reduced regression testing time by over 85%, saving eight resource days per cycle.
Ensured seamless Wave releases with significantly shortened testing periods.
Strengthened system reliability, with no major production issues reported.
Testing UNSW’s CRM posed unique challenges in a dynamic and fast-paced environment. Operating on a three-week development and one-week release cycle, the UNSW team faced tight timelines and occasional last-minute testing demands. Ad-hoc story testing may occur during the final sprint week, requiring additional diligence to ensure issues are addressed effectively before release.
Further complicating the process were constant environmental changes driven by Microsoft’s weekly updates to core products and bi-annual Wave releases. Understandably, these updates required frequent adjustments to automation scripts to maintain their reliability. The automation framework required high-touch maintenance to adapt to the evolving application landscape, placing additional strain on resources and timelines.
The team’s existing approach to automation included running scripts just once a month, managed by a part-time resource partially allocated to the CRM. This limited capacity meant automation scripts often failed due to application changes, reducing their effectiveness and eroding confidence in the testing framework.
Without a solution to these challenges, the university risked disruptions to the operations of hundreds of staff users, compromising the reliability of critical systems and impacting its ability to deliver essential services. The need for efficient, reliable, and adaptive testing processes became paramount to maintaining operational excellence and delivering on the university’s objectives.
Given the complexities of testing the CRM system, UNSW needed a partner who could overcome these challenges efficiently. The goal was to enhance automation reliability, increase test coverage across high, medium, and low-priority scenarios, and integrate seamlessly within the university’s agile framework.
UNSW turned to Planit for its proven track record in quality engineering and automation, along with its ability to adapt to the unique demands of fast-paced, high-touch environments. Our existing relationship with UNSW provided the confidence that we could integrate seamlessly into their systems, bringing not only technical expertise but also a prior understanding of their unique environment and needs.
Our approach involved refining the automation framework to achieve 94% test coverage, ensuring that both critical and lower-priority scenarios were comprehensively tested in the test and rehearsal environments. This strategy was complemented by manual verification and smoke testing during production verification to provide an additional safety net. In addition, automation scripts that previously required 40 hours to execute manually were optimised for efficiency, enabling the team to mitigate risks associated with compressed sprint cycles and frequent updates.
By leveraging automated testing, UNSW could quickly identify how weekly cloud product patches or new functionality introduced during a release might impact the system before these changes affected end-users.
Our consultants seamlessly integrated with UNSW’s local project team to deliver a structured, reliable, and efficient testing strategy. By leveraging a resource augmentation model, we ensured consistent collaboration and a deep understanding of project-specific needs. This local delivery approach allowed our quality engineers to support production verification testing (PVT) for monthly releases, contributing to a robust testing process.
Automation scripts were executed twice weekly in both test and rehearsal environments, identifying changes introduced by developers and Microsoft’s frequent updates. To complement automation, manual smoke testing and PVT further ensured stability, particularly during release nights. The testing process utilised multiple tools, including Selenium (Java), Cucumber, Serenity, Jenkins, and Atlassian Jira, to streamline execution and reporting.
We adopted a phased testing approach to maximise efficiency and reduce risks:
A key component of this approach was the execution of 161 behaviour-driven development (BDD) scenarios, encompassing 672 automated test cases, twice weekly in test and rehearsal environments. Additionally, prior to each sprint release, 55 BDD scenarios were executed monthly in the test environment, covering 214 medium- and low-priority test cases. This rigorous testing cadence minimised risks and improved the reliability of outcomes.
Our partnership with UNSW brought significant improvements to their CRM, enhancing both efficiency and system reliability. By updating the automation framework and increasing test coverage to 94% across high, medium, and low-priority test cases, our team successfully streamlined the testing process. We developed 123 new end-to-end BDD scenarios while refactoring and maintaining existing scripts to meet sprint requirements. This resulted in a total of 672 automated regression test cases out of 715 automatable scenarios, providing comprehensive coverage.
The efficiency gains were transformative. Manual regression testing, which previously required 60 hours of effort, was reduced to just 5.5 hours per cycle through automation, saving approximately eight days of resources compared to manual execution. Weekly regression tests in the rehearsal environment identified bugs introduced by Microsoft updates and internal builds, ensuring functionality remained stable and disruptions were prevented. These results were meticulously tracked and reported, providing clarity and actionable insights to the UNSW team.
Automation also revolutionised testing for Microsoft’s bi-annual Wave releases. What previously took close to two weeks was condensed to just 3-4 days, thanks to the expanded automation coverage and improved script reliability. The automation suites were also executed weekly in the test environment, enabling the early detection of mid-sprint functionality changes and facilitating timely resolutions.
UNSW stakeholders expressed high praise for Planit’s collaborative approach, highlighting the consultants’ commitment, adaptability, and proactive problem-solving. The partnership ensured high-quality delivery, streamlined processes, and reliable outcomes, enabling the university to confidently support hundreds of staff users with a resilient and efficient system.
We use cookies to optimise our site and deliver the best experience. By continuing to use this site, you agree to our use of cookies. Please read our Cookie Policy for more information or to update your cookie settings.
