Articles

AIArtificialIntelligenceQualityEngineering

Revolutionising testing with Agentic AI

Our experts explore agentic AI, the next evolution beyond Generative AI, capable of autonomously executing tasks and making decisions—bridging the gap between hype and reality.

Artificial Intelligence (AI) has evolved significantly over the past 80 years, but the advent of Generative AI (GenAI) accelerated its adoption at an unprecedented pace. Initially celebrated for its seemingly “remarkable” capabilities, GenAI generated enormous hype when it hit the market. Now, the next wave of AI is creating a lot of interest. Termed agentic AI, this new wave is boosted by large language model (LLM) technology, which is promising to produce the next level of productivity. If GenAI can be described as reactive where a user needs to prompt it for the information needed, agentic AI is proactive and can act autonomously to achieve goals without the need for human guidance once it understands the goal and the problem it is meant to solve. In short, it does not just tell you what to do but it does it for you.

Understanding agentic AI: Bridging hype and reality

Agentic AI can be described as both hype and reality, offering a transformative potential that integrates intelligence directly into workflows. While definitions may vary, agentic AI can best be described as autonomous or semi-autonomous software agents that use various AI techniques to achieve goals. In other words, agentic AI systems can autonomously plan and take actions to meet goals and objectives predefined by users. An AI agent has agency to make decisions and then, take action.

Agentic AI can take various forms and can include standalone AI agents and more complex multiagent systems, each capable of executing tasks and making decisions independently.

An AI agent can gather data which it then processes and interprets to generate insights, followed by a feedback loop to reassess through learning and thereby redefine future responses (see Figure 1).

Figure 1. Model of an AI agent

Microsoft’s Copilot Vison is an example of an AI agent that can take in the full context of what a user is doing online and can support the user by also scanning, analysing and then offering insights based on what is sees.

In multiagent AI architecture, agents can not only process within, but also have mechanisms to communicate with each other and this is where even higher value occurs.

Due to the extreme hype of this new technology, some entities will use the terminology although it’s not strictly correct and some “AI Agent washing” might occur.

Key strengths of agentic AI include:

  • Can operate automatically and independently without direct human intervention.
  • Programmed to achieve specific goals and objectives, its “job description.”
  • Continuously learn from interactions and new data to adapt and improve performance and refine strategies. It does this by identifying patterns as trends with machine learning (ML).
  • Contextual understanding of what considerations are relevant to include, mimicking human reasoning.
  • Ability to access information and applications across domains, platforms and vision or voice.

Transformative impact of AI agents across industries and domains

AI agents have the potential to transform workflows across industries, tackling everything from routine tasks to extremely complex use cases. AI agents can be used across workflows and although the technology is immature and in early stages, plenty of use cases and potential use cases already exists. The expected increasing use and reliance on AI agents is reflected in a Gartner predict that by 2028, at least 15% of day-to-day work decisions will be made autonomously through agentic AI, up from zero percent in 2024.

Multiple industries, including, pharmaceuticals, healthcare, financial services, automotive and high-tech have started adopting the technology. The following is an example to illustrate how a financial service company implemented multiagent systems by designing multiple but interactive agents able to communicate and reason with each other to analyse potential investment opportunities. Each agent was provided with different job tasks and objectives and the roles prescribed to the agents ranged from focusing on the P&L statement of a company to geopolitical risks where that company operated. Each agent had a distinct role in how they analysed the individual companies as investment opportunities. By weighting these results against each other, decisions could be made on whether customers should be advised to invest.

To quote Jeff Bezos “Just as electricity transformed almost everything 100 years ago, today I have no doubt that AI is going to have such an impact.” Interpreting this quote as electricity is not valuable in itself, but for what it can be used for in our society. AI is also not valuable in itself, but applying it for improvement, or even better, revolutionising how things can be improved with AI, is valuable. According to the declaration at the CES 2025 by the CEO of Nvidia, Jensen Huang; AI agents represent a multi-trillion dollar opportunity. Whether this prediction is true or not, it is a technology worth following closely.

Multiple use cases for agentic AI in testing and quality

AI is already enhancing the capabilities of developers and quality engineers throughout the Software Development Lifecycle (SDLC), and agentic AI promises to further elevate these processes by automating and optimizing various tasks.

People will guide agents to autonomously perform tasks for them while they concentrate on other items. An example is IBM developing a set of AI agents, powered by open LLMs for automatically resolving GitHub issues. These are primarily intended for developers but one of the AI agents will be able to develop and execute tests. UI path has also introduced Autopilot for testers which consists of agentic test design, agentic test automation and agentic test management. For example, agentic test design is programmed to evaluate requirements and generate manual test cases, agentic test automation to convert manual tests into automated and generate synthetic test data, and agentic test management provides insights into test results.

Quality engineers are expected to be able to work with AI agents in a collaborative format where tasks are assigned to available agents as needed, automating individual tasks with more accuracy and autonomy. (see Figure 2).

Figure 2. Examples of AI agents in Quality Engineering

Potential use cases for agentic AI in quality and testing include:

  • Automatically create and update documentation.
  • Test data generation per test requirement while learning patterns from existing datasets.
  • Manage deployment pipelines to enable rollbacks in case of failures and monitor application performance.
  • Identify flaky tests in CI/CD processes and suggest improvements.
  • Automate the running of certain tests and provide reporting on issues found.
  • Gather and share relevant information, document, and best practices from a centralised repository.
  • Analyse user feedback from various channels to identify common issues and by continuously monitoring user interactions, improvements can be suggested.
  • Finding defects by identifying patterns in past testing runs, predicting where issues are likely to occur and allow for focus on high-risk areas.
  • For testing of advanced AI features such as positive/negative intentions, true/false statements, and presence of sensitive information.

In the future, the potential applications of AI agents could expand to seamlessly linking individual agents responsible for different tasks, enabling the automation of entire workflows. For example, multiple agents could work collaboratively to create fully autonomous processes. Agent 1 might identify a defect, fix it, and commit the code. Agent 2 could then review the code, deploy it to the test environment, select the appropriate regression tests (and even write new tests if necessary), execute the tests, and manage the results. Based on the outcomes, the workflow could either proceed to address the next defect or iterate further until the current issue is resolved.

Focus on value when considering AI agents

Looking for use cases where AI agents can add value needs to be a priority for business leaders today. The expectation is that agentic AI will accelerate the potential use cases where AI can have an impact. As with other modern technologies, the best place to start is most often with the use case and what the best solution for improvement would be as the answer could be a range of solutions from functional automation to GenAI, or agentic AI. Certainly, the prediction is that agentic AI will have a large impact on how work will be done in the next few years although it is still early days. Agentic AI can provide business value by providing benefits including:

  • Enabling quality engineers to do more with less time adding more efficiency and speed and enabling faster time to market.
  • Freeing up quality professional’s time to perform higher value-add tasks.
  • Cost reduction and improved profitability.
  • Broader test coverage and improved reliability with ability to analyse massive datasets and achieve higher test coverage.
  • Support decision making with predictive analytics and providing insights into the QE process.

It is critical to track the impact of agentic AI through relevant metrics and KPIs to ensure the value expected is achieved.

Empowering quality engineers with AI agents will transform work

AI agents can be described as digital support staff, provided with a job description that they should perform. AI agents should be viewed as helping to complete tasks better and faster. AI agents will allow quality engineers to focus on higher value tasks, empowering them, and enhancing the work of a human in specific areas or tasks. Meanwhile, the AI agent will take care of other repetitive or mundane tasks, allowing for sequencing of the quality and testing tasks. It is almost inevitable that AI agents will have an impact on people’s jobs, but they are not likely to completely replace humans across quality engineering as that would be incredibly complex and an enormous effort to set up, if even possible.

In the Future of Jobs Report 2025, published by the World Economic Forum (WEF), it is forecasted that 41% of companies plan to reduce their workforce in the next five years as many tasks are automated with AI. However, at the same time, 70% of companies expect to hire people with AI skills. The WEF also forecasts that skills in AI and big data will be the fastest growth in demand by 2030. This describes how the overall job market will go through a transition due to AI technologies and with the right skills; opportunities abound through this shift in the market.

There is a risk to people if they do not keep up with what AI can do and harness this in their job role as it’s not people that will be replaced by AI agents but some people who do not use AI might be replaced by people using AI and AI agents.

Navigating the risks of agentic AI with a balanced approach

Like any emerging technology, agentic AI’s journey will be non-linear, marked by unforeseen challenges and initiatives that may fall short of expectations. It is crucial to be aware of these risks, applying a critical perspective and proactively address these risks while harnessing AI’s potential.

Some of these challenges include:

  • With more autonomy comes more security risks and more potential issues and failures which can be exacerbated with the level of complexity involved. The complex environment of multi-agent systems expands the attack surface of agentic AI. Engineers managing the systems must have specialised skills to discover and correct these problems.
  • As with all forms of AI, the data used to train these models must not be flawed or incomplete, as that can produce erroneous results.
  • Transparency can be an issue as an AI agent mostly function as a “black box,” which makes it challenging to understand how it arrives at specific conclusions.
  • LLMs are a frequent part of AI agents and therefore any issues associated with these can also apply to AI agents such as built in bias, hallucinations, ethics and the availability and use of quality data.
  • People have vastly different views on the use of AI and its use which will affect them in different manners from positive to negative. Some employees will be fiercely against it, viewing it as a threat to their role or even society, some will use it confidently, whilst others might trust AI too much.
  • Governance can be a concern as this is a technology that operates autonomously, and strict guardrails and accompanying skills will be required.

Conclusion

As we move into the future, AI will likely be a key part of business and life. Understanding and embracing agentic AI strategically, is not just an option but a necessity for businesses aiming to stay competitive. AI and AI agents should be a critical consideration in quality engineering for achieving optimal results. AI agents will revolutionise how software quality engineering is performed and although we are at early stages of the use of the technology, it is already transforming work processes. However, it is important to consider how AI agents can simplify and improve processes without adding unnecessary complexity.

There is promise of breakthroughs expected in 2025 onwards and it is important to understand both potential and challenges to be better prepared for the future in quality engineering.

Finally, the use of AI agents is not possible without people and human intelligence. However, real value can be achieved when people have access to agentic AI.

AUTHOR:

Susanne Matson

Practice Director - Customer Insight & Advisory

Accelerate Your AI Journey

AI is transformational, yet only 33% of leaders are confident their enterprise will mitigate the risks of AI. How ready are you?

No matter where you are on your AI journey Planit has the expertise and solutions to accelerate you towards Quality AI.

Get Updates

Get the latest articles, reports, and job alerts.

Security Center

We use cookies to optimise our site and deliver the best experience. By continuing to use this site, you agree to our use of cookies. Please read our Cookie Policy  for more information or to update your cookie settings.