Test Center of Excellence: Why Your Company Would Benefit and How to Set It Up

As companies shift into new ways of developing software, testing as a centralized service is becoming more common. As such, organizations are looking for ways to successfully deploy testers across multiple teams, without giving up the standardization and best practices some QA organizations have worked hard to create and maintain. A testing center of excellence can be a perfect way to maintain standardization across your teams and ensure your organization prioritizes testing innovation.

What is a TCoE

A testing center of excellence (TCoE) is a framework that defines, implements, and measures testing controls and standards across an organization. In this framework, testers themselves are shared resources across teams, but testing protocols, toolsets, and KPIs are maintained at a centralized level. This allows organizations to quickly deploy any tester to any team while continuously maintaining QA principles and processes.

When Is a TCoE Useful

A TCoE can be advantageous for companies that have complex organizational structures that sometimes result in testers spanning multiple teams where project goals may not align. However, there are several other situations when a TCoE may be useful for an organization. If any of these apply, a TCoE could be an ideal solution:

You have a complicated organizational structure: If your testers do not all report into the same manager or do not share common goals, it can be challenging or impossible to normalize processes and tooling across an organization.
You have a desire to identify common testing KPIs and track trends: Ensuring quality across multiple teams can be challenging, especially if you don’t have one person or a group whose primary focus is on it. You could see variations in how teams track certain KPIs while others track none at all. A TCoE can define common metrics and measure quality throughout your organization, reducing or even eliminating the challenge all together. For more information on WHY to measure, WHAT to measure and HOW to measure, check out my recent blog post on the topic.
Defects are an issue: By standardizing processes, tooling and KPIs, a TCoE can lead to fewer defects throughout your SDLC.
You want to homogenize processes and tooling across teams: A TCoE’s main function is to standardize processes and tools across teams. This normalization results in less time spent defining and implementing multiple variations unnecessarily. In addition, it encourages cross-team communication around best practices and guidelines related to test case writing, automation scripting, and execution.
You feel pressure to reduce time to production: The QA cycle of writing test cases, scripting and executing takes a considerable percentage of the overall software development lifecycle (SDLC). Having a TCoE in place cuts out the repetitive processes across teams, allowing them to focus solely on testing tasks that matter.
Your organization is challenged by not hiring and onboarding strong testing resources: A TCoE can establish reliable recruiting, hiring and onboarding protocols. This leads to strong testers across your organization, who are all onboarded with consistency.
You want to encourage persistent innovation: A tester’s day is filled with writing test cases or scripting, executing tests, and reporting defects. There is typically very little time for innovating and advancing the way they work. Having a TCoE ensures that someone in your organization is focused on this critical component.
Shifting projects and priorities leaves your testers shifting teams or deliverables often: In an agile environment, customer feedback loops sometimes lead to frequently shifting priorities. Having the ability to shift resources and maintain quality is key to being successful.

How to set up TCoE

Once an organization agrees to the framework of a TCoE, the hard work comes in the form of successfully implementing it. A successful implementation considers these steps:

Define the challenges you need your TCoE to solve or account for. At a minimum, your TCoE should standardize tools and processes. Additionally, you may customize your TCoE to include discovering and implementing new technologies, defining and measuring KPIs, or even hiring and onboarding new QA resources.
Identify who will govern your TCoE. This should be a dedicated team of individuals who properly represent your testing teams as a whole. Some organizations decide to partner with a vendor for this implementation while others keep it fully in house.
Outline your TCoE roadmap. Every organization is different in their needs and desired outcomes. Identify what areas are the most important and prioritize those accordingly.
Define how this group will interact with other teams. This requires leadership buy-in across your organization. Things to consider include how the TCoE will roll out new processes or tools and ensure proper adherence, and what level of guidance they can give to teams if the protocols aren’t followed. Defining this upfront will limit future missteps between your TCoE and teams.
Document your current tools, KPIs, processes, and methodologies. Prior to and during implementation, there will already be an agreed-upon set of processes or tools. Ensuring expectations are properly documented and an ongoing document repository is in place is important for future reference or onboarding.
Engage your teams to understand starting deficits. Perhaps you have testers that aren’t adhering to previously defined processes, or maybe they are using unapproved tools. Engaging each team to validate you understand their needs, as well as any gaps, is essential to building a strong starting foundation.
Communicate across your organization: By this point in your implementation, most people should be aware of the TCoE and what it means, however, don’t take that knowledge for granted. Make sure you communicate the existence of the TCoE, the purpose, and its goals to everyone in your organization.

Resources/Cost Involved

Depending on how your company approaches the implementation, your resources and costs may vary. For example, if you decide to partner with a third-party vendor to start-up and/or maintain the TCoE, the internal resources dedicated to this may be minimal, however, your partnership may result in higher costs. On the contrary, if you’re considering implementing this framework in-house, the following resources and costs should be considered:

Resources: A TCoE should be comprised of individuals who are fully dedicated to this initiative. When considering who should be included, contemplate recruiting testing managers, testing leads, and ensure someone from each testing competency is involved (automation, manual, performance, security, etc…)
Cost: The cost associated with starting up an internal TCoE includes resources that will be dedicated to the implementation of the TCoE and those that will formally sit within that group moving forward. In addition, there may be costs to consider when standardizing testing tools or purchasing a document repository solution.

TCoE Pros

When analyzing whether to implement a TCoE, you must fully consider the pros and cons of such. Here are some benefits of implementing a TCoE:

Enhanced core skill sets of all testers: by implementing a TCoE, you’re investing in your testers’ overall skills through training and innovation, resulting in higher quality products for your customers.
Standardization of automation frameworks and reduction of complexity: By having a defined automation framework you’re ensuring all teams are following basic coding standards. This leads to shorter scripting cycles and execution times, time reduction when onboarding new automation engineers, and improved testing quality and coverage.
Increased agility: Enforcing every tester to work within set guardrails allows priorities to quickly shift without testers having to learn varying processes or tools across teams. In addition, scaling up teams using an outsourcing model allows individuals to be quickly and consistently onboarded.
Continuous Improvement: A main component of having a well-rounded TCoE is the ongoing modernization of tools and processes. Having a dedicated team whose goal incorporates this, makes certain your organization is always operating in a modern testing world.
Cost Savings: Standardizing tools across teams could result in considerable cost savings for an organization over time.
Decrease testing costs: HCL published a case study detailing a TCoE implementation that led to an 11% decrease in testing costs for the organization. The full case study can be found here: https://www.hcltech.com/success-story/systems-integration/centralization-testing-services-enables-american-leading-casino-resort

TCoE Cons

A TCoE may not be the right path for your organization. Here are some cons to consider before deciding to make the leap:

A TCoE may overcomplicate things: If you have one or two teams with static testers, odds are their processes and tools are fairly aligned. Or maybe you have high functioning teams that would find standard ways of working an impediment to being successful. Either way, adding in an additional layer may add unnecessary complexity, resulting in delayed releases and frustration.
Insufficient support could lead to burnout and failure: Deciding to implement a TCoE without backing from all levels of your organization could lead TCoE members feeling discouraged and burned out if their process and tooling recommendations aren’t supported or adopted properly.

TCoE Pitfalls

With every new venture, there are pitfalls to avoid. Here are some pitfalls to consider when implementing a TCoE:

Not aligning TCoE goals to organizational outcomes: By definition, a TCoE is a centralized team of people who share the common goal of encouraging quality across the organization. Other teams will be subject to adhering to the outputs of the TCoE. It’s only logical the goals of the TCoE align with your organization’s goals.
Not defining how much authority the TCoE has: You will inevitably have a tester or team who fails to follow processes or use tools outlined by the TCoE. Failing to provide the TCoE with the ability to enforce guidelines will be counterproductive and lead to low adoption rates over time.
Failing to create feedback loops for communication, both ways: Having a group of individuals defining process or implementing new tools, without buy-in or direction from other teams in the organization, will drive an unsuccessful implementation. It is important that all testers are engaged and help form and drive decisions, not just in the beginning, but over time as well.
Creating a TCoE with bad collaborators and communicators: It’s not enough for this group to be comprised of people who understand testing principles in depth, it is also a must that they value communication and collaboration.
Trying to move too quickly during the implementation phase: Identifying, planning, and implementing a TCoE takes time. Ensuring you’ve gone through the steps above, and taking the time needed to plan up front, will pay off in the end.

TCoE KPIs

Identifying a solid set of KPIs upfront will help you understand if your implementation of the TCoE is adding value to your organization. As you continue to rollout new process, or tweak existing ones, KPIs will provide a good success measurement.

In my experience, identifying what KPIs you should measure is challenging and unique to every organization. While selecting your set of KPIs, consider team sizes and distribution, company culture, and current gaps or challenges your trying to fix.

Recommendations

As with any major organizational shift, analyzing your current state and understanding your gaps is the key to determining if a TCoE is right for you. When deciding to move forward, invest the time upfront to ensure you outline specifically what your TCoE is and isn’t and make certain you select the right people for the job. Enlisting testers who exhibit good collaboration and communication skills, in addition to a solid understanding of testing principles, is important in a successful implementation.

In addition, make certain you identify and communicate how you will measure success. If you’re using a set of KPIs, communicate what those are so teams understand what their success measurement is. In my experience, trying to measure too many things in the beginning becomes daunting and you may lose sight of the overall big picture.

Summary

A TCoE gives organizations the ability to implement standard testing principles and tooling across any number of teams while ensuring quality remains a priority. In addition, a TCoE helps define and measure KPIs, ensuring a consistent quality product to the customer. While this article references an agile organization, a TCoE can be enlisted within any organization, agile or not.

If implemented fittingly, a TCoE can help an organization scale testing without compromising quality. Analyzing where your organizational challenges lie today, and how you see those blocking your ability to scale and shift priorities in the future, will give you a good starting point in determining whether a TCoE is an appropriate solution for your organization. After concluding to move forward, arrange the time upfront to implement it successfully. Ensuring testers with good communication skills, a solid understanding of testing principles, and a desire to help the organization grow, are all attributes when looking for TCoE leaders. Make certain you fully define the success criteria for your TCoE, engage all levels of your organization, and convey the purpose and desired outcome appropriately. A solid built TCoE can bring many positive benefits to your organization when implemented thoughtfully.