D2M Blog
Building an AI Center of Excellence: Defining Success
Defining Success
D2M has advised several organizations on the creation of Centers of Excellence (CoE) focused on Machine Learning (ML) and automation. A critical first step for any organization undertaking a new project is defining the measures of success. Creating a CoE is no different.
Determining the business objectives for the formation of a CoE and developing quantifiable key performance indicators (KPIs) to measure the achievement of those objectives will support the case for creating a CoE and guide stakeholders through the process.
At D2M, we define goals using a method similarly to the Agile Methodology technique that creates user stories. These stories define the purpose of a CoE and determine the acceptance criteria for success
We have provided several examples below. While all examples may not be relevant to your organization, they provide a starting point for defining and measuring what a successful CoE will look like.
Example#1
As a data consumer, I want better data consistency so that I can more accurately compare today’s results with past ones.
Acceptance criteria:
- CoE supports and automates data pipelines to ensure consistent data treatment
- CoE provides documentation of data pipelines guaranteeing consistent data feeds
Example#2
As a business unit owner, I want to find cognitive automation opportunities, so that I can react to customer needs, faster.
- CoE supports use case discovery for business lines
- CoE provides a ranked list of automation opportunities with preliminary ROI estimates
- CoE maintains a list of common use cases with known ROI
Example#3
As a business unit owner, I want to ensure that my unit is always innovating
- CoE provides support for review of new and existing processes
- CoE provides suggestions for cognitive automation
- CoE produces frequent communications about the advances of ML applicable to business lines, both within the organization and outside
Example#4
As a technical manager, I want to reduce reliance on external skills
- CoE provides training for engineers who want to expand into ML
- CoE provides in-house resources for in-demand jobs, such as data scientist, and data engineer
Example#5
As a business unit owner, I want to ensure that my unit is meeting all data and regulatory guidance for machine learning
- CoE guides explainable AI in a legal setting
- CoE provides traceability guidance and documentation for pipeline and model deployment
Example#6
As a project owner, I want to ensure that my data project meets organizational standards for maintainability
- CoE provides organizational standards for
- Scalability
- Deployment
- Model acceptance criteria
Overall, a CoE requires measurable goals as a prerequisite to its establishment.
How do you measure success in your CoE? We’d love to hear from you.