CNTT Key Metrics 2020
- speedwyre
What are we doing?
Developing metrics for measuring quality, value, and realization of benefits of badging to RAx standards
Below Table: proposed Key Metrics for 2020.
Presented in Prague, CZ, January 13 - 16, 2020, and ongoing work in progress.
Designing and building data analytics tools as needed to gather and support data analysis
LF Developer Analytics Dashboard
Dashboard - In Beta Testing
Why are we doing it?
Establish platform to gather feedback on approach, metrics, and data.
Provide data driven insights to support CNTT mission and goals.
# | Area | Function | Segments | Targeted Benefits | Metric | Priority | Data Source(s) | Benchmark | 2020 Target | Future | Comment |
|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Governance | Engagement | All | Increase engagement of SP community members | Code/content contribution rates - by company | 1 | GitHub, Groups.io, Confluence | wip | wip | wip | |
2 | Governance | Engagement | All | Increase engagement of SP community members | Code/content contribution rates - by individual | 1 | GitHub, Groups.io, Confluence | wip | wip | wip | |
3 | Governance | Adoption | All | Improve Onboarding and Implementation | TBD: How to quantify TC metrics? (trial, cookbook, etc.)
| 1 | wip | wip | wip | wip | We need to develop benchmarks for pre-launch trials, e.g. is 50%, 70%, 85% pass rate 'good' for TCs pass/fail of expected results for RI? |
4 | Governance | Recruitment | Service Providers | Increase acquisition of community members | New member acquisition rate - by company | 2 | wip | wip | wip | wip | |
5 | Governance | Recruitment | Vendor Partners | Increase acquisition of community members | New member acquisition rate - by company | 2 | wip | wip | wip | wip | |
6 | Governance | Engagement | Labs | Increase engagement of community members | # of labs with CNTT verification capability | 2 | wip | wip | wip | wip | |
7 | Governance | Adoption | Service Providers | Increased adoption by SPs | rate of adoption of RAx - by company | 2 | wip | wip | wip | wip | |
8 | Governance | Adoption | Service Providers | Increased adoption by SPs | # of RFPs published required CNTT badged VNF | 2 | wip | wip | wip | wip | |
9 | Technology | Conformance | Service Providers | Reduce costs | See TCO Analysis | 2 | wip | wip | wip | wip | |
10 | Governance | Adoption | Vendor Partners | Increase adoption by VPs | rate of badges issued - by company | 3 | wip | wip | wip | wip | |
11 | Technology | Conformance | Service Providers | Accelerate time to deployment for new network services | # of weeks to deploy VNF product into lab (or deployment) | 3 | wip | wip | wip | wip | |
12 | Technology | Conformance | Service Providers | Improve interoperability and software quality | (TBD: criteria: technical, evolutionary, configuration, security, operations, performance) | 3 | wip | wip | wip | wip | |
13 | Technology | Conformance | Service Providers | Reduce in-house testing effort | # of weeks to test VNF product in lab to deploy (time to productivity) | 3 | wip | wip | wip | wip | |
14 | Technology | Conformance | Vendors | Improve time to revenue for new product offerings | # of weeks to develop new VNF | 3 | wip | wip | wip | wip | |
15 | Technology | Conformance | Vendors | Achieve greater alignment with service provider customer requirements | % of VNFs badged - by company | 3 | wip | wip | wip | wip | |
16 | Technology | Conformance | Vendors | Demonstrate product quality through open ecosystem testing | # of badges issued - by company | 3 | wip | wip | wip | wip |