Table of Contents

Overview

The intention of this page is to outline what CNTT "success" looks like by mid-April @ ONES-NA 2020, along with high level tracking items that need to be completed.

Success is defined as:

TBD - documentation created ? test suites defined ? test suites executed ?
TBD
TBD

To-Do (In General)

This is a comprehensive list of work that can be done. A subset of the list below will go into Baldy (and will be saved in the release notes for Baldy).

A: To be included in Baldy Release.

B: Nice to include in Baldy Release.

C: Low Priority for Baldy Release.

Reference Model

Mark Shostak to add general to-do list in here.

#	Deliverable	Comments
0	A: ms RM cleanup (ch content complete 80%, issue closure)
1	B: 1.x Various: May need to be rationalized w/ new Tech Steering document A: 1.9 Roadmap: This section needs perpetual maintenance (strongly consider moving the roadmap out of the RM, altogether) A: RA Compliance/traceability methodology A: Still needs a crisp executive summary identifying the problem statement, and the two key methodologies employed to address them (VNF Abstraction and a common NFVI)	RM-Ch01 - Intro A: RA Compliance/traceability methodology Needs to be Chapter 8 content (move to)
2	B: Needs a community aligned list of workloads the platform is intended to support and their priority, which can then be used as a basis for resource allocation and for weighting “suggestions” (notably “The public cloud does it this way, so we should too.”; understood, but we need an objective basis. Right now, it becomes a subjective debate) A: Align and execute on Prague proposal to park the Compute Intensive flavor, and supplement it by removing "over-subscription?" from the Basic flavour C: Flesh out quantitative details about supported workloads. (nice to have)	RM-Ch02 - VNF Requirements & Analysis
3	A: Needs tighter coupling w/ Container paradigm (i.e. decoupling from VM-based VNF) flavours for example might not be applicable to containers(CPU and memory, disc we want to find a way for containers) - possibly Ch04 harmonisation of template in Model. C: Needs a general review. Given the experience we’ve gained, can be made more succinct and more usable by a VNF developer C: Need to clearly explain slicing (and contrast to Tenant) from CNTT perspective B: Template attributes should be vetted w/ underlying capabilities, rationalized for duplication (i.e. CNTT value add, vs. what’s already in HEAT) and expanded to integrate Container paradigms B: Data Model - Create actual schema(s) and data model(s) for CNTT-specific information (potentially in an appendix) A: ms Shared w/ RM-Ch4: Need to define virtual networking strategy and related attributes/parameters (workload level)	RM-Ch03 - Modelling Pankaj.Goyal Flavours/Flavors are not mentioned in RM Ch03. The term "virtual resource" is used and that includes resources that result from both hardware (server) and Operating System virtualisation. Typically, VNFs and CNFs are mentioned together; there is an exception that should be corrected.
4	A: Integrate more Container paradigms/support C: ms Flesh out Generic Fabric Model (Fabric, Underlay and Overlay level (i.e. Infra, not workload) C: Research and enhance storage performance extensions using performance per unit of storage model (i.e. IOPs/GB) A) Hardware Profiles and Performance. Objectives/ Guidelines What level of granularity. Validation/Auditing. Proposals or white paper on how we plan to achieve deterministic performance from the reference model w/o having to specify hardware to down to a very granular level (e.g. w/o SKUs, clock speeds, etc.) A: Clarify diagrams and language around T-Shirts, Flavors and IT relationships A: ms Policies for Non-Conforming Technologies // assumes supporting frameworks are ratified SRIOV policy SmartNIC policy* Live Migration policy *includes Tomas’ concept of driving SmartNIC industry to standard, cloud-friendly ABI A: Shared w/ RM-Ch3: Need to define virtual networking strategy and related attributes/parameters (workload level). 4.2.2: need to consider aggregate network bandwidth instead of per Interface bandwidth. TBD: VNF Evolution / VNF Profile Generations TBD: Simplify VNF Profile Naming	RM-Ch04 - Infrastructure Capabilities, Measurements and Catalogue Pankaj.Goyal suggest that the White Paper be not priority A. Generic Fabric Model → Focus on terminology for Baldy.
5	A: Integrate cloud native concepts w/ sw profile (i.e. how abstraction and workload portability is maintained) A: Update s/w profile, pending parking of Compute Intensive IT Rationalize s/w profile to IT mapping after removal of third IT Audit chapter alignment w/ RM-Ch4 A: Incorporate VNF Profile Generations/Evolution (once ratified), designing and describing linkage/relationship to h/w profiles	RM-Ch05 - Feature set and Requirements from Infrastructure
6	B: Flesh out initial Enabler Services	RM-Ch06 - External Interfaces (APIs and Interfaces)
7	A: ms Worthy of review by SMEs for suggestions of areas to incorporate and/or refine	RM-Ch07 - Security
8	Cleanup to Ch08 A: Generic qualification model (i.e. NFVI and VNF) Badging types (requirements in RC) Methodology to support N-3 VNF Rls Profiles A: Overview of badging methodology Base-lining testbed before VNF qualification VNF testing without prior knowledge Plumbing testbed for supplier testing of VNF functionality, after VNF badging. A: ms Overview of novel VNF-related concepts Reference VNF (aka Golden VNF) NFVI characterization, and correction coefficients for normalizing performance A: What is out of scope for CNTT (e.g., VNF functional testing, performance, scalability, HA, etc.)	RM-Ch08 - Compliance, Verification, and Certification Should the RM content be VNF/CNF agnostic?
9	C: Create Generic Installer Model	RM-Ch09 - Infrastructure Operations and Lifecycle Management
A	C: Needs a strategy/purpose review and new commitment from original or new contributors to continue on it.	RM-Appendix-A - VNF Guidelines

Reference Architecture 1

Pankaj.Goyal to add general to-do list in here.

#	Deliverable	Target Date ( // )	Comments
1	A: RA-1 cleanup (ch content complete 80%, issue closure)	15 Mar 2020
2	A: Update OpenStack version based on newer version. (let continue discussion TSC call)		Overall
3	A: Update RA-1 Ch01: Introduction content		Develop after most Chapters greater than 60% complete
	A: Remove 1.6 as this will be moved into Overall Roadmap. A: Remove CI (as this has been decided to be parked for a bit and picked up later).		Ch01:
4	A: Update RA-1 Ch02: Requirements Align RM and RA-1 Requirements.		Ch02: A document provided to RM WSLs with a request for clean-up RM reqts (2020-01-27)
	A: Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation	7 Feb 2020	Ch02: (2020-02-07) Completed 2.3.1 - 2.3.5 (Scope of Baldy release)
5	A: Finalise RA-1 Ch03: NFVI + VIM Architecture		Ch03:
	Topology: Cover DC/Edge and SLA driven.		Ch03:
	Review Networking Section and suggest Improvements		Ch03:
	Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation		Ch03:
	Remove 3.6 as Traceability is in Ch02.
6	A: Finalise Ch04: NFVI + VIM Component Level Architecture		Ch04:
	Add Hardware acceleration		Ch04:
	Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation	7 Feb 2020	Ch04: PR #1046 Completed (waiting to be merged)
	Remove 4.7 as Traceability is in Ch02		Ch04: completed as part of PR #1065 (under review)
7	A: Finalise RA-1 Ch05: APIs and Interfaces		Ch05: completed
	Qualify APIs that are actually utilised.		Ch05:
	Add explanation on micro-versions	10 Feb 2020	Ch05: PR#1006 completed
	Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation	7 Feb 2020	Ch05: PR #1046 Completed (waiting to be merged)
8	B: Create content for RA-1 Ch06: Security: Create security requirements list		Ch06: Created RM Ch07 Consolidated Security Requirements++: currently under review
	Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation		Ch06:
9	B: Create updated ToC and Content for RA-1 Ch07: Operations and Life Cycle Management		Ch07:
	Develop content including Logging and Monitoring		Ch07:
	Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation
10	B: Update RA-1 Ch08: Gaps, Innovation, and Development		Ch08:
	Adopt Open Source (Tungsten Fabric) SDN APIs, map back to Requirements and push for adoption by OSTK as a Neutron extension		Ch08:
	Discovery: review ONAP A&AI. Support for specific use cases: packet acceleration (DPDK, SR-IOV, …)		Ch08:
	Capture Prague Etherpad Items

Reference Implementation 1

Qiao Fu to add general to-do list in here.

#	Deliverable	Comments
1	A: RI-1 cleanup (ch content complete 80%, issue closure)
2	C: Resolve Continuous Software Deployment Errors	(MikeF add) - Not clear what that is. → MikeF: Automatic deployment of OpenStack via the CICD Pipeline using AirShip fails. Manual intervention is needed to resolve issues with erroneous port 80 references, incorrect NOVA versions, and end-point URLs which do not work. (to name a few).
3	C: Document & Perform Repeatable CICD Deployment	(MikeF add) - some of it need to be documented in cookbook and some of it needs to be taken care of by OPNFV.
4	A: Document how to Perform Repeatable Compliance Validations (post deployment)	(MikeF add) - some of it need to be documented in cookbook and some of it needs to be taken care of in OPNFV.
5	C: Close Gaps in Cookbook [RC1 Ch9 - Snezka F2F] RC Cookbook Enhancements #945 [RI1 Ch07] User Manual initial content #408 User-Execution Common Failure codes, description, and resolutions Test Framework Clean-Up Utility, Process, and Support Expectations Process / Version Mgmt Process for version control Process for bug fixes and patches Containers Tagged Playbooks grab the tagged version	(MikeF add)
6	C: Conduct Friendly Cookbook Trial	(MikeF add) - That is not for RI team to worry about. → MikeF: How/who will own this item as there needs to be a friendly trial of the cookbook with feedback/improvement? This is about lab procurement, h/w and s/w validations, software installs, etc. All pertaining to RI WS. Rabi: Michael Fix i will send you an invite to the weekly adoption meeting where we are getting those vendors for the trials. I think it should be handled in there and I think you should get involved as per your role in RI & OPNFV
7	A: Select Vendor Candidates for RI installs	(MikeF add) - That is not for RI team to worry about. → MikeF: Can RI WS handle the handoff to the appropriate WS? The ask is for parallel/multiple RI installs. Who will identify & support these candidates?
8	A: Identify Target Labs for RI installs	(MikeF add) - That is not for RI team to worry about.
9	B: Outline Trial Partner Expectations & Establish Contact	(MikeF add) → → MikeF: Can RI WS handle the handoff to the appropriate WS? The ask is for parallel/multiple RI installs. Who will identify & support these candidates?
10	B: Descriptor File - finalize & perform PoC Refer to: [RI 1 Core]: Work with OPNFV Infra WG for the evolvment of PDF to fit into need for CNTT RI #526 [RI 1 Labs] Tooling to Generate a PDF matching Airship Internal Data Structure #525	(MikeF add) - That is OPNFV issue not CNTT. need to discuss who should own this. off course CNTT team needs to be part of the discussion, but someone from OPNFV has to take ownership of this and lead that discussion. → MikeF: Agree. RI WS is managing this today. Need handoff from RI WS to OPNFV. Should keep this open through the handoff.
11	A: 1.3 scope: clarify scope as per Prague discussion. A: 1.4 remove Roadmap (this will be /should be covered in the overall CNTT Roadmap).	Ch01
12	A: 2.2 Remove RA Req as it is not necessary. A: 2.3 put req into a table with reference numbers. A: 2.4 Diagram is not relevant: This should be removed or moved to Chapter 4 (Lab requirement) and improve diagram as it is low quality. Do we need an example section?	Ch02
15	A: 3.1 clarify the intention of the chapter (to be able to create a PDF/IDF from the content of the chapter). A: It will be better to simplify the presentation of the content here (metadata driven approach will be recommended) To make it simple to create PDF/IDF from the chapter. A: 3.4 it mixes requirement, with architecture with state. (need to clean up and remove any duplications). Remove any reference to functest or certification or test cases, this should be all about the state of NFVI.	Ch03
13	C: Need a Topology Diagram. B: Need Networking/Switching Requirements.	Ch04
14	B: 5.2 Add more installer general requirement (e.g. the need for it to be opensource, the result of it needs to be opensource). A: 5.3: needs to agree if this is something we need to have in CNTT. (ongoing discussion in OPNFV about it)	Ch05
15	A: Create a Cookbook for Labs (how to access labs, types of labs available, etc)	Ch06
16	A: lot of clean-ups needed. Not mix labs with installation (Chapter 6 deals with labs) 7.2, 7.3, 7.4 will need to move to Chapter 6 since they are lab related. 7.6, 7.7 (clarify difference between deployment validation and development validation). This should be sanity check and not extensive testing.	Ch07
17	C: Need some initial content (are there no gaps? )	Ch08
Development
18	B: Descriptor File - finalize & perform PoC	Work with OPNFV to agree on approach for handling Descriptor Fromat for various installers.
19	A: Ensure RI-1 lab is installed / available for tests.	Confirm that the CI/CD Scripting works (Natural bi-product of lab install success)
20	A: Endure that funkiest is able to repeatedly validate the installation of RI.
21	A: RI-1 passes the RC-1 sanity check	RI supports RC Define pass criteria for sanity check

Reference Conformance 1

Michael Fix to add general to-do list in here.

#	Deliverable	Comments
	A: Type of conformance: Installation Validation: Sanity Test (APIs), Conformance against CNTT Specs: Feature-set and Configuration (Functional) Performance, etc	Define pass, compliance, verification, validation Updates to: Ch01: Introduction
	A: Define Conformance A: Replace Certification w/ Conformance B: 1.7 Results Collation & Presentation - write section on where/how results will be normalized, collated, and presented C: 1.8 Governance - expand to include LCM, define partnerships and expectations from these partners as to what info or support is exchanged, or provided C: [RC1 Ch01] Provide Verification Process, including Life Cycle Management #159	Updates to: Ch01: Introduction
	A: (General) Finalize Test tooling/Framework	Updates to: (NFVI) Ch02: NFVI Testing Framework Requirements
	A: Replace Certification w/ Conformance A: Finalize Test Hardness/Framework B. 2.6 Entry & Exit Criteria - review for completeness, and reach consensus from the community that criteria satisfies objectives for CNTT (as intake to testing, and delivery to telcos) B. 2.7.3.3 Test Results - community review and consensus needed on collation and portal requirements; need to identify and document portal, or dashboard, used for results presentation B. 2.7.4 Badging - need alignment with OVP / CVC on badging steps and expectations; need alignment to add Xtesting as an alternative to Dovetail	Updates to: (NFVI) Ch02: NFVI Testing Framework Requirements
	A: (General) Finalize TC Req	Performance is not required for April Updates to: (NFVI) Ch03: NFVI Test Cases Requirements
	A: Replace Certification w/ Conformance C: 3.5 Software & Hardware Reference - content needs to be written to identify requirements for testing Software Configuration/Profiles, and Hardware Configuration/Profiles. C: 3.6 Options & Extensions - content needs to be written to create requirements for testing/evaluating Options and Extensions available and configured. C: 3.8.2 Resiliency Measurements - need to be written C: 3.9 Test Cases - remove (move) to Ch 4	Updates to: (NFVI) Ch03: NFVI Test Cases Requirements
	A: Cleanup & Finalise NFVI Testing Cookbook (General) Test suite is created	Updates to: (NFVI) Ch04: NFVI Testing Cookbook Identify and writeup missing TCs Performance is not required for Baldy Supporting OvS-DPDK in RI-1 Test suite details
	A: Mapping Test Cases to CNTT Req	Updates to: (NFVI) Ch05: NFVI Test Cases and Traceability to CNTT Requirements
	A: [RC1 Ch04] Define the test cases for exposed infrastructure capabilities #774 - This issue is to capture the tests the exposed infrastructure capabilities as defined in RM §4.1.2. A: Finalize TC Req Mapping & Close TC Gaps A: Port needed/missing Test Cases (also with RC1 Dev) port YardStick testcases to Xtesting port Bottlenecks to Xtesting port StorPerf testcases to Xtesting port NFVbench testcases to Xtesting update and integrate heat-tempest-plugin in Functest Heat API testing integrate KloudBuster in Functest disk benchmarking add tempest-stress in Functest stress testing port VTP test cases to Xtesting B: 5.3 Traceability Matrix - write introductory explaining this section defines the mapping, or traceability of RM/RA-1 requirements to test cases C: Review the applicability for, &/or Create content for Test Case Traceability of the following: 5.3.8 Tenants 5.3.9 LCM 5.3.10 Assurance 5.3.11 Security 5.3.13 Resilience 5.3.14 Bare-metal validations	Updates to: (NFVI) Ch05: NFVI Test Cases and Traceability to CNTT Requirements
	C: (General) RC-1 cleanup (ch content complete 80%, issue closure)	Example: https://github.com/cntt-n/CNTT/issues?q=mvp+label%3A%22RC+1+Dev%22
	C: (General) Create Tools & Perform Hardware & Manifest Verifications	Part of long-term RC program
	C: (General) Perform Empirical Validations (against prototype VNFs)	Part of long-term RC program
	B: (General) Collect & Normalize Results	Needed for RC program to simplify reviews & badging
	A: Replace Certification w/ Conformance C: 6.2.2. Prototype VNFs - Identify reference VNFs per Family Types to be used for Empirical Data Collection and evaluation against 'real' VNFs B.3 Badging Requirements - 6.3.1 Badging Scope - reach consensus with community, and OVP/CVC on badging framework defined C: Expand (elaborate) the following which has no/limited content today: 6.4.11 User & System Interfaces - lacks context, lists only UI and Programming Interfacce 6.4.12 Deliverables - needs content to describe by Docker and Standalone Installation Scripts are needed and pertinent for VNF Frameworks	(VNF) Ch06: VNF Testing Framework Requirements
	A: Replace Certification w/ Conformance C: 7.5 Interaction Type - Describe the types of Interactions: Extended Topology, Complex (Akraino), Functional, HA, Fault, Interoperability	(VNF) Ch07: VNF Test Cases Requirements
	B: Chapter needs community review and inputs - integrated NFVI framework (Section 8.3) is missing context, or reference to prior NFVI E2E Framework chapters. Clean up needed. e.g. Content examples to add Identify Framework Needs, Goals, and Dependencies Define Opensource Integration (OPNFV, OVP, Functest, CVC, others) Provide Automation Toolchain (list, topology, flow) Missing content needs to be reviewed/vetted for inclusion, and if needed, write content: 8.2.2. Yardstick - purpose, adoption &/or use of project, define why important 8.2.3 Bottlenecks - purpose, adoption &/or use of project, define importance	(Dev) Ch08: VNF Testing Cookbook
	C: All Chapter Content needs to be written: Introduction - Provide an overview of the purpose for the VNF TC Traceability to RM Requirements chapter RM/RA1 Requirements - define requirements Test Case Traceability - tracing test cases to requirements	(VNF) Ch09: VNF Test Cases and Traceability to CNTT Requirements

	A: 9.3 TC Mapping to Requirements needs explicit reference to RM/RA-1 requirements mapping. Very general mappings at present, with not explicit requirement traceability, or reference to NFCN traceability in prior chapter.	(Dev) Ch09: NFVI Tests Traceability to TC Requirements
	B: Content needs to be expanded: 10.2 OpenStack Release Comparisons - need to upload/add existing content regarding detail / comparison of OpenStack releases based on Pike baseline for CNTT RI-1 (e.g. Ocata, Pike, Queens, Stein, etc) 10.5 Framework Gaps - VTP is referenced as framework gap, but need to also include: Hardware (baremetal) Validations Software (manifest) Validations VVP heat artifact/template validations	(Dev) Ch10: Gap analysis & Development:
Development
	A: (General) RI-1 passes the RC-1 test suite execution (For sanity and APIs)

Reference Architecture 2

Tom Kivlin to add general to-do list in here.

#	Deliverable	Comments
1	A: Chapter completeness: Chapter 1 bogometer at 100% (stable scope/principles, etc.) Chapter 2 bogometer at 100% (stable requirements) Chapter 3 bogometer at 80% Chapter 4 bogometer at 80% Chapter 5 bogometer at 60% Chapter 6 bogometer at 60% Chapter 7 bogometer at 60% Chapter 8 bogometer at 80%	Align on RI2 on which chapters/sections are needed as a priority.
2	Chapter 1 A: Generic clean up, change bogometer, remove roadmap section to Technical document A: Clarify principles around cluster scope (per VNF, etc.), what RA2 specifies and what software vendors can specify
3	Chapter 2 A: Complete section 2.3 A: Close all backlog issues re. ch2
4	Chapter 3 A: TSC decided to remove Compute Intensive - remove from chapter A: Clearer link between Kubernetes / containerisation and sw/hw profiles B: Content addition to empty sections
5	Chapter 4 B: Way more detail required B: Detailing the constraints that make component decisions (e.g. kernel modules/versions/etc.) B: More detail on Kubernetes feature flags B: 4.7 - check on requirements driving ASM/NSM
6	Chapter 5 B: Add content
7	Chapter 6 B: Restructure and content review - focus on testable specification rather than guidance
8	Chapter 7 B: Add content
9	Chapter 8 B: Content needed in 8.3 - this defines the scope/requirements for any development effort
10	Appendix A B: Consider title change to reflect the focus being on how to move from VNF-only to VNF and CNF support from infrastructure C: Consider moving to a separate document, or even the CNCF TUG

Picks for Baldy

Picks for Baldy will go in Pull Request here: https://github.com/cntt-n/CNTT/pull/959

CNTT Baldy