• B: 1.x Various: May need to be rationalized w/ new Tech Steering document

• A: 1.9 Roadmap: This section needs perpetual maintenance (strongly consider moving the roadmap out of the RM, altogether)

• A: RA Compliance/traceability methodology 

• A: Still needs a crisp executive summary identifying the problem statement, and the two key methodologies employed to address them (VNF Abstraction and a common NFVI)

RM-Ch01 - Intro

• A: RA Compliance/traceability methodology 
  • Needs to be Chapter 8 content (move to)

• B: Needs a community aligned list of workloads the platform is intended to support and their priority, which can then be used as a basis for resource allocation and for weighting “suggestions” (notably “The public cloud does it this way, so we should too.”; understood, but we need an objective basis. Right now, it becomes a subjective debate)
• A: Align and execute on Prague proposal to park the Compute Intensive flavor, and supplement it by removing "over-subscription?" from the Basic flavour
• C: Flesh out quantitative details about supported workloads. (nice to have)

• A: Needs tighter coupling w/ Container paradigm (i.e. decoupling from VM-based VNF)
  • flavours for example might not be applicable to containers(CPU and memory, disc we want to find a way for containers) - possibly Ch04
  • harmonisation of template in Model.
• C: Needs a general review. Given the experience we’ve gained, can be made more succinct and more usable by a VNF developer
• C: Need to clearly explain slicing (and contrast to Tenant) from CNTT perspective
• B: Template attributes should be vetted w/ underlying capabilities, rationalized for duplication (i.e. CNTT value add, vs. what’s already in HEAT) and expanded to integrate Container paradigms
• B: Data Model - Create actual schema(s) and data model(s) for CNTT-specific information (potentially in an appendix)
• A: ms Shared w/ RM-Ch4: Need to define virtual networking strategy and related attributes/parameters (workload level)

• A: Integrate more Container paradigms/support
• C: ms Flesh out Generic Fabric Model (Fabric, Underlay and Overlay level (i.e. Infra, not workload)
• C: Research and enhance storage performance extensions using performance per unit of storage model (i.e. IOPs/GB)
• A) Hardware Profiles and Performance.
  • Objectives/
  • Guidelines
  • What level of granularity.
  • Validation/Auditing.
  • Proposals or white paper on how we plan to achieve deterministic performance from the reference model w/o having to specify hardware to down to a very granular level (e.g. w/o SKUs, clock speeds, etc.)
• A: Clarify diagrams and language around T-Shirts, Flavors and IT relationships
• A: ms Policies for Non-Conforming Technologies // assumes supporting frameworks are ratified
  • SRIOV policy
  • SmartNIC policy*
  • Live Migration policy
  • *includes Tomas’ concept of driving SmartNIC industry to standard, cloud-friendly ABI
• A: Shared w/ RM-Ch3: Need to define virtual networking strategy and related attributes/parameters (workload level).
  • 4.2.2: need to consider aggregate network bandwidth instead of per Interface bandwidth.
• TBD: VNF Evolution / VNF Profile Generations
• TBD: Simplify VNF Profile Naming

• A: Integrate cloud native concepts w/ sw profile (i.e. how abstraction and workload portability is maintained)
• A: Update s/w profile, pending parking of Compute Intensive IT
  • Rationalize s/w profile to IT mapping after removal of third IT
  • Audit chapter alignment w/ RM-Ch4
• A: Incorporate VNF Profile Generations/Evolution (once ratified), designing and describing linkage/relationship to h/w profiles

• B: Flesh out initial Enabler Services

• A: ms Worthy of review by SMEs for suggestions of areas to incorporate and/or refine

RM-Ch07 - Security

Added Security Requirements PR#1118

• Cleanup to Ch08
  • A: Generic  qualification model (i.e. NFVI and VNF)
    • Badging types (requirements in RC)
    • Methodology to support N-3 VNF Rls Profiles
  • A: Overview of badging methodology
    • Base-lining testbed before VNF qualification
    • VNF testing without prior knowledge
    • Plumbing testbed for supplier testing of VNF functionality, after VNF badging.
  • A: ms Overview of novel VNF-related concepts
    • Reference VNF (aka Golden VNF)
    • NFVI characterization, and correction coefficients for normalizing performance
  • A: What is out of scope for CNTT (e.g., VNF functional testing, performance, scalability, HA, etc.)

RM-Ch08 - Compliance, Verification, and Certification

Should the RM content be VNF/CNF agnostic?

• C: Create Generic Installer Model

• C: Needs a strategy/purpose review and new commitment from original or new contributors to continue on it.

A: RA-1 cleanup (ch content complete 80%, issue closure)

A: Update OpenStack version based on newer version. (let continue discussion TSC call)

Develop after most Chapters greater than 60% complete

Issue #1081

• A: Remove 1.6 as this will be moved into Overall Roadmap.
• A: Remove CI (as this has been decided to be parked for a bit and picked up later).

A: Update RA-1 Ch02: Requirements

Align RM and RA-1 Requirements.

Review Networking Section and suggest Improvements

Ch03: Move to RM?

Ch04: Cyborg Added

B: Create content for RA-1 Ch06: Security: Create security requirements list

Ch06: Created RM Ch07 Consolidated Security Requirements++: currently under review

Content being added – based on RM Ch07 requirements

B: Create updated ToC and Content for RA-1 Ch07: Operations and Life Cycle Management

C: Close Gaps in Cookbook

[RC1 Ch9 - Snezka F2F] RC Cookbook Enhancements #945 

[RI1 Ch07] User Manual initial content #408

User-Execution

• Common Failure codes, description, and resolutions
• Test Framework Clean-Up Utility, Process, and Support Expectations

Process / Version Mgmt

• Process for version control
• Process for bug fixes and patches
• Containers Tagged
• Playbooks grab the tagged version

B:  Descriptor File - finalize & perform PoC

Refer to: 

• [RI 1 Core]: Work with OPNFV Infra WG for the evolvment of PDF to fit into need for CNTT RI #526
• [RI 1 Labs] Tooling to Generate a PDF matching Airship Internal Data Structure #525

• A: 1.3 scope: clarify scope as per Prague discussion.
• A: 1.4 remove Roadmap (this will be /should be covered in the overall CNTT Roadmap).

• A: 2.2 Remove RA Req as it is not necessary.
• A: 2.3 put req into a table with reference numbers.
• A: 2.4 Diagram is not relevant: This should be removed or moved to Chapter 4 (Lab requirement) and improve diagram as it is low quality. Do we need an example section?

• A: 3.1 clarify the intention of the chapter (to be able to create a PDF/IDF from the content of the chapter).
• A: It will be better to simplify the presentation of the content here (metadata driven approach will be recommended)
  • To make it simple to create PDF/IDF from the chapter.
• A: 3.4 it mixes requirement, with architecture with state. (need to clean up and remove any duplications). Remove any reference to functest or certification or test cases, this should be all about the state of NFVI.

• C: Need a Topology Diagram.
• B: Need Networking/Switching Requirements.

• B: 5.2 Add more installer general requirement (e.g. the need for it to be opensource, the result of it needs to be opensource).
• A: 5.3: needs to agree if this is something we need to have in CNTT. (ongoing discussion in OPNFV about it)

A: Create a Cookbook for Labs (how to access labs, types of labs available, etc)

A:  lot of clean-ups needed.

• Not mix labs with installation (Chapter 6 deals with labs)
• 7.2, 7.3, 7.4 will need to move to Chapter 6 since they are lab related.
• 7.6, 7.7 (clarify difference between deployment validation and development validation).
• This should be sanity check and not extensive testing.

C: Need some initial content (are there no gaps? )

Confirm that the CI/CD Scripting works (Natural bi-product of lab install success) 

• RI supports RC
• Define pass criteria for sanity check

A: Type of conformance:

• Installation Validation: Sanity Test (APIs),
• Conformance against CNTT Specs:
  • Feature-set and Configuration (Functional)
  • Performance, etc

• Define pass, compliance, verification, validation
• Updates to: Ch01: Introduction

A:  Define Conformance

A:  Replace Certification w/ Conformance

B: 1.7 Results Collation & Presentation - write section on where/how results will be normalized, collated, and presented

C: 1.8 Governance - expand to include LCM, define partnerships and expectations from these partners as to what info or support is exchanged, or provided

C:  [RC1 Ch01] Provide Verification Process, including Life Cycle Management #159

• Updates to: Ch01: Introduction

• Updates to: (NFVI) Ch02: NFVI Testing Framework Requirements

A:  Replace Certification w/ Conformance

A:  Finalize Test Hardness/Framework

B. 2.6 Entry & Exit Criteria - review for completeness, and reach consensus from the community that criteria satisfies objectives for CNTT (as intake to testing, and delivery to telcos)

B. 2.7.3.3 Test Results - community review and consensus needed on collation and portal requirements; need to identify and document portal, or dashboard, used for results presentation

B. 2.7.4 Badging - need alignment with OVP / CVC on badging steps and expectations; need alignment to add Xtesting as an alternative to Dovetail 

• Updates to: (NFVI) Ch02: NFVI Testing Framework Requirements

A: (General) Finalize TC Req 

• Performance is not required for April
• Updates to: (NFVI) Ch03: NFVI Test Cases Requirements

A:  Replace Certification w/ Conformance

C: 3.5 Software & Hardware Reference - content needs to be written to identify requirements for testing Software Configuration/Profiles, and Hardware Configuration/Profiles.  

C: 3.6 Options & Extensions - content needs to be written to create requirements for testing/evaluating Options and Extensions available and configured.  

C: 3.8.2 Resiliency Measurements - need to be written

C: 3.9 Test Cases - remove (move) to Ch 4

• Updates to: (NFVI) Ch03: NFVI Test Cases Requirements

A: Cleanup & Finalise NFVI Testing Cookbook

(General) Test suite is created

• Updates to: (NFVI) Ch04: NFVI Testing Cookbook

• Identify and writeup missing TCs
• Performance is not required for Baldy
• Supporting OvS-DPDK in RI-1
• Test suite details

A: Mapping Test Cases to CNTT Req

• Updates to: (NFVI) Ch05:  NFVI Test Cases and Traceability to CNTT Requirements

A:  [RC1 Ch04] Define the test cases for exposed infrastructure capabilities #774 - This issue is to capture the tests the exposed infrastructure capabilities as defined in RM §4.1.2.

A:  Finalize TC Req Mapping & Close TC Gaps

A:  Port needed/missing Test Cases (also with RC1 Dev)

• port YardStick testcases to Xtesting
• port Bottlenecks to Xtesting
• port StorPerf testcases to Xtesting
  port NFVbench testcases to Xtesting
• update and integrate heat-tempest-plugin in Functest Heat API testing
• integrate KloudBuster in Functest disk benchmarking
• add tempest-stress in Functest stress testing
• port VTP test cases to Xtesting

B: 5.3 Traceability Matrix - write introductory explaining this section defines the mapping, or traceability of RM/RA-1 requirements to test cases

C: Review the applicability for, &/or Create content for Test Case Traceability of the following:

5.3.8 Tenants
5.3.9 LCM
5.3.10 Assurance
5.3.11 Security
5.3.13 Resilience
5.3.14 Bare-metal validations

• Updates to: (NFVI) Ch05:  NFVI Test Cases and Traceability to CNTT Requirements

A:  Replace Certification w/ Conformance

C: 6.2.2. Prototype VNFs - Identify reference VNFs per Family Types to be used for Empirical Data Collection and evaluation against 'real' VNFs

B.3 Badging Requirements - 6.3.1 Badging Scope - reach consensus with community, and OVP/CVC on badging framework defined

C: Expand (elaborate) the following which has no/limited content today:

• 6.4.11 User & System Interfaces - lacks context, lists only UI and Programming Interfacce

• 6.4.12 Deliverables - needs content to describe by Docker and Standalone Installation Scripts are needed and pertinent for VNF Frameworks

A:  Replace Certification w/ Conformance

C: 7.5 Interaction Type - Describe the types of Interactions: Extended Topology, Complex (Akraino), Functional, HA, Fault, Interoperability

B: Chapter needs community review and inputs - integrated NFVI framework (Section 8.3) is missing context, or reference to prior NFVI E2E Framework chapters.  Clean up needed.

e.g. Content examples to add 

• Identify Framework Needs, Goals, and Dependencies
• Define Opensource Integration (OPNFV, OVP, Functest, CVC, others)
• Provide Automation Toolchain (list, topology, flow)

Missing content needs to be reviewed/vetted for inclusion, and if needed, write content:

8.2.2. Yardstick - purpose, adoption &/or use of project, define why important

8.2.3 Bottlenecks  - purpose, adoption &/or use of project, define importance

C: All Chapter Content needs to be written:

• Introduction - Provide an overview of the purpose for the VNF TC Traceability to RM Requirements chapter
• RM/RA1 Requirements - define requirements
• Test Case Traceability - tracing test cases to requirements

A: 9.3 TC Mapping to Requirements needs explicit reference to RM/RA-1 requirements mapping.  Very general mappings at present, with not explicit requirement traceability, or reference to NFCN traceability in prior chapter.

B: Content needs to be expanded:

10.2 OpenStack Release Comparisons - need to upload/add existing content regarding detail / comparison of OpenStack releases based on Pike baseline for CNTT RI-1 (e.g. Ocata, Pike, Queens, Stein, etc) 

10.5 Framework Gaps - VTP is referenced as framework gap, but need to also include:

• Hardware (baremetal) Validations
• Software (manifest) Validations
• VVP heat artifact/template validations

A: Chapter completeness:

• Chapter 1 bogometer at 100% (stable scope/principles, etc.)
• Chapter 2 bogometer at 100% (stable requirements)
• Chapter 3 bogometer at 80% 
• Chapter 4 bogometer at 80%
• Chapter 5 bogometer at 60%
• Chapter 6 bogometer at 60%
• Chapter 7 bogometer at 60%
• Chapter 8 bogometer at 80%

Chapter 1

• A: Generic clean up, change bogometer, remove roadmap section to Technical document
• A: Clarify principles around cluster scope (per VNF, etc.), what RA2 specifies and what software vendors can specify

Chapter 2

• A: Complete section 2.3
• A: Close all backlog issues re. ch2

Chapter 3

• A: TSC decided to remove Compute Intensive - remove from chapter
• A: Clearer link between Kubernetes / containerisation and sw/hw profiles
• B: Content addition to empty sections

Chapter 4

• B: Way more detail required
• B: Detailing the constraints that make component decisions (e.g. kernel modules/versions/etc.)
• B: More detail on Kubernetes feature flags
• B: 4.7 - check on requirements driving ASM/NSM

Chapter 5

• B: Add content

Chapter 6

• B: Restructure and content review - focus on testable specification rather than guidance

Chapter 7

• B: Add content

Chapter 8

• B: Content needed in 8.3 - this defines the scope/requirements for any development effort

Appendix A

• B: Consider title change to reflect the focus being on how to move from VNF-only to VNF and CNF support from infrastructure
• C: Consider moving to a separate document, or even the CNCF TUG