Versions Compared

Version Old Version 32 New Version 33
Changes made by

Pankaj.Goyal

Michael Fix

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Update RI and RC with A, B, C, and identification of MVP for General & Chapter updates/needs.
Panel
titleTable of Contents
Table of Content Zone

Table of Contents
maxLevel2
exclude1\.1
stylenone

Panel
borderColorDarkSalmon
bgColorDarkSalmon

Overview

The intention of this page is to outline what CNTT "success" looks like by mid-April @ ONES-NA 2020, along with high level tracking items that need to be completed.

Success is defined as:

  1. TBD - documentation created ? test suites defined ? test suites executed ?
  2. TBD
  3. TBD
Panel
borderColorDarkSalmon
bgColorDarkSalmon

To-Do (In General)

This is a comprehensive list of work that can be done. A subset of the list below will go into  Baldy (and will be saved in the release notes for Baldy).

A: To be included in Baldy Release.

B: Nice to include in Baldy Release.

C: Low Priority for Baldy Release.

Reference Model

Mark Shostak to add general to-do list in here.

#Deliverable

Target Date

( // )

Comments0A: RM cleanup (ch content complete 80%, issue closure)1

  • B: 1.x Various: May need to be rationalized w/ new Tech Steering document

  • A: 1.9 Roadmap: This section needs perpetual maintenance (strongly consider moving the roadmap out of the RM, altogether)

  • A: RA Compliance/traceability methodology 

  • A: Still needs a crisp executive summary identifying the problem statement, and the two key methodologies employed to address them (VNF Abstraction and a common NFVI)

RM-Ch01 - Intro2
  • B: Needs a community aligned list of workloads the platform is intended to support and their priority, which can then be used as a basis for resource allocation and for weighting “suggestions” (notably “The public cloud does it this way, so we should too.”; understood, but we need an objective basis. Right now, it becomes a subjective debate)
  • A: Align and execute on Prague proposal to park the Compute Intensive flavor, and supplement it by removing "over-subscription?" from the Basic flavour
  • C: Flesh out quantitative details about supported workloads. (nice to have)
RM-Ch02 - VNF Requirements & Analysis3
  • A: Needs tighter coupling w/ Container paradigm (i.e. decoupling from VM-based VNF)
    • flavours for example might not be applicable to containers(CPU and memory, disc we want to find a way for containers) - possibly Ch04
    • harmonisation of template in Model.
  • C: Needs a general review. Given the experience we’ve gained, can be made more succinct and more usable by a VNF developer
  • C: Need to clearly explain slicing (and contrast to Tenant) from CNTT perspective
  • B: Template attributes should be vetted w/ underlying capabilities, rationalized for duplication (i.e. CNTT value add, vs. what’s already in HEAT) and expanded to integrate Container paradigms
  • B: Data Model - Create actual schema(s) and data model(s) for CNTT-specific information (potentially in an appendix)
  • A: Shared w/ RM-Ch4: Need to define virtual networking strategy and related attributes/parameters (workload level)

RM-Ch03 - Modelling

Pankaj.Goyal Flavours/Flavors are not mentioned in RM Ch03. The term "virtual resource" is used and that includes resources that result from both hardware (server) and Operating System virtualisation. Typically, VNFs and CNFs are  mentioned together; there is an exception that should be corrected.

4
  • A: Integrate more Container paradigms/support
  • A: Flesh out Generic Fabric Model (Fabric, Underlay and Overlay level (i.e. Infra, not workload)
  • C: Research and enhance storage performance extensions using performance per unit of storage model (i.e. IOPs/GB)
  • A) MVP: Hardware Profiles and Performance.
    • Objectives/
    • Guidelines
    • What level of granularity.
    • Validation/Auditing.
    • Proposals or white paper on how we plan to achieve deterministic performance from the reference model w/o having to specify hardware to down to a very granular level (e.g. w/o SKUs, clock speeds, etc.)
  • A: Clarify diagrams and language around T-Shirts, Flavors and IT relationships
  • A: Policies for Non-Conforming Technologies // assumes supporting frameworks are ratified
    • SRIOV policy
    • SmartNIC policy*
    • Live Migration policy
    • *includes Tomas’ concept of driving SmartNIC industry to standard, cloud-friendly ABI
  • A: Shared w/ RM-Ch3: Need to define virtual networking strategy and related attributes/parameters (workload level).
    • 4.2.2: need to consider aggregate network bandwidth instead of per Interface bandwidth.

RM-Ch04 - Infrastructure Capabilities, Measurements and Catalogue

Pankaj.Goyal Should the White Paper be an MVP?

5
  • A: Integrate cloud native concepts w/ sw profile (i.e. how abstraction and workload portability is maintained)
  • A: Update s/w profile, pending parking of Compute Intensive IT
    • Rationalize s/w profile to IT mapping after removal of third IT
    • Audit chapter alignment w/ RM-Ch4
  • A: Incorporate VNF Profile Generations/Evolution (once ratified), designing and describing linkage/relationship to h/w profiles
RM-Ch05 - Feature set and Requirements from Infrastructure6
  • B: Flesh out initial Enabler Services
RM-Ch06 - External Interfaces (APIs and Interfaces) 7
  • A: Worthy of review by SMEs for suggestions of areas to incorporate and/or refine
RM-Ch07 - Security8
  • A: Cleanup to Ch08
    • Generic  qualification model (i.e. NFVI and VNF)
      • Badging types (requirements in RC)
      • Methodology to support N-3 VNF Rls Profiles
    • Overview of badging methodology
      • Base-lining testbed before VNF qualification
      • VNF testing without prior knowledge
      • Plumbing testbed for supplier testing of VNF functionality, after VNF badging.
    • Overview of novel VNF-related concepts
      • Reference VNF (aka Golden VNF)
      • NFVI characterization, and correction coefficients for normalizing performance
    • What is out of scope for CNTT (e.g., VNF functional testing, performance, scalability, HA, etc.)
RM-Ch08 - Compliance, Verification, and Certification9
  • C: Create Generic Installer Model
RM-Ch09 - Infrastructure Operations and Lifecycle ManagementA
  • C: Needs a strategy/purpose review and new commitment from original or new contributors to continue on it.
RM-Appendix-A - VNF Guidelines

Reference Architecture 1

Pankaj.Goyal to add general to-do list in here.

#Deliverable

Target Date

( // )

Comments1

A: RA-1 cleanup (ch content complete 80%, issue closure)

 

2

Update OpenStack version based on newer version. (let continue discussion TSC call)

Overall3Update RA-1 Ch01: Introduction contentDevelop after most Chapters greater than 60% complete
  • Remove 1.6 as this will be moved into Overall Roadmap.
  • Remove CI (as this has been decided to be parked for a bit and picked up later).
Ch01:4

Update RA-1 Ch02: Requirements

Align RM and RA-1 Requirements.

Ch02:  A document provided to RM WSLs with a request for clean-up RM reqts (2020-01-27)Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediationCh02:5Finalise RA-1 Ch03: NFVI + VIM ArchitectureCh03: Topology: Cover DC/Edge and SLA driven.Ch03: 

Review Networking Section and suggest Improvements

Ch03: 

Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediationCh03: Remove 3.6 as Traceability is in Ch02.6Finalise Ch04: NFVI + VIM Component Level ArchitectureCh04: Add Hardware acceleration

Ch04: 

Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediationCh04: Remove 4.7 as Traceability is in Ch027Finalise RA-1 Ch05: APIs and InterfacesCh05:Qualify APIs that are actually utilised.Ch05:Add explanation on micro-versionsCh05:Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation8

Create content for RA-1 Ch06: Security: Create security requirements list

Ch06: Created RM Ch07 Consolidated Security Requirements++: currently under reviewCreate Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediationCh06:9

Create updated ToC and Content for RA-1 Ch07: Operations and Life Cycle Management

Ch07: Develop content including Logging and MonitoringCh07: Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation10Update RA-1 Ch08: Gaps, Innovation, and DevelopmentCh08: Adopt Open Source (Tungsten Fabric) SDN APIs, map back to Requirements and push for adoption by OSTK as a Neutron extensionCh08:Discovery: review ONAP A&AI. Support for specific use cases: packet acceleration (DPDK, SR-IOV, …)Ch08:Capture Prague Etherpad Items

Reference Implementation 1

Qiao Fu to add general to-do list in here.

#Deliverable

Target Date

( // )

Comments1Ensure RI-1 lab is installed / available for tests
  • Confirm that the CI/CD Scripting works (Natural bi-product of lab install success)
2


Panel
titleTable of Contents


Table of Content Zone

Table of Contents
maxLevel2
exclude1\.1
stylenone




Panel
borderColorDarkSalmon
bgColorDarkSalmon

Overview

The intention of this page is to outline what CNTT "success" looks like by mid-April @ ONES-NA 2020, along with high level tracking items that need to be completed.

Success is defined as:

  1. TBD - documentation created ? test suites defined ? test suites executed ?
  2. TBD
  3. TBD


Panel
borderColorDarkSalmon
bgColorDarkSalmon

To-Do (In General)

This is a comprehensive list of work that can be done. A subset of the list below will go into  Baldy (and will be saved in the release notes for Baldy).

A: To be included in Baldy Release.

B: Nice to include in Baldy Release.

C: Low Priority for Baldy Release.

Reference Model

Mark Shostak to add general to-do list in here.

#Deliverable

Target Date

( // )

Comments
0A: RM cleanup (ch content complete 80%, issue closure)

1

  • B: 1.x Various: May need to be rationalized w/ new Tech Steering document

  • A: 1.9 Roadmap: This section needs perpetual maintenance (strongly consider moving the roadmap out of the RM, altogether)

  • A: RA Compliance/traceability methodology 

  • A: Still needs a crisp executive summary identifying the problem statement, and the two key methodologies employed to address them (VNF Abstraction and a common NFVI)


RM-Ch01 - Intro
2
  • B: Needs a community aligned list of workloads the platform is intended to support and their priority, which can then be used as a basis for resource allocation and for weighting “suggestions” (notably “The public cloud does it this way, so we should too.”; understood, but we need an objective basis. Right now, it becomes a subjective debate)
  • A: Align and execute on Prague proposal to park the Compute Intensive flavor, and supplement it by removing "over-subscription?" from the Basic flavour
  • C: Flesh out quantitative details about supported workloads. (nice to have)

RM-Ch02 - VNF Requirements & Analysis
3
  • A: Needs tighter coupling w/ Container paradigm (i.e. decoupling from VM-based VNF)
    • flavours for example might not be applicable to containers(CPU and memory, disc we want to find a way for containers) - possibly Ch04
    • harmonisation of template in Model.
  • C: Needs a general review. Given the experience we’ve gained, can be made more succinct and more usable by a VNF developer
  • C: Need to clearly explain slicing (and contrast to Tenant) from CNTT perspective
  • B: Template attributes should be vetted w/ underlying capabilities, rationalized for duplication (i.e. CNTT value add, vs. what’s already in HEAT) and expanded to integrate Container paradigms
  • B: Data Model - Create actual schema(s) and data model(s) for CNTT-specific information (potentially in an appendix)
  • A: Shared w/ RM-Ch4: Need to define virtual networking strategy and related attributes/parameters (workload level)

RM-Ch03 - Modelling

Pankaj.Goyal Flavours/Flavors are not mentioned in RM Ch03. The term "virtual resource" is used and that includes resources that result from both hardware (server) and Operating System virtualisation. Typically, VNFs and CNFs are  mentioned together; there is an exception that should be corrected.


4
  • A: Integrate more Container paradigms/support
  • A: Flesh out Generic Fabric Model (Fabric, Underlay and Overlay level (i.e. Infra, not workload)
  • C: Research and enhance storage performance extensions using performance per unit of storage model (i.e. IOPs/GB)
  • A) MVP: Hardware Profiles and Performance.
    • Objectives/
    • Guidelines
    • What level of granularity.
    • Validation/Auditing.
    • Proposals or white paper on how we plan to achieve deterministic performance from the reference model w/o having to specify hardware to down to a very granular level (e.g. w/o SKUs, clock speeds, etc.)
  • A: Clarify diagrams and language around T-Shirts, Flavors and IT relationships
  • A: Policies for Non-Conforming Technologies // assumes supporting frameworks are ratified
    • SRIOV policy
    • SmartNIC policy*
    • Live Migration policy
    • *includes Tomas’ concept of driving SmartNIC industry to standard, cloud-friendly ABI
  • A: Shared w/ RM-Ch3: Need to define virtual networking strategy and related attributes/parameters (workload level).
    • 4.2.2: need to consider aggregate network bandwidth instead of per Interface bandwidth.

RM-Ch04 - Infrastructure Capabilities, Measurements and Catalogue






Pankaj.Goyal Should the White Paper be an MVP?

5
  • A: Integrate cloud native concepts w/ sw profile (i.e. how abstraction and workload portability is maintained)
  • A: Update s/w profile, pending parking of Compute Intensive IT
    • Rationalize s/w profile to IT mapping after removal of third IT
    • Audit chapter alignment w/ RM-Ch4
  • A: Incorporate VNF Profile Generations/Evolution (once ratified), designing and describing linkage/relationship to h/w profiles

RM-Ch05 - Feature set and Requirements from Infrastructure
6
  • B: Flesh out initial Enabler Services

RM-Ch06 - External Interfaces (APIs and Interfaces)
7
  • A: Worthy of review by SMEs for suggestions of areas to incorporate and/or refine

RM-Ch07 - Security
8
  • A: Cleanup to Ch08
    • Generic  qualification model (i.e. NFVI and VNF)
      • Badging types (requirements in RC)
      • Methodology to support N-3 VNF Rls Profiles
    • Overview of badging methodology
      • Base-lining testbed before VNF qualification
      • VNF testing without prior knowledge
      • Plumbing testbed for supplier testing of VNF functionality, after VNF badging.
    • Overview of novel VNF-related concepts
      • Reference VNF (aka Golden VNF)
      • NFVI characterization, and correction coefficients for normalizing performance
    • What is out of scope for CNTT (e.g., VNF functional testing, performance, scalability, HA, etc.)

RM-Ch08 - Compliance, Verification, and Certification
9
  • C: Create Generic Installer Model

RM-Ch09 - Infrastructure Operations and Lifecycle Management
A
  • C: Needs a strategy/purpose review and new commitment from original or new contributors to continue on it.

RM-Appendix-A - VNF Guidelines

Reference Architecture 1

Pankaj.Goyal to add general to-do list in here.

#Deliverable

Target Date

( // )

Comments
1

A: RA-1 cleanup (ch content complete 80%, issue closure)

 


2

Update OpenStack version based on newer version. (let continue discussion TSC call)


Overall
3Update RA-1 Ch01: Introduction content
Develop after most Chapters greater than 60% complete

  • Remove 1.6 as this will be moved into Overall Roadmap.
  • Remove CI (as this has been decided to be parked for a bit and picked up later).

Ch01:
4

Update RA-1 Ch02: Requirements

Align RM and RA-1 Requirements.


Ch02:  A document provided to RM WSLs with a request for clean-up RM reqts (2020-01-27)

Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation
Ch02:
5Finalise RA-1 Ch03: NFVI + VIM Architecture
Ch03: 

Topology: Cover DC/Edge and SLA driven.
Ch03: 

Review Networking Section and suggest Improvements


Ch03: 


Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation
Ch03: 

Remove 3.6 as Traceability is in Ch02.

6Finalise Ch04: NFVI + VIM Component Level Architecture
Ch04: 

Add Hardware acceleration

Ch04: 


Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation
Ch04: 

Remove 4.7 as Traceability is in Ch02

7Finalise RA-1 Ch05: APIs and Interfaces
Ch05:

Qualify APIs that are actually utilised.
Ch05:

Add explanation on micro-versions
Ch05:

Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation

8

Create content for RA-1 Ch06: Security: Create security requirements list


Ch06: Created RM Ch07 Consolidated Security Requirements++: currently under review

Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation
Ch06:
9

Create updated ToC and Content for RA-1 Ch07: Operations and Life Cycle Management


Ch07: 

Develop content including Logging and Monitoring
Ch07: 

Create Traceability entries in RA-1 Ch02. If Gaps and variances assign for completion/remediation

10Update RA-1 Ch08: Gaps, Innovation, and Development
Ch08: 

Adopt Open Source (Tungsten Fabric) SDN APIs, map back to Requirements and push for adoption by OSTK as a Neutron extension
Ch08:

Discovery: review ONAP A&AI. Support for specific use cases: packet acceleration (DPDK, SR-IOV, …)
Ch08:

Capture Prague Etherpad Items


Reference Implementation 1

Qiao Fu to add general to-do list in here.

#Deliverable

Target Date

( // )

Comments
1Ensure RI-1 lab is installed / available for tests
  • Confirm that the CI/CD Scripting works (Natural bi-product of lab install success)
2RI-1 passes the RC-1 test suite execution
  • RI supports RC
  • Define pass, compliance, verification, validation
3RI-1 cleanup (ch content complete 80%, issue closure)

4C: Resolve Continuous Software Deployment Errors
(MikeF add)
5C: Perform Repeatable CICD Deployment
(MikeF add)
6A: MVP Perform Repeatable Compliance Validations (post deployment)
(MikeF add)
7C: Close Gaps in Cookbook
(MikeF add)
8C: Conduct Friendly Cookbook Trial
(MikeF add)
9A: Select Vendor Candidates for RI installs
(MikeF add)
10A: Identify Target Labs for RI installs
(MikeF add)
11B: Outline Trial Partner Expectations & Establish Contact
(MikeF add)
12B:  Descriptor File - finalize & perform PoC
(MikeF add)
13
  • 1.3 scope: clarify scope as per Prague discussion.
  • 1.4 remove Roadmap (this will be /should be covered in the overall CNTT Roadmap)

Ch01
14
  • 2.2 Remove as it is not necessary.
  • 2.3 add in a table with reference numbers.
  •  2.4 This should be removed or moved to Chapter 4 (Lab requirement) and improve diagram as it is low quality. Do we need an example section?

Ch02
15
  • 3.1 clarify the intention of the chapter (to be able to create a PDF/IDF from the content of the chapter)
  • It will be better to simplify the presentation of the content here (metadata driven approach will be recommended)
    • To make it simple to create PDF/IDF from the chapter.
  • 3.4 it mixes requirement, with architecture with state. (need to clean up and remove any duplications). Remove any reference to functest or certification or test cases, this should be all about the state of NFVI.

Ch03
16
  • Need a Topology Diagram.
  • Need Networking/Switching Requirements.

Ch04
17
  • 5.2 Add more installer general requirement (e.g. the need for it to be opensource, the result of it needs to be opensource).
  • 5.3: needs to agree if this is something we need to have in CNTT. (ongoing discussion in OPNFV about it)

Ch05
18

Create a Cookbook for Labs (how to access labs, types of labs available, etc)


Ch06
19
  • A lot of clean-ups needed.
  • Not mix labs with installation (Chapter 6 deals with labs)
  • 7.2, 7.3, 7.4 will need to move to Chapter 6 since they are lab related.
  • 7.6, 7.7 (clarify difference between deployment validation and development validation).
    • This should be sanity check and not extensive testing.

Ch07
20

Need some initial content (are there no gaps? )


Ch08



Reference Conformance 1

Michael Fix to add general to-do list in here.

Work in progress - Mike will remove this banner when complete

#Deliverable

Target Date

( // )

Comments
1A: (General) MVP Finalize Test Harness/Framework
  • Updates to: (NFVI) Ch02: NFVI E2E C&V Framework Requirements
2

A: (General) MVP Finalize TC Req Mapping & Close TC Gaps


  • Performance is not MVP for April
  • Updates to: (NFVI) Ch04: NFVI TC Traceability to RA Requirements:
3

A: (General) MVP Test suite is created


  • Identify and Close Gaps in TCs
  • Performance is not MVP for April
  • Supporting OvS-DPDK in RI-1
  • Test suite details
4A: (General) MVP RI-1 passes the RC-1 test suite execution
RI supports RC#Deliverable

Target Date

( // )

Comments1Finalize Test Harness/Framework2

RC-1 Finalize TC Req Mapping & Close TC Gaps

  • Performance is not MVP for April
3

RC-1 test suite is created

  • Identify and Close Gaps in TCs
  • Performance is not MVP for April
  • Supporting OvS-DPDK in RI-1
  • Test suite details
4RI-1 passes the RC-1 test suite execution
  • Define pass, compliance, verification, validation
5RC-1 cleanup (ch content complete 80%, issue closure)Example:  https://github.com/cntt-n/CNTT/issues?q=mvp+label%3A%22RC+1+Dev%226

Restructure to remove confusion (https://github.com/cntt-n/CNTT/pull/961)

Overall

MVP: Define Conformance

MVP: Replace Certification w/ Conformance

1.7 Results Collation & Presentation - write section on where/how results will be normalized, collated, and presented

1.8 Governance - expand to include LCM, define partnerships and expectations from these partners as to what info or support is exchanged, or provided

(NFVI) Ch01: Introduction

MVP: Replace Certification w/ Conformance

2.6 Entry & Exit Criteria - review for completeness, and reach consensus from the community that criteria satisfies objectives for CNTT (as intake to testing, and delivery to telcos)

2.7.3.3 Test Results - community review and consensus needed on collation and portal requirements; need to identify and document portal, or dashboard, used for results presentation

2.7.4 Badging - need alignment with OVP / CVC on badging steps and expectations; need alignment to add Xtesting as an alternative to Dovetail 

(NFVI) Ch02: NFVI E2E C&V Framework Requirements

MVP: Replace Certification w/ Conformance

3.5 Software & Hardware Reference - content needs to be written to identify requirements for testing Software Configuration/Profiles, and Hardware Configuration/Profiles.  

3.6 Options & Extensions - content needs to be written to create requirements for testing/evaluating Options and Extensions available and configured.  

3.8.2 Resiliency Measurements - need to be written

3.9 Test Cases - remove (move) to Ch 4

(NFVI) Ch03: NFVI Test Case Requirements

MVP: [RC1 Ch04] Define the test cases for exposed infrastructure capabilities #774 - This issue is to capture the tests the exposed infrastructure capabilities as defined in RM §4.1.2.

MVP: Port needed/missing Test Cases (also with RC1 Dev)

  • port YardStick testcases to Xtesting
  • port Bottlenecks to Xtesting
  • port StorPerf testcases to Xtesting
    port NFVbench testcases to Xtesting
  • update and integrate heat-tempest-plugin in Functest Heat API testing
  • integrate KloudBuster in Functest disk benchmarking
  • add tempest-stress in Functest stress testing

4.3 Traceability Matrix - write introductory explaining this section defines the mapping, or traceability of RM/RA-1 requirements to test cases

Review the applicability for, &/or Create content for Test Case Traceability of the following:

4.3.8 Tenants
4.3.9 LCM
4.3.10 Assurance
4.3.11 Security
4.3.13 Resilience
4.3.14 Bare-metal validations

(NFVI) Ch04: NFVI TC Traceability to RA Requirements:

MVP: Replace Certification w/ Conformance

5.2.2. Prototype VNFs - Identify reference VNFs per Family Types to be used for Empirical Data Collection and evaluation against 'real' VNFs

5.3 Badging Requirements - 5.3.1 Badging Scope - reach consensus with community, and OVP/CVC on badging framework defined

Expand (elaborate) the following which has no/limited content today:

  • 5.4.11 User & System Interfaces - lacks context, lists only UI and Programming Interfacce
  • 5.4.12 Deliverables - needs content to describe by Docker and Standalone Installation Scripts are needed and pertinent for VNF Frameworks
(VNF) Ch05: VNF E2E C&V Framework Requirements

MVP: Replace Certification w/ Conformance

6.5 Interaction Type - Describe the types of Interactions: Extended Topology, Complex (Akraino), Functional, HA, Fault, Interoperability

(VNF) Ch06: VNF Test Case Requirements

All Chapter Content needs to be written:

  • Introduction - Provide an overview of the purpose for the VNF TC Traceability to RM Requirements chapter
  • RM/RA1 Requirements - define requirements
  • Test Case Traceability - tracing test cases to requirements
(VNF) Ch07: VNF TC Traceability to RM Requirements

Chapter needs community review and inputs - integrated NFVI framework (Section 8.3) is missing context, or reference to prior NFVI E2E Framework chapters.  Clean up needed.

e.g. Content examples to add 

  • Identify Framework Needs, Goals, and Dependencies
  • Define Opensource Integration (OPNFV, OVP, Functest, CVC, others)
  • Provide Automation Toolchain (list, topology, flow)

Missing content needs to be reviewed/vetted for inclusion, and if needed, write content:

8.2.2. Yardstick - purpose, adoption &/or use of project, define why important

8.2.3 Bottlenecks  - purpose, adoption &/or use of project, define importance

(Dev) Ch08: E2E Framework Integration

MVP: 9.3 TC Mapping to Requirements needs explicit reference to RM/RA-1 requirements mapping.  Very general mappings at present, with not explicit requirement traceability, or reference to NFCN traceability in prior chapter.

(Dev) Ch09: NFVI Tests Traceability to TC Requirements(Dev) Ch10: VNF Tests Traceability to TC Requirements

  • Define pass, compliance, verification, validation
3RI-1 cleanup (ch content complete 80%, issue closure)4
  • 1.3 scope: clarify scope as per Prague discussion.
  • 1.4 remove Roadmap (this will be /should be covered in the overall CNTT Roadmap)
Ch015
  • 2.2 Remove as it is not necessary.
  • 2.3 add in a table with reference numbers.
  •  2.4 This should be removed or moved to Chapter 4 (Lab requirement) and improve diagram as it is low quality. Do we need an example section?
Ch026
  • 3.1 clarify the intention of the chapter (to be able to create a PDF/IDF from the content of the chapter)
  • It will be better to simplify the presentation of the content here (metadata driven approach will be recommended)
    • To make it simple to create PDF/IDF from the chapter.
  • 3.4 it mixes requirement, with architecture with state. (need to clean up and remove any duplications). Remove any reference to functest or certification or test cases, this should be all about the state of NFVI.
Ch037
  • Need a Topology Diagram.
  • Need Networking/Switching Requirements.
Ch048
  • 5.2 Add more installer general requirement (e.g. the need for it to be opensource, the result of it needs to be opensource).
  • 5.3: needs to agree if this is something we need to have in CNTT. (ongoing discussion in OPNFV about it)
Ch059

Create a Cookbook for Labs (how to access labs, types of labs available, etc)

Ch0610
  • A lot of clean-ups needed.
  • Not mix labs with installation (Chapter 6 deals with labs)
  • 7.2, 7.3, 7.4 will need to move to Chapter 6 since they are lab related.
  • 7.6, 7.7 (clarify difference between deployment validation and development validation).
    • This should be sanity check and not extensive testing.
Ch0711

Need some initial content (are there no gaps? )

Ch08

Reference Conformance 1

Michael Fix to add general to-do list in here.

Work in progress - Mike will remove this banner when complete

  • Updates to: (NFVI) Ch01: Introduction
5C: (General) RC-1 cleanup (ch content complete 80%, issue closure)
Example:  https://github.com/cntt-n/CNTT/issues?q=mvp+label%3A%22RC+1+Dev%22
6

A: (General) MVP Restructure to remove confusion (https://github.com/cntt-n/CNTT/pull/961)


Overall

C: (General) Perform Hardware & Manifest Verifications
Part of long-term RC program

C: (General) Perform Empirical Validations (against prototype VNFs)
Part of long-term RC program

B: (General) Collect & Normalize  Results
Needed for RC program to simplify reviews & badging
7

A: MVP: Define Conformance

A: MVP: Replace Certification w/ Conformance

B: 1.7 Results Collation & Presentation - write section on where/how results will be normalized, collated, and presented

C: 1.8 Governance - expand to include LCM, define partnerships and expectations from these partners as to what info or support is exchanged, or provided


(NFVI) Ch01: Introduction
8

A: MVP: Replace Certification w/ Conformance

A: MVP: Finalize Test Hardness/Framework

B. 2.6 Entry & Exit Criteria - review for completeness, and reach consensus from the community that criteria satisfies objectives for CNTT (as intake to testing, and delivery to telcos)

B. 2.7.3.3 Test Results - community review and consensus needed on collation and portal requirements; need to identify and document portal, or dashboard, used for results presentation

B. 2.7.4 Badging - need alignment with OVP / CVC on badging steps and expectations; need alignment to add Xtesting as an alternative to Dovetail 


(NFVI) Ch02: NFVI E2E C&V Framework Requirements
9

A: MVP: Replace Certification w/ Conformance

C: 3.5 Software & Hardware Reference - content needs to be written to identify requirements for testing Software Configuration/Profiles, and Hardware Configuration/Profiles.  

C: 3.6 Options & Extensions - content needs to be written to create requirements for testing/evaluating Options and Extensions available and configured.  

C: 3.8.2 Resiliency Measurements - need to be written

C: 3.9 Test Cases - remove (move) to Ch 4


(NFVI) Ch03: NFVI Test Case Requirements
10

A: MVP: [RC1 Ch04] Define the test cases for exposed infrastructure capabilities #774 - This issue is to capture the tests the exposed infrastructure capabilities as defined in RM §4.1.2.

A: MVP: Finalize TC Req Mapping & Close TC Gaps

A: MVP: Port needed/missing Test Cases (also with RC1 Dev)

  • port YardStick testcases to Xtesting
  • port Bottlenecks to Xtesting
  • port StorPerf testcases to Xtesting
    port NFVbench testcases to Xtesting
  • update and integrate heat-tempest-plugin in Functest Heat API testing
  • integrate KloudBuster in Functest disk benchmarking
  • add tempest-stress in Functest stress testing

B: 4.3 Traceability Matrix - write introductory explaining this section defines the mapping, or traceability of RM/RA-1 requirements to test cases

C: Review the applicability for, &/or Create content for Test Case Traceability of the following:

4.3.8 Tenants
4.3.9 LCM
4.3.10 Assurance
4.3.11 Security
4.3.13 Resilience
4.3.14 Bare-metal validations


(NFVI) Ch04: NFVI TC Traceability to RA Requirements:
11

A: MVP: Replace Certification w/ Conformance

C: 5.2.2. Prototype VNFs - Identify reference VNFs per Family Types to be used for Empirical Data Collection and evaluation against 'real' VNFs

B.3 Badging Requirements - 5.3.1 Badging Scope - reach consensus with community, and OVP/CVC on badging framework defined

C: Expand (elaborate) the following which has no/limited content today:

  • 5.4.11 User & System Interfaces - lacks context, lists only UI and Programming Interfacce
  • 5.4.12 Deliverables - needs content to describe by Docker and Standalone Installation Scripts are needed and pertinent for VNF Frameworks

(VNF) Ch05: VNF E2E C&V Framework Requirements
12

A: MVP: Replace Certification w/ Conformance

C: 6.5 Interaction Type - Describe the types of Interactions: Extended Topology, Complex (Akraino), Functional, HA, Fault, Interoperability


(VNF) Ch06: VNF Test Case Requirements
13

C: All Chapter Content needs to be written:

  • Introduction - Provide an overview of the purpose for the VNF TC Traceability to RM Requirements chapter
  • RM/RA1 Requirements - define requirements
  • Test Case Traceability - tracing test cases to requirements

(VNF) Ch07: VNF TC Traceability to RM Requirements
14

B: Chapter needs community review and inputs - integrated NFVI framework (Section 8.3) is missing context, or reference to prior NFVI E2E Framework chapters.  Clean up needed.

e.g. Content examples to add 

  • Identify Framework Needs, Goals, and Dependencies
  • Define Opensource Integration (OPNFV, OVP, Functest, CVC, others)
  • Provide Automation Toolchain (list, topology, flow)

Missing content needs to be reviewed/vetted for inclusion, and if needed, write content:

8.2.2. Yardstick - purpose, adoption &/or use of project, define why important

8.2.3 Bottlenecks  - purpose, adoption &/or use of project, define importance


(Dev) Ch08: E2E Framework Integration
15

A: MVP: 9.3 TC Mapping to Requirements needs explicit reference to RM/RA-1 requirements mapping.  Very general mappings at present, with not explicit requirement traceability, or reference to NFCN traceability in prior chapter.


(Dev) Ch09: NFVI Tests Traceability to TC Requirements
16

C: All Chapter Content needs to be written:

  • Introduction - Define, and describe the purpose of this chapter to be a) Define RM/RA-1 Openstack requirements, and b) Map Framework to Requirements
  • RM/RA-1 Requirements - List out reference to Test Cases Requirement (coming from Chapter 6) and map them to real project test scripts.
  • TC Mapping to Requirements - Based on specific requirement, or use case, need to provide a Mapping of TCs to Requirements

(Dev) Ch10: VNF Tests Traceability to TC Requirements
17

B: Content needs to be expanded:

11.2 OpenStack Release Comparisons - need to upload/add existing content regarding detail / comparison of OpenStack releases based on Pike baseline for CNTT RI-1 (e.g. Ocata, Pike, Queens, Stein, etc) 

11.5 Framework Gaps - VTP is referenced as framework gap, but need to also include:

  • Hardware (baremetal) Validations
  • Software (manifest) Validations
  • VVP heat artifact/template validations

(Dev) Ch11: Gap analysis & Development:



Reference Architecture 2

Tom Kivlin to add general to-do list in here.

#Deliverable

Target Date

( // )

Comments
1TBD



Panel
borderColorDarkSalmon
bgColorDarkSalmon

Picks for Baldy

Picks for Baldy will go in Pull Request here: https://github.com/cntt-n/CNTT/pull/959