2020 Prague Daily Summaries

Area leaders should use this page to enter their key findings/impressions/concerns regarding the event

Monday, January 13



Track

Key Points

Challenges

Next Steps / AIs

Track

Key Points

Challenges

Next Steps / AIs

Testing

Have 3 participants providing VNFs for testing

  • ULAK

  • Aarna Networks

  • STC

Working on two infrastructures, with 3rd being setup

  • 2 OpenStack / ONAP ElAlto

  • 1 OpenStack / ONAP Dublin (virtual, being setup)

VNF Images have uploaded to infrastructure

  • Getting infrastructure up and ready for testing

  • Some issues with ONAP reaching Open Stack API (should be solved by end of today)

  • Running conformance test cases 

  • Run validation test cases

CNTT

Community alignment

  • Need to align release with main open source communities.

  • OPFN, ONAP, OVP.

  • cross-community sessions to go through the details.

Containers verification

  • Containers technology is evolving technology, how CNTT will adopt such a dynamic environment. 

  • Launching/supporting OVP Phase 2.0 taking MVP approach.

OPNFV

CIRV is approved project in OPNFV - this is the delivery of RI and RC for CNTT.



  • This is THE project for creating the RI



CI/CD Evolution to solve today's problems

  • Writing/fixing Jenkins is hard, need more modern solutions,...

  • but which one(s)?

  • Projects will have greater responsibilities/More on their own

  • propose a transition plan, once we agree / accept the migration pain and real $$ costs

OPNFV PDF to Airship Manifest transformation

  • ← They aren't the same (but certainly possible to evolve to new/better description).

VMware Integrated OpenStack Reference Implementation

  • Only one reference implementation

  • Exploring rapid RI deployment tools



This is a Project proposal! VCF Installer

Green Bay Packers won their playoff round



Go Pack Go! https://www.google.com/search?client=firefox-b-1-e&q=NFL+playoffs

ONAP




ONAP Usability





  • Unawareness of key documentation

  • Unawareness of what has been done i.e. Modularity

  • Promote what we have done- bi-weekly news

  • Provide guidelines to PTL

  • Focus on Core Use Cases/Core components to reduce the complexity - including prioritization 

Platform Maturity for Guilin

Are we making progress on platform maturity?

  • Performance, Stability - only Integration checks

  •  Resiliency, Scalability = OOM?

  •  Operability - upgrade build

  • During the project lifecycle & Reviews, let's review these potential assumptions with the PTL

  •  Platform enrichments vs full E2E use cases

  •  CI/CD investments

  •  Introduce S3P reqs through the new Requirements subcommittee (formerly Use Case reqs)

SO Enhancement

Next steps support:

  • CNF support

  • TOSCA 

  • SO proposed architecture key additions:

    • Puccini

    • Artifacts

    • k8s APIs

    • Custom resources

    • Custom controllers

SECCOM

Main requirements for Frankfurt

  • Python migration to version3

  • Java migration to version 11

  • CII badging with better code coverage

  • OSJI backlogs

  • Remove hardocded passwords

  • Get HTTPS for all external ports

How to get all OSJI solved by PTL ? => Need help from TSC to progress

Proposals to remove all Root privileges to containers and to  automatically check in Gating

User management is also a key topic to be addressed. Some experience from ODL could be used.

OOM: What's new

Main objectives for Frankfurt

  • Migrate to Kubernetes 1.16

  • Migrate to Helm3

  • Ingress controller introduction

  • Database consolidation ongoing

  • Storage class

  • Password management

Creation of template for Helm charts





Discussions on database: do we still need so many DB types ? Impacts on operations (backup/restore)

Discussions on password managementto contuniue with dedicated sessions tomorrow

Closed Loop

TOSCA based closed Loop

Extend closed loop from DACE/Policy to other appalications

CLAMP extension

  • pre-deployment verification

  • app deployment

  • CL monitoring

  • CL update/delete









Tuesday, January 14



Track

Key Points

Challenges

Next Steps / AIs

Track

Key Points

Challenges

Next Steps / AIs

LF Staff

Be respectful of everyone's time

  • Extended debates 

  • Presenters not concluding on time

  • Please be connected to the bridge 10 mins before schedule

  • Please request follow-up discussions

  • Please stick to your scheduled timeslot 

Testing

Have starting running static testing

  • ONAP Robot is not able to configure the demo instances in the ONAP El Alto instances.

  • Some documentation for running VNF test cases Dovetail is missing online (in the VNF specific guide).

  • Doing testing by hand (i.e. using ONAP portal).

  • Working to solve the issues with Robot, so automated tests can be run.

@Lincoln Lavoiesend info to ONAP TSC regarding issuses related to robot  Jan 14, 2020 

CNTT

Targeted 2020 Release Discussions: (Alpha, Beta, General Availability Launch)

  • Have 2 organizations who have volunteered to be part of the Friendly (Alpha) release testing. (One with the initials "Red" and "Hat")

  • Start getting feedback from them ASAP on RI/RC and test results to date.

  • Get them participating in WS release and RI/RC planning to help shape the onboarding process



OVP 2.0 Launched

RI & RC

  • More resources needed, more alignment needed



OPNFV

  • Several sessions on progress that was made in CNTT RI and RC, which leverages OPNFV LaaS, Intel pods 10 and 15, and Functest.

  • Validation of hardware to support the software testing; Pharos 2.0 discussion; Manifest validation.

  • Defining the next release of OPNFV Verified Program.



Marching towards reference!

RI + RC WS Update:

  • This is the ONLY RI.  All VNF vendors must certify against this deployment (in their own lab or via LFN) in order to get their CNTT badge for VNFs.

  • Cookbook intended to be complete instruction set for VNF suppliers to set up environment and run test suites.

  • Working on framework and community ratification

  • VNF certification is to be run against the RI, but then also against the a NFVI vendor's RC compliant stack.

  • Need to determine test scope - VNF is part of the scope

  • Issue that NFVI vendors must be able to Certify their products separately, with reference VNFs

  • Continue iterating on the documentation and cookbook.

  • Cookbook Validation and "Golden" VNF Creation.  Friendly Trial: Looking for volunteer VNF Vendors to work through the process.

Hardware Delivery Validation

  • Part of CNTT certification is checking the hardware

  • This can be considered step 0 of an RC suite: does the hardware meet a bare minimum specification in order to be RC compliant.

  • How to get all this information? (tooling)

  • What information do carriers consider to be relevant?

  • Session continued tomorrow at 9:00 in Rm 220

OPNFV Release Process Evolution

  • a series of observations of where we are and what challenges we have

  • Operating at installer/feature (scenario) as the release artifact for past 4 years

  • Installer base has dwindled

  • Tools (test suites) have become more important

  • CNTT is temporary entity to stand up new process/model and hand off to OPNFV

  • Moved to part time release management support

  • How does the Heritage of successful and still-active projects continue?  Do these projects use independent Release? (TSC questions)

    • Independent release will still benefit from Release Marketing by using the project delta(s) from previous OPNFV Release.

  • Check ODL precedent for "academic" vs "core" released projects

  • This needs to be taken to TSC for a decision (with prior review & demo at weekly Tech Discuss? these two things are usually paired-up).

  • Two personalities: 1) the stable RC and 2) the original concept of closing gaps by working upstream and verifying the tip of  master

OVP 2.0

  • Working session to help define mission and scope.

  • What is the scope of OVP Ph2.0?

  • Do we need to define "Cloud Native" or just what is being verified? It is for compliance with the documentation from RA - these are the behavioral aspects, and so the question becomes does the implementation match?

  • Define scope of C/VNF testing

  • Rabi / Lincoln will issue a call for participants to get the 2.0 work stream kicked off.

ETSI NFV Mano and VNF Testing

  • TST010 MANO api tests - conformance with SOL002, 003, 005

  • Specialist task force assigned, funded by ETSI

  • Result - document and automated test cases

  • First time the test cases were written first and then the document (Test Driven Documentation!)

  • Challenge - doc becomes too big to edit - looking at splitting under ETSI guidelines

  • How to automate fault management?

    • Subscribe to alarm

    • Simulate fault - this is currently a manual step

    • Verify notification

  • Moving to gitlab and reduce editing churn

  • Looking for ways to get alarms or metrics injected into SUT

  • Collaborate with Barometer to see how to inject test metrics

Manifest Validation

  • Setup documents to describe the infrastructure

  • Seeks to answer: Can manifest be verified pre-deployment?

  • Manifest consumed by installers

  • Such manifest is installer specific format

  • Such manifest is installer specific format

  • Challenges

    • Manual 

    • Many different formats

  • Need volunteers to work on this

ONAP

Change Management support



























Presented what is done in Frankfurt and what is planned in Guilin























Build and Replace worklfow should allow to modify existing instances of the VNFs on the fly.

Lack of VNFC level reconfiguration is one if the missing features on SO/Controller side which is not suppoirted today. We want to solve this issue in the Guilin release

PNF support by ONAP Change Management is the key challenge, planned to extend in Guilin

Schema/VSP Update is a chalange in the Upgrade process in ONAP. We want to addres this issue and to enable update of the schema with adequate modification of the existing instnces of services - for PNFs, VNFs and CNFs

Integration with CDS and K8s workloads as one of the key drivers for improvements of change management porocess in ONAP as K8s provides scaling, upgrades and traffic distribution capabilities by itself. For Frankfurt integration with CDS for CNFs creation is introduces. IN further releases K8s scaling and upgrade capabilities should be introduced.



Transport slicing support by ONAP

This presentation focused on:

  • Introduction to Transport Slicing in 5G context

  • Design of Transport Slicing

  • How to extend CCVPN to enable Transport Slicing

  • How Transport Slicing fit into E2E Network Slicing

  • How to work with SDOs on coordinating aspects of standards

  1. Continue to work with the SDO's to improve and finalize the IETF TEAS working group draft on Transport Slicing

  2. Finalize the architecture of Networking Slicing on how NFVO should be used

  3. Should the Transport Slicing solution support CNF Connectivity?

  4. Should Transport Slicing be considered as a Generic Network Building block?

Network slicing (core network) ongoing support by ONAP

CSMF and NSMF part of ONAP. External NSSMF to be used.

Demo for Frankfurt:

  • Transport subnet not in scope

  • Simulator RAN and core NSSMF simulator

ONAP components Impacts

  • No evolution in SDC

  • New WF for SO and NSSMFAdaptor

  • OOF used for NSI selection

  • &AI impact to include new 3GPP concepts (service profile, slice profile...)

  • 2 new portals: CSMF and NSMF (under UUI project)



Challenges on whether this is the only possible implementation or there may be more

Alignment on models and API  with 3GPP and  ORAN A1

Next step for G Release: include transport and mronitoring

Discussion on communication-service-profile introduced in ONAP

Warning on  templates that are deprecated within 3GPP

SECCOM: Password removal

All passwords should be stored only in Kubernetes secrets, users should be allowed to provide their own secrets for the deployment and allow to generate passwords at the deployment time.

Further exchanges with ONAP community on best practice approval and implementation - sessions at the PTLs and TSCs calls.

SECCOM: Ingress controller 

Ingress Controller introduced for Frankfurt as an option deployment

Effort on passing knowledge to ONAP teams on ingress controller and service mesh

Sessions at the PTLs calls.

ONAP SECCOM ISTIO opportunity for common continuation discussion   

Coexistance of AAF and Service Mesh in a long term security solution

 AAF and ISTIO to be treated as two separate tracks

 Use Case Cross-Carrier 5G 



 Get the Community Aligned on Network Slicing - AP for Architecture/Requirements subcommittee + Magnus (ONAP TCC)

What are the "baby steps" to build a "Slicing" roadmap

Promote ONAP through #1 use cases that does not require changes in ONAP; #2 POC so Innovation can continue while focus on ONAP Core. 

 Use Case Subcommittee

  •  Review of Frankfurt/Guilin Use cases



  •  Prepare comms about 'use cases' renamed as 'Requirements'

  • => Goals & Scope as per TSC vote last year

  •  Need to review TSC Chart

  •  Process to review to identify resources prior the M1 release to prepare the work upfront related to new reqs - to identify companies who will provide dev/test to implement them 

 Modeling Subcommittee

  • Review of the ongoing activities

  •  Latest feedback from ONAP/TM Forum from Magnus, our TCC representative

  • Information Model Documentation

  • API Swagger Documentation



Align Information Model with external SDO

Alignment with Documentation to include glossary from Modeling team and to improve API Documentation.

Communication to PTL to explain the API Swagger reco








Wednesday, January 15



Track

Key Points

Challenges

Next Steps / AIs

Track

Key Points

Challenges

Next Steps / AIs

Testing

  • CLI tests on Dublin working successfully; VNF vendors can/were able to run these to establish validation interop with ONAP

  • Got an instance of ONAP from AT&T (at 5pm) to try as a second approach to testing

  • Seem to have the HEAT VNF life-cycle tooling running on the lab ONAP instance as well (last failure was only a missing image in openstack).

  • Debugging of VNF test scripts is extremely challenging, logout, with most of the debugging relating to ONAP and not specifically the VNF on-boarding or running on the infrastructure.

  • Running the VNF tests on the AT&T infrastructure

    • Going to try and do this tonight, while we have AT&T CA folks online

  • Fix the missing image on the demo VNF

  • Running the VNF participants VNF through the testing.

CNTT

  • Technical Tracks:

    • RA-1

    • RA-2

    • RC

    • RI





  • Governance:

    • (On-boarding, Adoption, Release cadence, trials, OPNFV Alignment).





OPNFV

OPNFV's motto has long been working upstream.  The river analogy is taking new paths  and reaching other communities, embracing the changing tides of OPNFV Technical Steering.

RA 2 Deep Dive

  • Mostly review of Ch3 issues, followed by text review in first AM slot.



  • Forward-looking discussion of Programmable Network Fabric (P4). Has certain advantages unproven - Performance and scalability, but seen as challenging the usual NF vendor model by requiring application code in Smart NICs and the Controller Point  (split from User Plane).

CNTT | OPNFV Release & Lifecycle Planning

  • Ecosystem has shifted, OPNFV is refreshing and redefining what release artifacts will be

  • Straw model mode of operation presented

  • Chicken and Egg Problem →

  • RI1 is a good place to start as the processes are more mature and the upstream release cycles are better understood

  • RI2 is going to be much more of a learning process as CNF is still a very volatile environment wrt changes

  • So, which comes first?  The RC or the RI?  Can we release an RI without it having been certified?

  • Need to define release cadences, per stream (RI1, RI2)

CNTT RC Cookbook

Description:

  • Leverages existing OPNFV CI/CD toolchain (jenkins) and test results DB

  • Cookbook starts with Airship deployment, otherwise it is the same as RI

  • Test tools conform to RA Ch5, when RI available, can add any form of additional benchmarking

  • X-testing CI first creates a copy of the OPNFV CI in the local system

  • RC specifies what tests MUST be run, Functest Implements the tests 





  • There must be an exact record "Source of Truth" of what tests have been executed and the results.

    • Also, how do we patch in a reasonable way: Controls on the file that determines the tests run are needed. Processes needed. Essential to build the BADGING process up (from where we are.

  • Choices between Tools must be set by Policy == DETERMINISTIC

  • Trouble shooting needs to include Best Practices and General Guidelines, but also let the Community know (Open JIRA in Functest, or other tools) that something failed (more than log files from the failing test are needed). Need to open an issue for RC Appendix on Troubleshooting 

  • TSC must help find the PTLs of the necessary projects, but they haven't replied.  Also need to discuss the related "level-up" to CNTT question at TSC and Weekly Tech Discuss (with PTLs).

    • Need a real distinction between projects which are currently inactive but have responsible people available and projects that have working repos, but no support, (and should be closed).  See https://wiki.opnfv.org/display/PROJ/Project+Directory

    • CNTT RC has a list of projects they are looking at to fulfill their needs

OPNFV 2.0

Three big buckets of work: 

OPNFV mission refresh (which includes looking at the projects critically and make some hard decisions about projects that are outside the critical mission

Increase CNTT/OPNFV interactions (some projects and members heavily engaged)

Roadmap Development (Tactical now, multi-year next).

  • Solicit SPC help in development

Action Items: (8 total)

  • Catch the Balls from CNTT through Sunset

  • Plan OPNFV/CNTT Interactions

  • TSC faster/stronger/more involved in project and release objectives.

  • Focus on Adoption and why the Industry Should Care what OPNFV Accomplishes!

ONAP

Collaboration between 3GPP (and other SDOs) and ONAP -

  • 3 GPP/SA5 Liaisons to

  • TMF: many collaborations on API. TMF also launched various catalyst projects based on ONAP

How to provide links to ONAP implementations from 3GPP (eg Github, RTD link) ?

3GPP/SA5 collaboration to

Establish more communication wit TMF to get more feedback on ONAP (via catalyst experience)

Catalyst projects can be seen as use-case incubation

ONAP Integration

Gating retrospective

~2h to deploy ONAP

Dashboard: <add link>

Recap of KaaS (Kubernetes as a Service) POC to support Gating activities - deployment on Ms Azure

Avoid merge code when OOM gating is failing

Need to identify how we can pursue footprint reduction effort

Sync-up with SO, OOM team to identify additional 'build' improvement

Next focus on AAI



OVP Automation Augment with ONAP

  • urgently need to introduce automated testing tools to improve testing efficiency and solve problems in traditional testing

  • Can leverage ONAP and OVP projects to build the automated testing pipeline 

  • ONAP and OVP gap analysis

  • Suggestion for OVP:  provide common testing platform, focus on process automation and provide integration capability to integrate with different tools from  different upstream projects and vendors

  • The efforts have been done 

  • NFV automation testing survey

  • lack of automated tools for NFV tseting

  • ONAP and OVP project need to make up the gaps to achieve the automated testing

  • OVP should avoid duplicate work between different upstream project and push automation

need to work with OVP and ONAP  to identify the real telecom testing requirements and push the related work in ovp and ONAP to make up the gaps

<Other sessions>



Listen to the Daily Hudles recording for the full report



Thursday, January 16

Track

Key Points

Challenges

Next Steps / AIs

Track

Key Points

Challenges

Next Steps / AIs

Testing

  1. Got ONAP VVP testing (OOM Robot) running on two platforms.

  2. Worked on running testing on 3 commercial VNFs through these systems.

  3. Onboarded one of the VNFs through ONAP Dublin release.

  4. VNF static (template) validation is passing on all 3 VNFs.

  1. OPNFV XCI OpenStack setup provides HTTPS for OpenStack API by default, using self-signed certificates.  Within ONAP, this requires adding the self-signed CA to multiple pods. Should a step be added to the documentation / installed to allow a CA to be imported as part of the process?

  2. During ONAP deploy, the authentication keys should have been stored within correct formats for SO / Robot / etc. However, this seems to have failed during the install and required manual correction.  

  3. Repeatedly running e.g. the robot scripts while debugging can leak state into ONAP that requires manually cleaning databases. The option to rollback changes or having a “wipe clean” script for A&AI would be very useful.

  4. Initialization of values for ONAP (i.e. subscriber, cloudowner, line of business, etc.) isn’t clearly defined in the process, and if / who is responsible for setting those values.  For example “demo-k8s.sh onap init” will setup / provide one set of values, while the “instantiate-k8s.sh” for the VVP testing may assume a different set of values. It’s unclear in the documentation if VVP tooling would create these values if they aren’t yet existing in ONAP.

  5. VVP Validation false passed in the case where the vnf-details.json had a mismatch to the file name for the module preload file name.

  6. Two entry points for testing VNFs, based on VNF template types can be confusing to the users.

  7. Robot VVP script failures had to wait for timeout (i.e. script stopped) before logs became available to debug the issue.  

  8. Need to get some support from community to provide TOSCA based VNFs to run through the testing process.

  1. We (VNF participants) would like to continue debugging the testing next week (January 20-24), if the environments can be kept up.

    1. UNH-IOL environment will be kept up until at least 1/27/2020.

  2. For next DTF event, look into sending a weekly “briefing” email to all currently registered participants to point them to updated / latest resources, etc. This could also let them know about plugfest planning calls, etc.  Need to have at least one pre-event call specific to the plugfest, to make sure resources are aligned, etc.

CNTT







OPNFV

OVP Automation Augment with ONAP

  • urgently need to introduce automated testing tools to improve testing efficiency and solve problems in traditional testing

  • Can leverage ONAP and OVP projects to build the automated testing pipeline 

  • ONAP and OVP gap analysis

  • Suggestion for OVP:  provide common testing platform, focus on process automation and provide integration capability to integrate with different tools from  different upstream projects and vendors

  • The efforts have been done 

  • NFV automation testing survey

  • lack of automated tools for NFV tseting

  • ONAP and OVP project need to make up the gaps to achieve the automated testing

  • OVP should avoid duplicate work between different upstream project and push automation

need to work with OVP and ONAP  to identify the real telecom testing requirements and push the related work in ovp and ONAP to make up the gaps

ONAP

Cross-project AI Cooperation and Innovation

Network intelligence is ALL about DCAE or Close Loop in ONAP?
No

Need a focal point for both internal and external information exchange and efforts collaboration

  • Join the kick-off discussion next week for creating ONAP-AI WG

-Identify people, specify scope, establish process, and planning for Guilin

-Participate time poll by Friday at https://www.doodle.com/poll/q46kat8dx8xyd9dv



ONAP Projects Lifecycle and Review

Review the criteria associated to the lifecycle of a project from proposal, incubation, mature, Core and Archived, based on what we learned from other communities.  Goal is to focus resources on most mature and adopted projects.

Discussions about security/vulnerability issues that are identified post-mortem (after project termination/archived)

Checklist for the PTLs is in progress

Review will be performed as part of an architecture review outside the ONAP release cycle.



Release cadence in ONAP changes to a fixed cadence with three releases annually. The content of the releases is what is ready at the time of the releases. The proposal follows the same process as that used by other open source projects such as Openstack.

Formulate the process more fully.

Take the proposal to the TSC for approval.



+ Additional 5G Slicing topics

  • CSMF/NSMF Portal Design

  • Involvment of Data Lake services in 5G World 

  • SO