DRAFT (Table to be completed)

		Testsuite	VTP	Onap-tests
Project	repository	https://git.onap.org/testsuite/	https://git.onap.org/vnfsdk/refrepo/	https://gitlab.com/Orange-OpenSource/lfn/onap/onap-tests
	Leader	ONAP Integration team (Brian, Yang, Gary, Marco, ..)	VNFSDK (Victor, Kanag)	Orange
	Stats (February - may 2019)	27 contributors / 95 patches / more than 5 different companies)	3 contributors / 20 patches / 2 companies	9 contributors / 145 patches/ 3 companies
	Language	RobotFramework, Python	Java. Also supports to run test cases implemented in different languages	Python
	Links	https://wiki.onap.org/display/DW/Robot+Framework+Development+Guide https://wiki.onap.org/display/DW/Robot+Healthcheck+Tests+on+ONAP+Components	https://wiki.onap.org/pages/viewpage.action?pageId=43386304	https://wiki.onap.org/pages/viewpage.action?pageId=6593670&preview=%2F6593670%2F54722733%2Fonap_tests.pdf
Short Description		Robotframework testsuites to perform all types of tests: healthcheck, CSIT, VNF, web interface	Provides platform where VNFSDK, VVP communities test cases and others could be on-boarded and make them available over REST API, CLI and those test cases could be of different nature like compalince, life-cycle, performance and different run-time like java, python, etc. It provides test case managment, execution and result management, profile and vnf managment. Already Having integration with ONAP SDC, OPNFV dovetail, 3rd party registry. Provides capability to make use of ONAP command to consume directly as test cases over gPRC, which helps to run LCM with zero dev effort. More details LCM: https://wiki.onap.org/display/DW/OVP+LCM+Support	Python SDK for ONAP to manage VNF Life cycle based on VVP compliant Heat descriptor.
Current Usage in ONAP		release daily tests Dublin use case support including LCM tests Component CSIT test suites OOM Gating (Robot healthcheck + model distribution

)

Dublin use cases (e.g. 5G PNF)
Tosca verification

)	Tosca Compliance Check (OVP) VSP Compliance check (SDC design time, while on-boarding) End-End service provisioning for both HEAT and TOSCA (https://github.com/onap/integration/tree/master/test/hpa_automation )	OOM Gating (LCM tests) Third party VNF onboarding/instantiation Python CLI
VNF use cases currently supported		vFW vFWNG vFWCL vCPE vLB PNF registration ...	hpa Compalince testing	basic_vm (Ubuntu) for ONAP E2E sanity check on agting freeradius_nbi (CSIT NBI) clearwater_ims in theory any VNF with VVP compliant heat descriptor (generic framework, need config file but no additional code)
Integrations	Other ONAP components	natively integrated in onap namespace (robot pod)	Yes integrated with SDC	Black box approach versus ONAP - separation between SUT and the test tool, no hardcoded values in ONAP configurations embedded in xtesting framework (used also for infrastructure tests - same DB/API used by Dovetail for infra verification) Infra results: http://testresults.opnfv.org/test/api/v1/results?case_name=tempest_full&last=1 vnf results with ONAP: http://testresults.opnfv.org/onap/api/v1/results?case_name=basic_vm&last=1
	VVP		Yes	Yes consider VVP for heat template verification on code linting as a linter on patch submission (check all the compliancy of all refrenced heat files) based on ONAP official VVP docker
	Dovetail/OVP		Yes	No not natively but reuse all the same building block used by Dovetail (DB, test API, dockerization)
Item to do based on MVP page	Automation Script(s) to on-board VSP	Yes	Yes https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
	Automation Script(s) to Create VF	Yes	Yes https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
	Automation Script(s) to Create Service	Yes	Yes https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
	Automation Script(s) to Pre-Load

Image

Image	Yes	Yes NOT an ONAP operations, and is NFVI action, will use dovetail test case	Yes
Automation Script(s) to Submit Preloads	Yes	Yes https://wiki.onap.org/display/DW/OVP+LCM+Support	yes
Automation Script(s) to Instantiate VNF	Yes	Yes https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
Automation Script(s) to Healthcheck VNF	Yes	Yes https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
Automation to create OpenStack resources (via openstacksdk/shade)		Yes -NOT an ONAP operations, and is NFVI action, will use dovetail test case, -This could be managed thru MultiVIM test cases if required other than one listed https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
Automation to clean OpenStack resources		Yes -NOT an ONAP operations, and is NFVI action, will use dovetail test case -This could be managed thru MultiVIM test cases if required other than one listed https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
Automation to clean ONAP resources (Module, VNF, service instance)		Yes https://wiki.onap.org/display/DW/OVP+LCM+Support	Yes
Link with VNFREQTS	Yes

yes

Yes

No

Descriptors

Tosca support

Yes

No

but reuse of Tosca artifacts provided by SDC for runtime operations

Heat support

Yes

YesIntegration of VVP

Yes

as a linter on patch submission (check all the compliancy of all refrenced heat files) based on ONAP official VVP docker

CI/CD integration

Yes

With jenkins in onapci

though xtesting framework (used also for infrastructure tests - same DB/API used by Dovetail for infra verification)

NO

but Provides API, which could be integrated with CI//CD where ever requried

Yes

xtesting-onap with gitlabci for VVP verification on linting + chained-ci for On demand ONAP deployment + test

Results

management

dashboard

Yes

grafana dashboard http://onapci.org/grafana

Yes

Based on xtesting, all results pushed to a mongo DB (same than for infrastructure tests)

Several web front end (grafana, Kibana, static web pages) used as dahsboards

Versions Compared

Old Version 1

New Version Current

Key

DRAFT (Table to be completed)

Page Comparison

Versions Compared

Old Version 1

New Version Current

Key

DRAFT (Table to be completed)