...
As a first step to define goals the following table collects what is currently documented in LFN, what other organizations have defined and what tools could be used for measurement and verification.
The first column collects the quality/security criteria from various source added in the other columns. Colored rows organize criteria in groups and contain sources that can be used for the criteria under the group (if not further detailed in the rows).
Quality Goal | Priority | Related | LFN Wiki | OpenSSF | Key Measures | LFX | GitHub | Notes | ||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Project Vitals | Project Data Template (currently used both for induction ☑️ and health review ✅ *) | LFN Lifecycle states and guidelines (metrics per lifecycle stage) | Best Practices | Scorecard | |||||||
2 | Project Name | ☑️ | README-template.md | |||||||||
3 | Project Creation Date | ☑️ | Age? | |||||||||
4 | Project License | ☑️ | LICENSE; README-template.md | Degree of FOS | ||||||||
5 | Legal Details and checks | |||||||||||
6 | Community Size | ☑️ | #s | |||||||||
7 | Contributing organizations (Diversity) | ☑️ ✅ | ☑️ | #s | ||||||||
8 | Number of contributors | ☑️ | #s | |||||||||
9 | Lifecycle Stage | ☑️ | ||||||||||
10 | Release schedule | ☑️ | Months | |||||||||
11 | Adoption | ☑️ | Not sure how to measure. Downloads? | |||||||||
12 | Health Review | |||||||||||
13 | Release Information | ✅ | ||||||||||
14 | Number of commits (over last year) | ✅ | ||||||||||
15 | Number of active committers | ✅ | ||||||||||
16 | Number of Active committers per organization | ✅ | ||||||||||
17 | Number of PR/changeset | ✅ | ||||||||||
18 | Mailing list activity | ✅ | ||||||||||
19 | Project & Community Resources | |||||||||||
20 | Website | ☑️ | README.md | Yes/NO | ||||||||
21 | Wiki | ☑️ | README.md | Yes/NO | ||||||||
22 | Mailing List | ☑️ | README.md | Yes/NO | ||||||||
23 | Slack | ☑️ | README.md | Yes/NO | ||||||||
24 | Community Meetings | ☑️ | README.md | Yes/NO | ||||||||
25 | Project Governance | |||||||||||
26 | TSC/TOC | ☑️ | ☑️ | GOVERNANCE.md; GOVERNANCE-elections.md; GOVERNANCE-maintainer.md; | Yes/No | |||||||
27 | Charter | ☑️ | Yes/No | |||||||||
28 | Code of Conduct | ☑️ | CODE_OF_CONDUCT.md; README.md | Yes/No | ||||||||
29 | How to contribute | ☑️ | ☑️ | CONTRIBUTING.md; README.md | Yes/No | |||||||
30 | Project Roles | ☑️ | ☑️ | CONTRIBUTOR_LADDER.md | Yes/No | |||||||
31 | Maintainers | ☑️ | ☑️ | MAINTAINERS.md | ||||||||
32 | How to Review | ☑️ | ☑️ | REVIEWING.md | ||||||||
33 | Adding/Removing PTLs | ☑️ | ☑️ | MAINTAINERS.md ?? | ||||||||
34 | Sub-Project Lifecycle | ☑️ | ☑️ | GOVERNANCE-subprojects.md | ||||||||
35 | Dispute Resolution | ☑️ | ||||||||||
36 | Adding/removing committers | ☑️ | ☑️ | |||||||||
37 | Sub-projects without a lead | ☑️ | ||||||||||
38 | Documentation | [english] | ||||||||||
39 | Technical Documentation | ☑️ | ☑️ | what should be minimum criteria? or scale | ||||||||
40 | Contributor onboarding Documentation | ☑️ | Yes/No | |||||||||
41 | Company Diversity (past 12 months) | ☑️ | ||||||||||
42 | Number of Contributors | ☑️ | ||||||||||
43 | Release Management | ☑️ | [version_unique][version_semver][version_tags][release_notes][release_notes_vulns] | Release notes contains patched and outstanding defects ( details) | ||||||||
44 | CI CD integration | ☑️ | degree of automation, analysis tools, traceability of overrides, failures | |||||||||
45 | Adoption | ☑️ | ||||||||||
46 | Security Design Principals | ☑️ | ||||||||||
47 | Use Case/ Problem Statement | |||||||||||
48 | Problem that project solves | ☑️ | README.md | |||||||||
49 | Use Cases Scenarios | ☑️ | README.md | |||||||||
50 | Infrastructure Tooling | |||||||||||
51 | Wiki | ☑️ | ||||||||||
52 | Repos | ☑️ | ||||||||||
53 | Bug Tracking tool | ☑️ | ||||||||||
54 | Code review | ☑️ | ||||||||||
55 | CI/CD tooling | ☑️ | ||||||||||
56 | Collaboration Tooling | ☑️ | ||||||||||
57 | Roadmap | |||||||||||
58 | Near/long-term objectives | ☑️ | ||||||||||
59 | Milestones | ☑️ | ||||||||||
60 | Risks/Challenges | ☑️ | ||||||||||
61 | Timeline | ☑️ | ||||||||||
62 | Security Best Practices | |||||||||||
63 | Security Contacts | Yes, channels, | We are lacking security contacts from projects | |||||||||
64 | Code Scanning | ☑️ | license, code vulnerability, static, dynamic, manual | Snyk, Blubracket | ||||||||
65 | Seed code handoff | ☑️ | ||||||||||
66 | Coding Standards | ☑️ | ☑️ | |||||||||
67 | Security design principals | ☑️ | OSSF Scorecard; OSSF Best Practices | self assessment or audit, outside assessment/audit | ||||||||
68 | Vulnerability Reporting | ☑️ | SECURITY.md; incident-response.md | [release_notes_vulns][vulnerability_report_process][vulnerability_report_private][vulnerability_report_response] | ||||||||
69 | Bug reporting | [report_process][report_tracker][report_responses][enhancement_responses][report_archive] | days to patch, bug found during ( unit, static, dynamic, integration, field operations | |||||||||
70 | Demonstrate Security Awareness | ☑️ | all of this column. | |||||||||
71 | Practice Secure Lifecycle Management (per release) | ☑️ | cryptographic practices; | |||||||||
72 | Security Documentation | ☑️ | [vulnerabilities_fixed_60_days] [vulnerabilities_critical_fixed] | |||||||||
73 | CI/CD best practices | ☑️ | ||||||||||
74 | Secure project architecture | ☑️ | ||||||||||
75 | Supply Chain Security | ☑️ | code intake scans, 3rd party code ver alignment, 3rd party code vulnerability reporting | There is also OpenSSF S2C2F | ||||||||
76 | SBOM creation | ☑️ | Yes/NO | |||||||||
77 | Static Application Security Testing (SAST) | ☑️ | [static_analysis][static_analysis_common_vulnerabilities][static_analysis_fixed][static_analysis_often] | <-- | low hanging fruit | |||||||
78 | Dynamic Application Security Testing (DAST) | ☑️ | [dynamic_analysis][dynamic_analysis_unsafe][dynamic_analysis_enable_assertions][dynamic_analysis_fixed] | <-- | ||||||||
79 | Software Composition Analysis (SCA) | ☑️ | SonarCloud / SonarType could be used. | |||||||||
80 | Container vulnerablitiy scanning | ☑️ | ||||||||||
81 | Code Coverage Testing | ☑️ | [test][test_invocation][test_most][test_continuous_integration]; [test_policy][tests_are_added][tests_documented_added] | <-- | ||||||||
82 | Code Quality | ☑️ | ||||||||||
83 | ||||||||||||
84 |
...