How OpenSSF Scorecards Can Help Evaluate Open Source Software Risk

Everyone knows the expression “software is eating the world” by Marc Andreessen from over ten years ago. Software drives and touches nearly every aspect of modern society, both personal and professional, and is critical to the modern economy and national security.

It can also be said that open-source software (OSS) has eaten the software industry. The Linux Foundation and other groups have estimated that free and open-source software (FOSS) makes up 70% to 90% of every modern software product. Modern software is not only largely made up of OSS components, but IT leaders are also more likely to partner with vendors who also contribute to the OSS community.

The use of OSS is widespread because of its flexibility, cost savings, innovation through community-enabled projects, and arguably better security through increased code awareness, especially for large OSS projects. That said, OSS has its own concerns, including Common Vulnerabilities and Exposures (CVEs) for the affected code.

CVE is a MITRE project that aims to “identify, define and catalog publicly disclosed vulnerabilities in cybersecurity”. However, as the Cloud Native Computing Foundation (CNCF) Software Supply Chain Best Practices white paper notes, CVEs are a “trailing metric,” meaning they list vulnerabilities that are publicly revealed. They are also just one type of risk associated with software.

For this reason, organizations must use other methods to evaluate security health for OSS projects they use. One of the most notable is the Open Source Security Foundation (OpenSSF) Scorecards project.

What are OpenSSF scorecards for open source projects?

Scorecards, announced in late 2020, aims to automatically generate a security score for OSS projects to help consumers and organizations make risk-informed decisions about their OSS use. Organizations make overwhelming use of OSS dependencies, but determining the risk of consuming those dependencies remains a largely manual activity, especially at scale across the software ecosystem. The Scorecards project attempts to alleviate some of that burden using automated heuristics and security checks on a 0 to 10 scoring scale. It does this as it assesses OSS projects for security vulnerabilities that align with best practices such as signing or SAST, where already advocated by both public and private security leaders.

Chris Hughes

The Scorecards project doesn’t aim low either, scanning the one million most critical OSS projects based on direct dependencies and publishing the results to a public dataset on a weekly basis. In addition to leveraging this publicly available dataset, organizations can also run Scorecards on their own GitHub projects by using GitHub actions. When there is a change in the repository, the GitHub action is executed and alerts are issued to the administrators of those projects.

The Scorecards project uses a critical rating scale, high, medium, and low, the severity levels most security professionals are familiar with. It also uses a default list of checks that it runs on all the projects you target it at, whether they’re public projects or, if you’re using it natively, your own GitHub repositories.

You can go into what some of those checks are. They include fundamental security practices such as using industry protections, cryptographically signing releases, and the presence of unresolved vulnerabilities. To detect the presence of unresolved vulnerabilities, the Scorecards project uses the OSV Vulnerability Database. This is a distributed vulnerability database for OSS that uses the OpenSSF OSV format. OSV is at its core an aggregation of other databases of vulnerabilities that use the OSV schema, such as GitHub Security Advisories and the Global Security Database. OSC also supports both APIs and command line interface (CLI) tools for scanning SBOMs in CycloneDX or SPDX format.

The Scorecards project meets fortnightly and has an active Slack channel. It is led by facilitators from companies such as Google, Datto and Cisco. Since its inception, Scorecards has grown in popularity and is listed with over 3,000 Stargazers, or users who have essentially bookmarked the project. As organizations continue to strive for maturity of their OSS consumption management practices, the project will inevitably grow in popularity.

How can organizations use OpenSSF Scorecards?

The ability of organizations to conduct due diligence, governance and risk management of their OSS consumption is largely in its infancy. We see a big push to strengthen software supply chain resilience and software supply chain security practices of mature organizations. We have NIST’s Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations Rev.1, Secure Software Development Framework (SSDF), the OpenSSF OSS Security Mobilization Plan, Supply Chain Levels for Software Artifacts (SLSA)), and other best practices and sources of guidance emerge. They all touch on the need to control an organization’s OSS consumption and ensure consumption is aligned with the organization’s risk tolerance.

While that may sound simple, the idea of ​​doing that across the entire robust ecosystem of OSS projects and components that organizations consume is not so trivial. OpenSSF’s Scorecards project provides an automated way to understand security and risk in more than a million leading OSS projects and use the project natively for their own software and projects within the organization.

Organizations can use Scorecards for projects they do not own through the CLI and use a package manager for projects such as npm, PyPi or RubyGems. Scorecards is also available as a Docker container.

Organizations and individual contributors can participate in the project, including submitting needs checks to qualify for the score assessment. Organizations can also customize their use of Scorecards and only perform specific checks, possibly aligned with their organizational or industry-specific security requirements.

For more specific guidance, organizations can refer to NIST’s Best Practices for Open Source Software Controls, published in response to Cybersecurity Executive Order (EO) 14028, Improving the Nation’s Cybersecurity. These include tiered capabilities based on organizational maturity at three levels: foundational, supportive, and enhancing. Within those layers are capabilities such as using software composition analysis (SCA) source code assessments to identify vulnerable components, prioritizing the use of programming languages ​​with built-in guardrails, and automating the process of collecting, storing, and scanning OSS components in hardened internal repositories before being introduced to production environments.