U.S. government issues guidance for developers to secure the software supply chain: Key takeaways

Software supply chain attacks are on the rise, as cited in the Cloud Native Computing Foundation’s (CNCF’s) Catalog of Supply Chain Compromises. Industry leaders such as the Google, Linux Foundation, OpenSSF, and public sector organizations such as NIST have provide guidance on the topic over the past year or so.

The U.S. National Security Agency (NSA) alongside the Cybersecurity and Infrastructure Security Agency (CISA), and the Office of the Director of National Intelligence (ODNI) now join that list with their publication Securing the Software Supply Chain: Recommended Practices Guide for Developers. The announcement of the publication emphasizes the role developers play in creating secure software and states the guide strives to help developers adopt government and industry recommendations on doing so. Subsequent releases from Enduring Security Framework (ESF) will focus on the supplier and the software consumer, given the unique role each plays in the broader software supply chain and its resilience.

At a high-level the document is organized into three parts:

• Part 1: security guidance for software developers
• Part 2: software supplier considerations
• Part 3: software customer recommendations

The role of developers, software suppliers and customers

The guidance notes the unique role developers, suppliers and customers play in the broader software supply chain ecosystem.

U.S. Department of Defense

Software providers and their development teams may end up in the dichotomy of speed to market, versus secure and resilient software or software-enabled products.

As noted in the image above, each of the three roles has respective security activities it can and should be doing. These activities span the gamut from initial secure software development, composition and architecture all the way through security acceptance testing and integrity validation on the customers’ end.

Secure software begins with a secure software development lifecycle (SDLC) and the guidance cites many options teams can use, such as the U.S. National Institute of Standards and Technology’s (NIST’s) Secure Software Development Framework (SSDF), Carnegie Mellon University’s Secure Software Development Lifecycle Processes, and others such as the recently announced OpenSSF Secure Software Development Fundamentals courses.

U.S. Department of Defense

How to develop secure software

The guidance stresses not just using secure software development processes but producing tangible artifacts and attestations that are used for validation, both by the software producer and consumer to have assurances related to the security and resiliency of the software. These processes and activities include best practices such as threat modeling, SAST, DAST and pen testing but also using secure release activities such as digitally signing, a notable example being the increased adoption of Sigstore, which is a standard for signing, verifying and protecting software. The adoption and use of Sigstore is also cited in the OpenSSF’s Open Source Security Mobilization Plan as a method to deliver enhanced trust in the software supply chain.

Threat modeling gets a significant mention, recognizing that during product development and delivery teams should be examining the potential threat scenarios that can occur and what controls  could be put in place to mitigate them. Teams should also have established security test plans and associated release readiness criteria to ensure unacceptable vulnerabilities are not making it to production environments or getting to customers.

Mature product teams have established support and vulnerability handling policies as well. This includes having a system where product vulnerabilities can be submitted and an associated incident response team that is in a position to respond and get engaged should an incident occur. Given the impact developers can have on producing secure or insecure products, formalized assessment and training must occur. Determine what training is required and who is required to take it at a specified frequency. The OpenSSF’s Open Source Software Security Mobilization Plan lists upskilling developers on secure software development as a key objective that is recognized as an industry-wide necessity. Training topics include secure software development, code reviews, verification testing and using vulnerability assessment tools during development to drive down vulnerabilities that make it into their end products.

The activities and practices discussed above, such as secure development training, threat modeling, security test plan and developed security policies and procedures, are mapped to activities in the previously mentioned NIST SSDF, which will be a requirement soon for software vendors to self-attest to when selling software products to the U.S. federal government.

Secure code development has many aspects, including selecting programming languages that could mitigate vulnerabilities from the onset. There is also the need for organizations to address insider threats, which might be a compromised engineer or simply poorly trained engineers. Organizations can mitigate these threats by having codified source control processes with appropriate authentication, running static and dynamic tests on code, as well as looking for exposed secrets.

Organizations should also implement nightly builds and security regression testing to recognize and address flaws and vulnerabilities. Development efforts shouldn’t be ad hoc and should be mapped to specific system requirements with associated security testing to avoid feature creep which can introduce risk.

Code reviews should be prioritized, especially critical code to ensure fundamentals such as cryptography are in place and there are requirements for privilege escalation and access protection for resources. It isn’t just the code that must be secured but also the development environment. There has been notable incidents such as SolarWinds where the development environment can be compromised and poison downstream consumers so systems, such as developers endpoints, source code repositories and CI/CD pipelines, should be threat modeled and have vulnerability assessments conducted.

Open-source software (OSS) introduces its own unique risk and the guidance recommends using dedicated systems to download, scan and perform recurring checks on OSS components that internal development teams can use. This concept is also advocated by NIST in its Improving the Nation’s Cybersecurity executive order guidance for Section 4 and has been dubbed as continuous packaging.

Another practice emphasized is securing the developer environment, using secure development build configurations and secure third-party software toolchains and libraries. Development systems should be hardened and only used for development purposes, without internet access, and only with pre-approved tooling and software. The guidance recommends vetting third-party modules for CVEs against the NIST National Vulnerability Database (NVD). Tooling and automation can help facilitate this process and can even be done as part of the integrated development environment (IDE) using security dependency analyzers and similar tooling to identify vulnerabilities.

Hardening the build environment is critical, including the developer network, enterprise network and internal build environments. This mitigates threats being introduced from the internet and external malicious actors as well as integrity and validation measures to validate that malicious activities have not occurred to compromise products.

U.S. Department of Defense

Software components should be sourced from known trusted suppliers that meet organizational requirements and validated through methods such as SPDX or CycloneDX SBOM formats as well as the suppliers responsiveness to vulnerabilities with established methods for vulnerability reporting.

Securing the Software Supply Chain’s guidance goes beyond hardening the build environment to make recommendations such as using hermetic reproducible builds as well. This means fully declared build steps, immutable references, and no network access, as well as identical outputs and artifacts regardless of variable metadata changes to things such as timestamps.

Software should be delivered securely, including an SBOM of final composition to the customers. As part of package validation customers can use binary analysis outputs to ensure only intended software components are in place. To address compromises of software packages and updates, both product and components can make use of hashes and digital signatures for product distribution, components and upgrades. Organizations should also take steps to mitigate compromises of the distribution system itself. This can include applying security measures to repositories and package managers as well as using secure transport layer mechanisms.

Other resources for securing the software supply chain

The guidance includes a crosswalk among various scenarios with developers, suppliers and customers to specific practices outlined in SSDF. It also includes a mapping of dependencies and artifacts that exist among the supplier, third-party suppliers and the end customer.

A mapping to the SLSA framework shows how specific recommendations in the guidance map to the various levels of SLSA, ranging from L1 to L4. Lastly, there’s a comprehensive list of artifacts and checklists to be used throughout the SDLC and a list of informative references such as the cyber executive order, DoD and NIST documentation as well as industry organizations such as OWASP.

This secure software supply chain guidance is a critical resource that will undoubtedly be adopted by the industry as a go-to reference for organizations looking to bolster their software supply chain practices for both software producers and consumers. With this document taking a developer-centric focus, the industry would be well advised to look for the subsequent guidance, which will focus on software suppliers and consumers.