Tools & Services

Microsoft uses many tools and services in supporting its open source program. Here are just a few that you may find useful in your own company or open source community.

GitHub Enterprise Cloud with SAML single-sign on

Microsoft has over 70 GitHub organizations dedicated to open source activities. Collectively, these are part of an enterprise account at GitHub.

Some of the features the enterprise product brings to us include:

• SAML single sign-on adds an additional layer of protection for Microsoft by connecting with Microsoft's Azure Active Directory system, verifying access and protecting access tokens and SSH keys.
• Secure supply chain features such as private repo secret scanning help to discover accidental check-in of credentials, while vulnerability notifications and built-in dependency update pull request capabilities help projects to stay in the best shape.

GitHub Actions and Azure Pipelines

While Microsoft uses many different continuous integration systems, and open source projects adopt whatever common toolset an open source community prefers, many projects are powered by GitHub Actions and Azure Pipelines.

• GitHub Actions are the preferred way for projects to build and validate. Being built-in to the GitHub experience for developers, configuration is quick and easy.
• Azure Pipelines can be exposed to public, open communities, allowing many Microsoft projects to continue using the build system they've been using for many years, but with a collaborative angle.

Component Governance

At Microsoft, a internal extension for Azure DevOps called Component Governance was created to help provide automatic inventory of all the open source components used.

• Automatic inventory registration: every build at Microsoft is connected to Component Governance. Detected open source is registered, enabling teams to trace a specific build to the bill of materials it contained, and providing historical snapshots and build-by-build analysis of change. The automatic inventory is connected to an alerting system and business review system as needed.
• Security alerts: by connecting to security data sources, public known vulnerability data such as CVEs, and curated Microsoft-internal security knowledge, alerts can be raised at any time to help engineers protect Microsoft and its customers.
• Legal alerts: open source license data and algorithms specific to Microsoft's open source approach are used to generate alerts when specific components have additional legal obligations or requirements, such as publishing third-party source disclosures to https://3rdpartycode.microsoft.com/.
• Business review process: a legal and business review process, implemented as Azure Boards work items, is used for certain uses of open source.

GitHub Bots

Many different bots and applications are used as part of Microsoft's open source program. Bots help teams to scale and provide a great experience for communities.

Some of the common building blocks for GitHub Bots used at Microsoft include:

• Probot framework for building GitHub Apps to automate and improve your workflow
• octokit/rest.js REST API client for JavaScript and Node.js apps to connect to GitHub
• Octokit.net for .NET applications to integrate with GitHub

Self-service GitHub open source portal

When interfacing with third-party services such as GitHub, it is important to be able to identify employees at the same company working together on open source.

While GitHub allows organization members to publicize their organization membership on their individual profile, there is more to know. GitHub user management solutions will offer the following capabilities:

• Employment lifecycle automation: Employees should be able to join a company organization, if authorized, without needing a manual invitation. When an employee decides to leave the company, their access to company resources and GitHub organization membership should be removed.
• User linking: Authenticating an employee with IT and also GitHub to associate the GitHub account
• Communications: By linking an employee's corporate identity with their GitHub account, comms are possible without having to ask the user to share their email address or other information on their public profile.

Microsoft's self-service GitHub management portal is implemented in TypeScript and is a Node.js service. The portal is open source at https://github.com/microsoft/opensource-portal.

ClearlyDefined

ClearlyDefined provides license, source location, and attribution information on over 10 million open source components. We rely on this data for our compliance systems.

This open source project provides compliance data about open source components from across the package ecosystems. It uses multiple open source scanners and summarizes their results into a high-quality "definition" of the component upon which we base our business policy decisions internally. It also:

• Enables crowd-sourced curations: In cases where the data about a component is missing or incomplete, the project facilitates community curation of the data in order to improve it. This process is all done transparently and with multiple community reviewers.
• Facilitates data improvements upstream: Over time, the corpus of ClearlyDefined curations are intended to be submitted back to the upstream projects as pull requests so that future versions of the component are more clearly defined.

ClearlyDefined is an open source project of the Open Source Initiative (OSI) and its open source code is at https://github.com/clearlydefined.

Microsoft's Contributor License Agreement (CLA)

The Microsoft's Contributor License Agreement is a Contributor License Agreement solution that integrates nicely with GitHub to make sure that all contributors to a project have agreed to common terms by enabling contributors to sign CLAs from within a pull request.

GitHub data crawling

Microsoft has developed and adopted several different approaches to retrieving GitHub data about activity within our organizations: we use the GitHub REST API v3 and GraphQL to regularly make data about our GitHub repos, traffic data, issues and pull requests all available inside our big data systems.

By making data available in Azure Data Explorer, powered by Kusto, it's really quick for Microsoft engineers to be able to query data without having to build specialized GitHub API integrations.

Microsoft's 1ES team is in the process of open sourcing this technology.

Business review process with Azure Boards

Our business and legal review process - kicked off when a particular open source use, contribution, or release, scenario requires - integrates into the engineering system that includes Azure Boards. This helps meet engineers where they are, providing an easy way to review requirements, manage approvals and workflow, and eventually completing any necessary reviews.

This system is built by using the Work Item Tracking extensibility features and the Azure DevOps REST API.