Measure Your Open Source Program’s Success

Open source programs are proliferating within organizations of all types, and if yours is up and running, you may have arrived at the point where you want to measure the program’s success. Many open source program managers are required to demonstrate the ROI of their programs, but even if there is no such requirement, understanding the metrics that apply to your program can help optimize it. That is where the free Measuring Your Open Source Program’s Success guide comes in. It can help any organization measure program success and can help program managers articulate exactly how their programs are driving business value.

Once you know how to measure your program’s success, publicizing the results — including the good, the bad, and the ugly — increases your program’s transparency, accountability, and credibility in open source communities. To see this in action, check out example open source report cards from Facebook and Google.

Facebook’s open source program office periodically posts the month-over-month results from its open source projects internally and sends an executive report to management. “Reports are just a good way to raise awareness,” said Christine Abernathy, Open Source Developer Advocate at Facebook. “Even though Facebook places a high value on open source (as an organization), it’s still always a good thing to market yourself internally all the time and show your value.”

Existing tools can help you measure program success. You can begin by setting up the right tools for collecting data and make sure the data sources are clean and in a format that everyone can understand. Many organizations create a dashboard of metrics for their open source programs, to track all of the data in one place and provide project snapshots that can help assess progress at a glance. (See our guide on Tools for Managing Open Source Programs.)

Key metrics for measuring open source program success

There are countless ways to measure success and track progress for open source programs. Project health isn’t the only thing to track, but is important. “How do you actually get the smartest people in the world working at your company?” asks Chris Aniszczyk, Executive Director of the Open Container Initiative and COO of the Cloud Native Computing Foundation (and former head of open source programs at Twitter). “Well, you open source stuff and then you convince them to contribute to your projects.”

It helps to be able to quantify project health. GitHub’s guide on open source metrics gives a great overview of what project maintainers should pay attention to.  Some key project metrics to track are:

• Number of contributors (and ratio of internal to external contributors)
• Number of pull requests submitted, opened and accepted (and time remaining open)
• Number of issues submitted (and length of time remaining open)
• Number of commits per contributor (internal and external)
• Number of external adopters
• Number of projects created or contributed to (program wide)

Other metrics include popularity and awareness, influence, and program costs. As you delve into these metrics, you can concretely report everything from diversity of contributors to your projects to the number of followers you have across channels.

The Measuring Your Open Source Program’s Success guide can help you with all these initiatives and more, and it explores how to set program goals and measure whether or not they are being met. It is one of a new collection of free guides from The Linux Foundation and The TODO Group that are all extremely valuable for any organization running an open source program. The guides are available now to help you run an open source program office where open source is supported, shared, and leveraged. With such an office, organizations can establish and execute on their open source strategies efficiently, with clear terms.

You can read more in previous articles, on How to Create an Open Source Program and Tools for Managing Open Source Programs. We encourage you to check out all the guides and stay tuned for more coverage of them.