The Economist reports that “Some apartment building owners now require tenants to provide a DNA sample of their dog so that unscooped poop can be penalized.” Alex Pentland of the Massachusetts Institute of Technology has invented a “sociometric” badge that measures tone of voice and propensity to talk or listen.

The current trend towards evidence-based decision-making devalues opinions alone. In fact, an evidence-based meeting design becomes optimal.

What is an Evidence-Based Management?

Evidence-based is a term taken from the field of medicine in the 1990s. Its principles extend across education, public policy, social work, and most recently, management. Because an evidence-based practice supports making decisions through  the explicit use of the best available facts, it demands:

• Asking: turning an issue or problem into an answerable question
• Acquiring: looking for and getting the evidence
• Appraising: judging the trustworthiness of the evidence
• Aggregating: assembling the evidence
• Applying: building the evidence into the decision-making process
• Assessing: evaluating the outcome of the decision taken

 

According to the Center for Evidence-Based Management (CEBMa):

Our mission is to promote, develop and teach evidence-based practice to enhance the profession of management. We provide support and resources to managers, teachers, and others interested in evidence-based management.

 

Research published by the Harvard Business Review discovered that:

Evidence-based management is conducted best not by know-it-alls but by managers who profoundly value how much they do not know.

What better role to lead with the support of an evidence-based meeting design than a facilitator?

The Harvard writers conclude:

If taken seriously, evidence-based management can change how every manager thinks and acts. It represents a way of seeing the world and thinking about the craft of management. It proceeds from the premise that using better, deeper logic and employing facts, to the extent possible, permits leaders to do their jobs better. Facing the hard facts and truth about what works and what doesn’t, understanding the scary half-truths that constitute so much conventional wisdom about management, and denying the total nonsense that too often passes for sound advice will help groups perform better.

From an Agile perspective . . .

. . . while Scrum builds itself around an empirical practice, Michael Bodé suggests the following:

“EBP (Evidence Based Practice) gives a framework for the higher level decision making process than is implied with Scrum. Though there is certainly a stress on using empirical data, Scrum fails to clearly guide the practitioners in the decision making process. If we translate an evidence-based practice into project management, we create the following:

• • Asking: What will the minimum viable product (MVP) look like such that it produces value?
  • Acquiring: What are the PBIs that will reach the goal of the MVP and produce value?
  • Appraising: What is the level of complexity of PBIs and their applicability to accomplishing the business value (acceptance criteria)?
  • Aggregating: What different categorizations and business values do individual and aggregate PBIs produce?
  • Applying: What PBIs should we plan for sprints and releases based on the priority and business value?
  • Assessing: What shift in PBIs do we need to execute based on current progress and other factors?”

While opinions and feelings always slip into discussions, without much challenge, best-of-breed facilitators become aware of the challenge of facts and evidence while discounting vague assertions. We remain convinced that the intuitional mind appeals to a higher sense of reason than the purely rational. 

Want to increase the quality of your deliverables? Increase the amount of challenge and demand for more evidence-based proof, as the most complex decision-makers do. In God We Trust, but everyone else brings evidence.

Our alumni understand that leading and facilitating is simpler and easier than coming up with an optimal meeting design. Therefore, consider Scrum’s Evidence-Based Management for Software Organizations (EBMgt™)[1] which measures value to help improve your organizational agility.

The EBMgt or Scrum’s Evidence-based approach enables service groups to make rational, fact-based decisions, taking conversations from preferences and opinions to logic and insight. Above all, details and a modified meeting design have been based on the “Evidence-based Management Guide: Empirical Management for Software Organizations” written by Ken Schwaber, Patricia Kong, and David Starr.

Because Service Groups Struggle to Prove their Value

Within service groups, so much effort is focused on features and functions that benefits get overlooked. For example, monitoring the efficacy of the software does NOT provide evidence of a group value-add, rather . . .

Outcomes Provide Evidence of Value and Ways to Improve

The Current Value of any organization must be supported by evidence of its ability to meet market demand with timely delivery (Time-to-Market) while being able to sustain delivery over time (Ability to Innovate.)  Therefore, Scrum’s evidence-based approach encourages groups to focus on the following Key Value Areas (KVA) categories:

1. Current Value
2. Time-to-Market
3. Ability to Innovate

• Current Value

Current Value reveals the organization’s actual value in the marketplace but has no relevance to an organization’s ability to sustain value in the future.

• Time-to-Market

Time-to-market evaluates the organization’s efficacy at delivering new features, functions, services, and products. Hence, without actively managing Time-to-Market, the ability to sustain delivering value in the future remains uncertain.

• Ability to Innovate

The Ability to Innovate helps avoid software that is overloaded by low-value features. Consequently, as low-value features accumulate, more of the budget and time is consumed maintaining the product, not increasing the capacity to innovate.

What to Measure

Within the KVAs, EBMgt recommends eleven Key Value Measures (KVMs). Additionally, each should stand on its own and remain clear and transparent.

KVA: Current Value

KVM:	Measuring:
Budget or revenue per Employee	Approved budget or gross revenue / #employees
Product Cost Ratio	All expenses that develop, sustain, provide services, and administer the product or system.
Employee Satisfaction	Because engaged employees are a major asset of any software group or organization.
Customer Satisfaction	Sound management, solid software, and fulfilled stakeholders.

 

KVA: Time to Market

KVM:	Measuring:
Release Frequency	The time needed to satisfy the customer with effective products and services.
Release Stabilization	Impact of poor development practices and underlying design and code base.
Cycle Time	The time (including stabilization) to satisfy a key set of customers or to respond to a significant organizational request.

 

KVA: Ability to Innovate

KVM:	Measuring:
Installed Version Index	Because of the difficulty customers face adapting or changing to a new release.
Usage Index	Determines a product that is burdensome and difficult to use and excess software that must be maintained even if rarely used.
Innovation Rate	Growth of technical debt caused by poorly designed and developed software. Consequently, the trend of the budget percentage being consumed keeps old software alive.
Defects	Additionally, measures increasingly poor-quality software that leads to greater resources.

 

How to Improve Empirically through EBMgt

Monitoring KVAs provides a great start toward managing with evidence, but not enough to change the way agility is managed. The EBMgt approach recommends four phases that enable organizations to constantly learn and improve the value derived from software investments.

1. Measure KVMs

First, build actual values for the KVMs. Now you have an initial view of organizational value. In the figure below, KVMs are displayed in a radar graph that helps visualize relative strengths and weaknesses. In the example, the group’s ability to bring new features, functions, and products to its customers is strong, but its costs and defects are high.

 

2. Select KVAs to Improve

Thus, with a clear view of current organizational value and an understanding of the measures that reveal it, leaders can now make informed decisions about which KVAs to change. Incremental changes performed in small learning loops are the most effective method for increasing an organization’s overall agility.

3. Conduct Practice Experiments to Improve Value

Next in sequence, practitioners select a single or small set of practices to use in an experiment. For example, a software group may want to increase quality to reduce the Defects of KVM. Therefore, an experiment might implement test-first practices in development teams. Making this change in a time-boxed experiment allows observation of the impact of these practices on overall organizational value.

4. Consequently, Evaluate Results

Finally, assess the results and impact of an experiment to monitor the trend of value over time. Because understanding the changes of the KVMs prepares the organization for its next learning loop. Consequently, organizations that track changes periodically across time learn from patterns that emerge. Therefore, you can interpret the final chart below:

 

[1] ©2014 Scrum.Org. Offered for license under the Attribution Share-Alike license of Creative Commons, accessible at http://creativecommons.org/licenses/by-sa/4.0/legalcode and also described in summary form at http://creativecommons.org/licenses/by-sa/4.0/.

Citations

Evidence-Based Management: The Basic Principles, Barends, Rousseau & Briner, Center for Evidence-Based Management, 2014.

Evidence-Based Management, Pfeffer and Sutton, Harvard Business Review, January 2006.

Evidence-Based Process as Applied to Scrum and Facilitation, Michael Bodé, MS 2015

______

Don’t ruin your career by hosting bad meetings. Sign up for a workshop or send this to someone who should. MGRUSH workshops focus on meeting design and practice. Each person practices tools, methods, and activities daily during the week. We call it the road to building high-value facilitation skills.

Our workshops also provide a superb way to earn up to 40 SEUs from the Scrum Alliance, 40 CDUs from IIBA, 40 Continuous Learning Points (CLPs) based on Federal Acquisition Certification Continuous Professional Learning Requirements using Training and Education activities, 40 Professional Development Units (PDUs) from SAVE International, as well as 4.0 CEUs for other professions. (See workshop and Reference Manual descriptions for details.)

Want a free 10-minute break timer? Sign up for our once-monthly blog HERE and receive a free timer along with four other of our favorite meeting tools.