Case Study: How to Take the Pain out Portfolio Prioritization

Ed. Note: This is the story of how Decision Engine and the Collaboration Framework known as “Buy a Feature/Budget Games” transformed the fourth largest telecommunications company in the world. 

MasaMaeda
Masa K. Maeda, CEO of Valueinnova.

In June 2014, Masa K. Maeda, CEO of Valueinnova, Playcamp organizer and Conteneo Certified Collaboration Instructor, began work on an Agile Transformation project at the Ecuadorean office of the fourth largest telecommunication company in the world. As you’d expect, this company has a corporate presence in each country where it offers services and among all of its offices, the Ecuadorian headquarters was considered the most innovative by the senior leadership team.

“The agile transformation began with a very positive impact,” Masa relates, “spreading from 34 people in one department to more than 200 people in eight departments in only six weeks. This happened despite the fact that original contract was for the transformation of just one department.”

“The key to such an accelerated rate of adoption,” Masa continues, “was the ubiquitous introduction and widespread use of high collaboration frameworks (a.k.a. “serious games”) in the teams and at most levels of decision making.”

This initial success gave Valueinnova the opportunity to propose to the general manager that the company use Decision Engine and the collaboration framework “Buy a Feature/Budget Games” to prioritize the company’s 2015 project portfolio, and Valueinnova’s proposal was accepted.

The Backstory

The company’s typical project portfolio prioritization process would begin in October and be complete in December. Each of the company’s twelve departments first prioritized its own project portfolio, which was comprised of 10 to 15 project proposals. The set of twelve prioritized project lists were then handed to a board led by the general manager.

“The board would then go through the painstaking and time-consuming task of merging all those projects to generate one project portfolio of around 140 projects!” Masa relate. “They also preserved the order of projects from each department. No project proposed by any department was rejected, save rare exceptions.”

The issues with the original process were:
[unordered_list style=’circle’ animate=’no’]

  • Resource and time consumption: Many employees and decision makers were involved for too long—they had to give up a good portion of their daily activities during the three month period.
  • Quantity and quality repercussions: Because the company didn’t make hard prioritization choices, they ended up with too many projects, causing some projects to be delivered late due to insufficient resources and other projects to be delivered with poor quality due to cutting corners.
  • Local optimization: Since each department did its own project filtering, the board rarely rejected any projects, resulting in green-lit projects that had little relevance to the company’s bottom line. This localized optimization problem meant that some departments which should have been given more resources to grow faster were starved of their potential.
  • Silo mentality: Each department focused on its own projects without knowledge or interest in the projects from other departments. This is also why the board only merged the departments’ portfolios and did no filtering.
  • Failing economy: All the issues above ultimately had a negative impact on the overall profitability and economic viability of the company.

[/unordered_list]

“By using the collaboration framework “Buy a Feature/Budget Games” and the online prioritization platform Decision Engine, we sought to minimize—and possibly eliminate—those issues,” reported Masa.

In the Beginning

The first step was to ask all 12 departments to create a Business Model Canvas (BMC) for each project that was to become part of its proposed portfolio.the-business-model-canvas-shadow-hero

“There was some hesitation,” Masa said, “because the teams were afraid that this would increase the time needed to create each project portfolio.”

However, creating the Business Model Canvases ended up saving time overall; the act of creating the BMCs collaboratively meant that the teams actually better understood each project and were able to eliminate irrelevant projects early on. The total number of projects in the portfolio of each department was reduced by 30% to 45%, Masa reports, so the total number of projects to be sent to the board was considerably smaller than in past years.

To make sure the person was focused on the most important needs of the business, each project was classified as either strategic or progressive during the second and third week of November. The progressive projects remained under the decision-making control of the departments while the strategic projects were elevated to be used in Decision Engine under the belief that collaborative prioritization among the department heads would produce the best overall choices for the company.

Preparation

In the second week of December, each department generated a spreadsheet that included each project name, a one-paragraph description, and one paragraph indicating its benefits and compromises.

What Happened When: The overall project timeline and deliverables.
What Happened When: The overall project timeline and deliverables.

“I used these spreadsheets to prepare the three-round Decision Engine tournament,” said Masa. “I gave a copy of the list to all the managers and the board who were to participate, three days prior to the tournament for them to read and start getting acquainted with all the projects. In hindsight, I should have given them more time, but the schedule didn’t allow it.”

The day prior to the tournament, Masa organized two activities. First, the department managers and the board gathered together for a set of presentations by each department on its proposed projects. Each project was allotted 5-minutes (3 minutes for presentation and 2 minutes for Q&A). Second, everyone participated in a practice session using the online platform, Decision Engine, using dummy data to ensure everyone was comfortable with the platform and the game mechanics. “I wanted them to be able to focus entirely the prioritization activity,” reports Masa.

More Data

Masa also added two new elements to the process to gather even more data. The data analysis done after a Decision Engine forum typically compares the exhaustive data gathered by the online system (chats, bids, purchases etc.). Sometimes, the producers will also assign observers to work with the facilitator to record notes on participant behavior, which is very valuable information that influences the study for better results. In this case, Masa decided to add:

[ordered_list]

  1. Video and audio recordings of the sessions, and
  2. Heuristics based on fundamentals of Bayesian Statistics, to weigh variables taken from game observation such that applying the corresponding algorithm together with the game results would provide a better prioritization.

[/ordered_list]

Show Time

Room preparation began one hour in advance. In addition to Masa, the facilitation team included two volunteers with experience in high collaboration dynamics. One volunteer handled logistics, the other audio/video recording.

PrunetheTree
The team also used the collaboration framework “Prune the Product Tree” to collaboratively prioritize projects in the overall portfolio.

“We placed three session tables so that I could monitor all of them at the same time from a central table where I had 3 computers set up to facilitate the sessions,” detailed Masa. The team also had high-quality video and audio equipment set up to record each table. And they posted Prune the Product Tree posters on one wall, with large sticky notes printed with all the projects titles.

Everyone was on time and when the tournament began, all tables prioritized the first 50% of the projects in around 50 minutes.

“This first session started a bit slow,” Masa relates, “mostly due to discussion about the projects, and fortunately not due to the platform or game mechanics, demonstrating the benefit of the practice session done the previous day.”

The participants had a 15-minute coffee break at the end of the first session, so that Masa could set up the second forum. The participants then prioritized the remaining 50% of the projects in only 40 minutes.

“At that point I had to take the results of both games from all three tables and extract the top 10 projects to run the third session,” Masa reports. “We didn’t waste the time, however. While I set up the next set of forums, the two volunteers facilitated a Prune the Product Tree forum with all the participants to prune the entire project portfolio. I was ready to run the final prioritization session by the time they were done with the trees.”

The last Decision Engine forum took less than 30 minutes to complete, and all participants were able to leave earlier than scheduled. According to Masa, their familiarity with the projects was a huge contributor to more effective and proactive discussions. The discussions were also shorter because they focused on the value of the projects, rather than on understanding them.

The day-of agenda for the in-person prioritization.
The day-of agenda for the in-person prioritization.

Analysis

Masa collected the video and audio recordings, and the Prune the Product Trees (thus pruned!) and returned to his hotel room to begin analysis.

“This was a very involved activity,” said Masa. “I had to listed to every recording very carefully and map the information onto relevant variables to apply my algorithm. This was rather dynamic, since the variables emerged from the observation itself rather than being pre-determined, but this made it more effective.”

“I also added the results of Prune the Product Tree as a variable. Criteria included aspects such as the order in which projects were being discussed and purchased, the level of interest, amount of participation and other for a total of 15 variables.”

Masa reports that the analysis consumed the better part of two days. Once the data mapping was done, he ran the algorithm over the data. “I was very pleased with the results, because with the exception of one project, all were in agreement with what I had learned and observed during the past weeks. There was no bias since I didn’t participate on the games, and the data feeds were based on the observation captured by the cameras, microphones and the Prune the Product Trees.”

Masa used the one project that was in a higher priority than expected as a point of verification by reviewing all the data related to it. He found that the data effectively gave the project higher ranking. He then proceeded then to write the full report.

The results

Masa met with the team who helped him organize the Decision Engine tournament first, and they were amazed by the results and pleased with his explanation. Masa reports, “They were also surprised by the same project that I had surprised me. But they too agreed based on the data that its higher priority was correct.”

“The low esteem, so to speak, towards that project was because it wasn’t a sexy project. So while most people didn’t care for it, it absolutely needed to be done because it had to do with external governance.”

The next step was to present the results to the board. They were very impressed by the quality of the results, the process itself, the fact that the entire process took less than three weeks, the reduced number of projects and the already obvious economic benefit that was taking place.

The department heads and those who participated in the prioritization were also very pleased. The teams that generated the business model canvases and their department’s portfolio, also related to Masa that the experience was fun and helped them truly understand the projects.

“The decision makers said that it was the first time in the history of the company that they truly understood all the projects, and truly collaborated,” said Masa. They even gave higher priority to projects that weren’t their own; whereas in previous years, it was a battle to defend their own projects.

Moral of the story? Using Decision Engine and collaborative prioritization to prioritize their annual project portfolio brought the best out in all of them.


Conteneo and Persistent Systems Announce Partnership to Drive Enterprise Agile Transformation

Agile Experts Bernie Maloney & Luke Hohmann to Speak at Agile 2015 in Washington D.C.

 

Mountain View, CA — July 31, 2015 — Conteneo Inc., a leader in multidimensional collaboration, and Persistent Systems, a leading provider of services to build software-driven businesses, including the digital transformation of enterprises, today announced a partnership agreement to drive enterprise agile innovation for their respective customers, helping organizations leverage the advantage of agile across the enterprise.

“Engineering teams are experiencing a 30-300% productivity gain when applying Agile practices and methods to their work,” said Persistent Systems Agile Coach and Trainer Bernie Maloney. “It’s our belief that these collaborative agile practices can be applied beyond engineering, to the very business itself, driving tremendous benefits for these companies and their customers.”

Bernie Maloney will head two sessions at Agile 2015, touching on the advantages of agile across the organization:

 

  • August 4th at 9:00 AM – Create Influence, On Demand
  • August 6th at 3:45 PM – Bootstrap Your Business Model: Business Agility on the Back of a Napkin

 

“Effective collaboration is the foundation of agile methods and practices. It’s our mission to help organizations reap the benefits of agile, at scale, through multidimensional collaboration,” said Luke Hohmann, CEO & Founder of Conteneo Inc. and inventor of the Innovation Games® collaboration frameworks. “With Persistent Systems’ expertise in agile software development and enterprise digital transformation, and Conteneo’s cloud-based collaboration platform and services, our respective customers will benefit from agile methods across departments and divisions, from engineering to marketing to sales and beyond.”

Luke Hohmann will present the main stage keynote, “Awesome Superproblems” on August 3 at Agile 2015, focusing on how collaborative techniques can be used successfully for social good, as well as across the enterprise in the private sector. He will also be a presenter in the Agile Stalwarts track on Wednesday, August 5.

“We continue to strengthen our partner ecosystem to help our customers transform into software-driven businesses,” said Persistent Systems EVP Sudhir Kulkarni. “Conteneo’s prowess and focus on multidimensional collaboration, combined with our expertise in building software that enables our customers to drive their business, delivers an accelerated path to enterprise agile transformation for our customers.”

###

About Conteneo
Conteneo is the leading provider of multidimensional collaboration solutions for the public and private sector. The Conteneo Collaboration Cloud and our Collaboration Consulting services enable organizations to improve performance across the enterprise, including culture and change management, market research, strategy, complex sales, and innovation and product development. Current and past clients include Adobe Systems, Cisco, Emerson Climate Technologies, HP, Rackspace, Reed Elsevier, Qualcomm, SAP, Yahoo! and others. For more information, go to http://conteneo.co.

About Persistent Systems
Persistent Systems (BSE & NSE: PERSISTENT) builds software that drives the business of our customers; enterprises and software product companies with software at the core of their digital transformation.

Forward-looking and Cautionary Statements: For risks and uncertainties relating to forward-looking statements, please visit: http://content.persistent.com/Pslweb/forward_looking_cautionary_statement.shtml –

Media Contact

Luke Hohmann
CEO
Conteneo Inc.
info@conteneo.co
408-529-0319


On Choosing a Game (a.k.a. Collaboration Framework): Goals, Verbs, Nouns and Context

Designing and producing effective Innovation Games® and other collaboration frameworks boils down to gaining an understanding of these four key elements: goals, verbs, nouns and context. This post explains how to use these four key elements to design and produce great in-person and online forums, drawing on some examples from our client successes over the years.[separator type=’transparent’ color=” thickness=’1′ up=” down=”]

Goals

Goals are pretty straightforward: They are what you want to accomplish, ideally framed as a higher level business outcome. For example, you might be working for a company who wants to increase the average price per sale of large deals. Alternatively, you might want to improve your New Product Development (NPD) process by getting better concepts from the “fuzzy front end”.

Framing your goals in terms of measurable outcomes (“increase ASP by 10%”, or “generate 100 ideas that pass the first gate in our gate review process”) helps ensure that you’re choosing the right game – or no game at all. To illustrate, let’s look at two closely related business goals, both related to portfolio management.

A few years ago, Sallee Peterson, Senior Vice President of Global Customer Care for VeriSign, asked us to help her team prioritize a list of 46 projects. The business goal was to leverage the “wisdom of the crowd”, her globally distributed team, in selecting the best projects. (The careful reader will note that this last phrase is part of the context; I’ll return to this in a bit). We have a number of prioritization games, so this initial goal was already a good fit for the games.

Another leader contacted us to see how we might be able to use the games/collaboration frameworks to communicate the selection of projects within his company’s portfolio as they progressed through its gate process. In this case, the business goal of communication is not well served by the games. We recommended a simpler approach: a Wiki with an RSS feed and small, personal video updates on project status prepared by the project leaders. And it is easier than you think, especially if you own a phone that does video or a small video device. [separator type=’transparent’ color=” thickness=’1′ up=” down=”]

Verbs

Verbs are the actions that we take to accomplish our goals. Indeed, I am starting to think that if a business goal cannot be directly related to a verb, then it isn’t a good fit for a collaboration framework.

Returning to Sallee’s portfolio selection problem, the verb that equated to the goal was prioritize. Which is an amazingly powerful and common verb in business. We prioritize sales deals. We prioritize product features. We prioritize market segments. We even prioritize personas into “primary” and “secondary” personas.

When teaching the games, I will often take a few minutes to list as many business verbs as possible in five minutes or less. One group at Cisco generated more than 30 verbs, including generate, create, define, elaborate, group, establish, plan and develop.

As you gain experience in using collaboration frameworks, you’ll also gain experience in how the tense informs the business goals. Most of the time the tense is in the future, and you have to determine the time scale to determine the time frame of action (tactical or near term; strategic or long term). Other aspects of how the verbs of your business goals are used are important, so pay attention to the verbs associated with the goals.[separator type=’transparent’ color=” thickness=’1′ up=” down=”]

Nouns

Verbs are useless without the nouns they operate on. We prioritize sales deals, product features, the location where we’re going to have our corporate off-site, which customers we’re going to invite to our strategic advisory board meeting and so forth. The nouns/objects of the games become the metaphors we use in the visual collaboration forums, the items in a forum using the Buy a Feature collaboration framework, the objects and relationships between them in a Spider Web collaboration framework and so forth.

Once you understand the goals, verbs and nouns of the project, you’ll be well on your way to selecting one or two collaboration frameworks. Sometimes you don’t even realize how quickly this happens, so it is good to take a step back to ensure that you’ve got the right goals, verbs and nouns.

To illustrate, a few years ago Aladdin Knowledge Systems (since acquired by SafeNet) hired us to design and help produce a two-day sales training course for its world-wide distributors for a major new product launch, HASP SRM. Acting as strategic account managers, these distributors help SafeNet’s customer design and implement comprehensive Digital Rights Management solutions.

A simple statement of the goal was to train the distributors. A better goal was to focus on the actual outcomes a well-trained distributor would produce: more sales. We get more sales when the distributors can sell (verb!) HASP (noun!). We therefore designed a forum based on Product Box,  in which the distributors created boxes that demonstrated their ability to sell in complex situations.

To see if this was the right design, we had to compare it against the last element: context.[separator type=’transparent’ color=” thickness=’1′ up=” down=”]

Context

Context refers to such factors as who will be participating in the forum, their physical location, how many players will be included, whether or not the game will be part of a larger event and so forth.

Returning to Sallee’s challenge, the context included a 230+ team of customer support employees distributed across four locations around the world. In this case, the only option was to use Decision Engine and the Buy a Feature collaboration framework and construct a “feature tournament”. The resulting design enabled approximately 60% of the global workforce to participate in the games.

Changing the context will change how you design and produce the forums. NetApp, Wyse, and Rally Software Development are clients who have used the collaboration frameworks within customer advisory board meetings. In all of these cases, the small in-person context motivated them to use in-person versions of the collaboration frameworks.

Note that in these examples the selection of participants was also quite straightforward: Sallee wanted to include her employees, while the other examples wanted to include key/large accounts. Sometimes, however, choosing participants isn’t so straightforward, and you have to develop careful screening criteria to ensure the players are producing results that enable you to realize your goals.[separator type=’transparent’ color=” thickness=’1′ up=” down=”]

Two Kinds of Verbs

It is important to realize that there are often two verbs associated with a well-designed and well-produced set of games. The first verb is the ultimate action that the client who commissioned the event wants to take. The second verb is the verb that is associated with the selection of the frameworks. These may or may not be the same verbs.

To illustrate, let’s one more time refer to Sallee and the VeriSign portfolio event. The ultimate business goal of the customer care projects was to improve the quality of customer care. To serve this goal Sallee needed to prioritize her projects. An online Buy a Feature tournament was the best way to accomplish this goal. And by including as many of her employees as she could in the prioritization process, Sallee also created “global buy-in” for the results of the forums.

Of course, Sallee and her team still needed to implement the selected projects. And while frameworks like Remember the Future and Product Box can help plan projects, and frameworks like Start Your Day and Spider Web can identify hidden requirements, and games like Speed Boat can identify potential risks, ultimately Sallee and her team needed to implement the selected projects.

While this post has been informed by years of using collaboration frameworks to solve business problems, special thanks goes to Jenna Cline from Cisco, as our conversation on the nouns and verbs of collaboration enabled me to think that much more clearly about this topic. Thanks also to Harbinder Kang, also from Cisco, who further challenged these ideas.