October Event Review: “Bridging Data Into Action” panel discussion with Jennifer Berkley Jackson, Steve Chalgren, Tanguy Leborgne, Ajay Nilaver, and Brian Sohmers
by Dan Galatin
October 2013 Event
Jennifer Berkley Jackson, Founder at The Insight Advantage, moderated a lively panel discussion on October 2nd about how to take action based on market data. The panelists were Steve Chalgren, VP of Product Management and Strategy, Arena Solutions;
Tanguy Leborgne, Vice President, Consumer Solutions, Plantronics; Ajay
Nilaver, VP Product Management, Fusion IO; and Brian Sohmers, Executive
Vice President / General Manager, Equilar. The panelists fielded questions on how to filter and evaluate the quality of data, as well as how to use it to make product decisions.
Ms. Jackson began the discussion by asking the panelists what they love and hate about market data. Mr. Leborgne acknowledged that many of us have a love/hate relationship with data. Without data we’re blind, but data can be manipulated. One must be careful which questions one asks and pay attention to where the data is being obtained. Mr. Nilaver admitted that it’s hard to go into new markets and uncharted territory with no existing baseline data about the market – a certain amount of courage is necessary. Mr. Chalgren said we need data to look at where the market is going, but it’s important to understand that the data is often wrong. One needs to filter data and have at least 10 independent customers validate product ideas.
The panelists then discussed the challenges of bringing data into the product planning process. Mr. Sohmers said there were lots of pitfalls in doing so; one might ask the wrong questions or talk to the wrong customers. He was a strong advocate of the “buy-a-feature” technique: assemble a customer advisory board and ask them to “buy” features that are being considered for inclusion in the product. In addition to the quantitative ratings that result, one also obtains qualitative information from the participants’ discussions of the features.
Another pitfall that Mr. Leborgne mentioned was the tendency to draw conclusions from the data too early. An example is to assume that customers will try a new product just because its features address shortcomings they found in current products. One should ask more probing questions as a follow-up (“Why?”). Mr. Chalgren said that even in the cloud software business, one doesn’t have access to all information about user behavior, and even if one did, the data might be “lumpy” (for example, customers might be using a feature only at the end of a fiscal quarter). One may draw multiple conclusions from the same data.
Ms. Jackson then directed the discussion to the topic of how to source data. Mr. Nilaver recommended that one look at many sources of data, including roadmaps from one’s vendors throughout the product ecosystem and supply chain. One should talk to OEM and end customers, as well as pre-sales engineers. Mr. Leborgne said that one should concentrate on primary sources and obtaining data oneself (at least 70% of the time). PMs should not confuse what they do with what sales does: PMs should ask more questions what they discover problems, in order to turn the answers into product insights.
On the subject of third-party sources of information, Mr. Chalgren indicated that industry analysts will typically be behind the curve in a product space that’s truly innovative, but they can help one understand the competition or whether one’s product direction is truly unique. Sleuthing on LinkedIn and Google can also be useful. Mr. Leborgne recommended establishing relationships with analysts to use them as a sounding board for ideas. But Mr. Sohmers cautioned that one should be careful to understand the motivations of people that one’s trying to obtain information from.
When asked for final advice on how to get credible market information, Mr. Leborgne recommended getting data from multiple sources and sharing it with other PMs to help evaluate the data critically – this helps avoid drawing conclusions too quickly. Mr. Chalgren advised against leading conversations with one’s sources, and obtain “at least 10, better 20” sources to back up a conclusion. Mr. Nilaver advocated providing free products or services to customers in exchange for information that could help shape product decisions.
Dan Galatin has over 20 years of combined experience in product management and software engineering. He is currently a Senior Product Manager at Keynote Systems and can be contacted at firstname.lastname@example.org.