Doug Stark, Senior Director of Product Management at Medefinance spoke at the April 6, 2005 meeting of the SVPMA. Mr. Stark spoke about Running Successful User Surveys: Surprises and Lessons Learned. The speaker shared his recent experience conducting a user survey for his company in a presentation that was both candid and insightful.

Medefinance provides hosted analytics to hospitals to assist them in improving their financial performance. When Doug joined the company, he found that executive stakeholders were giving feedback, but very little information was being provided from actual users, thus, the genesis of a user survey. Doug considered outsourcing the survey, but found it would take three to four months to complete. Instead, he opted to conduct the survey himself to finish it in eight weeks and ahead of the upcoming user group meeting. The speaker commented that he would still consider outsourcing future surveys.

Working from the ground up, Doug researched surveys. He learned that he could expect a 25% - 30% response rate and needed a minimum sample size of 50 respondents. He set the goals for the survey and then led a cross-functional team to craft the questions. Doug had wanted to limit the survey to 25 questions, but ultimately settled on a four page survey with 35 total questions.

The speaker looked at a number of survey tools on the market and found Survey Monkey best met the needs: it was easy to use, low cost, and simple to brand. What Survey Monkey lacked was strong analytics for post survey examination. Doug completed much of the advanced analysis offline.

In order to get a good response rate, Doug sought verbal approval from all executive stakeholders at the target companies. He then sent the survey with a cover email from Medefinance's CEO to convey its importance. After one week, Medefinance had received responses from 25% of the group. A reminder email drove an additional 10% of users to respond for a total response rate of 35%. One lesson learned was to include an expected response date in the invitation.

The questions were a combination of quantitative and qualitative. The qualitative answers provided supporting detail for the quantitative responses, especially outliers. One of the most valuable questions turned out to be "Please describe how Medefinance can improve the service they provide?"

Doug shared key learnings with the company as they emerged, which made it easier to build consensus around the action plan. The speaker emphasized the need to take action on the data, and further, to take action in a timely fashion. You want the user to see the impact of sharing their opinion so they will continue to participate in future user research. You can view or download Doug Stark's presentation from http://www.svpma.org/etc/dstark.pdf

About the Speaker
As Senior Director, Product Management at MedeFinance, Doug Stark is responsible for managing the company's suite of products, including overseeing strategic direction, product and customer requirements, pricing, and deployment. Doug has over 13 years of experience defining, developing and launching products and services in both start-up and Fortune 500 companies including Accenture, IBM and Manugistics.