To produce a calculation of the positive return on investment of state-funded tourism promotion, Longwoods uses a tool it developed called “R.O.EYE,” a play on the common investment acronym for return on investment. Using this methodology, Longwoods estimated in the 2016 report, “Michigan 2015 Tourism Advertising Evaluation and Image Study,” that Pure Michigan’s out-ofstate advertising campaign generated $7.67 in revenue to state coffers for every $1.00 spent, an astonishing 667 percent return in just one year. Longwoods refuses to explicitly demonstrate how it obtains its return on investment figures, claiming their method is proprietary and leaving taxpayers and policymakers with a methodological “black box.”
Scientific studies assert to test a hypothesis, detail the methodology used to test this hypothesis, identify required assumptions and shortcomings and list all data sources. This method produces results that can be replicated and reviewed by others. Outcomes that are replicated by third parties are more robust than those that are generated by just one party (especially considering the interests one party may have in producing a particular outcome). Unfortunately, Longwoods’ reports fail to meet these common scientific standards, because they do not fully disclose their methodology or sources, making replication impossible.
Longwoods’ report on the impact of spending on Pure Michigan’s 2015 tourism promotion campaign only discloses the following information about its methodology: 1) It uses an online survey of a representative sample of American adult travelers; 2) The survey asks 4,000 participants about their opinion of Michigan as a place to vacation, their history of vacationing in Michigan and whether they are familiar with Pure Michigan’s advertisements; 3) Its trademarked R.O.EYE model deploys an experimental design and estimates how many more people vacationed in Michigan because of the Pure Michigan ads.
To its credit, Longwoods International has provided additional information when asked, such as the company’s treatment of “response bias” and the source of certain data used to derive its calculations. And Bill Siegel of Longwoods has always graciously fielded questions from author LaFaive. But an explicit derivation of the math behind the return on investment claims has never been provided, as Longwoods maintains its methodology is proprietary, and thus not open to public scrutiny, even though it is used explicitly to guide public policy decision making.