Traditional marketing tools, such as surveys and transactional data, are widely used to monitor ongoing marketing activities and to course-correct within a given marketing strategy. However, making decisions about changes in marketing strategy requires predicting how consumers will behave in a different market context than the one that currently exists. This course covers the use of experimental methods to quantify the causal effect of marketing decisions.
Experimental methods have been used since the early days of marketing in settings such as retail test-markets and direct mail. In recent years, technological change, particularly the proliferation of online A/B testing, has fundamentally altered the methods and benefits of marketing experimentation. This course will cover the fundamentals of conducting marketing experiments and students will learn how to incorporate experimental results into managerial decision making. In particular, we will discuss:
1. The kinds of decisions for which experimental tests are most beneficial compared to alternative approaches.
2. How to design experiments, taking into account factors including cost, sample size and effective treatment rates, analytic complexity, modeling and decision needs, potential information leakage and other sources of bias, and customer reaction.
3. How to statistically analyze experimental results to draw valid conclusions that will generalize reliably to the decisions being made, from basic tools to more advanced methods for complex experimental settings
4. How to incorporate experimental results into decision making, including issues in buy-in for experimentation, identifying and managing threats to internal and external validity, and incorporating experimentation into long-term knowledge-building.
The course will use a combination of lectures, case studies, hands-on exercises and an exam. Students will learn the tools needed to implement experimental methods in practice, from lab and survey-based experiments and concept tests to in-store, online and direct-communication field testing. We will discuss cases in which experimentation changed the way organizations made decisions, drawing on examples from advertising, online sales, consumer packaged-goods, consumer finance, fundraising and government.
