Traditional marketing tools, such as surveys and transactional data, are widely used to monitor ongoing marketing activities and to course-correct within a given marketing strategy. However, making decisions about changes in marketing strategy requires predicting how consumers will behave in a different market context than the one that currently exists. This course covers the use of experimental methods to quantify the causal effect of marketing decisions.

Experimental methods have been used since the early days of marketing in settings such as retail test-markets and direct mail. In recent years, technological change, particularly the proliferation of online A/B testing, has fundamentally altered the methods and benefits of marketing experimentation. This course will cover the fundamentals of conducting marketing experiments and students will learn how to incorporate experimental results into managerial decision making. In particular, we will discuss:

1. The kinds of decisions for which experimental tests are most beneficial compared to alternative approaches.

2. How to design experiments, taking into account factors including cost, sample size and effective treatment rates, analytic complexity, modeling and decision needs, potential information leakage and other sources of bias, and customer reaction.

3. How to statistically analyze experimental results to draw valid conclusions that will generalize reliably to the decisions being made, from basic tools to more advanced methods for complex experimental settings

4. How to incorporate experimental results into decision making, including issues in buy-in for experimentation, identifying and managing threats to internal and external validity, and incorporating experimentation into long-term knowledge-building.

The course will use a combination of lectures, case studies, hands-on exercises and an exam. Students will learn the tools needed to implement experimental methods in practice, from lab and survey-based experiments and concept tests to in-store, online and direct-communication field testing. We will discuss cases in which experimentation changed the way organizations made decisions, drawing on examples from advertising, online sales, consumer packaged-goods, consumer finance, fundraising and government.

Business 37000 (Marketing Strategy) and 41000/41100 (Statistics) or equivalent are required (strict), but can be taken concurrently. We will use statistical significance testing and regression analysis throughout the course. While we will review the basics of using these methods in our context, prior experience with statistical data analysis is important. Students who have not taken 41000 or 41100 must obtain the instructor's approval to enroll. Students who are not enrolled in one of the Booth programs must obtain permission from the instructor to enroll in this class. No auditors.

There is no textbook for this course. This course covers rapidly evolving methods and practices, and synthesizes from a wide range of sources, including academic research, policy experiments and business practices.