How to Understand and Influence Behavior Using Data Science

We talked with Florent Buisson, consultant and author, about his new book 'Behavioral Data Analysis with R & Python' (O’Reilly) and his perspective on data science and teaching.

Florent, you have over 10 years of experience in business analytics. You started and led for four years the behavioral science team of Allstate Insurance Company. Before that, you have published in peer-reviewed academic journals and you hold a Ph.D. in behavioral economics. In your view, what would you say are the biggest cultural differences between industry and academia when it comes to data analytics?

Having had the chance to work in very different environments (academia, consulting, industry), I have learned that regardless of your title, you only ever have one job: get things done. And regardless of your job title, the thing to get done is almost never data analysis per se. In academia, it’s usually articles and citations, and that means using rigorous methodologies that will be accepted by your peers. In the industry on the other hand, it’s driving business results. Because everyone is initially trained at university, junior analysts often try to maintain academic rigor (e.g., sticking with a 5% threshold for statistical significance), even if it makes their analyses inconclusive and therefore unhelpful. More seasoned analysts figure out “what you can get away with” (in the words of a UX researcher I once chat with) and focus more on building usable “data products” even if they’re not rigorous. Successful analysts also get much more worried with how analyses will be used than how they were run. For example, I have learned to never give preliminary results to my partners because they’ll just treat them as true and run with them if they like them, and then things get very uncomfortable if the final results go in the opposite direction.

How important do you think are causal questions for business decision-making?

I believe that causal questions are essential for business decision making. Business decisions implicitly rely on causal counterfactuals: what matters is not whether profits were higher when we did action A in the past (correlation) but whether profits would be higher if we did action A in the future (causation). Once you start looking for causality, you see it in many of the most successful business trends. It’s the secret ingredient that makes A/B testing so powerful, and it’s all over the place in the Lean Startup framework. On the negative side, one of the main causes of failure for analytics projects is trying to answer with predictive analytics a business question that is truly causal. Imagine for example that you had a crystal ball that would give you accurately the life-time value (LTV) of any of your customers. That sounds fantastic until you realize that it’s actually pointless: what matters is not how high a customer’s LTV is under business as usual, but how to increase it. Using predictive tools here will only make your business shower perks on customers who would have had a high LTV anyway.

Your new book “Behavioral Data Analysis with R & Python” (O’Reilly) came out this summer. Can you tell us a bit more about the role causal inference plays in your pedagogical approach?

Understanding and changing behaviors is one of the main goals of analytics in business. Unfortunately, behavioral variables are riddled with confounding and other issues that boil down to not understanding or measuring adequately causality. In the book, I develop what I call the “Causal-Behavioral Framework for data analysis” because I have found that thinking in causal and behavioral terms are mutually reinforcing approaches. If you take a rigorously causal approach but your variables don’t truly reflect behaviors, you won’t get the best results. Neither will you if your variables are neatly behavioral but you don’t account for confounding. A causal approach also vastly simplifies the analysis of experimental data. In the old days when computations were done by hand, it made sense to rely on strong statistical assumptions and use formulas each tailored to a specific design. But good luck calculating the power of a matching design with clustered subjects! With a unified causal approach, we can make the best of the computational power now at our fingertips: I show in the book how to analyze any data, either historical or experimental, with only thoughtfully crafted regressions and Bootstrap.

In your book, you frequently make use of causal diagrams to illustrate examples. What are the advantages of the graphical causal modeling approach from your perspective?

In data analyses, a lot of the business logic and reasoning is so to speak “under the hood”. Business partners are often uncomfortable interpreting code or equations, and showing a causal diagram provides a more accessible common language. It exposes business assumptions (e.g., “young people buy online”) and previously discovered mechanisms (e.g., “the effect of age on online behaviors is mediated by comfort with digital technologies”), which leads to more productive conversations. It also makes complex methods such as Instrumental Variables (IVs), moderation and mediation much more intuitive. When I was introduced to IVs during my master’s in econometrics, I really struggled to understand them from a purely statistical perspective. In the book I explain them with causal diagrams and I feel it makes them incomparably simpler to grasp.

What would you say are the biggest trends in data science right now that students should be aware of?

I think we’re reaching the end of the Wild West era of data science when, to keep it simple, physics PhDs with no business experience were tasked with analyzing a dataset for the very first time. Today’s students will enter a more mature data science world where most analytical low hanging fruits are gone, at the same time as the number of dedicated degrees has increased and tools are becoming more and more automated. Therefore, I think that the trend towards specialization that we can already partially observe will accelerate. Just being able to run XGBoost in Python will not be enough to be successful and I would encourage students to add a “minor” to their profile: a technical expertise such as deep learning or causal inference, or a transversal one such as behavioral science, business consulting or product management.

Thank you for the interview, Florent!

Tags

Share