Causal Meta-Mediation Analysis: Inferring Dose-Response Function From Summary Statistics of Many Randomized Experiments
(with Xuan Yin, Tianbo Li, and Liangjie Hong)
It is common in the internet industry to use offline-developed algorithms to power online products that contribute to the success of a business. Offline-developed algorithms are guided by offline evaluation metrics, which are often different from online business key performance indicators (KPIs). To maximize business KPIs, it is important to pick a north star among all available offline evaluation metrics. By noting that online products can be measured by online evaluation metrics, the online counterparts of offline evaluation metrics, we decompose the problem into two parts. As the offline A/B test literature works out the first part: counterfactual estimators of offline evaluation metrics that move the same way as their online counterparts, we focus on the second part: causal effects of online evaluation metrics on business KPIs. The north star of offline evaluation metrics should be the one whose online counterpart causes the most significant lift in the business KPI. We model the online evaluation metric as a mediator and formalize its causality with the business KPI as dose-response function (DRF). Our novel approach, causal meta-mediation analysis, leverages summary statistics of many existing randomized experiments to identify, estimate, and test the mediator DRF. It is easy to implement and to scale up, and has many advantages over the literature of mediation analysis and meta-analysis. We demonstrate its effectiveness by simulation and implementation on real data.
Open Science Practices are on the Rise: The State of Social Science (3S) Survey
(with Garret Christensen, Elizabeth Paluck, Nicholas Swanson, David J. Birke, Edward Miguel, and Rebecca Littman)
Companion paper in AER P&P
Has there been meaningful movement toward open science practices within the social sciences in recent years? Discussions about changes in practices such as posting data and pre-registering analyses have been marked by controversy—including controversy over the extent to which change has taken place. This study, based on the State of Social Science (3S) Survey, provides the first comprehensive assessment of awareness of, attitudes towards, perceived norms regarding, and adoption of open science practices within a broadly representative sample of scholars from four major social science disciplines: economics, political science, psychology, and sociology. We observe a steep increase in adoption: as of 2017, over 80% of scholars had used at least one such practice, rising from one quarter a decade earlier. Attitudes toward research transparency are on average similar between older and younger scholars, but the pace of change differs by field and methodology. According with theories of normal science and scientific change, the timing of increases in adoption coincides with technological innovations and institutional policies. Patterns are consistent with most scholars underestimating the trend toward open science in their discipline.
SmartStorage: Automated Storage System with Reinforcement Learning
(with Fengshi Niu)
This paper applies model-based deep reinforcement learning to solve a simplified storage assignment problem. First, we train an LSTM order predictor from a long historical order sequence and use it for state transformation and reward variance reduction. Second, we run an approximate value iteration until convergence. Our algorithm is specifically designed to address the tradeoff between travel-time efficiency and the repositioning costs. Our experiments evaluate this algorithm in a variety of simulated environments with a varying number of products (<= 1000) and different stochastic order processes. In all cases, our algorithm significantly reduces overall storage costs compared to random assignment heuristic. The performance gap between our algorithm and the oracle tabular value iteration with access to latent order probability is shown to be small. Our experiments also show tentative evidence that our algorithm scales up linearly in training time per iteration with respect to the number of products, despite the factorial growth of the permutations.
The Environmental and Economic Consequences of Internalizing Border Spillovers
(with Shaoda Wang)
This paper studies how centralized decision-making can help local governments internalize regional environmental spillovers, and investigates the associated economic and welfare consequences. Utilizing novel firm-level geocoded emission and production panel datasets, and exploiting more than 3000 cases of township mergers in China, we find that as township mergers eliminate the borders between neighboring townships, the negative externalities of polluting firms located on these borders are suddenly internalized by the new jurisdiction. As a result, these firms spend more effort on emission abatement, which leads to lower emissions, as well as lower output and profit levels. Further analysis suggests that household welfare improves with the internalization of border spillovers, as reflected by increased residential land price around the merging borders.
Home City Connection and Bureaucrat Performance: Evidence from China
Do bureaucrats perform better or worse when serving in their home city? By exploiting exogenous variations in city leadership vacancy and the turnovers in the personnel decision-making body, I find that Chinese municipal leaders’ biographical background indeed plays an important role in their governance decisions. Natives, who grew up in the city they serve, would implement policies that lead to a 7% reduction in total tax revenue. Estimates from firm-level data also show a significant drop in tax payment from firms during natives' tenure despite increases in outputs and profits. But further examination suggests that only firms in the home counties of native leaders benefit from the tax breaks. With respect to budgetary policies, native officials exhibit a pro-social tendency, allocating a higher share of municipal budget to education and health care, and a lower share to infrastructure. However, despite the changes in budget composition, real outcomes of public goods deteriorate under the native city leadership. Taken together, my results suggest that social proximity hampers bureaucrat performance and facilitates local favoritism.