Problem 1: Distributing your future riches
Many years from now, with a successful career behind you, you find yourself in a position to begin donating your considerable fortune to charitable causes. A non-profit called EduX approaches you to ask for a considerable donation. Their aim is to increase youth literacy with 1:1 after-school mentoring for children in first grade. The founder writes you a personal email that, among various moving anecdotes and appeals to your ego, indicates that “everyone’s saying EduX’s data is the strongest, the very best, showing HUGE — just the biggest — gains on children’s scores on standardized reading tests taken at the end of first grade.” Unperturbed by the founder’s anecdotes and bluster, you begin corresponding with the founder to understand the nature of this data about EduX’s impact.
Compose a reply directed to the founder — in language or with examples s/he would understand – – to each of the below claims the founder might make to you. Indicate why you are dissatisfied or satisfied with her/his answer.
a. “We know our program works because the students in our program are much better readers on their end-of-year exams at the end of first grade than they are on their end-of-year exams at the end of Kindergarten.” [2 points]
Copyright By PowCoder代写 加微信 powcoder
b. “When we compare the students in our program to students not in our program, the students in our program are much better readers.” [2 points]
c. “I see your point, but we’ve controlled for many, many things, such as teacher quality, what school students are in, etc.” [2 points]
d. “I have to admit, I’ve never got questions like this before. It sounds like you want us to run an experiment. But I don’t think that will work, because if we randomly assign a first grader to be eligible to our program, we can’t exactly force them to enroll.” [2 points]
e. “I see your point. But, we know parents also play a big role in whether students read well, and I don’t think your experiment would account for that since we don’t really have good measures of parental involvement.” [2 points]
f. “Ok, I’d be willing to start an experiment. I have a daughter and a son (twins) who are both in different first grade classes in the same school. I’ll randomly select one class and invite the entire class to take part in our program. Then we can see how that class versus the other does at the end of the year. Sound good?” [2 points]
g. “Great. While I have you, one more thing. Yale recently did a randomized experiment on a similar program. Not only did they find it was effective, they found that the effects were bigger on students who for whatever reason met with mentors in wealthier areas of their cities, relative to the effects on students who met with mentors in less wealthy areas of their cities. That made me think we should be busing students in poor areas into rich areas so that they can meet with their mentors there, as the Yale study shows that meeting with mentors in rich areas is more effective. Would you be willing to give us a gift to fund that
transportation? It’s much cheaper than the cost of a mentor, but the Yale research suggests it would triple our impact.” [2 points]
Problem 2: Driver’s License Tests
Several randomized experiments have assessed the effects of drivers’ training classes on the likelihood that a student will be involved in a traffic accident or receive a ticket for a moving violation. Suppose you want to estimate the impact of these classes on traffic accidents — that is, whether taking the classes reduces traffic accidents and tickets. A researcher on your team presents you the results of an experiment they have done they argue estimates this impact.
Suppose the researcher has done a randomized experiment. A complication arises because students who take drivers’ training courses typically obtain their licenses faster than students who do not take a course. (The reason is unknown but may reflect the fact that those who take the training are better prepared for the licensing examination.) If students in the control group start driving much later on average (because they had their license later), the number of students who have an accident or receive a ticket could well turn out to be higher in the treatment group.
Question a) The researcher observes a clearly statistically significant, positive ATE estimate of taking the classes on the outcome defined as “the number of accidents that occur within 3 years of first obtaining a driver’s license.” The researcher argues that this data shows accidents increase as a result of taking the classes. Do you agree? Why? (Note: I wrote “clearly statistically significant” to indicate that you should ignore the possibility that this positive estimate arose by chance.) [6 points]
Question b) Same question as above, but if the outcome measure were “the number of accidents per mile of driving within three years of getting a license” (and we were still observing a significant, positive ATE estimate)? [6 points]
Question c) Suppose researchers were to instead measure the outcome “the number of accidents that occurred over a period of three years starting the moment at which students were randomly assigned to be trained or not.” Assume we are still observing a significant, positive ATE on the outcome defined this way. Would you then agree with the claim “the number of accidents that occur within 3 years of when students would take these classes is higher as a result of taking them”? (Answer “yes” or “no”.) [2 points]
Question d) If your answer is “yes” in part c), discuss the implication of this claim on the effect of the program on the quality of drivers. If your answer is “no” in part c), explain your reasoning and provide the right claim. [8 points]
Problem 3: Estimating price elasticities [26 points]
Companies are often interested in estimating the price elasticities of their products. For example, Apple wants to know how its sales will change if it decides to increase the price of iPhone by 5%. In case you are not familiar with the concept: Price elasticity, which is a shorthand for price
elasticity of demand, is a measure that show the responsiveness of quantity demanded of a good or service to a change in its price when nothing but the price changes.
Uber is also interested in estimating price elasticities. This knowledge is essential for setting up the optimal surge price multiplier as a function of changes in demand and supply conditions. To study this question, Uber works with a few academics on a research project. For this problem, you should read the abstract and page 3 and page 4 in the introduction. Also, Figures 1, 4, and 5 should be helpful for understanding their research design. Note that I only expect you to understand the regression discontinuity design that they use to estimate demand elasticity. I do NOT expect you to understand how estimation of demand elasticities can be translated into consumer surplus.
a. What is the outcome in the study? What is the “treatment”? What is the discontinuity? [6 points]
b. Assume the RD was not available, and someone did a simple observational study comparing individuals who happen to get treatment versus those who do not for non- random reasons. What’s an alternative story for a simple association between the outcome and this non-random version of the treatment like this? [4 points]
c. What makes this RD evidence more convincing than this hypothetical observational study from part (b)? [8 points]
d. The RDD requires a critical assumption: no sorting at the cutoff(s). Do you think the assumption is justified in this scenario? Why? [8 points]
Problem 4: Capturing global customers with machine translation
As many e-commerce platforms try to grow their business beyond borders, they realize that language is one key barrier in international trade. For example, buyers in Mexico may not understand listing titles and descriptions in English, and therefore do not buy from U.S. sellers even though they are interested in the sellers’ products. To mitigate language barrier across borders, an e-commerce platform adopts machine translation (MT) to automatically translate listings for buyers. The platform first adopts MT from English to Spanish for buyers in Spanish-speaking Latin American countries. In particular, buyers from these countries will see listing titles and descriptions in Spanish, instead of in English.
Question a). The platform wants to evaluate the effect of the introduction of MT on international trade via a difference-in-difference (DiD) approach. The DiD approach compares intertemporal changes in exports from the U.S. to Spanish-speaking Latin American countries to intertemporal changes in exports from the U.S. to other countries. Use “data_DID.csv” to perform the DiD analysis. The outcome variable should be “log_revenue”. Report the line of code that you used for running the regression. Interpret the estimated coefficients (except the fixed effects and the constant). [8 points]
• “buyer_country” is the ID of the buyer’s country.
• “treatment” is the treatment status dummy: =1 for Spanish-speaking Latin American countries; =0 for other countries
• “norm_m” is the same as in part a).
• “post”: =1 if “norm_m”>=0; “post”=0 if “norm_m”<0
• “log_unit”: logarithm of U.S. exports to a country, where exports is measured in
units/quantity.
• “log_revenue”: logarithm of U.S. exports to a country, where exports is measured in
Question b). One data scientist in the company argues that we should control for “log_unit” in the regression because changes in revenue should obviously be correlated with changes in the number of units sold. Do you agree with this view? Why? [10 points]
Question c). What is an important assumption in this DiD approach? Do you think the assumption is justified in this scenario? Why? (You should use both “data_graph.csv” and your reasoning to answer this question) [10 points]
• “norm_m” is the normalized month, so that “0” means the first month that MT was introduced. “-1” means the month before MT was introduced.
• “norm_revenue_treated” is the average monthly U.S. exports in dollar terms to Spanish- speaking Latin American countries.
• “norm_revenue_control” is the (normalized) average monthly U.S. exports in dollar terms to other countries.
Problem 5: Are small samples better?
Randomized experiments in marketing are (in)famous for finding very large effects in relatively small samples. Marketers often defend their use of small sample sizes in their experiments with logic like the following: “If my sample size were really large, anything would be statistically significant! Therefore, it’s better to use smaller sample sizes since we’ll only get significant results when our treatment has a really big impact.” Do you agree with this logic? Why?
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com