PowerPoint Presentation
An unofficial guide to trying to do empirical work
Amy Finkelstein
January 17, 2007
Purpose
Graduate school is well-structured to teach you:
Economics
i.e. What are the interesting and important questions?
Technical skills
i.e. How to answer them
But what about the process of doing (or trying to do) research?
This is a semi-structured list of some tips I wish someone had told me….
Topics
How to come up with a question / find data
Covered in more detail last year
So you’ve got a question… now what?
The focus this year
The bottom line
1. It is hard to do good research
for everyone
2. Research is not a solo process
Talk about your ideas with people early and often
Work in a systematic and organized fashion:
Keep detailed notes on all your thoughts – you’ll forget them quickly
If you decide something will make or break the project, look into it sooner not later (in fact, look into it immediately!!)
This is not easy. They call it “re-search”. No one finds it easy.
Step 1: Coming up with ideas
All (good) research starts with a question that is interesting
Can you explain to others why it’s interesting and exciting?
Your peers and professors
Non economists
Your family
Non-economist friends (if any)
Are you interested in this question?
If you are not interested and excited in your project, how can you possibly expect anyone else to be?
And you will certainly not enjoying working on it for many years!
Some ways to come up with ideas…
Chicago adage
“vote early, vote often”
Think about ideas early, often and always
Write all your ideas down!
Especially why you care
Sources of ideas:
Classes – what are the important unanswered questions?
Seminars – what does the seminar make me think about?
In general do not go to the literature for ideas
Broad survey articles can stimulate ideas
JEL, JEP, Handbook Chapters
Read the newspaper with an eye towards economic questions
Look at the real world, not just the economics literature
Read non-economics non-fiction
Biography, history
Talk to people – economists and non economists
Ideas come at random times
Be sure to write them down whenever you have them
Keep at it
Nothing is worse than staring at a blank wall with no thoughts on what to do….
Seminar example: Bertrand and Mullainthan on CEOs rewarded for luck (corporate finance) Wolfers on are governors rewarded for luck (pol economy)
Baicker and Staiger: how does hopspital getting $$ help recipients– falsification test gone awry or really interesting alternative theory – spillovers and why we would care about them.
Someone else’s paper can be your instruemnt – David Matsa and union spillovers (Holmes)
Newspaper examples:
If you only look at the literature you’ll never find a new question.
If drug prices are low, firms won’t innovate. (is this true? Turns out no info. And why does it matter? Static vs dynamic policy effects. E.g. govt regulating low prices…)
2. Friedman article two days ago: high oil prices support autocratic regimes? (is that true? What determines regime shifts? Are natural resources good for a country in the long run?).
I often excuse myself and leave myself a voice mail message if during dinner something hits me.
Keep at it – there’s no other way.
Easier to come up with ideas when have other things to do.
Always be on the frontier
Interest / importance of question
How convincingly can you answer the question?
Avoid corner solutions
The more interesting / novel the question the less you have to nail it completely. If you are the 10th paper on X, you’d better have the cleanest approach by far.
People / fields have different tastes for where to be
Corner solutions never good
Useful tip #1: fast forward to the end
If you think you have a good idea:
Imagine you came up with a way to answer your question convincingly
I know it’s hard but “fast forward” to the end where you’ve produced a really convincing answer
Now ask yourself:
So what? Why is this interesting?
What would make it more interesting?
May help you modify / fine tune your question….
Useful tip #2: be a compulsive note-taker
So you think you have an idea…
Start a file on it and write down why it’s interesting
You’ll be surprised at how quickly you can lose sight of the forest for the trees!
I always go back to these notes before writing an introduction
Try out your motivation on people (fellow students, faculty members, other friends and family)
Update your notes as you get new ideas or spins from talking with them
Never delete old notes though!
Write down a description of your ideal data / variation
You’ll almost never have it but this should help you focus on what elements are essential as you consider (non-ideal) data
2. So you’ve got a question… now what?
Work in a focused and systematic fashion
What would be the ideal data and variation?
What are the essential data / variation you would need?
Does it look like anything is there?
What’s the relevant theory?
What issues / questions / concerns emerge and how can you deal with them?
Talk continuously with people about what you are doing
As you learn more, they will be able to give new comments and thoughts
Vague so going to give an example
That’s all really vague…
A specific example from my own recent work
Do less salient tax systems produce higher equilibrium tax rates
A caveat: this is selecting on the dependent variable
Many other projects will fail at any given step
The key is to figure that out sooner rather than later…
And keep at it
Vague so going to give an example
How it began…
I asked Jim about his experiences on the tax commission
Subliminal message #1: the more you talk / think / live economics….
Something clicked with my thoughts on driving to NYC
Subliminal message #2: real world experience doesn’t hurt either!
Subliminal message #3: especially if you’re always thinking about economics “out there in the real world”
Vague so going to give an example
What did I do first?
Started a file
Jotted down my idea and why it might be interesting
Mentioned the idea to several friends (that day) and saw if anyone could point out an obvious issue / tell me this topic was uninteresting
Took notes on additional motivations they gave me (e.g. Milton Friedman)
Starting looking for existing data sources
Google scholar on papers on tolls
Searches for trade organizations etc
NB: I did all of this immediately – put the other stuff I was working on aside for a few days
Harder to try to come up with / work on an idea when not inspired
Vague so going to give an example
Is there anything quick and easy to do?
Searching on line found several toll histories
Looked like there was something there
Sent out some haphazard inquiries to any toll facilities whose contact information I could find
Within a week had about 15 facilities and some suggestive evidence
Always good to try to do something “quick and easy” to get a sense of whether it is worth more time
Don’t wait until you have the final / perfect / all-t’s-crossed- data set before starting to look at it
Vague so going to give an example
Took stock
Was there really anything serious / fundamental to be learned from this
Went back to the initial motivation and followed up on readings
Talked to more people
Added to my motivation notes
Thought about how I could systematically collect a data set
Also did another check of whether the data didn’t already exist / there wasn’t some easier way to get it (e.g. would a trade organization do a survey…)
Worth investing in making sure you’re using the optimal data before you spend too much time with your data!!
Vague so going to give an example
As I began to get results
Talked whenever possible to whomever possible about my project
Got more ideas on motivation / links to other literatures
Got another empirical idea (looking at elasticity):
Suggested need for new data (on traffic)… so started in on that
NB: often ideas get refined / evolve and this suggests different or additional avenues to take the research
don’t wait until you think you’re done to get feedback!!!
People brought up several important problems I hadn’t thought of
So I started trying to think about how to deal with them and what additional data or designs I would need…
All this goes into the file!!
My file has a running stream of thoughts and suggestions on
Motivations – related literature / theories
Empirical concerns raised and my thoughts on what to do
Empirical suggestions
Constantly adding to it
Never delete old notes
Sometimes helpful to try to summarize thoughts thus far…
As I began to get results…
I also started writing the paper
Don’t wait until you think you’re “done”
Writing the intro / trying to motivate can stir additional thoughts for theory / empirics
Trying to write down in words your empirical strategy and its assumptions can give you a new perspective on whether they are reasonable / how you could improve them
The data section has to be written some time…
Sometimes in writing it can discover issues (e.g. sample definition)
Two tips for starting to write…
Go back to your file
Should help especially with the introduction
Why did I start this project? It wasn’t really because I was interested in tolls, was it?
Find a well-written published paper to use as a template
Don’t reinvent the wheel. Find a template and start by mimicking it.
Faculty can often help here if you need a suggestion…
Example: canada…
The bottom line re-visited.
Research is not a solo process
Form weekly working groups with your friends to talk about your latest thoughts
Force everyone to talk about at least one idea, no matter how lame they think it is
Think / talk / discuss your ideas or project constantly
As project progresses, will get different kinds of feedback
Keep a file on your thoughts and people’s comments
You’d be amazed how quickly you can forget…
Proceed in a systematic fashion:
If X is essential for the project’s success, look into X now, not later!