Political Science Research Design Handbook
This packet is to provide students with guidance and expectations for
completing a research design in political science at
You are then correct in noting, "Gee, this is important if I want to consider myself a political scientist. I had better work hard to master this!"
And yet for many of you, the research design will at first seem foreign, if not aversive. Many students are attracted to political science because they like history or current events; a secondary attraction is that as a social science, we seem removed from the hard sciences and mathematics. Then we hit you with a research design where we expect you to think and act scientifically and even mathematically. Generations before you have successfully completed research designs, and we are confident that you can, too.
This packet is our attempt to provide something of a road map for your
mastery. We have compiled it jointly, after much discussion amongst ourselves
(as well as with students who have been through senior seminar). By that time,
you will have written at least three of these designs, and you will be able to
help us improve this packet for future generations of Truman political
scientists.
back to top
B. Research Design Stages
Some political scientists study voting behavior, attempting to explain why people vote, and for whom they vote. Some study bureaucratic norms, others state debt financing, and others the conditions for the emergence of democratic parties and stability in formerly communist regimes. You probably have an area of interest that has drawn you to the study of politics. Hopefully you have questions that you want answered, or ideas about how things work in the political world.
Once attracted to an area, we tend to read all we can about that area to try to understand what has occurred, and why. Surely you have engaged in speculation and guessing based on hunches, whether it be about whether Zaire will fragment into five pieces, or whether Speaker Hastert can hold the conservatives in line to pass legislation, or whether Justices O'Connor and Ginsburg will vote together in sex discrimination cases -- or will the fact that they are females of a similar generation be outweighed by the fact one was appointed by a conservative Republican and the other by a moderate Democrat? As we read more about an area, we develop theories about how the (political) world works.
It is the research design that allows us to test theories to explain the
world and which might even let us predict successfully future political
phenomena.
Step 1: Topic Selection/ Problem Statement
"In its simplest terms a research design should pose a question you can answer with evidence that you can gather in the time available to you." (White, 404)
The research design should begin with you identifying some political phenomenon of interest. Good topics often come from puzzling over something -- reading or hearing about something that you find surprising, and you want to find if the world really works that way, or perhaps why it works that way. While humans are great at explaining away phenomenon which don't seem to fit with their world view, a good social scientist sees an opportunity to investigate further.
It is common at this preliminary stage for students to run into problems of two different sorts. Perhaps the most common problem we encounter is topics which are overly broad or too abstract to be meaningfully discussed or examined in a semester long project. A second common problem is that some topics have been beaten to death; there is nothing new to add. In either of these cases, your professor will work with you, and browbeat if necessary, to get you to sharpen your focus. The burden lies on you, however, to select a topic and justify it as worthy for a research design.
There are several ways to develop topics.
One type of study extends previous literature. So perhaps a study was conducted 20 years
ago, and you have some reason to think that behavior has changed since (money
in campaigns has increased: how does that affect voting rates?)? Another type of study identifies a gap in
existing literature and attempts to fill it.
A third type of study notes that there are competing explanations for
the same phenomena, and then devises a test of both theories in the same study. That is what Whitfield and Evans do in “Political
Culture versus Rational Choice: Explaining Responses to Transition in the Czech
Republc and
When you have identified a likely topic, consider the following questions. You should be able to answer them affirmatively.
This is the step that starts to bridge into the literature review. A starting point for literature relating to your topic might be JSTOR (with limitations noted, below); Parker reads relevant portions of introductory text books on the topic to see what literature is commonly cited, and then reads a couple of those studies to see what literature they cite. Keep in mind you are trying to participate in a scholarly debate or conversation, so you need to be a bit of an archeologist and trace that debate back a little bit. Not all of your excavation will go into your paper, but it will help you to know your topic, and how your topic ties into other materials.
One common misunderstanding is that you need to find literature exactly on your topic, such as "voting behavior of 18-21 year olds." We do not expect you to find something exactly on topic (after all you are adding to literature). Secondly, think of your topic more generally, such as “voting behavior.” Surely you can find studies on voter turnout more generally, and the variables from these might be adapted to your own study that is focused more narrowly on 18-21 year olds.
Our task then is to identify an interesting and engaging topic on which there has been some work to serve as a base. Use your professor as a sounding board. Remember that ultimately we will be evaluating your project. The selection of an appropriate topic can make or break this project. When you have completed this first step, you should have a statement of what you intend to investigate, and you should have full citations of 5-10 sources relating to your topic.
A well crafted statement often need be no longer than three to four sentences. One medium sized paragraph should be enough to state what it is that you intend to investigate.
"Having presented the general purpose of your study, you should then bring the reader up to date on the previous research in the area, pointing to general agreements and disagreements among previous researchers." (Babbie, A11)
Asnoted above, the problem statement merges into the literature review. After selecting a problem to investigate, you
need to read
As you read literature, pay attention to a couple of specific things.
· How do various authors treat their topic? do they all agree on the nature of the problem being studied? (if so, its conventional to accept the convention; if not, perhaps you have a disagreement in the literature that you can explore further).
· Be thinking ahead for your own study: what did these authors hypothesize? what variables did they employ? how did they measure the variables? what methods were used -- how did these authors gather data? how did they analyze the data? with what results in that study, and with what significance for the discipline of political science?
· Look to the bibliographies of your literature for leads on studies that your research has not yet uncovered. This is just good detective work of building a case that you have examined all the evidence in order to frame and answer your question.
After this onerous task of reading broadly and then more deeply in your chosen area, you are ready to write up what you have found. A literature review should present major findings and controversies that remain in the area under investigation.
First, a word of what NOT to do: do not write an annotated
bibliography, which presents the sources one at a time and summarizes the
articles. Instead, you want to integrate and synthesize the works you have
read. Discuss the literature based on the dimensions of the problem that you
are investigating. Pick up a journal and
see how these published authors do it. For
instance, consider “The Political Economy of IMF Lending in
If you are investigating an ongoing controversy, you might organize the information into opposing camps, and highlight not only the disagreements in conclusions, but also in assumptions, data, and methods.
Maybe you want to see if a finding in one area is applicable to other areas,
or is limited to the cases the previous authors chose to examine. For instance,
you might relate voting studies from one geographic region (say, 10
congressional districts in the U.S) will carry over to another region (say, the
provinces of
Here is an important point to note: you may not find material exactly on your topic. Fine. Find related studies and findings. So if you cannot find studies on the effects of splinter parties in mobilizing voters in formerly communist lesser developed countries, you surely can find studies on the role of parties in mobilization and literature on turnout (and associated variables) in emerging democracies. Again, your job is both to tell what is known and what is not known, but simply speculated, or theorized, about.
Perhaps you'd like to find the impact of some state policy; you would need
to discuss the general literature on state policy making
As you can see, we are moving directly toward the third major component of your research design, the hypotheses. Before we go on, review to make sure that you are doing what is asked, and also avoiding some common pitfalls.
DO:
· present the basic theory / theories in this field.
· attempt to be exhaustive; this means thinking of all related angles.
· make sure you get the very latest research included -- for instance, in many areas it would be common to cite literature from the last six months.
· organize the literature to provide the contours of the field.
· use names and dates of authors you are using.
· paraphrase or use quotes.
· Look at examples. Journals can be a good source for identifying what a lit review is to look like.
· Make sure the articles you are examining are research articles, and not editorials or book reviews.
·
use reference material available to you. (Many
of you will have used
DO NO'T:
·
think that you have to find something
·
plagiarize. This can be done in numerous ways,
purposefully or accidentally. It is a serious infraction on academic integrity
and will be treated as such. Three examples are drawn from Babbie (A-11):
"You cannot use another writer's exact words without using quotation marks and giving a complete citation, which indicates the source of the quotation such that your reader could locate the quotation in the original context."
"It is also not acceptable to edit or paraphrase another's words and present the revised version as your own work."
"Finally, it is not even acceptable to present another's ideas as your own -- even if you use totally different words to express those ideas."
Finally: you will want to consult with your professor if it
has crossed your mind to use a paper that you have written for another class.
It is the attitude of at least one of us that this is not acceptable. While it
is good for students to have a substantive interest that they pursue in more
than one paper, this is to be distinguished from the scenario of submitting in
two classes the same paper. When in doubt, (a) err on the cautious side, and
(b) talk with your professors.
Step 3: Hypothesis / Hypotheses
Another methods text reinforces this: "A tentative assertion linking
two or more phenomena, subject to testing and proof. In political analysis, the
most common and useful hypotheses are those that assert that two or more things
tend to be associated with each other in a specified manner." (White, Political Analysis)
The problem statement and literature review should be logically connected to the hypotheses.
Your literature review discusses the theories in the field (is the IMF a bad policy maker, or a bad policy enforcer? Is consumer confidence a function of objective economic conditions, or media reporting on presidential activities?) and it reports earlier findings (which theories seems to have support, and, perhaps derive theory from earlier findings).
Your hypothesis or hypotheses then follow from the theoretical discussion that you have carried out. If you have not reviewed all the relevant literature, you may miss something relevant. If you have not understood the literature, you may misform your hypotheses. If you have not put any time into understanding the literature, but have assumed a relationship ('everyone knows that...'), you are setting yourself up for inadequate hypotheses explication.
Your hypotheses should
As you can see, a hypothesis is a cross between an explanation and a guess. It
is more than a guess, because it has been informed by theory and all that
reading you have done already. It is less than an explanation because you have
yet to test it. At this point, we are trying to get you to think in terms of
the relationship that Y = F (X), which we say as "Y is a function of
X." Y, as your dependent variable, depends upon, or is a function of, your
independent variable, X. If you cannot articulate such a relationship, you are
not ready to proceed.
This requires practice. Usually we are moving from general expectations or near
hunches to formulating tight, straightforward hypotheses. For instance, you
might be reading Judge Robert Bork's book The Tempting of
As this stands, it is not a hypothesis, but it surely contains the seeds of one or more. First, strip away value statements or other statements that are not germane or not testable. From sentence one, remove "unfortunately." From sentence two, we can never know if Bork would have been confirmed under different standards. Sentence three presents the fodder for a different study -- whether groups worked together; here, its not important. From sentence four, words such as "distorted" and "misconstrued" are not important for this study (thought they might be if one were to compare press coverage with his own writings, for instance). Sentence five again is value laden with the term "rightful place", and sentence six uses the term "worse."
If we strip out these terms, we can construct the following assertion:
Nominees prior to Judge Bork were measured by their character and competence.
Judge Bork was measured on his (perceived) constitutional views. Nominees
subsequent to Judge Bork have also been measured on their constitutional views.
To turn this into hypotheses might be as follows. The first hypothesis would be
something like: "Prior to the 1987 hearings on Judge Bork, Supreme Court
nomination hearings focused on the nominee's competency." A second
hypothesis could be "Since Judge Bork, Supreme Court nomination hearings
have focused on the nominee's constitutional views." (In fact, such work has been done. Ayo Ogundele and Linda Camp Keith, conclude “that the
extra focus on the judicial philosophies of Supreme Court nominees by the
Judiciary Committee began earlier, with the first Rehnquist nomination, and
that the Bork nomination simply continued this process. Additionally, we find
that the level of constitutional questioning is significantly affected by the
individual characteristics of the nominees (qualification and political
closeness to the president) and one element of the political environment--the
president's fourth year in office.” see Research Note: "A
Another example illustrates how you can move from more general to more specific, testable statements. In "Electoral Systems, Party Competition, and Strength of Partisan Attachment: Evidence from three Countries," The Journal of Politics 56(4):991-1007 (1994), Bower, Lanoue, and Savoie state
"In sum, we expect that voters' loyalty
is, in part, dependent on the electoral choices they are offered in their
constituency."
Now, the authors recognize that this statement is too vague to count as a
meaningful hypothesis. For instance, assess it according to the general form
that
"Specifically, more extreme opposition parties should polarize voters, causing them to develop an even stronger attachment to their party of choice in the face of a more serious perceived threat."
While the authors do not identify this as their hypothesis, it clearly
is; again, assess it by the sample given above from
If your hypotheses do not have the characteristics that are listed
above, you are in a quandary when someone asks you, "how do you
know?" This is social science. We do not care what you think about
something, if it is merely opinion, or tied into one book you read once. We do
care that you say "given this previous work, I would expect this to
happen, and here is how I can find out if my expectations are accurate."
Step 4: Conceptualization / Operationalization
Having formulated a hypothesis (or two or three), your next step is to move from a level of abstract concepts to concrete indicators. In essence, you want to tell what would count as evidence for or against your hypotheses. This is often hard work, requiring thinking about logical possibilities, and problems with them. What are conditions that would need to be present for you to agree that there is a country is a democracy? What are the indicators of an "extreme opposition party"? Of "stronger partisan attachment"?
Consider again the example about Supreme Court nomination hearings. The researcher is essentially testing whether some event (the Bork hearings) marked a change in the confirmation process. In variable language, the dependent variable "hearing focus" (whether a senate confirmation hearing focuses on competency or constitutionalism) depends upon whether the hearing was pre- or post- 1987. Here, the job of telling whether a hearing was pre- or post 1987 would be easy.
What about the dependent variable, "hearing focus"? Two values for this variable have already been mentioned, those of "competency" and "constitutionalism." But how do we know whether a hearing focuses on constitutional issues or views, or the competency of the nominee? The researchers would have to tell us what are indicators of these variable values.
EXAMPLE: They would need to say that they would read the transcripts for all the hearings, and code the question as predominantly constitutional oriented or predominantly competent oriented. They would further have to say something like, "A question will be classified as constitutionally oriented if it asks the nominee's views of a specific case, or a hypothetical set of facts, or about the correct way to interpret the constitution."
The key here is that other researchers should be able to replicate -- or criticize -- your study: they should look at the same phenomena and classify it in the same way -- here agreeing what is a constitutionally oriented question, and what is a character oriented question.
The focus at this stage is to be very concrete: you are providing a road map. Your reader should know what you have identified as your dependent and as you independent variables; what different values these variables may take on. And you should be able to hand your research design to someone else who would be able to go and gather exactly the same data that you would gather. For instance, consider what variables might have the following values: (a) Yes/No; (b) Very Liberal, Liberal, Moderate, Conservative, Very Conservative; (c) High School Or Less, High School, Some College, College Graduate, Graduate Studies, Advanced Degree; (d) $0-$10,000; $10,001-$25,000; $25,001-$35,000; $35,001-$50,000; >$50,000.
For an exemplary treatment of variables, consider “How Initiators End
their Wars: The Duration of Warfare and the Terms of Peace,” by Branislav
L. Slantchev (48/4 American Journal of
Political Science 813-29, October 2004).
Pursuing the question posed in his title, Slantchev has a section titled
Principal Explanatory Variables, one of which is Rate of Loss: “To
measure the relative rate of loss for the initiator, I compute the ratio of its
military dead to the total military personnel and divide the result by the
total rate of losses for both sides.”
Someone else using the Correlates of War data set could readily compute
this figure, as well as Slantchev’s variable, “Outcome of War
[which] is an ordered categorical variable that takes one of four values (1) defeat, if the initiator was
exterminated or capitulated unconditionally because of inability to continue
fighting; (2) concessions, If the
initiator agreed to an armistice and concluded and agreement that was
disadvantageous with respect to its war aims or the prewar status quo; (3) gains … and (4) victory….”
(818)
Note here that your variables and their indicators will in part be
determined by what data is available to you! You can plan a wonderful study,
but if the data are not available, or are not available in the form you assumed
they would be, the design will be near meaningless.
Some projects will require you merely to explain what you would do, if you were to execute the study, while other projects require the full execution. Adapt the following language to suit what is required of you.
A. Data Collection. Since you have been identifying and defining your variables with an eye toward the available data, the next step is to go out and collect that data. Here you will be specific and straightforward about how your data were collected. We tend to divide data collection into the two categories of unobtrusive measures, such as looking through books for data, or using someone else's data set, and obtrusive measures, where you must confront the subject being studied. An example of an obtrusive measure is a survey. Surveys often seduce students who, at some level, think they are avoiding the nasty business of data collection, because all they need to do is hand out and collect surveys. The hard work though is the crafting of the careful, judicious survey which will gather the data that you desire. And you know from your reading that surveys have limitations; you will want to review these before you use them and before you write up your results.
If you use a survey, you will want to include it in your project. It might also be a wise idea to run the survey questions by your faculty member before administering it. While surveys are appropriate for some studies, many research questions can be answered by data that already exists. Sources, below, provide a list of sources in Pickler Memorial Library which might contain information that you could fruitfully use.
If you have appropriately identified your variables and the indicators and values of these variables, this data collection should be fairly straightforward. Remember the cardinal rule of data collection: get data in the most disaggregated or basic form that you can; later you can turn interval level data into nominal level data if you need to, but not vice versa.
Many researchers find at this stage that the data that they were absolutely
positively undeniably existed, doesn't. At least not in the form that they were
expecting. As a result, you may need to slightly rework your variables or your
study to accommodate what does exist. That's generally okay, as the research
process is not nearly as linear as the "Five Steps" we have laid out here
make it seem. But do be careful that you rework any earlier section of the
paper that might need it, so that it stands as a coherent whole. Again,
examples include statements like "A survey was be administered to a random
sample of 300 undergraduate students", or "Every fifth Supreme Court
decision was sampled between 1955 and 1995," or "Data were compiled
from the Book of the States and the State Manuals of Illinois,
B. Statistical Testing. After collecting your data, it is time to run
the appropriate statistical tests. Since you have been careful to identify your
variables and collect your data, this too should be relatively straightforward:
the tests you run depend upon the level of data you have (nominal, ordinal,
interval, ratio). Here is where Stat 190 and your Methods book will be helpful.
Or check out any basic level statistics or social science methodology book from
the library.
Prior to "running the data" you will need to enter it into a
statistical package.
Data entry is not altogether hard, but perhaps it is alien to some of you.
Get a manual and read it, and follow the instructions. Once the data are
entered, the analyses can be performed. See your statistics book or Political
Science Methodology book for further discussion. If your project requires that
you run the data, you will then need to report your findings. Most word
processors allow you to readily incorporate tables into your paper. Consider
doing this, and then summarizing the highlights. Do not describe the whole
table, or it need not be there. Do point to important findings or highlights.
C. Findings. Having collected and crunched the data, it is time to
discuss your findings. Are your hypotheses supported? As a discipline we are
cautious in our language. We speak not of "proving" something, but of
"lending support", or "providing evidence for." You should
also consider addressing limitations (some might say "shortcomings")
of your study at this point. For instance, if you surveyed 300 college
students, you really don't know what "all Americans believe" or
likely even all college students. This is a good place to consider what you
might do different in the future as well.
The following are a mix of "hard copy" and on-line data sources. Congress
recently passed legislation requiring data gathered with the support of federal
funds be made public. One source that
has data sets that are downloadable is the InterUniversity Consortium of
Political and Social Research, in
Also, talk with professors who are familiar with literature in your field of study. And check out the data bases that are indicated on Dr. Ishiyama’s web page and Dr. Quinn’s web page. You can also ask a real live reference librarian in Pickler.
I. International Statistics
1) Economic and Demographic:
United Nations World Economic and Social Survey
World Bank World Development Report (data on infrastructure)
United Nations Demographic Yearbook
International Monetary Fund World Economic Outlook
United Nations Industrial Statistics Yearbook
Statistical Abstract of Latin America
Soviet Statistics since 1950
World Business and Economic Review
2) Military statistics on
expenditures:
U.S. Arms Control and Disarmament Agency World Military Expenditures and Arms
Transfers
1991-1992 (lists 144 countries 1981-1991)
Jane's Defense UG 730 j3
U.S. Defense and Military Factbook UA 23.6 b67 1991
3) Political Statistics:
International Almanac of Electoral History JF 1001 m17 1991
Political Handbook of the World
Europa World yearbook and European Political Facts JN 10 c65; JN 12 c65
World Factbook G122 w67
World Government
Facts on File Yearbook
II. U.S Statistics
1) U.S. Economic and
Demographic:
Economic Indicators Handbook
State Rankings
Statistical Abstract of the U.S.
USA by the Numbers
Moody's Industrial Survey
2) Political Statistics:
Congress and the Nation
Congressional Quarterly Almanac
The Almanac of American Politics
The Election Data book
Handbook of Campaign Spending
Gallup Poll
Missouri Census Profile
County and City Extra
IV. Other sources
Vital Statistics on American Politics
The Book of the States
D. Commonly Used Political
Science Journals
The Journals marked by an asterisk are available in Pickler Memorial Library. And
of course you might use JSTOR. Keep in
mind that JSTOR has incomplete holdings:
several major journals are not indexed there until they are five years
old. Thus, if you rely only on
electronic searches, you may overlook important literature.
|
Administration and Society* |
American Journal of Political Science* |
|
American Political Science Review* |
American Politics Quarterly |
|
British Journal of Political Science* |
Comparative Political Studies* |
|
Comparative Politics* |
Foreign Affairs* |
|
Foreign Policy* |
International Organization |
|
International Political Science Review |
International Studies Quarterly* |
|
Journal of Conflict Resolution* |
Journal of Politics |
|
Legislative Studies Quarterly* |
Policy Studies Journal/Policy Studies Review |
|
Political Behavior |
Political Communication* |
|
Political Research Quarterly* |
Political Science Quarterly |
|
Political Studies* |
Political Theory* |
|
Polity* |
Public Administration Review* |
|
Public Opinion Quarterly |
Publius* |
|
Social Science Quarterly* |
World Politics |
You may well run across other journal in your research, both within political
science, and in related social science fields. Other big name journals in
related disciplines include American Behavioral Scientist, American Economic
Review, American Historical Review, American Journal of Sociology, American
Sociological Review, Journal of Abnormal Psychology, Journal of Experimental
Psychology, Journal of Marketing, Research, Social Forces, Sociology and Social
Research, and Urban Affairs Quarterly.
back to top
back to the Political Science Home Page