The 2019 Asian Political Methodology Meeting

[This post was contributed by Kentaro Fukumoto, Professor of Political Science at Gakushuin University.]

We held the joint conference of the 6th Asian Political Methodology Meeting and the second annual meeting of the Japanese Society for Quantitative Political Science at Doshisha University in Kyoto, Japan, on January 5 and 6, 2019. We have a small, intensive conference focusing on innovative quantitative methods and their applications. Although the conference seeks to promote the advancement of quantitative social science research in Asia, we have all regions of the world represented at the conference.

The program is at:

where you can download papers. Presented posters are available at:


We called for papers from July to September and received 56 applications for paper (oral) presentations and 40 for posters from 19 countries. The program is composed of a keynote address, 14 panels (29 papers), and two poster sessions (42 posters). We had 117 registrants from 12 countries, while we ended up with 105 attendees (among which we have 68 visitors from abroad). Prof. Robert J. Franzese Jr. (University of Michigan) delivered the keynote address, titled “21st Century Political Methodology: Advances in All Modes of Empirical Analysis in Political Science.” For the first time, the conference hosted papers from graduate students and post-docs.

This year we established the Best Poster Award, sponsored by the Japanese Journal of Political Science and the Society for Political Methodology (SPM). The winners, Soichiro Yamauchi (Harvard University) and Naijia Liu (Princeton University), receive $200 worth of paperback books from Cambridge University Press and the SPM invites them to its annual summer meeting.

This conference is sponsored by Doshisha University, Egusa Foundation for International Cooperation in the Social Sciences, Gakushuin University, the Kajima Foundation, Princeton University (Program for the Quantitative and Analytical Political Science), and SPM. The Program Committee is composed of Kentaro Fukumoto (Gakushuin University, Japan, committee chair), Fang-Yi Chiou (Academia Sinica, Taiwan), Benjamin Goldsmith (Australian National University, Australia), Takeshi Iida (Doshisha University, Japan), Kosuke Imai (Harvard University, U.S.A.), Koji Kagotani (Osaka University of Economics, Japan), Xun Pang (Tsinghua University, China), and Jong Hee Park (Seoul National University, Korea). We plan to have the next meeting in Hong Kong in 2020, which will be hosted by Jiangnan Zhu (University of Hong Kong).

Posted in Uncategorized | Leave a comment

Corrigendum to “Lowering the Threshold of Statistical Significance to p < 0.005 to Encourage Enriched Theories of Politics” and “Questions and Answers: Reproducibility and a Stricter Threshold for Statistical Significance”

Although The Political Methodologist is a newsletter and blog, not a peer-reviewed publication, I still think it’s important for us to recognize and correct substantively important errors.  In this case, I’m sad to report such errors in two things I wrote for TPM. The error is the same in both cases.

In“Lowering the Threshold of Statistical Significance to p < 0.005 to Encourage Enriched Theories of Politics,” I claimed that:

When K-many statistically independent tests are performed on pre-specified hypotheses that must be jointly confirmed in order to support a theory, the chance of simultaneously rejecting them all by chance is αK where p < α is the critical condition for statistical significance in an individual test. As K increases, the α value for each individual study can fall and the overall power of the study often (though not always) increases.

This argument is offered to support the conclusion that “moving the threshold for statistical significance from α = 0.05 to α = 0.005 would benefit political science if we adapt to this reform by developing richer, more robust theories that admit multiple predictions.”

Similarly, in “Questions and Answers: Reproducibility and a Stricter Threshold for Statistical Significance,” I claimed that:

Another measure to lower  Type I error (and the one that I discuss in my article in The Political Methodologist ) is to pre-specify a larger number of different hypotheses from a theory and to jointly test these hypotheses. Because the probability of simultaneously confirming multiple disparate predictions by chance is (almost always) lower than the probability of singly confirming one of them, the size of each individual test can be larger than the overall size of the test, allowing for the possibility that the overall test is substantially more powerful at a given size.

This reasoning, which is similar to reasoning offered in Esarey and Sumner (2018b), is incorrect; it would only be true when all predicted parameters were equal to zero. When the alternative hypothesis is that multiple directional predictions for parameters,  for example βi > 0 for i 1…K, separate t-tests rejecting each individual null (βi 0) separately using t-tests with size α will jointly reject all the null hypotheses at most α proportion of the time. The key insight is that the joint null hypothesis space includes the possibility that some βi parameters match the predictions while others do not; if (for example) β1 = 0 and all other βi=/=1 are very large, the probability of falsely rejecting the joint null hypothesis is the α for the test of β1. As we note in Esarey and Sumner (2018a), this is discussed and proved in Silvapulle and Sen (2005, Section 5.3), especially in proposition 5.3.1, and in Casella and Berger 2002, Section 8.2.3 and 8.3.3. Silvapulle and Sen cite Lehmann (1952); Berger (1982); Cohen, Gatsonis and Marden (1983); and Berger (1997) (among others) as sources for this argument. Associated calculations (such as that in Figure 4 of “Lowering the Threshold of Statistical Significance to p < 0.005 to Encourage Enriched Theories of Politics”) are based on the same error and therefore incorrect.

The upshot is that my argument for making additional theoretical predictions in order to facilitate lowering the threshold for statistical significance to α = 0.005 is based on faulty reasoning and incorrect.

I plan to post this correction as an addendum to both of the print editions featuring these articles.


Berger, Roger L. 1982. “Multiparameter Hypothesis Testing and Acceptance Sampling.” Technometrics 24(4):295–300.

Berger,  Roger  L.  1997.  Likelihood  ratio  tests  and  intersection-union  tests.  In  Advances  in statistical decision theory and applications, ed. Subramanian Panchapakesan and Narayanaswamy Balakrishnan.  Boston:  Birkhäuser pp. 225–237.

Casella, George and Roger L.. Berger. 2002. Statistical Inference, Second Edition. Belmont,
CA: Brooks/Cole.

Cohen, Arthur, Constantine Gatsonis and John I. Marden. 1983. “Hypothesis testing  for marginal probabilities in a 2 x 2 x 2 contingency table with conditional independence.”   Journal of the American Statistical Association 78(384):920–929.

Esarey, Justin and Jane Lawrence Sumner. 2018a. “Corrigendum to Marginal Effects in Interaction Models: Determining and Controlling the False Positive Rate.” Online. URL:

Esarey, Justin and Jane Lawrence Sumner. 2018b. “Marginal Effects in Interaction Mod-  els: Determining and Controlling the False Positive Rate.” Comparative Political Studies 51(9):1144–1176. DOI:

Lehmann, Erich L. 1952. “Testing multiparameter hypotheses.” The Annals of Mathematical Statistics pp. 541–552.

Silvapulle, Mervyn J. and Pranab K. Sen. 2005. Constrained Statistical Inference: Inequality, Order, and Shape Restrictions. Hoboken, NJ: Wiley.

Posted in Uncategorized | 2 Comments

Papers Written by Women Authors Are Cited Less Frequently, but the Etiology of this Finding is Complex

Justin Esarey, Wake Forest University
Kristin Bryant, Rice University


A recent symposium in Political Analysis, anchored around Dion, Sumner and Mitchell (2018), discusses their finding that articles authored by women are more likely to cite at least one paper authored by women. Our contribution to this symposium (Esarey and Bryant, 2018) noted that articles in the Dion, Sumner, and Mitchell (2018) data set with at least one female author are cited no more or less often than male-authored articles once we control for the publishing journal and the number of authors. In this paper, we present additional findings that place the results of our original paper into a broader context. This context is important to fully understand how scholarship by women is utilized by the discipline, how scholars’ careers are impacted as a result of this utilization, and how we might achieve greater gender parity in the field.

When looking at the the unadjusted data set, articles with at least one woman author are in fact cited fewer times on average. It is plausible that this citation gap does represent a substantively meaningful barrier to the advancement of women in the discipline. As we reported in Political Analysis, papers with women authors are no more or less likely to be cited once the number of authors and the publishing journal are controlled for via linear regression. However, simply controlling for author count is insufficient to eliminate the gender disparity in citations: controlling for the publishing journal is crucial. An implication is that women may be systematically disadvantaged in the field, but that this disadvantage is not a function of discrimination against women when articles are chosen to be cited. Instead, consistent with the findings of Teele and Thelen (2017), we find that articles in the most-cited journals of the discipline are less likely to have women authors. The etiology of that relationship (and the citation gender gap that it creates among political scientists) is difficult to unravel.

Full Text

Replication File

A replication file for this paper is available at

Posted in Uncategorized | Leave a comment

International Methods Colloquium: 2018-2019 Schedule!

On behalf of the advisory board (Michelle Dion, Cassy Dorff, Jeff Harden, Dustin Tingley, and Chris Zorn), I am pleased to announce the schedule of International Methods Colloquium series talks for the 2018-2019 academic year!

The International Methods Colloquium (IMC) is a weekly seminar series of methodology-related talks and roundtable discussions focusing on political methodology; the series is supported by Wake Forest University and was previously supported by Rice University and a grant from the National Science Foundation. The IMC is free to attend from anywhere around the world using a PC or Mac, a broadband internet connection, and our free software. You can find out more about the IMC at our website:

where you can join a talk in progress using the “Watch Now!” link. You can also watch archived talks from previous IMC seasons at this site. Registration in advance for a talk is encouraged, but not required.

Note that all talks begin at 12:00p Eastern Time and last precisely one hour.

Here is our schedule of presenters (and a link to our Google Calendar):

Fall Semester
  1. Oct 12: Matthew Blackwell, Harvard [register to attend]
  2. Oct 19: Masha Krupenkin, Stanford [register to attend]
  3. Nov 2: Roundtable on Gender, Citations, and the Methodology Community with Michelle Dion (McMaster), Sara Mitchell (Iowa), Dave Peterson (Iowa State), and Barbara Walter (UCSD) [register to attend]
  4. Nov 9: Luke Keele, University of Pennsylvania [register to attend]
  5. Nov 16: Kevin Munger, Princeton/Penn State [register to attend]
  6. Nov 30: Pablo Barbera, London School of Economics [register to attend]
Spring Semester
  1. Feb 1: Michelle Torres, Rice [register to attend]
  2. Feb 8: Marcel Neunhoeffer, Mannheim [register to attend]
  3. Feb 15: Winston Chou, Princeton [register to attend]
  4. Feb 22: Erin Rossiter, WUSTL [register to attend]
  5. Mar 1: Matthew Tyler/Christian Fong, Stanford [register to attend]
  6. Mar 8: Rob Carroll, Florida State [register to attend]
  7. March 29: Carlos Carvalho, University of Texas Statistics [register to attend]

Additional information for each talk (including a title and a link to a relevant paper) will be released closer to its date.

Please contact me if you need any more information; I hope to see many of you there!

Posted in Uncategorized | Leave a comment

Using Sequence Analysis to Understand Career Progression: An Application to the UK House of Commons

Matia Vannoni, IGIER, Bocconi University
Peter John, Department of Political Economy, King’s College London

Abstract: We argue that sequence analysis, mainly used in sociology, may be effectively deployed to investigate political careers inside legislatures. Career progression is a classic topic in political science, but political scientists have mainly examined access to legislatures. Although data reduction methods, for instance, can provide insight, we argue that sequence analysis can be used to understand better the career patterns inside parliaments. In this paper, we explain the method. Then we show how it can describe steps in political careers and map different patterns of advancement. We apply sequence analysis to a case study of MPs in the UK House of Commons from 1997 to 2015. We describe the variety of career paths and carry out regression analysis on the determinants of MP career progression.

Full Article


Online Appendix


Replication File

Replication files for this paper are located at:

Posted in Uncategorized | Leave a comment

MZES Open Social Science Conference 2019: Practicing New Standards in Transparency and Reproducibility

I received this message from Alexander Wuttke of the Mannheim Centre for European Social Research and the Leibniz Institute for the Social Sciences, announcing a new conference on Transparency and Reproducibility in the Social Sciences. This conference may be of interest to readers of The Political Methodologist!

— begin announcement —

MZES Open Social Science Conference 2019: Practicing New Standards in Transparency and Reproducibility

This conference is a forum for practicing and discussing credibility, transparency and replicability in the social sciences.

About a decade ago, John Ioannidis claimed that “most published research findings are false”. While seeming outrageous at the time, a growing body of meta-science research in the behavioral and social sciences substantiated this claim, causing uncertainty about the trustworthiness of published scientific findings. We believe that threats to the validity of published findings in the social sciences are widespread and systemic. Therefore, this conference promotes introspection about the current state of social science research and exchange on the opportunities for institutional and methodological improvement in the future.

The conference is supported by the Berkeley Initiative for Transparency in the Social Sciences (BITSS) and will take place from 25-27 January 2019 in Mannheim, Germany.

Conference Website – twitter

Keynote speakers:

  • Jeremy Freese (Stanford University)
  • Thomas König (APSR, University of Mannheim)
  • Arthur Lupia (OSF, University of Michigan)
  • Julia Rohrer (100% CI, Leipzig University)

Participate in the conference:

  • Give a talk: We call for researchers to advance discussion, debate, literature synthesis, or methods in open social science. We welcome methodological advances, e.g., p-curve analysis, systematic reviews, pre-analysis planning, and replication. We welcome general research findings that apply best practices of open science while conducting the research – Abstract submission DL: 22 August 2018  Read more
  • MZES-GESIS Pre-Registration Challenge:  We call for researchers to participate in a competition to win funding or survey time for the most innovative and rigorous pre-registration plan for a social science study. – Abstract submission DL: 22 August 2018  Read more
  • OSSC19 Crowdsourced Replication Initiative: We call for researchers to replicate and expand a previously published cross-national macro-comparative study. The goal is to explore and develop crowdsourcing methods and generate research surpassing what a single researcher could achieve. The replication comes from the field of immigration and social policy, but we encourage social science researchers of all disciplines and levels to participate. All full participants will be co-authors on the final paper. – Registration DL: 27 July 2018   Read more
  • Participate as a guest in Mannheim during the conference or during the subsequent Open Science Workshop, offered in collaboration with the Berkeley Initiative for Transparency in the Social Sciences (BITSS). Or use the live stream online.

Organizing Committee

Nate Breznau (MZES, University of Mannheim)
Eike Mark Rinke (MZES, University of Mannheim)
Alexander Wuttke (MZES, University of Mannheim)

Conference website:
Twitter: @opensocsci

Posted in Uncategorized | Leave a comment

New (but old) Print Edition Released!

The blog format of The Political Methodologist has made the release of print editions less pressing, as new content is immediately distributed to readers through our web presence. However, we still provide a LaTeX-compiled print edition, primarily so that our content can be properly cited in other work and circulated in print as necessary. This particular issue has been compiled rather belatedly: this is the Fall 2017 edition, and the content in this issue has been available on our website for months. We apologize for the lateness of this issue; at this time, TPM labors in the absence of any editorial assistance.

Posted in Uncategorized | Leave a comment