Conducting Research

< Back to Best Practices for Economists

1. Organize conferences, seminars, and visitor programs that are inclusive. 
   Each conference, panel, seminar series, or other forum should feature a diverse group of economists.

2. Ensure that the conferences and seminar discussions you host are constructive.
   Setting and enforcing rules of responsible behavior by attendees at conference and seminar presentations can increase the quality of the intellectual exchange. 

3. Provide equitable access to journals.  
   Ensuring that boards of editors at journals are diverse, that high standards of ethical and equitable behavior are set, and that referees adhere to them may help reduce biases in the editorial process. 

4. Read and cite diversely; think inclusively.
   Citing a diverse set of authors on syllabi and bibliographies and finding value in alternative research approaches can improve your research and the discipline.

5. Share research opportunities broadly. 
   Seeking diversity when identifying research assistants and potential collaborators can help counter existing biases.

Organize conferences, seminars, and visitor programs that are inclusive. 

Each conference, panel, seminar series, or other forum should feature a diverse group of economists.

How to Implement 

• Explicitly communicate your commitment to diversity when inviting submissions for conference sessions. Evaluate event criteria and traditions as potential sources of bias.
• Invite and secure a diverse group of economists to speak, serve as panelists, deliver keynote addresses, and participate in your event. Start the process by inviting members of underrepresented groups first to ensure diversity in the final set. Consult the Diversifying Economics Seminars Speakers List.
• Cultivate a network of diverse economists at all career stages. Familiarize yourself with research outside your bubble. Invite young scholars, as well as those from smaller institutions who may be less well connected, to attend your event even if they are not presenting.
• Leverage your status as an invited speaker. When you receive an invitation from an organizer, make sure that you will be part of a diverse set of speakers. If attention to inclusiveness is not evident in the agenda, decline to take part (Collins 2019). 
• To ensure affordability and flexibility for all potential participants, offer to reimburse travel costs for those without research accounts; provide accommodations that allow parents to balance professional and family obligations; and allow remote involvement.
• Keep records of the percentage of participants who are women and URM economists. Others will be tracking and competing against your outcomes (see, e.g., Doleac and Pancotti 2019, Suri and Niehaus 2019).

Research and Resources

The Econ Seminar Diversity project, fueled by undergraduate students, graduate students, and predoctoral research assistants, is gathering data from department websites on how often economics research seminars feature women and URM economists. Currently, 2 percent of the more than 3,000 speakers in their data are URM, compared with 8 percent of economics faculty (Doleac and Pancotti 2019).

The Diversifying Economics Seminars Speakers List, developed by CSMGEP in cooperation with CSQIEP and CSWEP, provides a growing database of URM, LGBTQ+, and women speakers by field. For additional assistance in finding diverse speakers, contact senior women and underrepresented minority economists and circulate invitations to members of the American Society for Hispanic Economists and the National Economic Association. You can reach junior economists from diverse backgrounds through the AEA Summer and Scholarship Programs, the CSMGEP Mentoring Program, the Summer Economic Fellows Program, the CeMENT Mentoring Program, and the Diversity Initiative for Tenure in Economics. Listening to the perspectives of women and URM economists helps strengthen your understanding and your research.

Building a diverse network of researchers works best when done in advance of an event. Conference organizers may attempt to redress a lack of diversity by inviting women and URM economists at the last minute. These last-minute invitations are usually not a good fit for the invitees and can end up being a burden. A better practice to ensure diverse research activities is to be intentional about building a diverse network of contacts. Oftentimes this will involve reading more broadly and reaching out personally, on an ongoing basis, to women and URM economists.

David Romer and Justin Wolfers at the Brookings Institution (Romer and Wolfers 2018) report that “being disorganized or rushed had negative consequences for gender equity.” More specifically, “in cases when our procrastination meant that we had to scramble to find a last-minute discussant, or our failure to notice that an author was not going to produce an acceptable paper in time meant we had to try to solicit a paper very late in the process, we tended to go with the first names that came to mind or to turn to people with whom we had personal connections. Unfortunately, those people were generally men. A behavioral economist might link this to ‘availability bias,’ and we suspect the greater availability of men’s names reflects both a personal element—we are both men—and a systemic one—we are in a male-dominated profession.”

Another way conferences can expand inclusivity is to offer childcare, especially when they are held on weekends. Alternatively, organizers can publicize a list of babysitters used by local faculty or recruit undergraduate students as babysitters. Conferences should offer a clean, private area for nursing mothers to pump breast milk. If possible, they should also provide space for older children. Organizers should budget to make attendance affordable and practicable for all potential participants. 

Ensure that the conferences and seminar discussions you host are constructive.

Setting and enforcing rules of responsible behavior by attendees at conference and seminar presentations can increase the quality of the intellectual exchange. 

How to Implement 

• Develop a set of guidelines for your department and/or conference series. As an example, refer to the “Guidance for a Constructive Culture of Exchange in MIT Economics Seminars” (MIT Department of Economics 2019).
• Mitigate power dynamics in seminars. For example, if a powerful person dismisses a good question from a junior, female, URM, or less-connected person, help to flatten the power dynamic by asking to hear the answer to that question.
• Repeated interruptions or refusals to accept an answer to a question should also be addressed. If it is not possible to do so during the seminar, follow up immediately afterward and explain how destructive the behavior is.
• Track how you and your colleagues behave. As a model, see the statistics collected in World Bank recruitment seminars (Ozler 2019).
• As a participant, your primary objective in attending a seminar is to learn from the speaker. You can also collaboratively help the speaker improve the quality of their scholarship. The objective should never be to prove that you are the smartest person in the room.
• Recognize the needs of different-abled economists when planning conferences. Ensure venues are accessible to those with visual and mobility impairments. Provide captioners to aid people with hearing disabilities, and make sure presenters use microphones when available to ensure proper delivery of captioning services.

Research and Resources

The economics profession is infamous for its aggressive seminar culture. “An adversarial and aggressive culture within academic economics is often advanced as a causal force in women’s stalled progress in the profession, though its impact is difficult to quantify. Economics seminars, for example, have a reputation for being particularly hostile environments. The culture of an academic discipline can have gendered implications if women either fail to fully adapt to the culture or if they receive differential treatment as a result of it” (Lundberg and Stearns 2019). 

New efforts to collect quantitative data on seminar dynamics suggest that women are treated differently than men (e.g., Modestino, Dupas, Niederle, and Wolfers 2019; Ozler 2019). These findings are consistent with a striking disparity recorded in the recent AEA professional climate survey (Allgood, Badgett, Bayer, Bertrand, Black, Bloom, and Cook 2019): 91 percent of respondents agree that men are respected within the field, while 41 percent overall (and only 16 percent of female respondents) agree that women are respected. The hostility and lack of respect can be detrimental to both the field and its practitioners; for example, to avoid possible harassment, discrimination, or unfair or disrespectful treatment, 40 percent of Black, Latinx, and Native American economists report having not presented a question, idea, or view and over 20 percent have not participated in a conference (Bayer 2020). “Harassment and exclusionary discourse are barriers to diversity in economics and, therefore, to the advancement of our scholarship. Tacit acceptance of such behavior contributes to a general perception that the economics profession is unwelcoming, or even actively hostile, to female and minority economists” (CSWEP 2018). 

Note that critique is not the problem. Economists who are members of groups underrepresented in the profession want close attention to and feedback on their work just as much as any economist. But, as participants in the Graduate Student Summit for Diversity in Economics note, “a common problem with seminars is low-quality communication. Time is precious for everyone, and low-quality communication prevents participants from getting the most value from seminars” (Blakeney, Kestelman, and Liu 2019). They provide examples of low-quality contributions from seminar audience members (paraphrased below): 

• Questions that aren’t questions 
• Questions that attack the whole idea—e.g., why would anyone study this?
• Overly technical questions, either to tear down or to show off
• Expecting norms from a different subfield to be applied
• Making the seminar about themselves
• Extended back-and-forth digressing from the main topic
• Talking between audience members

“Economists have a professional obligation to conduct civil and respectful discourse in all forums” (AEA 2018). The guidance provided by the MIT Economics Department (2019) can “help ensure that department seminars promote an open and vibrant exchange of ideas within a positive environment for both presenters and participants.” Given that women and URM speakers, especially, face premature, inappropriate, relentless, or patronizing questions, a neutral policy that, say, establishes a no-questions portion of the seminar (e.g., the first ten minutes) may help combat bias. Effective bystanders can also help change seminar norms (see the second Colleagues section). 

Provide equitable access to journals. 

Ensuring that boards of editors at journals are diverse, that high standards of ethical and equitable behavior are set, and that referees adhere to them may help reduce biases in the editorial process. 

How to Implement 

• Referees should remain mindful of documented biases in assessing the work of women and URM economists. Referees should judge research with an open mind and on its own merits without using proxies such as affiliation, age, race, or gender to evaluate papers.
• Editors must monitor and provide feedback to referees, recognizing that racial and/or gender bias may nevertheless be present in referees’ evaluations of papers. Editors encountering mistaken beliefs or racial/gender animus must overrule referees and dismiss these reports. Similarly, editors should screen out excessively aggressive reports and not share them with authors. Decisions should be communicated to the referees to encourage improvement. Track outcomes and be alert to concerning patterns. 
• Editors have a special responsibility to be mindful of their own biases as well and should exercise humility and self-reflection in editorial decisions.
• A referee or editor who has been involved in an intimate relationship, or has suggested an intimate relationship, with an author should recuse themselves from reviewing or editing that author’s work.
• Including women and URM economists on the board of editors can build a diverse pipeline of future journal editors. Providing discounts to graduate students or on the basis of income can make publishing more accessible to a broader set of submissions.

Research and Resources

In a study of submissions to four leading economics journals, Card, DellaVigna, Funk, and Iriberri (2019) find that that female authors appear to be held to a higher standard by referees in the editorial process, estimating that papers with female authors have a 1.7 percentage point lower probability of a revise and resubmit verdict relative to a citation-maximizing benchmark. Hengel (2019) finds female authors wait longer for peer review even though their papers are more readable; she concludes that “tougher editorial standards . . . cause senior female economists to write at least 7 percent more clearly than they otherwise would. Moreover, the cost to men of publishing a paper appears to be much lower than the cost to women: female-authored papers spend 3–6 months longer in peer review compared to observably equivalent male-authored papers (data from Econometrica and the Review of Economic Studies).” Although it might seem that using women reviewers could mitigate this disparity, Card, DellaVigna, Funk, and Iriberri (2019) find that female referees seem to hold female-authored submissions to the same higher standard as male referees do.

As a reviewer, include detailed and constructive comments. Avoid using glib statements such as “that is not economics” to criticize methodological approaches you may not like. Instead, articulate substantive critiques of a given methodology, and clarify whether a given critique applies to a particular paper or to a literature. In the case of a desk rejection, be explicit about the reason so that authors learn how to improve their papers.

In an initiative to improve gender balance at the Brookings Papers on Economic Activity, Romer and Wolfers (2018) find that “making an explicit decision to be concerned about these issues and to think consciously about them had the biggest impact on gender equity.”

Read and cite diversely; think inclusively.

Citing a diverse set of authors on syllabi and bibliographies and finding value in alternative research approaches can improve your research and the discipline.

How to Implement 

• Be intentional about citing a diverse set of authors on syllabi and in bibliographies. Broadening the set of papers you assign and cite expands your knowledge and diversifies the array of research and researchers that are seen as worthy. You can use this tool to assess the gender and racial/ethnic balance of syllabi and bibliographies (Sumner 2018).
• Whenever possible in written materials, as well as in presentations and social media (e.g., X (formerly Twitter), EconSpark), cite papers by all names or by an acronym of all family names. When writing as coauthors, consider using a certified random order of names when appropriate (Ray ⓡ Robson 2018).
• Maintain a diverse reading list, and build a diverse research network.
• Recognize value in alternative research methodologies and approaches. Exercise humility and self-reflection to understand how your own background and experiences influence your view on what constitutes an interesting research question, a valid methodological approach, or a compelling panelist.

Research and Resources

The resources and ideas summarized in this resource can help you evaluate the diversity in your own research activities and set explicit goals for improvement. Diversify the authors whose research you read, and expand your intellectual circle. Check the reference lists of the papers you are writing using Sumner’s (n.d., 2018) tool and correct imbalances. As Suri and Niehaus (2019) do for women in the field of development economics, track the seminar presentations you attend and the departments you visit and seek out ways to diversify them. Consider reaching out to economists at minority-serving institutions and historically Black colleges and universities. Explore diversity within groups; the circumstances and insights of white, Black, Latinx, and Asian women can differ (Sharpe 2019). Be open to alternative research approaches to increase the demographic and intellectual diversity of your networks and resources.

Share research opportunities broadly. 

Seeking diversity when identifying research assistants and potential collaborators can help counter existing biases. 

How to Implement 

• Proactively share professional opportunities with women and members of underrepresented groups to help counteract pervasive biases. 
• Involve diverse groups of students in research as early as possible in their academic careers. Connect these students with organizations that promote diversity through mentorship, such as the Leadership Alliance and the Sadie Collective.
• Recognize that some forms of research (e.g., field experiments in development) are increasingly valued but expensive propositions. Create ways to engage graduate students and faculty who are unable to access the necessary resources on their own. 

Research and Resources

Chari and Goldsmith-Pinkham (2018) show that women constitute a small fraction of presenters at the NBER Summer Institute, and Hospido and Sanz (2019) show that, other things equal, papers authored by women are less likely to be accepted at conferences than papers authored by men. Individuals and organizations can be deliberate about countering the effects of pervasive biases. 

In sum, the evidence suggests that poor research practices are impeding the trajectories of women and URM economists and of economic research itself. Economists need to redress discrimination intentionally for the good of the profession and of the discipline.

See References.

Please use the "Feedback on Best Practices for Economists” option on the AEA contact form to send questions and feedback.