Matching Recommender System for the Econ Job Market

Question

The job market involves common difficulties for both hiring committees and job candidates. Sometimes, search committees shoot too high, leaving the department with few options when those candidates fall through. Other times, departments bypass candidates they thought were out of reach, and they learn, too late, that the candidate would have preferred their offer to the one they accepted.

In parallel, candidates struggle because they have limited tools to credibly signal interest in jobs, a fact that makes both sides of the market worse off. It's particularly difficult for job candidates to signal useful information because they don't know what their option set will be; their interest in any particular job is contingent on available alternatives.

We've made a tool that helps solve these issues for both search committees and job candidates. The tool (EconMatch.com) takes preference-rankings from employers and candidates, after the AEA interviews (by midnight January 8), and then uses them to recommend each employer the top five “gettable” candidates using an adaption of the traditional matching algorithm. These recommendations represent an employer’s best (most preferred) attainable candidates. Moreover, the process allows candidates to credibly signal interest to their favorite, attainable employers.

Some candidates might worry that employers will learn they were not the candidates’ top choice. We preserve candidates’ privacy by limiting the recommended list to five names and blurring out one in five recommendations. Therefore, the hiring committee cannot infer that the candidate was not interested in the position if a candidate is not recommended. So participation helps you when recommended and doesn't reveal your preferences when you aren't.

We're rolling out a fully functional beta version of the tool this Fall, and we have gotten encouraging responses from several departments as well as a few interested economic associations. If you’re a candidate or search chair this year, we would value your feedback as you participate in the match. We’ve learned a lot just from questions and concerns people have posed. Feel free to post here or email me directly at econmatch@gmail.com.

If you're anything like me, you might find our five-minute video for candidates (https://youtu.be/YKkQzdIUX7k) and for employers (https://youtu.be/w8qGv0g6VAY) clarifying how to use the tool and why it's useful.

Thank you very much, and I look forward to learning from your suggestions. We hope to help you have a better job market.

Comments

I'm a current job market candidate--thank you guys for working on this problem!

One concern I have is that the signal sent by being on a "recommended list" might be a little ambiguous.  If I understand this correctly, a candidate is recommended to an institution if they were not ranked highly by institutions they preferred.  As a result, being "recommended" signals some combination of a candidate's interest in a position and a lack of interest in the candidate by other employers.  I'm concerned about whether this could lead to a "groucho marx" effect in cases where a candidate doesn't have other credible signals of interest in a position--a hiring committee might worry that their candidate didn't impress other departments.

The degree to which this is a problem seems like it depends in part on how "gettable" is defined--this would be a bigger problem if a candidate can't be highly rated by any institutions that they prefer in order to be gettable.  It also seems like something that could be avoided entirely with a second round of signals rather than a match, since a signal is an unambiguous indication of interest (though of course signals introduce strategic behavior on the part of candidates).

This might not be a concern in practice, but I thought I'd get your feedback on this.

Thanks again for building this tool and for opening a discussion about it!

---

It's our pleasure: you bet. This is an important question, and a good use of Groucho.

When a department pursues a candidate that later declines their offer, it can be extremely costly. While a department waits on their first offer to accept or decline, their other candidates receive and accept offers, leaving their remaining pool of candidates negatively selected.

For this reason, departments pursue candidates, disproportionately, that they suspect are attainable; bad guessing gums up the market, making everyone worse off. The match allows candidates to signal their interest in their best, attainable jobs. In our experience, it was hard to signal to places our advisers didn't have a friendship. The match allows you to signal credibly, regardless of your network.  

The reason why non-match signals are not very useful is that even a forthright candidate does not have enough information to signal truthfully in the normal process, because her interest in a place is contingent on the opportunity set, which is hard to predict. The matching recommender system saves time (producing diluted signals) while providing much better information.

To answer the question another way, the recommendation signals to the department that these candidates are the best (as indicated by the department's own ranking) candidates who won't likely have dominating offers. This could mean that other departments bypassed the candidate (if the department believes in their own assessment, that shouldn't matter); OR, it could mean that the candidate has a special interest in that department. We normally worry much more about whether a candidate is gettable than whether a candidate is worse than we assessed (we trust our own assessments).

Participation essentially gets the candidate noticed (1) by places that would have erroneously top-coded them and (2) places that would have pursued other candidates that weren't ever going to come. We believe that candidates that use EconMatch will have a greater number of well-targeted fly outs. The match trains the departments on the best candidates that are serious about the job.

---

Thank you--this is really helpful!  I definitely buy that departments are very concerned about attainability when offering flyouts.  I also believe that economists are on the whole confident in their assessments :).

However, this does make me a little more concerned about the negative consequences of not being listed, particularly the negative consequences of being not listed at random for a position to which I match.  Of course the 1 in 5 censoring is useful for making sure that not being listed isn't a strong signal that a candidate is unattainable, but it's still a weak signal that a candidate is unattainable (and if it wasn't, the tool would be less useful).  A 1 in 5 chance that a top choice that rates me highly will get a signal that I am not interested is a bit concerning.  Have you guys received feedback that the 1 in 5 blurring is important to people?

---

Another good question. This was a solution that was really important to some of our early testers, especially those that were matchers, and remains perhaps the most important secondary feature of the platform.

The concern is that a user might be concerned that employers will learn how a candidate ranked them from the result of the match, in particular if the candidate is not recommended. The differential-privacy scheme prevents employers from being able to infer anything about candidates that weren't recommended.

Recommended matches are very good for candidates: they are likely the best places a candidate could hope to receive an offer. At the same time, if a candidate is not recommended, the hiring committee cannot infer that the candidate was not interested in the position. Thus, participation helps you when recommended and doesn't reveal your preferences when you aren't.

Let me know if you have any suggestions!

Answer 1

Sounds a lot like our Israeli Law Signaling Program (which have been working well for the last two years, also based on DA as a non-commitment signaling mechanism).
But who are you guys? Went over the FAQ and everything and could not identify you...

Answer 2

Note that both departments and candidates are required to submit rankings on the last day of the meetings. It is not unusual for departments to discuss and rank candidates only after returning from the meetings. One can also imagine that some candidates might want to discuss options with their advisors.

Comments

That's a great point. We know that some departments begin inviting candidates for flyouts at the AEAs, so there's a bit of tension to figure out what is the cutoff day that includes the largest number of departments. The schools that have discussed this with us have tended to decide that they could arrange for a conference call with their department on the final day of the AEAs. That's not great for your Sunday afternoon, but it does allow for a common clearing date. Let me know if you have suggestions wrt/figuring out the best day to implement the match.

---

I agree that it is an empirical question as to the best date. And if EconMatch were to become widely used that might itself change the equilibrium. My guess would be a delay of a two or three days would be better, but you have more data than I do.

---

This is the primary reason my department does not want to use the system.  A delay of even one day would be helpful because we are likely to interview through Sunday afternoon.
Our other concern is with being a first-mover.  Does the fact that a department wants to participate as an employer (and in the first year of the program) signal that a department has had matching problems in the past?

---

Thanks for this. We're preparing to move back the ranking deadline to ***MIDNIGHT, JANUARY 8, 2019***, instead of the night of AEAs, January 6th.

I think it's also a thoughtful question to wonder whether participation itself is a signal of some kind.

If anything, it seems that the response to EconMatch has been most positive from more selective departments and graduate programs, suggesting that selection into EconMatch is not strongly related to hiring or placement difficulties.

Moreover, candidates are invited to rank *all* their interviews, and we don't tell them which employers are participating in the match, so candidates don't receive the "signal" that the employer is participating if employers don't want them to.

---

Thank you; this is very helpful and we are reconsidering our decision.  What is the last day that we could tell you we'd like to join?

---

My pleasure! The registration deadline is December 15; the search chair can make an account by Dec 15 and then decide not to submit rankings, of course.

Answer 3

Did you consult with a lawyer to see if this practice is legal? I suspect it may not be the case.
Entry level positions in economics are different from medical residencies (and from law internships in Israel) in that holding multiple offers allows candidates to negotiate their terms of employment.
The “Antitrust Guidance for Human Resources Professionals” (https://www.justice.gov/atr/file/903511/download) says, for example, that:
“You should not enter into agreements about employee compensation, other terms of employment, or employee recruitment with other HR professionals who work at competitors, meaning other companies that compete for the same types of employees.”

Comments

Thanks, good question. This particular concern is not an issue for EconMatch because our service does not constitute or create any agreements between employers or candidates. Since EconMatch does not create required appointments, employers are still engaged in direct competition when delivering offers and competing for candidates.

One way to see this more clearly is to notice that are service is more like other candidate recommender sites, like ZipRecruiter or Monster.com, in function than it is like the residency match which generates essentially fixed appointments. The correspondence between the NRMP and EconMatch is the algorithm, which isn't inherently anti-competitive. We use it to perform the function of a recommender, they use it to generate appointments.  

All of the case law that clamps down on employers for anti-competitive behavior is only successful in settings where employers explicitly collude to prevent competition.

For example, the Court struck down an agreement among engineers that prohibited competitive bidding for engineering projects, noting that it “is not price-fixing as such, [but] the ban on competitive bidding, like price fixing, impeded the ordinary give and take of the market place.”

In Federal Trade Commission v. Indiana Federation of Dentists, the Supreme Court reaffirmed this principle when it struck down an agreement among a group of dentists to withhold radiographs from insurance companies because it “disrupt[s] the proper functioning of the price-setting mechanism of the market.”

EconMatch merely allows candidates to signal their interest to employers. It, in no way, prevents employers from making more competitive offers in order to hire the best candidates possible. If anything, we believe that employers will exhibit increased competition as candidates have a larger number of viable flyouts and a smaller number of flyouts that are unlikely to yield a competitive offer.

Answer 4

I received a good question that others might have and I wanted to post it here:

"Just want to clarify. When candidates do their rankings, employers can see which of the candidates they interviewed are participating and which are not participating, correct? (This is my inference from having watched the employer video.) Assuming that's correct, suppose that an employer sees that Will Smith is participating and he's their favorite candidate. But Will is not on the match list the employer receives. That employer can now infer that Will didn't rate them very highly. Now Will may miss out on a flyout with them. Is my logic flawed anywhere here? You may say that Eric isn't missing out on much in this case because he doesn't value that flyout as much as others."

Comments

Great question!

We've implemented a differential-privacy scheme that prevents employers from being able to infer anything about candidates that weren't recommended.

We accomplish this by only providing to employers their top five matches while withholding the names of a random 20%.

Recommended matches are very good for candidates: they are likely the best places a candidate could hope to receive an offer.

At the same time, if a candidate is not recommended, the hiring committee cannot infer that the candidate was not interested in the position.

Thus, participation helps you when recommended and doesn't reveal your preferences when you aren't.

Answer 5

We're preparing to move back the ranking deadline to MIDNIGHT, JANUARY 8, 2019, instead of the night of AEAs, January 6th.

Please let me know if your department would be helped, or hurt, by this adjustment.

We believe that this will accommodate the largest number of departments while allowing candidates more time for reflection before ranking employers.

Comments

We would benefit.  This may be enough to make us join.

Answer 6

I am curious as to why the folks offering EconMatch have chosen to remain anonymous.

Comments

I think mainly to keep the focus on the platform.

---

I suspect that folks would have more faith in the system if they knew who was in charge. But I assume the schools participating  know who you are (as does the AEA). Since you're the ones putting in the work for the common good (and I assume a publication down the line), it's really up to you!

---

Allow me to point out that the person behind EconMatch has been publicly revealed elsewhere, and is a perfectly respectable economist.

Answer 7

You should reach out to more faculty members. I know several candidates whose advisors have asked them not to participate.

Comments

Also, it's helpful to know what concerns, if any, advisers have. I've answered several advisers in email, and I think they've almost all come around to having their students participate; if advisers would like to talk to more about whether they think their students should participate, they can kick the tires by emailing me directly at econmatch@gmail.com or post questions here.

---

Thanks! Let us know if you have ideas. My understanding is that the GPDs (graduate placement directors) at four top ten schools have proactively let us know that they are encouraging their candidates to use the service.

We've sent emails to ~2,610 economics faculty, each of them twice. The Southerns emailed an additional ~1,900 of their members two weeks ago. We've also emailed each of the candidates at ranked graduate economics programs who are on the market this year on their program's websites (another 900 emails). We're 100% open to suggestions.

Skepticism, on the part of graduate advisers, is probably a reasonable default position. The match signal is enormous benefit to candidates, and we're trying our best to show advisers that.

Answer 8

Logistical question:

How do you vet who signs up as the responsible party for a listing on the employer side? Seems like that role is there for the taking among depts/employers who are oblivious to this pilot venture.

Thanks.

Comments

Good question. This is up to search committees, but it would be natural for the search chair to make and manage the match process.

If two folks from the same department claim the same opening, we freeze both accounts and call the department to figure out which individual is managing the match.

What we don't know how to do is vet each employer that signs up to make sure they are the "right" person. So far, we haven't had any problems here. Let me know if you have a suggestion!

---

If there was AEA buy-in enough to let employers tell JOE to send you the relevant contact info then you could verify a match.

But I suppose that the hypothetical hijacking of a some employer accounts would not be likely to do much other than reduce the efficiency of the matching if you think of it like randomly missing data (maybe non-random is more likely but you all would know more about that than me). Good luck anyway. I am interested in giving it a try on the candidate side.

Answer 9

So who's in?

Comments

Deadline to register is December 15!! Coming right up!

Let me know if there are any other questions.

---

Given that your are handling somewhat sensitive personal data it makes me question your approach to confidentiality when you haven't implemented https. A SSL-certificate is quite cheap...

---

I just registered and received a confirmation e-mail that contains my password. That is, not only he's not using https but is also storing passwords in plain text.

RUN AWAY!

---

Thanks for your suggestions! With respect to sending the login information to users, we've seen and appreciated this feature in other services; it helps users be able to find their login details if they forget them.

We may be able to add an SSL certificate for next year. Someone had told us that the SSL certificate doesn't "actually mean much." Usually SSL is used in services collecting credit card information or social security numbers, which we don't collect.

---

This is a very unsafe practice. As Sune said, you're handling sensitive data and storing passwords as plain text is a strong indicator that security wasn't top priority in the development of the website.

SSL guarantees that your traffic data isn't being collected by, for example, someone in a public wifi, which is already something relevant. It also guarantees that I know that I'm accessing the "official" econmatch website and not some other site acting as such. I agree that these are less of a concern, but then again caring about these things usually come together with caring about other details in the background.