How to run a paper matching
In order to automatically assign Reviewers and Area Chairs, you must:
  1. 1.
    Enable the 'Review' or 'Post Submission' stage from your venue request form. This can only be done AFTER the submission deadline has passed.
  2. 2.
    Use the 'Paper Matching Setup' button on your venue request form to calculate affinity scores and conflicts.
After you complete these steps, a link for 'Paper Assignments' should appear on your Program Chair console.
Clicking on one of the assignment links will bring you to the assignment page, where you can create a new matching configuration. If members of your reviewer or area chairs group have profiles without publications, you will need to select ‘Yes’ for ‘Allow Zero Score Assignments’ in order to obtain a solution. Please note that all members of a group must have OpenReview profiles in order for the automatic assignment algorithm to run. Any members without profiles must be removed from the group before this step.
You can learn more about our automatic paper matching algorithm from its github repo: https://github.com/openreview/openreview-matcher. To create a new matching, click the 'New Assignment Configuration'. This will pull up a form with some default values pertaining to your matching settings:
User Demand
Max Papers
Min papers
Alternates
Paper Invitation
Match Group
Scores Specification
Conflicts Invitation
Custom User Demand Invitation
Custom Max Papers Invitation
Solver
Allow Zero Score Assignments
Randomized Probability Limits
After filling out the matching configuration form and hitting submit, you should see the following:
You can view, edit or copy the values you filled out in the matching form. When you are happy with your configuration, you should hit 'Run Matcher' and wait until its status is 'Complete'. This generates proposed assignments, with options to browse assignments, view statistics or deploy matching. If you click ‘Browse Assignments’ you will be brought to the edge browser, where you can browse, edit, and create proposed assignments.
If you get "No Solution" after running the matcher, you can view the configuration to see the entire error message. If the message is something like the following:
  • Error Message: Total demand (150) is out of range when min review supply is (34) and max review supply is (100)
that means that your constraints require more reviewers or area chairs than you currently have. The total demand is equal to (number of submissions * user demand) + (number of submissions * alternates). The max review supply is the number of reviewers available * max papers and the review supply is the number of reviewers available * min papers. Your total demand must fall within this range in order to obtain a solution.
Note that completion of this step does not make assignments, it only creates a proposed assignment configuration. Those assignments will need to be deployed before Reviewers or Area Chairs will see them.
Export as PDF
Copy link