OpenReview
  • Overview
    • OpenReview Documentation
  • Reports
    • Conferences
      • OpenReview NeurIPS 2021 Summary Report
      • OpenReview ECCV 2020 Summary Report
  • Getting Started
    • Frequently Asked Questions
      • I accidentally withdrew a submission, what do I do?
      • How do I add a Program Chair to my venue?
      • When will I be able to withdraw my submission?
      • I want to delete my withdrawn or desk-rejected paper, what do I do?
      • An author of a submission cannot access their own paper, what is the problem?
      • What should I do if I find a vulnerability in OpenReview?
      • How can I report a bug or request a feature?
      • What is the difference between due date (duedate) and expiration date (expdate)?
      • Will Reviewers be notified of their Assignments?
      • What is the max file size for uploads?
      • Why are the "rating" and "confidence" fields in my PC Console wrong?
      • What should I do if my question is not answered here?
      • My Profile is "Limited". What does that mean?
      • What field types are supported in the forms?
      • How do I recruit reviewers?
      • How do I obtain a letter of proof for my services as a reviewer?
      • How do I complete my tasks?
      • Can I automatically transfer my Expertise Selection to another venue?
      • Why does it take two weeks to moderate my profile?
      • What do the different 'status' values mean in the message logs?
      • I am an Independent Researcher, how do I sign up?
      • How do I locate the date a submission is made public?
      • I am a reviewer but I can't access my assigned submissions, what do I do?
      • Reviewers for my venue cannot see their assigned submissions, what should I do?
      • I am a reviewer and I don't have papers for Expertise Selection, what do I do?
      • How do I upload a publication with a license that is not listed?
      • I didn't receive a password reset email, what do I do?
      • How do I add/change an author of my submission after the deadline?
      • How do I find a venue id?
      • Why can't I update my DBLP link?
    • Using the API
      • Installing and Instantiating the Python client
      • Objects in OpenReview
        • Introduction to Groups
        • Introduction to Profiles
    • Hosting a venue on OpenReview
      • Creating your Venue Instance
      • Navigating your Venue Pages
      • Customizing your submission form
      • Enabling Supplementary Material Upload
      • Changing your submission deadline
      • Enabling an Abstract Registration Deadline
    • Creating an OpenReview Profile
      • Signing up for OpenReview
      • Resending an activation link
      • Expediting Profile Activation
      • Add or remove a name from your profile
      • Add or remove an email address from your profile
      • Finding your profile ID
      • Entering Institutional Data
      • Importing papers from DBLP
      • Manually adding a publication to your profile
      • Finding and adding a Semantic Scholar URL to your profile
      • Finding and adding your ACL Anthology URL to your profile
      • Merging Profiles
    • Customizing Forms
    • Using the New Forum Page
    • Live Chat on the Forum Page
  • Workflows
    • Example Workflow
    • ARR Commitment Venues
    • Exercises for workflow chairs
      • Prerequisites
      • Exercise: Posting LLM generated reviews
  • How-To Guides
    • Modifying Venue Homepages
      • How to customize your venue homepage
      • How to modify the homepage layout to show decision tabs
    • Managing Groups
      • How to Recruit and Remind Recruited Reviewers
      • How to have multiple Reviewer or Area Chair groups
      • How to Add and Remove Members from a Group
      • Publication Chairs
      • How to Copy Members from One Group to Another
    • Workflow
      • How to Programmatically Post Support Request Form
      • How to test your venue workflow
      • How to Post a Test Submission
      • How to support different tracks for a venue
      • How to Make Submissions Available Before the Submission Deadline
      • How to Change the Expiration Date of the Submission Invitation
      • Desk Reject Submissions that are Missing PDFs
      • How to begin the Review Stage while Submissions are Open
      • How to Change Who can Access Submissions After the Deadline
      • How to Enable Commenting on Submissions
      • How to Set a Custom Deadline for Withdrawals
      • How to Enable an Ethics Review Stage
      • How to Hide Submission Fields from Reviewers
      • How to modify the Review, Meta Review, and Decision Forms
      • How to release reviews
      • How to Enable the Rebuttal Period
      • How to Undo a Paper Withdrawal
      • How to enable Camera Ready Revision Upload for accepted papers
      • How to make papers public after decisions are made
      • How to enable bidding for Senior Area Chair Assignment
      • How to release the identities of authors of accepted papers only
      • How to enable the Review Revision Stage
    • Paper Matching and Assignment
      • How to Compute Conflicts Between Users
      • How to Post a Custom Conflict
      • How to create your own Conflict Policy
      • How to Bid on Submissions
      • How to add/remove bids programmatically
      • How to do manual assignments
      • How to do automatic assignments
        • How to setup paper matching by calculating affinity scores and conflicts
        • How to run a paper matching
        • How to modify the proposed assignments
        • How to deploy the proposed assignments
        • How to modify assignments after deployment
      • How to enable Reviewer Reassignment for Area Chairs
      • How to Sync Manual and Automatic Assignments
      • How to Compute Affinity Scores
      • How to Undo Deployed Assignments
      • How to Modify Reviewer Assignments as an Area Chair
      • How to Get all Assignments for a User
      • How to Update Custom Max Papers for Reviewers or ACs
      • How to Make Assignments using Subject Areas
    • Communication
      • How to send messages through the UI
      • How to customize emails sent through OpenReview
      • How to send messages with the python client
      • How to Send Decision Notifications Using the UI
      • How to view messages sent through OpenReview
      • How to email the authors of accepted submissions
      • How to get email adresses
    • Submissions, comments, reviews, and decisions
      • How to add formatting to reviews or comments
      • How to submit a Review Revision
      • How to add formulas or use mathematical notation
      • How to edit a submission after the deadline - Authors
      • How to upload paper decisions in bulk
      • How to hide/reveal fields
      • Update camera-ready PDFs after the deadline expires
    • Data Retrieval and Modification
      • How to get edges for Conflicts, Assignments, Custom Max Papers, and more
      • How to check the API version of a venue
      • How to view Camera-Ready Revisions
      • How to Export all Submission Attachments
      • How to create a custom submission export
      • How to add/remove fields from a submission
      • How to manually change the readers of a note
      • How to post/delete an Official Review using Python
      • How to Get Profiles and Their Relations
      • How to Get All the Reviews that I have written and their Corresponding Submissions
      • How to Get All Registration Notes
      • How to Get All Submissions
      • How to Get All Reviews
      • How to Export All Reviews into a CSV
      • How to get all Rebuttals
      • How to Get All Official Comments
      • How to Get All MetaReviews
      • How to Get All Decisions
      • How to Get All Venues
      • How to Retrieve Data for ACM Proceedings
      • How to Get Reviewer Ratings
      • How to Get All Profiles
  • Reference
    • API V1
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Note
          • Fields
        • Invitation
    • API V2
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Group
          • Fields
        • Note
          • Fields
        • Invitation
          • Types and Structure
          • Fields
          • Specifiers
          • Dollar Sign Notation
        • Edit
          • Fields
          • Inference
        • Profile
          • Fields
    • Stages
      • Revision
      • Registration Stage
      • Bid Stage
      • Review Stage
      • Rebuttal Stage
      • Meta Review Stage
      • Decision Stage
      • Comment Stage
      • Submission Revision Stage
      • Post Submission Stage
      • Post Decision Stage
      • Ethics Review Stage
    • Default Forms
      • Default Submission Form
      • Default Registration Form
      • Default Comment Form
      • Default Review Form
      • Default Rebuttal Form
      • Default Meta Review Form
      • Default Decision Form
      • Default Decision Notification
      • Default Ethics Review Form
    • OpenReview TeX
      • Common Issues with LaTeX Code Display
      • OpenReview TeX support
    • Mental Model on Blind Submissions and Revisions
Powered by GitBook
On this page
  • Getting all Conflict Edges for the Reviewers Group
  • Mapping reviewer IDs to their conflict edges
  • Mapping paper IDs to their conflict edges
  • How to get other types of edges
  • Optional parameters and other use cases
  • Using label to map reviewer IDs to their proposed assignments
  • Using select to map reviewer IDs to their assignment count

Was this helpful?

Export as PDF
  1. How-To Guides
  2. Data Retrieval and Modification

How to get edges for Conflicts, Assignments, Custom Max Papers, and more

PreviousData Retrieval and ModificationNextHow to check the API version of a venue

Last updated 5 days ago

Was this helpful?

If you want to get all edges for any type of Edge Invitation you can use the API call. The most common edges to retrieve are for Conflicts, Assignments and Custom Max Papers.

get_grouped_edges requires at least 2 fields:

  • : The ID of the Edge Invitation.

  • Groupby: The field of the you want to group the edges by. Typically you will group by the edge's head or tail. The value of head and tail are determined by their Invitation. You can view the invitation by going to: https://openreview.net/invitation/edit?id=invitation_id

get_grouped_edges will return list of dictionaries each containing 2 keys: id and values. The id will contain the field you grouped by and values will contain all the edges in that group.

Getting all Conflict Edges for the Reviewers Group

This is an example of how to use get_grouped_edges to retrieve Reviewer Conflict edges. You can use this method to get any type of Edge.

Mapping reviewer IDs to their conflict edges

  1. Get the conflict invitation ID for the API call:

Get the venue group. The content of the venue group contains many settings for your venue including invitation IDs. The Reviewer Conflict invitation ID is stored as reviewers_conflict_id in the content. It follows the format: venue_id/reviewers_name/-/Conflict.

venue_group = client.get_group(venue_id)
conflict_invitation_id = venue_group.content['reviewers_conflict_id']['value']
  1. Get the edges using get_grouped_edges:

Call get_grouped_edges, pass the invitation ID, and group by the tail. Conflict edges are configured so that the head is the paper ID and the tail is the user with the conflict. When you group by the head you can access all the conflicts for a specific paper. Alternatively, if you group by the tail, you can access all conflicts for a specific user:

grouped_edges = client.get_grouped_edges(
    invitation = conflict_invitation_id,
    groupby = 'tail'
)
  1. Map each reviewer profile ID with their conflict edges:

rev_conflict_map =  { group['id']['tail']: group['values'] for group in grouped_edges }

Mapping paper IDs to their conflict edges

If you want to see all conflicts for a specific paper, group by the head instead.

grouped_edges = client.get_grouped_edges(
    invitation = conflict_invitation_id,
    groupby = 'head'
)

paper_conflict_map =  { group['id']['head']: group['values'] for group in grouped_edges }

How to get other types of edges

You can use the same method to get any type of Edge by simply changing the invitation ID. For example:

  • To get all Reviewer Assignment edges, pass the Assignment invitation ID:

assignment_invitation_id = venue_group.content['reviewers_assignment_id']['value']
  • To get all Reviewer Custom Max Paper edges, pass the Custom Max Papers invitation ID:

custom_max_papers_invitation_id = venue_group.content['reviewers_custom_max_papers_id']['value']
  • To get all Reviewer Affinity Score edges, pass the Affinity Score invitation ID:

affinity_score_invitation_id = venue_group.content['reviewers_affinity_score_id']['value']

To get the corresponding invitation IDs for Area Chairs or Senior Area Chairs, just replace reviewers with area_chairs or senior_area_chairs. For example, area_chairs_assignment_id or senior_area_chairs_assignment_id. If your venue uses other role names, those names will automatically get replaced in the invitation IDs.

Optional parameters and other use cases

It may be helpful to pass other parameters to get_grouped_edges to further filter the edges.

Using label to map reviewer IDs to their proposed assignments

Some Edge Invitations allow for edges to be posted with a label. To know if this is possible, you have to examine the invitation and see if label is configured. If so, you may pass label='Some label' to get_grouped_edges to retrieve only a subset of those edges.

This is often used when retrieving Proposed Assignment edges. The matcher will post these edges where the label is the title of the matching configuration:

proposed_assignment_invitation_id = venue_group.content['reviewers_proposed_assignment_id']['value']

grouped_edges = client.get_grouped_edges(
    invitation = proposed_assignment_invitation_id,
    groupby = 'tail',
    label = 'rev-matching-1'
)

rev_proposed_assignment_map =  { group['id']['tail']: group['values'] for group in grouped_edges }

Using select to map reviewer IDs to their assignment count

You can pass select = 'count' to get the number of edges for each grouped entity. In this case, we will group by the user ID with tail so that we can map them to their assignment counts.

grouped_edges = client.get_grouped_edges(
    invitation = assignment_invitation_id,
    groupby = 'tail',
    select = 'count'
)

rev_assignment_count_map =  { group['id']['tail']: group['count'] for group in grouped_edges }

Notice how group['values'] changed to group['count'] in the mapping. This happens only when we select the count.

You may also select by any field(s) of the Edge if you don't need to retrieve all the data in the edges. For example, you may pass select='head,tail' if you only want to see the head and tail of the edge.

get_grouped_edges
Invitation
Edge