Exercise: Posting LLM generated reviews
Last updated
Was this helpful?
Last updated
Was this helpful?
Complete
Open the (step 17 in the )
In some cases, a venue may want to use LLMs to post reviews. There are 2 steps to enable this:
Create an LLM reviewer group for each paper.
Add the LLM reviewer groups to each submission reviewers group.
First, you will need to create a new reviewer Group for each paper. This will add the LLM as a Reviewer for the paper.
The reviewer group should have the following format:<venue_id>/Submission<number>/Reviewer_<new_name>
. So it's important to have the id
of the new group follow this format.
For example: MyConference/2025/Conference/Submission1/Reviewer_LLM
The full post_group_edit
call will look like this:
Then you will loop through all of the submissions in the venue.
Check your work: Go to one of the submission groups for the venue and check for your new reviewer group. (https://openreview.net/group/edit?id=<venue_id>/Submission<number>)
Group: The group ID to which you are adding members.
Members: The ID or IDs you are adding to the group.
The full add_members_to_group
call will look like this:
Check your work: Go to one of the submission groups for the venue and check for your new reviewer group in the reviewers group (https://openreview.net/group/edit?id=<venue_id>/Submission<number>/Reviewers)
Check your work: Go to one of the submission pages (you can navigate here from the PC Console), and check for your review.
Check your work: Go to the submission page (you can navigate here from the PC Console), and check that the review was deleted.
Next, you need to add the LLM Reviewers as members to each submission's overall reviewers group. You can do this by calling . It requires 2 fields, group
and members
.
Follow the directions here to using your LLM.
Follow the directions here to using your LLM.