To post a new Note, create a Note object with required fields, then use client_v2.post_note_edit().
Required fields include:
invitation: the invitation ID defining the note type (e.g., a submission, review, comment)
forum: the ID of the main submission this note belongs to (for replies)
signatures: who is posting (usually a profile ID)
readers: who can read this note
writers: who can modify the note
content: a dictionary of the note content fields (title, authors, review text, etc.)
Quickstart: Posting a submission with Python
Posting a submission with Python is typically reserved for testing venue workflows and changes.
PCs and Reviewers that are also authors of test submissions will be conflicted later on in the process. We recommend using test author profiles for the submission process.
QuickStart: Posting a Support Request Form With Python
While a support request form can most easily be submitted through the UI, some venues that have multiple deadlines a year and need to submit multiple venue requests with the same settings may find it easier to do this programmatically through the API. While most notes are posted with the API2 client, this note must be posted with the API 1 client.
#change the settings below
program_chair_emails = ['pc_email1@mail.com','pc_email2@mail.com']
content = {
"title": "My test venue",
"Official Venue Name": "My test venue",
"Abbreviated Venue Name": "MTV",
"Official Website URL": "mtv.com",
"program_chair_emails": program_chair_emails,
"contact_email": "pc@venue.org",
"Area Chairs (Metareviewers)": "Yes, our venue has Area Chairs",
"Paper Matching": [
"Reviewer Bid Scores",
"OpenReview Affinity"
],
"Venue Start Date": "2022/05/01",
"Author and Reviewer Anonymity": "Double-blind",
"reviewer_identity": [
"Assigned Area Chair"
],
"submission_readers": "All program committee (all reviewers, all area chairs, all senior area chairs if applicable)",
"venue_organizer_agreement":['OpenReview natively supports a wide variety of reviewing workflow configurations. However, if we want significant reviewing process customizations or experiments, we will detail these requests to the OpenReview staff at least three months in advance.',
'We will ask authors and reviewers to create an OpenReview Profile at least two weeks in advance of the paper submission deadlines.',
'When assembling our group of reviewers and meta-reviewers, we will only include email addresses or OpenReview Profile IDs of people we know to have authored publications relevant to our venue. (We will not solicit new reviewers using an open web form, because unfortunately some malicious actors sometimes try to create "fake ids" aiming to be assigned to review their own paper submissions.)',
"We acknowledge that, if our venue's reviewing workflow is non-standard, or if our venue is expecting more than a few hundred submissions for any one deadline, we should designate our own Workflow Chair, who will read the OpenReview documentation and manage our workflow configurations throughout the reviewing process.",
'We acknowledge that OpenReview staff work Monday-Friday during standard business hours US Eastern time, and we cannot expect support responses outside those times. For this reason, we recommend setting submission and reviewing deadlines Monday through Thursday.',
'We will treat the OpenReview staff with kindness and consideration.']
}
support_request = openreview.Note(
invitation = 'OpenReview.net/Support/-/Request_Form',
readers = ["OpenReview.net/Support"] + program_chair_emails,
writers = ["OpenReview.net/Support"] + program_chair_emails,
content = content,
signatures = [
<your_profile_id>
]
)
client_v1.post_note(support_request)
2. Editing Notes
Editing notes in API2 is done the same way as creating notes, by posting a note edit with new values for the fields.
Quickstart: Update the readers of a note or field
One common edit to make in bulk to notes is updating the readers of a note. This can be done in the UI for common configurations, but if your venue needs further fine-grained control over readership (for example custom tracks, groups, etc), then you can use the Python Client to change the readers of any subset of notes programmatically. The following script will change the readers of a note or field within a note to the new list.
#venue_id
venue_id = "<VENUE_ID>"
note_type = 'Official_Review' #Taken from the last part of the note invitation. Could replace with "Rebuttal", "Decision", "Metareview"
new_readers = ["<LIST_OF_NEW_READERS>"] # options are 1) ['everyone'] or 2) list of either group names ("Program_Chairs") or Submission + Group Name ("Submission Authors").
field_to_change = None #use None to change readers of the note, or use the field name to change the readers of a field
#set up invitations
edit_invitation = f'{venue_id}/-/Edit'
submission_invitation = f'{venue_id}/-/Submission'
#get a list of notes to edit (in this case all reviews)
submissions = client_v2.get_notes(invitation=submission_invitation,details = 'replies')
#reviews to edit
reviews=[openreview.api.Note.from_json(reply) for s in submissions for reply in s.details['replies'] if f'{venue_id}/Submission{s.number}/-/{note_type}' in reply['invitations']]
print(len(reviews))
notes_to_change = reviews
#change the readers of the notes
for note_to_change in notes_to_change:
submission_number = client_v2.get_note(note_to_change.forum).number
note_content = note_to_change.content
if new_readers == ['everyone']:
note_readers = ['everyone']
else:
#parse the new_readers:
group_ids = []
for g in new_readers:
if g.startswith('Submission'):
group_name = g.split(' ')[1]
new_reader_group = client_v2.get_group(f'{venue_id}/Submission{submission_number}/{group_name}').id
group_ids.append(new_reader_group)
else:
new_reader_group = client_v2.get_group(venue_id + '/' + g)
group_ids.append(new_reader_group.id)
#parse the new_readers:
if field_to_change == None:
note_readers = list(set(group_ids + note_to_change.signatures + [venue_id]))
else:
note_content[field_to_change]['readers'] = list(set(group_ids + note_to_change.signatures + [venue_id]))
client_v2.post_note_edit(invitation=edit_invitation,
signatures=[venue_id],
note=openreview.api.Note(
id=note_to_change.id,
readers=note_readers,
content=note_content
)
)
3. Deleting Notes and fields
Deleting notes should be used with caution as most changes can be done by editing a note, and once deleted, changes can be difficult to recover. Typically, we recommend deleting notes only in the case of testing a venue workflow on the dev site. To delete a note, you would post a note edit that includes a value (in millisecond time) for the ddate field, which has a value of None by default. An example is below:
Removing fields from submissions should be done sparingly. Oftentimes, changing the readers of the field its sufficient (see above), but in the case that you want to remove a field entirely, you can use the approach below"
#edit the review note
anon_groups = client_v2.get_groups(prefix=f'{test_venue_id}/Submission{submission_number}/Reviewer_', signatory='~Reviewer_One1')
anon_group_id = anon_groups[0].id
review_notes = client_v2.get_notes(
invitation=f'{venue_id}/Submission{submission_number}/-/Official_Review',
signatures=[anon_group_id]
)
# Assuming there's one review per reviewer
original_review = review_notes[0]
#Update the content of the review with the new content
review_content = original_review.content
review_content['rating']['value'] = 6
# Step 2: Edit the review to update the rating
edited_review = client_v2.post_note_edit(
invitation=f'{venue_id}/Submission1/-/Official_Review',
signatures=[anon_group_id],
note=openreview.api.Note(
id=original_review.id,
forum=original_review.forum,
replyto=original_review.replyto,
content=review_content
)
)
# Assuming the first one is the one we want to delete
note_to_delete = review_notes[0]
# Step 2: Post a deletion edit - to delete the whole review
deleted_note = reviewer_client.post_note_edit(
invitation=f'{test_venue_id}/Submission1/-/Official_Review',
signatures=[anon_group_id],
note=openreview.api.Note(
id=note_to_delete.id,
content=note_to_delete.content,
ddate = openreview.tools.datetime_millis(dt.datetime.now()))
)
