> This is part 1 of 5 of the full documentation (pages 1–100 of 458).
> The content is paginated: fetch every part to see all of it.
> Next part: https://docs.openreview.net/llms-full.txt/1
> Page index: https://docs.openreview.net/llms.txt

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
This site has an agentic ask interface you may use to query the documentation dynamically by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.openreview.net/index.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

---

# OpenReview Documentation

The OpenReview Documentation is divided into 4 main sections:

* [Getting Started](https://docs.openreview.net/getting-started/frequently-asked-questions): Contains the FAQ, how to create a Venue, how to create a profile, and how to interact with the API.
* [Workflows](https://docs.openreview.net/conference-review-workflow/overview): An overview of a typical conference workflow with step by step instructions for configuring each stage of your venue.
* [How-to Guides](https://docs.openreview.net/how-to-guides/modifying-venue-homepages): Mainly for Venue organizers that want to setup different parts of the workflow.
* [Reference](https://docs.openreview.net/reference/api-v2): Contains a technical reference on how to use more advanced features of OpenReview.

{% hint style="info" %}
You can use the Search bar in the top right corner to search for keywords in the entire OpenReview documentation.
{% endhint %}


# Overview

### 1. Introduction

This guide is for venue organizers using the new workflow configuration UI in OpenReview. This guide will walk through an overview of how to best interact with the UI.

{% hint style="info" %}
To see an example of a configured UI in an existing venue, use the following link and credentials:\
\
Link: <https://dev.openreview.net/group/edit?id=ICML.cc/2025/Workshop/DataWorld#workflowInvitations> email: <pc@midl.io>\
password: 1234
{% endhint %}

### **2. Accessing the Workflow Console**

Once your venue is created and you're assigned as a Program Chair, navigate to the console by:\
`Program Chairs Console → Workflow Setup Timeline`

This brings you to the **Workflow Setup Timeline** tab, where you can:

* View user groups (reviewers, authors, chairs)
* Configure review stages (bidding, assignments, reviews, decisions)
* Set email notifications and visibility options
* Automate release of submissions, reviews, and decisions

### **3. Understanding the UI Structure**

#### **3.1 Workflow Groups**

This section let's you view the members of major group in the venue.

* **Program Chairs** – oversee all configurations and decisions
* **Reviewers** – assigned to review submissions (also used for recruiting reviewers)
* **Authors** – those submitting papers
* **Automated Administrator** – used for auto-execution of stages like conflict detection

#### **3.2 Program Chair Configuration Tasks**

This section includes tasks to help guide the Program Chair through configuring major steps of the venue. Clicking on each task will take you to the relevant section of the workflow.

#### **3.3 Workflow Configurations**

This section contains a **chronological** list of stages in the submission and review pipeline. Each row shows:

* A **stage title** (e.g., “Create Reviewers Conflict”)
* An **invitation** label showing who can access it
* The **Enable / Disable** toggle
* Execution status (e.g., *Executed*, *Scheduled*, *Expired*)
* A link to view or edit the configuration form
* A color - Green means the stage is complete, Blue means stage is in progress, Grey means not started yet

### **4. How to Edit a Workflow Stage**

You can customize each stage via its configuration form.

1. Locate the stage in the **Workflow Configurations** list.
2. If not yet active, click **Enable**.
3. Click the **stage title** (e.g., “**Create Author Decision Notification**”).
4. In the configuration form you use the Edit button to change settings such as:
   * Set or adjust start and end dates
   * Customize email messages
   * Choose which users have access
   * Define visibility and data behavior
5. Click **Submit** or **Save** to apply changes. This will also re-order the stages, if the dates were changed.

#### **Notes:**

* Read the description of each invitation and field to understand how it works in the workflow.
* Re-execution of stages like “Create Reviewers Affinity Score” may overwrite existing data.
* When editing forms, use the '**Preview**' tab to see what the form will look like.

### **5. Key Workflow Stages Overview**

#### **Submissions**

* **Submission**: Opens the form for authors to submit papers. This invitation is time-bound and should end before reviewer assignment begins.
* **Withdrawal**: Enables authors to withdraw papers.
* **Desk Rejection**: Allows chairs to reject papers prior to review.

#### **Reviewer Assignment**

* **Reviewers Bid**: Enables reviewers to bid for reviewing specific papers.
* **Create Reviewers Conflict**: Detects institutional or co-authorship conflicts.
* **Create Reviewers Affinity Score**: Computes expertise match scores using OpenReview's models.
* **Create Reviewers Submission Group**: Creates a reviewer group for each paper. These groups record which reviewers are assigned to a submission, and need to be created before assignment begins.
* **Create Reviewer Assignment Deployment**: Finalizes reviewer-paper assignments based on prior bidding, affinity, or manual assignment.

#### **Review Stage**

* **Create Author Reviews Notification**: Defines and sends notification emails to authors when reviews are ready.
* Replies to Submissions
  * **Official Review**: Launches the review form reviewers must fill out.
  * **Official Comment:** Allows for comments to be made on the submission
  * **Author Rebuttal**: (If enabled) lets authors respond to reviews.
* Stats:
  * **Create Reviewers Review Assignment Count:** Count number of assigned reviews for each reviewer
  * **Create Reviewers Review Days Late:** Compute the days late for reviewers
  * **Create Reviewers Review Count:** Compute the number of reviews per reviewer

#### **Decision Stage**

* **Create Decision Upload**: Upload final decisions in batch (e.g., from a CSV).
* **Create Author Decision Notification**: Customize notification email content for decisions.
* **Decision Release**: Specifies the release time and visibility settings for decisions.

#### **Final Submissions**

* **Camera Ready Revision**: Opens the form for accepted papers to be revised for camera-ready versions.
* **Create Submission Release**: Controls which submissions are made public and under what license.

### **6. Customization Tips**

* Review and customize all stages prior to starting your venue - stages can now be scheduled to release ahead of time.
* Automated stages (e.g., affinity score, conflict detection) are executed by the **Automated Administrator** group.


# Example Workflow

Example workflow in the style of the existing ones

## 1. Request a Venue

To host a venue on OpenReview—such as a conference, workshop, or symposium—you must first submit a request via the **OpenReview Support Request Form**. This form is used to collect basic information about your venue and define the initial configuration of your review workflow.

#### Steps:

1. **Go to the Venue Request Page**:

   Visit [OpenReview Request Page](https://dev.openreview.net/group?id=OpenReview.net/Support#tab-venue-configuration-requests).
2. **Open the Request Form**:

   Select the **"Conference Review Workflow"** form under the *Venue Configuration Requests* tab.
3. **Fill in the Required Details**
4. **Submit the Form**:

   Once submitted, the OpenReview team will review your request. They may reach out to confirm details prior to deploying your venue.
5. **Venue Setup**:

   After approval, a venue group and initial setup will be created in the system. You will gain access to the **Program Chairs Console** for further configuration and management of the workflow.

{% hint style="info" %}
To see an example of a configured UI in an existing venue, use the following link and credentials:\
\
Link: <https://dev.openreview.net/group/edit?id=ICML.cc/2025/Workshop/DataWorld#workflowInvitations> email: <pc@midl.io>\
password: 1234
{% endhint %}

## 2. Complete Workflow Configuration Tasks

After your venue is created, the Program Chairs are responsible for configuring key stages of the peer review workflow. These tasks are listed in the **Workflow Console** under **“Program Chairs Configuration Tasks”** along with their respective deadlines.

Each tasks appears as a clickable item in the configuration panel. Once opened, they present a web form where settings can be customized and saved.

## 3. Review and Configure Workflow Steps

Once the Workflow Configuration Tasks are complete, also review and make any necessary changes to the steps of the workflow. Some stages will run automatically once their activation date passes, so we recommend configuring the stages as early as possible.

## 4. Recruitment

Reviewer recruitment in OpenReview is handled through the **Reviewers Recruitment** workflow stage. This task enables Program Chairs to invite users to participate in the review process—typically as **reviewers**, but potentially also for other roles like **area chairs** or **senior area chairs**, depending on the venue setup.

#### How to Recruit:

1. Go to the **Reviewers Group** in the Workflow Group console
2. Click the **Recruitment Request** button and fill out the Recruitment Form, including users to be invited, and the recruitment template.
3. **Submit changes**

   Once submitted, OpenReview will send personalized email invitations containing links where recipients can accept or decline.

#### Notes:

* Recruitment **can be done at any time**, but it's ideal to complete it well before the review stage begins.
* **Recruitment is independent from assignment matching**. Reviewers can be invited first, and **matching is performed later** once enough reviewers have accepted and submissions are in.
* The Recruitment page can also be used to send **recruitment reminders,** which will send an email to any users that have not already responded to the invitation

## 5. Assignment

Once recruitment is complete and submissions are in, Program Chairs can assign reviewers to papers using the **Create/Deploy Reviewer Assignment** task in the Program Chairs Console.

Assignments are not configured automatically, and must be created by the program chairs.

**How to Assign Reviewers:**

1. Make sure that **Reviewers Conflict** and **Reviewers Affinity Score** have been calculated
2. **Open the Assignment Tool**\
   Click on **“Create Reviewer Assignment”** in the workflow tasks list.\
   This page contains:
   * A link to **create draft assignments**
   * An option to **edit** and configure matching parameters
   * A link to deploy assignments
3. **Create a Draft Match**\
   In the draft assignment interface, you can:
   * Select the pool of eligible reviewers.
   * Choose the matching settings and constraints like maximum and minimum number of papers per reviewer.
4. **Review and Adjust**\
   After generating the draft, review assignments for conflicts of interest, balance, and workload. You can manually reassign reviewers if needed.
5. **Deploy Assignments**\
   Once the assignments are finalized, return to the **Create Reviewer Assignment Deployment** page.
   * Select the correct **Match Name** from the dropdown.
   * Choose a **Deploy Date** (or deploy immediately).
   * Click **Deploy** to make the assignments official.

**Notes:**

* Matching is **done after recruitment** but before the review stage opens.
* Errors during deployment (as seen in the UI) should be resolved by checking that:
  * The draft match exists and is selected.
  * Reviewer groups are correctly populated.
  * Constraints are met

## 6. Monitor Venue

After assignments are made, the rest of the conference workflow will run according to the settings and dates in the conference workflow. Monitor these stages and change them as necessary throughout the rest of the venue workflow.


# Exercises for workflow chairs

These exercises are designed to guide workflow chairs through specific tasks that they will need to be able to do in order to successfully support a venue. To use this page, first complete the [Prerequisites](https://docs.openreview.net/conference-review-workflow/exercises-for-workflow-chairs/prerequisites), then complete the relevant exercise below:

Basic Exercises

* [Understanding Notes](https://docs.openreview.net/conference-review-workflow/exercises-for-workflow-chairs/exercise-understanding-notes)

Advanced Exercises

* [Posting LLM generated reviews](https://docs.openreview.net/conference-review-workflow/exercises-for-workflow-chairs/exercise-posting-llm-generated-reviews)


# Prerequisites

All exercises should be practiced on the dev site. For any exercise, you will need to have the following prerequisites set up.

1. Review the documentation [here](https://docs.openreview.net/how-to-guides/workflow/how-to-test-your-venue-workflow) and create a venue request for the dev site.
2. Instantiate the python client using your dev profile credentials:

```python
import openreview
# API 1 client
dev_client_v1 = openreview.Client(
    baseurl='https://api.dev.openreview.net',
    username=dev_username,
    password=dev_pass
)

# API 2 client
dev_client_v2 = openreview.api.OpenReviewClient(
    baseurl='https://api2.dev.openreview.net',
    username=dev_username,
    password=dev_pass
)
```

3. Recommended: Familiarize yourself with the [Example Workflow](https://docs.openreview.net/conference-review-workflow/new-venue-ui-beta)


# Exercise: Posting LLM generated reviews

## Setup:

1. Complete [Prerequisites](https://docs.openreview.net/conference-review-workflow/exercises-for-workflow-chairs/prerequisites)
2. [Add submissions](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) (make sure this is on the dev site)
3. Open the **Official Review** step: in the Workflow Timeline, click **Edit** next to its **Dates** and set the **Activation Date** to the current time (see the [Example Workflow](https://docs.openreview.net/conference-review-workflow/new-venue-ui-beta))

## Posting LLM Generated Reviews Overview

In some cases, a venue may want to use LLMs to post reviews. This exercise assumes the venue already has access to an LLM that generates the review and covers how to post them as replies to submissions.

* Create a `venue_id/-/AI_Review` invitation using the [Custom Stage](https://docs.openreview.net/how-to-guides/workflow/using-the-customstage).
* **(Optional)** Create an AI Reviewer group to sign the AI Reviews.
  * Venues may want this to make it more clear that it's an AI Review, for example:

    <figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FKWMi95nvZNX4CV70bWhL%2Fai-review-note-heading.png?alt=media&#x26;token=fbda4f1f-8f19-49a6-9609-0697dccda77d" alt=""><figcaption></figcaption></figure>
  * If you choose to do this, you can either create:
    * One overall AI Reviewer group (`venue_id/AI_Reviewer`) to sign all the notes
    * Or create an AI Reviewer group per paper (`venue_id/SubmissionXX/AI_Reviewer`)
  * Otherwise, the notes will be signed with the Program Chairs group.
* Write a script to iterate through all submissions and post the AI Review to each paper.

### 1. Create the AI Review invitation

You can use the [`CustomStage`](https://docs.openreview.net/how-to-guides/workflow/using-the-customstage) to create the `venue_id/-/AI_Review` invitation and you will post your LLM generated reviews using this invitation.

The invitees can be the PCs and you can configure the other settings however you need (e.g. form content, readers, etc.).

### 2. (Optional) Create an AI reviewer group and update the AI Review invitation

#### Creating the AI reviewer group(s)

If you don't want to sign the AI reviews using the PC group, you can create AI Reviewer group(s). Otherwise you can skip this step.

The following creates a single AI Reviewer group you can use to sign all the reviews:

```python
client.post_group_edit(
    invitation='<venue_id>/-/Edit',
    signatures=['<venue_id>'],
    group=openreview.api.Group(
        id = '<venue_id>/AI_Reviewer',
        readers = ['<venue_id>'],
        signatures = ['<venue_id>'],
        writers = ['<venue_id>'],
        signatories = ['<venue_id>'],
    )
)
```

Alternatively, you can loop through [all submissions](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-get-all-notes-for-submissions-reviews-rebuttals-etc#quickstart-getting-all-submissions) and create an AI Reviewer group for each paper. Note the change in the `id` format. The `venue_id/SubmissionXX` group contains all the groups for that paper, including the assigned reviewers, ACs, and now the AI Reviewer.

```python
for sub in submissions:
    client.post_group_edit(
        invitation='<venue_id>/-/Edit',
        signatures=['<venue_id>'],
        group=openreview.api.Group(
            id = '<venue_id>/Submission<sub.number>/AI_Reviewer',
            readers = ['<venue_id>'],
            signatures = ['<venue_id>'],
            writers = ['<venue_id>'],
            signatories = ['<venue_id>'],
        )
    )
```

**Check your work:** Go to one of the submission groups for the venue and check for your new reviewer group. (`https://openreview.net/group/edit?id=<venue_id>/Submission<number>`)

#### Updating the AI Review invitation

If you decide to use AI Reviewer groups, you will also need to update the AI Review invitation to allow you to sign the notes using the AI Reviewer group. The signature is different depending on whether you created 1 AI Reviewer group or a group for each paper.

The following posts an edit to the `/-/AI_Review` invitation. Note that the signature for the per-paper AI Reviewer groups use [dollar sign notation](https://docs.openreview.net/reference/api-v2/entities/invitation/dollar-sign-notation) to retrieve the submission number:

```python
ai_review_inv_id = f"{venue_id}/-/AI_Review"

## Signature for single overall AI Reviewer group
ai_reviewer_group_id = f"{venue_id}/AI_Reviewer"

## Signature for per-paper AI Reviewer groups
# ai_reviewer_group_id = f"{venue_id}/Submission${{7/content/noteNumber/value}}/AI_Reviewer"

client_v2.post_invitation_edit(
    invitations=f'{venue_id}/-/Edit',
    readers=[venue_id],
    writers=[venue_id],
    signatures=[venue_id],
    invitation=openreview.api.Invitation(
        id=ai_review_inv_id,
        edit={
            "invitation": {
                "edit": {
                    "signatures": {
                        "param": {
                            "items": [ { "value": ai_reviewer_group_id, "optional": True } ]
                        }
                    }
                }
            }
        }
    )
)
```

### 3. Post reviews using your LLM

Follow the directions here to [post reviews](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) using your LLM. You should be able to sign the notes using the PC group or the AI\_Reviewer group(s) you created.

**Check your work**: Go to one of the submission pages (you can navigate here from the PC Console), and check for your review.

### 4. Delete an LLM review

Follow the directions here to [delete reviews](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) using your LLM.

**Check your work**: Go to the submission page (you can navigate here from the PC Console), and check that the review was deleted.


# Exercise: Understanding Notes

#### Objectives

Learn to use the OpenReview API to:

1. Post new notes
2. Retrieve a specific note
3. Fetch all submissions for a given venue
4. Edit a submission (readers and content)
5. Delete a note

Use the [OpenReview documentation](https://docs.openreview.net/getting-started/using-the-api/objects-in-openreview/introduction-to-notes) and linked guides as your primary reference. Ensure that you have instantiated the API client in a variable `client` before beginning.

#### 1. Post New Test Submissions

**Task**: Create and post at least two submissions to a venue of your choice.

**Hints**:

* See [**How to create, change, and delete notes**](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) — *Posting a submission with Python*.

**Check your work:**\
Open the [PC Console](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages) for your venue. Your new submissions should appear under the Submission Status tab.

#### 2. Get a Single Note

**Task**: Fetch and inspect one of the submissions you just posted by its note id.

**Hints**:

* Read [**Introduction to Notes**](https://docs.openreview.net/getting-started/using-the-api/objects-in-openreview/introduction-to-notes).
* Look up [`get_note(id)` in the API reference](https://openreview-py.readthedocs.io/en/latest/api.html#openreview.api.OpenReviewClient.get_note).

**Check your work:**\
Run the following code, the output should match the note ID you requested.

```python
print(note.id)
```

#### 3. Get All Submissions for a Venue

**Task**: Retrieve all submissions for your chosen venue.

**Hints**:

* See [**How to get all notes for submissions, reviews, rebuttals, etc.**](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-get-all-notes-for-submissions-reviews-rebuttals-etc).

**Check your work:**\
Run the following code, the output should match the number of submissions in your venue.

```python
print(len(submissions))
```

#### 4a. Edit a Submission - Title

**Task**: Choose one of your submissions and edit the title\
**Hints**:

* Same guide: [**How to create, change, and delete notes**](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) — *Update the content of a note*.

**Check your work:**\
Check the submission page on the OpenReview site. The submission should show the new title. When you click 'Show Revisions', you should see the old title in the original edit, and the current title in a more recent edit.

#### 4b. Edit a Submission - Readers

**Task**: Choose one of your submissions and change the readers to \['everyone'], then change it back to its original settings\
**Hints**:

* Same guide: [**How to create, change, and delete notes**](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) — *Update the readers of a note*

**Check your work:**\
Check the submission page on the OpenReview site. There should be a property just under the title of the submission with the readers of the note listed.

#### 4c. Edit a Submission - Field Readers

**Task**: Choose one of your submissions and edit the readers of the `authors` field\
**Hints**:

* Same guide: [**How to create, change, and delete notes**](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) — *Update the field readers of a note*

**Check your work:**\
Check the submission page on the OpenReview site. There should be an eye icon next to the edited field saying who the readers of the field are

#### 5. Delete a Note

**Task**: Remove one of your test submissions from the system.

**Hints**:

* Same guide: [**How to create, change, and delete notes**](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes) — *Delete a note*.

**Check your work:**\
Run the following code, the output should indicate the note does not exist

```python
client.get_note("<NOTE_ID>")
```


# Exercise: Getting profile information

## Setup:

1. Complete [Prerequisites](https://docs.openreview.net/conference-review-workflow/exercises-for-workflow-chairs/prerequisites)

## Part 1: Getting all profiles from a list

### 1. Create a list of profiles to query

First, create a `list` of profiles to query. This could be a custom list, all authors of the submissions, all members of a group, such as reviewers, etc. This should be a list of profile IDs.

Some useful examples from the documentation:

* [How to get all author profiles](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-loop-through-accepted-papers-and-print-the-authors-and-their-affiliations)
* [How to use get\_group](https://docs.openreview.net/how-to-guides/managing-groups/copying-members-from-one-group-to-another)

**Check your work:** Print the list of profiles. You should see a list of profile IDs in the format `['~First_Last1', ...]`

### 2. Get all profiles from that list

Review the example [here](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-profiles) to get profiles from a list. This will return a list of profile objects. The relevant code is:

```python
profiles = openreview.tools.get_profiles(client_v2,profile_id_list,as_dict=True)
```

**Check your work:** The result should be a dictionary with the query profile ID and the OpenReview Profile Object.

### 3. Parse the profiles

Review the example [here](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-profiles) to understand how profile information is stored, and one example of how to flatten and tabulate the data.

Profiles don't include email addresses. To get the preferred email of each user in your venue, pass your venue's preferred emails invitation:

```python
profiles = openreview.tools.get_profiles(client_v2, profile_id_list, with_preferred_emails='<venue_id>/-/Preferred_Emails', as_dict=True)
```

The preferred email is then in `profile.content['preferredEmail']`.

**Check your work**: Use Python to get the profile ID property, the preferred email, and the current institutional affiliation. Then print the profile information and check that the data you extracted is correct.


# Exercise: Checking Registration Completion

#### Objectives

Learn to use the OpenReview API to:

1. Retrieve members of a group
2. Check whether reviewers have accepted the invitation
3. Check whether reviewers have completed registration
4. Sentd a message to users who haven't completed registration

**Setup:** Complete [Prerequisites](https://docs.openreview.net/conference-review-workflow/exercises-for-workflow-chairs/prerequisites)

### 1. Retrieve

Using the API, retrieve the notes for Reviewer Registration. The invitation is typically in the format `<YOUR_VENUE_ID>/Reviewers/-/Registration`

**Resources:** [Working with notes](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-get-all-notes-for-submissions-reviews-rebuttals-etc)

The signature of each note is a list containing the profile ID of the user who submitted the note. We use these signatures to identify who has completed the registration.

<pre class="language-python"><code class="lang-python">##assuming a registration_notes object
<strong>users_registration_completed = [r.signatures[0] for r in registration_notes]
</strong></code></pre>

Users may have multiple profile IDs, so it is important to normalize profile IDs when note signatures. To do so, first retrieve the profiles for members of the `users_registration_completed` list from Step 1 above.

**Resources:** See [this exercise](https://docs.openreview.net/conference-review-workflow/exercises-for-workflow-chairs/exercise-getting-profile-information) and [Introduction to Profiles](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-profiles)

```python
profiles = client.get_profiles(users_registration_completed)
normalized_profiles = [p.id for p in profiles]
```

**Check your work:** You should have a list of profile ids of the users

### 2. Retrieve Reviewer Group

Next, retrieve the members of the Reviewer group for your venue. This group shows all reviewers that have accepted the invitation to review for your venue and hence received the registration task. Normalize the reviewer profiles as well.

Use the function `client.get_group()` . Typically the reviewer group ID is `<YOUR_VENUE_ID>/Reviewers` .

```python
reviewer_group = client.get_group('GROUP_ID').members
```

**Resources:** See [Introduction to Groups](https://docs.openreview.net/getting-started/objects-in-openreview/groups)

### 3. Compare Registration Signatures to Reviewer Group

Finally, identify the users in `reviewer_group` who are not in `normalized_profiles` to make a list of users who have not completed their registration:

```python
registration_not_completed = [profile for profile in reviewer_group if not profile in normalized_profiles]
```

### 4. Send message to reviewers who have not completed registration

Use `post_message` to send an email to the users who have not completed registration.

**Resources**: [How to send emails with python client](https://docs.openreview.net/how-to-guides/communication/how-to-send-messages-with-the-python-client)


# Conferences

Reports from previous conferences


# OpenReview NeurIPS 2021 Summary Report

**OpenReview NeurIPS 2021 Summary Report**

[Andrew McCallum](https://people.cs.umass.edu/~mccallum/) (Professor, UMass Amherst; Director OpenReview project)

[Melisa Bok](https://www.cics.umass.edu/people/bok-melisa) (Lead Developer, OpenReview project)

[Alina Beygelzimer](https://hunch.net/~beygel/) (Senior Research Scientist, Yahoo Research; NeurIPS 2021 Program Co-chair)

In 2021 the organizers of NeurIPS (one of the flagship conferences in machine learning) decided to move from CMT to OpenReview. This report provides a summary of the NeurIPS 2021 workflow, the OpenReview services provided, the system performance, and enhancements planned for the next NeurIPS.

The volume of NeurIPS paper submissions has been increasing dramatically over the past decade. NeurIPS also has a history of innovation in peer review.

NeurIPS 2021 workflow was very similar to its previous years: double blind, closed reviewing, with area chairs and senior area chairs, and meta-reviews, with the addition in 2021 of rolling reviewer discussion with authors (rather than a single author response).

Workflow details and timing were planned extensively with the OpenReview team, and coordinated through Google Docs, several video conference meetings, and conversations through a shared Slack channel. Throughout the submission and reviewing process OpenReview technical staff provided 24/7 support to the NeurIPS program chairs, including rapid responses and custom work.

Below is a summary of key workflow steps and services. (Detailed workflow is described [here](https://docs.google.com/document/d/1JKNmfEBC2yVXCU-3YLn4MQd4KKWrP-hU9TmAWaBzZCs/edit).)

***

**Reviewer recruiting**. NeurIPS PCs invited over 13k reviewers, 1k area chairs and 155 senior area chairs. With the permission of ICLR, OpenReview also shared with the PCs the list of accepted authors of the previous ICLR conference from 2016 until 2021.

* **Reviewer & author registration**. OpenReview already had profiles for approximately 228k researchers. During the reviewer recruiting and the paper submission 11k profiles were created, and incorporated their papers from DBLP, running our own version of author coreference, augmented by verification performed by OpenReview staff. NeurIPS required all authors (not just submitting authors) to register with OpenReview (mostly for the purposes of conflict-of-interest resolution, and gathering multiple email addresses per person). During the month of May 12,537 new user profiles were created, more than in any month of OpenReview’s history.
* **Conflicts-of-interest gathering**. Author and reviewer profiles include not only current institution domain names, but also a DBLP URL (from which authors imported all their publications), Google Scholar URL, and extensive conflict-of-interest information, including institutional history, advisors, other collaborators, and social connections, and other non-institutional conflicts. As requested by NeurIPS, we also added the ability to record private conflicts (not shown in the public web site). For NeurIPS review matching, OpenReview computed the conflicts based on institution history, all conflict relations listed above, and paper co-authorship within the last 3 years.
* **Reviewer expertise modeling**. Expertise models were built for all reviewers, using OpenReview’s own modern deep learning embedding methods run on titles and abstracts of reviewers’ papers. NeurIPS decided to use only our expertise model instead of TPMS or Semantic Scholar.
* **Paper submissions**. As requested by NeurIPS 2021 PCs, draft paper titles and abstracts were submitted one week before the full-paper deadline. OpenReview received 11,729 paper submissions. In the 24 hours before the final deadline, OpenReview received over 42k submission updates, and had over 28k active users (over 2.3k active simultaneous users during the last hour of submissions). The OpenReview multi-server system never surpassed 50% CPU usage, and maintained smooth operation with rapid system response throughout. In addition, during the submission period over 110k email messages were sent to authors (sent to each author for each update).
* **Bidding**. SACs bid on ACs and both ACs and reviewers bid on papers, assigned as a “task” that was not complete until a given number of bids had been entered. During reviewer bidding, SACs, ACs and reviewers were able to sort the ACs/papers by affinity scores or search by metadata.
* **Paper-reviewer assignment**. Paper-reviewer affinities included: the OpenReview reviewer expertise model, reviewer bids, and conflicts of interest. Optimization of paper-reviewer matching was performed by both [Min-Cost-Flow](https://developers.google.com/optimization/flow/mincostflow) and [FairFlow](https://arxiv.org/abs/1905.11924v1) \[Kobren, et al, 2019]. The optimizer’s meta-parameters can be easily tuned, and the NeurIPS 2021 program chairs ran the optimizer many times (with \~60 minute turn-around time). Each resulting match comes with various requested summary statistics about the quality of the match. The results of a paper-reviewer match could be browsed by PCs and ACs using OpenReview’s “Edge Browser,” which provides a MacOS-Finder-“column-view”-like nested browsing, as well as extensive searching, and the ability to make suggested edits to the assignment (including inviting new reviewers not already in the NeurIPS reviewing pool), while seeing reviewer loads, and meta-data for reviewers (including their institution, job title, and link to profile). The same paper matching system was used to do secondary area chair assignment, and emergency reviewer assignment during the reviewing stage.
* **Specialized consoles**: OpenReview provided specialized custom consoles for reviewers, area chairs, senior area chairs, ethic reviewers, ethic chairs, and program chairs––including functionality such as task lists, reviewing status, filtering entries with an filtering language (such as “papers with missing reviews”, or “papers where the average rating is higher than 3”), keyword search, reviewer re-assignment, aggregate statistics, status of bids for each revidewer, status of review completion, sending email to remind reviewers, the ability to dump data as downloadable CSV files.
* **Reviewing and discussion**. Reviews were entered directly into the OpenReview system, visible immediately to the ACs, then visible to authors and reviewers of the same paper after the reviewing deadline. An enhancement created specially at the request of NeurIPS, OpenReview implemented multiple tabs in the discussion forum of a paper (author discussion, committee discussion, all reviewing discussion, post-reviewing public discussion). OpenReview processed 37,284 reviews, 8,103 meta reviews and 452 ethics reviews. In addition, 101,112 confidential comments.
* **Review rating**. NeurIPS PCs requested that area chairs be able to rate the quality of each review. The PCs also allowed authors of the submissions to provide review feedback. The ratings and feedback were only visible to the Program Chairs.
* **Ethics reviews.** As requested by NeurIPS, for the first time OpenReview added configuration to handle ethics reviews. The Ethics Review Chairs assigned ethics reviewers to papers flagged with ethical issues. The OpenReview expertise matching system was used to suggest reviewers with the appropriate topical expertise.
* **Decisions**. OpenReview provides the ability to download various CSV files, which PCs downloaded into Google Sheets, including AC decisions. Some decisions were modified by the PCs. Then OpenReview emailed and posted the decision based directly on this Google Sheet. (In future, OpenReview may provide browsing, sorting, and editing directly through its UI; avoiding the need for Google Sheets. Alternatively, we may more closely embrace Google Sheets––leveraging its features––with live bi-directional data updates between OpenReview and the Google Sheet.)
* **Camera-ready revisions**. OpenReview created additional upload invitations and tasks for accepted paper authors, including copyright form, supplementary materials (including videos), camera-ready LaTeX zip file.
* **Conference track formation**. OpenReview also provided affinity scores between accepted papers, as input to paper clustering, for conference track assignments.

**System Responsiveness**

Throughout the submission period, the OpenReview system provided smooth service, with rapid response and smooth uptime.

**Peer Review Experiments**:

With the help and guidance of the team at OpenReview, NeurIPS 2021 ran the following experiments:

* **Consistency experiment**: In 2014, NeurIPS ran an experiment in which 10% of submissions were reviewed by two independent program committees to quantify the randomness in the review process. Since then, the number of annual NeurIPS submissions has increased more than fivefold. To check whether decision consistency has changed as the conference has grown, we ran a variant of this experiment again in 2021. Thе results of this experiment are reported here: ​[​https://blog.neurips.cc/2021/12/08/the-neurips-2021-consistency-experiment/](https://blog.neurips.cc/2021/12/08/the-neurips-2021-consistency-experiment/)
* To discourage **resubmissions** without substantial changes, authors were asked to declare if a previous version of their submission had been rejected from another peer-reviewed venue. Like the year before, authors of resubmissions were asked to describe the improvements made. This information was entered into OpenReview during the submission process. To evaluate **resubmission bias**, resubmission information was made visible to reviewers and area chairs only for a randomly chosen 50% of submissions. While the experiment allowed us to eliminate a significant bias, we can’t confidently ascertain there is none.
* **Author perception experiment**: OpenReview implemented a two-part author survey to help NeurIPS understand how well authors’ perception of their submissions agrees with reviewing outcomes. The results of this experiment are forthcoming.

**Releasing the data to the public**:

Submissions under review were visible only to assigned program committee members, and we did not solicit comments from the general public during the review process. After the notification deadline, **accepted papers were made public and open for non-anonymous public commenting**, along with their anonymous reviews, meta-reviews, and author responses.

By default, rejected submissions were not made public, but authors of **rejected submissions were given 2 weeks to opt in** to make their de-anonymized papers public and open for commenting in OpenReview. If they chose to do so, this also opened up the reviews, meta-reviews, and any discussion with the authors for these papers. This policy does give authors a mechanism to publicly flag and expose potential problems with the review process. In the end, only about 2% of rejected papers opted in.

**Feedback from Alina Beygelzimer, NeurIPS 2021 Program Co-chair**:

“As Program Chairs for NeurIPS 2021, we decided to shift the entire reviewing workflow to OpenReview. OpenReview is a flexible platform that allows heavy customization, and will be easy to adapt as the needs of the conference evolve. It brings a number of infrastructural improvements including persistent user profiles that can be self-managed, accountability in conflict-of-interest declarations, and improved modes of interaction during the discussion process. NeurIPS has a long history of experimentation with the goal of informing and improving the review process (e.g., the widely known “NeurIPS Consistency Experiment” of 2014). This year we took full advantage of the great flexibility of OpenReview’s workflow configuration to run several key experiments (including a version of the noise audit that hasn’t been done since 2014). We are grateful to the OpenReview team for supporting all requested experimentation.

Our experience with OpenReview has been a delight. Not only did the paper deadline proceed smoothly (with sub-second system response time throughout the arrival of thousands of submissions just before the submission deadline), but OpenReview gracefully handled more than 20K authors accessing the system roughly at the same time to read and respond to preliminary reviews, and enabled 10K reviewers and Area Chairs and 20K authors to engage in discussions in the weeks that followed. The feedback we received from our authors and program committee members has been overwhelmingly positive.

I hope that NeurIPS will continue to work with OpenReview for years to come. We are hugely grateful to the OpenReview team, for their unparalleled level of support to everyone involved in the review process. OpenReview has also supported the Data & Benchmarks track (new this year) as well as the Ethics Review process for both the main conference and the Data & Benchmarks track. It is also notable that over 20 of the NeurIPS workshops have chosen to use OpenReview for their reviewing workflow this year.”

**OpenReview team’s plans for improvement**. The OpenReview system is ready for re-use for future NeurIPS conference reviewing needs. The OpenReview team continues to make improvements and new features. Current work likely to be ready for NeurIPS 2022 includes

* We are currently designing a new version of the paper reviewing discussion forum, and would be eager for feedback and feature requests. NeurIPS concerns about “rolling discussions” could be addressed here.
* Further improvements to the reviewer-paper matching system.
* Deployment of a new API providing (1) additional flexibility for fine-grained per-field control of visibility, (2) ease of changing readership permission of content, (3) better storage and access of the history of changes to a paper, review, or comment, (4) creation of “CRON”-jobs for automated sending of reminders.

In future, we will also have support for synchronous chat-style communication among reviewers, area chairs, and program chairs––which we hope will encourage more interactive, open, scientifically-flexible communication during the reviewing period. We are also building support for live conferences, integrated into the OpenReview reviewing platform.


# OpenReview ECCV 2020 Summary Report

**OpenReview ECCV 2020 Summary Report**

[Andrew McCallum](https://people.cs.umass.edu/~mccallum/) (Professor, UMass Amherst; Director OpenReview project)

[Melisa Bok](https://www.cics.umass.edu/people/bok-melisa) (Lead Developer, OpenReview project)

[Thomas Brox](https://lmb.informatik.uni-freiburg.de/people/brox/) (Professor, U. Freiburg; ECCV 2020 Program Co-chair)

[Rene Vidal](http://cis.jhu.edu/~rvidal/) (Professor, JHU; Computer Vision Foundation Board member)

In 2020 the organizers of [ECCV](https://eccv2020.eu) (one of the flagship conferences in computer vision) decided to move from CMT to OpenReview. This report provides a summary of the ECCV 2020 workflow, the OpenReview services provided, the system performance, and enhancements planned for the next ECCV.

(The Computer Vision Foundation, CVF, has the long-term goal of unifying the CVPR conference workflow tools under one integrated infrastructure. Seeing the success of OpenReview for ICLR over the past seven years, CVF has been providing the OpenReview Foundation with a multi-year financial gift towards this new software development. CVPR is hoping to move to OpenReview in the future.)

ECCV 2020 workflow was not fundamentally different from its previous years: double blind, closed reviewing, with area chairs, closed reviewer discussion, author reponses, and meta-reviews.

Workflow details and timing were planned extensively with shared Google Docs, and three video conference meetings with the OpenReview team. Through the submission and reviewing process OpenReview technical staff provided 24/7 support to the ECCV program chairs, including rapid responses and custom work over weekends and evenings.

Below is a summary of key workflow steps and services. (Detailed workflow is described [here](https://docs.google.com/document/d/1J_QyRNQpJHLNkethTYtGXLhHgf3ZtTGenOuUQYEdSpQ/edit#heading=h.czbul1eao42z).)

* **Reviewer recruiting**. Based on a list provided by ECCV PCs, OpenReview invited over 5k reviewers, and automatically gathered their responses. We also coordinated with ICLR to invite additional reviewers from ICLR’s 2020 reviewer pool.
* **Reviewer & author registration**. OpenReview already had profiles for approximately 100k researchers. For ECCV we added an additional \~3k reviewer profiles, and incorporated their papers from DBLP, running our own version of author coreference, augmented by verification performed by OpenReview staff. ECCV required all authors to register with OpenReview (mostly for the purposes of conflict-of-interest resolution, and gathering multiple email addresses per person); this resulted in \~12k additional profiles being created.
* **Conflicts-of-interest gathering**. Author and reviewer profiles include not only current institution domain names, but DBLP url, Google Scholar url, past advisors, and other non-institutional conflicts. OpenReview could in the future also create conflicts based on paper co-authorship within the last *N* years; in future ECCV may use this feature also.
* **Reviewer expertise modeling**. Expertise models were built for all reviewers, using modern deep learning embedding methods run on titles and abstracts of reviewers’ papers. In future, OpenReview expertise modeling will also use paper full-text and citation graphs. ECCV 2020 decided to use a combination of both OpenReview reviewer-paper affinities as well as those from TPMS.
* **Paper submissions**. As requested by ECCV 2020 PCs, draft paper titles and abstracts were submitted one week before the full-paper deadline. OpenReview received 7646 paper submissions. In the 24 hours before the final deadline, OpenReview received over 24k submission updates, and had over 19k active users (over 3.7k active simultaneous users during the last hour of submissions). The OpenReview multi-server system never surpassed 50% CPU usage, and maintained smooth operation with rapid system response throughout. (In contrast, the ECCV static web server simply providing submission deadline information became unresponsive.) In addition, during the submission period over 55k email messages were sent to authors (sent to each author for each update).
* **Paper double-submission check**. ECCV used the OpenReview service that checks for double submissions against ICML 2020 and NeurIPS 2020.
* **Bidding**. Both ACs and reviewers bid on papers, assigned as a “task” that was not complete until a given number of bids had been entered. During reviewer bidding, ACs and reviewers were able to search submissions by keyword.
* **Paper-reviewer assignment**. Paper-reviewer affinities included: the OpenReview reviewer expertise model, TMPS affinity scores, area chair reviewer suggestions, reviewer bids, conflicts of interest. During area chair reviewer suggestions, candidate reviewers could be shown ordered by various criteria, including OpenReview affinity, TPMS affinity, and reviewer bids, (and custom reviewer loads). Optimization of paper-reviewer matching was performed by both [Min-Cost-Flow](https://developers.google.com/optimization/flow/mincostflow) and [FairFlow](https://arxiv.org/abs/1905.11924v1) \[Kobren, et al, 2019]. The optimizer’s meta-parameters can be easily tuned, and the ECCV 2020 program chairs ran the optimizer many times (with \~30 minute turn-around time). Each resulting match comes with various requested summary statistics about the quality of the match. The results of a paper-reviewer match could be browsed by PCs and ACs using OpenReview’s “Edge Browser,” which provides a MacOS-Finder-“column-view”-like nested browsing, as well as extensive searching, and the ability to make suggested edits to the assignment, while seeing reviewer loads, and meta-data for reviewers, including their institution, job title, and link to profile. (The same paper matching system was used to do secondary area chair assignment, and emergency reviewer assignment during the reviewing stage.)
* **Specialized consoles**: OpenReview provided specialized consoles for reviewers, area chairs, and program chairs, including functionality such as task lists, reviewing status, search, reviewer re-assignment, aggregate statistics, status of bids for each revidewer, status of review completion, sending email to remind reviewers, the ability to dump data as downloadable CSV files.
* **Reviewing and discussion**. Reviews were entered directly into the OpenReview system, visible immediately to the ACs, then visible to authors and reviewers of the same paper after the reviewing deadline. As a specially-ECCV-requested enhancement, OpenReview implemented [MarkDown](https://www.markdownguide.org) in time for the entry of author responses. (LaTeX formula rendering has already been available since Spring 2019.) OpenReview processed 15,152 reviews, 4,117 meta reviews and 2,752 secondary meta reviews. In addition, 9,506 confidential comments and 10,874 rebuttal comments were entered.
* **Review rating**. ECCV PCs requested that area chairs be able to rate the quality of each review on the scale -1, 0, 1, 2. From this reviewers were assigned an aggregate rating, what also included information about their tardiness. These aggregated reviewer ratings are stored (privately) in OpenReview, so that they will be easily (and programmatically) available to future ECCV program chairs. (We are also hoping to encourage private sharing of these ratings across conferences.)
* **Paper ranking**. ECCV program chairs requested that ACs be able to enter a ranking of their assigned papers.
* **Decisions**. PCs downloaded various CSV files into Google Sheets, including AC decisions. Some decisions were modified by the PCs. Then OpenReview emailed and posted the decision based on this Google Sheet. (In future, OpenReview may provide browsing, sorting, and editing directly through its UI; avoiding the need for Google Sheets. Alternatively, we may more closely embrace Google Sheets––leveraging its features––with live bi-directional data updates between OpenReview and the Google Sheet.)
* **Camera-ready revisions**. OpenReview created additional upload invitations and tasks for accepted paper authors, including copyright form, supplementary materials (including videos), camera-ready LaTeX zip file.
* **Conference track formation**. OpenReview also provided affinity scores between accepted papers, as input to paper clustering, for conference track assignments.

**Feedback from Thomas Brox, ECCV 2020 Program Co-chair**: Very happy with how OpenReview worked, and would recommend it to future program chairs. Particularly liked: (a) very stable and reliable system, (b) great response time and availability of the team, (c) excellent custom service (even implementing custom features we needed), (d) expressive conflict management (this was a primary impetus for moving to OpenReview, (e) reviewer assignment tools. Improvements that would be helpful for next year: feature allowing program chairs to impersonate another user (as CMT allows); additional reviewer-assignment constraints limiting the number of papers from the same institution on one paper, and multiple of the new features listed below.

**OpenReview team’s plans for improvement**, including

new system features:

* Allow program chairs to impersonate another user (as CMT allows), for purposes of understanding reviewer and area chair questions.
* Additional reviewer-assignment constraints limiting the number of papers from the same institution on one paper.

new UI features:

* Reviewer re-assignment directly from the convenient OpenReview “Edge Browser” interface (without the need to visit the PC console).
* Faster load times of the PC console when there are >5k submitted papers.
* Improved UI and organization of the “forum” page containing per-paper reviews and discussion: Easier way to read one-to-one discussion and distinguish between different types of replies: reviews, comments, rebuttals. More self-documenting “idiot-proof” UI widget for discussion participants to select the readers of the comments they enter.
* Allow ACs to download all their assigned paper files in a zip file.
* Add the ability for ACs and reviewers to bid “in blocks,” for example, bidding (positively or negatively) on all submissions containing a keyword, or in an area.’
* Add additional UI options for filtering papers, area chairs, or reviewers by various criteria, and then taking actions (such as sending email) on those objects satisfying the criteria.

new data gathering features:

* Improved expertise data, by automatically gathering the most recent computer vision conference publications that are not yet in DBLP. Improved expertise model based on the full-text and citations of each reviewers’ papers.
* Provide summary statistics of the number of past computer vision publications authored by each reviewer.

simple, alternative configuration for the next ECCV (no new system features needed):

* Restrict the list of papers shown to reviewers during the bidding stage: only the top *N* relevant submissions to each reviewer (rather than allowing reviewers to see all submissions).
* Allow the reviewer to edit the review after the rebuttal stage without showing the change to the authors until final decisions are released.


# Frequently Asked Questions

#### I have a question about:

* [My profile](#for-openreview-profiles)
* [My venue](#for-program-chairs)
* [My submission](#for-authors)
* [My reviews](#for-reviewers)

## For OpenReview Profiles

* [I am an Independent Researcher, how do I sign up?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-am-an-independent-researcher-how-do-i-sign-up)
* [I didn't receive a password reset email, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-didnt-receive-a-password-reset-email-what-do-i-do)
* [I don't want my profile visible to the public, how do I change this?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-dont-want-my-profile-visible-to-the-public-how-do-i-change-this)
* [I had an account previously, but lost access to my email, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-had-an-account-previously-but-lost-access-to-my-email-what-do-i-do)
* [Why can't I update my DBLP link?](https://docs.openreview.net/getting-started/frequently-asked-questions/why-cant-i-update-my-dblp-link)

## For Program Chairs

### Setting up a venue

* [How can I access the request form for my venue](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages)?
* [How do I find a venue id?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-find-a-venue-id)
* [How do I add a Program Chair to my venue?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-add-a-program-chair-to-my-venue)
* [How can I override the information on my venue homepage](https://docs.openreview.net/how-to-guides/modifying-venue-homepages/how-to-customize-your-venue-homepage)?
* [How do I recruit reviewers?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-recruit-reviewers)
* [How can I edit the Submission Form?](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/customizing-your-submission-form)
* [How can I test my venue workflow](https://docs.openreview.net/how-to-guides/workflow/how-to-test-your-venue-workflow)?
* [What customizations are available for emails sent through OpenReview](https://docs.openreview.net/how-to-guides/communication/how-to-customize-emails-sent-through-openreview)?
* [Reviewers for my venue cannot see their assigned submissions, what should I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/reviewers-for-my-venue-cannot-see-their-assigned-submissions-what-should-i-do)
* [What do the default submission, review, metareview, and decision forms look like](https://docs.openreview.net/reference/default-forms)?
* [How can I allow LLM generated submissions?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-can-i-allow-llm-generated-submissions)

### Managing active venues

* [How can I extend the Submission deadline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/changing-your-submission-deadline)?
* [How can I release reviews](https://docs.openreview.net/how-to-guides/workflow/how-to-release-reviews-decisions-and-metareviews)?
* [How can I enable Camera-Ready Revisions](https://docs.openreview.net/how-to-guides/workflow/how-to-use-the-camera-ready-revision-step-for-accepted-papers)?
* [How can I manually assign Reviewers/Area Chairs](https://docs.openreview.net/how-to-guides/paper-matching-and-assignment/how-to-do-manual-assignments)?
* [How can I automatically assign Reviewers/Area Chairs](https://docs.openreview.net/how-to-guides/paper-matching-and-assignment/how-to-do-automatic-assignments)?
* [What do the different 'status' values mean in the message logs?](https://docs.openreview.net/getting-started/frequently-asked-questions/what-do-the-different-status-values-mean-in-the-message-logs)
* [How can I contact Reviewers, Area Chairs, and/or Authors](https://docs.openreview.net/how-to-guides/communication/how-to-send-messages-through-the-ui)?
* [An author of a submission cannot access their own paper, what is the problem](https://docs.openreview.net/getting-started/frequently-asked-questions/an-author-of-a-submission-cannot-access-their-own-paper-what-is-the-problem)?
* [What is the max file size for uploads](https://docs.openreview.net/getting-started/frequently-asked-questions/what-is-the-max-file-size-for-uploads-1)?
* [What is the difference between due date (duedate) and expiration date (expdate)](https://docs.openreview.net/getting-started/frequently-asked-questions/what-is-the-difference-between-due-date-duedate-and-expiration-date-expdate)?
* [Will Reviewers be notified of their Assignments?](https://docs.openreview.net/getting-started/frequently-asked-questions/will-reviewers-be-notified-of-their-assignments)
* [How do I hide a submission field?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-hide-a-submission-field-from-the-submissions-readers)

## For Reviewers

* [I am a reviewer and I don't have papers for Expertise Selection, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-am-a-reviewer-and-i-dont-have-papers-for-expertise-selection-what-do-i-do)
* [How can I add formatting to my reviews/comments](https://docs.openreview.net/how-to-guides/submissions-comments-reviews-and-decisions/how-to-add-formatting-to-reviews-or-comments)?
* [I am a reviewer but I can't access my assigned submissions, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-am-a-reviewer-but-i-cant-access-my-assigned-submissions-what-do-i-do)
* [How do I obtain a letter of proof for my services as a reviewer](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-obtain-a-letter-of-proof-for-my-services-as-a-reviewer)?

## For Authors

* [How do I add/change an author of my submission after the deadline?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-add-change-an-author-of-my-submission-after-the-deadline)
* [Why is my LaTeX code not displaying properly](https://docs.openreview.net/reference/openreview-tex/common-issues-with-latex-code-display)?
* [When will I be able to withdraw my submission?](https://docs.openreview.net/getting-started/frequently-asked-questions/when-will-i-be-able-to-withdraw-my-submission)
* [I accidentally withdrew a submission, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-accidentally-withdrew-a-submission-what-do-i-do)
* [I want to delete my withdrawn or desk-rejected paper, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-want-to-delete-my-withdrawn-or-desk-rejected-paper-what-do-i-do)

### Other

* [How do I find a venue and contact the venue organizers?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-find-a-venue-and-contact-the-venue-organizers)
* [What should I do if my question is not answered here](https://docs.openreview.net/getting-started/frequently-asked-questions/what-should-i-do-if-my-question-is-not-answered-here)?
* [What should I do if I find a vulnerability in OpenReview](https://docs.openreview.net/getting-started/frequently-asked-questions/what-should-i-do-if-i-find-a-vulnerability-in-openreview)?
* [How can I report a bug or request a feature](https://docs.openreview.net/getting-started/frequently-asked-questions/how-can-i-report-a-bug-or-request-a-feature)?
* [How do I locate the date a submission is made public?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-locate-the-date-a-submission-is-made-public)


# I accidentally withdrew a submission, what do I do?

To revert a withdrawal, please contact the Program Chairs of your conference using the contact email on the venue homepage.


# How do I add a Program Chair to my venue?

Navigate to the Program Chairs group in the [Workflow Timeline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages). Add the Program Chair's OpenReview ID to the members using the "Add to Group" button.


# When will I be able to withdraw my submission?

The timeline for submission withdrawals is determined by the Program Chairs of each venue. Therefore, there is not a single overarching OpenReview withdrawal policy. If you have questions about when in a venue workflow you will be able to withdraw your submission, you should reach out to the Program Chairs of that venue directly.


# I want to delete my withdrawn or desk-rejected paper, what do I do?

Individual conferences determine the policy on the visibility of withdrawn and desk-rejected submissions. Please refer to the conference website for further information on the policy, and contact the venue organizers with any questions or concerns.


# Why can't I import my publication data from Google Scholar?

We understand the frustration, unfortunately, Google Scholar does not have an API that enables us to import author publications.


# An author of a submission cannot access their own paper, what is the problem?

The author must make sure that the email address associated with the submission is added to their profile and confirmed.


# What should I do if I find a vulnerability in OpenReview?

A vulnerability is a defect (bug) in the system that compromises the integrity of the application or its data. If you found one, please [Contact Us](https://openreview.net/contact) as soon as possible so that it can be fixed. Please do **NOT** post about the vulnerability elsewhere.


# How can I report a bug or request a feature?

You can report a bug or request a feature by going to our [issue tracker repository](https://github.com/openreview/openreview). Before you do so, please read the [guidelines](https://github.com/openreview/openreview/blob/main/CONTRIBUTING.md) so that we can understand and address your bug report or request. Once you are familiar with the procedure, you can make use of one of these [templates](https://github.com/openreview/openreview/issues/new/choose) to help us better understand your issue or feature request.


# What is the difference between due date (duedate) and expiration date (expdate)?

Due date is the advertised deadline.

Expiration date is the hard deadline.


# Will Reviewers be notified of their Assignments?

Reviewers will see their assigned papers in their Reviewer console and in their Task list. Depending on what method you used to assign Reviewers to papers, they may have already been notified of their assignments:

#### Automatic Assignment Deployed through the Edge Browser

If you deployed an assignment configuration from the edge browser, Reviewers would not have received a notification of their assignments.

#### Manual Assignment through the Edge Browser

If you use the "Assign" button to add a Reviewer to a paper, the Reviewer will receive a notification of their new assignment.


# What is the max file size for uploads?

The max file size for uploads is 100 MB.


# Why are the "rating" and "confidence" fields in my PC Console wrong?

If you removed the rating or confidence fields from your Official Review form, the PC console will show an average rating and/or confidence of 0 for each paper. If you replaced them with custom values, you can customize your PC console to show the average of those values instead.

1. From the Workflow Timeline, edit the Official Review step.
2. Under Form Fields--> Edit, ensure that the fields that you chose are in the form fields. The options reviewers can select for that field must follow the format "number: description", for example "1: Very Poor".
3. Update the value for the "Rating Field Name" and "Confidence Field Name". The name must match an existing field in the Review form fields.
4. Click "Submit".


# What should I do if my question is not answered here?

The best way to get help with a specific issue is to contact the program chairs or organizers of the venue you are participating in. Contact info can usually be found on the venue's OpenReview page. See [How do I find a venue and contact the venue organizers?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-find-a-venue-and-contact-the-venue-organizers)

For general inquiries, you can contact the OpenReview team by using the [Feedback Form](https://openreview.net/contact). We are most responsive from 9AM - 5PM EST Monday through Friday. With the exception of urgent issues, requests made on weekends or US holidays can expect to receive a response on the next business day.


# My Profile is "Limited". What does that mean?

OpenReview’s goal is to provide scientific communication to the entire global community. Initially OpenReview was not available to users in Iran, Cuba, Syria, and several other countries because our cloud service provider blocked access as a simple approach to comply with United States economic sanctions and trade laws.

Recently the OpenReview staff has done extra work to make OpenReview available in all countries, including those above. In order to comply with relevant laws, OpenReview must guarantee that none of our registered users appear in the Office of Foreign Assets Control of the U.S. Department of the Treasury (OFAC) [SDN list](https://sanctionssearch.ofac.treas.gov/). We primarily accomplish this by looking for first-last-name matches in the list. When there is a name match, we ask for additional information that would disambiguate the OpenReview user from the person on the SDN list.

If you were notified that your profile has been limited, this means that the name on your profile matches that of a person on the SDN list. While in a limited state, you can log in to OpenReview, edit your profile, and view the same amount of data as before, but you will be unable to author any notes on the OpenReview system. This means you will be prevented from performing most common actions, including submitting to venues or posting reviews, meta-reviews, or comments. In order to reactivate your profile, you will need to do the following:

* Log in to OpenReview at <https://openreview.net/login>
* Navigate to your Profile at <https://openreview.net/profile>
* Click the Edit button.
* Find the field where you can enter your year of birth, and enter the information.
* Click Save at the bottom of the page.

OpenReview will routinely check for SDN matches, and if your birth year has been updated, your profile will return to active. Note that this change will not be immediate. If you believe you have been waiting longer than expected and your profile has not been reactivated, you can reach out to us using the [Feedback Form](https://openreview.net/contact).


# How do I recruit reviewers?

Go to the Reviewers group in the Workflow Timeline and click the "Recruitment Request" button to send emails to potential reviewers.


# How do I obtain a letter of proof for my services as a reviewer?

This is not something that is generated by OpenReview. Please contact the organizers of the venue where you served as a reviewer to see if they can assist you.

To see a history of all emails sent to you by the organizers, please login and go to openreview\.net/messages. To see your recent activity, including any reviews you may have submitted, you can go to openreview\.net/activity.

There is also the option of [retrieving the public reviews](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-get-all-the-reviews-that-i-have-written-and-their-corresponding-submissions) you've written with their corresponding submissions using the API.


# How do I complete my tasks?

Throughout the course of a conference/workshop, members of the committee groups are assigned tasks to complete, such as, Expertise Selection, Reviewing, and the Reviewer License Agreement. Users can find their tasks by clicking on the Tasks button (openreview\.net/tasks) on the red nav bar at the top of the page. They can also find these tasks in their respective venue consoles.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FO2grw5VZVax40JRTwksa%2FScreen%20Shot%202024-01-30%20at%203.15.16%20PM.png?alt=media&amp;token=2fa7a22b-54b7-498c-8331-271482367392" alt="" width="375"><figcaption><p>Tasks button</p></figcaption></figure>

They'll see a list of different venue IDs and the number of pending and completed tasks.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F0kCny7sVE9rqXtiiHetD%2FScreen%20Shot%202024-01-30%20at%203.22.02%20PM.png?alt=media&amp;token=e1a6373d-7270-4093-8633-e6a93768c2b2" alt="" width="375"><figcaption><p>Tasks page</p></figcaption></figure>

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FKVENd8brmcDq3Adibm9g%2FScreen%20Shot%202024-01-30%20at%203.59.47%20PM.png?alt=media&amp;token=771f6163-d465-4488-a525-a105864ce0f0" alt="" width="375"><figcaption><p>Reviewer console listing tasks</p></figcaption></figure>

To view the tasks, click on the arrow next to the venue ID or "Show pending tasks and completed tasks". Clicking on a task link will redirect the user to another page to complete the task. When the task is completed it will be grayed out on the Tasks page. When the expiration date for that task expires the task will disappear from the Tasks page.

Note: Some tasks, such as Expertise Selection, will continue to show as pending until the task expires, at which time it will be removed from the queue.

See also: [I am a reviewer and I don't have papers for Expertise Selection, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-am-a-reviewer-and-i-dont-have-papers-for-expertise-selection-what-do-i-do)


# Can I automatically transfer my Expertise Selection to another venue?

Expertise Selection is a task for Program Committee members (such as reviewers) that gives them the opportunity to identify which previous publications they would like to be considered when being matched with submissions to review. If you want to use the same Expertise Selection configuration for multiple venues, it is possible to transfer the expertise from one venue to another using the OpenReview Python client using the following steps.

{% hint style="info" %}
These instructions are for venues that use the default OpenReview setting of using all papers except those that are **excluded** by the user in the Expertise Selection Task. Before transferring expertise, please check that both venues use this method for Expertise Selection.
{% endhint %}

### 1. [Download and install the Python Client](https://docs.openreview.net/getting-started/using-the-api/installing-and-instantiating-the-python-client)

```python
import openreview
from openreview.api import OpenReviewClient

live_client_v2 = OpenReviewClient(
    baseurl='https://api2.openreview.net',
    username=YOUR_USERNAME,
    password=YOUR_PASSWORD
)
```

### 2. Get Expertise Selection edges from the previous venue

OpenReview by default considers all publications as relevant expertise for paper matching and assignment. In Expertise Selection, edges are posted indicating the papers that should be *excluded* from paper matching. You can get the edges from a previous venue using the following method:

```python
#set up venue and user information
profile_id = YOUR_TILDE_ID 
from_venue_id = FROM_VENUE_ID 
from_group_name = FROM_GROUP_NAME  #Typically 'Reviewers' or 'Area_Chairs'

#get edges for from_venue
expertise_edges = openreview_client.get_all_edges(invitation=f"{from_venue_id}/{from_group_name}/-/Expertise_Selection",
 tail=profile_id)
```

You will need the following information:

* **profile\_id:** Your profile ID (such as \~First\_Last1). Also see [Finding your Profile ID](https://docs.openreview.net/getting-started/creating-an-openreview-profile/finding-your-profile-id)
* **from\_venue\_id:** The id of the venue you want to transfer expertise from. Also see: [How do I find a venue ID?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-find-a-venue-id)
* **group\_name:** The name of the group you are in on the Program Committee (typically this is Reviewers or Area\_Chairs)

The number of edges should correspond with the number of papers that you are excluding from Expertise Selection.

### 3. Copy edges to new venue

Finally, post an edge to the new venue that you are doing Expertise Selection for:

```python
to_venue_id = TO_VENUE_ID
to_group_name = TO_GROUP_NAME
invitation = f"{to_venue_id}/{to_group_name}/-/Expertise_Selection"

for edge in expertise_edges:
    openreview_client.post_edge(openreview.api.Edge(
        invitation = invitation,
        head = edge.head,
        tail = edge.tail,
        signatures = [profile_id],
        label = 'Exclude'
    ))
```


# What do the different 'status' values mean in the message logs?

**Sent**: The initial status value, meaning the message was sent to the email service.

**Dropped**: The email was not sent to the recipient for delivery.

**Deferred**: The email cannot immediately be delivered, but has not been completely rejected. The service will continue to try for 72 hours to deliver the deferred message. After 72 hours the defferal turns into a block.

**Bounce**: The server cannot or will not deliver the message. This is often caused by outdated or an invalid email address.

**Delivered**: The email has been accepted by the receiving server. This does not mean that the email made it to the recipient's inbox.

**Blocked**: The email was denied temporarily by the server. This is unrelated to the validity of the address.


# I am an Independent Researcher, how do I sign up?

We've received many emails from concerned researchers who want to sign up quickly but aren't affiliated with an academic institution. Please do not use an old role or affiliation as your current position, as that may delay moderation.

If you are not affiliated with an academic institution, but are employed at a company, you can use that domain and position information to sign up. This is recommended if at all possible.

If you do not have a company domain, you can use a personal email. All personal emails will be moderated - see [Expediting Profile Activation](https://docs.openreview.net/getting-started/creating-an-openreview-profile/expediting-profile-activation) for additional tips on how to make sure your profile is activated as quickly as possible.

When filling out the profile details for registration, there is a section that will ask for your Career & Education History, which will require a current position. For the Position field select **'Independent Researcher'**, which will lock the Institution Info domains automatically. See the page [Entering Institutional Data](https://docs.openreview.net/getting-started/creating-an-openreview-profile/entering-institutional-data) for an example of how to fill out institutional information as an independent researcher. Please include as much of your background as possible. If the 'Career & Education History' section only lists 'Independent Researcher' your profile will be rejected with a request to add another institution record.

If you are not affiliated with an institution, set your current position to **Independent Researcher**. When you select this position, OpenReview automatically fills in the institution domain (`independent-researcher.org`) and disables the Institution Domain and Institution Name fields — you don't need to enter them yourself.&#x20;

Because personal emails are moderated, adding and confirming an institutional email is still the fastest way to get your profile activated.


# How do I locate the date a submission is made public?

Publication date(pdate) means when the venue marked the paper as accepted.

Modification date(mdate) means when the last time the paper was modified.

The odate means when the submission becomes public. To check the odate of a submission, see the note object [https://api2.openreview.net/notes?id=](https://api2.openreview.net/notes?id=RAA0vCLMhp)

To find the id you can check the url of the submission, for example <https://openreview.net/forum?id=KS8mIvetg2> . Use `id=KS8mIvetg2` , the note object url will be <https://api2.openreview.net/notes?id=KS8mIvetg2>.


# I am a reviewer but I can't access my assigned submissions, what do I do?

Ensure that you are logged in with an active OpenReview account that has the email the invitation was sent to added and confirmed to it. To make an OpenReview account, see [Signing up for OpenReview](https://docs.openreview.net/getting-started/creating-an-openreview-profile/signing-up-for-openreview). To add an email to your account, see [Add or remove an email address from your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/add-or-remove-an-email-address-from-your-profile).

If you are logged in and are still not able to see your assigned submissions (but can see assignments under your Tasks), then reviewers have likely not been given reader access for submissions by the organizers of the venue.

Please contact the venue organizers notifying them of the issue. The contact email can be found on the homepage of the venue.

### Missing Fields

If part of the submission is missing fields such as the PDF, the most likely causes are:

1. These fields were optional in the submission
2. The venue organizers have decided to hide the field in review. If you believe this to be in error, please contact the venue organizers.


# Reviewers for my venue cannot see their assigned submissions, what should I do?

In order for a reviewer to see their assigned submissions, they need to be logged into an OpenReview account that has read permissions for the submission.

If a specific reviewer cannot access their assigned submission, they may need to check that the email they received the invitation to is added and confirmed to their account- The email should be listed in their profile with (Confirmed) next to it. If this is not the case, they should follow the steps to [add an email ](https://docs.openreview.net/getting-started/creating-an-openreview-profile/add-or-remove-an-email-address-from-your-profile)to their account.

If all reviewers cannot access the forum page of their assigned submission, check that they are readers of the submission they are assigned to.

Reviewers are made readers of submissions in the Submission Change Before Reviewing step in the Workflow Timeline. If the step has not run yet, you can edit the date to the current time to activate the stage.

If you have added fields to the submission form that should be hidden from reviewers, see [How do I hide a submission field?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-hide-a-submission-field-from-the-submissions-readers).


# I am a reviewer and I don't have papers for Expertise Selection, what do I do?

The OpenReview system uses the reviewer's publications from Expertise Selection in order to match reviewers and papers via affinity scores. Reviewers who do not have existing publications should leave the Expertise Selection form blank, and the reviewer will not be assigned an affinity score for a paper.

Decisions on how to assign reviewers without affinity scores is at the discretion of the venue organizers. Please contact the venue organizers if you have further questions about how review assignment will work for a specific venue.

Note: There is no 'Submit' button for Expertise Selection, and the task will show in your pending queue (where you can make changes) until the task expires at which time the task will automatically leave the queue.

See also: [How do I complete my tasks?](https://docs.openreview.net/getting-started/frequently-asked-questions/how-do-i-complete-my-tasks)


# How do I upload a publication with a license that is not listed?

OpenReview requires every publication to have a license. If you have a paper that is under a license that is not listed, please contact us through the [Feedback Form](https://openreview.net/contact).


# I didn't receive a password reset email, what do I do?

There are a few common reasons why you might not have received the password reset email. You can try the following steps to debug the issue

1. Check that the email was not sent to a spam or junk folder.
2. Try sending the password reset email again and double check that there are no typos in the email.
3. Reset your password using a different email that is listed in your profile (such as personal email, previous institutional email).

If after these steps, you are still having trouble resetting your password, contact us through the [Feedback Form](https://openreview.net/contact).


# How do I add/change an author of my submission after the deadline?

Each venue decides its own policy for how to handle requests to change the authorship list, order, or author details after the deadline. This is sometimes listed on the venue website or FAQ. If you need to request a change to your submission after the submission deadline has passed, please contact the venue organizers.


# How do I find a venue id?

There are a few different ways to find your venue id.

1. If you are a Program Chair, go to your [venue request form](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages) and copy the information in the field Venue ID
2. Use the URL of the submission page. The venue id is the portion of the url after `group?id:`For example for ICLR 2024 the submission homepage is : <https://openreview.net/group?id=ICLR.cc/2024/Conference#tab-accept-oral>, and the venue id is `ICLR.cc/2024/Conference`

{% hint style="info" %}
If you are having trouble with your venue id, ensure that there is no trailing backslash (e.g. `ICLR.cc/2024/Conference/` is incorrect)
{% endhint %}


# Why can't I update my DBLP link?

{% hint style="danger" %}
The DBLP staff cannot support the exceptionally high number of support and error correction requests they've received from authors submitting to venues hosted on OpenReview. It will currently take several weeks, if not months, to read and address requests. **Do not send your request multiple times**.
{% endhint %}

You can try the following steps to debug issues related to adding your DBLP link to your profile:

1. **Ensure that the DBLP link is properly formatted and uses the persistent URL** (see [here](https://docs.openreview.net/getting-started/creating-an-openreview-profile/importing-papers-from-dblp) for more details about persistent URLs)
2. **Ensure that your DBLP link is a single author page.** If the link has a banner that states that it is a "disambiguation page", or if the link contains publications that belong to another researcher (See this [post](https://blog.dblp.org/2020/03/18/name-disambiguation-suffixes-in-dblp/?utm_source=chatgpt.com) by DBLP for more information) there will be issues with importing papers. Please see the above warning about contacting DBLP with correct requests.
3. **Check for previous profiles**. Sometimes users have previously created profiles with DBLP links. If you get an error stating another user is using your profile, check for previous accounts using past emails.
4. **Check for claimable profiles.** There are some profiles that have been pre-created by OpenReview. You can go to the [sign-up](https://openreview.net/signup) page and type the first and last name in the conflicting profile and check for claimable profiles.
5. **Contact support.** If none of the above steps work, please [Contact Us](https://openreview.net/contact). Include your profile id, the DBLP link, and the error message you receive when you attempt to update your profile.


# I had an account previously, but lost access to my email, what do I do?

If you have lost access to your OpenReview account, please try the following steps.

1. Try resetting your password with alternate email accounts. You can reset your password with any email that was associated with it.
2. Contact OpenReview Support using the [Contact Form](https://openreview.net/contact) from your new email with the following information: your old email or profile ID and any available documentation verifying your institutional history in order to gain assistance in resetting your password.


# How do I hide a submission field?

1. In the Workflow Timeline, click Submission Change Before Reviewing
2. Under Restrict Field Visibility, edit the Content Readers JSON to hide the requested fields. The `authors` field, which holds the author names and profile IDs, is hidden automatically. Use its entry as a guide for hiding other fields.
3. Under Edit Dates, set the Activation Date to now to trigger the stage.


# I don't want my profile visible to the public, how do I change this?

If you don't want your profile to be publicly visible, you can uncheck the Profile Visibility setting from your Profile/Edit page. With this setting disabled, only authenticated (logged in) OpenReview users can view your profile.

1. Go to <https://openreview.net/profile/edit>
2. Click on "Personal Info"
3. Under "Profile Visibility", uncheck the box to hide your profile from the public

If your profile is public, unauthenticated individuals can see your:

* Name
* Personal Links
* Career & Education History
* Advisors, Relations & Conflicts (you can hide these items from your profile edit page)
* Expertise

You can see this by copying your profile link and opening it in an incognito browser.

{% hint style="warning" %}
Please note that OpenReview only displays obfuscated emails (\*\*\*\*@umass.edu) and never releases full email addresses to the public.
{% endhint %}


# How can I allow LLM generated submissions?

All submissions must be represented and submitted by human authors. There are no author profiles representing AI.

A field should be added to the submission form to indicate the content was generated by an LLM. Please see our guide on [how to customize the forms](https://docs.openreview.net/getting-started/customizing-forms).


# How do I find a venue and contact the venue organizers?

Each venue on OpenReview is run by its own organizers, the Program Chairs. For questions about a venue, contact its organizers.

## Find the venue

On the [OpenReview homepage](https://openreview.net):

* **Open for Submissions** lists the venues that are accepting submissions now.
* **All Venues** lists every organization that hosts venues on OpenReview, in alphabetical order. You can also open the list at [openreview.net/all-venues](https://openreview.net/all-venues). Click an organization, then follow the links to the year and the venue.

The search bar at the top of each page finds papers, authors, and reviews, not venues.

## Contact the organizers

The top of the venue page shows the venue's details. The email address next to the envelope icon is the organizers' contact address. The link next to it opens the venue's own website, which often has more ways to reach them.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2Fu7cl9ldSLuUnaJ3PcOzK%2Fimage.png?alt=media&#x26;token=2227745f-7d56-468b-bb93-a8c13db731ea" alt="The header of a venue page showing the venue name, its location, date, website link, and a contact email address next to an envelope icon."><figcaption><p>The header of a venue page. The contact email is next to the envelope icon.</p></figcaption></figure>


# Objects in OpenReview

OpenReview consists of different objects used to hold or interact with data. This section defines briefly each of the objects and how they funciton within OpenReview.

**Notes**: Primary data objects representing submissions, reviews, comments, and decisions. Each Note contains flexible fields and fine-grained access controls, including `readers`, `writers`, and `signatures`. Notes are created as responses to Invitations, ensuring they adhere to predefined structures and permissions.

**Invitations**: Templates that define the structure, permissions, and rules for creating or editing entities like Notes and Edges. Invitations specify who can perform certain actions (`invitees`), who can read the resulting entities (`readers`), and the expected format of the content. They ensure consistency and enforce workflow rules.

[**Groups**](https://docs.openreview.net/getting-started/objects-in-openreview/groups): Collections of users (Profiles) that represent roles like authors, reviewers, or progiram chairs. Groups define permissions and access controls within the system. They are essential for managing who can read, write, or sign various entities.

**Edits**: Mechanism for creating or modifying entities like Notes, Groups, and Invitations. An Edit encapsulates the changes and, upon submission, undergoes an inference process to apply these changes to the target entity. This allows for a record of all changes within the system.

[**Profiles**](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-profiles): Represent individual users with unique identifiers (e.g., `~First_Last1`). They store user information such as names, emails, affiliations, and relationships (e.g., co-authors, advisors). Profiles are linked to different roles and permissions in the system via Groups.

*


# Introduction to Groups

See also:

* [Technical reference for Groups](https://docs.openreview.net/reference/api-v2/entities/group)
* [Managing Groups for Venue Workflows](https://docs.openreview.net/how-to-guides/managing-groups)

In OpenReview, a **Group** object represents a collection of users (individuals or other groups) and serves as the core mechanism for managing permissions, roles, and access control. Groups define who can read, write, sign, or modify content such as Notes and Invitations. They are hierarchical and addressable using a path-like ID (e.g., `ICLR.cc/2025/Conference/Reviewers`).

To give fine-grained control over readership, individuals within a conference are typically members of many groups. For example, a reviewer for the conference `Test/2025/Conference` assigned to Submissions 1 and 10 would likely be in the following groups:

* **Reviewer group:** This contains all reviewers that accepted the recruitment invitation`Test/2025/Conference/Reviewers`
* **Relevant submission groups:** These groups give assigned reviewers (rather than all reviewers) the ability to view and post reviews for submissions they are assigned to. In this case it would be: `Test/2025/Conference/Submission1/Reviewers` `Test/2025/Conference/Submission10/Reviewers`
* **Relevant anonymous groups:** These groups give the reviewer the ability to remain anonymous during the review process: `Test/2025/Conference/Submission1/Reviewer_Axyz` , `Test/2025/Conference/Submission10/Reviewer_Xyab`

### **Structure**

You can find a description of all fields of the `Group` object in the [API Reference](https://docs.openreview.net/reference/api-v2/entities/group). You will most often interact with the following key fields:

* **id**: Unique identifier for the group (e.g., `ICLR.cc/2025/Conference/Reviewers`).
* **members**: List of user IDs or group IDs that are members.

### **Retrieving Groups**

You can retrieve groups using the `get_group` or `get_groups` method:

<pre class="language-python"><code class="lang-python"><strong>reviewer_group = client.get_group('Test/2025/Conference/Reviewers')
</strong>conference_groups = client.get_groups(prefix='Test/2025/Conference')
</code></pre>

The `prefix` parameter allows filtering by group ID hierarchy.

#### Quickstart: Get a list of all active venues

```python
venues = client_v2.get_group(id='active_venues').members
print(venues)
```

#### Quickstart: See a venue's information

```python
venue_group = client_v2.get_group("<VENUE_ID>")
```

Within this group, `content` is a dictionary that contains information about the venue.

To see all possible keys contained in a venue group's `content` object, you can use the following:

```python
venue_group.content.keys()
```

You can query the information within this dictionary to get more information about the venue. For example: `venue_group.get_content_value('submission_id')` will tell you the name of the Submission invitation.

#### Quickstart: Get pending invited Area Chairs/ Reviewers

```python
#venue_id
venue_id = "<VENUE_ID>"
role = "Area_Chairs" #Alternatively, Reviewers, Senior_Area_Chairs

accepted = client_v2.get_group(venue_id + '/' + role)
invited = client_v2.get_group(venue_id + '/' + role +'/Invited')
declined = client_v2.get_group(venue_id + '/' + role + '/Declined')

pending_members = [m for m in invited.members if not m in accepted.members and not m in declined.members]
```

### Working with groups

Most of the changes that you want to make with groups can be made from the UI. See the section [Managing Groups](https://docs.openreview.net/how-to-guides/managing-groups) for specific instructions on working with groups. In some specific cases, you may need to programmatically create or modify groups.

#### Creating Groups

The majority of groups are created automatically in the workflow. However, Program Chairs (but not other roles) have the access necessary to create groups, should they need to.

To add a group, you would use `post_group_edit` function. This creates a group with a specific id that you can then add members to. As an example, if you ant to add an LLM Reviewer to each Submission Group 1:

The new group ID would be: `MyConference/2025/Conference/Submission1/Reviewer_LLM`

The full `post_group_edit` call will look like this:

```python
client.post_group_edit(
    invitation='<venue_id>/-/Edit',
    signatures=['<venue_id>'],
    group=openreview.api.Group(
        id = '<venue_id>/Submission<number>/Reviewer_New_Group',
        readers = ['<venue_id>'],
        signatures = ['<venue_id>'],
        writers = ['<venue_id>'],
        signatories = ['<venue_id>'],
    )
)
```

#### Adding Members

To add members to a group, you use the function `add_members_to_group` , where `group` is the group id to add to, and `members` is group or list of profile ids to add.

```python
client.add_members_to_group(group='<venue_id>/Submission<number>/Reviewers', 
members='<venue_id>/Submission<number>/Reviewer_New_Group')
```

An alternative approach uses the more general `post_group_edit` function to modify groups which gives more flexibility how to modify groups. To use this function to add mamebers:

```python
group = client_v2.get_group('<GROUP_ID>')
members = []
self.post_group_edit(invitation = f'{group.domain}/-/Edit', 
    signatures = group.signatures, 
    group = Group(
        id = group.id, 
        members = {
            'add': list(set(members)) #the keyword 'add' is used to add the members here
        }
    ), 
    readers=group.signatures, 
    writers=group.signatures
)
```

#### Removing Members

The function `post_group_edit` can also be used to remove members from a group by using the keyword `remove`:

```python

group = client_v2.get_group('<GROUP_ID>')
members = []
self.post_group_edit(invitation = f'{group.domain}/-/Edit', 
    signatures = group.signatures, 
    group = Group(
        id = group.id, 
        members = {
            'remove': list(set(members)) #the keyword 'remove' is used to remove the members here
        }
    ), 
    readers=group.signatures, 
    writers=group.signatures
)
```

#### Replacing Members

Finally, `post_group_edit` can also be used to replace the members of a group with new members with the keyword `replace` .

```python
members = []
self.post_group_edit(invitation = f'{group.domain}/-/Edit', 
    signatures = group.signatures, 
    group = Group(
        id = group.id, 
        members = {
            'replace': { 'index': 2, 'value': '~OpenReview_Profile1' }
        }
    ), 
    readers=group.signatures, 
    writers=group.signatures
)
```


# Introduction to Profiles

[Jump to QuickStart script for getting all profiles](#quickstart-getting-all-profiles)

[Technical Reference for Profiles](https://docs.openreview.net/reference/api-v2/entities/profile)

**Profiles** in OpenReview represent the identity of users on the platform—such as authors, reviewers, and area chairs. Each profile serves as a record user's publications, affiliations, and roles within the OpenReview system.

### **What’s in a Profile?**

A profile typically includes:

* **Profile ID:** This is the unique identifier in the system, in the format `~First_Last1` This is also sometimes referred to as a "tilde id". While each tilde ID points to a single profile, a profile may have multiple tilde IDs associated with it as a result of adding an alternate name or merging profiles.
* **Name(s)**: Full name and any alternate names (e.g., name changes, nicknames).
* **Email(s)**: Verified email addresses associated with the user. Only the email domains are publicly displayed- even PCs will not see the full emails for users submitting to their venue. Note, if you need to get the full author emails, refer to the instructions [here](https://docs.openreview.net/how-to-guides/communication/how-to-get-email-addresses).
* **Affiliations**: Work history or institutional associations.
* **Publications**: Papers the user has authored or co-authored.

### **Getting a Profile or Profiles**

You can get one or more profiles using the Python API using the function `openreview.tools.get_profiles()` . This function takes a list of profiles or emails, and for each item in the list, returns one of the following: It is possible to query a profile either using the profile's tilde id, or using any confirmed email to the profile

```python
openreview.tools.get_profiles(client_v2, <LIST_OF_PROFILE_IDS_OR_EMAIL>)
```

To get multiple profiles, you would use the function `openreview.tools.get_profiles()` . This function takes a list of profiles or emails, and for each item in the list, returns a dictionary with the following:

```python
profile_id_list = []
profiles = openreview.tools.get_profiles(client_v2,profile_id_list,as_dict=True)

```

Other arguments that can be used to get additional informations along with the profiles are:

* `with_publications`
* `with_relations`
* `with_preferred_emails`

### **Structure of Profiles**

Profiles are an OpenReview object that contains properties. The main three properties that you can expect to interact with are: `id` , `state`, and `content`. For more details about each of these fields, see [here](https://docs.openreview.net/reference/api-v2/entities/profile/fields). Several of the fields of `content` include lists or lists of dictionaries, which means that it is necessary to understand the structure of the profile in order to get the profile information.in order to access this data, it is Because you need to flatten the dictionary to create the fields, then extract the content, similarly to how the submission content was extracted. The original profile information looks something like this:

```
{'active': True,
 'state': 'Active'
 'content': {'emails': ['name@university.edu'],
             'emailsConfirmed': ['name@university.edu'],
             'history': [{'end': None,
                          'institution': {'country': 'US',
                                          'domain': 'university.edu'},
                          'position': 'PhD Student',
                          'start': 2017}],
             'homepage': 'https://test.com',
             'names': [{'fullname': 'First Last',
                        'preferred': True,
                        'username': '~First_Last2'}],
             'preferredEmail': 'name@university.edu',
             'relations': []},
 'id': '~First_Last2',
 ...<other metacontent>...
 
 }
```

To get a tabular format, it is necessary to flatten the profile. After flattening (sample code below) a profile would look like this:

<table><thead><tr><th width="51.8984375"></th><th>preferredEmail</th><th>homepage</th><th>emails_0</th><th>names_0_preferred</th><th>names_0_fullname</th><th>names_0_username</th><th>history_0_position</th><th>history_0_start</th><th>history_0_end</th><th>history_0_institution_country</th><th>history_0_institution_domain</th><th>emailsConfirmed_0</th><th>profile_id</th></tr></thead><tbody><tr><td>0</td><td>name@university.edu</td><td>https://test.com</td><td>name@university.edu</td><td>True</td><td>First Last</td><td>~First_Last2</td><td>PhD Student</td><td>2017</td><td>None</td><td>US</td><td>university.edu</td><td>name@university.edu</td><td>~First_Last2</td></tr></tbody></table>

There will be multiple columns for some profile fields recording each of the entries, for example: `names_0_preferred, names_0_fullname`. Because profiles have different numbers of affiliations in their profile, some of these columns will be null for some profiles.

### QuickStart: Getting All Profiles

The code below takes a list of profile IDs or emails, and returns a DataFrame with all of the profile information in it in a tabular format.

```python
client_v2 = #connect to the OpenReview Client (API2) with your credentials

from collections.abc import MutableMapping

list_of_profile_ids = []
profile_list = openreview.tools.get_profiles(client_v2,list_of_profile_ids)


def flatten_dict(d, parent_key='', sep='_'):
    """
    Recursively flattens a dictionary, concatenating nested keys.
    """
    items = []
    for k, v in d.items():
        new_key = f"{parent_key}{sep}{k}" if parent_key else k
        if isinstance(v, MutableMapping):
            items.extend(flatten_dict(v, new_key, sep=sep).items())
        elif isinstance(v, list):
            for i, elem in enumerate(v):
                # Handle lists of dictionaries by adding an index
                if isinstance(elem, MutableMapping):
                    items.extend(flatten_dict(elem, f"{new_key}_{i}", sep=sep).items())
                else:
                    # Just add the element if it's not a dictionary
                    items.append((f"{new_key}_{i}", elem))
        else:
            items.append((new_key, v))
    return dict(items)

def extract_content(d):
    flattened = flatten_dict(d.content)
    content = {k: v for k, v in flattened.items()}
    content['profile_id'] =d.id
    return(content)


#Create a DataFrame with the flattened profile content + profile ID
profile_df = pd.DataFrame([extract_content(note) for note in profile_list)

#extract the columns you want included in the data
relevant_columns = ['profile_id'] + [c for c in profile_df.columns if 'history_0' in c] 
profile_df_subset = profile_df[relevant_columns]
```

Once the DataFrame is created, it is possible to create a CSV with this data, or merge it with other OpenReview data. See [here](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-loop-through-accepted-papers-and-print-the-authors-and-their-affiliations) for examples on how to combine profile with submission data.


# Introduction to Notes

See also:

* [How-to guide for notes](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes)
* [Technical Reference for Notes](https://docs.openreview.net/reference/api-v2/entities/note)
* [Guide for getting notes (submissions, reviews, etc)](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-get-all-notes-for-submissions-reviews-rebuttals-etc)
* [Guide for exporting submission and other information](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-loop-through-accepted-papers-and-print-the-authors-and-their-affiliations)

### What is a note?

The **Note** is the core data object in OpenReview that is used to represent content such as submissions, reviews, comments, official responses, and more. Notes are created and governed by **invitations**, which define their schema and permissions.

For detailed information about the Note object and its fields, see [here](https://docs.openreview.net/reference/api-v2/entities/note). A few fields that are particularly important for interacting with notes are `id`, `invitation` ,`readers` , and `content`.

* `id` - unique identifier for the Note
* `invitation` is used to identify the schema used to generate and validate the note. In practice, this is most commonly used for identifying all notes for example identifying all Reviews or all Rebuttals.
* `readers` identifies which groups can retrieve the Note. Individual fields inside `content` can carry their own `readers` as well, see [Readers of a Note and readers of a field](#readers-of-a-note-and-readers-of-a-field)
* `content` is a dictionary that contains all of the fields of a note.

### Notes, Replies, and Forums

A **Reply** is a specific type of Note that is **linked to another Note**—such as a comment on a submission or a review in response to a paper.

**`id`**

* The unique identifier of a specific Note.
* Every Note, whether standalone or a reply, has its own `id`.

**`forum`**

* The `forum` field indicates the **top-level Note thread** the current Note belongs to.
* For a submission, `forum == id`, since it starts its own thread.
* For a reply, `forum` is the `id` of the original submission Note it belongs to.

**`replyto`**

* The `replyto` field indicates the **direct parent** of the current Note.
* A top-level Note (like a submission) has `replyto=None`.
* A review or comment will have `replyto` set to the `id` of the Note it is directly responding to (could be a submission or another comment).

In OpenReview, typically Submissions are the top-level Note, and other Notes posted in relationship to the Submission is a reply. The example below shows a forum page for an example submission. In this case, the Submission is the top-level note, the Official Review is a reply to the Submission, and the Official Comment is a reply to the Official Comment.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FTc4rtkSKZoHY7Q6SDw7E%2FScreenshot%202025-06-09%20at%201.34.03%E2%80%AFPM.png?alt=media&#x26;token=fa8cafe7-3840-4900-8678-cb0807f9e1e5" alt=""><figcaption></figcaption></figure>

The data structure of the above forum is as below:

```
{'cdate': 1749500872124,
 'content': {'abstract': {'value': 'This is an abstract 1'},
             'authorids': {'readers': ['TC/2024/Conference',
                                       'TC/2024/Conference/Submission1/Authors'],
                           'value': ['~SomeFirstName_User1',
                                     'author@mail.com',
                                     'andrew1@amazon.com']},
             'authors': {'readers': ['TC/2024/Conference',
                                     'TC/2024/Conference/Submission1/Authors'],
                         'value': ['SomeFirstName User',
                                   'First Last',
                                   'Andrew Mc']},
             'keywords': {'value': ['machine learning', 'nlp']},
             'paperhash': {'readers': ['TC/2024/Conference',
                                       'TC/2024/Conference/Submission1/Authors'],
                           'value': 'user|paper_title_1'},
             'pdf': {'value': '/pdf/pppppppppppppppppppppppppppppppppppppppp.pdf'},
             'title': {'value': 'Paper title 1'},
             'venue': {'value': 'TC 2024 Conference Submission'},
             'venueid': {'value': 'TC/2024/Conference/Submission'}},
 'ddate': None,
 
 'domain': 'TC/2024/Conference',
 'forum': 'WEIzUBsDX6',
 'id': 'WEIzUBsDX6',
 'invitations': ['TC/2024/Conference/-/Submission',
                 'TC/2024/Conference/-/Post_Submission'],
 'license': 'CC BY-SA 4.0',
 'mdate': 1749500934613,
 'nonreaders': None,
 'number': 1,
 'odate': None,
 'parent_invitations': None,
 'pdate': None,
 'readers': ['TC/2024/Conference',
             'TC/2024/Conference/Submission1/Senior_Area_Chairs',
             'TC/2024/Conference/Submission1/Area_Chairs',
             'TC/2024/Conference/Submission1/Reviewers',
             'TC/2024/Conference/Submission1/Authors'],
 'replyto': None,
 'signatures': ['TC/2024/Conference/Submission1/Authors'],
 'tcdate': 1749500872124,
 'tmdate': 1749500934613,
 'writers': ['TC/2024/Conference', 'TC/2024/Conference/Submission1/Authors']},
 'details': {'replies': [{'cdate': 1749501156614,
                          'content': {'confidence': {'value': 5},
                                      'rating': {'value': 10},
                                      'review': {'value': 'Excellent paper, '
                                                          'accept'},
                                      'title': {'value': 'Good paper, accept'}},
                          'domain': 'TC/2024/Conference',
                          'forum': 'WEIzUBsDX6',
                          'id': 'OvggXPAvUK',
                          'invitations': ['TC/2024/Conference/Submission1/-/Official_Review'],
                          'license': 'CC BY 4.0',
                          'mdate': 1749501156614,
                          'nonreaders': ['TC/2024/Conference/Submission1/Authors'],
                          'number': 1,
                          'parentInvitations': 'TC/2024/Conference/-/Official_Review',
                          'readers': ['TC/2024/Conference/Program_Chairs',
                                      'TC/2024/Conference/Submission1/Senior_Area_Chairs',
                                      'TC/2024/Conference/Submission1/Area_Chairs',
                                      'TC/2024/Conference/Submission1/Reviewer_Nxce'],
                          'replyto': 'WEIzUBsDX6',
                          'signatures': ['TC/2024/Conference/Submission1/Reviewer_Nxce'],
                          'tcdate': 1749501156614,
                          'tmdate': 1749501156614,
                          'version': 2,
                          'writers': ['TC/2024/Conference',
                                      'TC/2024/Conference/Submission1/Reviewer_Nxce']},
                         {'cdate': 1749501221454,
                          'content': {'comment': {'value': 'Thank you for your '
                                                           'review!'}},
                          'domain': 'TC/2024/Conference',
                          'forum': 'WEIzUBsDX6',
                          'id': '9mJiUp5fsF',
                          'invitations': ['TC/2024/Conference/Submission1/-/Official_Comment'],
                          'license': 'CC BY 4.0',
                          'mdate': 1749501221454,
                          'number': 1,
                          'parentInvitations': 'TC/2024/Conference/-/Official_Comment',
                          'readers': ['TC/2024/Conference/Program_Chairs',
                                      'TC/2024/Conference/Submission1/Senior_Area_Chairs',
                                      'TC/2024/Conference/Submission1/Area_Chairs',
                                      'TC/2024/Conference/Submission1/Reviewer_Nxce'],
                          'replyto': 'OvggXPAvUK',
                          'signatures': ['TC/2024/Conference/Submission1/Senior_Area_Chairs'],
                          'tcdate': 1749501221454,
                          'tmdate': 1749501221454,
                          'version': 2,
                          'writers': ['TC/2024/Conference',
                                      'TC/2024/Conference/Submission1/Senior_Area_Chairs']}]},
}
```

### Readers of a Note and readers of a field

`readers` appears at two different levels, and the distinction decides what a user actually gets back when they query a Note.

**Note level.** The `readers` field at the top of the Note lists the groups that can retrieve the Note at all. A user who is not in any of these groups will not see the Note in the UI and will not get it back from the API.

**Field level.** Any field inside `content` can carry its own `readers` list. Users outside that list still retrieve the Note, but the field is omitted from the `content` they receive. This is how a submission can be visible to reviewers while its `authors` and `authorids` stay hidden from them.

In the forum example above, the submission is readable by the venue, the Senior Area Chairs, the Area Chairs, the Reviewers, and the Authors, but `authors`, `authorids`, and `paperhash` each narrow their own readers to the venue and the paper's Authors:

```
'readers': ['TC/2024/Conference',
            'TC/2024/Conference/Submission1/Senior_Area_Chairs',
            'TC/2024/Conference/Submission1/Area_Chairs',
            'TC/2024/Conference/Submission1/Reviewers',
            'TC/2024/Conference/Submission1/Authors'],
'content': {'title': {'value': 'Paper title 1'},
            'authors': {'readers': ['TC/2024/Conference',
                                    'TC/2024/Conference/Submission1/Authors'],
                        'value': ['SomeFirstName User', 'First Last', 'Andrew Mc']}}
```

A Reviewer querying this Note receives it with `title` but without `authors`.

Two consequences are worth keeping in mind:

* Field readers can only **narrow** access. A group that cannot read the Note cannot see any of its fields, no matter what the field's `readers` say.
* A field with no `readers` of its own is visible to everyone who can read the Note.

Which level to change depends on what you are trying to do. To change who can see a whole submission, review, or comment, change the readers of the Note, usually by running the relevant stage. To hide one field from part of the committee, set the readers of that field. See [How to hide/reveal fields](https://docs.openreview.net/how-to-guides/submissions-comments-reviews-and-decisions/how-to-hide-reveal-fields) for the procedure, and [Setting the Readers of a Field](https://docs.openreview.net/getting-started/customizing-forms#setting-the-readers-of-a-field) for how to declare field readers when customizing a form.

### Creating, Editing, and Deleting Notes (Inference)

Operations on Notes are performed via [**Edits**](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-edits)**.** The first Edit posted is for the creation of the note, any revisions of the notes can be made by posting Edits with the relevant fields, and deleting a Note object is done by posting and Edit adding a date to the `ddate` field. See below for examples of each operation.

To create/modify a Note, you post an Edit with the necessary fields:

* **`invitation`**: The ID of the Invitation governing the Note. This is used to ensure that the Edit is well-formed and conforms to the specifications in the invitation.
* **`signatures`**: List of group IDs authorizing the Edit.
* **`note`**: The Note object containing fields like `readers`, `writers`, `signatures`, and `content` . All notes need to have these four fields. `readers`, `writers`, and `signatures` are all lists of strings consisting of the IDs of groups or profiles. `content` is a dictionary containing the fields that will show up in the final Note. the dictionary needs to include only fields specified in the invitation. Between the notes and all edits of the note, all required fields must be present in the note.

### Note Validation

Before the [Edit](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-edits) is posted, the content of both the Edit and the Note within is checked to ensure that it fits the parameters of the Invitation. In particular, the dictionary `content` needs to include only fields specified in the invitation. Between the notes and all edits of the note, all required fields must be present in the note.


# Introduction to Edits

See also:

* [Technical Reference for Edits](https://docs.openreview.net/reference/api-v2/entities/edit)
* [How to create, edit, and delete notes](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes)

In OpenReview, an **Edit** is the object used for creating, modifying, or deleting entities (such as Notes, Groups, Edges, etc.). Edits are posted and then **applied via inference rules** defined by invitations.

See the[ API reference ](https://docs.openreview.net/reference/api-v2/entities/edit)for a comprehensive description of the fields of the Edit object. Most commonly, you will interact with the following key fields:

* `invitation`: ID of the invitation that governs the operation
* `signatures`: Who is performing the edit
* `note`: The data of the note (new content or modifications)
* `ddate`: A timestamp to **delete** a note (in ms time). The default value for this field is `None`

### Edits and Inference

Inference refers to **how the system processes edits**. The target invitation defines **what kind of edits are allowed**, **what metadata is required**, and **how the system responds** to them (e.g., assigning readers, creating edges, triggering workflows).

* Invitations can specify a `process` function that determines side effects.
* The system checks permissions, field formats, and required values using the invitation’s rules.

Below is an example of an edit to a review. The Edit object includes the metadata for the Edit, and nested within is the metadata and content for the [Note](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-notes) to be updated.

{% hint style="info" %}
There are two different reader fields here - one for the Edit (outer layer) and one within the Note object which are the readers of the Note to update. When updating the readers, it is helpful to double-check that the correct values are being update.
{% endhint %}

```
{'cdate': 1750702362496,
 'content': None,
 'ddate': None,
 'domain': 'TC/2024/Conference',
 'group': None,
 'id': 'a9tHyMftoj',
 'invitation': 'TC/2024/Conference/Submission1/-/Official_Review',
 'invitations': None,
 'nonreaders': ['TC/2024/Conference/Submission1/Authors'],
 'note': Note(id = 'vz9qfSrB0P',
              number = 1,
              cdate = None,
              pdate = None,
              odate = None,
              mdate = None,
              tcdate = None,
              tmdate = None,
              ddate = None,
              content = {'title': {'value': 'Good paper, accept'}, 
                        'review': {'value': 'Excellent paper, accept'}, 
                        'rating': {'value': 8}, 
                        'confidence': {'value': 5}},
              forum = 'lOns9yOgZf',
              replyto = 'lOns9yOgZf',
              readers = ['TC/2024/Conference/Program_Chairs', 'TC/2024/Conference/Submission1/Senior_Area_Chairs', 'TC/2024/Conference/Submission1/Area_Chairs', 'TC/2024/Conference/Submission1/Reviewer_4iHs'],
              nonreaders = ['TC/2024/Conference/Submission1/Authors'],
              signatures = ['TC/2024/Conference/Submission1/Reviewer_4iHs'],
              writers = ['TC/2024/Conference', 'TC/2024/Conference/Submission1/Reviewer_4iHs'],
              details = None,
              invitations = None,parent_invitations = None,domain = None,license = None),
 'readers': ['TC/2024/Conference/Program_Chairs',
             'TC/2024/Conference/Submission1/Senior_Area_Chairs',
             'TC/2024/Conference/Submission1/Area_Chairs',
             'TC/2024/Conference/Submission1/Reviewer_4iHs'],
 'signatures': ['TC/2024/Conference/Submission1/Reviewer_4iHs'],
 'tauthor': 'openreview.net',
 'tcdate': 1750702362496,
 'tmdate': 1750702362496,
 'writers': ['TC/2024/Conference']}
```

### How to Get Edits

To get all edits, use the following code:

```python
note_id = ''
edits = client_v2.get_note_edits(note_id)
edits
```


# Introductions to Invitations

See also:

* [Technical Reference for Invitations](https://docs.openreview.net/reference/api-v2/entities/invitation)
* How to Guides:
  * [Change Submission Expiration Date](https://docs.openreview.net/how-to-guides/workflow/how-to-change-the-expiration-date-of-the-submission-invitation)
  * [Change Submission Readers](https://docs.openreview.net/how-to-guides/workflow/how-to-change-who-can-access-submissions-after-the-deadline)
  * [Hide Submission Fields](https://docs.openreview.net/how-to-guides/workflow/how-to-hide-and-unhide-submission-fields-from-reviewers)
  * [Customizing Forms](https://docs.openreview.net/getting-started/customizing-forms)

In OpenReview, **invitations** define the rules and permissions for creating or editing entities such as [notes](https://docs.openreview.net/getting-started/objects-in-openreview/introduction-to-notes), [groups](https://docs.openreview.net/getting-started/objects-in-openreview/groups), and edges. They act as **templates** that control who can perform specific actions, what content is expected, and how submissions are processed.

The vast majority of invitations will be created automatically at the appropriate point of the workflow, and you will interact with invitations via the UI and stages within to change settings.

### Structure of Invitations

See the [API Reference](https://docs.openreview.net/reference/api-v2/entities/invitation) for a detailed description of each field of the invitation. The main fields that you will interact with are:

* `id`: A unique identifier (e.g., `ICLR.cc/2025/Conference/-/Area_Chair_Registration`)
* `invitees`: Who is allowed to use the invitation (e.g. Area Chairs)
* `signatures`: Who issued the invitation (typically a venue or program chair group)
* `edit`: Defines the type of object that can be posted (e.g., `note`, `edge`)
* `reply` or `content`: Schema specifying required fields, validation, and structure
* `readers`, `writers`, and `signatures` constraints for resulting entities

### Creating Invitations

The vast majority of invitations will be created automatically at the appropriate point of the workflow, and you will interact with invitations via the UI and stages within to change settings.

In some cases, you may need to create custom invitations to accommodate less commons workflow. In this case, see guidance for those specific stages, or the Custom Stage.

### Editing invitations

1. **Pre-created invitations:** The vast majority of edits to invitations are made from the [Workflow Timeline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#workflow-timeline). Examples: [Hiding submission fields](https://docs.openreview.net/how-to-guides/workflow/how-to-hide-and-unhide-submission-fields-from-reviewers), [releasing reviews](https://docs.openreview.net/how-to-guides/workflow/how-to-release-reviews-decisions-and-metareviews)
2. **Custom Invitations:** For custom invitations that are not configurable in the venue configuration form, you may need to make changes directly to those invitations. In those cases, changes can be made in the Invitation's page, found on your venue's group page: <kbd><http://openreview.net/group/edit?id={venue\\_id}></kbd>
3. **Changes to per-paper invitations:** If you need to make programmatic changes to per-paper invitations for many submissions, you may want to make these changes programmatically.

{% hint style="warning" %}
Each time you edit a step, OpenReview recreates its invitation for every submission from the step's settings, so **changes made directly to those invitations are overwritten**.
{% endhint %}

### Deleting invitations

Rather than outright deleting invitations, we use the expiration date. Update the `exp_date` field of the invitation to now or a past time (in ms time), which will stop any new posts from being made with the invitations. Example: [Updating Submission Expiration](https://docs.openreview.net/how-to-guides/workflow/how-to-change-the-expiration-date-of-the-submission-invitation). To turn off a whole step of your venue, click **Disable** next to it in the Workflow Timeline.


# Using the API

There are currently two APIs supported. The current API (referred to in the documentation as API or API 2) is the current API version, and is the default version used for all operations unless otherwise specified.

The legacy API v1 is being phased out, but is still used for some conferences (primarily those before 2024)

While most operations will work on both APIs, pay careful attention when that is not the case, for example, the JSON format for each API is different.

## You need an OpenReview account

You need an OpenReview account to use the API. You authenticate with the same username and password you use on the website, and the API enforces the same permissions as the site does: you can only read and edit what your account is allowed to read and edit.

If you do not have an account yet, [sign up for one](https://docs.openreview.net/getting-started/creating-an-openreview-profile/signing-up-for-openreview) before going any further. Once you have one, see [Installing and Instantiating the Python client](https://docs.openreview.net/getting-started/using-the-api/installing-and-instantiating-the-python-client) for how to authenticate.


# Installing and Instantiating the Python client

1. You will need to install the **openreview-py** package.

```bash
pip install openreview-py
```

2\. Create a client object with your OpenReview credentials. If you do not yet have an OpenReview profile, [you will need to make one now](https://docs.openreview.net/getting-started/creating-an-openreview-profile/signing-up-for-openreview).

```python
import openreview

# API V2
client = openreview.api.OpenReviewClient(
    baseurl='https://api2.openreview.net',
    username=<your username>,
    password=<your password>
)

# API V1
client_v1 = openreview.Client(
    baseurl='https://api.openreview.net',
    username=<your username>,
    password=<your password>
)
```

{% hint style="info" %}
If your profile has multi-factor authentication enabled, the client asks for a verification code in your terminal or notebook when it logs in. For scripts that run unattended, where nobody can enter the code, see [Using multi-factor authentication in unattended scripts](https://docs.openreview.net/getting-started/using-the-api/using-multi-factor-authentication-in-unattended-scripts).
{% endhint %}


# Using multi-factor authentication in unattended scripts

If your profile has [multi-factor authentication](https://docs.openreview.net/getting-started/creating-an-openreview-profile/setting-up-multi-factor-authentication) enabled, the Python client asks for a second verification step when it logs in. Whether you need this page depends on whether someone is there to answer it:

* **You run the script or notebook yourself:** nothing to change. The client asks in your terminal or notebook for the code from your authenticator app or your email, or opens your browser to confirm with a passkey, and then continues.
* **The script runs unattended:** nobody is there to answer, so the login fails with `MfaRequiredException`. This page shows how to set up an authenticator app so the script can generate the code itself.

Unattended login works with the **authenticator app** method only. Email codes and passkeys still need a person to complete them.

## How it works

An authenticator app generates a new 6-digit code every 30 seconds from a secret shared between you and OpenReview. Any program that knows the secret produces the same codes, so you can keep the secret in Google Authenticator on your phone **and** give it to your scripts, and both will work.

Your script generates the code with [PyOTP](https://pypi.org/project/PyOTP/) and hands it to the client through the `mfa_code_provider` parameter. The client never receives the secret itself, so you decide where it is stored.

## 1. Install PyOTP

```bash
pip install pyotp
```

## 2. Enable the authenticator app and get its secret

The **Password & Security** page on the website shows the setup secret only as a QR code. To use the secret in scripts you need it as text, so enable the authenticator app with the following script instead. Run it once, in a terminal.

```python
import base64

import openreview
import pyotp

client = openreview.api.OpenReviewClient(
    baseurl='https://api2.openreview.net',
    username=<your username>,
    password=<your password>
)

response = client.session.post(client.mfa_setup_totp_init_url, headers=client.headers)
response.raise_for_status()
setup = response.json()
secret = setup['secret']

with open('openreview-totp-qr.png', 'wb') as f:
    f.write(base64.b64decode(setup['qrCodeDataUrl'].split(',', 1)[1]))

response = client.session.post(client.mfa_setup_totp_verify_url, headers=client.headers, json={'code': pyotp.TOTP(secret).now()})
response.raise_for_status()

print(f'Authenticator app enabled. Secret: {secret}')
print('QR code saved to openreview-totp-qr.png')
recovery_codes = response.json().get('recoveryCodes')
if recovery_codes:
    print('Recovery codes:')
    print('\n'.join(recovery_codes))
```

When it finishes:

1. **Save the recovery codes.** They are printed only if your profile has no recovery codes yet (usually, when this is your first multi-factor method), and they cannot be shown again.
2. **Store the secret** somewhere safe, such as a password manager, your operating system's keychain, or your CI system's secret store. Never put it in a script or a Git repository.
3. **Optional:** scan `openreview-totp-qr.png` with Google Authenticator or another authenticator app to get the same codes on your phone.
4. **Delete `openreview-totp-qr.png`.** It contains the secret.

{% hint style="warning" %}
A profile has a single authenticator-app secret. Running this script again replaces it, and codes from the old secret stop working everywhere. That includes your phone, so scan the new QR code there too.
{% endhint %}

{% hint style="info" %}
If you already enabled the authenticator app on the website, the secret behind its QR code is not shown anywhere. Run the script above to replace it with one you can use in scripts.
{% endhint %}

## 3. Log in from your scripts

Pass a function that returns the current code as `mfa_code_provider`. This example reads the secret from the `OPENREVIEW_TOTP_SECRET` environment variable:

```python
import os

import openreview
import pyotp

client = openreview.api.OpenReviewClient(
    baseurl='https://api2.openreview.net',
    username=<your username>,
    password=<your password>,
    mfa_code_provider=lambda method: pyotp.TOTP(os.environ['OPENREVIEW_TOTP_SECRET']).now()
)
```

The API V1 client, `openreview.Client`, accepts the same parameter.

The function receives the name of the method, which is always `'totp'`, and must return the code as a string. You can get the code from anywhere: an environment variable, a secrets manager, or a password manager that can generate codes itself.

What the client does with it:

* The function is called only when your profile has the authenticator app enabled, and at most once per login.
* A wrong code is not retried. The login fails with an error that says how many attempts remain.
* If the function returns `None` or an empty string, the client falls back to the interactive prompt. With no terminal available, it raises `MfaRequiredException`.
* Errors raised inside the function reach your script unchanged.

{% hint style="warning" %}
After 5 incorrect codes in a row, verification is locked for 15 minutes. Codes depend on the current time, so keep the clock of the machine that runs your scripts synchronized (for example, with automatic time sync enabled).
{% endhint %}

## Keeping your account safe

Anyone who has both your password and your authenticator secret can sign in as you. Treat the secret like your password:

* Store it in a secrets manager or keychain, not in plain-text files or source code.
* Give it only to the machines that need it.
* If you think it has leaked, change your password and run the setup script again to replace the secret.


# Hosting a venue on OpenReview


# Navigating your Venue Pages

After your venue is deployed, you will have access to multiple pages in OpenReview. These can be accessed through two methods:

1. Navigating to them from the Program Chair Console on the OpenReview website
2. Clicking the relevant link in the venue deployment confirmation email (subject: 'Your Venue is Deployed")

### Venue Homepage

This is the public-facing page for your venue, and can be found under 'Active Venues' on the OpenReview homepage.

### Program Chair Console

You can find a link to your PC console under ‘Active Consoles’ on the OpenReview homepage after you have logged into your account. Use this page to access your workflow timeline, venue request form, committee consoles, and edge browser.

### Workflow Timeline

The workflow timeline contains the timeline and customizations for your venue. Each step has a description inlcuding the important changes that the step will implement. You can use the edit button under each step to make changes to the forms, dates, and other settings for your venue.

### Venue Request Form

The venue request form is primarily used for notifications regarding your venue and communication with OpenReview Support. Add a comment to this page to request support.

### Edge Browser

The edge browser is the place where PCs can see reviewers, papers, and the links between them. This is where PCs will typically go to see and modify assignments for roles like reviewers and area chairs. To open it, go to the Assignments page, at `https://openreview.net/assignments?group=<venue_id>/Reviewers`, or use the **Reviewers Paper Assignment** link in your Program Chairs console. Create an assignment configuration there, and once its status is "Complete", click **Browse Assignments**.

### Committee Member Groups and Consoles

Each type of committee member (Reviewers, Area Chairs, and Senior Area Chairs) for your venue will have their own console. You can access these consoles through each of the links under ‘Venue Roles’ at the bottom of your PC console. This link will take you to the group edit page, then you can use the 'View Group' button to see the console.


# Customizing your submission form

For an overview and basics of form customization, see the comprehensive [Customizing Forms](https://docs.openreview.net/getting-started/customizing-forms).

To make changes to your submission form, navigate to your Workflow Timeline. Then go to the "Submission" Step — > Edit Fields — > Widget. This will allow you to edit the fields in your submission form.

{% hint style="info" %}
To hide a new field from reviewers, go to "Submission Change Before Bidding" and/or "Submission Change Before Reviewing" — > Edit Restrict Field Visibility. You can check which fields are visible to reviewers by going to any submission page and hovering over the eye icon next to the field name.
{% endhint %}

## Common Customizations

### Asking authors to agree to conference policies

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FOpLSJksv0cxqMsABr5RS%2FScreenshot%202024-08-20%20at%2011.13.02%E2%80%AFAM.png?alt=media&#x26;token=6abf77a1-4bf8-4652-aaa7-beb548497773" alt=""><figcaption></figcaption></figure>

<pre class="language-json" data-title=""><code class="lang-json"><strong>{
</strong>  "confirmation": {
    "description": "Please confirm you have read the workshop's policies.",
    "order": 2,
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "I have read and agree with the workshop's policy on behalf of myself and my co-authors."
        ],
        "input": "radio"
      }
    }
  }
}
</code></pre>

### Limit read-permissions for certain fields

The `readers` field can be used to list who will be allowed to read a specific field of the submission form. The example below limits the readers of the `private comments` field to just authors, Assigned Senior Area Chairs, and Program Chairs.

Note: Authors will not be able to read these fields if they are not in the readers list

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FvU0GWDltsGL0FoLTw9Qp%2FScreenshot%202024-08-20%20at%2011.17.59%E2%80%AFAM.png?alt=media&#x26;token=0189e96d-6af2-4aa0-bca8-d83203319f4a" alt=""><figcaption></figcaption></figure>

```json
{
  "private_comments": {
    "value": {
      "param": {
        "type": "string",
        "optional": true
      }
    },
    "readers": [
      "Your/Venue/ID/Program_Chairs",
      "Your/Venue/ID/Submission${4/number}/Senior_Area_Chairs",
      "Your/Venue/ID/Submission${4/number}/Authors"
    ]
  }
}
```

### Adding tracks to your venue

Once you have [reviewed our support for "tracks" in a single venue](https://docs.openreview.net/how-to-guides/workflow/how-to-have-different-tracks-or-types-of-submissions-for-a-single-venue) and you think this is what your venue needs, you can add a "track" field to your submission form if you are using separate reviewing pools for track submissions.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FtMSUgpKAXIlaIM8o8iLx%2FScreenshot%202024-08-20%20at%2011.23.48%E2%80%AFAM.png?alt=media&#x26;token=4a00f07e-9a91-4285-99a7-63318e77b0b6" alt=""><figcaption></figcaption></figure>

```json
{
  "track": {
    "description": "Please select the track you are submitting to.",
    "order": 2,
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "Track 1",
          "Track 2",
          "Track 3"
        ],
        "input": "radio"
      }
    }
  }
}
```


# Allowing authors to nominate an author as reviewer

You can add a field for authors to nominate co-authors as reviewers by editing the submission form.

To make changes to your submission form, navigate to your Workflow Timeline. Then go to the "Submission" Step — > Edit Fields — > Content JSON. Add the following code to the submission JSON.

```json
"serve_as_reviewer": {
    "order": 11,
    "description": "Enter the profile ids of the authors of this submission who will serve as reviewers.",
    "value": {
        "param": {
            "type": "string[]",
            "enum": ["${3/authors/value/*/username}"],
            "input": "select"
        }
    }
}
```

This code will show any users added as authors of the submission in a dropdown. The submitted author will be able to choose one or more authors to serve as reviewer. The validation that the user added is an author of the submission is done by the field, so no need to validate this field with a pre/post-process.


# Enabling Supplementary Material Upload

To add a supplementary material field to the submission form, go to the Workflow Timeline and locate the `Submission` step. Click on "Submission" to expand the configuration for the Submission step and click on Edit next to the Form Fields. You can copy the JSON below and paste it directly to your Content JSON editor:            (make sure to change any details before saving the changes to your submission form)

{% code title="API 2" %}

```
{
    "supplementary_material": {
        "value": {
            "param": {
                "type": "file",
                "extensions": ["zip", "pdf"],
                "maxSize": 50,
                "optional": true,
                "deletable": true
            }
        },
        "description": "All supplementary material must be self-contained and zipped into a single file. Ensure all material is anonymized. The maximum file size is 50MB.",
        "order": 7,
        "readers": [ 
            "ICML.cc/2023/Conference", 
            "ICML.cc/2023/Conference/Submission${4/number}/Authors"
        ]
    }
}
```

{% endcode %}

The field `readers` is optional and it can be used to restrict the readers of the field, if you don't specify the readers then all the readers of the submission will be able to see the supplementary material. Make sure you **use the right group ids** to specify the readers. An placeholder venueid is used in the above example, make sure to change the values for your venue.

This will add a supplementary material field to upload zipped files of size up to 50 MB. You can also enable a Submission Revision Stage to allow a separate deadline for Supplementary Material.


# Changing your submission deadline

Note that you can only have one submission deadline for all submissions, and it is not possible to have different deadlines for different types of papers, or to extend the deadline only for a subset of authors.

To change your submission deadline, edit the Dates field in the Submission step of the workflow timeline.


# Customizing settings and forms for a Journal

These are the settings that you can change for your journal:

Note: AE - Action Editor; EIC - Editor-in-Chief

### Journal Settings

* **website\_urls**
  * Definition: URLs to relevant pages of the website
  * Possible values: dictionary in the format {page\_name : url}
  * Default value: None
* **editors\_email**
  * Definition: List of emails of the editors-in-chief of the journal
  * Possible values: list of strings
  * Default value: None
* **issn**
  * Definition: ISSN of the journal
  * Possible values: string
  * Default value: "XXXX-XXXX"
* **has\_publication\_chairs**
  * Definition:
  * Possible values:
  * Default value: False

### Submission Settings

* **submission\_public**
  * Definition: Whether submissions can be seen by the general public (anyone with the link)
  * Possible values: True, False
  * Default value: True
* **author\_anonymity**
  * Definition: True if authors are anonymous.
  * Possible values: True, False
  * Default value: True
* **assignment\_delay**
  * Definition: Delays the notification to the reviewer in case the AE make a mistake and decides to remove the assignment (in seconds)
  * Possible values: integer
  * Default value: 5
* **expertise\_model**
  * Definition:
  * Possible values:
  * Default value: "specter+mfr"
* **submission\_name**
  * Definition: The name of the submission invitation in the system
  * Possible values: string
  * Default value: "Submission"
* **submission\_length**
  * Definition: The type of the submission
  * Possible values: List of strings from this set:
  * Default value: \[ "Regular submission (submissions may be any length)" ]
* **submission\_license**
  * Definition:
  * Possible values:
  * Default value: "CC BY-SA 4.0"
* **release\_submission\_after\_acceptance**
  * Definition:
  * Possible values: True, False
  * Default value: True
* **eic\_submission\_notification**
  * Definition:
  * Possible values: True, False
  * Default value: False
* **skip\_camera\_ready\_revision**
  * Definition:
  * Possible values: True, False
  * Default value: False

### **Review Settings**

* **skip\_ac\_recommendations**
  * Definition: Set as True to skip the phase where authors recommend AEs for their paper.
  * Possible values: True, False
  * Default value: False
* **number\_of\_reviewers**
  * Definition: Number of reviewers needed to trigger the next step.
  * Possible values: integer
  * Default value: 3
* **reviewers\_max\_papers**
  * Definition: Maximum number of papers that can be assigned to a given reviewer per year
  * Possible values: integer
  * Default value: 6
* **action\_editors\_max\_papers**
  * Definition:
  * Possible values: integer
  * Default value: 12
* **archived\_action\_editors**
  * Definition: If True it allows AEs to keep current assignments, however AEs are moved to the archival group so they aren't given new assignments.
  * Possible values: True, False
  * Default value: True
* **archived\_reviewers**
  * Definition:
  * Possible values:
  * Default value: False
* **expert\_reviewers**
  * Definition: If True creates the Expert\_Reviewers group. Related to certifications.
  * Possible values: True, False
  * Default value: True
* **skip\_reviewer\_responsibility\_acknowledgement**
  * Definition:
  * Possible values: True, False
  * Default value: False
* **skip\_reviewer\_assignment\_acknowledgement**
  * Definition:
  * Possible values: True, False
  * Default value: False
* **show\_conflict\_details**
  * Definition:
  * Possible values: True, False
  * Default value: False

### Duration Settings

* **ae\_recommendation\_period**
  * Definition: Number of weeks authors are able to recommend AEs. (soft deadline)
  * Possible values: integer
  * Default value: 1
* **under\_review\_approval\_period**
  * Definition: Number of weeks the AE has to approve a submission for review.
  * Possible values: integer
  * Default value: 1
* **reviewer\_assignment\_period**
  * Definition: Number of weeks the AE has to assign reviewers.
  * Possible values: integer
  * Default value: 1
* **review\_period**
  * Definition: Length of the review period (in weeks)
  * Possible values: integer
  * Default value: 2
* **discussion\_period**
  * Definition: Length of the discussion period between reviewers (in weeks)
  * Possible values: integer
  * Default value: 2
* **recommendation\_period**
  * Definition: Length of the reviewer recommendation period (in weeks)
  * Possible values: integer
  * Default value: 2
* **decision\_period**
  * Definition: Length of the decision period for AEs (in weeks)
  * Possible values: integer
  * Default value: 1
* **camera\_ready\_period**
  * Definition: Length of the camera ready revision period for authors (in weeks)
  * Possible values: integer
  * Default value: 4
* **camera\_ready\_verification\_period**
  * Definition: Length of the camera ready verification period (in weeks)
  * Possible values: integer
  * Default value: 1

### Form customizations:

See [here](https://github.com/openreview/openreview-py/blob/6afa3b9c6152aaab5993e276e619d71e445c58a2/openreview/journal/invitation.py#L893) to see the default fields of the different forms. (Search for the set\_\[Form Name]\_invitation function).

* **submission\_additional\_fields**
  * Definition: Additional fields/customization for the submission form
  * Possible values: JSON
  * Default value: None.
* **review\_additional\_fields**
  * Definition: Additional fields/ customization for the additional review form
  * Possible values: JSON
  * Default value: None.
* **official\_recommendation\_additional\_fields**
  * Definition: Additional fields/ customization for the official recommendation form
  * Possible values: JSON
  * Default value: None.
* **decision\_additional\_fields**
  * Definition: Additional fields/ customization for the decision form
  * Possible values: JSON
  * Default value: None.

### **Certifications**

* **certifications:**
  * Definition:
  * Possible values: string\[]
  * Default value:
* **eic\_certifications:**
  * Definition:
  * Possible values: string\[]
  * Default value:
* **event\_certifications**
  * Definition:
  * Possible values: string\[]
  * Default value:

These settings are formatted into JSON and updated in the Journal Request Form. An example JSON is below:

```
'submission_public': True,
'author_anonymity': True,
'assignment_delay': 5,
'submission_name': 'Submission',
'certifications': [
    'Featured Certification',
    'Reproducibility Certification',
    'Survey Certification'
],
'eic_certifications': [
    'Outstanding Certification'
],                            
'submission_length': [
    'Regular submission (no more than 12 pages of main content)', 
    'Long submission (more than 12 pages of main content)'
],
'issn': '2835-8856',
'website_urls': {
    'editorial_board': 'https://jmlr.org/tmlr/editorial-board.html',
    'evaluation_criteria': 'https://jmlr.org/tmlr/editorial-policies.html#evaluation',
    'reviewer_guide': 'https://jmlr.org/tmlr/reviewer-guide.html',
    'editorial_policies': 'https://jmlr.org/tmlr/editorial-policies.html',
    'faq': 'https://jmlr.org/tmlr/contact.html'                     
},
'editors_email': 'tmlr-editors@jmlr.org',
'skip_ac_recommendation': False,
'number_of_reviewers': 3,
'reviewers_max_papers': 6,
'ae_recommendation_period': 1,
'under_review_approval_period': 1,
'reviewer_assignment_period': 1,
'review_period': 2,
'discussion_period' : 2,
'recommendation_period': 2,
'decision_period': 1,
'camera_ready_period': 4,
'camera_ready_verification_period': 1,
'archived_action_editors': True,
'expert_reviewers': True,
```


# Creating an OpenReview Profile

Use these pages to create, activate, and maintain your OpenReview profile.

#### Create and activate

* [Signing up for OpenReview](https://docs.openreview.net/getting-started/creating-an-openreview-profile/signing-up-for-openreview): Create an account, complete registration, and understand moderation.
* [Resending an activation link](https://docs.openreview.net/getting-started/creating-an-openreview-profile/resending-an-activation-link): Get a fresh signup/activation link.
* [Expediting Profile Activation](https://docs.openreview.net/getting-started/creating-an-openreview-profile/expediting-profile-activation): Add and confirm an institutional email to speed activation.
* [Information for high school students](https://docs.openreview.net/getting-started/creating-an-openreview-profile/information-for-high-school-students): Information on how to sign up for users aged 13-18

#### Edit your profile details

* [Add or remove a name from your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/add-or-remove-a-name-from-your-profile): Add name variants and set a preferred name.
* [Add or remove an email address from your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/add-or-remove-an-email-address-from-your-profile): Add, confirm, and set your preferred email.
* [Selecting your pronouns](https://docs.openreview.net/getting-started/creating-an-openreview-profile/selecting-your-pronouns): Set pronouns (including custom values).
* [Finding your profile ID](https://docs.openreview.net/getting-started/creating-an-openreview-profile/finding-your-profile-id): Locate your profile identifier.
* [Entering Institutional Data](https://docs.openreview.net/getting-started/creating-an-openreview-profile/entering-institutional-data): Fill education/career history and affiliation fields correctly.

#### Publications and external author pages

* [Importing papers from DBLP](https://docs.openreview.net/getting-started/creating-an-openreview-profile/importing-papers-from-dblp): Import publications using your persistent DBLP URL.
* [Importing publication data from ORCID to your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/importing-publication-data-from-orcid-to-your-profile): Import externally sourced ORCID records.
* [Manually adding a publication to your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/manually-adding-a-publication-to-your-profile): Upload missing publications directly.
* [Finding and adding a Semantic Scholar URL to your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/finding-and-adding-a-semantic-scholar-url-to-your-profile): Find your author URL and add it to profile links.
* [Finding and adding your ACL Anthology URL to your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/finding-and-adding-your-acl-anthology-url-to-your-profile): Add your ACL Anthology author page URL.

#### Fix duplicates

* [Merging Profiles](https://docs.openreview.net/getting-started/creating-an-openreview-profile/merging-profiles): Merge accounts if you have multiple profiles.


# Signing up for OpenReview

### 1. Making or finding a profile

To create a profile, go to <https://openreview.net/signup>. Enter your full name as you would like it to appear in publications. A common format is: First Last

By checking the confirmation box, existing profiles with your name will be listed.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FlknSf3ArbCie3Dxui2TG%2FScreenshot%202026-02-27%20at%2011.14.59%E2%80%AFAM.png?alt=media&#x26;token=3dcde47e-56bb-47fe-8f3c-cec000f78d96" alt=""><figcaption><p>Enter your full name on the signup page and click the confirmation checkbox.</p></figcaption></figure>

Enter an email address and click the "Sign Up" button. You'll be prompted to create a password.

{% hint style="info" %}
OpenReview restricts duplication of certain profile information. Users who create multiple accounts will run into errors when updating their profile, so it is recommended that you make a new account only if there is no existing account.
{% endhint %}

#### Existing Profile Email

If a profile registered to this email already exists, an email is sent to that address to notify them.

#### **Resend Activation Link**

If there is a profile created with your name, but you didn't finish the activation process, you can resend the confirmation link, refer to: [Resending an activation link](https://docs.openreview.net/getting-started/creating-an-openreview-profile/resending-an-activation-link). Enter your email address and click ‘Didn't receive email confirmation?’, you will receive an email with a link that will bring you to the OpenReview [registration page](#profile-registration) (Step 2).

#### **Reset Password**

If you don't remember the password to your profile and your profile is active, from the [login page](https://openreview.net/login) click on the 'Forgot your password?' link. You can also visit the [Request Password Reset](https://openreview.net/reset) page to send a password reset link to your registered email.

If you do not receive a password reset email, refer to: [I didn't receive a password reset email, what do I do?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-didnt-receive-a-password-reset-email-what-do-i-do)

#### **Create a new account**

If you have no existing profile, to create a new one fill in your email address in the text field next to the ‘Sign Up’ option and click the 'Sign Up' button. We recommend that you sign up with your institutional (university, company, or organization) email.

If you sign up with a public domain or an email domain that is not included in our institution list you will see a warning:

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FFIfVi3ZRll1GNtwJzGG7%2FScreenshot%202026-02-27%20at%2010.20.10%E2%80%AFAM.png?alt=media&#x26;token=68a2d906-752d-417e-a5cb-cb4af4fddd18" alt="The warning reads: gmail.com does not appear in our list of publishing institutions. It can take up to 2 weeks for profiles using public email services to be activated. To activate immediately, please sign up with an email address that uses an educational or employing institution domain. If your institution is not yet in our list, contact us to request that it be added."><figcaption><p>Email domain warning.</p></figcaption></figure>

If you are using an institutional email and get this warning, please proceed with registration, we routinely add new, verified domains to the list. Signing up with a public domain may take up to 2 weeks in moderation. See [here](https://docs.openreview.net/getting-started/creating-an-openreview-profile/expediting-profile-activation) for tips on expediting this process.

After typing your email you will then be prompted to enter a new password and send a confirmation email. Clicking the link in the confirmation email will bring you to the registration page.

### **2. Profile Registration**

If you have resent an activation link, or created a new profile, you will receive an email with the subject 'OpenReview Signup Confirmation'. Follow the link in this email to complete your profile registration. Profile registration is a required step in order to have an active OpenReview Profile.

Providing as complete information as possible, especially an institutional email address and valid personal homepage, will help your account be activated quickly.

A valid homepage is a website that shows your name, affiliation, and email that you used to register for OpenReview. This could be a Google Scholar profile, personal webpage, GitHub, institutional profile page, or other professional homepage.

{% hint style="info" %}
Note: A landing page for an entire company, institution, or department, or a social media account that uses a screen name, is not a valid homepage
{% endhint %}

If you do not have an institutional affiliation, you may sign up as an Independent Researcher. See [here](https://docs.openreview.net/getting-started/frequently-asked-questions/i-am-an-independent-researcher-how-do-i-sign-up) for detailed instructions on what to include in your profile.

#### Common issues for Profile registration:

* **Validation errors for career/education history or institutional data**: See [Entering Institutional Data](https://docs.openreview.net/getting-started/creating-an-openreview-profile/entering-institutional-data) for a detailed description and examples.
* **Homepage duplication:** If you get an error that one of your links can't be added, it is either in the wrong format or already added to another account. Check that there are no exiting profiles with your email, and if you still don't recognize the account listed in the error, contact OpenReview Support through our [Feedback form](https://openreview.net/contact).

### 3. Moderation

After clicking ‘Register for OpenReview’, your profile will either be activated immediately or sent to moderation. The support team reviews moderation regularly, and will follow up with additional actions necessary.

If you receive an email indicating that additional information is required to activate your OpenReview profile, return to the Profile Registration Step by [resending the activation link](https://docs.openreview.net/getting-started/creating-an-openreview-profile/resending-an-activation-link), and complete the missing information. See [Expediting Profile Activation](https://docs.openreview.net/getting-started/creating-an-openreview-profile/expediting-profile-activation) for additional information.


# Resending an activation link

The expiration date of an activation link is listed in the activation email. If you need to access the link again to complete your profile registration, confirm an email or update a pending profile, you can request another activation link by going to[ https://openreview.net/login](https://openreview.net/login) entering your email and click the ‘Didn’t receive confirmation email?’ button.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FA2M27WcnETVDZPIVGYcS%2FScreenshot%202024-08-23%20at%2012.34.04%E2%80%AFPM.png?alt=media&amp;token=d7df47f4-918a-4b5e-9c48-db9468391c4a" alt="" width="369"><figcaption></figcaption></figure>


# Expediting Profile Activation

If you received an email with the subject ‘OpenReview profile activation pending’, your profile is in moderation. The fastest way to activate an OpenReview Account is to ensure that you have an email associated with your school or company added and confirmed to your profile.

{% hint style="info" %}
Note: Do not try to register a second time with your institutional email, which often causes users to run into issues with duplicate information in their multiple profiles. Rather, add your institutional email to your pending account to expedite profile activation
{% endhint %}

## How to add an institutional email to a pending account

Please note that all three steps must be completed for the institutional email to be confirmed.

1. Open the activation link sent to your email (subject: OpenReview signup confirmation). If your link is expired resend it following the directions [here](https://docs.openreview.net/getting-started/creating-an-openreview-profile/resending-an-activation-link).
2. Add an email associated with your institution (university, lab, or company) to your profile. Make sure to press Confirm to save the email to your account.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FPeUUtO1lCbfRGoWeeV2O%2FScreenshot%202024-08-23%20at%2011.26.50%E2%80%AFAM.png?alt=media&amp;token=9dbc6cd5-58b1-482c-bd5f-0f86f5d6812c" alt="" width="563"><figcaption></figcaption></figure>

3. DO NOT close your current window/tab. The verification number sent to your inbox needs to be input in the same window/tab.
4. Confirm that email by pasting the verification code and clicking on Verify.

## Common questions about pending profiles

### I want to publish / receive emails under a personal email, what do I do?

After creating an account, you can set your preferred email to your personal email account, and this will be the email used for submissions and notifications.

### I have an institutional email in my profile, why is my profile still pending?

Please check that your email has (confirmed) next to it in your profile page. If your email is confirmed, it is likely that your institution has not yet been added to our list. We routinely add new domains to the institution list.

### I don’t have an institutional email, what else can I do to speed up the process?

If you don’t have any institutional emails, please ensure your other profile information is complete. Ensuring your homepage displays your email and career details, along with a fully completed profile, will facilitate the activation process. We will follow up if we need more information.

### Related pages:

[I am an Independent Researcher, how do I sign up?](https://docs.openreview.net/getting-started/frequently-asked-questions/i-am-an-independent-researcher-how-do-i-sign-up)


# Add or remove a name from your profile

Users can keep multiple names in their profiles and select one as preferred, which will be used for author submission and identity display. Names can be replaced by new names in the profile and in some submissions as long as the organizers of the venue allow it.

#### Adding a name to your profile and mark it as preferred

1. Go to your profile page at <https://openreview.net/profile>
2. Click 'Edit Profile'.
3. Locate the 'Names' section and click the blue plus sign underneath your name.
4. Enter your name and then click ‘Save Profile Changes’ at the bottom of the page.

If you would like to make that name preferred, you can do so by clicking ‘Edit profile’ once more and selecting ‘Make Preferred’ next to your desired name. The option to make a name preferred will not appear until you have saved the new name to your profile.

{% hint style="info" %}
Submissions can be updated with the user's preferred name if the organizers of the venue that the user submitted to allow it.
{% endhint %}

#### Removing a name from your profile

1. Go to your profile page at <https://openreview.net/profile>
2. Click 'Edit Profile'.
3. Locate the 'Names' section and make sure you have a name marked as preferred that you want to keep.
4. Click on the button 'Request Deletion' next to the name you want to remove.
5. In the new window, explain why you want that name to be removed and click 'Submit'

The process for removing the name will take some time until has been reviewed and accepted. You will receive email updates regarding the status of the name removal. Once the process is complete the name will be removed and any publications with the previous name will be updated with the one you marked as preferred in your profile.

{% hint style="info" %}
OpenReview doesn't take any action on PDF files so the name change in the file should also be authorized by the organizers of the venue.
{% endhint %}


# Add or remove an email address from your profile

#### Adding email addresses to your profile and mark them as preferred

1. Go to your profile page at <https://openreview.net/profile>
2. Click 'Edit Profile'.
3. Locate the 'Emails' section and click the blue plus sign underneath your name.
4. Enter your email and then click 'Confirm'. An email will be sent to your new address.
5. DO NOT close your current window/tab. The verification number sent to your inbox needs to be input in the same window/tab.
6. Confirm that email by pasting the verification code and clicking on Verify.

If you would like to make that email preferred, you can do so by clicking ‘Edit profile’ once more and selecting ‘Make Preferred’ next to your desired email. The option to make an email preferred will not appear until you have confirmed the new email to your profile.

{% hint style="info" %}
We recommend adding at least one permanent (non-institutional) email to your profile to allow you to retain ownership of your account should you lose access to your institutional email
{% endhint %}

#### Removing email addresses from your profile

OpenReview uses email addresses associated with current or former affiliations for profile deduplication, conflict detection, and paper coreference. For this reason, OpenReview prefers to keep all current and former institutional email addresses on each user's profile. OpenReview will only send messages to the address marked as “Preferred”. OpenReview only displays obfuscated emails (\*\*\*\*@umass.edu) and never releases full email addresses to the public.

If there is an email address you want removed from your profile, please fill out the [Feedback Form](https://openreview.net/contact). Please note that if you have changed institutions, we ask you to add and confirm your new institutional email before requesting that we remove an old or expired email.


# Selecting your pronouns

The Profile Edit page will allow users to select pronouns for their profile. This dropdown field is optional and it contains four pre-defined options to choose from: they/them, he/him, she/her, and not specified.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FqeAeOi3vnggePauTIJF4%2FScreenshot%202025-12-12%20at%203.31.54%E2%80%AFPM.png?alt=media&amp;token=9524054e-cd4e-49b1-b24d-ca8872596c35" alt=""><figcaption></figcaption></figure>

To customize your pronouns, type them into the textbox and click on your custom value in the dropdown to select it.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F3VMqSQRSBgles86JWXeo%2FScreenshot%202025-12-12%20at%203.33.11%E2%80%AFPM.png?alt=media&amp;token=b1fe772a-80f2-494d-bf1c-9053d48be417" alt=""><figcaption></figcaption></figure>

Note: To protect user privacy, pronoun data cannot be retrieved in the API response.


# Finding your profile ID

Your OpenReview profile ID is a unique string made up of a tilde concatenated with your full name and a number, for example ∼First\_Last1. If you go to your OpenReview profile, your ID will be at the end of the url (for example, <https://openreview.net/profile?id=><mark style="background-color:purple;">∼Your\_Id1</mark>)


# Entering Institutional Data

To add institutional data to your OpenReview profile, go to your profile page at <https://openreview.net/profile> and click 'Edit profile'. You must enter at least one position under 'Career & Education History' for your profile to be saved.

Each position requires at least the following information: Position, Start date, Institution Info (domain, name, and country).

For the Position field, you can choose a position from the dropdown or, if none of the existing positions are a good fit, **you may type in a custom role**. Make sure to click in the custom role as it appears in the dropdown in order to select it for that field, otherwise no value will be entered.

Similarly for the institution name and domain, choose from the dropdown or type the name and domain yourself. If you are not sure what domain to type in for your institution, use the email domain used in institutional emails.

If you do not have a current affiliation and are independent, select '**Independent Researcher'** as your position, which will lock the institution name and domain fields automatically.

Independent researchers should use the position Independent Researcher. In this case, the institution domain is filled in automatically (`independent-researcher.org`) and the domain and name fields are disabled. You don't need to enter any additional information.

{% hint style="info" %}
When you type in custom information that data will appear in the dropdown. Click to select it or the textbox will return to blank when you click away.
{% endhint %}

A start date is required for all affiliations. If your affiliation is current, leave the end date blank, otherwise include the end date for past affiliations.

Below are some examples of complete Education and Career History sections:

### Example - complete profile with institutional information

As an example of complete institutional information, this sample user has one past affiliation as a PhD student and one current affiliation. Note that the institution domain, name, and country are all required for the changes to be saved.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F3Q3L1MFmDpIzrlw9fCQy%2FScreenshot%202024-10-18%20at%201.55.14%E2%80%AFPM.png?alt=media&amp;token=bf579519-e625-4f4d-b84e-bebaba23c058" alt=""><figcaption></figcaption></figure>

### Example - complete profile with freeform data

This sample profile includes one past affiliation with a small company not listed in the drop-down menu, and is currently an independent researcher. Here the domain and position information are typed in for the small company.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FsSB4HBfK04LhvawcJllx%2FIndependent%20Researcher%20Example.png?alt=media&amp;token=62c40e55-fbdc-468f-90f8-78d22337c5fb" alt=""><figcaption></figcaption></figure>


# Importing papers from DBLP

### **To add existing DBLP publications to your OpenReview profile**:

#### 1. Login to OpenReview

You can click on the 'Login' button on the right of the navigation menu, or go to <https://openreview.net/login>

#### 2. Navigate to the profile page

Click on your name on the right of the navigation menu and click 'Profile' from the dropdown options, or go to <https://openreview.net/profile>.

![](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FK4XoD1BJdnDOVw72PDoz%2Fimage.png?alt=media\&token=9e999e8b-ee02-42b5-8db0-694b7933f329)

Click 'Edit Profile'.

![](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FqsuuhceIHXlLlavZG57h%2Fimage.png?alt=media\&token=432f9c4c-026a-4dc4-9429-6709fdb19758)

#### 3. Add a DBLP URL

Locate the 'DBLP URL' text field under the 'Personal Links' section.

![](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F9KEXKIiUuCphvPKS1dHP%2Fimage.png?alt=media\&token=0cc33350-d57e-41b5-b928-941908ebd692)

You will need to get the 'Persistent DBLP URL' from your DBLP homepage. To do so, hover over the share icon to the right of your name in DBLP page heading and copy the persistent URL from the hover menu.

![](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FZO9RQZyyh8Ra51H5u9ug%2Fimage.png?alt=media\&token=e15316d4-4402-4e81-a334-17e6ef611c0e)

Paste this persistent url into the DBLP URL field.

**4. Click the "Add DBLP Papers to Profile" button**

If your persistent DBLP url was valid, the option to 'Add DBLP Papers to Profile' will appear. Click the button and your DBLP publications will appear in a modal window.

#### 5. Select papers to upload

Use the checkbox in front of each paper to select those which you would like to import to your OpenReview profile.

#### 6. Click the 'Add to your Profile' button

Click the 'Add to your Profile' button at the bottom of the modal window to import the selected papers.

![](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FyHBegtriQn1Terh6Qe0n%2Fimage.png?alt=media\&token=19b56e4e-9dc8-41c8-9944-69a7023f76e5)

If you get an error that says "please ensure the provided DBLP URL is yours", make sure that the name (or one of the names) in your OpenReview profile matches **exactly** with the name used in DBLP publications. If it does not, you can [add a new name to your profile](https://docs.openreview.net/getting-started/creating-an-openreview-profile/add-or-remove-a-name-from-your-profile), click 'Save Profile Changes', and try again to import your papers.

### To remove publications imported from DBLP

Go to your profile page and click 'Edit Profile'. Click on the 'Personal Links' section to view the publications. All publications associated with your profile will be listed here but those imported from DBLP or ORCID will have a minus icon displayed after the title.

![](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F6rflPh9HngTorNjt1TEe%2Fimage.png?alt=media\&token=071079fb-dcbc-4296-aeb1-db827d196e5e)

You can use this minus button to remove a DBLP publication from your profile. If you mistakenly remove a publication, you can click the icon again to reverse it.

When you are finished, click 'Save Profile Changes' in order to remove the selected papers from your profile.


# Manually adding a publication to your profile

If one or more publications are not present in your DBLP homepage, you can use our direct upload feature to manually upload your missing publications. Please go to the [OpenReview.net Archive page](https://openreview.net/group?id=OpenReview.net/Archive) and follow the instructions there.

<br>


# Finding and adding a Semantic Scholar URL to your profile

To locate your Semantic Scholar URL, go to [https://semanticscholar.org](https://semanticscholar.org/) and search for the name you publish by.

If Semantic Scholar has your data, an author tile with your name will appear under the search bar. If your name is not immediately one of the top tiles, click the "See All Results for" link to expand the tile section. Click on the author tile.

![](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FB888SOE1Lis9WKhuO35i%2FScreenshot%202026-01-13%20at%201.47.40%E2%80%AFPM.png?alt=media\&token=6a54b905-475d-4edc-ae1d-ec557b984d97)

Once you have identified your author page with the associated papers, t**he URL in the browser address bar is the Semantic Scholar URL that you can use in OpenReview profile edit page**.

If you would like to edit your Semantic Scholar author page or add additional metadata (e.g. affiliation data) you may use the "Claim Author Page" button located under your name at the top left of your Semantic Scholar author page.

<img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2Fw8dz5xMpGPsVuJ8sqFCv%2FScreenshot%202026-01-13%20at%201.50.13%E2%80%AFPM.png?alt=media&amp;token=d0025e29-11cb-4a53-8fb6-84f086d76a20" alt="" width="358">

After you have claimed your page and the claim has been approved, you will receive an email from Semantic Scholar with instructions to edit and update your author page. You will have the option to edit or add metadata, remove papers or add additional papers to your claimed Semantic Scholar author page (in case there are multiple author pages with your name).<br>


# Finding and adding your ACL Anthology URL to your profile

An option is now availible to add your ACL Anthology URL to your OpenReview profile. This is an optional field, and if you don't have a personal ACL anthology page it can be ignored.

To locate your ACL Anthology URL, go to [https://aclanthology.org](https://aclanthology.org/), type your publishing name into the Search bar at the top of the page, and then click on the magnifying glass icon to initiate the search.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FQnsBmJmwHOpU6JHGz4QX%2FScreen%20Shot%202024-03-21%20at%201.58.06%20PM.png?alt=media&amp;token=caeb1468-1dc4-4ac6-a0bb-845b1151a698" alt="" width="368"><figcaption><p>Type your name into the search bar</p></figcaption></figure>

After the search results are returned, click on the Authors tab and select your name. The personal author pages are auto generated, so it's possible a page for you was not created. If you notice an error in the content of your author page, you can create an issue or submit a pull request <https://github.com/acl-org/acl-anthology/>.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FqQiuZwo8xuxTxgsrLOll%2FScreen%20Shot%202024-03-21%20at%201.58.22%20PM.png?alt=media&amp;token=1c08c373-98e5-40e7-9846-f2217f4fa932" alt="" width="375"><figcaption><p>Search results page</p></figcaption></figure>

The author page will list known published name variations, publication list, and links to coauthors as well as venues the author submitted papers to. Your ACL Anthology URL can be found in the address bar.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FLEfwfxMGeNl7gQ1MlRWe%2FScreen%20Shot%202024-03-21%20at%202.16.17%20PM.png?alt=media&amp;token=4b57422f-fa93-4b5e-9c7e-b10547ec9284" alt=""><figcaption><p>ACL Anthology author publication page</p></figcaption></figure>

To add this link to your OpenReview profile, login and click on Edit Profile. Scroll down to the Personal Links Section and you'll see a textbox labeled "ACL Anthology URL". Copy the ACL Anthology URL and paste it in this textbox. Make sure to scroll all the way down the Edit page to submit your changes.


# Merging Profiles

### If you know you have multiple profiles (and access to them):

1. Log in using the email address associated with any one of your profiles. Preferably, the one you want to merge the other profiles into.
2. Add the email address that is already confirmed in your other profile to the profile you're currently logged into. (You can skip to step 3 if you have already added this email address.)
3. Click on "Confirm."
4. You will receive an email to confirm the profile merge. Click on this link - make sure you are logged into the account from which you requested to do the merge.

If you get an 'Merge Forbidden' message, try logging out, logging into the other OpenReview profile, then opening that link again.

### If you got an error during profile registration/edit:

Sample: \[link] is already in use by \[user]

This can happen because this information is associated with another profile.

1. Finish profile registration **without** the link that caused the error.
2. Fill out the [Feedback Form](https://openreview.net/contact) and include in the message your profile ID and the profile ID that you wish to merge with.

### Other cases

If you do not recognize a profile flagged as duplicate being yours, or do not have access to the email address of the other profile, please [contact us](https://openreview.net/contact).


# Importing publication data from ORCID to your profile

**What is ORCID?**

[ORCID](https://orcid.org/) is a global, not-for-profit organization that provides unique identifiers for those participating in research, scholarship, and innovation.

Researchers primarily interact with ORCID to obtain an ORCID iD and a record unique to them. This record is populated with publication and service information added either by the researcher directly or by trusted organizations.

### Importing ORCID records to OpenReview

{% hint style="info" %}
OpenReview will only import publication records that are "Externally Sourced", meaning they were added to the ORCID record by an institution, publisher, etc.
{% endhint %}

Importing ORCID records functions like [importing records from DBLP](https://docs.openreview.net/getting-started/creating-an-openreview-profile/importing-papers-from-dblp).

1. **Login to OpenReview**

You can click on the 'Login' button on the right of the navigation menu, or go to <https://openreview.net/login>

2. **Navigate to the profile page**

Click on your name on the right of the navigation menu and click 'Profile' from the dropdown options, or go to <https://openreview.net/profile>. Click 'Edit Profile'.

3. **Add an ORCID URL**

Locate the 'ORCID URL' text field under the 'Personal Links' section.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FT6OtzMTmoqt9o7luFgrU%2FScreenshot%202025-09-17%20at%205.11.23%E2%80%AFPM.png?alt=media&amp;token=2f3777e0-1cbe-4b10-9cdf-9638b2993a14" alt=""><figcaption></figcaption></figure>

Input your ORCID iD link, and click the **"Add ORCID Papers to Profile"** button. If your ORCID iD is not valid, the button will remain unresponsive.

If your ORCID iD was valid, publications will appear in a modal window.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FXGxg6VvQ1QzYhJ8IxVRd%2FScreenshot%202025-09-18%20at%204.26.00%E2%80%AFPM.png?alt=media&amp;token=da87a5c6-c086-4401-8e52-52a6b1672723" alt=""><figcaption></figcaption></figure>

**5. Select papers to upload**

Use the checkbox in front of each paper to select those which you would like to import to your OpenReview profile.

**6. Click the 'Add to your Profile' button**

Click the 'Add to your Profile' button at the bottom of the modal window to import the selected papers.

### Unlinking imported publications

1. Go to your profile page and click 'Edit Profile'. Navigate to the 'Personal Links' section.
2. All publications associated with your profile will be listed here but those imported from DBLP or ORCID will have a minus icon displayed after the title.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FyDignHOe4CN3hj2E5cou%2FScreenshot%202025-09-18%20at%204.44.28%E2%80%AFPM.png?alt=media&amp;token=ef79f1aa-d78a-410d-8e5d-5e2c215bb560" alt=""><figcaption></figcaption></figure>

3. You can use this minus button to remove a publication from your profile. If you mistakenly remove a publication, you can click the icon again to reverse it.
4. When you are finished, click 'Save Profile Changes' in order to remove the selected papers from your profile.


# Information for high school students

OpenReview welcomes younger researchers, aged 13 and above, to participate in scholarly peer review.

\
No one under the age of 13 may create or hold an OpenReview account. If OpenReview learns that an account holder is under 13, the account will be suspended and the associated personal data is deleted in accordance with the [Privacy Policy](https://openreview.net/legal/privacy).

\
If you are at least 13 and under 18 years of age, you must have the consent of a parent or legal guardian. Please have your parent or legal guardian complete and sign the form, then upload it as a single file (PDF) to your profile with the unique link sent to you in the temporary rejection message. Do not email the consent form to OpenReview directly. If you do not receive the link or it has expired, please submit a message through the [Contact Form](https://openreview.net/contact).

In the 'Career & Education History' section, select the **'High School Student'** position, or add any internships if applicable. If the history section only lists 'Independent Researcher' your profile will be rejected with a request to add another institution record.

A parent or guardian may withdraw consent at any time. Upon request, accounts can be closed and data will be deleted.

{% file src="/files/GiWPfI6vszHuxVyL0Me7" %}


# Setting up multi-factor authentication

Multi-factor authentication (MFA) adds a second verification step to your OpenReview login. After you enter your password, OpenReview asks for a one-time code or a passkey, so your profile stays protected even if someone learns your password.

OpenReview supports three verification methods:

* **Authenticator app** — a 6-digit code generated by an app such as Google Authenticator, Microsoft Authenticator, or 1Password.
* **Passkey** — Touch ID, Face ID, Windows Hello, or a hardware security key.
* **Email** — a 6-digit code sent to your email address.

You can enable more than one method, and you can switch between them when you sign in.

## Opening your security settings

1. Sign in to OpenReview.
2. Click your name in the top right corner and select **Password & Security**.
3. Expand the **Multi-Factor Authentication** section.

Each method is shown as a card that indicates whether it is currently enabled. If you have not set up any method yet, the page displays a notice that multi-factor authentication is disabled, and a similar reminder appears on your profile edit page.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FRsmphkrfO8ZR6r9Y12QA%2Fimage.png?alt=media&#x26;token=b7eba4de-7cc1-4acb-b20c-67e50c62e032" alt="The Multi-Factor Authentication panel with Authenticator App, Passkey and Email cards, and a notice that multi-factor authentication is currently disabled"><figcaption><p>The Multi-Factor Authentication section before any method has been set up.</p></figcaption></figure>

## Setting up an authenticator app

1. On the **Authenticator App** card, click **Set up**.
2. Scan the QR code with your authenticator app.
3. Enter the 6-digit code shown in your app and click **Verify**.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F8BhrG7Rg2jcc466JE4Kx%2Fimage.png?alt=media&#x26;token=d3640a37-0ed2-4d4c-b229-12bbbabbfba5" alt="The authenticator app setup panel showing a QR code, a field to enter the code from the authenticator app, and a Verify button"><figcaption><p>Scanning the QR code and confirming the first generated code enables the authenticator app.</p></figcaption></figure>

{% hint style="info" %}
The QR code is valid for 10 minutes. If setup expires before you finish, close the panel and click **Set up** again to generate a new code.
{% endhint %}

{% hint style="info" %}
Scripts you run yourself with the Python client simply ask you for the code. For scripts that run unattended, follow [Using multi-factor authentication in unattended scripts](https://docs.openreview.net/getting-started/using-the-api/using-multi-factor-authentication-in-unattended-scripts) instead of these steps: they need the setup secret as text, and the website only shows it as a QR code.
{% endhint %}

## Setting up a passkey

1. On the **Passkey** card, click **Set up**.
2. Enter a name that will help you recognise the device later, such as `Laptop Touch ID`. A name is required.
3. Click **Add Passkey** and complete the prompt shown by your browser or operating system.

You can register several passkeys — for example, one per device. To add or remove them later, click **Edit** on the Passkey card. Removing every passkey disables the passkey method.

## Setting up email codes

On the **Email** card, click **Enable**. The method is switched on immediately; there is no confirmation step.

{% hint style="warning" %}
When you sign in, the one-time code is sent to the email address you signed in with, which is not necessarily the preferred address on your profile. Make sure you can receive mail at the address you use to log in.
{% endhint %}

Each emailed code is valid for 10 minutes.

## Saving your recovery codes

The first time you enable any method, OpenReview generates **10 recovery codes** and displays them once. Use **Download** or **Copy to Clipboard** and store them somewhere safe — they are your last way into your account if you lose access to every other method.

Each recovery code can be used only once. The Recovery Codes card shows how many remain unused.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FYqaHiFujZv5OeIOKMEFo%2Fimage.png?alt=media&#x26;token=e552c1c5-14c1-43fb-9365-4a2ac2f42cb8" alt="Ten recovery codes listed above Download and Copy to Clipboard buttons, with a warning that the codes will not be shown again"><figcaption><p>Recovery codes are displayed once, when you enable your first method.</p></figcaption></figure>

{% hint style="danger" %}
Recovery codes are shown only once and cannot be retrieved later. Clicking **Generate New Code** issues a fresh set of 10 and invalidates every previously issued code, including unused ones.
{% endhint %}

## Choosing your preferred method

If you enable more than one method, click **Set as Preferred** on the card you want OpenReview to offer first when you sign in. You can still choose any of your other enabled methods at login.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FTo1GLQQlcLQ40CAWgwU9%2Fimage.png?alt=media&#x26;token=fe82a8ae-4864-4728-abcd-a4e1584470a3" alt="The Multi-Factor Authentication panel with Authenticator App and Email enabled, Passkey not set up, and a Recovery Codes card showing ten codes unused"><figcaption><p>Enabled methods are highlighted, and the Recovery Codes card tracks how many codes remain.</p></figcaption></figure>

## Signing in with multi-factor authentication

After you enter your username and password, OpenReview shows a verification step using your preferred method. On that screen you can:

* Enter the code from your authenticator app or email, or confirm with your passkey.
* Select **Log in using…** to switch to another method you have enabled.
* Select **Log in using Recovery Code** to use one of your recovery codes.
* Tick **Do not ask for a code on this device for the next 30 days** to skip verification on that browser in future.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FdScIdDEVfbfeGCOIUmFX%2Fimage.png?alt=media&#x26;token=035ea0b4-ecfd-4c1b-af84-5418399ce87b" alt="The Two-Factor authentication login step with a six-digit code field, a checkbox to skip codes on the device for 30 days, and links to log in using Email OTP or a recovery code"><figcaption><p>The verification step shown after your password, with alternative methods listed below.</p></figcaption></figure>

## If you cannot sign in

After 5 consecutive incorrect codes, verification is locked for 15 minutes. Wait for the lockout to expire, then try again.

If you have lost access to your authenticator app, passkey, and email, sign in with one of your recovery codes and set up a new method straight away. If you no longer have your recovery codes either, contact OpenReview through the [Feedback Form](https://openreview.net/contact).

## Turning off a method

Open **Password & Security → Multi-Factor Authentication**, then:

* **Authenticator App** — click **Disable**, then confirm with **Disable Authenticator App**.
* **Email** — click **Disable**.
* **Passkey** — click **Edit** and delete the passkeys you no longer want. Removing the last one disables the method.

Disabling your last remaining method turns multi-factor authentication off for your profile.

{% hint style="info" %}
We strongly recommend keeping at least one method enabled. Author, reviewer, and program chair accounts control submissions and reviews, and a password on its own is a single point of failure.
{% endhint %}


# Customizing Forms

A guide for program chairs on how to customize the different forms available to the reviewing committee.

## Where to customize forms

The form fields for all invitations are modified via the [Workflow Timeline](https://docs.openreview.net/conference-review-workflow/overview#id-2.-accessing-the-workflow-console). Here's an example of where to edit the fields for the submission form.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FCydGySFn9NyOnfoBJg9D%2FScreenshot%202026-06-10%20at%2010.55.55%E2%80%AFAM.png?alt=media&#x26;token=1e218b4a-cb45-4a4f-bcca-9f796d7eda2e" alt="This image is a screenshot of the Submission invitation edit interface. It is cropped to focus on the section on editing form fields." width="375"><figcaption><p>Screenshot of the Submission Invitation edit interface,<br>focused on how to edit form fields.</p></figcaption></figure>

However, at this time, `readers` can only be added to fields via the JSON content tab. Please see [this guide](#setting-the-readers-of-a-field) about adding readers to fields to limit which groups can see them.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FeyqA0NL6BebkGRRzGuWP%2FScreenshot%202026-06-10%20at%2010.56.07%E2%80%AFAM.png?alt=media&#x26;token=c70dfce3-e367-4ea3-9ada-95c940ab36fe" alt="Screenshot of the Edit form fields interface, focused on the various tabs starting with left to right: &#x22;Content JSON&#x22;, &#x22;Widgets&#x22;, and &#x22;Preview&#x22;" width="375"><figcaption><p>Screenshot of the Edit Form Fields interface,<br>and the tabs to edit and preview the fields.</p></figcaption></figure>

## Essential structure of custom fields

These text boxes accept a valid JSON object with fields and values. The following is an example where the *title* field gets replaced with a radio button, like so:

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FtgMLfYGnay2sk6Y7VdAS%2Fimage.png?alt=media&#x26;token=b38f62ce-f5fb-4df3-ad8d-b4af4627ae62" alt="A required Title field rendered as a radio button with the single option Test Submission Title, under the description Title of the paper."><figcaption><p>Preview of how this field will be rendered on OpenReview</p></figcaption></figure>

```
{
    "title": {
        "order": 1,
        "description": "Title of the paper",
        "value": {
            "param": {
                "type": "string",
                "enum": ["Test Submission Title"],
                "input": "radio"
            }
        }
    }
}
```

Note that correct indentation levels and matched brackets are necessary for valid JSON.

The primary fields of the entry are:

`title` - This will be the name of the field in the form

`order` - Determines where in the form the field will appear

`description` - Will show up in the form as instructions or description

`value` - Will have subfields (under `param`) determining the format of the field and the options for responses

```
"value": {
    "param": {
        "type": "string",
        "enum": ["Test Submission Title"],
        "input": "radio"
    }
}
```

When making a field that is asking for user input, you will ***always*** see this pattern of `"value": { "param": {...} }`. Inside the `param` object are fields determining what the user sees in the input form along with what the user is allowed to submit: these are *representation specifiers* and *validation specifiers*.

{% hint style="danger" %}
Both validation and representation specifiers can be found inside the `param` object
{% endhint %}

## Specifiers

This section will introduce common specifiers used in customizing forms. Further information about specifiers can be found [here](https://docs.openreview.net/reference/api-v2/entities/invitation/specifiers).

### Representation Specifiers

Representation specifiers determine how the user will input their response into the field (for example a textbox or a checklist). These will be defined in the `param` object.

The `input` specifier determines the rendering on the form and can have the following values (see below for examples of how different input types render):

* `text`
* `select`
* `checkbox`
* `textarea`
* `radio`

`default` is the default value of the box that will appear when the widget is initialized

`markdown` is a boolean value (true/false) that enables markdown for this text field

`scroll` is a boolean that adds a scroll bar to a `textarea` input

### Validation Specifiers

Validation specifiers are used by the back-end to ensure data submitted through the form conforms to certain requirements. In the example above, only a single string is allowed, and that string must be one of the values defined in the `enum` array. Specifically, a string that has the value "`Test Submission Title`"

`optional` is a boolean (true/false) value that indicates whether or not this field is required to be present when the form is submitted. By default all fields in the form are required, and you can add `optional : true` to indicate an optional field.

{% hint style="info" %}
Required fields have their field names prefixed with an asterisk in the preview tab and note edit interface.
{% endhint %}

`type` specifiers require the input to be of a specific type: options are `string`, `string[]` (string array) and `file`.

`string` fields can be further validated by using fields to describe the structure of a valid string input. Some of these field are:

* `"maxLength":` set the maximum number of characters of the input
* `"minLength":` set the minimum number of characters of the input
* `"regex":` use regular expressions to define acceptable string structures
* `"enum":` restrict the user to a predefined set of strings
* `"items":` an array of strings as indicated by its type (only used with `type: string[]` ) \*\*

\*\* All values in `"items"` will be considered required unless specified otherwise with `"optional": true`. Please see [multiple choices](#multiple-choices) for an example.

`file` fields are specifically validated with `maxSize` and `extensions`. `maxSize` is an integer that specifies the size of the largest file that can be uploaded on the form in megabytes. `extensions` is a list of strings that are extensions, for example `"extensions": ["pdf", "zip"]`.

{% hint style="warning" %}
Extensions that have a "." in them are **not** supported. The following field would be invalid because "`tar.gz`" is not supported:

```
"extensions": ["zip", "tar.gz"]
```

{% endhint %}

## Examples of Common Form Patterns

### Text Input

{% tabs %}
{% tab title="Small Text Area" %}

```
{
  "title": {
    "order": 1,
    "description": "(Optional) Brief summary of your comment.",
    "value": {
      "param": {
        "type": "string",
        "maxLength": 500,
        "optional": true
      }
    }
  }
}
```

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F5Ydl6krLLqyIJtFb81cO%2FcommentTitle.png?alt=media&#x26;token=45ce94f3-f00a-4ef6-bdbe-cfa32169a702" alt=""><figcaption></figcaption></figure>
{% endtab %}

{% tab title="Large Text Area" %}

```
{
  "comment": {
    "order": 2,
    "description": "Your comment or reply (max 5000 characters). Add formatting using Markdown and formulas using LaTeX. For more information see https://openreview.net/faq",
    "value": {
      "param": {
        "type": "string",
        "maxLength": 5000,
        "markdown": true,
        "input": "textarea"
      }
    }
  }
}
```

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FDJmRQ7jDtAUHkPVL5wXz%2Fcommentcomment.png?alt=media&#x26;token=13688e64-2ce8-49d6-9654-744fd191bc6f" alt=""><figcaption></figcaption></figure>
{% endtab %}
{% endtabs %}

### Single Choice

{% tabs %}
{% tab title="Radio Button" %}

```
{
  "recommendation": {
    "order": 10,
    "description": "Please provide your recommendation based on the manuscript, reviews, author responses and your discussion with the reviewers.",
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "Reject",
          "Accept",
          "Nominate for best paper"
        ],
        "input": "radio"
      }
    }
  }
}
```

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F72wbZpO6KSjaKfFlt1jO%2Fmetareviewupdate.png?alt=media&#x26;token=4711afc8-e716-44f8-b6ac-703a73dc7eae" alt=""><figcaption></figcaption></figure>
{% endtab %}

{% tab title="Dropdown" %}

```
{
  "submission_track": {
    "order": 11,
    "description": "Please select a preferred track, which will be considered in the paper-reviewer assignment. Please note that your choice of a track will be taken into account, but the chairs may decide to move your paper to a different area when appropriate.",
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "Commonsense Reasoning",
          "Computational Social Science and Cultural Analytics",
          "Dialogue and Interactive Systems",
          "Discourse and Pragmatics",
          "Efficient Methods for NLP",
          "Ethics in NLP",
          "Human-Centered NLP",
          "Information Extraction",
          "Information Retrieval and Text Mining",
          "Interpretability, Interactivity, and Analysis of Models for NLP",
          "Language Grounding to Vision, Robotics and Beyond",
          "Language Modeling and Analysis of Language Models",
          "Linguistic Theories, Cognitive Modeling, and Psycholinguistics",
          "Machine Learning for NLP",
          "Machine Translation",
          "Multilinguality and Linguistic Diversity",
          "Natural Language Generation",
          "NLP Applications",
          "Phonology, Morphology, and Word Segmentation",
          "Question Answering",
          "Resources and Evaluation",
          "Semantics: Lexical",
          "Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.",
          "Sentiment Analysis, Stylistic Analysis, and Argument Mining",
          "Speech and Multimodality",
          "Summarization",
          "Syntax, Parsing and their Applications",
          "Theme Track: Large Language Models and the Future of NLP"
        ],
        "input": "select"
      }
    }
  }
}
```

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FVjwSyouj3iCsiyFZT0nI%2Fdropdownselect.png?alt=media&#x26;token=8352cf5f-be1a-447f-8846-e4eb0aa24606" alt=""><figcaption></figcaption></figure>
{% endtab %}
{% endtabs %}

### Multiple Choices

{% tabs %}
{% tab title="Checkboxes" %}

```
{
  "confirmation_of_submission_requirements": {
    "order": 17,
    "description": "Before you submit this paper, please make sure that the following requirements are met. If any of these requirements are not fulfilled, your submission will be rejected and will not be reviewed.",
    "value": {
      "param": {
        "type": "string[]",
        "optional": true,
        "items": [
          { "value": "Requirement 1", "description": "Requirement 1", "optional": true},
          { "value": "Requirement 2", "description": "Requirement 1", "optional": true}
        ],
        "input": "checkbox"
      }
    }
  }
}
```

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FEbznFBDevTgTF7RTnIB3%2Frequirements.png?alt=media&#x26;token=c3b91a90-baf6-4824-8789-e1cd8a61a68d" alt=""><figcaption></figcaption></figure>
{% endtab %}

{% tab title="Dropdown Values" %}

<pre><code>{
  "submission_tracks": {
    "order": 11,
    "description": "Please select your preferred tracks, which will be considered in the paper-reviewer assignment. Please note that your choice of tracks will be taken into account, but the chairs may decide to move your paper to a different area when appropriate.",
    "value": {
      "param": {
        "type": "string[]",
        "items": [
            { "value": "Commonsense Reasoning", "description": "Commonsense Reasoning", "optional": true },
            { "value": "Computational Social Science and Cultural Analytics", "description": "Computational Social Science and Cultural Analytics", "optional": true },
            { "value": "Dialogue and Interactive Systems", "description": "Dialogue and Interactive Systems", "optional": true },
            { "value": "Discourse and Pragmatics", "description": "Discourse and Pragmatics", "optional": true },
            { "value": "Efficient Methods for NLP", "description": "Efficient Methods for NLP", "optional": true },
            { "value": "Ethics in NLP", "description": "Ethics in NLP", "optional": true },
            { "value": "Human-Centered NLP", "description": "Human-Centered NLP", "optional": true },
<strong>            { "value": "Information Extraction", "description": "Information Extraction", "optional": true },
</strong>            { "value": "Information Retrieval and Text Mining", "description": "Information Retrieval and Text Mining", "optional": true },
            { "value": "Interpretability, Interactivity, and Analysis of Models for NLP", "description": "Interpretability, Interactivity, and Analysis of Models for NLP", "optional": true },
            { "value": "Language Grounding to Vision, Robotics and Beyond", "description": "Language Grounding to Vision, Robotics and Beyond", "optional": true },
            { "value": "Language Modeling and Analysis of Language Models", "description": "Language Modeling and Analysis of Language Models", "optional": true },
            { "value": "Linguistic Theories, Cognitive Modeling, and Psycholinguistics", "description": "Linguistic Theories, Cognitive Modeling, and Psycholinguistics", "optional": true },
            { "value": "Machine Learning for NLP", "description": "Machine Learning for NLP", "optional": true },
            { "value": "Machine Translation", "description": "Machine Translation", "optional": true },
            { "value": "Multilinguality and Linguistic Diversity", "description": "Multilinguality and Linguistic Diversity", "optional": true },
            { "value": "Natural Language Generation", "description": "Natural Language Generation", "optional": true },
<strong>            { "value": "NLP Applications", "description": "NLP Applications", "optional": true },
</strong><strong>            { "value": "Phonology, Morphology, and Word Segmentation", "description": "Phonology, Morphology, and Word Segmentation", "optional": true },
</strong>            { "value": "Question Answering", "description": "Question Answering", "optional": true },
            { "value": "Resources and Evaluation", "description": "Resources and Evaluation", "optional": true },
            { "value": "Semantics: Lexical", "description": "Semantics: Lexical", "optional": true },
<strong>            { "value": "Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.", "description": "Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.", "optional": true },
</strong>            { "value": "Sentiment Analysis, Stylistic Analysis, and Argument Mining", "description": "Sentiment Analysis, Stylistic Analysis, and Argument Mining", "optional": true },
            { "value": "Speech and Multimodality", "description": "Speech and Multimodality", "optional": true },
            { "value": "Summarization", "description": "Summarization", "optional": true },
            { "value": "Syntax, Parsing and their Applications", "description": "Syntax, Parsing and their Applications", "optional": true },
            { "value": "Theme Track: Large Language Models and the Future of NLP", "description": "Theme Track: Large Language Models and the Future of NLP", "optional": true }
        ],
        "input": "select"
      }
    }
  }
}
</code></pre>

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2Fdy0pDDQb79UJ1KAlLLtG%2Fdropdownvalues.png?alt=media&#x26;token=e835b7cd-3a3d-45db-8b58-76b64d829c6d" alt=""><figcaption></figcaption></figure>
{% endtab %}
{% endtabs %}

### Miscellaneous

{% tabs %}
{% tab title="Files" %}

```
{
  "supplementary_material": {
    "order": 10,
    "description": "All supplementary material must be self-contained and zipped into a single file. Note that supplementary material will be visible to reviewers and the public throughout and after the review period, and ensure all material is anonymized. The maximum file size is 200MB.",
    "value": {
      "param": {
        "type": "file",
        "extensions": [
          "zip",
          "pdf",
          "tgz",
          "gz"
        ],
        "maxSize": 200,
        "optional": true
      }
    }
  }
}
```

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FB2STzNHT0nZLvE1pGKYC%2Fsubmissionsupplementary.png?alt=media&#x26;token=3118eb6a-9848-43ad-a90b-ac62a49b17ca" alt=""><figcaption></figcaption></figure>
{% endtab %}
{% endtabs %}

### Setting the Readers of a Field

If you want to limit who in the committee can see a particular field in a form, this is done by adding a `readers` field. Please follow this link for more detailed information on [hiding or revealing fields](https://docs.openreview.net/how-to-guides/submissions-comments-reviews-and-decisions/how-to-hide-reveal-fields). Below are two examples, one for the [submission form](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/customizing-your-submission-form) and one for the meta review form. Notice the different use of dollar sign notation. The notation used for the meta review form will also work for other replies to the forum: reviews, comments, and decisions. Please scroll to see how best to setup `readers` depending on which venue configuration was deployed.

{% tabs %}
{% tab title="Submission" %}

```
{
  "supplementary_material": {
    "order": 10,
    "description": "All supplementary material must be self-contained and zipped into a single file. Note that supplementary material will be visible to reviewers and the public throughout and after the review period, and ensure all material is anonymized. The maximum file size is 200MB.",
    "value": {
      "param": {
        "type": "file",
        "extensions": [
          "zip",
          "pdf",
          "tgz",
          "gz"
        ],
        "maxSize": 200,
        "optional": true
      }
    },
    "readers": [
    "Your/Venue/ID/Program_Chairs",
    "Your/Venue/ID/Submission${4/number}/Senior_Area_Chairs",
    "Your/Venue/ID/Submission${4/number}/Authors"
    ]
  }
}
```

{% endtab %}

{% tab title="Meta Review" %}

```
{
  "recommendation": {
    "order": 10,
    "description": "Please provide your recommendation based on the manuscript, reviews, author responses and your discussion with the reviewers.",
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "Accept (Oral)",
          "Accept (Poster)",
          "Reject"
        ],
        "input": "radio"
      }
    },
    "readers": [
    "Your/Venue/ID/Program_Chairs",
    "Your/Venue/ID/Submission${7/content/noteNumber/value}/Area_Chairs",
    "Your/Venue/ID/Submission${7/content/noteNumber/value}/Reviewers"
  ]
  }
}
```

{% endtab %}
{% endtabs %}


# Using the New Forum Page

Learn about the differences and new features of the  forum page.

OpenReview is releasing a major update of the forum page for venues using the new API (v2). While the interface will look familiar, it should be faster, more flexible, and offer new ways to quickly find the content you are looking for.

#### Anatomy of a Forum Reply

Here is an example of a typical forum comment:

![An annotated example of a forum comment](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2Fs6HSGhx1FVOcYOyvnE9O%2FScreen%20Shot%202022-07-06%20at%208.33.36%20PM.png?alt=media\&token=c1e5b009-0465-47bf-8ee1-2b77e0d05e3d)

Every post on the new forum page contains the following information:

1. Title: title may be provided by the author of the post or it may be a generic title. Clicking the link icon next to the title copies a direct link to this post to your clipboard.
2. Reply Type: represents the type of forum reply, and comes from the name of the invitation that was used to create the post.
3. Signature: shows the identity of the user who posted the reply. If the reply comes from a group and you have permission to see the members of the group, their identities will be shown in parenthesis next to the signature.
4. Creation Date: shows when the reply was posted and when it was last modified
5. Readers: shows who this post is visible to
6. Revisions Link: see a list of all edits made (for more see "Editing Posted Content" below)
7. Edit & Delete: allows you to modify or hide the reply (for more see "Editing Posted Content" below)
8. Content: shows the complete content of the forum reply
9. Collapse Toggles: allows you to show more or less of the content of the note. The top button will collapse everything down to a single line displaying just basic information, the middle button will only show the first 5-10 lines of content, and the bottom button will display the entire contents of the note.
10. Reply Buttons: show all the available options for replying to this post. Clicking one of the buttons will open a form that allows you to submit your reply.

#### Sorting and Filtering Replies

The new forum page provides advanced controls for sorting and filtering replies such as comments, reviews, private responses, and PC decisions. The new controls look something like this:

![An annotated screenshot of the filter controls on a new forum page](https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F9qo7b5hxhEjybR6gP4fH%2FScreen%20Shot%202022-06-30%20at%204.24.36%20PM.png?alt=media\&token=c7a02d3e-70df-4129-a4c1-4b2ea5cfc1fa)

1. Invitation Filter: show only replies of a certain type. Can select multiple invitations (types) to show replies matching any of those invitations.
2. Author Filter: show only replies signed by the selected user or group. Can select multiple authors to show all replies that include any one of the selections as a signature.
3. Keyword Filter: show all replies that match the phrase. Matches can come from the content of the reply, the title, or the invitation.
4. Sort Control: change the order of replies shown to either most recent first or oldest first
5. Layout Control: change the level of nesting of the replies. The three options are Linear, Threaded, and Nested
6. Collapse Control: change how much of all the replies is shown. By default the entire contents of the reply is visible, this corresponds to the three lines. Selecting the middle button (two lines) will only show the first 5-10 lines of content, and selecting the left button (one line) will condense the replies down to a single line displaying just basic information.
7. Link Button: copy a URL to your clipboard that includes all the currently selected filter and sort options. This is useful for sharing specific views of a forum page with other people or bookmarking for later.
8. Readers Filter: show all the replies that have the selected users or groups listed as a reader. Clicking a button twice will turn the button red – this means that only the replies that DO NOT include that group as a reader will be shown.
9. Preset Views: venues can configure sets of filters and layout options that are displayed as tabs above the filter form. Clicking on a given tab will switch the current filters over to those settings.

#### Forum Layout Options

As mentioned in the section above there are currently three layout modes available for forum pages. These are:

* Linear: all replies are shown at the same level of nesting (no indentation). This is useful to quickly see a chronological feed of all forum replies. If a given post is a reply to another reply (not a general reply to the submission note) the title of that reply will be shown in gray above the title. Clicking on this gray title will scroll the page to that reply.
* Threaded: one level of nesting is shown. This layout is useful to group conversations into overarching topics.
* Nested: two levels of nesting are shown. This is useful for breaking larger conversations down into sub threads.

#### Editing Posted Content

If you are logged into OpenReview and have permission to modify the content of a submission or a forum reply (aka a [Note](https://github.com/openreview/openreview/blob/main/docs/getting-started/broken-reference/README.md)) you will see a dropdown button labeled Edit to the right of the title. Clicking this button will display a list of all the available ways to modify the note (aka edit Invitations). For a submission note this might include options to revise the submission or withdraw the submission, and for a reply it might include the option to edit the content of the reply.

You can see a list of all the edits of a given note by clicking on the Revisions link below the title.

#### Providing Feedback

We hope that you find the new functionality useful. If there is anything you would like to see changed or added please fill out the [Feedback Form](https://openreview.net/contact) with the subject "New Forum Page Feedback".


# Live Chat on the Forum Page

Live chat is a new OpenReview feature available to all venues to support synchronous dialog among reviewers and ACs. Similar to a Slack channel or chat room, messages will be visible to all participants and appear in real time as they are posted. In addition, there is an option to enable browser notifications so participants can be alerted when new messages are posted.

This feature was developed with encouragement and funding from the Alfred P. Sloan Foundation. For more information, see the Motivations section below.

#### Enabling Live Chat

Live chat can't be turned on from the Workflow Timeline. To enable or disable it, contact OpenReview Support by commenting on your [venue request form](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#venue-request-form).

Once enabled, the live chat interface will be visible on all forum pages under the “Committee Members Chat” tab. Chat messages are hidden in the “Discussion” tab to prevent confusion and keep the discussions with paper authors and other users separate.

#### Chat Features

* Real-time: Type a simple message, and it appears immediately to other reviewers who have the same OpenReview forum page open in their browser, enabling interactive discussion.
* Private: By default, messages are visible all program chairs, area chairs, and reviewers of a paper. The messages will never be visible to the authors or the public. As an extra reminder, the chat participants are below the text input box. The ability to send private messages to a subset of the chat participants is planned for a future release.
* Rich text: Chat messages can use Markdown for formatting and LaTeX for math notation. When composing a message you can preview how it will look by clicking the Preview button. To copy the unformatted text of a posted chat message, hover over the message and click the View Raw button.
* Replies: Users can reply to specific message from earlier in the chat by hovering over the message and clicking the Reply button.
* Permalinks: Users can copy a URL pointing to a specific chat message by hovering over the message and clicking the Copy Link button.
* Notifications: Browser and email notifications are supported. User should enable browser notifications from their browser. Email notifications will be sent in bulk either every 5 messages or every 4 hours.
* Reactions: Users can add emoji "reactions" to chat messages, in the same way they can react to messages on apps like Slack and GitHub. To add a reaction, hover over the message and click the "Add Reaction" button, then select the emoji you want to use. All the reactions a message has gotten will be shown below the message content as separate buttons. Clicking one of these buttons will either add a reaction of that type, or remove your reaction if you have already added one.

#### Motivation

In the past, many peer-reviewed yearly conferences in computer science (such as NeurIPS and CVPR) held an in-person meeting among the 20-30 “area chairs” of the conference to discuss and debate each paper. These meetings enabled extensive synchronous communication and serendipitous discussion during the final decision-making phase of the peer review process.

Unfortunately, these practices of in-person area-chair meetings are increasingly rare due to growing conference size, travel expenses, and (more recently) risks from the pandemic. Given the current universal use of web tools for reviewing, one would hope that the scientific communities would take advantage of internet technology to greatly increase interactivity during the peer review process, but unfortunately, reviewing workflows remain largely unchanged.

One exception is the introduction of “rebuttal statements” (an opportunity for authors to write a response to first drafts of the reviews), and asynchronous messages among the reviewing team. However, due to the lack of synchronous communication, discussion of these rebuttals (and scientific debate among the decision-making team) is hampered – often lackluster or incomplete.

Synchronous communication creates beneficial social pressure for engagement and the opportunity for interactive clarification without the extensive “context switching” inherent in asynchronously juggling many tasks. One highly experienced computer science reviewer and OpenReview user told us, “The burden of context switching after 24 hours away from a conversation can sometimes be the bottleneck in writing a response; re-remembering the details of a paper often takes even more time than writing a reply to a co-reviewer. Synchronous communication could dramatically reduce the burden and increase quality of reviewing by removing this context-switching bottleneck.”

While reviewers may naturally find each other online at the same time, and benefit from the chat feature, scheduled chat discussion sessions may be even more helpful. Scheduling synchronous sessions will be one key aspect of recapturing the benefits of in-person synchronous discussion. The details and cultural considerations of this scheduling will be driven by the program chairs. The in-person meetings were a burden, but also provided a scientific/social occasion that was enjoyed and cherished; we anticipate that with some sensitive design, program chairs can provide a similar occasion. The burden can be reduced by culling from the process submissions that are clear rejections.

Why text messaging rather than audio/video? Although speaking is easy, typing has multiple advantages: typed messages can be more easily archived, read later, skimmed quickly, analyzed, and studied. Many people are used to interactive synchronous text chat tools, such as Slack, and find them very productive. Furthermore, text has high accessibility, and is recommended by the Web Content Accessibility Guidelines (WCAG 2.0). With text entry, users can naturally make some statements publicly and mark others as private; scientists are already used to this ability in typed chat systems such as Slack or Zoom. Typed chat messages support not only fully-synchronous interaction, but also interactive semi-synchronous interaction (e.g. with two minute interruption or delay), which fits modern work styles. If some users are slow typists, they can use voice recognition on their devices to type messages at speaking speed.\ <br>


# Modifying Venue Homepages


# How to customize your venue homepage

You can customize your venue homepage using Markdown. All Markdown should be validated to ensure that it will not break the layout of the page: <https://stackedit.io/>

Many properties can be modified using the Group Content tab of your venue's workflow page: [https://openreview.net/group/edit?id=Your/Venue/ID#groupContent](https://openreview.net/group/edit?id=HCAL/2026/Call_for_Chapters#groupContent). Here you can use the `Edit` button to update properties of the venue including the Venue URL, location, and start date, among others.

To add additional custom text to your venue, go to the UI Code tab of the same page [https://openreview.net/group/edit?id=Your/Venue/ID#groupUICode](https://openreview.net/group/edit?id=HCAL/2026/Call_for_Chapters#groupUICode) and edit the instruction variables to include a custom markdown string in the format `"instructions": "Your Markdown String"` .


# Managing Groups


# How to Recruit and Remind Recruited Reviewers

Every committee group of your venue (Reviewers, and Area Chairs and Senior Area Chairs if your venue uses them) has its own recruitment tools. You send invitations and reminders from the **Workflow Groups** section at the top of your [Workflow Timeline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#workflow-timeline). The steps below use the Reviewers group; the steps for Area Chairs and Senior Area Chairs are the same.

{% hint style="info" %}
Recruitment can be done at any point. The recruitment request is available as soon as your venue is deployed.
{% endhint %}

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FYA2cgxqkMqGlE7WAdiTq%2Fimage.png?alt=media&#x26;token=745aae79-efbb-44f6-9582-bc54f611b4b8" alt="The Reviewers group in the Workflow Groups section, with the Recruitment Request and Recruitment Request Reminder buttons."><figcaption><p>The Reviewers group in the Workflow Groups section, as Program Chairs see it.</p></figcaption></figure>

## Send recruitment invitations

1. Open your Workflow Timeline and go to the **Workflow Groups** section.
2. In the row for the Reviewers group, click the **Recruitment Request** button.
3. Fill out the form:
   * **Invitee Details**: enter the invitees, one per line. Invitees must be formatted in a specific way, or they will be reported as errors and will not receive the email:

     > Enter a list of invitees with one per line. Either tilde IDs (\~Captain\_America1), emails (<captain_rogers@marvel.com>), or email,name pairs (<captain_rogers@marvel.com>, Captain America) expected. If only an email address is provided for an invitee, the recruitment email is addressed to "Dear invitee". Do not use parentheses in your list of invitees.
   * **Invite Message Subject Template** and **Invite Message Body Template**: the recruitment email. You can use `{{fullname}}` (the name of the invitee) and `{{invitation_url}}` (the link to accept or decline). The body must contain `{{invitation_url}}`, otherwise the request is rejected.
4. Click **Submit**.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FgIoy50DpYtZ15AXxFcGY%2Fimage.png?alt=media&#x26;token=7655cf27-48b4-4198-90e3-abd3ff19e9ce" alt="The Recruitment Request form for the Reviewers group, with the Invitee Details, Invite Message Subject Template and Invite Message Body Template fields."><figcaption><p>The Recruitment Request form, as Program Chairs see it.</p></figcaption></figure>

When the request has been processed, the Program Chairs receive an email titled "Recruitment request status for \<venue> Reviewers Group", and the same status is posted as a comment on your venue request form. It lists how many users were invited, how many were already invited, how many were already members, and any errors, with links to the invited list.

{% hint style="info" %}
You can also send recruitment invitations from the Reviewers group page: open the group, go to the **Content** tab, click **Edit** and select **Recruitment Request**.
{% endhint %}

## Where invitees end up

* Every invited user is added to the group venue\_id/Reviewers/Invited.
* Users who accept are added to venue\_id/Reviewers.
* Users who decline are added to venue\_id/Reviewers/Declined.

Invitees can change their answer later with the link in the invitation email. Accepting removes them from the Declined group, and declining removes them from the Reviewers group. A reviewer who has already been assigned to a submission cannot decline and is asked to contact the Program Chairs or the submission's Area Chair. The Reviewers group and its Invited and Declined subgroups are listed in the **Workflow Groups** section, with the number of members in each.

## Remind invitees who have not responded

**Automatic reminder:** seven days after each recruitment request, OpenReview sends a reminder to the users from that request who have neither accepted nor declined. The reminder reuses your invitation email, with "\[Reminder]" added to the subject.

**Manual reminder:** to send a reminder at any other time:

1. In the **Workflow Groups** section, click the **Recruitment Request Reminder** button in the row for the Reviewers group.
2. Edit the **Invite Reminder Message Subject Template** and **Invite Reminder Message Body Template**. The body must contain `{{invitation_url}}`.
3. Click **Submit**.

The reminder is sent to every member of venue\_id/Reviewers/Invited who is not in venue\_id/Reviewers or venue\_id/Reviewers/Declined.

## Configure the recruitment steps

Recruitment also has two steps in the Workflow Timeline, which you can change with the **Edit** link under each step:

* **Create Reviewers Recruitment Request**
  * **Dates**: when Program Chairs can send recruitment invitations.
  * **Request Emails**: the default subject and body of the recruitment email.
* **Reviewers Recruitment Response**
  * **Dates**: when invitees can accept or decline. By default, invitees can respond until 12 weeks after your venue was deployed.
  * **Reduced Load**: the reduced load options invitees can choose from, and whether they can pick one when they accept. By default, reduced loads are not offered.
  * **Response Emails**: the emails sent when an invitee accepts or declines.
  * **Overlap Committees**: committee groups whose members cannot accept this invitation. If your venue has Area Chairs, by default a user cannot serve as both a Reviewer and an Area Chair: a user who has already accepted the Area Chair invitation is asked to decline it before accepting the Reviewer invitation. Delete the value to allow users to serve in both roles.


# How to have multiple Reviewer or Area Chair groups

Although OpenReview can support multiple groups for each role, it cannot support distinct workflows, different deadlines, or different review forms for multiple groups within the same venue.

1. [Contact OpenReview support](https://openreview.net/contact) to request the ability to support multiple groups for a particular role.
2. After we have customized your venue to support multiple groups, you can recruit members directly into each group with the 'Recruitment' button.
3. When it is time to assign the group members to submissions, you will need to run separate matchings for each group. When you deploy your assignments, the members of each group will be moved into the individual paper groups. For example, if a reviewer from Group A and Group B are each assigned to paper 1, they would both be moved into the Paper1/Reviewers group.
4. From this point forward the reviewers will be treated identically, regardless of their original group.


# How to Add and Remove Members from a Group

The program committee is represented as a group, like Reviewers, Area Chairs, Action Editors, etc. Each of these groups have a property 'members' that can contain a set of groups. Email addresses and profile ids are also represented as groups.

### **Adding Members to the Group**

If you want to build you own group of reviewers, you can simply add them as members to the group.

**Using the Group Editor**

1. Access the group editor at the following URL, replacing `group_id` with the actual ID of your group: `https://openreview.net/group/edit?id=group_id`
2. Under the Group Members section select the text box area, add their profile ID or email, and click on Add to Group

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F5ELcFsHISvH8O8hffC3t%2Fimage.png?alt=media&amp;token=550d8611-283e-4330-b6c7-7274108f40a1" alt=""><figcaption><p>Group UI</p></figcaption></figure>

**Using the Python Library**

To remove members using the python client, utilize the `add_members_to_group` function:\
\
`client.add_members_to_group(group, ['~Jane_Doe1', '~Melisa_Bok1'])`

Replace `group` with your target group's ID, and include the profile ids or emails of the members you wish to remove in the list.

### Removing Members from the Group

Should you need to remove members from a group, you can also use the group editor or the python library.

**Using the Group Editor**

1. Access the group editor at the following URL, replacing `group_id` with the actual ID of your group: `https://openreview.net/group/edit?id=group_id`
2. In the Group Members section, locate the profiles/emails of the members you wish to remove.
3. Select the profile ID or email and click 'Remove' or 'Remove Selected' to delete them from the group.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FKw8kkMuo1WHlZo6BtCfX%2FScreen%20Shot%202024-02-05%20at%204.35.17%20PM.png?alt=media&amp;token=a9068469-79ca-49cc-a6db-37ee867dc2f0" alt=""><figcaption><p>Group UI</p></figcaption></figure>

**Using the Python Library**

To programmatically remove members using the python library, utilize the `remove_members_from_group` function:

```python
client.remove_members_from_group(group, ['~Jane_Doe1', '~Melisa_Bok1'])
```

Replace `group` with your target group's ID, and include the profile ids or emails of the members you wish to remove in the list.


# Publication Chairs

The Conference Review Workflow doesn't create a Publication Chairs group, and you can't add one from the venue request form or the Workflow Timeline. To add one, contact OpenReview Support by commenting on your [venue request form](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#venue-request-form).

When a venue has a Publication Chairs group, it appears in the **Workflow Groups** section of the [Workflow Timeline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#workflow-timeline). Publication Chairs are not one of the reader options of the release steps, so they only see accepted papers once those are released to **Public**, and author names only if the authors are revealed.


# How to Copy Members from One Group to Another

Some venues with multiple deadlines a year may want to reuse the same reviewer and area chairs from cycle to cycle. In those cases, it is useful to use the python client to copy group members from one group to another rather than recruiting the same people each time. This can be done programmatically or from the UI.

#### Programmatically

1. If you have not done so, you will need to [install and instantiate the openreview-py client](https://docs.openreview.net/getting-started/using-the-api/installing-and-instantiating-the-python-client).
2. Get the group you are taking members from. You can get an individual group by its id. If you are copying reviewers from one venue iteration to another, for example, do the following:

```
first_reviewer_group = client.get_group("<first_venue_id>/Reviewers")
```

3\. Each group has a field 'members' which is a list of profile IDs or emails belonging to the members of that group. Extract the members:

```
reviewers = first_reviewer_group.members
```

4\. Finally, you can use add\_member&#x73;*\_*&#x74;o\_group to add those members to your new Reviewers group.

```
client.add_members_to_group("<second_venue_id>/Reviewers", reviewers)
```

#### From the UI

1. Navigate to <https://openreview.net/group/edit?id=\\>\<first\_venue\_id>/Reviewers
2. Under Group Members section click on Select All

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FwDEZRPMLauihbO92Dsb4%2Fimage.png?alt=media&amp;token=cdd90aca-d3f8-4737-983d-55211885cfa8" alt=""><figcaption><p>Select All</p></figcaption></figure>

3. Then click on Copy Selected

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FRCdEexesea9FuIpqLHec%2Fimage.png?alt=media&amp;token=d2e8aa2f-b82a-4f3c-9f90-d07c25713399" alt=""><figcaption><p>Copy Selected</p></figcaption></figure>

4. Navigate to <https://openreview.net/group/edit?id=\\>\<second\_venue\_id>/Reviewers
5. Under the Group Members section select the text box area and click on Add to Group

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F5ELcFsHISvH8O8hffC3t%2Fimage.png?alt=media&amp;token=550d8611-283e-4330-b6c7-7274108f40a1" alt=""><figcaption></figcaption></figure>


# Workflow


# How to create, change, and delete notes

Jump to:

* [Quickstart: posting a test submission](#id-1.-posting-creating-a-note)
* [Quickstart: automatically posting desk rejections](#quickstart-posting-a-desk-rejection-for-submissions-missing-pdfs)
* [Quickstart: changing readers](#quickstart-update-the-readers-of-a-note)
* [Quickstart: posting and deleting test reviews](#quickstart-posting-editing-and-deleting-a-test-review)
* [Quickstart: deleting fields](#id-3.-deleting-notes-and-fields)

## 1. Posting (Creating) a Note

To **post a new Note**, create a `Note` object with required fields, then use `client_v2.post_note_edit()`.

#### Required fields include:

* `invitation`: the invitation ID defining the note type (e.g., a submission, review, comment)
* `forum`: the ID of the main submission this note belongs to (for replies)
* `signatures`: who is posting (usually a profile ID)
* `readers`: who can read this note
* `writers`: who can modify the note
* `content`: a dictionary of the note content fields (title, authors, review text, etc.)

### Quickstart: Posting a submission with Python

Posting a submission with Python is typically reserved for testing venue workflows and changes.

{% hint style="info" %}
PCs and Reviewers that are also authors of test submissions will be conflicted later on in the process. We recommend using test author profiles for the submission process.
{% endhint %}

```python
#create the submission note
venue_id = test_venue_id
author_ids_or_emails = [ "~Author_One1",'email@mail.com']
your_id = "~PC_ID1"
i = 1
test_pdf_path = './data/paper.pdf'
note = openreview.api.Note(
        license = 'CC BY-SA 4.0',
        content = {
            'title': { 'value': 'Paper title ' + str(i) },
            'abstract': { 'value': 'This is an abstract ' + str(i) },
            'authorids': { 'value': author_ids_or_emails },
            'authors': { 'value': author_ids_or_emails},
            'keywords': { 'value': ['machine learning', 'nlp'] },
            'pdf': {'value': '/pdf/' + 'p' * 40 +'.pdf' },
        }
    )
#url = client.put_attachment(test_pdf_path, f'{venue_id}/-/Submission', 'pdf')
#note.content['pdf'] = url

#post the note
note = openreview_client.post_note_edit(
    invitation=f"{venue_id}/-/Submission",
    signatures=[your_id],
    readers=[venue_id] + author_ids_or_emails,
    writers = [venue_id] + author_ids_or_emails,
    note=note
)
```

### Quickstart: Posting a desk rejection for submissions missing PDFs

PCs may want to programmatically desk-reject submissions that are missing PDF files once the submission deadline has passed. You can use the script below to do so.

```python
venue_id = "<VENUE_ID>"
​
submissions = client.get_all_notes(invitation=f'{venue_id}/-/Submission')
#gets the desk rejection name of the invitation
desk_rejection_name = client.get_group(venue_id).content['desk_rejection_name']['value']
    
# for each submission note that does not contain a pdf field, post a desk rejection note
for submission in submissions:
    #Check for a pdf field value
    if not submission.content.get('pdf', {}).get('value'): 
        desk_reject_note = client.post_note_edit(
                    #desk rejection invitation
                    invitation=f'{venue_id}/Submission{submission.number}/-/{desk_rejection_name}',
                    signatures=[f'{venue_id}/Program_Chairs'],
                    note=openreview.api.Note(
                        content = {
                        'desk_reject_comments': { 'value': 'No PDF.' }
                        }
                    )
                )
        print(submission.number, "is desk rejected")
```

### Quickstart: Posting a Support Request Form With Python

While a support request form can most easily be [submitted through the UI](https://openreview.net/group?id=OpenReview.net/Support), some venues that have multiple deadlines a year and need to submit multiple venue requests with the same settings may find it easier to do this programmatically through the API. While most notes are posted with the API2 client, this note must be posted with the [API 1 client](https://docs.openreview.net/getting-started/using-the-api/installing-and-instantiating-the-python-client).

```python
#change the settings below
program_chair_emails = ['pc_email1@mail.com','pc_email2@mail.com']
content = {
        "title": "My test venue",
        "Official Venue Name": "My test venue", 
        "Abbreviated Venue Name": "MTV", 
        "Official Website URL": "mtv.com", 
        "program_chair_emails": program_chair_emails, 
        "contact_email": "pc@venue.org", 
        "Area Chairs (Metareviewers)": "Yes, our venue has Area Chairs",
        "Paper Matching": [
            "Reviewer Bid Scores", 
            "OpenReview Affinity"
        ],
        "Venue Start Date": "2022/05/01",
        "Author and Reviewer Anonymity": "Double-blind",
        "reviewer_identity": [
            "Assigned Area Chair"
        ],
        "submission_readers": "All program committee (all reviewers, all area chairs, all senior area chairs if applicable)",
        "venue_organizer_agreement":['OpenReview natively supports a wide variety of reviewing workflow configurations. However, if we want significant reviewing process customizations or experiments, we will detail these requests to the OpenReview staff at least three months in advance.',
  'We will ask authors and reviewers to create an OpenReview Profile well in advance of the paper submission deadlines.',
  'When assembling our group of reviewers and meta-reviewers, we will only include email addresses or OpenReview Profile IDs of people we know to have authored publications relevant to our venue.  (We will not solicit new reviewers using an open web form, because unfortunately some malicious actors sometimes try to create "fake ids" aiming to be assigned to review their own paper submissions.)',
  "We acknowledge that, if our venue's reviewing workflow is non-standard, or if our venue is expecting more than a few hundred submissions for any one deadline, we should designate our own Workflow Chair, who will read the OpenReview documentation and manage our workflow configurations throughout the reviewing process.",
  'We acknowledge that OpenReview staff work Monday-Friday during standard business hours US Eastern time, and we cannot expect support responses outside those times.  For this reason, we recommend setting submission and reviewing deadlines Monday through Thursday.',
  'We will treat the OpenReview staff with kindness and consideration.'
  'We acknowledge that authors and reviewers will be required to share their preferred email.'
  'We acknowledge that certain metadata for accepted papers, specifically the paper title, abstract and author list, will be publicly released on OpenReview.']
    }
    
support_request = openreview.Note(
    invitation = 'OpenReview.net/Support/-/Request_Form', 
    readers = ["OpenReview.net/Support"] + program_chair_emails,
    writers = ["OpenReview.net/Support"] + program_chair_emails,
    content = content,
    signatures = [
        <your_profile_id>
    ]
)
client_v1.post_note(support_request)
```

## 2. Editing Notes

Editing notes in API2 is done the same way as creating notes, by posting a note edit with new values for the fields. The basic note structure looks like this<br>

<pre class="language-python"><code class="lang-python"><strong>note = openreview.api.Note(
</strong>    invitation = 'OpenReview.net/Support/-/Request_Form', 
    readers = ["OpenReview.net/Support"] + program_chair_emails,
    writers = ["OpenReview.net/Support"] + program_chair_emails,
    content = content,
    signatures = [
        &#x3C;your_profile_id>
    ]

</code></pre>

The typical workflow for editing a note is this:

<pre class="language-python"><code class="lang-python"><strong>##get note
</strong><strong>note_to_change = client_v2.get_note(&#x3C;note_id>)
</strong><strong>##modify note
</strong><strong>new_content = note_to_change.content
</strong><strong>new_content[&#x3C;field>]["value"] = new_value
</strong>
<strong>##post change to venue
</strong>client_v2.post_note_edit(invitation=&#x3C;venue_id>/Edit,
        signatures=[venue_id],
        note=openreview.api.Note(
            id=note_to_change.id,
            readers=note_to_change.readers,
            content=new_content
        )
    )


</code></pre>

**Editing fields:** To edit fields, you can edit the `content` object within the note. This is a dictionary object with the key as field names and the following structure:

```
'authorids': {'readers': ['TC/2024/Conference',
                                       'TC/2024/Conference/Submission1/Authors'],
               'value': ['~SomeFirstName_User1',
                           'author@mail.com',
                           'andrew1@amazon.com']},
```

The `value` is the value of the field, while the `readers` are who can see the field (see below).

So to update the 'authorids' field, you would update `new_content['authorids']['value']`

**Editing field readers:** The visibility of individual fields can be controlled by adding, removing, or editing the `readers` property of each field. This takes a list of ID or group names.

Common types of readers are:

* \["everyone"] - Public note, anyone can read it
* \[\<venue\_id>] - Visible only to Program Chairs
* \[\<venue\_id>, \<venue\_id>/Senior\_Area\_Chairs] - Visible only to Program Chairs and SACs
* \[\<venue\_id>, \<venue\_id./Submission1/Authors] - Visible to the Program Chairs and the authors of Submission1

{% hint style="info" %}
The readers for individual fields should always consist of a subset of the note readers.
{% endhint %}

**Editing note readers:** To edit the readers of the entire note, you would edit the `readers` parameter of the entire Note object. This takes a list of ID or group names.

### Quickstart: Update the readers of a note or field

One common edit to make in bulk to notes is updating the readers of a note. This can be done in the UI for common configurations, but if your venue needs further fine-grained control over readership (for example custom tracks, groups, etc), then you can use the Python Client to change the readers of any subset of notes programmatically. The following script will change the readers of a note or field within a note to the new list.

```python
#venue_id
venue_id = "<VENUE_ID>"
note_type = 'Official_Review' #Taken from the last part of the note invitation. Could replace with "Rebuttal", "Decision", "Metareview"
new_readers = ["<LIST_OF_NEW_READERS>"]  # options are 1) ['everyone'] or 2) list of either group names ("Program_Chairs") or Submission + Group Name ("Submission Authors").
field_to_change = None #use None to change readers of the note, or use the field name to change the readers of a field

#set up invitations
edit_invitation = f'{venue_id}/-/Edit' 
submission_invitation = f'{venue_id}/-/Submission'

#get a list of notes to edit (in this case all reviews)
submissions = client_v2.get_notes(invitation=submission_invitation,details = 'replies')
#reviews to edit
reviews=[openreview.api.Note.from_json(reply) for s in submissions for reply in s.details['replies'] if f'{venue_id}/Submission{s.number}/-/{note_type}' in reply['invitations']]
print(len(reviews))
notes_to_change = reviews

#change the readers of the notes
for note_to_change in notes_to_change:
    submission_number = client_v2.get_note(note_to_change.forum).number
    note_content = note_to_change.content
    if new_readers == ['everyone']:

         note_readers = ['everyone']

    else:
        #parse the new_readers:
        group_ids = []
        for g in new_readers:
            if g.startswith('Submission'):
                group_name = g.split(' ')[1]
                new_reader_group = client_v2.get_group(f'{venue_id}/Submission{submission_number}/{group_name}').id
                group_ids.append(new_reader_group)
            else:
                new_reader_group = client_v2.get_group(venue_id + '/' + g)
                group_ids.append(new_reader_group.id)
        #parse the new_readers:
        if field_to_change == None:
            note_readers = list(set(group_ids + note_to_change.signatures + [venue_id]))
        else:
            note_content[field_to_change]['readers'] = list(set(group_ids + note_to_change.signatures + [venue_id]))
        
    client_v2.post_note_edit(invitation=edit_invitation,
            signatures=[venue_id],
            note=openreview.api.Note(
                id=note_to_change.id,
                readers=note_readers,
                content=note_content
            )
        )


```

For more examples, see: how to [hide/reveal fields](https://docs.openreview.net/how-to-guides/submissions-comments-reviews-and-decisions/how-to-hide-reveal-fields).

## 3. Deleting Notes and fields

Deleting notes should be used with caution as most changes can be done by editing a note, and once deleted, changes can be difficult to recover. Typically, we recommend deleting notes only in the case of testing a venue workflow on the dev site. To delete a note, you would post a note edit that includes a value (in millisecond time) for the `ddate` field, which has a value of `None` by default. An example is below:

```python

venue_id = '<VENUE ID>'
note_to_delete = client_v2.get_note('<NOTE_ID>')
time_now = openreview.tools.datetime_millis(dt.datetime.now()))
)

client_v2.post_note_edit(
    invitation=f'{VENUE ID}/-/Edit',
    signatures=note_to_delete.signatures,
    note=openreview.api.Note(
        id=note_to_delete.id,
        content=note_to_delete.content,
        ddate = time_now
        )
```

### Quickstart: Remove a field from a submission

Removing fields from submissions should be done sparingly. Oftentimes, changing the readers of the field its sufficient (see above), but in the case that you want to remove a field entirely, you can use the approach below"

```python
venue_id = "<YOUR_VENUE_ID>"
forum_id_to_change = '123dsavdf'
field_to_delete = 'keywords'
submission = client_v2.get_note(forum_id_to_change)
time_now = openreview.tools.datetime_millis(dt.datetime.now()))
)

edit_content = submission.content
edit_content[field_to_delete][['value']['ddate'] = time_now

client_v2.post_note_edit(invitation=f'{venue_id}/-/PC_Revision',
    signatures=[f'{venue_id}/Program_Chairs'],
    note=openreview.api.Note(
        id = submission.id,
        content = edit_content
    ))
```

### Quickstart: Posting, editing, and deleting a test review

1. Posting a new review note:

```python
submission_number = '<SUBMISSION_NUMBER>'
venue_id = '<VENUE ID>'

anon_groups = client_v2.get_groups(prefix=f'{venue_id}/Submission{submission_number}/Reviewer_', signatory='~Reviewer_One1')
anon_group_id = anon_groups[0].id

review_edit = client_v2.post_note_edit(
    invitation=f'{venue_id}/Submission{submission_number}/-/Official_Review',
    signatures=[anon_group_id],
    note=openreview.api.Note(
        content={
            'title': { 'value': 'Good paper, accept'},
            'review': { 'value': 'Excellent paper, accept'},
            'rating': { 'value': 10},
            'confidence': { 'value': 5},
        }
    )
```

2. Editing an existing review note, you must specify the note ID.

You can [get all reviews](https://docs.openreview.net/how-to-guides/data-retrieval-and-modification/how-to-get-all-notes-for-submissions-reviews-rebuttals-etc#quickstart-getting-reviews-meta-reviews-comments-decisions-rebuttals), get a specific review, or get all reviews for one submission to edit (below):

```python
# Edit the review note

# Get all reviews for a submission
review_notes = client_v2.get_notes(
    invitation=f'{venue_id}/Submission{submission_number}/-/Official_Review'
)

# Get the review to edit
original_review = review_notes[0]

# Update the content of the review with the new content 
review_content = original_review.content
review_content['rating']['value'] = 6

# Step 2: Edit the review to update the rating
edited_review = client_v2.post_note_edit(
    invitation=f'{venue_id}/Submission{submission_number}/-/Official_Review',
    signatures=original_review.signatures, # Using the signatures from the original note
    note=openreview.api.Note(
        id=original_review.id,
        content=review_content
    )
)
```

3. Deleting a review note:

```python
# Assuming the first one is the one we want to delete
note_to_delete = review_notes[0]

# Step 2: Post a deletion edit - to delete the whole review
deleted_note = reviewer_client.post_note_edit(
    invitation=f'{venue_id}/Submission{submission_number}/-/Official_Review',
    signatures=[anon_group_id],
    note=openreview.api.Note(
        id=note_to_delete.id,
        content=note_to_delete.content,
        ddate = openreview.tools.datetime_millis(dt.datetime.now()))
)
```


# How to test your venue workflow

See also:

* [Post a test submission](https://docs.openreview.net/how-to-guides/workflow/how-to-create-change-and-delete-notes)
* Post, edit, and remove reviews

If you wish to test your venue workflow, please use the dev site, as testing on your live venue page can cause collisions and issues when you run your workflow live. To do so, please go to [dev.openreview.net](https://dev.openreview.net/) and complete the following steps:

1. Create a profile for yourself
2. Submit a venue request
3. Email us at <info@openreview.net> so that we can deploy the venue for you.

#### Notes about the dev site

* The **dev site uses a separate database** from the live site- this means that no information is shared between the two sites.
* **Emails are disabled** on the dev site, except for **signup confirmation links**. This means that your test emails will not actually be sent, but they will show up in the messages tab of the site.
* If you want to test with **real author names**, you must **create those profiles on the dev site**. Some profiles may already exist from previous testing.
* The Post Decision stage should **only** be run after testing is complete.

### Resetting the Venue

Most stages can be reset by changing the dates and settings to reopen the stage. If you want to reset to the beginning of the review phase:

1. Delete all review [notes](https://docs.openreview.net/reference/api-v1/entities/note) from the venue
2. Undeploy [assignments](https://docs.openreview.net/how-to-guides/paper-matching-and-assignment/how-to-undo-deployed-assignments) in the assignments browser
3. Re-[assign](https://docs.openreview.net/how-to-guides/paper-matching-and-assignment) reviewers as necessary
4. Change the dates to reopen the review stage


# How to support different tracks for a venue

OpenReview does support a limited functionality for "tracks". If your workflow requires different deadlines and different review forms for each track, then please fill out a separate venue request for each track.

What we currently support:

* Create reviewer, area chair, and senior area chair group roles for each track. Please limit the group roles to a maximum of 10 tracks.
* Assign reviewers and ACs in these independent pools to the submissions with their track selected.
* ACs can modify reviewer assignments after the PCs have initiated the assignments.
* All tracks follow the same workflow, including deadlines and forms.
* Currently, PCs are responsible for recruiting to all track roles, as well as the paper matching setup to compute conflicts and affinity scores and create matching configuration.

If the needs for your venue are not addressed here regarding tracks, please [Contact Support](https://openreview.net/contact). We can try to accommodate custom requests, however, we need to know at least one month before submissions open.

If you want to accept distinct types of submissions that will all follow the same workflow, you can [add a ‘track’ field to your submission form](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/customizing-your-submission-form). If you reach out to [OpenReview support](https://openreview.net/contact), we can then customize your PC console to allow you to filter and sort by track or add roles for reviewers and ACs if you need a group per track.


# How to Make Submissions Available Before the Submission Deadline

Submissions are released to your committee by the **Create Submission Change Before Bidding** step in your [Workflow Timeline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#workflow-timeline). By default this step runs automatically 30 minutes after the submission deadline, and it gives all Senior Area Chairs, Area Chairs and Reviewers access to all submissions, with author identities and PDFs hidden. To release submissions earlier, move the step's activation date forward.

1. Open the Workflow Timeline for your venue and expand the **Create Submission Change Before Bidding** step.
2. Click **Edit** next to **Readers** to choose who can see the submissions. The options are **Program Chairs**, **All Senior Area Chairs**, **All Area Chairs**, **All Reviewers**, **Submission Authors** and **Public** (the Senior Area Chair and Area Chair options only appear if your venue has those roles).
   * To make submissions public, select **Public** on its own. It can't be combined with other options.
   * Otherwise, **Program Chairs** and **Submission Authors** must both be selected.
3. Click **Edit** next to **Restrict Field Visibility** if you want to change which submission fields stay hidden. For more on this, see [How to Hide Submission Fields from Reviewers](https://docs.openreview.net/how-to-guides/workflow/how-to-hide-and-unhide-submission-fields-from-reviewers).
4. Click **Edit** next to **Dates** and set the **Activation Date** to the time you want submissions released. Click **Submit**.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2Fs1Pa1Pvq1jnTb1cQ908N%2Fimage.png?alt=media&#x26;token=b276b7ce-bb9c-4aca-b62e-5fac4f365900" alt="The Create Submission Change Before Bidding step expanded in the Workflow Timeline, with the Readers editor open and the Public option shown in the reader dropdown."><figcaption><p>The Readers editor of Create Submission Change Before Bidding, as Program Chairs see it.</p></figcaption></figure>

When the activation date arrives, the step updates every active submission with the readers you chose. If you selected **Public**, each submission also gets a publication date and a BibTeX entry.

{% hint style="info" %}
The step only updates submissions that exist when it runs. Submissions posted afterwards are not released until the step runs again. While submissions are open, authors can still edit them. Each edit sets the submission's readers back to the Program Chairs and its authors, so the committee loses access to it until the step runs again. To include new submissions and restore access to edited ones, set the **Activation Date** again to a time after the submission deadline, so the step runs once more when all submissions are in.
{% endhint %}

{% hint style="warning" %}
If you later change the dates of the **Submission** step (or the **Full Submission** step, if your venue has two deadlines), a **Create Submission Change Before Bidding** activation date that is earlier than the end of the new submission period (the deadline plus the 30-minute grace period) is moved to that time.
{% endhint %}


# How to Change the Expiration Date of the Submission Invitation

The submission deadline set through the venue request form is actually only the advertised due date. The true deadline is the expiration date, which is set to 30 minutes after the submission deadline. If you would like to change the expiration date to allow the authors more or less wiggle room around the submission deadline, you can do so using the python client.

1. If you have not done so, you will need to [install and instantiate the openreview-py client](https://docs.openreview.net/getting-started/using-the-api/installing-and-instantiating-the-python-client).
2. Choose your desired expiration date. You will need to convert it into epoch time in milliseconds using an [epoch time converter.](https://www.epochconverter.com/) For example:

```python
expdate = 1650639704000
```

Create an Invitation Edit

```python
client.post_invitation_edit(
    invitations='<invitation-id>',  # This is the Invitation used to create this Invitation Edit
    readers=[venue_id],
    writers=[venue_id],
    signatures=[venue_id],
    invitation=openreview.api.Invitation(
        id="<Your/Venue/ID/-/Submission",
        expdate=expate
        signatures=[venue_id]
    )
)
```


# How to begin the Review Stage while Submissions are Open

In the [Workflow Timeline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#workflow-timeline), the steps that prepare submissions for review are scheduled relative to the submission deadline. To start reviewing while submissions are still open, move the **Activation Date** of each of the following steps to a time before the deadline. For each step, expand it, click **Edit** next to **Dates**, change the **Activation Date** and click **Submit**.

1. **Create Reviewers Submission Group**: creates a Reviewers group for each submission. Once this step is active, a group is also created for each new submission as it comes in.
2. **Create Reviewers Conflict** and **Create Reviewers Affinity Score**: compute conflicts of interest and affinity scores between your reviewers and the submissions that exist when the step runs. Before the activation date, choose the conflict policy with **Edit** next to **Policy** on the conflict step, and the expertise model with **Edit** next to **Model** on the affinity score step.
3. Run the reviewer matching: go to the Assignments page, at `https://openreview.net/assignments?group=<venue_id>/Reviewers`, or use the **Reviewers Paper Assignment** link in your Program Chairs console. Click **New Assignment Configuration**, fill in the form and click **Submit**. Then click **Run Matcher** for that configuration and wait until its status is "Complete".
4. **Create Reviewers Assignment Deployment**: click **Edit** next to **Match** and set **Match Name** to the title of the matching configuration you want to deploy. The configuration must have status "Complete". When the step runs, it deploys the assignments and schedules **Create Submission Change Before Reviewing** to run 30 minutes later. That step gives assigned reviewers access to their submissions.
5. **Official Review**: set the **Activation Date** to when reviewers can start submitting reviews, and set the **Due Date** and **Expiration Date**. Once this step is active, new submissions also get a review form when they're posted.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2F4OPZUPjPinWTIk9GUjZz%2Fimage.png?alt=media&#x26;token=f0685de7-df10-4080-b7d4-39ed4e9a315c" alt="The Create Reviewers Assignment Deployment step expanded in the Workflow Timeline, showing its Dates and Match controls."><figcaption><p>The Create Reviewers Assignment Deployment step, as Program Chairs see it.</p></figcaption></figure>

{% hint style="warning" %}
The **Create Reviewers Assignment Deployment** step deploys assignments only once. If a configuration has already been deployed, the step does nothing when it runs again. PCs can make manual assignments for any submissions that arrive after the first matching is deployed.
{% endhint %}

{% hint style="info" %}
Conflicts and affinity scores only cover the submissions that exist when the steps run. To include later submissions, set the step's **Activation Date** again so it runs once more after the deadline.
{% endhint %}

{% hint style="warning" %}
While submissions are open, authors can still edit them. Each edit sets the submission's readers back to the Program Chairs and its authors, so the committee loses access to it until **Create Submission Change Before Reviewing** runs again. After the submission deadline, set its **Activation Date** to the current time to restore access.
{% endhint %}

{% hint style="warning" %}
If you move the **Submission** deadline (or the **Full Submission** deadline) later, any submission group step scheduled before the new deadline is moved to 30 minutes after it, so the groups are no longer created early. After changing the deadline, check the **Activation Date** of **Create Reviewers Submission Group** and the Area Chair and Senior Area Chair submission group steps, and set them again if needed.
{% endhint %}


# How to Change Who can Access Submissions After the Deadline

At any point during your venue's worflow, you can click on the ‘Post Submission’ button and use the field ‘Submission Readers’ to change the readers for all submissions:

* **All program committee (all reviewers, all area chairs, all senior area chairs if applicable)**: all papers are private and only released to all reviewers, area chairs and senior area chairs (if your venue has them)
* **Assigned program committee (assigned reviewers, assigned area chairs, assigned senior area chairs if applicable)**: all papers are private and only released to each paper's assigned reviewers, area chairs and senior area chairs (if your venue has them)
* **Program chairs and paper authors only**: papers are private and released only to program chairs and paper authors
* **Everyone (submissions are public)**: papers are released to the public

You can use this field to change the submission readers as many times as needed.


# How to Enable Commenting on Submissions

Venues created with the Conference Review Workflow include an **Official Comment** step in the [Workflow Timeline](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#workflow-timeline). By default, commenting opens 4 weeks after the submission deadline (the full submission deadline, if your venue has one) and closes 6 weeks after it. The Program Chairs, assigned Senior Area Chairs, assigned Area Chairs, assigned reviewers, and the submission's authors can post comments, and the person commenting chooses who can read each comment.

To change when and how commenting is enabled, click **Official Comment** in the Workflow Timeline to expand the step, then use the **Edit** link next to each control and click **Submit**.

## Set when commenting is open

Edit **Dates**:

* **Activation Date** – when the comment form becomes available on each submission.
* **Expiration Date** – when the comment form is no longer available.

## Choose who can comment and who can read comments

Edit **Writers And Readers**:

* **Writers** – "Who should be able to participate in this stage (read and write comments)?" Choose from Program Chairs, Assigned Senior Area Chairs, Assigned Area Chairs, Assigned Reviewers, Assigned Reviewers who already submitted their review, and Submission Authors. Program Chairs are always included.
* **Readers** – "Who should be able to only read comments?" These are the reader options commenters can choose from. Program Chairs are always included.

Senior Area Chair and Area Chair options only appear if your venue has those roles.

<figure><img src="https://622636955-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FVorH499wd7ipUjYX5etp%2Fuploads%2FR5RzGRmZClT6fUGZLceG%2Fimage.png?alt=media&#x26;token=d9e080b8-69b6-4b0e-b8f9-e78e14c98b33" alt="The Official Comment step expanded in the Workflow Timeline, with the Writers And Readers editor open showing the Writers and Readers selections."><figcaption><p>The Official Comment step, as Program Chairs see it, with the Writers And Readers editor open.</p></figcaption></figure>

## Choose who is notified

Edit **Notifications** and select which groups are emailed when a comment is posted. By default, assigned reviewers and submission authors are notified. An email is only sent to a selected user if they are also a reader of the comment.

## Change the comment form

Edit **Form Fields** to add, change, or remove fields of the comment form. See [How to modify the Review, Meta Review, and Decision Forms](https://docs.openreview.net/how-to-guides/workflow/how-to-modify-the-review-meta-review-and-decision-forms) for how the form editor works.

{% hint style="info" %}
**Public** is not one of the reader options of the **Official Comment** step, and comments from users outside your venue's committee (public comments) cannot be turned on from the Workflow Timeline. To allow either, contact OpenReview Support by posting a comment on your [venue request form](https://docs.openreview.net/getting-started/hosting-a-venue-on-openreview/navigating-your-venue-pages#venue-request-form).
{% endhint %}




---

[Next Page](https://docs.openreview.net/llms-full.txt/1)

