OpenReview
  • Overview
    • OpenReview Documentation
  • Reports
    • Conferences
      • OpenReview NeurIPS 2021 Summary Report
      • OpenReview ECCV 2020 Summary Report
  • Getting Started
    • Frequently Asked Questions
      • I accidentally withdrew a submission, what do I do?
      • How do I add a Program Chair to my venue?
      • When will I be able to withdraw my submission?
      • I want to delete my withdrawn or desk-rejected paper, what do I do?
      • An author of a submission cannot access their own paper, what is the problem?
      • What should I do if I find a vulnerability in OpenReview?
      • How can I report a bug or request a feature?
      • What is the difference between due date (duedate) and expiration date (expdate)?
      • Will Reviewers be notified of their Assignments?
      • What is the max file size for uploads?
      • Why are the "rating" and "confidence" fields in my PC Console wrong?
      • What should I do if my question is not answered here?
      • My Profile is "Limited". What does that mean?
      • What field types are supported in the forms?
      • How do I recruit reviewers?
      • How do I obtain a letter of proof for my services as a reviewer?
      • How do I complete my tasks?
      • Can I automatically transfer my Expertise Selection to another venue?
      • Why does it take two weeks to moderate my profile?
      • What do the different 'status' values mean in the message logs?
      • I am an Independent Researcher, how do I sign up?
      • How do I locate the date a submission is made public?
      • I am a reviewer but I can't access my assigned submissions, what do I do?
      • Reviewers for my venue cannot see their assigned submissions, what should I do?
      • I am a reviewer and I don't have papers for Expertise Selection, what do I do?
      • How do I upload a publication with a license that is not listed?
      • I didn't receive a password reset email, what do I do?
      • How do I add/change an author of my submission after the deadline?
      • How do I find a venue id?
      • Why can't I update my DBLP link?
    • Using the API
      • Installing and Instantiating the Python client
      • Groups
    • Hosting a venue on OpenReview
      • Creating your Venue Instance
      • Navigating your Venue Pages
      • Customizing your submission form
      • Enabling Supplementary Material Upload
      • Changing your submission deadline
      • Enabling an Abstract Registration Deadline
    • Creating an OpenReview Profile
      • Signing up for OpenReview
      • Resending an activation link
      • Expediting Profile Activation
      • Add or remove a name from your profile
      • Add or remove an email address from your profile
      • Finding your profile ID
      • Entering Institutional Data
      • Importing papers from DBLP
      • Manually adding a publication to your profile
      • Finding and adding a Semantic Scholar URL to your profile
      • Finding and adding your ACL Anthology URL to your profile
      • Merging Profiles
    • Customizing Forms
    • Using the New Forum Page
    • Live Chat on the Forum Page
  • Workflows
    • Example Workflow
    • ARR Commitment Venues
    • Exercises for workflow chairs
      • Prerequisites
      • Exercise: Posting LLM generated reviews
  • How-To Guides
    • Modifying Venue Homepages
      • How to customize your venue homepage
      • How to modify the homepage layout to show decision tabs
    • Managing Groups
      • How to Recruit and Remind Recruited Reviewers
      • How to have multiple Reviewer or Area Chair groups
      • How to Add and Remove Members from a Group
      • Publication Chairs
      • How to Copy Members from One Group to Another
    • Workflow
      • How to Programmatically Post Support Request Form
      • How to test your venue workflow
      • How to Post a Test Submission
      • How to support different tracks for a venue
      • How to Make Submissions Available Before the Submission Deadline
      • How to Change the Expiration Date of the Submission Invitation
      • Desk Reject Submissions that are Missing PDFs
      • How to begin the Review Stage while Submissions are Open
      • How to Change Who can Access Submissions After the Deadline
      • How to Enable Commenting on Submissions
      • How to Set a Custom Deadline for Withdrawals
      • How to Enable an Ethics Review Stage
      • How to Hide Submission Fields from Reviewers
      • How to modify the Review, Meta Review, and Decision Forms
      • How to release reviews
      • How to Enable the Rebuttal Period
      • How to Undo a Paper Withdrawal
      • How to enable Camera Ready Revision Upload for accepted papers
      • How to make papers public after decisions are made
      • How to enable bidding for Senior Area Chair Assignment
      • How to release the identities of authors of accepted papers only
      • How to enable the Review Revision Stage
    • Paper Matching and Assignment
      • How to Compute Conflicts Between Users
      • How to Post a Custom Conflict
      • How to create your own Conflict Policy
      • How to Bid on Submissions
      • How to add/remove bids programmatically
      • How to do manual assignments
      • How to do automatic assignments
        • How to setup paper matching by calculating affinity scores and conflicts
        • How to run a paper matching
        • How to modify the proposed assignments
        • How to deploy the proposed assignments
        • How to modify assignments after deployment
      • How to enable Reviewer Reassignment for Area Chairs
      • How to Sync Manual and Automatic Assignments
      • How to Compute Affinity Scores
      • How to Undo Deployed Assignments
      • How to Modify Reviewer Assignments as an Area Chair
      • How to Get all Assignments for a User
      • How to Update Custom Max Papers for Reviewers or ACs
      • How to Make Assignments using Subject Areas
    • Communication
      • How to send messages through the UI
      • How to customize emails sent through OpenReview
      • How to send messages with the python client
      • How to Send Decision Notifications Using the UI
      • How to view messages sent through OpenReview
      • How to email the authors of accepted submissions
      • How to get email adresses
    • Submissions, comments, reviews, and decisions
      • How to add formatting to reviews or comments
      • How to submit a Review Revision
      • How to add formulas or use mathematical notation
      • How to edit a submission after the deadline - Authors
      • How to upload paper decisions in bulk
      • How to hide/reveal fields
      • Update camera-ready PDFs after the deadline expires
    • Data Retrieval and Modification
      • How to check the API version of a venue
      • How to view Camera-Ready Revisions
      • How to Export all Submission Attachments
      • How to loop through Accepted Papers and print the Authors and their Affiliations
      • How to add/remove fields from a submission
      • How to manually change the readers of a note
      • How to post/delete an Official Review using Python
      • How to Get Profiles and Their Relations
      • How to Get All the Reviews that I have written and their Corresponding Submissions
      • How to Get All Registration Notes
      • How to Get All Submissions
      • How to Get All Reviews
      • How to Export All Reviews into a CSV
      • How to get all Rebuttals
      • How to Get All Official Comments
      • How to Get All MetaReviews
      • How to Get All Decisions
      • How to Get All Venues
      • How to Retrieve Data for ACM Proceedings
      • How to Get Reviewer Ratings
  • Reference
    • API V1
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Note
          • Fields
        • Invitation
    • API V2
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Group
          • Fields
        • Note
          • Fields
        • Invitation
          • Types and Structure
          • Fields
          • Specifiers
          • Dollar Sign Notation
        • Edit
          • Fields
          • Inference
    • Stages
      • Revision
      • Registration Stage
      • Bid Stage
      • Review Stage
      • Rebuttal Stage
      • Meta Review Stage
      • Decision Stage
      • Comment Stage
      • Submission Revision Stage
      • Post Submission Stage
      • Post Decision Stage
      • Ethics Review Stage
    • Default Forms
      • Default Submission Form
      • Default Registration Form
      • Default Comment Form
      • Default Review Form
      • Default Rebuttal Form
      • Default Meta Review Form
      • Default Decision Form
      • Default Decision Notification
      • Default Ethics Review Form
    • OpenReview TeX
      • Common Issues with LaTeX Code Display
      • OpenReview TeX support
    • Mental Model on Blind Submissions and Revisions
Powered by GitBook
On this page

Was this helpful?

Export as PDF
  1. Getting Started

Live Chat on the Forum Page

PreviousUsing the New Forum PageNextExample Workflow

Last updated 11 months ago

Was this helpful?

Live chat is a new OpenReview feature available to all venues to support synchronous dialog among reviewers and ACs. Similar to a Slack channel or chat room, messages will be visible to all participants and appear in real time as they are posted. In addition, there is an option to enable browser notifications so participants can be alerted when new messages are posted.

This feature was developed with encouragement and funding from the Alfred P. Sloan Foundation. For more information, see the Motivations section below.

Enabling Live Chat

Program Chairs can enable this feature from the venue management forum using a . First, click the Comment Stage button, then select the option "Enable Chat Between Committee Members". Members of the committee must be selected as participants.

Similarly, live chat can be disabled for all forum pages by clicking Comment Stage and selecting the disable chat option.

Once enabled, the live chat interface will be visible on all forum pages under the “Committee Members Chat” tab. Chat messages are hidden in the “Discussion” tab to prevent confusion and keep the discussions with paper authors and other users separate.

Chat Features

  • Real-time: Type a simple message, and it appears immediately to other reviewers who have the same OpenReview forum page open in their browser, enabling interactive discussion.

  • Private: By default, messages are visible all program chairs, area chairs, and reviewers of a paper. The messages will never be visible to the authors or the public. As an extra reminder, the chat participants are below the text input box. The ability to send private messages to a subset of the chat participants is planned for a future release.

  • Rich text: Chat messages can use Markdown for formatting and LaTeX for math notation. When composing a message you can preview how it will look by clicking the Preview button. To copy the unformatted text of a posted chat message, hover over the message and click the View Raw button.

  • Replies: Users can reply to specific message from earlier in the chat by hovering over the message and clicking the Reply button.

  • Permalinks: Users can copy a URL pointing to a specific chat message by hovering over the message and clicking the Copy Link button.

  • Notifications: Browser and email notifications are supported. User should enable browser notifications from their browser. Email notifications will be sent in bulk either every 5 messages or every 4 hours.

  • Reactions: Users can add emoji "reactions" to chat messages, in the same way they can react to messages on apps like Slack and GitHub. To add a reaction, hover over the message and click the "Add Reaction" button, then select the emoji you want to use. All the reactions a message has gotten will be shown below the message content as separate buttons. Clicking one of these buttons will either add a reaction of that type, or remove your reaction if you have already added one.

Motivation

In the past, many peer-reviewed yearly conferences in computer science (such as NeurIPS and CVPR) held an in-person meeting among the 20-30 “area chairs” of the conference to discuss and debate each paper. These meetings enabled extensive synchronous communication and serendipitous discussion during the final decision-making phase of the peer review process.

Unfortunately, these practices of in-person area-chair meetings are increasingly rare due to growing conference size, travel expenses, and (more recently) risks from the pandemic. Given the current universal use of web tools for reviewing, one would hope that the scientific communities would take advantage of internet technology to greatly increase interactivity during the peer review process, but unfortunately, reviewing workflows remain largely unchanged.

One exception is the introduction of “rebuttal statements” (an opportunity for authors to write a response to first drafts of the reviews), and asynchronous messages among the reviewing team. However, due to the lack of synchronous communication, discussion of these rebuttals (and scientific debate among the decision-making team) is hampered – often lackluster or incomplete.

Synchronous communication creates beneficial social pressure for engagement and the opportunity for interactive clarification without the extensive “context switching” inherent in asynchronously juggling many tasks. One highly experienced computer science reviewer and OpenReview user told us, “The burden of context switching after 24 hours away from a conversation can sometimes be the bottleneck in writing a response; re-remembering the details of a paper often takes even more time than writing a reply to a co-reviewer. Synchronous communication could dramatically reduce the burden and increase quality of reviewing by removing this context-switching bottleneck.”

While reviewers may naturally find each other online at the same time, and benefit from the chat feature, scheduled chat discussion sessions may be even more helpful. Scheduling synchronous sessions will be one key aspect of recapturing the benefits of in-person synchronous discussion. The details and cultural considerations of this scheduling will be driven by the program chairs. The in-person meetings were a burden, but also provided a scientific/social occasion that was enjoyed and cherished; we anticipate that with some sensitive design, program chairs can provide a similar occasion. The burden can be reduced by culling from the process submissions that are clear rejections.

Why text messaging rather than audio/video? Although speaking is easy, typing has multiple advantages: typed messages can be more easily archived, read later, skimmed quickly, analyzed, and studied. Many people are used to interactive synchronous text chat tools, such as Slack, and find them very productive. Furthermore, text has high accessibility, and is recommended by the Web Content Accessibility Guidelines (WCAG 2.0). With text entry, users can naturally make some statements publicly and mark others as private; scientists are already used to this ability in typed chat systems such as Slack or Zoom. Typed chat messages support not only fully-synchronous interaction, but also interactive semi-synchronous interaction (e.g. with two minute interruption or delay), which fits modern work styles. If some users are slow typists, they can use voice recognition on their devices to type messages at speaking speed.

Comment Stage