OpenReview
  • Overview
    • OpenReview Documentation
  • Reports
    • Conferences
      • OpenReview NeurIPS 2021 Summary Report
      • OpenReview ECCV 2020 Summary Report
  • Getting Started
    • Frequently Asked Questions
      • I accidentally withdrew a submission, what do I do?
      • How do I add a Program Chair to my venue?
      • When will I be able to withdraw my submission?
      • I want to delete my withdrawn or desk-rejected paper, what do I do?
      • An author of a submission cannot access their own paper, what is the problem?
      • What should I do if I find a vulnerability in OpenReview?
      • How can I report a bug or request a feature?
      • What is the difference between due date (duedate) and expiration date (expdate)?
      • Will Reviewers be notified of their Assignments?
      • What is the max file size for uploads?
      • Why are the "rating" and "confidence" fields in my PC Console wrong?
      • What should I do if my question is not answered here?
      • My Profile is "Limited". What does that mean?
      • What field types are supported in the forms?
      • How do I recruit reviewers?
      • How do I obtain a letter of proof for my services as a reviewer?
      • How do I complete my tasks?
      • Can I automatically transfer my Expertise Selection to another venue?
      • Why does it take two weeks to moderate my profile?
      • What do the different 'status' values mean in the message logs?
      • I am an Independent Researcher, how do I sign up?
      • How do I locate the date a submission is made public?
      • I am a reviewer but I can't access my assigned submissions, what do I do?
      • Reviewers for my venue cannot see their assigned submissions, what should I do?
      • I am a reviewer and I don't have papers for Expertise Selection, what do I do?
      • How do I upload a publication with a license that is not listed?
      • I didn't receive a password reset email, what do I do?
      • How do I add/change an author of my submission after the deadline?
      • How do I find a venue id?
      • Why can't I update my DBLP link?
    • Using the API
      • Installing and Instantiating the Python client
      • Groups
    • Hosting a venue on OpenReview
      • Creating your Venue Instance
      • Navigating your Venue Pages
      • Customizing your submission form
      • Enabling Supplementary Material Upload
      • Changing your submission deadline
      • Enabling an Abstract Registration Deadline
    • Creating an OpenReview Profile
      • Signing up for OpenReview
      • Resending an activation link
      • Expediting Profile Activation
      • Add or remove a name from your profile
      • Add or remove an email address from your profile
      • Finding your profile ID
      • Entering Institutional Data
      • Importing papers from DBLP
      • Manually adding a publication to your profile
      • Finding and adding a Semantic Scholar URL to your profile
      • Finding and adding your ACL Anthology URL to your profile
      • Merging Profiles
    • Customizing Forms
    • Using the New Forum Page
    • Live Chat on the Forum Page
  • Workflows
    • Example Workflow
    • ARR Commitment Venues
    • Exercises for workflow chairs
      • Prerequisites
      • Exercise: Posting LLM generated reviews
  • How-To Guides
    • Modifying Venue Homepages
      • How to customize your venue homepage
      • How to modify the homepage layout to show decision tabs
    • Managing Groups
      • How to Recruit and Remind Recruited Reviewers
      • How to have multiple Reviewer or Area Chair groups
      • How to Add and Remove Members from a Group
      • Publication Chairs
      • How to Copy Members from One Group to Another
    • Workflow
      • How to Programmatically Post Support Request Form
      • How to test your venue workflow
      • How to Post a Test Submission
      • How to support different tracks for a venue
      • How to Make Submissions Available Before the Submission Deadline
      • How to Change the Expiration Date of the Submission Invitation
      • Desk Reject Submissions that are Missing PDFs
      • How to begin the Review Stage while Submissions are Open
      • How to Change Who can Access Submissions After the Deadline
      • How to Enable Commenting on Submissions
      • How to Set a Custom Deadline for Withdrawals
      • How to Enable an Ethics Review Stage
      • How to Hide Submission Fields from Reviewers
      • How to modify the Review, Meta Review, and Decision Forms
      • How to release reviews
      • How to Enable the Rebuttal Period
      • How to Undo a Paper Withdrawal
      • How to enable Camera Ready Revision Upload for accepted papers
      • How to make papers public after decisions are made
      • How to enable bidding for Senior Area Chair Assignment
      • How to release the identities of authors of accepted papers only
      • How to enable the Review Revision Stage
    • Paper Matching and Assignment
      • How to Compute Conflicts Between Users
      • How to Post a Custom Conflict
      • How to create your own Conflict Policy
      • How to Bid on Submissions
      • How to add/remove bids programmatically
      • How to do manual assignments
      • How to do automatic assignments
        • How to setup paper matching by calculating affinity scores and conflicts
        • How to run a paper matching
        • How to modify the proposed assignments
        • How to deploy the proposed assignments
        • How to modify assignments after deployment
      • How to enable Reviewer Reassignment for Area Chairs
      • How to Sync Manual and Automatic Assignments
      • How to Compute Affinity Scores
      • How to Undo Deployed Assignments
      • How to Modify Reviewer Assignments as an Area Chair
      • How to Get all Assignments for a User
      • How to Update Custom Max Papers for Reviewers or ACs
      • How to Make Assignments using Subject Areas
    • Communication
      • How to send messages through the UI
      • How to customize emails sent through OpenReview
      • How to send messages with the python client
      • How to Send Decision Notifications Using the UI
      • How to view messages sent through OpenReview
      • How to email the authors of accepted submissions
      • How to get email adresses
    • Submissions, comments, reviews, and decisions
      • How to add formatting to reviews or comments
      • How to submit a Review Revision
      • How to add formulas or use mathematical notation
      • How to edit a submission after the deadline - Authors
      • How to upload paper decisions in bulk
      • How to hide/reveal fields
      • Update camera-ready PDFs after the deadline expires
    • Data Retrieval and Modification
      • How to check the API version of a venue
      • How to view Camera-Ready Revisions
      • How to Export all Submission Attachments
      • How to loop through Accepted Papers and print the Authors and their Affiliations
      • How to add/remove fields from a submission
      • How to manually change the readers of a note
      • How to post/delete an Official Review using Python
      • How to Get Profiles and Their Relations
      • How to Get All the Reviews that I have written and their Corresponding Submissions
      • How to Get All Registration Notes
      • How to Get All Submissions
      • How to Get All Reviews
      • How to Export All Reviews into a CSV
      • How to get all Rebuttals
      • How to Get All Official Comments
      • How to Get All MetaReviews
      • How to Get All Decisions
      • How to Get All Venues
      • How to Retrieve Data for ACM Proceedings
      • How to Get Reviewer Ratings
  • Reference
    • API V1
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Note
          • Fields
        • Invitation
    • API V2
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Group
          • Fields
        • Note
          • Fields
        • Invitation
          • Types and Structure
          • Fields
          • Specifiers
          • Dollar Sign Notation
        • Edit
          • Fields
          • Inference
    • Stages
      • Revision
      • Registration Stage
      • Bid Stage
      • Review Stage
      • Rebuttal Stage
      • Meta Review Stage
      • Decision Stage
      • Comment Stage
      • Submission Revision Stage
      • Post Submission Stage
      • Post Decision Stage
      • Ethics Review Stage
    • Default Forms
      • Default Submission Form
      • Default Registration Form
      • Default Comment Form
      • Default Review Form
      • Default Rebuttal Form
      • Default Meta Review Form
      • Default Decision Form
      • Default Decision Notification
      • Default Ethics Review Form
    • OpenReview TeX
      • Common Issues with LaTeX Code Display
      • OpenReview TeX support
    • Mental Model on Blind Submissions and Revisions
Powered by GitBook
On this page
  • Where to customize forms
  • Essential structure of custom fields
  • Specifiers
  • Representation Specifiers
  • Validation Specifiers
  • Examples of Common Form Patterns
  • Text Input
  • Single Choice
  • Multiple Choices
  • Miscellaneous
  • Setting the Readers of a Field

Was this helpful?

Export as PDF
  1. Getting Started

Customizing Forms

A guide for program chairs on how to customize the different forms available to the reviewing committee

PreviousMerging ProfilesNextUsing the New Forum Page

Last updated 3 months ago

Was this helpful?

Where to customize forms

Many OpenReview forms are customizable from the different buttons on the venue request form. You can find where to input your customizations by clicking on a button (for example, "Review Stage") and finding the large text box under "Additional _____ Options". Under "Revision", you'll find that this box modifies the submission form under the heading "Additional Submission Options". For the "Review Stage", the heading will be "Additional Review Form Options"

Some buttons configure other parts of the workflow that do not have an associated form, in which case there will not be an additional options text box.

Whenever possible, forms should be customized through the venue request form following the directions above rather than editing an invitation directly - this saves your changes in the case that you update any other settings for your venue.

Essential structure of custom fields

These text boxes accept a valid JSON object with fields and values. The following is an example where the title field gets replaced with a radio button, like so:

{
    "title": {
        "order": 1,
        "description": "Title of the paper",
        "value": {
            "param": {
                "type": "string",
                "enum": ["Test Submission Title"],
                "input": "radio"
            }
        }
    }
}

Note that correct indentation levels and matched brackets are necessary for valid JSON.

The primary fields of the entry are:

title - This will be the name of the field in the form

order - Determines where in the form the field will appear

description - Will show up in the form as instructions or description

value - Will have subfields (under param) determining the format of the field and the options for responses

"value": {
    "param": {
        "type": "string",
        "enum": ["Test Submission Title"],
        "input": "radio"
    }
}

When making a field that is asking for user input, you will always see this pattern of "value": { "param": {...} }. Inside the param object are fields determining what the user sees in the input form along with what the user is allowed to submit: these are representation specifiers and validation specifiers.

Both validation and representation specifiers can be found inside the param object

Specifiers

This section will introduce common specifiers used in customizing forms. Further information about specifiers can be found here.

Representation Specifiers

Representation specifiers determine how the user will input their response into the field (for example a textbox or a checklist). These will be defined in the param object.

The input specifier determines the rendering on the form and can have the following values (see below for examples of how different input types render):

  • text

  • select

  • checkbox

  • textarea

  • radio

default is the default value of the box that will appear when the widget is initialized

markdown is a boolean value (true/false) that enables markdown for this text field

scroll is a boolean that adds a scroll bar to a textarea input

Validation Specifiers

Validation specifiers are used by the back-end to ensure data submitted through the form conforms to certain requirements. In the example above, only a single string is allowed, and that string must be one of the values defined in the enum array. Specifically, a string that has the value "Test Submission Title"

optional is a boolean (true/false) value that indicates whether or not this field is required to be present when the form is submitted. By default all fields in the form are required, and you can add optional : true to indicate a required field.

Required fields have their field names prefixed with an asterisk

type specifiers require the input to be of a specific type: options are string, string[] (string array) and file.

string fields can be further validated by using fields to describe the structure of a valid string input. Some of these field are:

  • "maxLength": set the maximum number of characters of the input

  • "minLength":set the minimum number of characters of the input

  • "regex": use regular expressions to define acceptable string structures

  • "enum": restrict the user to a predefined set of strings

  • "items": an array of strings as indicated by its type (only used with type: string[] ) **

** All values in "items" will be considered required unless specified otherwise with "optional": true. Please see multiple choices for an example.

file fields are specifically validated with maxSize and extensions. maxSize is an integer that specifies the size of the largest file that can be uploaded on the form in megabytes. extensions is a list of strings that are extensions, for example "extensions": ["pdf", "zip"].

Extensions that have a "." in them are not supported. The following field would be invalid because "tar.gz" is not supported:

"extensions": ["zip", "tar.gz"]

Examples of Common Form Patterns

Text Input

{
  "title": {
    "order": 1,
    "description": "(Optional) Brief summary of your comment.",
    "value": {
      "param": {
        "type": "string",
        "maxLength": 500,
        "optional": true
      }
    }
  }
}
{
  "comment": {
    "order": 2,
    "description": "Your comment or reply (max 5000 characters). Add formatting using Markdown and formulas using LaTeX. For more information see https://openreview.net/faq",
    "value": {
      "param": {
        "type": "string",
        "maxLength": 5000,
        "markdown": true,
        "input": "textarea"
      }
    }
  }
}

Single Choice

{
  "recommendation": {
    "order": 10,
    "description": "Please provide your recommendation based on the manuscript, reviews, author responses and your discussion with the reviewers.",
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "Reject",
          "Accept",
          "Nominate for best paper"
        ],
        "input": "radio"
      }
    }
  }
}
{
  "submission_track": {
    "order": 11,
    "description": "Please select a preferred track, which will be considered in the paper-reviewer assignment. Please note that your choice of a track will be taken into account, but the chairs may decide to move your paper to a different area when appropriate.",
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "Commonsense Reasoning",
          "Computational Social Science and Cultural Analytics",
          "Dialogue and Interactive Systems",
          "Discourse and Pragmatics",
          "Efficient Methods for NLP",
          "Ethics in NLP",
          "Human-Centered NLP",
          "Information Extraction",
          "Information Retrieval and Text Mining",
          "Interpretability, Interactivity, and Analysis of Models for NLP",
          "Language Grounding to Vision, Robotics and Beyond",
          "Language Modeling and Analysis of Language Models",
          "Linguistic Theories, Cognitive Modeling, and Psycholinguistics",
          "Machine Learning for NLP",
          "Machine Translation",
          "Multilinguality and Linguistic Diversity",
          "Natural Language Generation",
          "NLP Applications",
          "Phonology, Morphology, and Word Segmentation",
          "Question Answering",
          "Resources and Evaluation",
          "Semantics: Lexical",
          "Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.",
          "Sentiment Analysis, Stylistic Analysis, and Argument Mining",
          "Speech and Multimodality",
          "Summarization",
          "Syntax, Parsing and their Applications",
          "Theme Track: Large Language Models and the Future of NLP"
        ],
        "input": "select"
      }
    }
  }
}

Multiple Choices

{
  "confirmation_of_submission_requirements": {
    "order": 17,
    "description": "Before you submit this paper, please make sure that the following requirements are met. If any of these requirements are not fulfilled, your submission will be rejected and will not be reviewed.",
    "value": {
      "param": {
        "type": "string[]",
        "optional": true,
        "items": [
          { "value": "Requirement 1", "description": "Requirement 1", "optional": true},
          { "value": "Requirement 2", "description": "Requirement 1", "optional": true}
        ],
        "input": "checkbox"
      }
    }
  }
}
{
  "submission_tracks": {
    "order": 11,
    "description": "Please select your preferred tracks, which will be considered in the paper-reviewer assignment. Please note that your choice of tracks will be taken into account, but the chairs may decide to move your paper to a different area when appropriate.",
    "value": {
      "param": {
        "type": "string[]",
        "items": [
            { "value": "Commonsense Reasoning", "description": "Commonsense Reasoning", "optional": true },
            { "value": "Computational Social Science and Cultural Analytics", "description": "Computational Social Science and Cultural Analytics", "optional": true },
            { "value": "Dialogue and Interactive Systems", "description": "Dialogue and Interactive Systems", "optional": true },
            { "value": "Discourse and Pragmatics", "description": "Discourse and Pragmatics", "optional": true },
            { "value": "Efficient Methods for NLP", "description": "Efficient Methods for NLP", "optional": true },
            { "value": "Ethics in NLP", "description": "Ethics in NLP", "optional": true },
            { "value": "Human-Centered NLP", "description": "Human-Centered NLP", "optional": true },
            { "value": "Information Extraction", "description": "Information Extraction", "optional": true },
            { "value": "Information Retrieval and Text Mining", "description": "Information Retrieval and Text Mining", "optional": true },
            { "value": "Interpretability, Interactivity, and Analysis of Models for NLP", "description": "Interpretability, Interactivity, and Analysis of Models for NLP", "optional": true },
            { "value": "Language Grounding to Vision, Robotics and Beyond", "description": "Language Grounding to Vision, Robotics and Beyond", "optional": true },
            { "value": "Language Modeling and Analysis of Language Models", "description": "Language Modeling and Analysis of Language Models", "optional": true },
            { "value": "Linguistic Theories, Cognitive Modeling, and Psycholinguistics", "description": "Linguistic Theories, Cognitive Modeling, and Psycholinguistics", "optional": true },
            { "value": "Machine Learning for NLP", "description": "Machine Learning for NLP", "optional": true },
            { "value": "Machine Translation", "description": "Machine Translation", "optional": true },
            { "value": "Multilinguality and Linguistic Diversity", "description": "Multilinguality and Linguistic Diversity", "optional": true },
            { "value": "Natural Language Generation", "description": "Natural Language Generation", "optional": true },
            { "value": "NLP Applications", "description": "NLP Applications", "optional": true },
            { "value": "Phonology, Morphology, and Word Segmentation", "description": "Phonology, Morphology, and Word Segmentation", "optional": true },
            { "value": "Question Answering", "description": "Question Answering", "optional": true },
            { "value": "Resources and Evaluation", "description": "Resources and Evaluation", "optional": true },
            { "value": "Semantics: Lexical", "description": "Semantics: Lexical", "optional": true },
            { "value": "Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.", "description": "Semantics: Lexical, Sentence level, Document Level, Textual Inference, etc.", "optional": true },
            { "value": "Sentiment Analysis, Stylistic Analysis, and Argument Mining", "description": "Sentiment Analysis, Stylistic Analysis, and Argument Mining", "optional": true },
            { "value": "Speech and Multimodality", "description": "Speech and Multimodality", "optional": true },
            { "value": "Summarization", "description": "Summarization", "optional": true },
            { "value": "Syntax, Parsing and their Applications", "description": "Syntax, Parsing and their Applications", "optional": true },
            { "value": "Theme Track: Large Language Models and the Future of NLP", "description": "Theme Track: Large Language Models and the Future of NLP", "optional": true }
        ],
        "input": "select"
      }
    }
  }
}

Miscellaneous

{
  "supplementary_material": {
    "order": 10,
    "description": "All supplementary material must be self-contained and zipped into a single file. Note that supplementary material will be visible to reviewers and the public throughout and after the review period, and ensure all material is anonymized. The maximum file size is 200MB.",
    "value": {
      "param": {
        "type": "file",
        "extensions": [
          "zip",
          "pdf",
          "tgz",
          "gz"
        ],
        "maxSize": 200,
        "optional": true
      }
    }
  }
}

Setting the Readers of a Field

If you want to limit who in the committee can see a particular field in a form, this is done by adding a readers field. Please follow this link for more detailed information on hiding or revealing fields. Below are two examples, one for the submission form and one for the meta review form. Notice the different use of dollar sign notation. The notation used for the meta review form will also work for other replies to the forum: reviews, comments, and decisions.

{
  "supplementary_material": {
    "order": 10,
    "description": "All supplementary material must be self-contained and zipped into a single file. Note that supplementary material will be visible to reviewers and the public throughout and after the review period, and ensure all material is anonymized. The maximum file size is 200MB.",
    "value": {
      "param": {
        "type": "file",
        "extensions": [
          "zip",
          "pdf",
          "tgz",
          "gz"
        ],
        "maxSize": 200,
        "optional": true
      }
    },
    "readers": [
    "Your/Venue/ID/Program_Chairs",
    "Your/Venue/ID/Submission${4/number}/Senior_Area_Chairs",
    "Your/Venue/ID/Submission${4/number}/Authors"
    ]
  }
}
{
  "recommendation": {
    "order": 10,
    "description": "Please provide your recommendation based on the manuscript, reviews, author responses and your discussion with the reviewers.",
    "value": {
      "param": {
        "type": "string",
        "enum": [
          "Accept (Oral)",
          "Accept (Poster)",
          "Reject"
        ],
        "input": "radio"
      }
    },
    "readers": [
    "Your/Venue/ID/Program_Chairs",
    "Your/Venue/ID/Submission${7/content/noteNumber/value}/Area_Chairs",
    "Your/Venue/ID/Submission${7/content/noteNumber/value}/Reviewers"
  ]
  }
}
Preview of how this field will be rendered on OpenReview