OpenReview
  • Overview
    • OpenReview Documentation
  • Reports
    • Conferences
      • OpenReview NeurIPS 2021 Summary Report
      • OpenReview ECCV 2020 Summary Report
  • Getting Started
    • Frequently Asked Questions
      • I accidentally withdrew a submission, what do I do?
      • How do I add a Program Chair to my venue?
      • When will I be able to withdraw my submission?
      • I want to delete my withdrawn or desk-rejected paper, what do I do?
      • An author of a submission cannot access their own paper, what is the problem?
      • What should I do if I find a vulnerability in OpenReview?
      • How can I report a bug or request a feature?
      • What is the difference between due date (duedate) and expiration date (expdate)?
      • Will Reviewers be notified of their Assignments?
      • What is the max file size for uploads?
      • Why are the "rating" and "confidence" fields in my PC Console wrong?
      • What should I do if my question is not answered here?
      • My Profile is "Limited". What does that mean?
      • What field types are supported in the forms?
      • How do I recruit reviewers?
      • How do I obtain a letter of proof for my services as a reviewer?
      • How do I complete my tasks?
      • Can I automatically transfer my Expertise Selection to another venue?
      • Why does it take two weeks to moderate my profile?
      • What do the different 'status' values mean in the message logs?
      • I am an Independent Researcher, how do I sign up?
      • How do I locate the date a submission is made public?
      • I am a reviewer but I can't access my assigned submissions, what do I do?
      • Reviewers for my venue cannot see their assigned submissions, what should I do?
      • I am a reviewer and I don't have papers for Expertise Selection, what do I do?
      • How do I upload a publication with a license that is not listed?
      • I didn't receive a password reset email, what do I do?
      • How do I add/change an author of my submission after the deadline?
      • How do I find a venue id?
      • Why can't I update my DBLP link?
    • Using the API
      • Installing and Instantiating the Python client
      • Groups
    • Hosting a venue on OpenReview
      • Creating your Venue Instance
      • Navigating your Venue Pages
      • Customizing your submission form
      • Enabling Supplementary Material Upload
      • Changing your submission deadline
      • Enabling an Abstract Registration Deadline
    • Creating an OpenReview Profile
      • Signing up for OpenReview
      • Resending an activation link
      • Expediting Profile Activation
      • Add or remove a name from your profile
      • Add or remove an email address from your profile
      • Finding your profile ID
      • Entering Institutional Data
      • Importing papers from DBLP
      • Manually adding a publication to your profile
      • Finding and adding a Semantic Scholar URL to your profile
      • Finding and adding your ACL Anthology URL to your profile
      • Merging Profiles
    • Customizing Forms
    • Using the New Forum Page
    • Live Chat on the Forum Page
  • Workflows
    • Example Workflow
    • ARR Commitment Venues
    • Exercises for workflow chairs
      • Prerequisites
      • Exercise: Posting LLM generated reviews
  • How-To Guides
    • Modifying Venue Homepages
      • How to customize your venue homepage
      • How to modify the homepage layout to show decision tabs
    • Managing Groups
      • How to Recruit and Remind Recruited Reviewers
      • How to have multiple Reviewer or Area Chair groups
      • How to Add and Remove Members from a Group
      • Publication Chairs
      • How to Copy Members from One Group to Another
    • Workflow
      • How to Programmatically Post Support Request Form
      • How to test your venue workflow
      • How to Post a Test Submission
      • How to support different tracks for a venue
      • How to Make Submissions Available Before the Submission Deadline
      • How to Change the Expiration Date of the Submission Invitation
      • Desk Reject Submissions that are Missing PDFs
      • How to begin the Review Stage while Submissions are Open
      • How to Change Who can Access Submissions After the Deadline
      • How to Enable Commenting on Submissions
      • How to Set a Custom Deadline for Withdrawals
      • How to Enable an Ethics Review Stage
      • How to Hide Submission Fields from Reviewers
      • How to modify the Review, Meta Review, and Decision Forms
      • How to release reviews
      • How to Enable the Rebuttal Period
      • How to Undo a Paper Withdrawal
      • How to enable Camera Ready Revision Upload for accepted papers
      • How to make papers public after decisions are made
      • How to enable bidding for Senior Area Chair Assignment
      • How to release the identities of authors of accepted papers only
      • How to enable the Review Revision Stage
    • Paper Matching and Assignment
      • How to Compute Conflicts Between Users
      • How to Post a Custom Conflict
      • How to create your own Conflict Policy
      • How to Bid on Submissions
      • How to add/remove bids programmatically
      • How to do manual assignments
      • How to do automatic assignments
        • How to setup paper matching by calculating affinity scores and conflicts
        • How to run a paper matching
        • How to modify the proposed assignments
        • How to deploy the proposed assignments
        • How to modify assignments after deployment
      • How to enable Reviewer Reassignment for Area Chairs
      • How to Sync Manual and Automatic Assignments
      • How to Compute Affinity Scores
      • How to Undo Deployed Assignments
      • How to Modify Reviewer Assignments as an Area Chair
      • How to Get all Assignments for a User
      • How to Update Custom Max Papers for Reviewers or ACs
      • How to Make Assignments using Subject Areas
    • Communication
      • How to send messages through the UI
      • How to customize emails sent through OpenReview
      • How to send messages with the python client
      • How to Send Decision Notifications Using the UI
      • How to view messages sent through OpenReview
      • How to email the authors of accepted submissions
      • How to get email adresses
    • Submissions, comments, reviews, and decisions
      • How to add formatting to reviews or comments
      • How to submit a Review Revision
      • How to add formulas or use mathematical notation
      • How to edit a submission after the deadline - Authors
      • How to upload paper decisions in bulk
      • How to hide/reveal fields
      • Update camera-ready PDFs after the deadline expires
    • Data Retrieval and Modification
      • How to check the API version of a venue
      • How to view Camera-Ready Revisions
      • How to Export all Submission Attachments
      • How to loop through Accepted Papers and print the Authors and their Affiliations
      • How to add/remove fields from a submission
      • How to manually change the readers of a note
      • How to post/delete an Official Review using Python
      • How to Get Profiles and Their Relations
      • How to Get All the Reviews that I have written and their Corresponding Submissions
      • How to Get All Registration Notes
      • How to Get All Submissions
      • How to Get All Reviews
      • How to Export All Reviews into a CSV
      • How to get all Rebuttals
      • How to Get All Official Comments
      • How to Get All MetaReviews
      • How to Get All Decisions
      • How to Get All Venues
      • How to Retrieve Data for ACM Proceedings
      • How to Get Reviewer Ratings
  • Reference
    • API V1
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Note
          • Fields
        • Invitation
    • API V2
      • OpenAPI definition
      • Entities
        • Edge
          • Fields
        • Group
          • Fields
        • Note
          • Fields
        • Invitation
          • Types and Structure
          • Fields
          • Specifiers
          • Dollar Sign Notation
        • Edit
          • Fields
          • Inference
    • Stages
      • Revision
      • Registration Stage
      • Bid Stage
      • Review Stage
      • Rebuttal Stage
      • Meta Review Stage
      • Decision Stage
      • Comment Stage
      • Submission Revision Stage
      • Post Submission Stage
      • Post Decision Stage
      • Ethics Review Stage
    • Default Forms
      • Default Submission Form
      • Default Registration Form
      • Default Comment Form
      • Default Review Form
      • Default Rebuttal Form
      • Default Meta Review Form
      • Default Decision Form
      • Default Decision Notification
      • Default Ethics Review Form
    • OpenReview TeX
      • Common Issues with LaTeX Code Display
      • OpenReview TeX support
    • Mental Model on Blind Submissions and Revisions
Powered by GitBook
On this page
  • Value Pattern or Validation Specifier
  • Value Range or Domain Specifiers
  • Field attributes
  • Representation specifiers

Was this helpful?

Export as PDF
  1. Reference
  2. API V2
  3. Entities
  4. Invitation

Specifiers

There are two types of specifiers: Validation Specifiers and Representation Specifiers.

Both specifiers are defined inside the param field. The Representation Specifiers define how the requested data will be rendered in the UI.

Validation Specifiers define valid values that can be assigned to the fields of the objects (Edits, Edges, Tags) that will be created using the Invitation. Most Validation Specifiers outside content do not require a type field since its type can be inferred from the field name. For instance, the readers field will always be of type group[]. However, other Validation Specifiers need to be defined for the readers field, such as regex.

Only the mandatory fields head and tail in an Edge must specify the field type even though they are not inside a content field.

Validation Specifiers inside the content field will always require a type. This is because content can have any field name and its valid values need to be specified so that the UI can render them and the backend validate them.

In most cases, you can only have one Validation Specifier. The only exceptions are value range specifiers, such as minimum, maximum, minLength and maxLength.

There is no restriction on the amount of Representation Specifiers.

Value Pattern or Validation Specifier

The param should have one of the following specifiers needed to validate the expected value.

Whenever param is present, the user is expected to input a value for that field. Otherwise, the value is considered to be a constant and the user cannot change it.

  • const: restricts the value of the field to whatever it's written in const. When const is present, the user does not need to pass the value, since it will be assigned by the Invitation. In the example below, the title value has to be exactly "This is a title".

"title": {
    "value": {
        "param": {
            "type": "string",
            "const": "This is a title"
        }
    }
}

It is preferred to use the following shorthand whenever possible, which is the same as the example above:

"title": { "value": "This is a title" }

Fields that are specified as constants in the Invitation do NOT need to be passed when creating an object (Note Edit, Invitation Edit, Group Edit, Tag, or Edge).

  • enum: valid values must be listed in the enum array. The values in enum must be of the same type and should match the type specified in the field type. When the type of enum is string its values can be regexes too. In the example below "This is a title" and "This issss b regex" are valid values for title.

"title": {
    "value": {
        "param": {
            "type": "string",
            "enum": ["This is a title", "This is{3,5} [a|b] regex"]
        }
    }
}
  • items: valid values must be listed in the items array. Notice that each item is an object that must contain the field value or prefix and optionally the fields optional and description. The values in items must be of the same type and should match the type specified in the field type. In the example below the title field will be an array of strings as indicated by its type.

    "title": {
        "value": {
            "param": {
                "type": "string[]",
                "items": [
                  { "value": "title 1", "description": "This is the first title" },
                  { "value": "title 2", "description": "This is the second title" },
                ],
            },
        }
    }
  • regex: valid values need to match the specified regex. The type of the field should be string. In the example below, "This asdf title" is a valid value.

The maximum allowed value for repetition size is 1000. For example, the following regular expression is invalid "[A-Za-z0-9]{0,1001}".

Having a large repetition size in regular expressions can potentially lead to performance issues, security vulnerabilities, or even crashes in certain cases. This is because regular expressions with large repetition sizes can be computationally expensive to process, especially when applied to long input strings. The engine needs to check all possible combinations within the specified repetition range, which can result in significant slowdowns and resource consumption. This can lead to poor application performance and sluggish user experiences.

"title": {
    "value": {
        "param": {
            "type": "string",
            "regex": "^This .+ title$"
        }
    }
}
  • range: valid values are within the range [a, b] which means a <= value <= b. The type of the field must be float or integer. In the example below, the values 0, 3, and 10 are valid.

"grade": {
    "value": {
        "param": {
            "type": "integer",
            "range": [0, 10]
        }
    }
}
  • withInvitation: valid values are IDs of entities that have as invitation the value specified in withInvitation.

Suppose we have the following note (this is a simplified version of the note):

{
    "id": "or2022",
    "invitations": ["OpenReview.net/2022/Conference/-/Submission"],
    "readers": ["everyone"],
    "writers": ["~Some_Author1"],
    "signatures": ["~Some_Author1"],
    "content": {
        "title": {
            "value": "Title"
        }
    }
}

Given the example below, the value "or2022" would be valid.

"id": {
    "param": {
        "withInvitation": "OpenReview.net/2022/Conference/-/Submission"
    }
}
  • withVenueid: valid values are IDs of entities that have as venueid the value specified in withVenueid.

Suppose we have the following note (this is a simplified version of the note):

{
    "id": "or2023",
    "invitations": ["OpenReview.net/2023/Conference/-/Submission"],
    "readers": ["everyone"],
    "writers": ["~Some_Author1"],
    "signatures": ["~Some_Author1"],
    "content": {
        "title": {
            "value": "Title"
        },
        "venueid": {
            "value": "OpenReview.net/2023/Conference"
        }
    }
}

Given the example below, the value "or2023" would be valid.

"id": {
    "param": {
        "withVenueid": "OpenReview.net/2023/Conference"
    }
}
  • withForum: valid values are IDs of entities that have as forum the value specified in withForum.

Suppose we have the following note (this is a simplified version of the note):

{
    "id": "com2024",
    "invitations": ["OpenReview.net/2024/Conference/-/Submission"],
    "readers": ["everyone"],
    "writers": ["~Some_Author1"],
    "signatures": ["~Some_Author1"],
    "forum": "or2024",
    "content": {
        "title": {
            "value": "Title"
        }
    }
}

Given the example below, the value "com2024" would be valid.

"id": {
    "param": {
        "withForum": "or2024"
    }
}
  • inGroup: valid values are group IDs that are members of the group ID specified in inGroup.

Suppose we have the following group (this is a simplified version of the group):

{
    "id": "OpenReview.net/2022/Conference/Reviewers",
    "readers": ["OpenReview/2022/Conference"],
    "writers": ["OpenReview/2022/Conference"],
    "signatures": ["OpenReview/2022/Conference"],
    "members": ["~Some_Reviewer1", "~Some_Reviewer2"]
}

Given the example below, the value "~Some_Reviewer1" would be valid.

"tail": {
    "param": {
        "type": "profile",
        "inGroup": "OpenReview.net/2022/Conference/Reviewers"
    }
}

Value Range or Domain Specifiers

  • minLength: valid values are strings that have at least the length specified in minLength. In the example below, the values "title" and "long title" are valid.

"some_field": {
    "param": {
        "type": "string",
        "minLength": 5
    }
}
  • maxLength: valid values are strings that have at most the length specified in maxLength. In the example below, the values "title" and "abc" are valid.

"some_field": {
    "param": {
        "type": "string",
        "maxLength": 5
    }
}
  • maxSize: specifies the maximum size that a file can have in megabytes. It must be used with type file. In the example below

  • extensions: specifies the valid types with the extension of the file. The extensions cannot have dots, which means that files that have the extension tar.gz are not supported.

"supplemtary_material": {
    "param": {
        "type": "file",
        "maxSize": 5,
        "extensions": ["pdf", "zip"]
    }
}
  • minimum: valid values are numbers (floats or integers) that have at least the value specified in minimum. In the example below, 1 and 6000 are valid values.

"some_number": {
    "param": {
        "type": "integer",
        "minimum": 1
    }
}
  • maximum: valid values are numbers (floats or integers) that have at most the value specified in maximum. In the example below, 20.5 and 16.7 are valid values.

"some_number": {
    "param": {
        "type": "float",
        "maximum": 20.5
    }
}

Field attributes

  • type: define the type value that field is going to have.

The field type is mandatory in all the fields whose type cannot be determined by the name of the field. Note how in the example below the field invitation/edit/signatures defines param/regex without a type keyword. This is because we know the type of the field signatures. We know is of type groups[], so there is no need to define it. On the other hand, the field invitation/edit/note/content/title/value has to define the type since the fields inside content can be anything.

"invitation": {
    "id": "OpenReview.net/2022/Conference/-/Submission",
    "signatures": [ "~Super_User1" ],
    "readers": [ "~" ],
    "writers": [ "groupId" ],
    "invitees": [ "~" ],
    "edit": {
        "signatures": { "param": { "regex": ".*" } },
        "readers": [ "groupId" ],
        "writers": [ "groupId" ],
        "note": {
            "signatures": [ "~Super_User1" ],
            "readers": [ "groupId" ],
            "writers": [ "groupId" ],
            "content": {
                "title": {
                    "value": { "param": { "type": "string", "regex": ".*" } }
                }
            }
        }
    }
}

Allowed types:

  1. string

  2. integer

  3. boolean

  4. date

  5. file

  6. profile

  7. group

  8. note

The field types profile, group, and note ask for an ID, not the entire object.

Square brackets can be appended to each type to expect an array of values, for example:

"keywords": {
    "type" "string[]"
}

Not all types can become arrays. The exceptions are date and file.

type can be an optional property in some cases where it is known which type is expected, for example: readers, writers and signatures of a note.

  • optional: boolean value that indicates if the value is optional or mandatory.

  • deletable: boolean value that indicates if the value can be deleted.

The fields optional and deletable can be used at the same time or omitted. Their behavior is defined below:

Optional
Deletable
Behavior

True

True

Field is optional and can be deleted

True

False

Field can be added but not deleted

False

True

Undefined

False

False

Field is mandatory and cannot be deleted

True

undefined

Field can be added but not deleted

undefined

True

Field is optional and can be deleted

undefined

undefined

Field is mandatory and cannot be deleted

Representation specifiers

Representation specifiers are not used for validation, they are only used to correctly display the data in the UI. They are not mandatory and in case they are not present the UI will have default values for each field.

  • order: define the order in which the field is going to be rendered in the form.

  • description: short text to explain the expected value.

  • input: type of widget to use when requesting data.

    • text

    • select

    • checkbox

    • textarea

    • radio

  • default: value that is going to be shown when the widget is initialized.

  • markdown: boolean value that enables the use of markdown.

  • scroll: boolean value that specifies whether a textarea input will be scrollable.

PreviousFieldsNextDollar Sign Notation

Last updated 3 months ago

Was this helpful?