How to become a Community Reviewer?

This page explains the steps and key information to join the Community Review stage

LV0 - New Community Reviewers

• Community reviewers must register and carry out reviews using Ideascale.

• Community reviewers will be asked to specify which challenges they would prefer to work on

• Community reviewers will receive an email confirming their registration has been accepted and must read through the guidance documentation provided

• Community Reviewers will login into Ideascale and carry out their reviews providing scores and reasons for giving those scores, using the scoring rubric supplied

• LV0 and LV1 Reviewers can review proposal from any challenge, however Catalyst suggests that opinions are most valuable when reviewing proposals from areas and subject matter that the Community Reviewer is familiar with

• LV0 and LV1 Reviewers will be provided an allocation of proposals to review. Reviewers can choose to review outside of the allocation, however to minimize risks of gaming, non-allocated proposal reviews will have a lower weighted score compared to allocated reviews (More details in the Score Weight section)

• Once a review is submitted, it will pass through Catalyst scripts which analyze the review for: ‘length’, ‘similarity’, ‘profanity’ and ‘use of AI’ (details can be found in the Catalyst flags section)

• All flagged reviews will be randomly allocated to at least 2 LV2 Community Moderators for analysis. Community Moderators will determine whether a flagged review will be published or discarded.

• Once voting has concluded, the rewards calculations for all reviewers (LV0-LV2) are prepared

• Community Reviewer rewards will be distributed at the earliest possible opportunity

LV1 - Experienced Community Reviewers

Experienced Community Reviewers must follow the same guidance and steps as LV0 Community Reviewers noting that after Step3 above, LV1 community reviewers will have the following additional steps.

• Catalyst will generate a random allocation of proposals to be reviewed for eligible LV1 and LV0 reviewers. Allocations are private and sent to reviewers securely via email. Reviews of allocated proposals will be rewarded at a fixed rate per review for LV1 and at a variable rate per review for LV0 determined by the number of reviewers participating and total reviews published.

• LV1 Community reviewer allocations are based on the challenge preferences selected during the registration process. While LV1s are able to review any proposal, the higher rewards will be paid for the review of proposals identified in the allocation.

• When a LV1 reviewer chooses to review proposals outside of the prescribed allocation, the review will still be counted (subject to passing the flagging and LV2 stage) and rewards for these additional reviewers will be counted as part of the LV0 pot of rewards

An LV1 reviewer may perform more than the allocation of 80 reviews. The rewards for each review above and beyond the allocation will be paid as part of the LV0 pool of reviews. Those excess reviews will also carry a lower weight in score as outlined in the score weight formula section.