Community Review Process
This page provides information regarding Community Reviewer process and information on how to find allocations in the Ideascale
New Community Reviewers (LV0)
Community reviewers must register, complete a required identity verification process and carry out reviews using IdeaScale.
Community reviewers will receive an email confirming their registration has been accepted and must read through the guidance documentation provided.
Community Reviewers will log into Ideascale and carry out their reviews, providing scores and reasons for giving those scores using the scoring rubric supplied.
LV0 and LV1 Reviewers can review proposals from any Fund categories; however, Catalyst suggests that opinions are most valuable when reviewing proposals from areas and subject matter that the Community Reviewer is familiar with.
LV0 and LV1 Reviewers will be provided an allocation of proposals to review. Reviewers can choose to review outside of the allocation however, to minimize risks of gaming, non-allocated proposal reviews will have a lower weighted score compared to allocated reviews. (More details in the Score Weight section).
Once a review is submitted, it will pass through Catalyst scripts which analyze the review for: ‘length,’ ‘similarity,’ ‘profanity,’ and ‘use of AI’ (details can be found in the Catalyst flags section).
All flagged reviews will be randomly allocated to at least 2 LV2 Community Moderators for analysis. Community Moderators will determine whether a flagged review will be published or discarded.
Once voting has concluded, the reward calculations for all reviewers (LV0-LV2) are prepared.
Community Reviewer rewards will be distributed as soon as possible.
Experienced Community Reviewers (LV1)
Experienced Community Reviewers must follow the same guidance and steps as LV0 Community Reviewers, noting that after Step 3 above, LV1 community reviewers will have the following additional steps.
Catalyst will generate a random allocation of proposals to be reviewed for eligible LV1 and LV0 reviewers. Allocations are private and sent to reviewers securely via email. Reviews of allocated proposals will be rewarded at a fixed rate per review for LV1 and at a variable rate per review for LV0, determined by the number of reviewers participating and the total reviews published.
LV1 Community reviewer allocations are based on the category preferences selected during registration. While LV1s can review any proposal, the higher rewards will be paid for reviewing proposals identified in the allocation.
When an LV1 reviewer chooses to review proposals outside of the prescribed allocation, the review will still be counted (subject to passing the flagging and LV2 stage), and rewards for these additional reviewers will be counted as part of the LV0 pot of rewards.
An LV1 reviewer may perform more than the allocation of 80 reviews. The rewards for each review above and beyond the allocation will be paid as part of the LV0 pool of reviews. Those excess reviews will also carry a lower weight in the score as outlined in the score weight formula section.
In Fund13, the first set of allocations for LV1 Reviewers included proposals outside of the category preferences chosen during registration. For this reason, another set of allocation that satisfies the category preferences was generated for affected LV1 Reviewers. Every review that was allocated to LV1 (in the old batch and in the new one) will still be considered and rewarded as LV1 review. The maximum total of 80 will still be in place and the excess will be trated as LV0 review as per original guideline.
How to find allocations in the Ideascale?
Last updated