Community Moderation Process

In Fund14, the Community Moderation process is conducted through the Community Review Module, which has replaced the previous Moderation module. A user guide explaining how to use this new tool is available here to help you understand its features and functionality.

All reviews provided by the Community Reviewers will be subject to an automated review process using Catalyst Flags. Catalyst Flags* are pieces of code known as scripts that are run to analyze all reviews by the Community Reviewers. Catalyst scripts will raise flags in case of:

• Similarity ~ reviews which are identical or almost identical to a another review

• Profanity ~ reviews that contain profanities or abusive content or language

• Use of AI ~ reviews which do not reflect the opinion of the Community Reviewer or which are provided by others or an AI-agent acting on their behalf.

* Length requirements have already been applied for automatically (reviews containing less than 150 characters in impact, feasibility, and value for money criteria)

All reviews that have been flagged will go through a moderation process involving Community Moderators (LV2).

• Community Moderators will be randomly allocated flagged reviews for further assessment. In addition, between 1-5% percent of reviews not flagged by scripts will be allocated to LV2 Community Reviewers for a spot check (depending on the total reviews received).

• Community Moderators have to register in the Community Review module to perform moderation

• Community Moderators can see which flags have been raised by scripts about the review/reviewer.

• After reading the review and flags, Community Moderators will submit their moderations, marking reviews they believe were legitimately flagged and providing a reason for why the review should be removed. Eg. Where the flag is raised because of potential ChatGPT AI involvement or plagiarism, the Moderator must make a judgment whether they believe the review not to be a reviewer's own opinion. If the Moderator agrees with the flag, they mark it as REMOVE and give a rationale.

• Rationale should be a short written statement, and a selection must be made from a menu of reasons. For example, the moderator would select from options such as:

  • There is no evident issue with review

  • I believe this is not the person’s own opinion

  • There is profanity

  • There is too much similarity with another review

  • Rationale doesn’t correspond with the score given

  • Rationale contains a false statement

  • If Moderators disagree with the flag and believe the review should be included for voters to consider, they mark it as VALID and give a rationale.

  • Moderators should only discard reviews if there is a valid reason why a voter shouldn’t see that review. Unless the review has an obvious reason, such as the below flags or another obvious reason (spamming, for instance), then the review should be included.

• Moderators should only discard reviews if there is a valid reason why a voter shouldn’t see that review. Unless the review has an obvious reason, such as the below flags or another obvious reason (spamming, for instance), then the review should be included.

• If Moderators disagree with the flag and believe the review should be included for voters to consider, they mark it as VALID and give a rationale.

Rules to be followed by Community Moderators (LV2)

• All reviews should be provided to voters unless a Community Moderator has good reason not to include it.

• It is anticipated that Community Moderators will approve many flagged reviews and will be included for voters to consider.

• Moderators are there to provide a sanity check to verify if the flag(s) have raised a valid concern.

• Moderators should use their best judgment and provide a reason if they agree the flag is valid, meaning that the review should not be included.

• The majority of reviews will be displayed in the voting application, meaning only those legitimately flagged and tagged by LV2 Moderators will not be included.

• Simple majority forms consensus about including the review or not. In a scenario whereby 2 moderators (or an equal number of moderators on each side of the decision) do not form a consensus about including the review or not, then the benefit of the doubt will be given and the review included.

  • In a scenario where only 1 moderator checked a review, then the Catalyst will fulfill the second necessary moderation

  • In a scenario where flagged or spot-checked reviews go without moderation by either LV2 moderators allocated to check, then the Catalyst team will fulfill the moderation duty.