I'm not going to litigate the specifics of this situation, but there are some critical lessons here for people who are thinking of running a labeler (and to some extent they're the lessons of T&S in general, but they are even more important given the paradigm of composable moderation).
This thread covers the two fundamental things all labelers need to decide on up front and stick to:
1) Who is doing the moderation, what are their biases, and how are those biases mitigated?
2) Are you moderating/labeling objective actions/content, or subjective characteristics?
Each of these two points have a lot (and I mean A LOT) of nuance. (Like everything having to do with T&S!) Let's start with #1: bias mitigation. People who oppose community-driven moderation are now smugly parading around going "of course anyone who wants to be a mod is biased!"
This is the wrong way to look at it. It's not an inherent problem with community moderation: it's an inherent problem with people. Everyone is biased, in a million different ways. We all have our viewpoints of what we think is good vs bad.
Elon Musk thinks the word "cis" is a slur and should be moderated: that's a bias. I think people who create accounts only to advertise things are spammers and should be moderated: bias. You may think associating a wallet name with an account name is doxing and should be moderated: bias. Etc.
T&S, inherently, is a biased process: it involves someone's definitions of what should and shouldn't be actioned. There is no such thing as neutral, unbiased moderation. Anyone who says otherwise is simply asserting societal prejudices that are declared "objective" because of who holds them.
And, crucially, people don't want moderation to be "unbiased", or to fall back solely on externalities such as "is this content legal". Don't believe me? Look at the months-long Discourse on child safety: most of the content many people very loudly want removed is legal under US law.
What people are calling "bias" here, me included (because it's shorter), is actually better termed "viewpoint". Moderation is a function of viewpoint. You choose a viewpoint lens through which to moderate and apply it to your policies and actions.
The neat thing about Bluesky's experiment in composable moderation (which, as everyone who's been following me for ages knows, I am still dubious about the long term likelihood of success of, but this is *not* the reason why) is that you can pick which viewpoint you want to view the site through.
What people starting up labelers are going to have to do, though, is work out how to ensure the agents doing the work to action reports are going to apply *the labeling service's* viewpoint and not their own. This is an incredibly, incredibly difficult problem.
The fundamental tension here: a labeler with a strong viewpoint built from the (actual or perceived) consensus of a specific group as to what should be moderated will naturally want to draw its agents from members of that group, who have a familiarity with the group's social norms and practices.
This allows contextual interpretation of reported content. Failures of cultural competency result in problems where the members of the group can easily understand why a post should be moderated, but an outsider has no idea and thinks the post is inocuous. This happens *all the time*.
However, members of the group, who will have social connections within the group and have already formed opinions and reads on people in the group, will, always, need to compensate for the human tendency to read charitably when you agree with/like the speaker and uncharitably when you don't.
Let me be very clear here: this is not an individual failing of any specific person. It's fundamental human nature. You can compensate for it when you know the tendency exists, but you can never eliminate it. I do it. You do it. Every moderator ever has done it.
There are various process methods a team manager can use to compensate for it, over and above the methods individuals can use. People commonly propose a double-agree system, where two people have to sign off on an action. That can help, but is deeply impractical at any kind of volume.
You can do escalating levels of agreement needed for more severe actions; this might look like "single agree for labeling a post, double agree for labeling an entire account". But there are problems with that, too! First: 99.9% of whole account actions will be completely uncontroversial.
So you're *still* increasing your workload for no reason. Second, since 99.9% of actions are uncontroversial, the person doing the check is going to be strongly inclined to agree with the first person on the .1% too, because they're used to most of their decisions being right.
Our brains are, fundamentally, bad at spotting .1% events. Again: this is fundamental human nature! There's only so far you can process your way out of it. Third, if everyone on the team comes from the same group, they're more likely to have the same predispositions to read charitably/uncharitably.
You can set a policy that agents should not act on reports where they already had a pre-existing opinion on one party to the conflict. That gets you closer to fixing the problem, but it, too, has issues: for one, it's entirely self-reported and relies on agents being honest about recusal.
And most people *are* honest about this! But if you have a lot of undone work and very few people doing it, the agent who has the energy to slam through a ton of reports will want to make things easy on their coworkers and reason with themselves "well, it wasn't a *strong* opinion..."
Or "but this is a *really obvious* violation" or "but we're getting so many reports about this and if we don't do something they're just going to keep piling up" or "people are demanding to know why we haven't Done Something..."
No matter how much you emphasize that the "don't act on a case where you have a pre-formed opinion" is a hard and fast rule, there will always be a point at which someone thinks they have a good reason to bend it. Often it is a very good reason! And it usually turns out fine! Until it doesn't.
Two: a ton of reports come in about people who have had past reports. Sometimes lots of past reports. Sometimes involving a long running conflict between two people that requires deep understanding of the underlying conflict.
(We occasionally see one of those long running conflicts flare up again on DW from, I shit you not, 2002. My old LJAT folks will know exactly who I mean if I say that it involves a person with the initials KD.)
Handling those conflicts requires expertise about that backstory, usually from someone who handled an earlier case. Under a pure "no cases where you already have an opinion", anyone on a team could only handle one report from a long-running feud and then they'd be conflicted out forevermore.
This is a bad and counterproductive standard! And just knowing "one of my comods took/didn't take action against this person in the past" is technically enough to make most agents form an opinion anyway, so even a one and done policy isn't enough to fully mitigate this problem!
Which leads me to three: sometimes, and especially when your team is mostly recruited from the same group, there is no person on the team who hasn't already formed an opinion about one party involved. Often not even the same opinion or the same person!
But there are a lot of reports about people within a community where everyone from that community is conflicted out because that person is just *that* well-known. It happens with very famous people (ie the Trump Twitter suspend) and well-connected people (ie highly socially networked people).
So what do you do? One option is to require unanimity from the entire team even though they're all conflicted out: that covers "we all have an opinion but they're all different opinions". For "we all hate this person", less so.
That is gonna require an outside opinion from an uninvolved person. But! Either the uninvolved person is not culturally competent, in which case they may miss things, or they are, and they have opinions of their own.