Okay. I am probably going to regret wading into this again, but The Discourse about child protection issues on Bluesky has escalated to the point where people are making extremely serious accusations against the head of Bluesky T&S, and y'all need to stop. It is unhinged QAnon batshittery.
I am now going to state a series of facts. I am neither agreeing nor disagreeing with any of these facts. These facts are phrased as neutrally and as clinically as I can manage to phrase them. Most of The Discourse has been incorrect about one or more of these facts.
1. United States law criminalizes any image or video that depicts the sexual abuse of an actual, identifiable minor. The law regarding this is 18 USC §2252. It only applies to image or video that depicts an actual, identifiable minor.
2. The material covered by 18 USC §2252 -- image and video that depicts an actual act of sex abuse against a specific, identifiable minor -- is the only material that is presumptively illegal under US law and the only material the term "CSAM", child sex abuse material, refers to.
3. Drawings, cartoons, paintings, sculptures, etc, that depict an apparent minor engaged in sexually explicit conduct are not presumptively illegal under US law. They are governed by 18 USC §1466A, and are only illegal if they meet the legal definition of "obscenity".
4. The US legal definition of "obscenity" is a three-prong test set forth in the Supreme Court case Miller v. California, 413 U.S. 15 (1973) en.m.wikipedia.org/wiki/Miller_...
5. Even drawn material that violates 18 USC §1466A on grounds of meeting the legal definition of "obscenity" is not CSAM. CSAM only refers to the memorialization of an actual act of child sexual abuse.
6. Both 18 USC §2252 (photorealistic images and video) and 18 USC §1466A (drawings, paintings, cartoons, and, likely, AI-generated images) have very specific definitions of what they cover and what constitutes "sexually explicit conduct".
7. The definition of "sexually explicit conduct" in US law is 18 USC §2256(2) and "identifiable minor" is 18 USC §2256(9). www.law.cornell.edu/uscode/text/...
8. 18 USC §2252A criminalizes the advertising or solicitation of material covered under 18 USC §2252 (photorealistic images) or §1466A (non-photorealistic images that are obscene).
9. Because the advertising or solicitation of this material is illegal, the people who do it have developed an extensive vocabulary for doing so in plausibly deniable ways. Conclusively determining that a specific reference is an offer or solicitation is often difficult.
11. The formal name of the mental disorder that involves a sexual attraction to minors in the DSM-V is "Pedophiliac Disorder". There is significant disagreement among mental health professionals who treat people with Pedophiliac Disorder about effective treatments and interventions.
12. It is impossible to ethically conduct effective empirical scientific research into the effectiveness of most interventions intended to prevent people with Pedophiliac Disorder from sexually abusing children. Any research science can do is imperfect and limited.
13. It is common for people and organizations to overstate the conclusions of the imperfect, limited scientific research that can be done in order to support their preferred interpretations and their desired policies.
14. This means that a service trying to make content policies involving child safety issues where the disposition is not obvious in US law, or where there is genuine dispute about the interpretation of the law, does not have a single resource to consult.
15. Making policies about content where the disposition is not obvious in US law, or where there's genuine dispute about the interpretation of the law, takes time. It is not immediate.
16. The head of T&S for a site is not the only person who has input into the content policy of the site and does not have sole discretion over how quickly or slowly a site creates or changes its content policies.
17. The head of T&S for a site is responsible for advising the people who decide on the content policy. This includes reading widely about multiple positions on multiple issues from multiple sources on every possible issue you can imagine.
18. Because the people who engage in policy advocacy are the people who have strong opinions about the topic they are advocating about, this often means reading the posts of terrible people with whom you deeply, fundamentally disagree.
19. Because the people who engage in policy advocacy are the people who have strong opinions the topic they are advocating about, they often present themselves as being more neutral, more objective, or more detached from the topic than they actually are.
20. This is why it is even more important for the people whose job it is to synthesize the advocacy on a particular topic to read widely and broadly in order to ensure they are not relying on biased sources (or that they're able to identify the bias in those sources).
21. Every single one of us who works in this job has at least one story of "I didn't realize how biased this person's advocacy was until I really dug into it". Sometimes we drop reading them when we realize; sometimes we keep them and apply a significant correction factor.
If you have read my posts for any length of time, you will know the one thing I repeatedly harp on is that this job fucking sucks. I have done it for 22 years. It fucking sucks. Everything that people think is simple and clear-cut actually has a million nuances and exceptions.
Child protection issues that do not fall into the category of "things easily identifiable as illegal" are a nightmare to deal with even when you're only dealing with one country's laws. Add more than one country and it gets even worse.
Bluesky is also operating with a significant penalty because they are making policy both for Bsky Dot App, the site you're reading this on, but also for the ATProto federated network. This elevates "complete nightmare" into the category of "the hardest professional challenge in the industry today".
T&S in a federated protocol is an entirely new challenge. ActivityPub (Mastodon et al) has dealt with it by devolving the responsibility entirely onto the individuals running individual nodes of the network. That approach has been, to put it charitably, a fucking disaster.
Bluesky is trying a different approach with offering multiple moderation services that people can subscribe to. Those of you who have been following me for a while know that I've been deeply interested in the outcome while also deeply concerned about some aspects of the approach.
This specific scenario is exactly (like, I mean, *the* exact scenario) what I have repeatedly been most concerned about the ability of the composable moderation system to properly handle. I have been very vocal about there being content that can't be handled by consensus or by third party labeling.
Bluesky the company is discovering the proof of this in real time (and has been for a while). It is going to take time for the people making the decisions -- who again, are not Aaron -- to come to terms with this fact.
I do not have any insight into what discussions are happening at Bluesky the company. However, because I have been in this industry for 22 years, I know that the job of head of T&S involves endless fights about everything from tiny details to huge fucking problems, and sometimes you lose them.
And when you lose them, you still have to turn around and take endless shit from users who don't see any of the fighting, just the outcome. And it fucking sucks. This job fucking sucks. It will never not suck.
Please do not contribute to the fucking suck of Aaron's job by falling into the trap of believing that he is the sole person who sets the policy he is executing or that failure to act on an issue yet means that the issue will never be decided.
Please assume every post in the thread was liked (actually going through and liking them all feels vaguely "overdoing it" to me for some reason).
The nexus of technology and humanity is deeply, profoundly complex, because people are not machines & do not behave in predictable ways.
Dealing with nuance & implications scales at a rate far faster than many folks can conceive, as a userbase grows; when you mix in groups who are convinced they alone are correct, and go all pitchforks-and-torches when their simplistic and usually-wrong beliefs aren't immediately acted on by TPTB...
...it makes the job insanely difficult.
That's without even taking into account the bad actors who are constantly looking to exploit the system in ways that the devs never even conceived of.
Social media is just so good at dealing with nuance. /s
...and the scale doesn't even need to be as large as people imagine.
"We're going to do this. It will only affect a few users."
"These are the most immediate consequences of that choice. It's a Really Bad Idea."
"You're catastrophising."
...
"OK, maybe you weren't catastrophising."
great thread, really appreciate you going into the facts of the law, and how that doesn't line up exactly with the vocal community's views (and beliefs as to what the law is), and how bridging that gap is a really hard problem, where no matter where you draw the line, someone will be unhappy.
There is without question a category of "material that is not illegal under US law that a specific site should still remove". Writing a policy that concretely and actionably separates it from "material that is not illegal under US law that a site should leave alone" is incredibly fucking difficult!
Thanks for educating people on the complexities involved. If I may ask, in your experience, how often/intimately are the general counsel's at SM companies (or those that can afford in-house counsel) involved in these discussions? How do small companies even navigate these issues?
Companies large enough to have in-house teams will usually have one specific person who specializes in content issues (and larger ones will have specific people for specific countries/regions/etc). Companies that can't afford in-house either consult on an as-needed basis, or Google and pray.
like I've said a few times, my site's policies are as concrete, specific, and actionable as 22 years of experience can get them, and we still run into another case of "the policy does not make what to do with this content immediately obvious" every few years.
Thank you for this. I wish that I was in possession of a magic wand that would make it so that people really understood how complex systems (tech, law, policy, dealing with CSAM, people ) are fucking hard, and create wicked, fractal problems without easy solutions.
*Sigh*
My misanthropy springs from the fact that most people live blissfully without this knowledge, and the rest of us are just permanently scarred by the Knowing.
Oh, my version is still so much less shit (privacy + tech + law) than yours, but complex nuance and almost zero easy answers being a daily thing still sucks ass.
I dislike this particular issue coming up constantly these days as it is triggering but I really appreciate that you bring your thoughtfulness and expertise and experience to the convo.
Thank you for (again) firmly grabbing that third rail and trying to talk sense instead of screaming incoherently through the smell of burning hair. I only hope it gets through to-
...ah, who am I kidding. Your mentions gonna be bad today 😥
