If the jailbreak is essentially saying “don’t worry, I’m asking for a friend / for my fanfic” then that isn’t a jailbreak, it is a hole in safeguarding protections, because the ask from society / a legal standpoint is to not expose children to material about self-harm, fictional or not.

This is still OpenAI doing the bare minimum and shrugging about it when, to the surprise of no-one, it doesn’t work.

Try it with lyrics and see if you can achieve the same. I don’t think "we’ve tried nothing and we’re all out of ideas!” is the appropriate attitude from LLM vendors here.

Sadly they’re learning from Facebook and TikTok who make huge profits from e.g. young girls swirling into self harm content and harming or, sometimes, killing themselves. Safeguarding is all lip service here and it’s setting the tone for treating our youth as disposable consumers.

Try and push a copyrighted song (not covered by their existing deals) though and oh boy, you got some splainin to do!

ChatGPT to a consumer isn’t just a LLM. It’s a software service like Twitter, Amazon, etc. and expectations around safeguarding don’t change because investors are gooey eyed about this particular bubbleware.

You can confirm this yourself by asking ChatGPT about things like song lyrics. If there are safeguards for the rich, why not for kids?

In those cases broadcasters take one of two roads:

1. Don’t broadcast it - many extreme sports are simply not broadcast by many, many broadcasters.

2. Properly mitigate the risk to an acceptable level - this is done frequently for sports and other media. This is the reason you can watch Jackass and Dirty Sanchez even though the risk of death for many stunts is non-zero.

Once the death occurs though, they can only rely on their demonstration of #2 here to offset legal culpability. They are also then generally bound to remove the material and not re-air (in this case, Kick did make the content available again for whatever reason)

It seems like this is the road the defense will take in this particular case is to prove the death (illegal to air if preventable) was not caused by the preceding consensual torture (legal to air, seemingly).

Yes, that is the law. You are required not to broadcast death and to create circumstances in which the likelihood of this is minimised.

That’s not calling for censorship because it doesn’t preclude a level of consensual harm that doesn’t lead to high risk of death.

As I said earlier, your point stands: it is not for these platforms to act as moral compasses for viewers of consensual but provocative content.

However, that’s irrelevant to the law which wants to avoid incentivising people dying / being killed on broadcast streams for a profit.

I think this is ratified by the fact that there will be less of a burden of blame on the service provider if this proves not to be the case

They aren’t deciding, they’re being held to laws that they didn’t create nor necessarily agree with.

I’d assume that, given the option, they’d like this kind of thing to be legal so they can continue making money from it legitimately

This is why we can’t have nice things

Because they profited from his torture and subsequent death?

To your point though, they aren’t responsible in the moral sense that you’re implying. However, they committed a crime when they platformed, promoted and profited from it.