But we've already established there is no public oversight over the contents of the NCMEC database and that there cannot ever be, by design. Furthermore, it's known to contain hashes of non-CSAE images simply because they were found in a CSAE-related context.

So how can this system guarantee civil freedom? How can it be guaranteed that it won't be exploited by the small number of people in power to actually inspect it and manipulate it?

Have we established that? Certainly not a reality I recognize. The processes involved in CSAM databases like NCMEC/ICSE are many years old. If it leads to widespread civil rights abuses, where are they?

Google Drive has 1bn users. Google scans content for CSAM already. Shouldn't we be seeing these negative side effects already?

Proponents of these systems can point to thousands upon thousands of actual "wins" (such as identifying teachers, police officers, sports coaches, judges, child minders etc who are pedophiles) and detractors cannot provide actual evidence of their theoretical disadvantages.

No system is perfect, no system "guarantees civil freedom", this is not a fair test. The actual evidence suggests automated scanning for CSAM is a net win for society.

“ detractors cannot provide actual evidence of their theoretical disadvantages.”

This is the concern. The system is hidden and you have no way to challenge. You are simply “trusting” that this system is working well

Who watches the watchers?

That’s the major concern I have: take as a given that NCMEC is on the side of the angels here, what happens when some government demands that Apple help identify anyone who has an image shared from a protest, leak, an underground political movement, etc.? The database is private by necessity, so there’s no way to audit for updates.

Now, currently this is only applied to iCloud photos which can already be scanned server side, making this seem like a likely step towards end to end encryption of iCloud photos but not a major change from a privacy perspective. What seems like more of a change would be if it extended to non-cloud photos or the iMessage nude detection scanner since those aren’t currently monitored, and in the latter case false positives become a major consideration if it tries to handle new content of this class as well.

> If it leads to widespread civil rights abuses, where are they?

This is disingenuous, since up to this point these databases weren't used in systems residing on end user devices, scanning their local files.

The disadvantages aren't merely "theoretical"; they just haven't materialized yet since the future does not yet exist. To ignore these obvious faults by hand-waving them as theoretical is to blatantly ignore valid concerns raised by thousands of informed citizens.

No system is perfect, but that doesn't mean there aren't some systems that simply go too far.

The distinction between scanning locally before upload, and on server after upload, is an implementation detail. The only reason Apple have done this is to allow them to implement E2EE for iCloud without hosting CSAM.

All arguments relating to anything other than this are arguing against something that doesn't exist.

I don't dispute that fictional proposals in the imaginations of HNers might pose a grave (fictional) threat to civil freedoms.