Twitter acquired Threader! Learn more

Sarah Jamie Lewis
+ Your AuthorsArchive @SarahJamieLewis Executive Director @OpenPriv. Cryptography and Privacy Researcher. @cwtch_im icyt7rvdsdci42h6si2ibtwucdmjrlcb2ezkecuagtquiiflbkxf2cqd Apr. 21, 2021 3 min read

I see that we are talking about "Hypocrite Commits" again and I want to clarify a few things.

Despite what their paper says they didn't get an IRB-exemption until *after* they posted about their IEEESP paper acceptance and a group of researchers (inc myself) expressed concern...

Our complaints were based on the abstract and a screenshot of the first page of the paper. They have since published the whole paper:

 https://raw.githubusercontent.com/QiushiWu/qiushiwu.github.io/main/papers/OpenSourceInsecurity.pdf 

They lied to people in order to assess their response, with no system in place for prior informed consent or debriefing.

That any IRB could conclude that it wasn't a deception study on human subjects speaks to the overall ability of many IRBs to reason about internet studies.

I also want to take a moment to point out the original wording of their abstract (in their screenshot IEEESP announcement) v.s. the paper published in that repository.

"successfully introduced multiple exploitable...vulnerabilities"

v.s.

"safely demonstrate it is practical"

Some people who have grown extremely cynical of academia, like myself, might classify the original wording of the abstract (accepted to IEEESP) - as "a lie intended to bolster the impact of the paper"

Quoting myself from a previous thread:

"What if people submit code that has bugs in it, and the maintainers don't catch it!.......

but intentionally"

To be fair to the researchers...the future research section basically writes itself...

Without controversial studies like this we may have never gotten great conclusions like "make contributors agree not to introduced bugs" and "verify everyone's identity which is definitely an effective mitigation against malicious behaviour"

They apparently learned nothing, seemingly conducted another round of experiments with more incorrect patches...

Got caught, and in the resulting fallout they blamed a new static analysis tool, and accused the maintainers of (bordering on) slander...

 https://lore.kernel.org/linux-nfs/YH%2FfM%[email protected]/#t 

> "but they still did demonstrate a flaw"

It was a known flaw, one practically every maintainer is aware of. The solution is safer languages with stronger security semantics coupled with automated testing and analysis tools. Initiatives that many people are actively working on.

Anyway this is the latest in a long line of computer science researchers stumbling into human subject research, disregarding any and all ethical considerations, getting a paper published, and leaving to find a new community fuck around in.

That this behaviour continues to be supported and even encouraged by university departments, institutional review boards and conference program committees demonstrates that this is an institutional problem permeating across academic computer science.

An update: The authors have decided to withdraw their paper. IEEE S&P "plan to publish a statement before the conference".

Update 2: Report by the Linux Foundation's Technical Advisory Board:  https://lwn.net/ml/linux-kernel/[email protected]/ 

"All patch submissions that were invalid were caught, or ignored...Our patch-review processes
worked as intended"

This raises more questions about the papers claims & acceptance.

The original claim of "successfully introduced multiple exploitable...vulnerabilities" was complete fiction.

Even the eventually watered down claim of "safely demonstrate it is practical" seems suspect given the TAB's analysis of the patches involved.

Update 3 to cap this thread off: Statement from IEEESS&P PC

 https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_Statement.pdf 

fwiw:

1) I'm incredibly skeptical of any separation of ethical review from technical review.

2) If reviewers end up placing a paper with as many technical and ethical flaws as this one as "top 5% of submitted papers" I'm left wondering what's the point of review at all.

Finally, kudos to the PC for a such a deeply honest retrospective.


You can follow @SarahJamieLewis.



Bookmark

____
Tip: mention @threader on a Twitter thread with the keyword “compile” to get a link to it.

Follow Threader