Comments with LLM-generated fixes when issues are already verified and closed

[evverx]

I saw https://issues.oss-fuzz.com/issues/388905046#comment3 the other day

Hi, here is a potential patch generated by the LLM agent...

and I'm kind of curious what is going on. The issue was closed on Feb 25 and the comment was posted on Mar 9.

[maflcko]

The issue tracker is fully public, so I guess it is expected that there is some level of spam on issues once they are fixed or made public after the deadline. If it happens rarely, it is probably best to just ignore.

[evverx]

I don't know how often it happens but to judge from https://issues.oss-fuzz.com/issues?q=coderover there are 18 issues with comments like that and to judge from things like

We'll try to offer more suggestions for 0-day vulnerability next.
We'll try to contribute to 0-day vulnerability next.

it appears the intention is to keep posting that stuff. My guess would be that it's part of some research so it's probably possible for OSS-Fuzz to influence the direction.

[evverx]

Looks like one patch was sent to the assimp project in the end (assimp/assimp#6055) so it looks a little less spammy now. Given that I haven't seen comments like that recently I think this issue can be closed.

[yuntongzhang]

Hi @evverx, I am part of the research team who are developing an automated vulnerability remediation agent. We are currently working with the OSS-Fuzz team to provide experimental patch suggestions to security vulnerabilities. An initial report of the research can be found here: https://arxiv.org/pdf/2411.03346

Recently we have sent several patches to the issues that are still open on the issue tracker. We have checked that these patches can fix the exploit input. We have also manually examined the patches and only sent those that we think can fix the vulnerability.

Regarding https://issues.oss-fuzz.com/issues/388905046#comment3: there was a time window between when we collected the open issues and when we posted the fix as comment, and in this case, the issue has been closed during this period. I apologize for this oversight and going forward we will check the issue status again before sending a patch.

Regarding your comment about posting patches on the OSS-Fuzz issue tracker vs. sending them directly to each project: Initially, we aimed to minimize disruption, so we posted patches as comments on the issue tracker rather than opening PRs in each project. If the project maintainers think that directly opening PRs would be more efficient, we would be happy to proceed with that approach instead. Ultimately, the goal is to reduce the workload on developers and maintainers when fixing the reported vulnerabilities.