Add gru component

[yuuee-www]

What does this PR do?

This PR introduces the integration of the Gradient Rectified Unlearning (GRU) with the Gradient Ascent (GA) unlearning objective. Currently, we are providing the GRU with the GA version due to time constraints. Integrations with other unlearning objectives, such as NPO, SimNPO, WGA, and GD, are planned and will be added in subsequent updates as soon as possible. For more detailed information, please refer to the original paper on arXiv.

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Have you gone through the documentation of adding new components?
• Did you make sure to update the documentation with your changes? Here are the pointers to documentation
  documentation guidelines.

Sample Implementation and Some Reproduction Results

We run some examples following the common hyperparameters in docs/evaluation.md for reproducibility purposes. As DeepSpeed does not support gradient manipulation currently, it is not used in our implementation. An example command for executing the GRU on forget10 is as follows::

CUDA_VISIBLE_DEVICES=0 python src/train.py --config-name=unlearn.yaml experiment=unlearn/tofu/default forget_split=forget10 retain_split=retain90 trainer=GRU task_name=gru_forget10

The reproducible results on the Llama-3.2-1B-Instruct architecture:

TOFU unlearning on the Llama-3.2-1B-Instruct architecture

Method	forget01			forget05			forget10
	forget_quality	model_utility	forget_truth_ratio	forget_quality	model_utility	forget_truth_ratio	forget_quality	model_utility	forget_truth_ratio
Finetuned	0.01	0.60	0.47	2.96e-13	0.6	0.47	8.08e-22	0.6	0.48
Retain	1.0	0.60	0.65	1.0	0.6	0.63	1.0	0.59	0.63
GradAscent	0.27	0.33	0.59	1.94e-119	0	2.52e-23	1.06e-239	0	2.25e-18
GradDiff	0.77	0.43	0.57	1.94e-119	0.53	3.87e-34	1.06e-239	0.49	3.53e-27
IdkDPO	0.01	0.51	0.60	1.12e-05	0.07	0.62	4.64e-12	0.23	0.6
NPO	0.92	0.56	0.66	0.14	0.45	0.7	0.02	0.46	0.7
SimNPO	0.58	0.46	0.55	5.01e-100	0.58	4.19e-03	2.47e-203	0.54	1.07e-05
RMU	0.16	0.55	0.70	4.87e-10	0.58	0.77	3.15e-15	0.59	0.76
GradAscent w/ GRU	0.03	0.59	0.50	1.39e-06	0.49	0.52	6.83e-09	0.54	0.55

[molereddy]

can you clean up the code - e.g. remove this print and any other unnecessary prints/comments from the code?

[yuuee-www]

Thanks for the feedback! I've removed unnecessary print statements. Let me know if anything else needs updating!

[rtx-1999]

Hi!

Sorry for jumping into the conversation, and really interesting method by the way! I tried to reproduce the TOFU forget10 results of GRU on the Llama-3.2-1B-Instruct, following the example command you had above, but the result doesn't seem to match. I ran twice but both time the model_utility is about 0.364 (table shows 0.54), and forget_truth_ratio is about 0.512 (table shows 0.55). I also ran src/eval.py following the same default settings, and the two metrics gave the same results as above.

I also ran the same configurations with RMU, and its model_utility is about 0.574, which is close to the table (0.59), but the forget_truth_ratio is 0.588, which is actually worse.

So I'm just wondering what your exact evaluation settings were and if you could double-check the settings, especially regarding model utility, which seems to be the main argument for GRU.

[yuuee-www]

Hi!

Sorry for jumping into the conversation, and really interesting method by the way! I tried to reproduce the TOFU forget10 results of GRU on the Llama-3.2-1B-Instruct, following the example command you had above, but the result doesn't seem to match. I ran twice but both time the model_utility is about 0.364 (table shows 0.54), and forget_truth_ratio is about 0.512 (table shows 0.55). I also ran src/eval.py following the same default settings, and the two metrics gave the same results as above.

I also ran the same configurations with RMU, and its model_utility is about 0.574, which is close to the table (0.59), but the forget_truth_ratio is 0.588, which is actually worse.

So I'm just wondering what your exact evaluation settings were and if you could double-check the settings, especially regarding model utility, which seems to be the main argument for GRU.

Hi @rtx-1999,

Thank you for your interest in our method and for taking the time to test it out! We appreciate your efforts. Let me address your points:

We’ve uploaded an eval output JSON log file
TOFU_EVAL.json
that matches the results reported in our table (0.54 for model_utility and 0.55 for forget_truth_ratio). If possible, could you share the specific evaluation and unlearning settings you used or even upload your own eval JSON log file? This would help us figure out where the issue might lie.

Regarding the RMU results you mentioned, we’d like to kindly clarify that the numbers in the table were directly taken from the original table in results.md. We didn’t conduct any additional evaluations for RMU ourselves. The only row we added to the table was for GradAscent w/ GRU, which reflects the results from our work.

Finally, we want to note that the results in our table were produced using the common hyperparameters provided by the open-unlearning authors in results.md. These were used for reproducibility purposes, and the table isn’t intended for direct comparison between methods, as the authors themselves have stated.

Once again, we truly appreciate your interest in our work and your efforts to test it. Let us know if there's anything more we can assist with!

[rtx-1999]

Hi @rtx-1999,

Thank you for your interest in our method and for taking the time to test it out! We appreciate your efforts. Let me address your points:

We’ve uploaded an eval output JSON log file TOFU_EVAL.json that matches the results reported in our table (0.54 for model_utility and 0.55 for forget_truth_ratio). If possible, could you share the specific evaluation and unlearning settings you used or even upload your own eval JSON log file? This would help us figure out where the issue might lie.

Regarding the RMU results you mentioned, we’d like to kindly clarify that the numbers in the table were directly taken from the original table in results.md. We didn’t conduct any additional evaluations for RMU ourselves. The only row we added to the table was for GradAscent w/ GRU, which reflects the results from our work.

Finally, we want to note that the results in our table were produced using the common hyperparameters provided by the open-unlearning authors in results.md. These were used for reproducibility purposes, and the table isn’t intended for direct comparison between methods, as the authors themselves have stated.

Once again, we truly appreciate your interest in our work and your efforts to test it. Let us know if there's anything more we can assist with!

Thanks for your response and clarification on RMU results! Just wanted to clarify that making direct comparison between methods wasn't my purpose. I'm just trying to make sure that I have the same evaluation setup as yours.

Here are the hydra configs, my evaluation script, eval.log, TOFU_EVAL.json, GRU.log and trainer_state.json, zipped in settings.zip as the yaml format is not supported for attachment here. As I checked, I used the exact setting for unlearning:

CUDA_VISIBLE_DEVICES=0 python src/train.py --config-name=unlearn.yaml experiment=unlearn/tofu/default forget_split=forget10 retain_split=retain90 trainer=GRU task_name=GRU_forget10

And for the evaluation script, I followed the default configs in tofu_unlearn.sh, equivalently:

CUDA_VISIBLE_DEVICES=0 python src/eval.py \
experiment=eval/tofu/default.yaml \
forget_split=forget10 \
model=Llama-3.2-1B-Instruct \
task_name=GRU_forget10 \
model.model_args.pretrained_model_name_or_path=saves/unlearn/GRU_forget10 \
paths.output_dir=saves/unlearn/GRU_forget10/evals \
retain_logs_path=saves/eval/tofu_Llama-3.2-1B-Instruct_retain90/TOFU_EVAL.json

[Dornavineeth]

CUDA_VISIBLE_DEVICES=0 python src/train.py --config-name=unlearn.yaml experiment=unlearn/tofu/default forget_split=forget10 retain_split=retain90 trainer=GRU task_name=GRU_forget10

@rtx-1999 I see that you are using only 1 device for unlearning. Using different number of GPU can effect the results quite a lot as the effective hyper parameters such as batch size etc are changed.

Additionally, even if we manage to keep the effective hyper params same, using deepspeed can lead to different results.

@yuuee-www can you confirm how many GPUs have you used for unlearning?

[Dornavineeth]

Hi @yuuee-www,

Thank you for taking the time to add your method to our repository. We've added guidelines on contributing to our repository, including how to share your settings, results, and reproducible scripts. Could you review them and update this PR accordingly?

This should help others use and reproduce your work in their studies.

Quick summary of the contribution guide:

1. Create a folder under community/methods/.
2. Include your method details, results, hyperparameter search scripts, and reproducibility steps.
3. Update the leaderboard with your results.

We've provided a template folder to get you started.

Let us know if you have any questions!

[yuuee-www]

Hi @yuuee-www,

Thank you for taking the time to add your method to our repository. We've added guidelines on contributing to our repository, including how to share your settings, results, and reproducible scripts. Could you review them and update this PR accordingly?

This should help others use and reproduce your work in their studies.

Quick summary of the contribution guide:

1. Create a folder under community/methods/.
2. Include your method details, results, hyperparameter search scripts, and reproducibility steps.
3. Update the leaderboard with your results.

We've provided a template folder to get you started.

Let us know if you have any questions!

Hi @Dornavineeth,

Understood. We’ll follow the guidelines and update the PR soon.

Cc @rtx-1999.

Thanks!

[molereddy]

A couple of requests:

1. Can you mark out the trainer code so that it is clearer which of the code is your changes and which is from the HuggingFace source code? There are some comments indicating that, but it is not totally clear. Also, to the extent convenient, add comments that explain what is going on with your additions and clean up the code.
2. Given GRU can add onto many unlearning losses, it would be better if you could implement this in a more extensible way. You may introduce a forget_loss_type in the trainer config's method_args (like the retain_loss_type) so that users can vary the underlying forget loss with a config attribute. This can take values like NPO, DPO, GradAscent, SimNPO etc. You may use the trainer utils file to get these functionalities (and refer other trainer class implementations for their usage). In your GRU trainer you can now have if conditions on forget_loss_type that use different loss functions depending on the config.

For your leaderboard results, you may report your best-performing forget_loss_type with GRU.

If you have a better way of doing this, or face difficulties with this plan, do let us know.

[molereddy]

I see that ruff quality tests are failing here. Make sure to follow the instructions in https://github.com/locuslab/open-unlearning/blob/main/docs/contributing.md#create-a-pull-request to ensure you are in dev env and can apply ruff formatting.

[molereddy]

clean code up

[molereddy]

clean code up

[molereddy]

clean up

[molereddy]

you can

• remove all the template instructions from this doc which we've written ("provide a concise summary", etc.)
• add your results here

[molereddy]

We can get this in once:

• the few small nit comments are resolved
• synced with remote origin's latest changes
• code is tested
• tests pass

[molereddy]

Also, we use DeepSpeed for distributed training in our code and experiments, and provide documentation with that in mind. Please test if your code works with DeepSpeed and if not (looking at the code it seems it may not), add a note to your documentation describing in what your method works.

If DeepSpeed is not supported, mention that