feat(docs): add half-precision training section in using_simulator docs #678
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PabloCarmona
added
documentation
jubueche
previously approved these changes
Aug 12, 2024
PabloCarmona
force-pushed
the
feat-677-half-precision-docs
branch
from
e017345 to
0877c23
Compare
|
@coreylammie @anu-pub Take a look at it and let me know what you think, thanks! |
|
Compilation is currently not working for the half precision type. To reproduce: Error: |
|
@maljoras I think this is something that you could maybe easily fix. Or you could tell me how I could resolve it. |
maljoras
reviewed
Aug 25, 2024
| dataset = datasets.MNIST("data", train=True, download=True, transform=transform) | ||
| train_loader = torch.utils.data.DataLoader(dataset, batch_size=32) | ||
|
|
||
| model = model.to(device=device, dtype=torch.bfloat16) |
There was a problem hiding this comment.
Does this also work with other tiles that TorchInference? I had added a HALF type, like in the above: rpu_config.runtime.data_type = RPUDataType.HALF
because the HALF could either be bfloat16 of float16 depending on the compilation options.
There was a problem hiding this comment.
There also a runtime.data_type.as_torch() or something function to convert it to the corresponding torch type, see https://github.com/IBM/aihwkit/blob/master/src/aihwkit/simulator/parameters/enums.py#L23
|
Yes, one has to enable the FP16 auto-conversion during the compilation if I remember correctly. You have to make sure that |
|
You could try to set additionally |
|
You might also want to use bfloat as default FP16 type (with |
|
Actually, could also be related to the rather old compiler you are using? One way to go about it might to change each conversion |
|
I suspect the compiler version to be the problem. Do you know what compiler version worked? On the system I using to check this it is not working. Regarding the other things you mentioned: I tried |
|
@jubueche have you tried a different compiler version? what is the status on this testing? |
|
Tested now in my local env with enabled CUDA GPU and CPU and both making the example working correctly. Please check if everything seems correct on the example and on the docs are clear, thanks! |
Signed-off-by: Pablo Carmona Gonzalez <pablocarmonagonzalez@gmail.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
Signed-off-by: Pablo Carmona Gonzalez <pablocarmonagonzalez@gmail.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* Draft fix * Remove DS Store * Ran mypy and pylint and passing * add ignore on unused ctx params temporarly for linting pass --------- Co-authored-by: Pablo Carmona Gonzalez <pablocarmonagonzalez@gmail.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* add release content for 0.9.2 Signed-off-by: Pablo Carmona <pablocarmonagonzalez@gmail.com> * remove added patch text Signed-off-by: Pablo Carmona <pablocarmonagonzalez@gmail.com> * modify changelog.md and version Signed-off-by: Pablo Carmona <pablocarmonagonzalez@gmail.com> * ufix(examples/31_custom_drift_models.py): change way rpu config is setup to a cleaner way Signed-off-by: Pablo Carmona <pablocarmonagonzalez@gmail.com> * fix(examples/31_custom_drift_models.py): fix errors with pycodestyle Signed-off-by: Pablo Carmona <pablocarmonagonzalez@gmail.com> * feat(notebooks): clean up and enhacenments on notebook contents for clarification and typos fixings * feat(converter/conductance): new conductance converters development * updates to custom drift model suggested by Malte (#690) Co-authored-by: Charles Mackin <charles.mackin@ibm.com> * Update hermes.py (#691) * fix(hermes.py): fix linting errors related to whitespacing * fix(requirements.txt): solve problem with numpy versions in some cases * feat(CHANGELOG.md): add PR number for new related developments and modify the date of release to a proper one * feat(CHANGELOG.md): add PR number for new related developments and modify the date of release to a proper one * feat(travis): update travis build with latest pytorch version --------- Signed-off-by: Pablo Carmona <pablocarmonagonzalez@gmail.com> Co-authored-by: charlesmackin <45808803+charlesmackin@users.noreply.github.com> Co-authored-by: Charles Mackin <charles.mackin@ibm.com> Co-authored-by: Athanasios Vasilopoulos <152185220+atvasilopoulos@users.noreply.github.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* add release content for 0.9.2 * feat(notebooks/tutorial): add notebook outputs for reference * fix(pcm.py): add a check to not get custom_drift_model as None * fix(travis/build): fix proper version for pytorch on mac osx wheel build * fix(travis/build): fix torch version to match compatibility with osx version Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* add release content for 0.9.2 * feat(notebooks/tutorial): add notebook outputs for reference * fix(pcm.py): add a check to not get custom_drift_model as None * fix(travis/build): fix proper version for pytorch on mac osx wheel build * fix(travis/build): fix torch version to match compatibility with osx version * feat(docs): update advance install docs to match different GPU versions Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* add release content for 0.9.2 * feat(notebooks/tutorial): add notebook outputs for reference * fix(pcm.py): add a check to not get custom_drift_model as None * fix(travis/build): fix proper version for pytorch on mac osx wheel build * fix(travis/build): fix torch version to match compatibility with osx version * feat(license): change LICENSE.txt and license references to MIT License * fix(array.py): add temporal ignore to bypass pylint * fix: pycodestyle errors Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* Initital changes. Signed-off-by: Julian Buechel <jub@zurich.ibm.com> * Executed notebook Signed-off-by: Julian Buechel <jub@zurich.ibm.com> --------- Signed-off-by: Julian Buechel <jub@zurich.ibm.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* passing code checkers Signed-off-by: Charles Mackin <charles.mackin@ibm.com> * minor updates Signed-off-by: Charles Mackin <charles.mackin@ibm.com> --------- Signed-off-by: Charles Mackin <charles.mackin@ibm.com> Co-authored-by: Charles Mackin <charles.mackin@ibm.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* passing code checkers Signed-off-by: Charles Mackin <charles.mackin@ibm.com> * minor updates Signed-off-by: Charles Mackin <charles.mackin@ibm.com> * updated examples README.md Signed-off-by: Charles Mackin <charles.mackin@ibm.com> --------- Signed-off-by: Charles Mackin <charles.mackin@ibm.com> Co-authored-by: Charles Mackin <charles.mackin@ibm.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
Signed-off-by: Julian Buechel <jub@zurich.ibm.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
* Added Notebook on Device Non_Idealities Signed-off-by: Ghazi Sarwat Syed <ghs@zurich.ibm.com> * Added the updated tutorial Signed-off-by: Ghazi Sarwat Syed <ghs@zurich.ibm.com> * Add files via upload Signed-off-by: Ghazi Sarwat Syed <ghs@zurich.ibm.com> * Cleanup of the notebooks directory. (#444) Signed-off-by: Ghazi Sarwat Syed <ghs@zurich.ibm.com> * fix typo (#445) Signed-off-by: Ghazi Sarwat Syed <ghs@zurich.ibm.com> * [ImgBot] Optimize images (#448) *Total -- 450.13kb -> 41.19kb (90.85%) /examples/img/replay_fake_images_gan.gif -- 423.50kb -> 24.00kb (94.33%) /notebooks/examples/imgs/xbar.png -- 26.62kb -> 17.19kb (35.44%) Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com> Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com> Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com> Signed-off-by: Ghazi Sarwat Syed <ghs@zurich.ibm.com> * for pr 437 * ghs-tutorial * changed to non executable code * change code to markdown * trying again with markdown changes --------- Signed-off-by: Ghazi Sarwat Syed <ghs@zurich.ibm.com> Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com> Co-authored-by: Kaoutar El Maghraoui <kaoutar.elmaghraoui@gmail.com> Co-authored-by: Malte J. Rasch <17587387+maljoras@users.noreply.github.com> Co-authored-by: imgbot[bot] <31301654+imgbot[bot]@users.noreply.github.com> Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com> Co-authored-by: Borja Godoy Gago <Borja.Godoy.Gago@gmail.com> Co-authored-by: pablocarmona <pablocarmonagonzalez@gmail.com> Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
Signed-off-by: Pablo Carmona González <pxg@Pablos-MacBook-Pro.local>
PabloCarmona
force-pushed
the
feat-677-half-precision-docs
branch
from
1b0197d to
fd3e438
Compare
|
@jubueche can you please check if this is ok to merge. The testing that Pablo did passed. |
@kaoutar55 the branch first needs to be updated by @PabloCarmona |
Done! |
jubueche
approved these changes
Apr 2, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related issues
#623
#677
Description