Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API Issues with initial sync (RateLimitExceeded) #222

Open
wokawoka opened this issue Mar 27, 2024 · 165 comments
Open

API Issues with initial sync (RateLimitExceeded) #222

wokawoka opened this issue Mar 27, 2024 · 165 comments

Comments

@wokawoka
Copy link

wokawoka commented Mar 27, 2024

Expected vs. actual behavior

After installation and having granted a fresh token I expect to be able able to download and sync my evernote database (around 17k notes) locally on nixnote2.

Unfortunately the sync get interrupted for an API error.
I installed nixnote2 on this machine just yesterday, the token is from yesterdays, to my knowledge the API limits should be initially lifted, instead this is what I find in the log:

ERROR 2024-03-27 16:10:56.596 src/communication/communicationmanager.cpp:546 Exception stacktrace:
ERROR 2024-03-27 16:10:56.601 src/communication/communicationerror.cpp:88 "RateLimitExceeded[code=19]: API rate limit exceeded. Enable 'Sync' > 'Restart sync on API limit' or try again in 53 minutes ## >
ERROR 2024-03-27 16:10:56.601 src/threads/syncrunner.cpp:278 Error retrieving chunk
ERROR 2024-03-27 16:12:57.723 src/communication/communicationmanager.cpp:546 Exception stacktrace:
ERROR 2024-03-27 16:12:57.726 src/communication/communicationerror.cpp:88 "RateLimitExceeded[code=19]: API rate limit exceeded. Application will continue to sync in 48 minutes ## EDAMSystemException: EDAMErrorCode::RATE_LIMIT_REACHED rateLimitDuration= 2868 sec. # rateLimitDuration=\u0B34"

Restart sync on API limit (experimental) is flagged in my preferences.

Do you have any suggestion?
Thank you so much!

Specifications

  • Version: 2023100407162.1.10ubuntu22.04.1
@boo-yee
Copy link

boo-yee commented Mar 29, 2024

This limit is from server side. I am afraid we have to wait until the time is out.

@wokawoka
Copy link
Author

Alright, but I have more than 20k notes to download locally in order to finish the sync
I had to restart the client at least 20 times in order to be able to download 1k of them (at the beginning I was not receiving the ratelimitexceed error above but the synchronization was getting stuck after about 10 minutes)
It would be next to impossible to complete the first sync with these limits.

I have read that with a fresh token the API limits should be initially lifted, then why am I facing all this issues?
Do you guys have any suggestion about that?
Thank you very much again!

@boo-yee
Copy link

boo-yee commented Mar 30, 2024

I have read that with a fresh token the API limits should be initially lifted, then why am I facing all this issues?

Do you mean this link? https://dev.evernote.com/doc/articles/rate_limits.php
This page says that the rate limit of initial sync for a new token can be temporarily increased within 24 hours only, but it may be not large enough for your case, I think. This may be the reason.

If you still have Evernote installed, and your Evernote has most of your notes synced, you can transfer locally.

If your notes consist of text mainly, and not too many multimedia files, you could consider to sync with Evernote and then transfer locally. I remember that Evernote client has a limit for data amount every month for free user, I am not sure if it is enough for your notes.

I don't have so many notes, and I don't know whether the option of 'Restart sync on API limit (experimental)' works or not in reality. But if nixnote doesn't restart the sync automatically, what about restarting nixnote every one hour with an outside shell commands or something like that, with the option of auto 'sync on start up' enabled? This method may take a long time, but it is available.

@wokawoka
Copy link
Author

I have read that with a fresh token the API limits should be initially lifted, then why am I facing all this issues?

Do you mean this link? https://dev.evernote.com/doc/articles/rate_limits.php This page says that the rate limit of initial sync for a new token can be temporarily increased within 24 hours only, but it may be not large enough for your case, I think. This may be the reason.

If you still have Evernote installed, and your Evernote has most of your notes synced, you can transfer locally.

If your notes consist of text mainly, and not too many multimedia files, you could consider to sync with Evernote and then transfer locally. I remember that Evernote client has a limit for data amount every month for free user, I am not sure if it is enough for your notes.

I don't have so many notes, and I don't know whether the option of 'Restart sync on API limit (experimental)' works or not in reality. But if nixnote doesn't restart the sync automatically, what about restarting nixnote every one hour with an outside shell commands or something like that, with the option of auto 'sync on start up' enabled? This method may take a long time, but it is available.

Thank you very much for your reply.
I' m on linux and evernote for linux has been disabled months ago. Until a couple of days ago I was using evernote legacy through wine and a repackage windows version that a guy was releasing on github here , and evernote disabled the API endpoint for both of them a couple of days ago.
That's how I landed here.
I still have both of them installed and I also have some recent backups in enex format.
I'm a subscribed user (not free user).
Should I import the backup notes on a fresh installation of nixnote and then pull down the new notes through a server sync?
Frankly I'm very afraid of doing a mess with the notes on the server, I keep very important information to me there.

The idea of a script killing and restarting nixnote every 20 minutes is also a good idea and I could try that.
But, as I' m not using nixnote since many many years, would be all this syncing/importing work worth it? Is the client completely functional at this point?
It would be a great way to be able to keep using evernote on linux (other than migrating to a different platform).
Thank you again

@boo-yee
Copy link

boo-yee commented Mar 31, 2024

Until a couple of days ago I was using evernote legacy through wine and a repackage windows version that a guy was releasing on github here

You were using evernote legacy through wine, so the latest evernote client can't work with wine, right?

I still have both of them installed and I also have some recent backups in enex format.

If there are only a few unsynced notes, how about using the web evernote and backing them up manually?

But, as I' m not using nixnote since many many years, would be all this syncing/importing work worth it?

You have backup files, so you don't need to sync all of them, backing up the unsynced notes is enough. Once you get all the necessary notes you want, you could not only import them to nixnote but also other apps you like.

Should I import the backup notes on a fresh installation of nixnote and then pull down the new notes through a server sync? Frankly I'm very afraid of doing a mess with the notes on the server, I keep very important information to me there.
The idea of a script killing and restarting nixnote every 20 minutes is also a good idea and I could try that.

If you decide to use nixnote or other apps which are integrated with Evernote cloud, then this is a safe way, it just takes a long time.

Is the client completely functional at this point?

Nixnote is functional in most of the time. Some bugs may appear, but not often. The ui may be a little legacy. and don't offer some modern, fancy functions just as other newer note apps such as obsidian.

@wokawoka
Copy link
Author

Thank you very much for your reply.

Until a couple of days ago I was using evernote legacy through wine and a repackage windows version that a guy was releasing on github here

You were using evernote legacy through wine, so the latest evernote client can't work with wine, right?

Yes, the new evernote version doesn't run under wine.

I still have both of them installed and I also have some recent backups in enex format.

If there are only a few unsynced notes, how about using the web evernote and backing them up manually?

The backup is from about two weeks ago, and in the meanwhile I might have probably edited many old notes other than adding new ones.
Even if I'm sure that it's a viable option, I'm afraid of messing with imports/exports of data that is not updated.

But, as I' m not using nixnote since many many years, would be all this syncing/importing work worth it?

You have backup files, so you don't need to sync all of them, backing up the unsynced notes is enough. Once you get all the necessary notes you want, you could not only import them to nixnote but also other apps you like.

So my last backup is not very fresh, but it looks like that tools like this are still working. So maybe I could pull down a fresh enex backup with that tool and then import the notes in nixnotes (but, by the way, why this tool is able to download and export the notes and nixnotes can't do the same?)

Should I import the backup notes on a fresh installation of nixnote and then pull down the new notes through a server sync? Frankly I'm very afraid of doing a mess with the notes on the server, I keep very important information to me there.
The idea of a script killing and restarting nixnote every 20 minutes is also a good idea and I could try that.

If you decide to use nixnote or other apps which are integrated with Evernote cloud, then this is a safe way, it just takes a long time.

Yes, I agree, I would much prefer to do a clean sync from the client itself. Scripting a recurrent shutdown/restart of nixnote like you were suggesting could be a good option.

Is the client completely functional at this point?

Nixnote is functional in most of the time. Some bugs may appear, but not often. The ui may be a little legacy. and don't offer some modern, fancy functions just as other newer note apps such as obsidian.

I'm not concerned about the look of the UI (and as I said, I was using the legacy version of evernote which was really looking oldshool, but it was quite snappy and functional). What I'm afraid of is inconsistencies between notes written in nixnote and in the new evernote official client. Recently I was using a mix a evernote legacy on one machine and evernote 10 on another machines and very often I was stepping into notes that were missing information in one client or in the other (it was happening especially with image attachments). I guess that overtime they changed things in their architecture and this is one of the reasons why they also dismissed the possibility to use the old clients.

@boo-yee
Copy link

boo-yee commented Apr 1, 2024

So my last backup is not very fresh, but it looks like that tools like this are still working. So maybe I could pull down a fresh enex backup with that tool and then import the notes in nixnotes

If this tool works and has no rate limit, it will be easy to solve the problem.

(but, by the way, why this tool is able to download and export the notes and nixnotes can't do the same?)

I have not studied the code, I just read the README page, and I guess that it syncs the notes by calling the offical Evernote client APIs. The offical Evernote client may call another group of APIs, which are different from the third-party ones called by nixnote, and have no rate limit, I guess.

What I'm afraid of is inconsistencies between notes written in nixnote and in the new evernote official client. Recently I was using a mix a evernote legacy on one machine and evernote 10 on another machines and very often I was stepping into notes that were missing information in one client or in the other (it was happening especially with image attachments). I guess that overtime they changed things in their architecture and this is one of the reasons why they also dismissed the possibility to use the old clients.

Evernote changed part of the note content format these years. If evernote 10 note format is not compatible with legacy client, then it's very possible that evernote 10 is not compatible with nixnote, because nixnote note format is designed to be compatible with legacy client, and did not follow the development and changes of evernote these years.

@wokawoka
Copy link
Author

wokawoka commented Apr 1, 2024

If this tool works and has no rate limit, it will be easy to solve the problem.

I pulled a backup with that tool yesterday night and it correctly updated the backup I previously did last month (he even just downloaded the new and changed notes). Just great

I have not studied the code, I just read the README page, and I guess that it syncs the notes by calling the offical Evernote client APIs. The offical Evernote client may call another group of APIs, which are different from the third-party ones called by nixnote, and have no rate limit, I guess.

I guess that integrating their syncing method or the tricks they are using would benefit this project a lot.
At this stage the barrier of entry for someone who is using evernote since a long time and has to import a lot of notes (hitting the API limit) is too high

Evernote changed part of the note content format these years. If evernote 10 note format is not compatible with legacy client, then it's very possible that evernote 10 is not compatible with nixnote, because nixnote note format is designed to be compatible with legacy client, and did not follow the development and changes of evernote these years.

That makes me think that issues and inconsistencies might appear if someone is using nixnote and at the same time evernote 10 on other machines or on mobile clients

@d1vanov
Copy link

d1vanov commented Apr 1, 2024

Evernote changed part of the note content format these years. If evernote 10 note format is not compatible with legacy client, then it's very possible that evernote 10 is not compatible with nixnote, because nixnote note format is designed to be compatible with legacy client, and did not follow the development and changes of evernote these years.

Any source? I haven't heard of them changing anything. Considering that they still have lots of business API integrations, it seems unlikely such moves would be done, especially silently.
Here's the specification for their ENEX format: http://xml.evernote.com/pub/evernote-export3.dtd.

What they could have actually changed is many other things which are present in note metadata but not in the note content (i.e. ENEX) itself - like all these reminder things and such.

@wokawoka
Copy link
Author

wokawoka commented Apr 1, 2024

Evernote changed part of the note content format these years. If evernote 10 note format is not compatible with legacy client, then it's very possible that evernote 10 is not compatible with nixnote, because nixnote note format is designed to be compatible with legacy client, and did not follow the development and changes of evernote these years.

Any source? I haven't heard of them changing anything. Considering that they still have lots of business API integrations, it seems unlikely such moves would be done, especially silently. Here's the specification for their ENEX format: http://xml.evernote.com/pub/evernote-export3.dtd.

What they could have actually changed is many other things which are present in note metadata but not in the note content (i.e. ENEX) itself - like all these reminder things and such.

I have no source but I have written what is my direct experience.

After many years of exclusively using the legacy version of evernote (under wine on linux), in the last 6 months I started using it alongside evernote 10 (on other machines), evernote android client, and evernote web.
Over the course of these months I ran into a lot of problems with synchronisation (especially on the evernote legacy side), problems with attachments (especially images which could no more be displayed natively in the client but could be downloaded) and even situations where whole chunks of text were missing.

This didn't happen when I was using legacy evernote exclusively.
My experience in these recent months has been that the two versions of evernote were less and less interoperable.
If the enex protocol will certainly have remained the same other things may have probably changed.

@boo-yee
Copy link

boo-yee commented Apr 2, 2024

I pulled a backup with that tool yesterday night and it correctly updated the backup I previously did last month (he even just downloaded the new and changed notes). Just great

It is nice.

I guess that integrating their syncing method or the tricks they are using would benefit this project a lot.
At this stage the barrier of entry for someone who is using evernote since a long time and has to import a lot of notes (hitting the API limit) is too high

I see that the API rate limit was often talked about on github before, if there are not too many notes, just waiting and restarting the sync can solve it. In recent years, I feel that the users are not as many as before, so I think the improvement on syncing will not be needed by too many people.

That makes me think that issues and inconsistencies might appear if someone is using nixnote and at the same time evernote 10 on other machines or on mobile clients

I am afraid so.

@boo-yee
Copy link

boo-yee commented Apr 2, 2024

@d1vanov

Any source? I haven't heard of them changing anything. Considering that they still have lots of business API integrations, it seems unlikely such moves would be done, especially silently.
Here's the specification for their ENEX format: http://xml.evernote.com/pub/evernote-export3.dtd.

What they could have actually changed is many other things which are present in note metadata but not in the note content (i.e. ENEX) itself - like all these reminder things and such.

What you said is reasonable, Evernote should be very serious about the format. But there are some posts about the note format on the evernote forum. For example, this link: https://discussion.evernote.com/forums/topic/148763-old-notes-bulleted-and-numbered-lists-messed-up/. To this kind of issues, I just guessed it was because of the format changes, because except the note format changes, I cannot think out other reasons. To find the source Evernote released, I searched just now, and found this link: https://evernote.com/blog/legacy-decommissioning#:~:text=Legacy%20versions%20of%20Evernote%20use,and%20update%20at%20lightning%20speed.
This link says: Expect a slower experience the first time you open a note in v10. Legacy versions of Evernote use a different note format than v10. As such, each note must be converted to the new format. This action takes place when the note is opened for the first time in v10 and can last a couple of seconds. Going forward, it’ll load and update at lightning speed.

Maybe the word I used and my understanding are not precise enough, but it seems that there is incompatibility between the legacy client and the latest one.

@wokawoka
Copy link
Author

wokawoka commented Apr 2, 2024

I see that the API rate limit was often talked about on github before, if there are not too many notes, just waiting and restarting the sync can solve it. In recent years, I feel that the users are not as many as before, so I think the improvement on syncing will not be needed by too many people.

the situation with the API syncing is making it very very difficult to switch to nixnote for any user that has more than 2k notes and above stored on evernote servers.
That's a pity, and it is certainly a reason that keeps nixnote's user base low, even on a platform like linux where at the moment there is a desperate need for an evernote local client

@d1vanov
Copy link

d1vanov commented Apr 2, 2024

To this kind of issues, I just guessed it was because of the format changes, because except the note format changes, I cannot think out other reasons

Well, I don't know for certain either but I believe what has actually changed is not the ENEX format spec itself but the note editor implementation between legacy Evernote client and v10 and in how each of them interprets the format. ENEX in its core is just a stripped down XHTML which in turn is just a little more strict and XML-like HTML. So it all kinda boils down to how a web browser implements a rich text editor. V10 might use different HTML tags for lists than the legacy client used. But I don't think a change in ENEX spec was necessary for that to happen.

@wokawoka
Copy link
Author

wokawoka commented Apr 2, 2024

To this kind of issues, I just guessed it was because of the format changes, because except the note format changes, I cannot think out other reasons

Well, I don't know for certain either but I believe what has actually changed is not the ENEX format spec itself but the note editor implementation between legacy Evernote client and v10 and in how each of them interprets the format. ENEX in its core is just a stipped down XHTML which in turn is just a little more strict and XML-like HTML. So it all kinda boils down to how a web browser implements a rich text editor. V10 might use different HTML tags for lists than the legacy client used. But I don't think a change in ENEX spec was necessary for that to happen.

Yes, it makes sense

@boo-yee
Copy link

boo-yee commented Apr 3, 2024

the situation with the API syncing is making it very very difficult to switch to nixnote for any user that has more than 2k notes and above stored on evernote servers.
That's a pity, and it is certainly a reason that keeps nixnote's user base low, even on a platform like linux where at the moment there is a desperate need for an evernote local client

Yes, it is one of the reasons. On many sides, nixnote is not close enough to perfect. On the other hand, it is not an easy thing to make it a perfect software through working on it just in spare time.

@wokawoka
Copy link
Author

wokawoka commented Apr 3, 2024

the situation with the API syncing is making it very very difficult to switch to nixnote for any user that has more than 2k notes and above stored on evernote servers.
That's a pity, and it is certainly a reason that keeps nixnote's user base low, even on a platform like linux where at the moment there is a desperate need for an evernote local client

Yes, it is one of the reasons. On many sides, nixnote is not close enough to perfect. On the other hand, it is not an easy thing to make it a perfect software through working on it just in spare time.

Of course, I was making a general point because I think it's a shame that a ready-made, working project is not easily accessible to a potentially very large group of users who are in need of an evernote client on linux.

In all cases investing in a free project available to all is always a great act of generosity that should be appreciated

@boo-yee
Copy link

boo-yee commented Apr 5, 2024

the situation with the API syncing is making it very very difficult to switch to nixnote for any user that has more than 2k notes and above stored on evernote servers.
That's a pity, and it is certainly a reason that keeps nixnote's user base low, even on a platform like linux where at the moment there is a desperate need for an evernote local client

Yes, it is one of the reasons. On many sides, nixnote is not close enough to perfect. On the other hand, it is not an easy thing to make it a perfect software through working on it just in spare time.

Of course, I was making a general point because I think it's a shame that a ready-made, working project is not easily accessible to a potentially very large group of users who are in need of an evernote client on linux.

In all cases investing in a free project available to all is always a great act of generosity that should be appreciated

I studied evernote-backup code, and I think we could have an easy solution for this problem. I'll try it, but I can't promise.

@boo-yee
Copy link

boo-yee commented May 7, 2024

Hi, wokawoka. I hope this reply is not too late. I made a PR for this issue and it may probably solve it, since I compared the logic of evernote-backup and nixnote, and removed tedious part of nixnote. For the details you can visit #223. It works fine for me, but I am not sure whether the rate limit exception will appear when there are thousands of notes with resource files. Could you please help to test it?

@boo-yee
Copy link

boo-yee commented May 7, 2024

If you'd like to, I can offer you a binary under Windows here: https://github.com/boo-yee/nixnote2/releases/download/v2.1.10/nixnote.zip.

Update: And also for Linux there is appimage now. https://github.com/robert7/nixnote2/releases/download/continuous-develop/NixNote2-x86_64.AppImage

@wokawoka
Copy link
Author

wokawoka commented May 8, 2024

this is very good news
I am going to test the appimage later tonight when at home
Thank you very much again!

@boo-yee
Copy link

boo-yee commented May 8, 2024

I feel very glad to receive your reply and appreciate your help. I think, if syncing all the notes takes too long time, syncing only part of your notes is enough, as long as there are no exception reported, for a longer period of time than previously. Syncing 2-3 thousand notes is enough I think. I created one thousand notes and synced them without exception. The only thing I am not sure is if there is any side effect after my changes.

And a reminder for you for the sake of data safety: please do not let the new nixnote appimage mess up your old local database.

@wokawoka
Copy link
Author

wokawoka commented May 9, 2024

yesterday night I tried to pull a full sync through the appimage you posted above but unfortunately I failed one again.
The first time the sync stopped at 800 notes.
The second one stopped at 1790.
And the third one at 2794.
Do you have any suggestion? Is there something that I could try in order to make it work?
Thank you

@boo-yee
Copy link

boo-yee commented May 9, 2024

Do you have any suggestion? Is there something that I could try in order to make it work?

It should be the code, not your mkstake. I think I have to ask you several questions. You said it stopped, did it report the rate limit exceeded exception? Or no exception reported, just the note list not updated? And do you know how long time around does it take to sync the first 800 notes? (I think the network speed may also determine the api calling frequency.) Do you think the new appimage peformed better than previous version?

@wokawoka
Copy link
Author

wokawoka commented May 9, 2024

I never stepped into a "rate limit exceeded" error, the syncing each time just stopped pulling the notes (even if the syncing icon was still showing as spinning).
Regarding the speed, I can't really tell, I was on wifi and the connection was quite stable, I just left the program downloading in the background and I was monitoring from time to time the note count through the "tools -> database status" menu option.
If you need, just let me know how I could monitor the syncing rate in a more specific way and I'll do more test.
Thank you very much again

@boo-yee
Copy link

boo-yee commented May 10, 2024

If the rate limit exceeded exception does not appear, then the issue about the api should be fixed. As I was not sure if my changes would bring any side effects, I just focused on the rate limit issue, and left the other things alone, hoping to fix all the issues one by one.

the syncing each time just stopped pulling the notes (even if the syncing icon was still showing as spinning)

For this issue, I think I have to continue to inspect the code to find the reason. (This issue is probably nothing about the rate limit exception. I think it is something about optimization. And the thread number is also another reason that nixnote is not as fast as evernote-backup, which uses multi-thread. So nixnote can be still improved.)

Regarding the speed, I can't really tell, I was on wifi and the connection was quite stable, I just left the program downloading in the background and I was monitoring from time to time the note count through the "tools -> database status" menu option.
If you need, just let me know how I could monitor the syncing rate in a more specific way and I'll do more test.

As the rate limit exception is fixed, we do not need to focus on the api and the network, I think. When I asked that question about the network speed, I thought the rate limit exception appeared again. However it did not in fact.

@boo-yee
Copy link

boo-yee commented May 10, 2024

The resources such as the images in the newly synced notes is correctly synced together, right?

@wokawoka
Copy link
Author

The resources such as the images in the newly synced notes is correctly synced together, right?

I actually didn´t inspect any note because I didn't want to interfere in any way with the syncing.
Since we are here, do you think that it's safe for me doing these tests with you on my (very much important) notes?
Should I pull a backup before going on with these tests or the syncing is (for now) just a download of the notes through the API from the server to client?
Thank you

@boo-yee
Copy link

boo-yee commented May 11, 2024

Since we are here, do you think that it's safe for me doing these tests with you on my (very much important) notes?
Should I pull a backup before going on with these tests or the syncing is (for now) just a download of the notes through the API from the server to client?

Yes, it is just a download. Only syncing the notes from a fresh nixnote client will not affect the notes on the cloud server.

As you synced the notes with evernote-backup earlier, just importing and using them is OK. Nixnote syncing is still slower, full sync will take too much time.

For the very important notes, I think it's best to backup them well and carefully, as Nixnote is not reliable sometimes, one of my note's content once went out of order when I was using it. It is about charater encoding. It happend only once, so I didn't put it at the first priority. I also saw similar issues from another user: baumgarr#446 (comment) (I guess that it was because of the thread mutex not set correctly and not write back to the disk when exiting abnormally. And I'll work on this later.)

@prahal
Copy link

prahal commented Jun 24, 2024

'Restart sync on API limit (experimental)' works.
The only issue I have and is likely an Evernote API bug is that when the limit is reached the evernote API returns the exception and nixnote waits for the returned timeout. But once this timeout is reached, the sync restart only for around 5 minutes then an API limit exceed is triggered anew and waits for an hour. The next attempt works fine (it syncs for half an hour or more before the API limit triggers), then the next is 5 minutes ... and so forth.
So it takes twice the time of the theoretical API limit.
But I don't think this is a nixnote/qevernote issue. Either way, I cannot devote time to document this issue properly in a bug report for now. But I compile notes about it.

@boo-yee
Copy link

boo-yee commented Oct 16, 2024

@vitaly-zdanevich

Hi, are the shared notes synced now?

@vitaly-zdanevich
Copy link

I deleted ~/.conig/nixnote and ~/.local/share/nixnote2, doing full resync now...

@boo-yee
Copy link

boo-yee commented Oct 23, 2024

I deleted ~/.conig/nixnote and ~/.local/share/nixnote2, doing full resync now...

How is the resync going?

@vitaly-zdanevich
Copy link

I deleted ~/.conig/nixnote and ~/.local/share/nixnote2, doing full resync now...

How is the resync going?

ERROR 2024-10-25 15:38:44.763 src/communication/communicationerror.cpp:88 "StdException[code=16]: Connection closed"

@boo-yee
Copy link

boo-yee commented Oct 26, 2024

The server closed the connection, just syncing again is OK.

@vitaly-zdanevich
Copy link

Again

ERROR 2024-10-29 02:17:17.852 src/communication/communicationerror.cpp:88 "EDAMUserException[PERMISSION_DENIED]: EDAMUserException: EDAMErrorCode::PERMISSION_DENIED parameter=authenticationToken"

looks like finally it fetched all notes.

@boo-yee
Copy link

boo-yee commented Oct 30, 2024

Again

ERROR 2024-10-29 02:17:17.852 src/communication/communicationerror.cpp:88 "EDAMUserException[PERMISSION_DENIED]: EDAMUserException: EDAMErrorCode::PERMISSION_DENIED parameter=authenticationToken"

looks like finally it fetched all notes.

Including the shared notes?

About the exception, did you try to share the notebook in which the shared notes are located with your secondary account?

@vitaly-zdanevich
Copy link

Solved by removing a few shared notes...

@boo-yee
Copy link

boo-yee commented Dec 2, 2024

Solved by removing a few shared notes...

I cannot reproduce this issue, so I do not know how to continue.

@vitaly-zdanevich
Copy link

Maybe it were notes shared to/from an account that not exist anymore... very old.

@boo-yee
Copy link

boo-yee commented Dec 11, 2024

@wokawoka

Hi, wokawoka, the new release is out now, and you could have a test in any time you want.
The link for the release is: https://github.com/robert7/nixnote2/releases/tag/continuous-develop

Thank you for the information @boo-yee ! I did try the new appimage and unfortunately I'm facing the same instant crash as described here. I guess there's a problem with nixnote database: I might try to redownload again everything from scratch but I guess I can' t find the time for that, the last time the sync took two days (and was nearly finished when I started to experience errors with the notes it downloaded). Thank you very much again

It seems that nixnote sync works now. And the sync can be accelerated. If you are interested, you could have one more test, and report new exceptions as soon as they appear, if there are any.

me: And also I added a setting of downloading thread number, in the sync tab of preference dialog. It is 5 as default, you can set it according to your network speed, so that the test can be accelerated, and make the most of the initial boost. For getting to know the downloading speed, you could start Nixnote with --logLevel=0, and when Nixnote is downloading notes, you can see such messages as DEBUG ...... src/communication/communicationmanager.cpp:795 downloadNote guid=... The faster it is printed, the faster the downloading. (If you find Nixnote works fine, you could start it without --logLevel=0, so that the sync can be faster.)

@wokawoka
Copy link
Author

@boo-yee
thank you for the message!
A couple of weeks ago I did start a new sync from scratch of my 18k+ notes and after stepping into different errors and crashes, in a couple of days I nearly went until the end.
Today, after reading your message, I relaunched the sync and after about one hour of waiting it seems that now all the notes are there. Thank you!

Now, since I'm here (and I don´t know where else I could ask this) I just would like to ask a couple of quick suggestions about using nixnote itself (but let me know in case there's a forum or another place where I should ask for this):

  • My system is set to a dark theme. Is there a way to set the background of the notes in white color? Many notes are unreadable (black letters on black or grey background). I already tried to change the theme from the "edit->theme" menu, but all of that themes seem to be dark and the background of the notes is always grey/black.
  • Is there a way to disconnect nixnote from evernote servers and use it just as an archive? Would revoking the authorization on the user account maintenance menu still let me use nixnote as an offline browsable database of my notes?
    Thank you very much once again!

@boo-yee
Copy link

boo-yee commented Dec 13, 2024

Today, after reading your message, I relaunched the sync and after about one hour of waiting it seems that now all the notes are there. Thank you!

I just realized that I forgot to paste the latest release link in my last comment, which is https://github.com/boo-yee/nixnote2/releases/tag/continuous, did you use this version? (This version has the option of setting thread number.)

My system is set to a dark theme. Is there a way to set the background of the notes in white color? Many notes are unreadable (black letters on black or grey background). I already tried to change the theme from the "edit->theme" menu, but all of that themes seem to be dark and the background of the notes is always grey/black.

The dark theme is a long standing issue, I did not focus on it too much because there are other issues, but I'll work on it later.

Is there a way to disconnect nixnote from evernote servers and use it just as an archive? Would revoking the authorization on the user account maintenance menu still let me use nixnote as an offline browsable database of my notes?
Thank you very much once again!

Yes, just revoking the authorization is enough.

@wokawoka
Copy link
Author

I just realized that I forgot to paste the latest release link in my last comment, which is https://github.com/boo-yee/nixnote2/releases/tag/continuous, did you use this version? (This version has the option of setting thread number.)

Yes, I did use the last appimage available (September).

I can live with the dark theme, no problem.

The real issue now is that, while I was trying out nixnote, I kept modyfing notes from evernote web.
I added a new notebook with a new note on evernote web and nixnote didn't pick up the new notebook.
The note inside that notebook has been downloaded in nixnote but it now appears in a new "" notebook instead inside the newly created notebook (which is missing on nixnote).
Nothing has changed after one day and many syncs.

Do you have any suggestion to fix this? Is there a way to force this change? How can I be sure that all the changes on evernote web's side are being reflected on nixnote as well?

Thank you very much once again!

@boo-yee
Copy link

boo-yee commented Dec 13, 2024

I added a new notebook with a new note on evernote web and nixnote didn't pick up the new notebook.
The note inside that notebook has been downloaded in nixnote but it now appears in a new "" notebook instead inside the newly created notebook (which is missing on nixnote).

Does the notebook name contain special characters or just consist of latin characters?

@wokawoka
Copy link
Author

Does the notebook name contain special characters or just consist of latin characters?

the name it's very similar to other notebooks that got synced correclty and it just contains latin characters

@wokawoka
Copy link
Author

the name it's very similar to other notebooks that got synced correclty and it just contains latin characters

Is there by chance a way to just resync the notebooks (and not full 18k+ notes) of my account?
Thank you

@boo-yee
Copy link

boo-yee commented Dec 18, 2024

I can live with the dark theme, no problem.

I updated the dark theme, which is named with 'global dark'. It is still https://github.com/boo-yee/nixnote2/releases/tag/continuous.
It is white text on dark blue background now. If it is not enough for you, then you could modify the theme.ini by yourself, which is finding editorCss in the theme.ini and changing it to editorCss=body { color: black; background-color: white;}.

Is there by chance a way to just resync the notebooks (and not full 18k+ notes) of my account?

Yes. Use a sql statement to set all the notes you want to resync to dirty, thie field name is 'isDirty', in the 'notetable' table, then these notes will be resynced.

And then again,

A couple of weeks ago I did start a new sync from scratch of my 18k+ notes and after stepping into different errors and crashes, in a couple of days I nearly went until the end.

The errors and crashes you met come from my release in September or previous release?

the name it's very similar to other notebooks that got synced correclty and it just contains latin characters

I'll have to study it later.

@wokawoka
Copy link
Author

I updated the dark theme, which is named with 'global dark'. It is still https://github.com/boo-yee/nixnote2/releases/tag/continuous. It is white text on dark blue background now. If it is not enough for you, then you could modify the theme.ini by yourself, which is finding editorCss in the theme.ini and changing it to editorCss=body { color: black; background-color: white;}.

The new dark theme is much much better. Still, there are some notes that I wrote myself on evernote with dark blue text on blue background.
But, again, I can live with them. Thank you very much

Is there by chance a way to just resync the notebooks (and not full 18k+ notes) of my account?

Yes. Use a sql statement to set all the notes you want to resync to dirty, thie field name is 'isDirty', in the 'notetable' table, then these notes will be resynced.

Thank you for the explanation, but the problem here is that a notebook is missing, and the note that belogs to it are erroneusly marked as belonging to a "" notebook.
Would flagging that note as dirty force nixnote to add the missing notebook while resyncing the note?

And then again,

A couple of weeks ago I did start a new sync from scratch of my 18k+ notes and after stepping into different errors and crashes, in a couple of days I nearly went until the end.

The errors and crashes you met come from my release in September or previous release?

They come from this release from September.
I even saved the different errors (some were on the log, and some other were popping up as kde notifications).
I just don't have them here since I'm on a different machine.

the name it's very similar to other notebooks that got synced correclty and it just contains latin characters

I'll have to study it later.

Thank you very much for your work

@vitaly-zdanevich
Copy link

I updated the dark theme, which is named with 'global dark'. It is still https://github.com/boo-yee/nixnote2/releases/tag/continuous.

thanks
image

@boo-yee
Copy link

boo-yee commented Dec 19, 2024

@wokawoka

The new dark theme is much much better. Still, there are some notes that I wrote myself on evernote with dark blue text on blue background.
But, again, I can live with them. Thank you very much

If so, I'm afraid that you have to change the background color by yourself in theme.ini or change the blue text to another color.

Thank you for the explanation, but the problem here is that a notebook is missing, and the note that belogs to it are erroneusly marked as belonging to a "" notebook.
Would flagging that note as dirty force nixnote to add the missing notebook while resyncing the note?

Can you rename the notebook named with "" in Nixnote?

@boo-yee
Copy link

boo-yee commented Dec 19, 2024

@vitaly-zdanevich Sorry for being so late. As there are other issues, I almost forgot this issue. The dark theme is not perfect now, there are still some little flaws.

@vitaly-zdanevich
Copy link

@vitaly-zdanevich Sorry for being so late. As there are other issues, I almost forgot this issue. The dark theme is not perfect now, there are still some little flaws.

Yep, but better than nothing, dark themes are essential for my eyes.

@wokawoka
Copy link
Author

@boo-yee

If so, I'm afraid that you have to change the background color by yourself in theme.ini or change the blue text to another color.

I will try to modify the theme.ini file as soon as possible

Can you rename the notebook named with "" in Nixnote?

In the meanwhile I did already recreate the notebook and moved the note back in so everything is fine. Still, I don't know why nixnote was not picking up the notebook correctly and I am a bit afraid that it might happen again without me realizing it. But I will try to monitor what happens during the syncs from now on.

Other than that, I am just experiencing a last (very small) annoyance about the note list: whenever I start nixnote the note list is showing the notes ordered from the first ever created no matter what I have previously set on View -> "Sort notes by".
Each time I start nixnote I have to manually select again the "Sort notes by" menu option to "Relevance Date updated (desc)".
Is there a way to set nixnote to always respect the "Sort notes by" selection since it starts up? I played with some options but nothing changed.
Thank you!

@vitaly-zdanevich
Copy link

I will try to modify the theme.ini file as soon as possible

And maybe contribute it here, thanks.

@boo-yee
Copy link

boo-yee commented Dec 20, 2024

@wokawoka

In the meanwhile I did already recreate the notebook and moved the note back in so everything is fine. Still, I don't know why nixnote was not picking up the notebook correctly and I am a bit afraid that it might happen again without me realizing it. But I will try to monitor what happens during the syncs from now on.

Evernote web does note allow the free users, such as me, to create a second notebook via it, so I can't reproduce it and know the reason I'm afraid. But I found that I can still create a new notebook and sync it normally via Nixnote.

Other than that, I am just experiencing a last (very small) annoyance about the note list: whenever I start nixnote the note list is showing the notes ordered from the first ever created no matter what I have previously set on View -> "Sort notes by".
Each time I start nixnote I have to manually select again the "Sort notes by" menu option to "Relevance Date updated (desc)".
Is there a way to set nixnote to always respect the "Sort notes by" selection since it starts up? I played with some options but nothing changed.

You can do it by clicking at the note list headers, then Nixnote will remember your setting.

@boo-yee
Copy link

boo-yee commented Dec 20, 2024

@vitaly-zdanevich

Yep, but better than nothing, dark themes are essential for my eyes.

I should have fixed it earlier.

And maybe contribute it here, thanks.

If it is only the editor color that does not fit you, you could also modify it by yourself, it is very easy.

@wokawoka
Copy link
Author

@boo-yee

Other than that, I am just experiencing a last (very small) annoyance about the note list: whenever I start nixnote the note list is showing the notes ordered from the first ever created no matter what I have previously set on View -> "Sort notes by".
Each time I start nixnote I have to manually select again the "Sort notes by" menu option to "Relevance Date updated (desc)".
Is there a way to set nixnote to always respect the "Sort notes by" selection since it starts up? I played with some options but nothing changed.

You can do it by clicking at the note list headers, then Nixnote will remember your setting.

I tried to look what you meant with "click at the note list headers" but unfortunately I wasn't able to find anything. Could you please explain where am I supposed to click or what I am supposed to select? Thank you very much!

(as a side question: I see that nixnote is not importing evernote's favourite notes to its shortcuts panel. But how to manually set the shortcuts on nixnote? I tried to "pin" some notes but, apparently, nothing changed. Thank you again)

@boo-yee
Copy link

boo-yee commented Dec 21, 2024

I tried to look what you meant with "click at the note list headers" but unfortunately I wasn't able to find anything. Could you please explain where am I supposed to click or what I am supposed to select? Thank you very much!

The note list headers I mean are:
image

(as a side question: I see that nixnote is not importing evernote's favourite notes to its shortcuts panel. But how to manually set the shortcuts on nixnote? I tried to "pin" some notes but, apparently, nothing changed. Thank you again)

You could drag the note from the list to the Shortcuts area with the mouse.

@wokawoka
Copy link
Author

I tried to look what you meant with "click at the note list headers" but unfortunately I wasn't able to find anything. Could you please explain where am I supposed to click or what I am supposed to select? Thank you very much!

The note list headers I mean are: image

(as a side question: I see that nixnote is not importing evernote's favourite notes to its shortcuts panel. But how to manually set the shortcuts on nixnote? I tried to "pin" some notes but, apparently, nothing changed. Thank you again)

You could drag the note from the list to the Shortcuts area with the mouse.

Thank you, the note headers were totally empty at first.
I was finally able to set it up the way I needed.
Thank you again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants