Strava Not Importing More than 37 Activities

[CyberSparkCloud]

Hi, I used the Strava integration and the first time the sync ran it imported 37 activities and when I looked at my Strava developer account it looked like it had hit the API limit so I assumed it was due to that it didn't import anymore. However even waiting to the following day for the sync to run at 02:00 it didn't import anymore.

I then completely deleted all the data from the docker folder and re-created it and re-set up the Strava integration etc. The first import ran and imported 37 activities but again, even after waiting 24 hours it hadn't imported anymore.. I checked the Strava API and it had not hit any limits so it wasn't due to this.

I've logged into Pocketbase but I cannot see any logs at all at 02:00. I did try setting POCKETBASE_CRON_SYNC_SCHEDULE in the compose file and recreating the containers to force it to sync but again, it doesn't seem to attempt to import anymore than the 37 already in there.

In Strava I have about 130 activities.

Any ideas?

Cheers.

[Flomp]

Hey,

Thanks for creating a dedicated issue. This will be tough for me to debug since the issue is probably dependent on your specific strava data and apparently, there are no logs (could you please double check? The error would be in pocketbase with log level 4). Is there anything special about the 38th activity?

[dyuri]

I have probably a similar issue, syncing strava activities stuck at some point. Probably something specific with the next activity.

Logs from docker:

2025-03-26T02:00:04.916267830Z panic: runtime error: index out of range [0] with length 0
2025-03-26T02:00:04.982791842Z
2025-03-26T02:00:04.982871997Z goroutine 2874 [running]:
2025-03-26T02:00:04.982901623Z pocketbase/integrations/strava.createTrailFromActivity({0x159b190, 0xc000383400}, 0xc000373c08, 0xc0004d44c0, {0xc0000ee260, 0xf})
2025-03-26T02:00:04.982953227Z  /Users/christianbeutel/Documents/svelte/wanderer/db/integrations/strava/strava.go:506 +0x9f4
2025-03-26T02:00:04.982996840Z pocketbase/integrations/strava.syncTrailsWithActivities({0x159b190, 0xc000383400}, {0xc000620090, 0x28}, {0xc0000ee260, 0xf}, {0xc0003bc008?, 0xc00049a630?, 0x0?})
2025-03-26T02:00:04.983036345Z  /Users/christianbeutel/Documents/svelte/wanderer/db/integrations/strava/strava.go:397 +0x2c5
2025-03-26T02:00:04.983068302Z pocketbase/integrations/strava.SyncStrava({0x159b190, 0xc000383400})
2025-03-26T02:00:04.983109057Z  /Users/christianbeutel/Documents/svelte/wanderer/db/integrations/strava/strava.go:121 +0x5f7
2025-03-26T02:00:04.983143752Z main.registerCronJobs.func1()
2025-03-26T02:00:04.983172603Z  /Users/christianbeutel/Documents/svelte/wanderer/db/main.go:729 +0x31
2025-03-26T02:00:04.983205002Z github.com/pocketbase/pocketbase/tools/cron.(*Job).Run(...)
2025-03-26T02:00:04.983236432Z  /Users/christianbeutel/go/pkg/mod/github.com/pocketbase/pocketbase@v0.26.1/tools/cron/job.go:25
2025-03-26T02:00:04.983271634Z created by github.com/pocketbase/pocketbase/tools/cron.(*Cron).runDue in goroutine 520
2025-03-26T02:00:04.983305050Z  /Users/christianbeutel/go/pkg/mod/github.com/pocketbase/pocketbase@v0.26.1/tools/cron/cron.go:225 +0x285
(the container dies here, then it's restarted a few minutes later)
2025-03-26T02:04:35.283168559Z 2025/03/26 02:04:35 Server started at http://0.0.0.0:8090
...

[dyuri]

Hmm, it might be StartLatlng is empty for some cases, like non-gpx based activities?

[CyberSparkCloud]

I think you guys are right. The 38th activity mine got stuck on was an indoor rowing one with no GPX data in it and 0m distance travelled.
I deleted it from Strava as it was the only one like this and I tried running Wanderer sync again but it wouldn't import any more.

I then wiped the docker data folders and started again from scratch and pulled the latest images but I couldn't get any activities imported at all. And I noticed that the oauth settings weren't in Pocketbase in the later image. So I rolled back to a previous version where the oauth settings are there but I still can't get any activities at all imported now even though I'm starting again completely from scratch. I can see the Strava API calls going up by 3 each time I run the sync but no activities imported.

[fs090]

Got the same issue, for me the 28th activitiy which is also without GPX data (indoor training) and also only set to visible for myself. Also all my strava activities get imported as routes - is that correct or should they be in activities?

[dyuri]

My fix looks OK here, already got my first ~100 activity synced (where strava's rate-limit hit) w/o any error.

[Flomp]

@CyberSparkCloud Sorry, I think this has to do with #209. Basically, I messed up the secret encryption, and your strava credentials are probably not stored correctly in the DB. This is hopefully fixed on the main branch and is included in the next patch.

In my testing, I found that an empty activity has no GPX and thus throws an error, which stops the import process. To mitigate the problem, I made the error handling a bit more graceful:
c8dcebf

So the import does not stop when encountering a single error, but rather logs a warning and continues.

@fs090 Yes, that is correct for now. I plan to implement leaderboards and competitive trails in one of the next major updates. Then I might rethink the import logic.

[Flomp]

Hopefully fixed in v0.16.1. Please reopen if the issue remains.

[fs090]

@Flomp thanks, it worked now but was not complete. Maybe as an input for your rethinking of the import logic I have ~750 activities in Strava and there were now >2000 requests to the API which resulted in wanderer_db logs to return "fetching activities from strava: failed to fetch activities: received status 429" with too many requests exceeding the 2000 request limit. I stopped wanderer_db at 3000 requests as it seemed to continue regardless of the 429. Maybe the import logic requires a import from/to for users with a lot of activities.

[dyuri]

@fs090 for me it resumed properly on the next run:

With the last version, it failed on the second day after ~60 successful imports where I had the strength activity (I also reset everything). (red)
Then when I was experimenting with my fix, and manually executed the cron job in pocketbase (waiting 15 mins between the runs) it imported ~150+ activity properly. (green)
This night, the cronjob successfully ran and imported the next ~50 activities. (blue)
(in average 2 calls / activity)

[fs090]

@dyuri true, it should then pick up the rest tomorrow, will check tomorrow again

[dyuri]

You can also log into pocketbase (wanderer_db:8090, the URL with the superadmin token should be displayed at start) and manually execute the cron job in every 15 minutes, but there's also a daily limit of 1000 API calls.

[fs090]

It loaded another 30 activities but did not stop the requests somehow first returned 429 for too many requests then also 401 not authorized when it´s probably been blocked

[CyberSparkCloud]

Mine is also showing a similar number of daily requests from when I set it up yesterday. I only ran the sync twice today which looks OK.

[dyuri]

Hmm, I only have the daily ~100, but I'm running 0.16 + my patch, so that high number of requests might be related to an other recent change, like the graceful error handling (c8dcebf) where previously the processing just stopped.