Skip to content

Sync error #1361

Closed
Bug
Closed
Sync error#1361
Bug

Description

@Sanji78
Sanji78
opened on Sep 11, 2026

Priority/Impact

No response

Description

Description

[Please describe what you were doing when the error occurred.]

Error details

Error title: Sync error
Error message: Exception: Failed to fetch token

Sync status:

server: self-hosted
connected: false, connecting: false, downloading: false, uploading: false
last successful sync: 2026-08-26T19:54:22.624Z
pending uploads: 1
checkpoint request: target 9223372036854775807, seen -, applied -
local buckets: last op 560166, ops since checkpoint 25000, oplog rows 9936
error (Uncategorised): Exception: Failed to fetch token

App logs (last 23 entries):

2026-09-11T16:11:10.813477Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/structure/, attempt 3 (1233ms), reason: status code 500
2026-09-11T16:11:10.732536Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-gym/, attempt 3 (1223ms), reason: status code 500
2026-09-11T16:11:10.641081Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-display/, attempt 3 (1229ms), reason: status code 500
2026-09-11T16:11:08.929536Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/structure/, attempt 2 (542ms), reason: status code 500
2026-09-11T16:11:08.821955Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-gym/, attempt 2 (551ms), reason: status code 500
2026-09-11T16:11:08.765608Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-display/, attempt 2 (569ms), reason: status code 500
2026-09-11T16:11:07.335576Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-display/, attempt 1 (307ms), reason: status code 500
2026-09-11T16:11:07.334605Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-gym/, attempt 1 (276ms), reason: status code 500
2026-09-11T16:11:07.332340Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/structure/, attempt 1 (293ms), reason: status code 500
2026-09-11T16:11:06.904863Z WARNING [powersync-django] Upload deferred: Upload of UpdateType.patch on core_userprofile deferred: retryable status 500
2026-09-11T16:11:02.808488Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/structure/, attempt 3 (1146ms), reason: status code 500
2026-09-11T16:11:02.806672Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-display/, attempt 3 (1090ms), reason: status code 500
2026-09-11T16:11:02.706137Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-gym/, attempt 3 (1188ms), reason: status code 500
2026-09-11T16:11:01.161443Z WARNING [powersync-django] Upload deferred: Upload of UpdateType.patch on core_userprofile deferred: retryable status 500
2026-09-11T16:11:01.160441Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-display/, attempt 2 (602ms), reason: status code 500
2026-09-11T16:11:01.160125Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-gym/, attempt 2 (561ms), reason: status code 500
2026-09-11T16:11:01.158683Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/structure/, attempt 2 (546ms), reason: status code 500
2026-09-11T16:10:59.879438Z INFO [powersync] Connecting to the sync service (manual retry)
2026-09-11T16:10:59.680834Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-display/, attempt 1 (257ms), reason: status code 500
2026-09-11T16:10:59.674816Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/structure/, attempt 1 (252ms), reason: status code 500
2026-09-11T16:10:59.667999Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-gym/, attempt 1 (260ms), reason: status code 500
2026-09-11T16:10:58.024082Z WARNING [powersync-django] Upload deferred: Upload of UpdateType.patch on core_userprofile deferred: retryable status 500
2026-09-11T16:10:56.074364Z INFO [WgerBaseProvider] Retrying fetch for https://gym.xxx.ddns.net/api/v2/routine/14/structure/, attempt 3 (1114ms), reason: status code 500

Server version

No response

Mobile app version

2.1.0+270 on android, installed by com.android.vending

Activity

  1. Sanji78
    added the
    Bug
    issue type on Sep 11, 2026

  2. rolandgeider commented on Sep 27, 2026

    @rolandgeider
    rolandgeider
    on Sep 27, 2026
    Member

    Hi! This looks like a server problem, all the requests return a server error, e.g. https://gym.xxx.ddns.net/api/v2/routine/14/date-sequence-display/, attempt 1 (257ms), reason: status code 500

    is there anything in the docker logs for the web service?

  3. Sanji78 commented on Sep 27, 2026

    @Sanji78
    Sanji78
    on Sep 27, 2026
    ContributorAuthor

    Which docker logs shall I inspect?

  4. rolandgeider commented on Sep 27, 2026

    @rolandgeider
    rolandgeider
    on Sep 27, 2026
    Member

    the web service, it's the one producing the server errors

  5. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    " django.db.utils.ProgrammingError: column core_userprofile.time_zone does not exist" I dunno if this is the root cause... in powersynch I have: " user_profile:
    auto_subscribe: true
    queries:
    - SELECT * FROM core_userprofile WHERE CAST(user_id AS TEXT) = auth.user_id()
    - SELECT * FROM gallery_image WHERE CAST(user_id AS TEXT) = auth.user_id()"

  6. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    I fixed it :)

    stefano@ServerNVR:~/Gym$ docker exec -it wger-db psql -U wger -d wger \
 -c "SELECT column_name FROM information_schema.columns WHERE table_name = 'core_userprofile' ORDER BY ordinal_position;"
       column_name
---------------------------
id
is_temporary
workout_reminder_active
workout_reminder
workout_duration
last_workout_notification
age
height
gender
sleep_hours
work_hours
work_intensity
sport_hours
sport_intensity
freetime_hours
freetime_intensity
calories
weight_unit
gym_id
notification_language_id
user_id
num_days_weight_reminder
birthdate
added_by_id
can_add_user
repetitions_rounding
weight_rounding
trophies_enabled
(28 rows)

stefano@ServerNVR:~/Gym$ docker exec -it wger-web python3 manage.py migrate --check
stefano@ServerNVR:~/Gym$ docker exec -it wger-web python3 manage.py showmigrations core
core
[X] 0001_initial
[X] 0002_auto_20141225_1512
[X] 0003_auto_20150217_1554
[X] 0004_auto_20150217_1914
[X] 0005_auto_20151025_2236
[X] 0006_auto_20151025_2237
[X] 0007_repetitionunit
[X] 0008_weightunit
[X] 0009_auto_20160303_2340
[X] 0010_auto_20170403_0144
[X] 0011_auto_20201201_0653
[X] 0012_auto_20210210_1228
[X] 0013_auto_20210726_1729
[X] 0013_userprofile_email_verified
[X] 0014_merge_20210818_1735
[X] 0015_alter_language_short_name
[X] 0016_alter_language_short_name
[X] 0017_language_full_name_en
[X] 0018_rounding
[X] 0019_delete_daysofweek
[X] 0020_add_trophies_enabled_to_userprofile
[X] 0021_add_unit_type_to_repetitionunit
[X] 0022_move_email_verified_to_emailaddress
[X] 0023_create_publication
[X] 0024_backfill_emailaddress
[X] 0025_remove_unused_fields_in_userprofile
[X] 0026_alter_userprofile_birthdate_alter_userprofile_height
[ ] 0027_powersync_publication
[ ] 0028_longlivedsession
[ ] 0029_userprofile_timezone
stefano@ServerNVR:~/Gym$ docker exec -it wger-web python3 manage.py migrate
Operations to perform:
 Apply all migrations: account, actstream, allauth_idp_oidc, auth, authtoken, axes, config, contenttypes, core, easy_thumbnails, exercises, gallery, gym, mailer, manager, measurements, mfa, nutrition, sessions, sites, socialaccount, token_blacklist, trophies, weight
Running migrations:
 Applying allauth_idp_oidc.0001_initial... OK
 Applying allauth_idp_oidc.0002_client_default_scopes... OK
 Applying allauth_idp_oidc.0003_client_allow_uri_wildcards... OK
 Applying manager.0028_backfill_session_day... OK
 Applying core.0027_powersync_publication... OK
 Applying core.0028_longlivedsession... OK
 Applying core.0029_userprofile_timezone... OK
 Applying manager.0029_alter_workoutsession_options_and_more... OK
 Applying measurements.0006_health_sync... OK
 Applying measurements.0007_migrate_weight... OK
 Applying measurements.0008_dynamic_type... OK
 Applying weight.0006_delete_weightentry... OK
stefano@ServerNVR:~/Gym$ docker exec -it wger-web python3 manage.py showmigrations core
core
[X] 0001_initial
[X] 0002_auto_20141225_1512
[X] 0003_auto_20150217_1554
[X] 0004_auto_20150217_1914
[X] 0005_auto_20151025_2236
[X] 0006_auto_20151025_2237
[X] 0007_repetitionunit
[X] 0008_weightunit
[X] 0009_auto_20160303_2340
[X] 0010_auto_20170403_0144
[X] 0011_auto_20201201_0653
[X] 0012_auto_20210210_1228
[X] 0013_auto_20210726_1729
[X] 0013_userprofile_email_verified
[X] 0014_merge_20210818_1735
[X] 0015_alter_language_short_name
[X] 0016_alter_language_short_name
[X] 0017_language_full_name_en
[X] 0018_rounding
[X] 0019_delete_daysofweek
[X] 0020_add_trophies_enabled_to_userprofile
[X] 0021_add_unit_type_to_repetitionunit
[X] 0022_move_email_verified_to_emailaddress
[X] 0023_create_publication
[X] 0024_backfill_emailaddress
[X] 0025_remove_unused_fields_in_userprofile
[X] 0026_alter_userprofile_birthdate_alter_userprofile_height
[X] 0027_powersync_publication
[X] 0028_longlivedsession
[X] 0029_userprofile_timezone
stefano@ServerNVR:~/Gym$ docker compose restart powersync
[+] Restarting 1/1
✔ Container wger-powersync  Started

  7. rolandgeider commented on Sep 28, 2026

    @rolandgeider
    rolandgeider
    on Sep 28, 2026
    Member

    yes that was the problem!

    you probably had DJANGO_PERFORM_MIGRATIONS set to false in the config, in that case you need to take care of these things yourself :)

  8. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    I still have issue in logging in ..

    Error message: ClientException with SocketException: Failed host lookup: 'gym.XXX' (OS Error: No address associated with hostname, errno = 7), uri=https://gym.XXX/sync/sync/stream Sync status: server: self-hosted connected: true, connecting: false, downloading: true, uploading: false last successful sync: 2026-08-26T19:54:22.624Z pending uploads: 0 checkpoint request: target 1, seen -, applied - local buckets: last op 793226, ops since checkpoint 258050, oplog rows 10212 download progress: 258050 / 258287 error (Connection error): ClientException with SocketException: Failed host lookup: 'gym.XXX' (OS Error: No address associated with hostname, errno = 7), uri=https://gym.XXX/sync/sync/stream App logs (last 26 entries): 2026-09-28T07:54:54.152145Z FINE [AuthNotifier] revalidation: session still valid 2026-09-28T07:54:54.095079Z WARNING [ServerGating] serverUpdateRequired: could not parse server version "2.8.0-dev.0" (sanitized: "2.8.0-"), skipping check 2026-09-28T07:54:53.376563Z FINE [AuthNotifier] refreshAccessToken: success, new expiry 2026-09-28 07:59:53.000Z, rotated refresh token: true 2026-09-28T07:54:52.654976Z FINE [AuthNotifier] revalidation: access token within leeway, refreshing first 2026-09-28T07:54:52.654877Z FINE [AuthNotifier] revalidation: app resumed with stale access token 2026-09-28T07:54:52.615642Z FINE [AuthHttpClient] Pre-emptive refresh: access token within leeway window 2026-09-28T07:54:52.615612Z FINER [TrophyStateNotifier] Fetching user trophy progression 2026-09-28T07:54:52.615568Z FINE [AuthHttpClient] Pre-emptive refresh: access token within leeway window 2026-09-28T07:54:52.615472Z FINER [TrophyStateNotifier] Fetching user trophies 2026-09-28T07:54:52.614698Z FINE [AuthNotifier] refreshAccessToken: starting 2026-09-28T07:54:52.614590Z FINE [AuthHttpClient] Pre-emptive refresh: access token within leeway window 2026-09-28T07:54:52.614256Z FINER [TrophyStateNotifier] Fetching trophies 2026-09-28T07:54:52.613134Z INFO [NetworkStatus] Network status: online (probe answered) 2026-09-28T07:54:52.520621Z FINE [PowerSync] Credentials: PowerSyncCredentials<endpoint: https://gym.XXX/sync/ userId: 7 expiresAt: 2026-09-28 07:59:48.450629Z> (repeated 19x since last logged) 2026-09-28T07:51:58.122813Z INFO [NetworkStatus] Network status: offline (probe failed: ClientException with SocketException: Failed host lookup: 'gym.XXX' (OS Error: No address associated with hostname, errno = 7), uri=https://gym.XXX/api/v2/version/) 2026-09-28T07:51:28.599358Z WARNING [PowerSync] Sync error: Sync service error (repeated 19x since last logged) | ClientException with SocketException: Failed host lookup: 'gym.XXX' (OS Error: No address associated with hostname, errno = 7), uri=https://gym.XXX/sync/sync/stream #0 IOClient.send (package:http/src/io_client.dart:227) #1 StreamingSyncImplementation._postStreamRequest (package:powersync/src/sync/streaming_sync.dart:332) #2 streamFromFutureAwaitInCancellation. (package:powersync/src/sync/stream_utils.dart:142) 2026-09-28T07:51:25.401467Z FINER [TrophyStateNotifier] Fetching user trophy progression 2026-09-28T07:51:25.401291Z FINER [TrophyStateNotifier] Fetching user trophies 2026-09-28T07:51:25.400794Z FINER [TrophyStateNotifier] Fetching trophies 2026-09-28T07:51:25.398796Z INFO [NetworkStatus] Network status: online (network adapter available, probe pending) 2026-09-28T07:51:20.649376Z INFO [NetworkStatus] Network status: offline (probe failed: ClientException with SocketException: Failed host lookup: 'gym.XXX' (OS Error: No address associated with hostname, errno = 7), uri=https://gym.XXX/api/v2/version/) 2026-09-28T07:50:50.321712Z INFO [PowerSync] Starting Rust sync iteration (repeated 17x since last logged) 2026-09-28T07:49:48.450684Z FINE [PowerSync] Credentials: PowerSyncCredentials<endpoint: https://gym.XXX/sync/ userId: 7 expiresAt: 2026-09-28 07:59:48.450629Z> 2026-09-28T07:49:48.450322Z INFO [powersync-django] Connecting to PowerSync endpoint https://gym.XXX/sync/ 2026-09-28T07:49:48.221317Z FINE [AuthNotifier] refreshAccessToken: success, new expiry 2026-09-28 07:54:47.000Z, rotated refresh token: true 2026-09-28T07:49:47.423058Z FINE [AuthNotifier] refreshAccessToken: starting

  9. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    Where should I add DJANGO_PERFORM_MIGRATIONS ? I don't have it anywhere...

  10. rolandgeider commented on Sep 28, 2026

    @rolandgeider
    rolandgeider
    on Sep 28, 2026
    Member

    Where should I add DJANGO_PERFORM_MIGRATIONS ? I don't have it anywhere...

    it's in the default prod.env config, which one are you using?

  11. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    This one

    # Django core
DJANGO_DEBUG=False
SECRET_KEY=longsecretstring
SIGNING_KEY=XXXX
JWT_PRIVATE_KEY=XXXX==
JWT_PUBLIC_KEY=XXXX==

#SITE_DOMAIN=192.168.1.62:8800
SITE_DOMAIN=gym.XXX

POWER_SYNC_ENABLED=True
POWER_SYNC_URL=https://gym.XXX/sync
POWERSYNC_URL_PATH=sync

SITE_URL=https://gym.XXX
DJANGO_ALLOWED_HOSTS=localhost,127.0.0.1,192.168.1.62,gym.XXX

CSRF_TRUSTED_ORIGINS=http://192.168.1.62:8800,http://localhost:8800,https://gym.XXX

USE_X_FORWARDED_HOST=True
SECURE_PROXY_SSL_HEADER=HTTP_X_FORWARDED_PROTO
X_FORWARDED_PROTO_HEADER_SET=True

# Database
DJANGO_DB_ENGINE=django.db.backends.postgresql
DJANGO_DB_DATABASE=wger
DJANGO_DB_USER=wger
DJANGO_DB_PASSWORD=WgerSync2026
DJANGO_DB_HOST=db
DJANGO_DB_PORT=5432

# Cache
DJANGO_CACHE_BACKEND=django_redis.cache.RedisCache
DJANGO_CACHE_LOCATION=redis://cache:6379/1
DJANGO_CACHE_TIMEOUT=1296000
DJANGO_CACHE_CLIENT_CLASS=django_redis.client.DefaultClient

# Celery
CELERY_BROKER=redis://cache:6379/2
CELERY_BACKEND=redis://cache:6379/2

# File paths
DJANGO_MEDIA_ROOT=/home/wger/media
DJANGO_STATIC_ROOT=/home/wger/static

# App behaviour
ALLOW_REGISTRATION=False
ALLOW_GUEST_USERS=False
USE_RECAPTCHA=False
AXES_ENABLED=False

# Locale
TIME_ZONE=Europe/Rome
LANGUAGE_CODE=it

    How can I fix the above error?

  12. rolandgeider commented on Sep 28, 2026

    @rolandgeider
    rolandgeider
    on Sep 28, 2026
    Member

    the one we provide has all the common (and possible) options, you can then just add one that has the specific overrides you need

      web:
    env_file:
      - ./config/prod.env
      - ./config/local.env # or only this if you use an override file

    As for the OS error, that sometimes happens in android when you put the app in the background and the OS cuts internet access. Does it sync correctly when you are using the app? (that being said, we should perhaps hide that specific error a bit)

  13. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    After I patched the above, the sync started to work and download, but it never completes... now it's at 264791/265028 and it's stuck...

  14. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    I compared your prod.env with mine and this is the result:

    Variable | In your file | In prod.env | Recommendation
-- | -- | -- | --
TZ | Missing | Europe/Berlin | Add Europe/Rome
PS_STORAGE_PG_URI | Missing | Configured | Check Required for PowerSync
PS_PORT | Missing | 8080 | Check
WGER_INSTANCE | Missing | https://wger.de | Add
ALLOW_UPLOAD_VIDEOS | Missing | True | Optional
SYNC_EXERCISES_CELERY | Missing | True | Add
SYNC_EXERCISE_IMAGES_CELERY | Missing | True | Add
SYNC_EXERCISE_VIDEOS_CELERY | Missing | True | Optional
SYNC_INGREDIENTS_CELERY | Missing | True | Optional
DOWNLOAD_INGREDIENTS_FROM | Missing | WGER | Optional
USE_CELERY | Missing | True | Check
CELERY_WORKER_CONCURRENCY | Missing | 4 | Optional
ACCESS_TOKEN_LIFETIME | Missing | 10 | Optional
REFRESH_TOKEN_LIFETIME | Missing | 2880 | Optional
DJANGO_COLLECTSTATIC_ON_STARTUP | Missing | True | Add
WGER_USE_GUNICORN | Missing | True | Optional
NUMBER_OF_PROXIES | Missing | 1 | Check
LOG_LEVEL_PYTHON | Missing | INFO | Optional
EXPOSE_PROMETHEUS_METRICS | Missing | False | Optional
AXES_FAILURE_LIMIT | Missing | 10 | Add
AXES_COOLOFF_TIME | Missing | 30 | Add
AXES_HANDLER | Missing | Cache handler | Add
AXES_LOCKOUT_PARAMETERS | Missing | ip_address | Add

    Any line I need to add?

  15. rolandgeider commented on Sep 28, 2026

    @rolandgeider
    rolandgeider
    on Sep 28, 2026
    Member

    if those are the options that are missing, then you should be set

    As for why your sync doesn't finish (it seems it can't sync the last 237 entries), I have no idea. There's however a new version of both the flutter sdk and the docker image for the service, which from what I've seen from their issues at the very least have better logging. I'll try to push a new version of the app tomorrow or so, before we start searching for this in the logs like in the other issue

  16. Sanji78 commented on Sep 28, 2026

    @Sanji78
    Sanji78
    on Sep 28, 2026
    ContributorAuthor

    I added "proxy_buffering off;
    proxy_request_buffering off;
    proxy_set_header Connection '';
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;" to my ontop nginx proxy manager that routes the traffic to gym.xxxx and it fixed the issue. Now the app connects properly.

  17. rolandgeider commented on Oct 8, 2026

    @rolandgeider
    rolandgeider
    on Oct 8, 2026
    Member

    oh it was the buffering option! We definitely need to put this more prominently in the docs!!

    (closing here, if you encounter further sync errors, just open a new issue)

  18. rolandgeider
    closed this as completedon Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Bug

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions