Generate seed file presigned URL in crawl operator, update prior to expiry - #3600

Open

tw4l wants to merge 11 commits into

mainfrom

issue-3576-seed-file-presigned-url

Commits on Aug 25, 2026

Generate new seed file presigned URL on each configmap load
tw4l
committed
8ab8d1e
Temp: Add working TODOs to reference later
tw4l
committed
44ead67
Only update configmap and get new presigned if crawler restarts/resumes
tw4l
committed
698c965
Switch to snake_case
tw4l
committed
f0fb2c9
Set status.update_seed_file_presigned after any crawl pod exited
tw4l
committed
0d45250
Pass seed file id for scheduled crawls as well
tw4l
committed
a7d6e08
Regenerate seed file presigned url when close to expiring
tw4l
committed
e21e9ea
Make get_public_file_out consistent with get_file_out
tw4l
committed
b14699f
Change to one hour
tw4l
committed
087d167
Add more logging
tw4l
committed
c3e9681
Remove temporary debug logging
tw4l
committed
eebfd88