Skip to content

Offload trigger inputs at registration#1176

Merged
pingsutw merged 20 commits into
mainfrom
katrina/eng26-531-sdk-create-trigger-implement-selectcluster-flow-to-offload
Jun 12, 2026
Merged

Offload trigger inputs at registration#1176
pingsutw merged 20 commits into
mainfrom
katrina/eng26-531-sdk-create-trigger-implement-selectcluster-flow-to-offload

Conversation

@katrogan

@katrogan katrogan commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

And read kickoff time input from the run context, using the newly added run_start_time from #1100

Depends on flyteorg/flyte#7482

Testing
[x] regular trigger (TaskTriggerSpec) Ran https://github.com/flyteorg/flyte-sdk/blob/main/examples/triggers/basic.py and verified output included valid kickoff time
[x] deployed trigger (TriggerSpec) Using

import flyte
from flyte.remote import Trigger

CONFIG = ".flyte/config-oss-local.yaml"
TASK_NAME = "cached_example_task.cached_task"

TRIGGER_NAME = "standalone_minutely"  # distinct from the embedded trigger names

flyte.init_from_config(CONFIG)

Trigger.create(
    trigger=flyte.Trigger.minutely("start_time", name=TRIGGER_NAME),
    task_name=TASK_NAME,
)

print(f"Created standalone trigger '{TRIGGER_NAME}' on task '{TASK_NAME}'.")

[x] cached trigger (TaskTriggerSpec) Using this script

import flyte

env = flyte.TaskEnvironment(
    name="cached_example_task",
    env_vars={"_U_USE_ACTIONS": "1"},
)


@env.task(
    cache=flyte.Cache(behavior="auto", ignored_inputs="start_time"),
    triggers=flyte.Trigger.minutely("start_time", name="minutely_cached"),
)
def cached_task(start_time: datetime, x: int = 1) -> str:
    return f"Cached task executed at {start_time.isoformat()} with x={x}"


if __name__ == "__main__":
    flyte.init_from_config()
    flyte.deploy(env)

And verified subsequent runs used cached output from the first run.

[x] run time always injected: Using this script

import flyte

env = flyte.TaskEnvironment(
    name="hello_run_start_time",
)


@env.task()
async def hello_run_start_time() -> str:
    ctx = flyte.ctx()
    assert ctx is not None
    # run_start_time is populated by the backend from the run's start time (for scheduled-trigger
    # runs, the trigger's fire time) via the {{.runStartTime}} container arg. For ad-hoc runs it
    # defaults to the run-creation time.
    run_start_time = ctx.run_start_time
    print(f"Run start time: {run_start_time.isoformat()}")
    return run_start_time.isoformat()


if __name__ == "__main__":
    flyte.init_from_config()

    run = flyte.run(hello_run_start_time)
    print(run.name)
    print(run.url)
    run.wait()

Verified individual runs output the kickoff time

ref eng26-533

katrogan added 4 commits June 8, 2026 11:08
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Comment thread pyproject.toml Outdated
"aiolimiter>=1.2.1",
"flyteidl2==2.0.20",
# LOCAL DEV: pinned to local gen/python via [tool.uv.sources] below. Restore "flyteidl2==2.0.20" before merge.
"flyteidl2",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will update to an official release version before merge

Comment thread examples/basics/hello_run_start_time.py Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll revert all these examples before merge

@pingsutw pingsutw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread src/flyte/_internal/runtime/trigger_serde.py Outdated
katrogan added 7 commits June 9, 2026 09:58
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
@katrogan katrogan marked this pull request as ready for review June 10, 2026 08:59
katrogan added 2 commits June 10, 2026 11:00
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
pingsutw
pingsutw previously approved these changes Jun 10, 2026
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
pingsutw
pingsutw previously approved these changes Jun 11, 2026
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
pingsutw
pingsutw previously approved these changes Jun 11, 2026
await controller.finalize_parent_action(tctx.action)


def _inject_kickoff_time_from_run_start(inputs: Inputs, run_start_time: datetime) -> Inputs:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we not do it here and instead do it at -

literals = {named_literal.name: named_literal.value for named_literal in inputs.proto_inputs.literals}

you have it in the ctx already so you can just get it from the ctx internal_ctx or task context

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, done


# Offloaded triggers carry the kickoff-time input arg name in inputs.context; fill that input
# from run_start_time (the scheduled fire time) before native conversion. No-op otherwise.
if inputs is not None and run_start_time is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this part, we dont need to reopen the inputs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread src/flyte/remote/_trigger.py Outdated
# on the trigger spec. At fire time the backend passes this straight through as the run's
# OffloadedInputData and stamps run_start_time, instead of inlining a kickoff literal.
# The task was just fetched above and is already registered, so we reference it by task_id.
offloaded_input_data = await trigger_serde.offload_trigger_inputs(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only if we have data?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
katrogan and others added 3 commits June 12, 2026 08:54
Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
@pingsutw pingsutw merged commit 7f705ef into main Jun 12, 2026
40 checks passed
@pingsutw pingsutw deleted the katrina/eng26-531-sdk-create-trigger-implement-selectcluster-flow-to-offload branch June 12, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants