Repository navigation
ptodsl: support Python and/or short-circuit in AST rewrite (#1332) #1346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
bc2a295
e89bb34
7ea21a6
965a615
5d10b71
62eaf58
cdcfa87
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -29,8 +29,10 @@ def rewrite_jit_function( | |
| ): | ||
| """Return a function with PTODSL lexical sections lowered safely. | ||
|
|
||
| ``pto.section`` is a physical SSA region, not a Python ``with`` hint. The | ||
| section body therefore gets a small source-level lexical rewrite even when | ||
| ``rewrite_control_flow`` controls both runtime statement rewriting and the | ||
| ``and``/``or`` short-circuit rewrite. ``pto.section`` is a physical SSA | ||
| region, not a Python ``with`` hint. Its body therefore gets a small | ||
| source-level lexical rewrite even when | ||
| the optional control-flow rewrite is disabled. This keeps Python's | ||
| function-local assignment rules from leaking a section-local SSA value into | ||
| a sibling physical section. | ||
|
|
@@ -68,6 +70,8 @@ def rewrite_jit_function( | |
| static_env.update(static_bindings or {}) | ||
| _inject_closure_defaults(function_def, closure_vars.nonlocals) | ||
| _sanitize_signature_for_exec(function_def) | ||
| if rewrite_control_flow: | ||
| function_def = _BoolOpRewriter().visit(function_def) | ||
| function_def = _ConditionalExpressionNormalizer().visit(function_def) | ||
| section_rewriter = _SectionLexicalRewriter() | ||
| function_def = section_rewriter.visit(function_def) | ||
|
|
@@ -381,6 +385,77 @@ def _normalize_ifexp_assignment(self, stmt, value): | |
| return ast.copy_location(self.generic_visit(if_stmt), stmt) | ||
|
|
||
|
|
||
| def _zero_arg_lambda(body): | ||
| return ast.Lambda( | ||
| args=ast.arguments( | ||
| posonlyargs=[], | ||
| args=[], | ||
| vararg=None, | ||
| kwonlyargs=[], | ||
| kw_defaults=[], | ||
| kwarg=None, | ||
| defaults=[], | ||
| ), | ||
| body=body, | ||
| ) | ||
|
|
||
|
|
||
| class _BoolOpRewriter(ast.NodeTransformer): | ||
| """Rewrite Python ``and``/``or`` into device-side short-circuit helpers. | ||
|
|
||
| Python ``and``/``or`` are not overloadable operators: evaluating ``a and b`` | ||
| forces a truth check on ``a``, which calls ``__bool__`` on a PTODSL runtime | ||
| value during tracing and raises. This pass replaces every ``ast.BoolOp`` | ||
| with a right-nested lazy helper call so the RHS is only traced inside a | ||
| device-side ``scf.if`` region:: | ||
|
|
||
| a and b and c -> pto._short_circuit_and(a, lambda: pto._short_circuit_and(b, lambda: c)) | ||
| a or b or c -> pto._short_circuit_or(a, lambda: pto._short_circuit_or(b, lambda: c)) | ||
|
|
||
| The transformation is purely syntactic and composes with every expression | ||
| context: assignments, call arguments, ``return``, and ``if``/``while`` | ||
| conditions. Nested function bodies are rewritten as well, mirroring the | ||
| control-flow rewriter. Statically-known ``bool``/``int`` operands are | ||
| short-circuited at trace time by the helpers themselves, so the RHS is not | ||
| traced at all when Python semantics would skip it. | ||
| """ | ||
|
|
||
| @staticmethod | ||
| def _reject_rhs_assignment_expressions(node): | ||
| """Reject walrus expressions that would move into a generated lambda.""" | ||
| if any( | ||
| isinstance(child, ast.NamedExpr) | ||
| for value in node.values[1:] | ||
| for child in ast.walk(value) | ||
| ): | ||
| raise PTODSLAstRewriteError( | ||
| "ast_rewrite=True cannot rewrite an and/or expression containing " | ||
| "an assignment expression (walrus operator) on the RHS; the " | ||
| "assignment would bind inside the generated helper lambda" | ||
| ) | ||
|
|
||
| def visit_BoolOp(self, node): | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 缺陷:海象表达式(
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 已修复。现在会检测短路表达式 RHS 中的 ast.NamedExpr,并明确抛出 PTODSLAstRewriteError |
||
| self._reject_rhs_assignment_expressions(node) | ||
| node = self.generic_visit(node) | ||
| if isinstance(node.op, ast.And): | ||
| helper = "_short_circuit_and" | ||
| elif isinstance(node.op, ast.Or): | ||
| helper = "_short_circuit_or" | ||
| else: | ||
| return node | ||
| values = list(node.values) | ||
| if len(values) < 2: | ||
| return values[0] if values else node | ||
| result = values[-1] | ||
| for value in reversed(values[:-1]): | ||
| result = ast.Call( | ||
| func=_pto_attr(helper), | ||
| args=[value, _zero_arg_lambda(result)], | ||
| keywords=[], | ||
| ) | ||
| return result | ||
|
|
||
|
|
||
| @dataclass(frozen=True) | ||
| class _NameInfo: | ||
| loads: set[str] | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
覆盖缺口:
_BoolOpRewriter挂在rewrite_control_flow开关下。如果存在ast_rewrite=True但rewrite_control_flow=False的使用方式,and/or仍会在 trace 时触发__bool__并抛出旧的不透明错误。请确认该组合不存在/非法;否则建议把 BoolOp 改写挂到 ast_rewrite 总开关下(它不依赖控制流重写也能正确工作)。There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已确认当前不存在 ast_rewrite=True、rewrite_control_flow=False 的独立公开组合。rewrite_control_flow 同时控制语句控制流和 and/or 重写;