trunk/1781f63f19fdfb119cc58a9a9b6c6ef0650c7cc4: Codegen backward prologue subclass unwrapping (#178927)
Generate a codegen'd identity unwrap function for the backward prologue's non-tangent args (saved tensors, symints, opaque objects). In AOT dispatch, these are always plain tensors since the compiled forward operates on unwrapped inner tensors, so the codegen eliminates the per-element is_traceable_wrapper_subclass checks from runtime_unwrap_tensor_subclasses. In AOT dispatch, non-tangent backward args (saved tensors, symints, opaque objects) are always plain tensors since the compiled forward operates on unwrapped inner tensors. The codegen'd function is therefore an identity: def unwrap_fn(args): return list(args) This eliminates the per-element is_traceable_wrapper_subclass checks from runtime_unwrap_tensor_subclasses that ran on every backward call. Unwrap step in isolation (us/call):
Generate a codegen'd identity unwrap function for the backward prologue's non-tangent args (saved tensors, symints, opaque objects). In AOT dispatch, these are always plain tensors since the compiled forward operates on unwrapped inner tensors, so the codegen eliminates the per-element is_traceable_wrapper_subclass checks from runtime_unwrap_tensor_subclasses.Generate a codegen'd identity unwrap function for the backward prologue's non-tangent args (saved tensors, symints, opaque objects). In AOT dispatch, these are always plain tensors since the compiled forward operates on unwrapped inner tensors, so the codegen eliminates the per-element is_traceable_wrapper_subclass checks from runtime_unwrap_tensor_subclasses.In AOT dispatch, non-tangent backward args (saved tensors, symints, opaque objects) are always plain tensors since the compiled forward operates on unwrapped inner tensors. The codegen'd function is therefore an identity:
def unwrap_fn(args): return list(args)
This eliminates the per-element is_traceable_wrapper_subclass checks
from runtime_unwrap_tensor_subclasses that ran on every backward call.
Unwrap step in isolation (us/call):
| Case | Before (isinstance loop) | After (codegen identity) | Speedup |
|---|---|---|---|
| 5 args | 0.78 us | 0.14 us | 5.5x |
| 10 args | 1.43 us | 0.15 us | 9.5x |
| 20 args | 2.70 us | 0.16 us | 16.9x |
| 50 args | 6.61 us | 0.20 us | 33.1x |
The codegen'd identity (list(args)) eliminates O(n) isinstance /
is_traceable_wrapper_subclass checks per backward call. Speedup scales
linearly with the number of non-tangent args since the old loop cost is
per-element while the new path is O(1).
Pull Request resolved: https://github.com/pytorch/pytorch/pull/178927
Approved by: https://github.com/aorenste
ghstack dependencies: #178675`
Assets 2
PyTorch Releases
https://github.com/pytorch/pytorch/releases/tag/trunk%2F1781f63f19fdfb119cc58a9a9b6c6ef0650c7cc4Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
github![trunk/5d6292dfff853cd0559300c88d7330752c185e40: [Native DSL] Add torch.backends.python_native (#178902)](https://avatars.githubusercontent.com/u/4992598?s=60&v=4)
trunk/5d6292dfff853cd0559300c88d7330752c185e40: [Native DSL] Add torch.backends.python_native (#178902)
Summary: Adds user-facing control of python_native op overrides defined in torch._native . Allows for: Per-DSL control and information via. torch.backends.python_native.$dsl .name # (property) .available # (property) .enabled # (property, settable) .version # (property) .disable() # (method) .enable() # (method) .disabled() # (context manager) And module-level control via. torch.backends.python_native .available_dsls (property) .all_dsls (property) .get_dsl_operations() (method) .disable_operations() (method) .enable_operations() (method) .disable_dispatch_keys() (method) .enable_dispatch_keys() (method) .operations_disabled() (context) .operations_disabled() (context manager) Tests and docs for this functionality are also added. Test Plan: pytest -sv test/python_native/test_torch_backends
Sources: Mark Zuckerberg is back to writing code after a two-decade hiatus, submitting three diffs to Meta s monorepo, and is a heavy user of Claude Code CLI (Gergely Orosz/The Pragmatic Engineer)
Gergely Orosz / The Pragmatic Engineer : Sources: Mark Zuckerberg is back to writing code after a two-decade hiatus, submitting three diffs to Meta's monorepo, and is a heavy user of Claude Code CLI Mark Zuckerberg and Garry Tan join the trend of C-level folks jumping back into coding with AI. Also: a bad week for Claude Code and GitHub, and more
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
![Billion dollar AI company was built on lies [video]](https://d2xsxph8kpxj0f.cloudfront.net/310419663032563854/konzwo8nGf8Z4uZsMefwMr/default-img-brain-ai-7TDBUd4FYSNsUnz3sSAmvo.webp)
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!