The Surprising Effectiveness of Noise Pretraining for Implicit Neural Representations
arXiv:2603.29034v1 Announce Type: cross Abstract: The approximation and convergence properties of implicit neural representations (INRs) are known to be highly sensitive to parameter initialization strategies. While several data-driven initialization methods demonstrate significant improvements over standard random sampling, the reasons for their success -- specifically, whether they encode classical statistical signal priors or more complex features -- remain poorly understood. In this study, we explore this phenomenon through a series of experimental analyses leveraging noise pretraining. We pretrain INRs on diverse noise classes (e.g., Gaussian, Dead Leaves, Spectral) and measure their ability to both fit unseen signals and encode priors for an inverse imaging task (denoising). Our anal
View PDF HTML (experimental)
Abstract:The approximation and convergence properties of implicit neural representations (INRs) are known to be highly sensitive to parameter initialization strategies. While several data-driven initialization methods demonstrate significant improvements over standard random sampling, the reasons for their success -- specifically, whether they encode classical statistical signal priors or more complex features -- remain poorly understood. In this study, we explore this phenomenon through a series of experimental analyses leveraging noise pretraining. We pretrain INRs on diverse noise classes (e.g., Gaussian, Dead Leaves, Spectral) and measure their ability to both fit unseen signals and encode priors for an inverse imaging task (denoising). Our analyses on image and video data reveal a surprising finding: simply pretraining on unstructured noise (Uniform, Gaussian) dramatically improves signal fitting capacity compared to all other baselines. However, unstructured noise also yields poor deep image priors for denoising. In contrast, we also find that noise with the classic $1/|f^\alpha|$ spectral structure of natural images achieves an excellent balance of signal fitting and inverse imaging capabilities, performing on par with the best data-driven initialization methods. This finding enables more efficient INR training in applications lacking sufficient prior domain-specific data. For more details, visit project page at this https URL
Comments: Accepted to CVPR 2026. Project page: this https URL
Subjects:
Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
Cite as: arXiv:2603.29034 [cs.CV]
(or arXiv:2603.29034v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2603.29034
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Kushal Vyas [view email] [v1] Mon, 30 Mar 2026 22:01:00 UTC (38,220 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
trainingannounceapplication
XR is XR: Rethinking MR and XR as Neutral Umbrella Terms
arXiv:2603.29939v1 Announce Type: new Abstract: The term XR is currently widely used as an expression encompassing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). However, there is no clear consensus regarding its origin or meaning. XR is sometimes explained as an abbreviation for Extended Reality, but multiple interpretations exist regarding its etymology and formation process. This paper organizes the historical formation of terminology related to VR, AR, MR, and XR, and reexamines the context in which the term XR emerged and how it has spread. In particular, by presenting a timeline that distinguishes between the coinage of terms and the drivers of their adoption, we suggest that XR, as an umbrella term, functions not as an abbreviation of Extended Reality, but rat
Fisher Information Limits of Satellite RF Fingerprint Identifiability for Authentication
arXiv:2603.29766v1 Announce Type: new Abstract: RF fingerprinting authenticates satellite transmitters by exploiting hardware-specific signal impairments, yet existing methods operate without theoretical performance guarantees. We derive the Fisher information matrix (FIM) for joint estimation of in-phase/quadrature (IQ) imbalance and power amplifier (PA) nonlinearity parameters, establishing Cram\'{e}r-Rao bounds (CRBs) whose structure depends on constellation moments. A necessary condition for full IQ identifiability is that the identifiability factor~$\beta$ exceeds zero; for binary phase-shift keying (BPSK), $\beta = 0$ yields a rank-deficient FIM, rendering IQ parameters unidentifiable. This provides a plausible theoretical explanation for OrbID's near-random performance (area under t
Think Anywhere in Code Generation
arXiv:2603.29957v1 Announce Type: new Abstract: Recent advances in reasoning Large Language Models (LLMs) have primarily relied on upfront thinking, where reasoning occurs before final answer. However, this approach suffers from critical limitations in code generation, where upfront thinking is often insufficient as problems' full complexity only reveals itself during code implementation. Moreover, it cannot adaptively allocate reasoning effort throughout the code generation process where difficulty varies significantly. In this paper, we propose Think-Anywhere, a novel reasoning mechanism that enables LLMs to invoke thinking on-demand at any token position during code generation. We achieve Think-Anywhere by first teaching LLMs to imitate the reasoning patterns through cold-start training
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Products
Active Inference with People: a general approach to real-time adaptive experiments
arXiv:2603.29003v1 Announce Type: cross Abstract: Adaptive experiments automatically optimize their design throughout the data collection process, which can bring substantial benefits compared to conventional experimental settings. Potential applications include, among others: computerized adaptive testing (for selecting informative tasks in ability measurements), adaptive treatment assignment (when searching experimental conditions maximizing certain outcomes), and active learning (for choosing optimal training data for machine learning algorithms). However, implementing these techniques in real time poses substantial computational and technical challenges. Additionally, despite their conceptual similarity, the above scenarios are often treated as separate problems with distinct solutions
Machine Learning in the Wild: Early Evidence of Non-Compliant ML-Automation in Open-Source Software
arXiv:2603.29698v1 Announce Type: new Abstract: The increasing availability of Machine Learning (ML) models, particularly foundation models, enables their use across a range of downstream applications, from scenarios with missing data to safety-critical contexts. This, in principle, may contravene not only the models' terms of use, but also governmental principles and regulations. This paper presents a preliminary investigation into the use of ML models by 173 open-source projects on GitHub, spanning 16 application domains. We evaluate whether models are used to make decisions, the scope of these decisions, and whether any post-processing measures are taken to reduce the risks inherent in fully autonomous systems. Lastly, we investigate the models' compliance with established terms of use.
Deep Learning-Assisted Improved Differential Fault Attacks on Lightweight Stream Ciphers
arXiv:2603.29382v1 Announce Type: new Abstract: Lightweight cryptographic primitives are widely deployed in resource-constraint environment, particularly in the Internet of Things (IoT) devices. Due to their public accessibility, these devices are vulnerable to physical attacks, especially fault attacks. Recently, deep learning-based cryptanalytic techniques have demonstrated promising results; however, their application to fault attacks remains limited, particularly for stream ciphers. In this work, we investigate the feasibility of deep learning assisted differential fault attack on three lightweight stream ciphers, namely ACORNv3, MORUSv2 and ATOM, under a relaxed fault model, where a single-bit bit-flipping fault is injected at an unknown location. We train multilayer perceptron (MLP)
Uncovering Relationships between Android Developers, User Privacy, and Developer Willingness to Reduce Fingerprinting Risks
arXiv:2603.29063v1 Announce Type: new Abstract: The major mobile platforms, Android and iOS, have introduced changes that restrict user tracking to improve user privacy, yet apps continue to covertly track users via device fingerprinting. We study the opportunity to improve this dynamic with a case study on mobile fingerprinting that evaluates developers' perceptions of how well platforms protect user privacy and how developers perceive platform privacy interventions. Specifically, we study developers' willingness to make changes to protect users from fingerprinting and how developers consider trade-offs between user privacy and developer effort. We do this via a survey of 246 Android developers, presented with a hypothetical Android change that protects users from fingerprinting at the co
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!