cvpr2024waterf

Presentation Video

Abstract

The advances in the Neural Radiance Fields (NeRF) research offer extensive applications in diverse domains, but protecting their copyrights has not yet been researched in depth. Recently, NeRF watermarking has been considered one of the pivotal solutions for safely deploying NeRF-based 3D representations. However, existing methods are designed to apply only to implicit or explicit NeRF representations. In this work, we introduce an innovative watermarking method that can be employed in both representations of NeRF. This is achieved by fine-tuning NeRF to embed binary messages in the rendering process. In detail, we propose utilizing the discrete wavelet transform in the NeRF space for watermarking. Furthermore, we adopt a deferred back-propagation technique and introduce a combination with the patch-wise loss to improve rendering quality and bit accuracy with minimum trade-offs. We evaluate our method in three different aspects: capacity, invisibility, and robustness of the embedded watermarks in the 2D-rendered images. Our method achieves state-of-the-art performance with faster training speed over the compared state-of-the-art methods.

Method

Phase 1: We train the encoder and the decoder to extract messages. After training is done, we do not use the encoder. Phase 2: We fine-tune the NeRF to embed the messages into the rendered images automatically. (a) We disable auto-differentiation and render a full-resolution image in order to save memory. (b) We use DWT for the rendered images and choose the LL coefficient as the input of the pre-trained decoder. (c) We enable auto-differentiation and render the images patch by patch.

Experiment Results

Bit accuracies with different message lengths are compared with the state-of-the-art method i.e CopyRNeRF. Our method generally outperforms other methods with state-of-the-art bit-accuracy. HiDDeN + NeRF is pre-embedded before training. The results are the average of LLFF and Blender datasets.

Our method demonstrates a trade-off between bit accuracy and the quality of reconstructed data, as detailed in the Table. Bit accuracies and reconstruction qualities comparision with baselines. ↑ (↓) means higher (lower) is better. We show the results on 16 bits. The results are evaluated in the same way as baselines. The best performances are highlighted in bold.

We show the results on 16 message bits. The best performances are highlighted in bold. We evaluate the robustness of the watermarked image to different attacks by applying Gaussian noise, Rotation, Scaling, Gaussian Blur, Crop, Brightness, JPEG compression and combination of these distortions. As shown in Table, We experiment on both TensoRF and Vanila NeRF when applied with our method, resulting in robustness against various attacks compared to CopyRNeRF.

Even though our proposed method necessitates the use of a pre-trained model’s weights as a starting point for fine-tuning, the time requirement for training is significantly less compared to that of CopyRNeRF. As depicted in Figure, it is evident that the model utilizing our techniques substantially outperforms CopyRNeRF regarding training efficiency.