question about dataset

[xiaozibuqi]

I would like to run the algorithm on my own dataset, but my event voxels are accumulated directly over time periods and not represented by SCER in EFNet, in which case there is a problem with the calculation of res in the source code right?

[chenkang455]

Hi @xiaozibuqi, thank you so much for your interest in our work!

If you'd like to run the algorithm on your own dataset, you can refer to the implementation of SCER on the REBlur dataset:

def get_voxel_SCER(self, index, num_bin, dynamic=False, keypoints=None):
    """
    Get SCER voxel
    Reference: Event-Based Fusion for Motion Deblurring with Cross-modal Attention
    Convert event streams to a [num_bin, H, W] event tensor.
    """
    if not hasattr(self, 'data_h5'):
        self.open_hdf5()
    if dynamic and keypoints is None:
        raise ValueError("Keypoints are required when dynamic is True.")
        
    img_size, start_time, end_time = self.extract_time_from_index(index)
    E, _ = event2tensor_time(self.events, img_size, start_time, end_time, num_bin, voxel_type='EFNet')
    voxel = E[:, 0, :, :] - E[:, 1, :, :]
    
    re_voxel = np.zeros_like(voxel)
    left_voxel = voxel[:num_bin // 2, :, :]
    right_voxel = voxel[num_bin // 2:, :, :]
    
    right_voxel_sum = np.cumsum(right_voxel, axis=0)
    left_voxel = left_voxel[::-1]
    left_voxel_sum = np.cumsum(left_voxel, axis=0)
    left_voxel_sum = left_voxel_sum[::-1]
    
    re_voxel[:num_bin // 2, :, :] = -left_voxel_sum
    re_voxel[num_bin // 2:, :, :] = right_voxel_sum
    
    return re_voxel

Additionally, if you decide to use accumulated event voxels over time periods instead, the calculation of res in the source code should work correctly, but note that it may lead to lower performance. SCER is a more suitable representation in our work for accurately estimating residuals.

[xiaozibuqi]

thanks,I'll try it again

[chenkang455]

thanks,I'll try it again

Welcome ask me further qs.

[xiaozibuqi]

I noticed in the code: for i in range(len(sharp_image)): sharp_image_g.append(colour.rgb2gray(sharp_image[i]))
Is there multiple sharp images for a blurry image in your dataset?And in my own dataset there is only one corresponding GT for a blurry image

[chenkang455]

I noticed in the code: for i in range(len(sharp_image)): sharp_image_g.append(colour.rgb2gray(sharp_image[i]))

Is there multiple sharp images for a blurry image in your dataset?And in my own dataset there is only one corresponding GT for a blurry image

Yes, that is correct. Our model requires a sequence of sharp images per blurry one and it can not be trained on the dataset with only one gt frame.

[chenkang455]

I noticed in the code: for i in range(len(sharp_image)): sharp_image_g.append(colour.rgb2gray(sharp_image[i]))

Is there multiple sharp images for a blurry image in your dataset?And in my own dataset there is only one corresponding GT for a blurry image

Despite it, you can try comment the loss function for supervising the residual sequence training and leave the loss for the central frame supervision. However, this process will deteriorate the restoration performance and the estimated residual sequence is meaningless.

[chenkang455]

I would like to run the algorithm on my own dataset, but my event voxels are accumulated directly over time periods and not represented by SCER in EFNet, in which case there is a problem with the calculation of res in the source code right?

Hi, we provide a simple implementation of our method in event_deblur.py，where you can learn how to convert the raw events to the SCER representation and how to deblur the image with the assistance of estimated residual sequence.