How to Obtain RGB Output in the Code?

[lyz140204842]

Thank you for sharing this code. I noticed that in your code, the inputs and outputs seem to retain only the luminance channel. How can I restore the RGB image in the code?

[seabro917]

Similar question here, I notice that the repo provides dataset with single-channel images for testing (Set11), but not other datasets with RGB images, how can I do the test with RGB images?

[hong460]

Thank you for sharing this code. I noticed that in your code, the inputs and outputs seem to retain only the luminance channel. How can I restore the RGB image in the code?

from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained("sd-legacy/stable-diffusion-v1-5").to(device)
model = DDP(Net(T, pipe.unet).to(device), device_ids=[rank])
model._set_static_graph()

Could you please let me know if you used these lines of code directly? I am unable to load the model with this code. How did you handle it? Please help me

[lyz140204842]

Thank you for sharing this code. I noticed that in your code, the inputs and outputs seem to retain only the luminance channel. How can I restore the RGB image in the code?

from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained("sd-legacy/stable-diffusion-v1-5").to(device) model = DDP(Net(T, pipe.unet).to(device), device_ids=[rank]) model._set_static_graph()

Could you please let me know if you used these lines of code directly? I am unable to load the model with this code. How did you handle it? Please help me

Please ensure that the ‘HF_ENDPOINT=https://hf-mirror.com’ environment variable is correctly configured during runtime,this mirror is typically accessible and in China. If the issue persists after setting it, please provide more detailed error information.

[hong460]

Thank you for sharing this code. I noticed that in your code, the inputs and outputs seem to retain only the luminance channel. How can I restore the RGB image in the code?

from diffusers import StableDiffusionPipeline pipe = StableDiffusionPipeline.from_pretrained("sd-legacy/stable-diffusion-v1-5").to(device) model = DDP(Net(T, pipe.unet).to(device), device_ids=[rank]) model._set_static_graph()
Could you please let me know if you used these lines of code directly? I am unable to load the model with this code. How did you handle it? Please help me

Please ensure that the ‘HF_ENDPOINT=https://hf-mirror.com’ environment variable is correctly configured during runtime,this mirror is typically accessible and in China. If the issue persists after setting it, please provide more detailed error information.

(IDM) PS D:\model\IDM-main> python test.py
D:\model\IDM-main\backprop.py:41: FutureWarning: torch.cuda.amp.custom_fwd(args...) is deprecated. Please use torch.amp.custom_fwd(args..., device_type='cuda') instead.
def apply(x, layers):
D:\model\IDM-main\backprop.py:50: FutureWarning: torch.cuda.amp.custom_fwd(args...) is deprecated. Please use torch.amp.custom_fwd(args..., device_type='cuda') instead.
def forward(ctx, x, layers):
D:\model\IDM-main\backprop.py:61: FutureWarning: torch.cuda.amp.custom_bwd(args...) is deprecated. Please use torch.amp.custom_bwd(args..., device_type='cuda') instead.
def backward(ctx, dx):
cs ratio = 0.1
Couldn't connect to the Hub: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/sd-legacy/stable-diffusion-v1-5 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000014A895E1100>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。'))"), '(Reques4e417fe)').
Will try to load from local cache.
Traceback (most recent call last):
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connection.py", line 198, in _new_conn
sock = connection.create_connection(
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
raise err
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
response = self._make_request(
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connectionpool.py", line 488, in _make_request
raise new_e
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connectionpool.py", line 464, in _make_request
self._validate_conn(conn)
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connectionpool.py", line 1093, in _validate_conn
conn.connect()
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connection.py", line 704, in connect
self.sock = sock = self._new_conn()
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connection.py", line 213, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x0000014A895E1100>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主 机没有反应，连接尝试失败。

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\anaconda\envs\IDM\lib\site-packages\requests\adapters.py", line 667, in send
resp = conn.urlopen(
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\connectionpool.py", line 841, in urlopen
retries = retries.increment(
File "D:\anaconda\envs\IDM\lib\site-packages\urllib3\util\retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/sd-legacy/stable-diffusion-v1-5 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000014A895E1100>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\anaconda\envs\IDM\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1291, in download
info = model_info(pretrained_model_name, token=token, revision=revision)
File "D:\anaconda\envs\IDM\lib\site-packages\huggingface_hub\utils_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "D:\anaconda\envs\IDM\lib\site-packages\huggingface_hub\hf_api.py", line 2518, in model_info
r = get_session().get(path, headers=headers, timeout=timeout, params=params)
File "D:\anaconda\envs\IDM\lib\site-packages\requests\sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "D:\anaconda\envs\IDM\lib\site-packages\requests\sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "D:\anaconda\envs\IDM\lib\site-packages\requests\sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "D:\anaconda\envs\IDM\lib\site-packages\huggingface_hub\utils_http.py", line 96, in send
return super().send(request, *args, **kwargs)
File "D:\anaconda\envs\IDM\lib\site-packages\requests\adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/sd-legacy/stable-diffusion-v1-5 (Caused by NewConnection
Error('<urllib3.connection.HTTPSConnection object at 0x0000014A895E1100>: Failed to establish a new connection: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。'))"), '(Request ID: df2fca20-706b-452c-aa10-e480d4e417fe)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\model\IDM-main\test.py", line 42, in
pipe = StableDiffusionPipeline.from_pretrained("sd-legacy/stable-diffusion-v1-5").to(device)
File "D:\anaconda\envs\IDM\lib\site-packages\huggingface_hub\utils_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "D:\anaconda\envs\IDM\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 699, in from_pretrained
cached_folder = cls.download(
File "D:\anaconda\envs\IDM\lib\site-packages\huggingface_hub\utils_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "D:\anaconda\envs\IDM\lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1536, in download
raise EnvironmentError(
OSError: Cannot load model sd-legacy/stable-diffusion-v1-5: model is not cached locally and an error occurred while trying to fetch metadata from the Hub. Please check out the root cause in the stacktrace above.

你好，这是我添加完环境变量之后的报错，仍然是无法从远端加载模型。请你帮我看一下这是什么原因

[Guaishou74851]

@lyz140204842 @seabro917 Thanks for your interest in our work!

Currently, this repo focuses primarily on the Y-channel for compressive sensing tasks, and we haven't included direct support for RGB images yet. If you'd like to experiment with RGB images, one approach is to reshape the RGB data from the format (b, 3, h, w) into (b*3, 1, h, w), treating each RGB channel as a separate grayscale batch. This way, you can directly apply the existing compressive sensing pipeline (sampling matrix multiplication and transpose operation) without major changes.

Unfortunately, due to our current workload and limited resources, we don't have plans to extend and release dedicated RGB support in the near future. But please feel free to explore the approach mentioned above, and we'll be happy to help if you have further questions.

Thanks again for your understanding!

[mlkk518]

@lyz140204842 @seabro917 Thanks for your interest in our work!

Currently, this repo focuses primarily on the Y-channel for compressive sensing tasks, and we haven't included direct support for RGB images yet. If you'd like to experiment with RGB images, one approach is to reshape the RGB data from the format (b, 3, h, w) into (b*3, 1, h, w), treating each RGB channel as a separate grayscale batch. This way, you can directly apply the existing compressive sensing pipeline (sampling matrix multiplication and transpose operation) without major changes.

Unfortunately, due to our current workload and limited resources, we don't have plans to extend and release dedicated RGB support in the near future. But please feel free to explore the approach mentioned above, and we'll be happy to help if you have further questions.

Thanks again for your understanding!

Thanks for making your code publicly available. In your paper you report results related to RGB images, and here you say that your code does not support RGB maps. I wonder how you arrived at these results for RGB images in your paper?