LoadImaged for 2D/3D grayscale tiff files

[perlmutter]

Using LoadImaged on a 2D grayscale tiff file with WSIReader and backend=cucim returns an array with 3 "color" channels, which are all identical.

For example,

data_dict = {"image": "test.tif"}
x = np.random.randint(low=0,high=256,size=(128,128),dtype=np.uint8)
io.imsave(data_dict["image"],x) 

print("cucim: shape = " + str(cucim.CuImage(data_dict["image"]).shape))
transform = LoadImaged(keys=["image"], reader=WSIReader, backend='cuCIM')
print("LoadImaged-cucim: shape = " + str(transform(data_dict)["image"].shape))

returns,

cucim: shape = [128, 128, 1]
LoadImaged-cucim: shape = (3, 128, 128)

When using backend=tifffile the files are misread entirely

print("tifffule: shape = " + str(tifffile.imread(data_dict["image"]).shape))
transform = LoadImaged(keys=["image"], reader=WSIReader, backend='tifffile')
print("LoadImaged-tifffile: shape = " + str(transform(data_dict)["image"].shape))

returns,

tifffule: shape = (128, 128)
LoadImaged-tifffile: shape = (3, 128)

Is this the expected behavior? Is there a better way to read 2D grayscale tiff files into monai? How about 3D grayscale tiffs?

Thanks,
David

[vikashg]

Hi David,
Can you please tell me which version of Monai are you using? I have a bunch of .svs file and I am trying to use the LoadImaged with backend cuCIM and I get the following the error

 Please install the reader libraries, see also the installation instructions:
    https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies.
   The current registered: [<monai.data.image_reader.NumpyReader object at 0x7f47313ebbe0>, <monai.data.image_reader.PILReader object at 0x7f4731397550>, <monai.data.image_reader.WSIReader object at 0x7f4731391670>].
Traceback (most recent call last):
  File "/workspace/MONAI/monai/transforms/io/array.py", line 211, in __call__
    img = reader.read(filename)
  File "/workspace/MONAI/monai/data/image_reader.py", line 785, in read
    img = self.wsi_reader(name, **kwargs_)
ValueError: Cannot load /workspace/data/train_and_validation_data/Train/WSIs/Cancerous/DCIS/BRACS_1999.svs!

Traceback (most recent call last):
  File "/workspace/MONAI/monai/transforms/io/array.py", line 211, in __call__
    img = reader.read(filename)
  File "/workspace/MONAI/monai/data/image_reader.py", line 662, in read
    img = PILImage.open(name, **kwargs_)
TypeError: open() got an unexpected keyword argument 'backend'

Traceback (most recent call last):
  File "/workspace/MONAI/monai/transforms/io/array.py", line 211, in __call__
    img = reader.read(filename)
  File "/workspace/MONAI/monai/data/image_reader.py", line 572, in read
    img = np.load(name, allow_pickle=True, **kwargs_)
TypeError: load() got an unexpected keyword argument 'backend'

[Plugin: cucim.kit.cuslide] Unloaded.
[Plugin: cucim.kit.cumed] Unloaded.

For some reason it goes to read the PIL Reader.

[perlmutter]

moani version 0.8.0

I got the same error messages before specifying reader=WSIReader and backend=... following this documentation: #3161

Perhaps WSIReader is not appropriate for grayscale tiffs, but as far as I can tell it's the only reader that opens tiffs at all. Would be great to have a default "tifffilereader" option for LoadReader that can read 2D/3D/grayscale/color tiffs

[wyli]

this is partly a cucim issue:

import numpy as np                                                                                                                                                                                                                                                                                                                                                                                                                                                                          
import cucim                                                                                                                                                                                                                                     
from skimage import io                                                                                                                                                                                                                           
                                                                                                                                                                                                                                                 
data_dict = {"image": "test.tif"}                                                                                                                                                                                                                
x = np.random.randint(low=0, high=256, size=(128, 128), dtype=np.uint8)                                                                                                                                                                          
io.imsave(data_dict["image"], x)                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                 
region = cucim.CuImage("test.tif").read_region(location=(0, 0), size=(128, 128), level=0)                                                                                                                                                        
print(region.shape) 

log:

[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/cucim.kit.cuslide@22.02.00.so
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cumed] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/cucim.kit.cumed@22.02.00.so
Initializing plugin: cucim.kit.cumed (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cumed)
[128, 128, 4]
[Plugin: cucim.kit.cuslide] Unloaded.
[Plugin: cucim.kit.cumed] Unloaded.

cc @gigony

[wyli]

this doesnt work with the input without a channel dim,

MONAI/monai/data/image_reader.py

Lines 922 to 924 in d6327a7

	# remove alpha channel if exist (RGBA)
	if raw_region.shape[-1] > 3:
	raw_region = raw_region[..., :3]

could you please help @drbeh, thanks in advance!

[bhashemian]

Hi @perlmutter,

The intended use ofWSIReader and its backends (cuCIM and OpenSlide) are not for loading any arbitrary array in a TIFF file but for Whole Slide Images in pathology (which are 2D colored images).
First of all, as @wyli mentioned, you need to have a channel dimension. Otherwise, the backends may generate different outputs:

• cuCIM will add the new dimension by repeating values (@gigony could you please confirm this?),
• Tifffile will not add any channel dimension so WSIReader consider the last dim as the channel dim and will truncate it to 3 (to have RGB image),
• OpenSlide will fail to load the image.

Second of all, how do you have grayscale images? I have never encountered a grayscale WSI image, could you please explain the use case so we can see if we should support it here of elsewhere?

[vikashg]

Hello @drbeh

• cuCIM will add the new dimension by repeating values (@gigony could you please confirm this?),

I dont think its true that cucim will repeat values, because a WSI is an RGB image. If we were to repeat values it will be a gray scale image. Also thanks to your (#3853 (comment)) I was able to read the WSI and I tested the values in all 3 channels at random locations and they are all different.

[bhashemian]

Hello @drbeh

• cuCIM will add the new dimension by repeating values (@gigony could you please confirm this?),

I dont think its true that cucim will repeat values, because a WSI is an RGB image. If we were to repeat values it will be a gray scale image. Also thanks to your (#3853 (comment)) I was able to read the WSI and I tested the values in all 3 channels at random locations and they are all different.

Right, it doesn't do that if you provide an RGB TIFF image. In that case, all the three backends should return a 3D ndarray, representing an RGB 2D image.

Glad that it worked!

[perlmutter]

Hi @perlmutter,

The intended use ofWSIReader and its backends (cuCIM and OpenSlide) are not for loading any arbitrary array in a TIFF file but for Whole Slide Images in pathology (which are 2D colored images). First of all, as @wyli mentioned, you need to have a channel dimension. Otherwise, the backends may generate different outputs:

• cuCIM will add the new dimension by repeating values (@gigony could you please confirm this?),
• Tifffile will not add any channel dimension so WSIReader consider the last dim as the channel dim and will truncate it to 3 (to have RGB image),
• OpenSlide will fail to load the image.

Second of all, how do you have grayscale images? I have never encountered a grayscale WSI image, could you please explain the use case so we can see if we should support it here of elsewhere?

@drbeh , you're right, my application is not whole slide imaging. I work with microCT data, at a user-facility for scientific applications: https://als.lbl.gov/beamlines/8-3-2/. We don't use medical imaging formats like dicom. Instead we store raw data in hdf5, but typically give users reconstructions in tiffs (either 3D tiffs or individual 2D slices). Would very much appreciate general tiff support in monai.

[bhashemian]

@drbeh , you're right, my application is not whole slide imaging. I work with microCT data, at a user-facility for scientific applications: https://als.lbl.gov/beamlines/8-3-2/. We don't use medical imaging formats like dicom. Instead we store raw data in hdf5, but typically give users reconstructions in tiffs (either 3D tiffs or individual 2D slices). Would very much appreciate general tiff support in monai.

@perlmutter, I believe we already support functionality via ITK. You should be able to use ITKReader for your use case. For your example above you can change it to:

import numpy as np                                                                                                                                                                                                                           
from skimage import io
from monai.transforms import LoadImaged
from monai.data.image_reader import ITKReader       

data_dict = {"image": "test.tif"}
x = np.random.randint(low=0,high=256,size=(128,128),dtype=np.uint8)
io.imsave(data_dict["image"],x) 

transformD = LoadImaged(keys=["image"], reader=ITKReader)
image = transformD(data_dict)["image"]
print(f"ITKReader: image shape = {image.shape}")

ITKReader: image shape = (128, 128)

[wyli]

perhaps in the wsireader it's useful to raise a runtime error when the input doesn't have the required colour channels?