Skip to content

Correct Resize pipeline #211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Correct Resize pipeline #211

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
b1f972b
Update augmentation.py
makecent Jan 9, 2022
159e5ce
Update augmentation.py
makecent Jan 9, 2022
b0ab0dd
Update lsun-car_pad_512.py
makecent Jan 10, 2022
e98d5b8
Update imagenet_rgb.py
makecent Jan 10, 2022
12568ad
Update imagenet_rgb.py
makecent Jan 10, 2022
41ea798
Update augmentation.py
makecent Jan 10, 2022
1246986
Update augmentation.py
makecent Jan 10, 2022
6a263be
Update augmentation.py
makecent Jan 11, 2022
880e94b
Update augmentation.py
makecent Jan 11, 2022
156d50b
revise pre-commit
makecent Jan 11, 2022
6564429
Update augmentation.py
makecent Jan 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion configs/_base_/datasets/imagenet_rgb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

test_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='Resize', size=(256, -1), backend='pillow'),
dict(type='Resize', size=(-1, 256), backend='pillow'),
dict(type='CenterCrop', crop_size=224),
dict(type='Normalize', **img_norm_cfg),
dict(type='ImageToTensor', keys=['img']),
Expand Down
2 changes: 1 addition & 1 deletion configs/_base_/datasets/lsun-car_pad_512.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
key='real_img',
io_backend='disk',
),
dict(type='Resize', keys=['real_img'], scale=(512, 384)),
dict(type='Resize', keys=['real_img'], scale=(384, 512)),
dict(
type='NumpyPad',
keys=['real_img'],
Expand Down
2 changes: 1 addition & 1 deletion mmgen/datasets/pipelines/augmentation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ def __call__(self, results):
scale = (self.scale[-1], int(self.scale[-1] / w * h))
else:
# direct use the given ones
scale = self.scale
scale = self.scale[::-1]
Copy link
Collaborator

@LeoXing1996 LeoXing1996 Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider the situation when the input scale is float or int.

Copy link
Author

@makecent makecent Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the mmcv.imresize can take argument scale of float or int.
https://github.com/open-mmlab/mmcv/blob/51b40c332aff9d2927fcc252b248d295850a4d55/mmcv/image/geometric.py#L51-L95

Copy link
Author

@makecent makecent Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the original Resize pipeline in the mmgen does NOT support taking as input scale of float or int. I tried and got an error raised.

Copy link
Collaborator

@LeoXing1996 LeoXing1996 Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, mmcv.imrescale can take float as input. In MMGen's Resize, we compose mmcv.imrescale and mmcv.imresize in one function. Refers to

mmgeneration/mmgen/datasets/pipelines/augmentation.py

Lines 162 to 177 in 69333cf

if self.keep_ratio:
img, scale_factor = mmcv.imrescale(
img,
scale,
return_scale=True,
interpolation=self.interpolation,
backend=self.backend)
else:
img, w_scale, h_scale = mmcv.imresize(
img,
scale,
return_scale=True,
interpolation=self.interpolation,
backend=self.backend)
scale_factor = np.array((w_scale, h_scale), dtype=np.float32)
return img, scale_factor

Copy link
Collaborator

@LeoXing1996 LeoXing1996 Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may use the following command to run the unit test locally.

 coverage run --branch --source mmgen -m pytest -s tests/test_datasets/test_pipelines/test_augmentation.py

Copy link
Author

@makecent makecent Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see... I think it's a little bit confusing though. Current Resize:

Cases Arguments Comments
scale with a factor & keep ratio scale=float, keep_ratio=True lack of asserting keep_ratio=True; not support scale=int
scale with factors (fh, fw) & not keep ratio not supported
scale with size (h, w) & not keep ratio scale=(h, w), keep_ratio=False Misplaced h and w
scale with size (h, -1) & keep ratio scale=(h, -1), keep_ratio=True; lack of asserting keep_ratio=True; variable max_long_edge could be the actual short edge

Copy link
Author

@makecent makecent Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can help resolve the problems:

  1. lacking asserting keep_ratio=True;
  2. not support scale=int;
  3. Misplaced h and w (solved).

But as for the " variable max_long_edge could be the actual short edge", it requires futher discussion.

Copy link
Collaborator

@LeoXing1996 LeoXing1996 Jan 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your analysis of cases 1 and 2 are correct. And for case 3 we automatically rescale short edge and this behavior is independent of the position of -1 in scale.


# here we assume all images in self.keys have the same input size
for key in self.keys:
Expand Down