Incorrect dimension when input image has shape 480 * 640

[MarkChenYutian]

Hi, I'm trying to use PWCDCNet implemented in PyTorch to calculate the optical flow on some dataset with image shape of 3 * 480 * 640 (C x H x W).

However, when I fed the images into the network, the following error occurred:

File "...\pwc_model.py", line 143, in warp
    vgrid = grid + flo
RuntimeError: The size of tensor a (15) must match the size of tensor b (16) at non-singleton dimension 2

Setting breakpoint and analyze the intermediate results in the forward() method of PWCDCNet shows that the variables c2x in forward() has following shape:

PWC-Net/PyTorch/models/PWCNet.py

Lines 182 to 193 in 07df6eb

	c11 = self.conv1b(self.conv1aa(self.conv1a(im1)))
	c21 = self.conv1b(self.conv1aa(self.conv1a(im2)))
	c12 = self.conv2b(self.conv2aa(self.conv2a(c11)))
	c22 = self.conv2b(self.conv2aa(self.conv2a(c21)))
	c13 = self.conv3b(self.conv3aa(self.conv3a(c12)))
	c23 = self.conv3b(self.conv3aa(self.conv3a(c22)))
	c14 = self.conv4b(self.conv4aa(self.conv4a(c13)))
	c24 = self.conv4b(self.conv4aa(self.conv4a(c23)))
	c15 = self.conv5b(self.conv5aa(self.conv5a(c14)))
	c25 = self.conv5b(self.conv5aa(self.conv5a(c24)))
	c16 = self.conv6b(self.conv6a(self.conv6aa(c15)))
	c26 = self.conv6b(self.conv6a(self.conv6aa(c25)))

# c21.shape, ..., c26.shape
torch.Size([1, 16, 240, 320]),
torch.Size([1, 32, 120, 160]),
torch.Size([1, 64, 60, 80]),
torch.Size([1, 96, 30, 40]),
torch.Size([1, 128, 15, 20]),
torch.Size([1, 196, 8, 10])

When calculating upflow_6, the upflow_6 has shape of torch.Size([1, 2, 16, 20]).

And hence the exception is raised on this line:

PWC-Net/PyTorch/models/PWCNet.py

Line 210 in 07df6eb

warp5 = self.warp(c25, up_flow6*0.625)

Is there any continent fix (i.e. without modifying the architecture which requires re-training the network) to this problem?

Thanks.

[MarkChenYutian]

For now, the fix is to resize the input image such that both width and height are multiple of 64:

PWC-Net/PyTorch/script_pwc.py

Lines 47 to 54 in 07df6eb

	divisor = 64.
	H = im_all[0].shape[0]
	W = im_all[0].shape[1]
	H_ = int(ceil(H/divisor) * divisor)
	W_ = int(ceil(W/divisor) * divisor)
	for i in range(len(im_all)):
	im_all[i] = cv2.resize(im_all[i], (W_, H_))

[flaviopinzarrone]

I have the same problem and this fix didn't solve the problem for me

[MarkChenYutian]

I cropped my image (originally 480 * 640) to 448 * 640 to fix this problem.