I have a complex tensor: tensor1 = tensor([0.0000+0.j, -106990.0794+0.j], device='cuda:1', dtype=torchplex128)

The both elements have no imaginary part (0.j), however, when squaring the variable tensor1**2 it yields an imaginary part in the second element

tensor([0.0000e+00+0.0000e+00j, 1.1447e+10-2.8037e-06j], device='cuda:1', dtype=torchplex128)

This behavior is not exhibited when performing the square in numpy

tensor1.cpu().numpy()**2

Out : array([0.00000000e+00+0.j, 1.14468771e+10-0.j])

Why are they different?

I have a complex tensor: tensor1 = tensor([0.0000+0.j, -106990.0794+0.j], device='cuda:1', dtype=torchplex128)

The both elements have no imaginary part (0.j), however, when squaring the variable tensor1**2 it yields an imaginary part in the second element

tensor([0.0000e+00+0.0000e+00j, 1.1447e+10-2.8037e-06j], device='cuda:1', dtype=torchplex128)

This behavior is not exhibited when performing the square in numpy

tensor1.cpu().numpy()**2

Out : array([0.00000000e+00+0.j, 1.14468771e+10-0.j])

Why are they different?

• pytorch
• tensor
• complex-numbers

2 Answers 2

This looks to be a numeric issue impacting complex numbers on GPU. Weirdly, it impacts negative numbers but not positive.

values = [5.0, -5.0]
devices = ['cpu', 'cuda']

for device in devices:
    for value in values:
        x = torch.tensor(value+0.j, device=device, dtype=torchplex128)
        x_pow2 = x.pow(2)
        x_x_conj = x * x.conj()
        check1 = x_pow2 == x_x_conj
        check2 = x_pow2.imag == 0
        check3 = x_x_conj.imag==0
        
        print(f"{device}\t{value}\t{check1.item()}\t{check2.item()}\t{check3.item()}")

cpu 5.0 True    True    True
cpu -5.0    True    True    True
cuda    5.0 False   True    True
cuda    -5.0    False   False   True

For a given input x = value + 0.j, the code computes x_pow2 = x.pow(2) and x_x_conj = x * x.conj(). The code checks if x_pow2 == x_x_conj, and if the imaginary component of those tensors is zero.

We see on CPU, x_pow2 == x_x_conj for both input values, and both x_pow2 and x_x_conj have a zero imaginary component.

On GPU, we see that in both cases x_pow2 != x_x_conj. For the case of value = 5.0, both x_pow2 and x_x_conj have zero imaginary component. For the case of value = -5.0, x_x_conj has zero imaginary component but x_pow2 does not.

I've tested this with a few values, and I consistently see inputs with positive real values yielding x_pow2 with zero imaginary component, while inputs with negative real values yielding x_pow2 with nonzero imaginary component.

This is likely due to something weird happening in the pow kernel that isn't present in the mul kernel.

NumPy has a special case that implements x**2 as x*x. For complex x, this uses the straightforward componentwise multiplication formula, which produces a purely real output for purely real inputs.

Apparently Torch isn't doing that. Torch seems to be doing a general complex**real exponentiation. That converts a complex number to angle-and-magnitude representation, then computes magnitude**power and angle*power and converts back to componentwise representation.

The angle of a negative number is pi, which is not exactly representable. So squaring a negative number this way produces rounding error in the angle, which translates to a very small imaginary component in the result when converted back to real and imaginary components.