[BUG] [flytekit] pyflyte register fails for Azure Blob Storage (Missing Headers)

[olandr]

Describe the bug

When trying to register a launchplan to Azure Blob Storage, using:

pyflyte --verbose register -p usecase -d development --image myacr.azurecr.io/myimage:123 launch_plan.py

I get the error (excerpt):

│                                                                                                  │
│    84 │   │   │   if not resp.ok:                                                                │
│    85 │   │   │   │   raise AssertionError(f"Failed to upload file {self._filename} to {res.si   │
│    86 │   │   except Exception as e:                                                             │
│ ❱  87 │   │   │   raise AssertionError(f"Failed to upload file {self._filename} reason {e}")     │
│    88                                                                                            │
│    89                                                                                            │
│    90 def get_flyte_fs(remote: FlyteRemote) -> typing.Type[FlyteFS]:                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AssertionError: Failed to upload file 00000 reason Failed to upload file 00000 to 
https://mystorageaccount.blob.core.windows.net/flyte/usecase/development/00000/1
235bj1u5351959x94?se=2024-09-19T12%3A56%3A12Z&sig=LesP283512395Rdyf%2FKm5135GVcg%aaaaa9Qnnf3%2
1431mM%3D&sp=aw&spr=https&sr=b&st=2024-09-19T11%3A26%3A12Z&sv=2024-
05-04 reason An HTTP header that's mandatory for this request is not specified.

Related Issues, PRs

#3942
flyteorg/flytekit#1784

Expected behavior

When running:

pyflyte --verbose register -p usecase -d development --image myacr.azurecr.io/myimage:123 launch_plan.py

It should not fail and the workflow should register properly.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[welcome]

Thank you for opening your first issue here! 🛠

[olandr]

After some discussion with @Tom-Newton, the workaround for now seems to be the following:

https://github.com/flyteorg/flytekit/blob/2452c74fba00b21a4428d548fe4100966d779c72/flytekit/remote/remote_fs.py#L83

            FlytePathResolver.add_mapping(self.path, res.native_url)
-           resp = requests.put(res.signed_url, data=data)
+           resp = requests.put(res.signed_url, data=data, headers={"x-ms-blob-type": "BlockBlob"})
            if not resp.ok:
                raise AssertionError(f"Failed to upload file {self._filename} to {res.signed_url} reason {resp.reason}")
        except Exception as e:
            raise AssertionError(f"Failed to upload file {self._filename} reason {e}")

[kumare3]

You can do this specifically for Azure (in the fs layer)