Language Server does not find extension attribute: `Cannot access member "Meta" for type "Markdown" Member "Meta" is unknown`

[PlexSheep]

When using the meta extension, the Meta field of the Markdown class is not found by Pyright. Perhaps I missed something, can we import a stub file to make this error go away?

For clarity: The code works, but the language server does not find the Meta Member for type Markdown.

This might also happen with the attributes added by other extensions, but I've found it for the meta extension specifically.

Minimal example:
test.py:

import markdown
MD = markdown.Markdown(extensions=['meta'])
with open("foo.md", "r") as f:
    content: str = f.read()
    html: str = MD.convert(content)
    meta = MD.Meta    # Cannot access member "Meta" for type "Markdown"    Member "Meta" is unknown
    print(meta)

foo.md:

key: 19

this is the content

executing the script:

$ python test.py           
{'key': ['19']}

Proposed Fix
Add a python stub file for the Meta attribute.

[waylan]

I can't replicate this. In fact, notice below that I have not even called convert yet. Simply loading the extension ensures that the Meta attribute exists.

>>> md = markdown.Markdown(extensions=['meta'])
>>> md.Meta
{}
>>> markdown.__version__
'3.4.4'

[facelessuser]

I don't think the argument is that md.Meta doesn't exist, only that LSP (Language Server Provider) in their editor can't automatically recognize md.Meta and its type.

As Python Markdown does not provide type annotations, and any that are available are provided by 3rd parties, I'm not sure if this is an issue for Python Markdown. If we were currently providing type annotations, then maybe it could be argued we should do something, but currently, this is not a typed library, even if someone else has provided typing that you are using in conjunction with your LSP.

[waylan]

Now I have reread your report and you note that things are working. But apparently some tool is inspecting the code and not finding the attribute. That is correct. Many extensions add their own attributes (most of them being third-party) and there is no way that we could possible anticipate or stub every possible attribute added by every extension. So, no we will not be giving this one extension special treatment.

That said, if anyone has suggestions for a general purpose solution, I am willing to consider it.

[waylan]

As Python Markdown does not provide type annotations

That will no longer be true with the next release. See #1379. BTW, @facelessuser I have a question for you in my last comment there. Based on your comments here perhaps you've been ignoring that one and missed it.

[facelessuser]

That will no longer be true with the next release.

Well, you are documenting the API, yes, but there are no type annotations. I'm fairly certain that is what LSP uses, not function doc strings. At least that is my understanding.

BTW, @facelessuser I have a question for you in my last comment there. Based on your comments here perhaps you've been ignoring that one and missed it.

I have been following along at a very high level. I am aware of what is proposed and the improvement to documentation. but that PR was generating a lot of noise, so it is likely I dismissed the notification without looking.

[waylan]

Well, you are documenting the API, yes, but there are no type annotations.

Yes, we are adding type annotionations, which is my point. It will no longer be true that the library does not provide type annotations. However, I'm not really familiar with LSP or what is does or what it expects, so I don't know how this is relevant.

[facelessuser]

Ah, I take that back then. Like I said, I wasn't following that closely. Unless you define a Meta property upfront and type it, it will show up as a "mystery" attribute. But since this is a dynamic attribute that only appears when the extension is loaded, I'm not quite sure. While I've used type annotations in a number of libraries, I cannot argue I'm an expert in all cases.

[waylan]

Unless you define a Meta property upfront and type it, it will show up as a "mystery" attribute.

Ah okay, so this is the complaint then.

It recently occurred to me that we could provide some hook for extensions to use for this. For example, we could provide an attribute which stored key/value pairs where the key would be a name provided by the extension and the value would be whatever data the extension provides. Then it could be type annotated as dict[str, Any] (or whatever format we went with).

The issue of course is that many existing extensions already provide their own custom solutions and this would be a new API. Users would then need to alter their code to check the new location. Maybe we could use a custom getter and setter which would cause the extension to write to our custom location and the user to read from our custom without any changes on their end (extension or user).

Although, now that I think about it, some projects make extensive use of custom getters and setters on classes (see Django for example). They have a whole collections of potential attributes which are unknown. How do they handle this? Maybe that is the approach we should take.

[facelessuser]

Although, now that I think about it, some projects make extensive use of custom getters and setters on classes (see Django for example). They have a whole collections of potential attributes which are unknown. How do they handle this? Maybe that is the approach we should take.

Maybe, there are a lot of complex typing paths that I have yet to explore.

[waylan]

To be clear I'm talking about the __setattr__(self, key, value) and __getattr__(self, key) methods which will accept any non-existent attribute on a class instance and set/get it. Would annotating those methods be sufficient to satisfy type checkers?

[PlexSheep]

If it came to it, you could just generally set a type hinting for the Meta attribute as TypeOfMeta | None. This would indicate to the user that the attribute can indeed be none (if the extension is unloaded).

In this case, I think my LSP picks up on the type if I check before using with if hasattr(MD, "Meta").

[waylan]

If it came to it, you could just generally set a type hinting for the Meta attribute as TypeOfMeta | None. This would indicate to the user that the attribute can indeed be none (if the extension is unloaded).

But where would we do that? Not on the Markdown class. It has no knowledge of any specific extension or what attributes an extension might add. In fact, many different extensions all add attribute and we can't cover them all.

So we have two options:

1. Do nothing.
2. Come up with some generic solution that works for any extension with any attribute that it might add to the class.

[oprypin]

Apparently I bumped into this topic here:

• Fix all mypy typechecking errors, add a lot of type annotations #1399 (comment)

[oprypin]

There are no solutions for any typechecker that would allow these attributes to be properly declared with known types, other than actually declaring them on the core object itself.

So these are the options:

1. Declare the 3 core attributes. Very pragmatic as per my comment. People get the benefit of knowing that they're accessing the correct attributes and the attributes' types are known and the values are further type-checked. 95+% of use cases are served well. Adding third-party extensions there will be summarily rejected.

2. Trick the typecheckers by defining __getattr__ and letting it return Any. Then all type checking will be shut down for accessing unknown attributes on Markdown and for any values obtained from them.

3. Define a new "data bag" sub-attribute. Similar outcome to the previous one but is not really backwards compatible.

4. The current state: Not declare any attributes. You'll get a type error consistently. There's still some benefit to this: at least you can't make a mistake when trying to access attributes that do consistently exist.