Clean title come from meta tags

[cikay]

Apply HTMLTITLE_REGEX cleanup to titles extracted from og:title, twitter:title, meta name title, and itemprop headline. Previously, extract() with with_metadata=True returned titles with site name suffixes from meta tags, while extract_title() correctly returned clean titles from h1 tags.

• Add clean_title() helper function to remove site name suffix/prefix
• Apply clean_title() in extract_opengraph() for og:title
• Apply clean_title() in examine_meta() for meta name titles and itemprop headlines
• Add tests for clean_title() and title cleaning in metadata extraction

The issue is encountered in the following website
https://www.nuhev.com

example url: https://www.nuhev.com/gelo-jiyan-de-li-sala-2050yan-cawa-be/

titles from meta: <meta property="og:title" content="Gelo Jîyan dê li Sala 2050yan Çawa Be? - Nûhev Co. %100 Kurdî">
titles from h1: <h1 class="jeg_post_title">Gelo Jîyan dê li Sala 2050yan Çawa Be?</h1>

[adbar]

Hi @cikay, your PR looks OK but for some reason the tests haven't run.

I suggest you edit your PR or add a commit like git commit --allow-empty -m "ci: retrigger" && git push to hopefully bypass this Github issue.

[cikay]

Hi @cikay, your PR looks OK but for some reason the tests haven't run.

I suggest you edit your PR or add a commit like git commit --allow-empty -m "ci: retrigger" && git push to hopefully bypass this Github issue.

Thanks. I just did

[adbar]

@cikay Are you still working on the PR? Maybe it's now fixed due to the recent changes.

[cikay]

@cikay Are you still working on the PR? Maybe it's now fixed due to the recent changes.

Still the same

>>> output_string = extract(downloaded, output_format="json", with_metadata=True, include_comments=False)
>>> output = json.loads(output_string)
>>> output["title"]
'Gelo Jîyan dê li Sala 2050yan Çawa Be? - Nûhev Co. %100 Kurdî'
>>> 

[cikay]

Existing tests have failed now. We have to find a reliable signal for a clean title.