Thank you

[mviereck]

I just want to say "Thank you!" for this project! :-)

My wiki pages now appear in search engines. First visitors come from https://github-wiki-see.page/ , this was how I got attention of your project.

[nelsonjchen]

Thanks! It's really a shame Wikis aren't indexed directly.

My first "if I knew about the Wiki entry on this" issue was something involving the Nlog library. Not realizing something nuanced that was explained in an obscure wiki entry caused a time-bombed nationwide production outage for a large customer on thousands of machines. That was about 5 years ago. Work project exploding? Just another day. Didn't do anything about this.

Years later, my "gotta do something about this" event was answering yet another repeated question in a FAQ on an openpilot Discord. ARE PEOPLE READING THE FAQ?! Annoyed in a personal interest? That was just too much. And then I made GHWSEE.

Clearly I have my priorities in order 😄 .

I'm very happy to see your Wikis and others like it are ranking at least on the first page of Google results with GHWSEE. GHWSEE shouldn't exist though and a real solution is yet to be implemented. Let's hope GitHub implements something more acceptable as the current status quo isn't.

[nelsonjchen]

@mviereck

Also, you might want to do something like this in your READMEs for your projects:

https://github.com/MiSTer-devel/Main_MiSTer#main_mister-main-binary-and-wiki-repo

It helps prioritize Google towards crawling your wiki deeper.

[mviereck]

Also, you might want to do something like this in your READMEs for your projects:
It helps prioritize Google towards crawling your wiki deeper.

Thank you for the tip!
I wonder if that would help much. If I look at my wiki main page on github-wiki-see, its links directly lead to github: https://github-wiki-see.page/m/mviereck/x11docker/wiki
If the subpages on github cannot be crawled, too, than only the main page would get a ranking advantage from linking it in my README.md.
So i would have to link all my wiki pages on github-wiki-see in README.md to get a severe advantage for crawling.

Just wondering: I don't find a robots entry like x-robots-tag: none in the header of my wiki pages.
Nonetheless, only the copy on github-wiki-see appears in search engines.

[nelsonjchen]

Ah, I synthesize a "https://github-wiki-see.page/m/mviereck/x11docker/wiki_index" which is linked at the bottom of every mirrored page. That one pulls a page list from GitHub, links the pages and is indexable. The index page might be the better "page" to put in the readme. That did not exist when the MiSTer people added the crawlable link to their README.

As for the headers issue, rather than writing a 1000 words, here's a picture of the header I was talking about which I do see:

It's in the HTTP Headers, not the HTML. While my screenshot is of looking at it in incognito view like a search engine, the header is also present in a logged-in session.

[mviereck]

Ah, I synthesize a "https://github-wiki-see.page/m/mviereck/x11docker/wiki_index" which is linked at the bottom of every mirrored page. That one pulls a page list from GitHub, links the pages and is indexable. The index page might be the better "page" to put in the readme. That did not exist when the MiSTer people added the crawlable link to their README.

Thank you! I've put a link at the bottom of my README.md.

Additionally a technical link for search engine crawlers to [x11docker wiki](https://github-wiki-see.page/m/mviereck/x11docker/wiki_index) 
because github does not allow search engines to access github wiki pages directly. 
Compare workaround helper [github-wiki-see](https://github.com/nelsonjchen/github-wiki-see-rs).

It's in the HTTP Headers, not the HTML. While my screenshot is of looking at it in incognito view like a search engine, the header is also present in a logged-in session.

Well hidden! Thank you for clarification.

[mviereck]

Just a thought:
It might make sense to provide a page that contains all links to all wiki pages of github.
This page could be linked in several READMEs as well to empower search rankings.

[nelsonjchen]

That might be a bit too spammy and the page is likely to be ignored as a mass link farm or something. There are also about 3 million pages in GitHub from my last survey. It costs a lot to do the survey once in a while and I'm not too gung ho about storing that list in the repo.

[nelsonjchen]

@mviereck It looks like your project's GitHub Wiki is now directly indexable. I think it fits in under the 1000+ stars and un-publicly editable gate that GitHub haven't publically defined yet as a requirement to have an index-able wiki.

An auto-disengagement system was implemented to make this service drop out automatically for directly indexable Wikis and it seems to be working fine!

There are still other Wikis that don't fit that criteria but also don't deserve the death of not being indexable so I'm going to keep this thing going.

[mviereck]

Thank you for telling me!
After your hint concerning "publicly editable" I've removed the possibility for others to edit the wiki some month ago just to see if github will add x11docker wiki to the indexable sites.
It seems this succeeded.
Currently not much of x11docker wiki pages appears on google, but in the long term I am sure they will become popular. I've collected a lot of useful information there (like many other github users do), and it is a shame that github hides this from search engines by default.
Keep up your good work!

There are still other Wikis that don't fit that criteria but also don't deserve the death of not being indexable so I'm going to keep this thing going.

As someone pointed out elsewhere, especially the less popular repositories need to be indexed. There are lots of useful but barely known projects.