Eta hangs when accessing etlas.typelead.com

[mspaulding06]

Hello,

I am running etlas 1.0.2.0 on macOS 10.12.6 and having a problem on the first run when the configuration is first generated. The first thing etlas does is it downloads repositories which it is unable to do. I tried running both etlas init and etlas update which both ran into the same problem.

It gets to this point:

Downloading the latest package lists from:
- hackage.haskell.org
- etlas.typelead.com

Eventually it will quit since it is unable to make a connection to Github. Looking at the generated configuration file it appears that it is using the wrong protocol so I switched it from http to https and then set secure to true. After running etlas this time I get this message.

Warning: No mirrors found for https://github.com/typelead/etlas-index
Unexpected response 404for https://github.com/typelead/etlas-index/root.json

Removing the repository altogether will cause an infinite loop looking for patches with the following message.

Downloading the latest package list from hackage.haskell.org
To revert to previous state run:
    etlas update --index-state='2017-08-19T03:39:40Z'
Updating the eta-hackage patch set.

Then if I clone the Github repository myself into ~/.etlas/packages/etlas.typelead.com it fails with a different error.

Downloading the latest package lists from:
- hackage.haskell.org
- etlas.typelead.com
etlas: Could not read index. Did you call 'checkForUpdates'?

I have tried both the binary from the website and building from source with the same result.

[rahulmutt]

This looks interesting. Can you wipe out ~/.eta, ~/.etlas and do a fresh binary installation. And the first command you run after the installation is done should be have -v3 and share the output.

e.g. if your first command is etlas update, please run etlas update -v3 instead and share the result.

[mspaulding06]

Thanks for your quick response. Here you go.

Welcome to Etlas, the awesome build tool for the Eta programming language!

We are working hard to make the best platform for pure functional programming
on the Java Virtual Machine.

Etlas will periodically send usage statistics to help us:

* Make decisions on how to evolve the language

* Build a community portal that displays the data we collect in visual form

* Build an Etlas package repository that will accurately display package popularity

* And more!

For more details, please consult our privacy policy:

        http://eta-lang.org/docs/html/privacy-policy.html

Would you like to help us make Eta the fastest growing programming language,
and help pure functional programming become mainstream? (y/n)

Thank you for trying out Eta.

If you face any issues, you can:

* File an issue at:
      https://github.com/typelead/eta/issues/new

* Post your question on Eta-Discuss:
      https://groups.google.com/forum/#!newtopic/eta-discuss

* Ask on the Gitter channel:
      https://gitter.im/typelead/eta

Config file path source is default config file.
Config file /Users/matt/.etlas/config not found.
Writing default configuration to /Users/matt/.etlas/config
no user package environment file found at /Users/matt
Trying to locate mirrors via DNS for initial bootstrap of secure repository
'http://hackage.haskell.org/' ...
Searching for nslookup in path.
Found nslookup at /usr/bin/nslookup
/usr/bin/nslookup '-query=TXT' _mirrors.hackage.haskell.org
located 2 mirrors for http://hackage.haskell.org/ :
- http://hackage.fpcomplete.com/
- http://objects-us-west-1.dream.io/hackage-mirror/
Selected mirror http://hackage.haskell.org/
Downloading root
Searching for curl in path.
Found curl at /usr/bin/curl
Searching for powershell in path.
Cannot find powershell on the path
Searching for wget in path.
Found wget at /usr/local/bin/wget
Selected http transport implementation: curl
/usr/bin/curl 'http://hackage.haskell.org/root.json' --output /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/transportAdapterGet1622650073984943658 --location --write-out '%{http_code}' --user-agent 'etlas/1.0.2.0 (osx; x86_64)' --silent --show-error --dump-header /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/curl-headers1144108930470211272.txt --header 'Cache-Control: no-transform'
Downloading the latest package lists from:
- hackage.haskell.org
- etlas.typelead.com
Selected mirror http://hackage.haskell.org/
Downloading timestamp
/usr/bin/curl 'http://hackage.haskell.org/timestamp.json' --output /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/transportAdapterGet14587779232007237709 --location --write-out '%{http_code}' --user-agent 'etlas/1.0.2.0 (osx; x86_64)' --silent --show-error --dump-header /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/curl-headers8235644401115438165.txt --header 'Cache-Control: no-transform'
Searching for git in path.
Found git at /usr/local/bin/git
/usr/local/bin/git --version
/usr/local/bin/git is version 2.14.1
/usr/local/bin/git clone '--depth=1' 'http://github.com/typelead/etlas-index' /Users/matt/.etlas/packages/etlas.typelead.com
Downloading snapshot
/usr/bin/curl 'http://hackage.haskell.org/snapshot.json' --output /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/transportAdapterGet1148079871137522503 --location --write-out '%{http_code}' --user-agent 'etlas/1.0.2.0 (osx; x86_64)' --silent --show-error --dump-header /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/curl-headers144128232716531729.txt --header 'Cache-Control: no-transform'
Downloading mirrors
/usr/bin/curl 'http://hackage.haskell.org/mirrors.json' --output /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/transportAdapterGet8965443031474833169 --location --write-out '%{http_code}' --user-agent 'etlas/1.0.2.0 (osx; x86_64)' --silent --show-error --dump-header /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/curl-headers12648177091998097157.txt --header 'Cache-Control: no-transform'
Cannot update index (no local copy)
Downloading index
/usr/bin/curl 'http://hackage.haskell.org/01-index.tar.gz' --output /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/transportAdapterGet8933518161505795335 --location --write-out '%{http_code}' --user-agent 'etlas/1.0.2.0 (osx; x86_64)' --silent --show-error --dump-header /var/folders/sj/9wf9v8jx5dbgx5lrbv9x5b3r0000gp/T/curl-headers19548990971636807826.txt --header 'Cache-Control: no-transform'
Updating index cache file
/Users/matt/.etlas/packages/hackage.haskell.org/01-index.cache ...
Index cache updated to index-state 2017-08-19T03:39:40Z
/usr/local/bin/git returned ExitFailure 128 with error message:
Cloning into '/Users/matt/.etlas/packages/etlas.typelead.com'...
fatal: unable to access 'http://github.com/typelead/etlas-index/': Failed to
connect to github.com port 80: Operation timed out

[rahulmutt]

And doing a git clone http://github.com/typelead/etlas-index works fine when doing it manually? You mentioned you get a different error if you do a manual clone. Can you try that again and use -v3 this time and share the output?

[mspaulding06]

I don't think I was clear on that, the command that I used to clone the repository was this:

$ cd ~/.etlas/packages
$ git clone https://github.com/typelead/etlas-index.git etlas.typelead.com

So I had to use https instead of http which etlas itself was using in order for the clone to work. I'm not sure what happened this time but when I attempted this method again it works. After cloning the repository myself etlas will continue to install eta and the libraries.

---

Update

So, I decided I would try this method one more time just to see what happens. I did this again and it failed like it had the previous time. Here is the output after cloning the repository and then running etlas update -v3 again.

no user package environment file found at
/Users/matt/.etlas/packages/etlas.typelead.com
Downloading the latest package lists from:
- hackage.haskell.org
- etlas.typelead.com
Searching for curl in path.
Found curl at /usr/bin/curl
Searching for powershell in path.
Cannot find powershell on the path
Searching for wget in path.
Found wget at /usr/local/bin/wget
Selected http transport implementation: curl
Updating index cache file
/Users/matt/.etlas/packages/hackage.haskell.org/01-index.cache ...
Searching for git in path.
Found git at /usr/local/bin/git
/usr/local/bin/git --version
etlas: Could not read index. Did you call 'checkForUpdates'?

I then attempted this method again by:

1. Running etlas update and waiting for it to fail
2. Cloning the etlas.typelead.com repository into ~/.etlas/packages
3. Removing the directory ~/.etlas/packages/hackage.haskell.org

This time the update worked again.

My guess is that when git fails to the clone the first time etlas update is run it immediately fails the entire process even though the hackage repository is still in the process of being updated leaving it in a corrupted state, since the repository downloads are running concurrently. Then when I go to run etlas again the error message about Could not read index is actually referring to the hackage repository index being corrupt.

[rahulmutt]

Thank you for sharing your observations in detail! Does this mean I should make https the default for cloning that repo?

This is indeed a bug, the behaviour should be both to:

1. Avoid killing the hackage download if the etlas-index download fails
2. If the hackage.haskell.org local cache is corrupted, start over from scratch.

[mspaulding06]

Yes, I think that you should be using https for the clone to work, but as I mention above when I did this change to the etlas configuration file I got a different error yet.

Warning: No mirrors found for https://github.com/typelead/etlas-index
Unexpected response 404for https://github.com/typelead/etlas-index/root.json

So there must be some other changes needed in etlas to make this possible.

[rahulmutt]

That's because etlas has two types of repos: Hackage-like (index is backed by a giant tar.gz) and Git-like (index is backed by a git repo). The secure option only applies to Hackage-like indexes, and hence when applied to the etlas-index it's failing (another bug). Currently, the url used to download atlas-index is hardcoded.

So another change:

• Make the secure option on by default for atlas-index and also make it behave properly by setting the URI scheme before initiating a git clone.

Thanks a lot!