download an entire directory

[rcsingh81]

Is there a way to download entire directory (just like we do for an individual file) from opengrok results? For a file we see it has an option "Download" to download it but I don't see(or aware of) any such feature for downloading the entire directory.

[vladak]

There's not, marking as enhancement.

[alxjsn]

Dirty way to download a folder using wget + mitmproxy + proxychains:

opengrok.py

from mitmproxy import http

def request(flow: http.HTTPFlow):
    if flow.request.path.endswith("/"):
        pass
    else:
        flow.request.path=flow.request.path.replace('xref', 'raw')

1. Configure proxychains to use the following proxy http 127.0.0.1 8081
2. Run mitmproxy with this command: mitmproxy -p 8081 -s opengrok.py
3. Export cookies.txt file with a browser extension
4. proxychains wget -e robots=off --load-cookies ~/Downloads/cookies.txt -r -np -R "index.html*" --no-check-certificate https://opengrokserver/path/to/folder/

[meir555]

Dirty way to download a folder using wget + mitmproxy + proxychains:

opengrok.py

from mitmproxy import http

def request(flow: http.HTTPFlow):
    if flow.request.path.endswith("/"):
        pass
    else:
        flow.request.path=flow.request.path.replace('xref', 'raw')

1. Configure proxychains to use the following proxy http 127.0.0.1 8081
2. Run mitmproxy with this command: mitmproxy -p 8081 -s opengrok.py
3. Export cookies.txt file with a browser extension
4. proxychains wget -e robots=off --load-cookies ~/Downloads/cookies.txt -r -np -R "index.html*" --no-check-certificate https://opengrokserver/path/to/folder/

Thanks @alxjsn
Here are some additional improvements

1. Disable proxy_dns in proxychains.conf

2. Before running mitmproxy, use

export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8

3. Set the proxy script to return 404 for files with ? in the name

def request(flow: http.HTTPFlow):
    if flow.request.path.endswith("/"):
        pass
    else:
        flow.request.path=flow.request.path.replace('xref', 'raw')
    if (flow.request.path.find("?") != -1):
        flow.response = http.HTTPResponse.make(404)

4. For saving the session in firefox I used Export Cookies

5. If download stopped before finishing, use --no-clobber flag for wget to skip existing files

[AceSineX]

I wrote a simple script to make my life easier:
https://github.com/AceSineX/opengrok_directory_downloader