Skip to content

CoR-Forum/regnum-online-resources-crawler

Repository files navigation

regnum-online-resources-crawler

Content

  • Resource Crawler (for Regnum Paradise): crawl.sh / crawl_parallel.sh
  • Patchserver Mirrorer (creates a mirror of the whole patch server, including resources): ``mirror_patchserver.sh`
  • Patch Server Archiver (archives all install-related files from the regnum download01 server): archiver.sh/ archiver_borg.sh

crawl.sh

This will download all ressource files from the Regnum Patchserver and covert them to their corresponding format, be it a sound file, texture or whatever.

Please note that this clearly violates Regnums Terms of Use. Use at your own risk. NGE is known for randomly banning players so we strongly suggest to not use the same IP of playing Regnum and crawling the patch server.

Please note: Interrupting this script might give you inconsistent files!

Prerequisites

Make sure the following packages are installed:

apt install gzip bzip2 imagemagick qpdf zlib1g-dev xxd xz-utils parallel

Make sure the script is executable:

chmod +x crawl.sh

Usage

./crawl.sh

This will start downloading all files from the patch server and convert them in their corresponding format.

ls res/
ANIMATION  FONT  MESH  TERRAIN_MATERIAL  TEXTURE

Troubleshooting

If the script is terminated, it will restart where it ended.

If you cancel the script on your own or it terminated, it's highly recommended to remove the last file that was downloaded.

For example,

31108: bzip2: (stdin) is not a bzip2 file.

means you need to remove

raw/res_31108
extracted/res_31108

then restart the script.

crawl_parallel.sh

A GNU parallel-compatible, and thus faster version of crawl.sh. Running this on an IP address used for playing the game will lead you to a pretty sure PERMANENT ban. If you have imagemagick already installed in your system, then it's heavily recommended to use a clean system to run it.

It has the same prerequisites as crawl.sh, but you need GNU parallel on top of that :

apt install parallel

And you will also need ImageMagick7, to avoid color endianess issues (red and blue may be swapped), which is not yet available on many Linux repos. You can find an easy installer with instructions at https://github.com/SoftCreatR/imei. IM7 is incompatible with the system's IM6 that can be found in repos and needs to be uninstalled first.

Then make the script executable:

chmod +x ./crawl_parallel.sh

You need then to run the script as-is:

seq 0 90000 | parallel -j N ./crawl_parallel.sh

Where N is the number of parallel jobs you want to run. Don't abuse it too much if you don't have some good machine. A sweet spot may be 6 jobs by available CPU thread, but running just 1 will allow to act the same as crawl.sh, but with proper texture colors.

prepare.sh

This script will prepare the files to be usable by CoRT's texture paradise.

Use --no-thumbnails to skip the generation of texture thumbnails.

update approach

This is dirty.

mkdir update && \
cp crawl*.sh update/ && \
cp prepare.sh update/ && \
cd update/ && \
seq 0 90000 | parallel -j 50 ./crawl_parallel.sh && \
./prepare.sh

If this goes well...

rm -rf ../res && \
mv res ../

and update the res/files.json in CoRT accordingly.

About

Download all files from the Regnum Online Patchserver and convert them.

Resources

Stars

Watchers

Forks

Contributors

Languages