Raw Align Sub-step Error

[singhua-agri]

Hello! Thanks for developing such a powerful genome assembly tool. I was confronted with an error bellow when I ran the software:

[20861 INFO] 2024-10-01 00:48:40 Submitted jobID:[140827] jobCmd:[/public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align1/nextDenovo.sh] in the local_cycle.
[20861 INFO] 2024-10-01 00:48:41 Submitted jobID:[141098] jobCmd:[/public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align2/nextDenovo.sh] in the local_cycle.
[20861 INFO] 2024-10-01 00:48:41 Submitted jobID:[141105] jobCmd:[/public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh] in the local_cycle.
[20861 INFO] 2024-10-01 00:48:42 Submitted jobID:[141242] jobCmd:[/public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align4/nextDenovo.sh] in the local_cycle.
[141105 CRITICAL] 2024-10-01 04:37:46 Command '/bin/sh /public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh > /public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh.o 2> /public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh.e' returned non-zero exit status 139, error info: .
Traceback (most recent call last):
File "/opt/anaconda3/envs/genome_assembly/bin/nextDenovo", line 857, in
main(args)
File "/opt/anaconda3/envs/genome_assembly/bin/nextDenovo", line 675, in main
task.run.start()
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/site-packages/paralleltask/task_control.py", line 344, in start
self._start()
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/site-packages/paralleltask/task_control.py", line 368, in _start
self.submit(job)
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/site-packages/paralleltask/task_control.py", line 252, in submit
_, stdout, _ = self.run(job.cmd)
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/site-packages/paralleltask/task_control.py", line 288, in run
log.critical("Command '%s' returned non-zero exit status %d, error info: %s." % (cmd, p.returncode, stderr))
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/logging/init.py", line 1355, in critical
self._log(CRITICAL, msg, args, **kwargs)
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/logging/init.py", line 1444, in _log
self.handle(record)
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/logging/init.py", line 1454, in handle
self.callHandlers(record)
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/logging/init.py", line 1516, in callHandlers
hdlr.handle(record)
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/logging/init.py", line 865, in handle
self.emit(record)
File "/opt/anaconda3/envs/genome_assembly/lib/python3.6/site-packages/paralleltask/kit.py", line 42, in emit
raise Exception(record.msg)
Exception: Command '/bin/sh /public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh > /public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh.o 2> /public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh.e' returned non-zero exit status 139, error info: .
[20861 ERROR] 2024-10-01 11:16:27 sort_align failed: please check the following logs:
[20861 ERROR] 2024-10-01 11:16:27 /public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh.e

So I followed the log info and opened the nextDenovo.e:

/public/genome/nextdenovo/nextdenovo_out/01.raw_align/03.sort_align.sh.work/sort_align3/nextDenovo.sh: line 5: 141109 segment error

I am really doubted (1) how to solve this problem. Besides, after the problem solved, (2) how to continue my program with the run.cfg file modified?

[moold]

1. Rerun sort_align3/nextDenovo.sh, and if it still report error, you need to rerun all subtasks to produce the input file of this subask: sort_align3/input.fofn.
2. see here.

[xiaoyu-stars]

i have the same problem.
When running NextDenovo for error correction and assembly of Nanopore data in a cluster, I encountered some issues.
the contain of run.cfg is below

[General]
job_type = local
job_prefix = nextDenovo
task = all
rewrite = yes
deltmp = yes
parallel_jobs = 10
input_type = raw
read_type = ont # clr, ont, hifi
input_fofn = input.fofn
workdir = output

[correct_option]
read_cutoff = 1k
seed_cutoff=10k
genome_size = 928m # estimated genome size
blocksize=2g
sort_options = -m 20g -t 15
minimap2_options_raw = -x ava-ont -t 4
pa_correction = 3
correction_options = -p 15 -dbuf

[assemble_option]
minimap2_options_cns = -t 2
nextgraph_options = -a 1

Here is the error log content from output/01.raw_align/02.raw_align.sh.work/raw_align12/nextDenovo.sh.e.

hostname
+ hostname
cd /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01
+ cd /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01
time /home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/nextdenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 4 /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.seed.003.2bit /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.part.003.2bit -o input.seed.003.2bit.0.ovl;
+ /home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/nextdenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 4 /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.seed.003.2bit /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.part.003.2bit -o input.seed.003.2bit.0.ovl
/home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh: line 5:  5808 Terminated              /home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/nextdenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 4 /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.seed.003.2bit /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.part.003.2bit -o input.seed.003.2bit.0.ovl

I’ve been trying to solve this problem for a long time and hope to find a solution on how to run this task on the cluster！！！

[xiaoyu-stars]

i have the same problem. When running NextDenovo for error correction and assembly of Nanopore data in a cluster, I encountered some issues. the contain of run.cfg is below

[General]
job_type = local
job_prefix = nextDenovo
task = all
rewrite = yes
deltmp = yes
parallel_jobs = 10
input_type = raw
read_type = ont # clr, ont, hifi
input_fofn = input.fofn
workdir = output

[correct_option]
read_cutoff = 1k
seed_cutoff=10k
genome_size = 928m # estimated genome size
blocksize=2g
sort_options = -m 20g -t 15
minimap2_options_raw = -x ava-ont -t 4
pa_correction = 3
correction_options = -p 15 -dbuf

[assemble_option]
minimap2_options_cns = -t 2
nextgraph_options = -a 1

Here is the error log content from output/01.raw_align/02.raw_align.sh.work/raw_align12/nextDenovo.sh.e.

hostname
+ hostname
cd /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01
+ cd /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01
time /home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/nextdenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 4 /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.seed.003.2bit /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.part.003.2bit -o input.seed.003.2bit.0.ovl;
+ /home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/nextdenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 4 /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.seed.003.2bit /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.part.003.2bit -o input.seed.003.2bit.0.ovl
/home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh: line 5:  5808 Terminated              /home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/nextdenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 4 /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.seed.003.2bit /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/input.part.003.2bit -o input.seed.003.2bit.0.ovl

I’ve been trying to solve this problem for a long time and hope to find a solution on how to run this task on the cluster！！！

Let me add some command line error output information:

Command '/bin/sh /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh > /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh.o 2> /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh.e' returned non-zero exit status 143, error info: .
Traceback (most recent call last):
  File "/home/u2220114062/miniforge3/envs/nextDenovo/bin/nextDenovo", line 857, in <module>
    main(args)
  File "/home/u2220114062/miniforge3/envs/nextDenovo/bin/nextDenovo", line 651, in main
    task.run.start()
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/paralleltask/task_control.py", line 344, in start
    self._start()
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/paralleltask/task_control.py", line 368, in _start
    self.submit(job)
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/paralleltask/task_control.py", line 252, in submit
    _, stdout, _ = self.run(job.cmd)
                   ^^^^^^^^^^^^^^^^^
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/paralleltask/task_control.py", line 288, in run
    log.critical("Command '%s' returned non-zero exit status %d, error info: %s." % (cmd, p.returncode, stderr))
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/logging/__init__.py", line 1586, in critical
    self._log(CRITICAL, msg, args, **kwargs)
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/logging/__init__.py", line 1684, in _log
    self.handle(record)
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/logging/__init__.py", line 1700, in handle
    self.callHandlers(record)
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
    hdlr.handle(record)
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/logging/__init__.py", line 1028, in handle
    self.emit(record)
  File "/home/u2220114062/miniforge3/envs/nextDenovo/lib/python3.12/site-packages/paralleltask/kit.py", line 42, in emit
    raise Exception(record.msg)
Exception: Command '/bin/sh /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh > /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh.o 2> /home/u2220114062/work/Onychostoma_macrolepis/02.assemble/nextDenovo-nextDenovo/output/01.raw_align/02.raw_align.sh.work/raw_align01/nextDenovo.sh.e' returned non-zero exit status 143, error info: .

[moold]

Try to reduce the value of parallel_jobs = 10 and run again.