Skip to content

Commit a56b79b

Browse files
RoyalSkyeRoyalSkye
committed
update README
1 parent 8ed0ece commit a56b79b

File tree

6 files changed

+122
-11
lines changed

6 files changed

+122
-11
lines changed

‎README.md

+115-4
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,118 @@
1-
# Routing_Anything
1+
<h1 align="center"> MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts </h1>
22

3-
## Model Structure
3+
<p align="center">
4+
<a href="https://openreview.net/forum?id=lsQnneYa8p"><img src="https://img.shields.io/static/v1?label=OpenReview&message=Forum&color=green&style=flat-square" alt="Paper"></a>&nbsp;&nbsp;&nbsp;&nbsp;<a href=""><img alt="License" src="https://img.shields.io/static/v1?label=ICML'24&message=Vienna&color=9cf&style=flat-square"></a>&nbsp;&nbsp;&nbsp;&nbsp;<a href="https://github.com/RoyalSkye/Routing-MVMoE/blob/main/LICENSE"><img src="https://img.shields.io/static/v1?label=License&message=MIT&color=orange&style=flat-square" alt="Paper"></a>
5+
</p>
6+
The PyTorch Implementation of *ICML 2024 Poster -- [MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts]()*. MVMoE is a unified neural solver that can cope with 16 VRP variants simultaneously, even in a zero-shot manner. Concretely, the training tasks include `CVRP`, `OVRP`, `VRPB`, `VRPL`, `VRPTW`, and `OVRPTW`. The test tasks include `OVRPB`, `OVRPL`, `VRPBL`, `VRPBTW`, `VRPLTW`, `OVRPBL`, `OVRPBTW`, `OVRPLTW`, `VRPBLTW`, and `OVRPBLTW`.
47

5-
## Multiple VRPs
8+
* ☺️ *We will attend ICML 2024 in person. Welcome to stop by our poster for discussion.*
69

7-
## Few-shot (e.g, Prompt-tuning)
10+
<p align="center"><img src="./assets/mvmoe.png" width=98%></p>
11+
12+
## Dependencies
13+
14+
* Python >= 3.8
15+
* Pytorch >= 1.12
16+
17+
## How to Run
18+
19+
<details>
20+
<summary><strong>Train</strong></summary>
21+
22+
```shell
23+
# Default: --problem_size=100 --pomo_size=100 --gpu_id=0
24+
# 0. POMO
25+
python train.py --problem={PROBLEM} --model_type=SINGLE
26+
27+
# 1. POMO-MTL
28+
python train.py --problem=Train_ALL --model_type=MTL
29+
30+
# 2. MVMoE/4E
31+
python train.py --problem=Train_ALL --model_type=MOE --num_experts=4 --routing_level=node --routing_method=input_choice
32+
33+
# 3. MVMoE/4E-L
34+
python train.py --problem=Train_ALL --model_type=MOE_LIGHT --num_experts=4 --routing_level=node --routing_method=input_choice
35+
```
36+
37+
</details>
38+
39+
<details>
40+
<summary><strong>Evaluation</strong></summary>
41+
42+
```shell
43+
# 0. POMO
44+
python test.py --problem={PROBLEM} --model_type=SINGLE --checkpoint={MODEL_PATH}
45+
46+
# 1. POMO-MTL
47+
python test.py --problem=ALL --model_type=MTL --checkpoint={MODEL_PATH}
48+
49+
# 2. MVMoE/4E
50+
python test.py --problem=ALL --model_type=MOE --num_experts=4 --routing_level=node --routing_method=input_choice --checkpoint={MODEL_PATH}
51+
52+
# 3. MVMoE/4E-L
53+
python test.py --problem=ALL --model_type=MOE_LIGHT --num_experts=4 --routing_level=node --routing_method=input_choice --checkpoint={MODEL_PATH}
54+
55+
# 4. Evaluation on CVRPLIB
56+
python test.py --problem=CVRP --model_type={MODEL_TYPE} --checkpoint={MODEL_PATH} --test_set_path=../data/CVRP-LIB
57+
```
58+
59+
</details>
60+
61+
<details>
62+
<summary><strong>Baseline</strong></summary>
63+
64+
```shell
65+
# 0. LKH3 - Support for ["CVRP", "OVRP", "VRPL", "VRPTW"]
66+
python LKH_baseline.py --problem={PROBLEM} --datasets={DATASET_PATH} -n=1000 --cpus=32 -runs=1 -max_trials=10000
67+
68+
# 1. HGS - Support for ["CVRP", "VRPTW"]
69+
python HGS_baseline.py --problem={PROBLEM} --datasets={DATASET_PATH} -n=1000 --cpus=32 -max_iteration=20000
70+
71+
# 2. OR-Tools - Support for all 16 VRP variants
72+
python OR-Tools_baseline.py --problem={PROBLEM} --datasets={DATASET_PATH} -n=1000 --cpus=32 -timelimit=20
73+
```
74+
75+
</details>
76+
77+
78+
## How to Customize MoE
79+
80+
MoEs can be easily used in Transformer-based models by replacing a Linear/MLP with an MoE layer. The input and output dimensions are kept the same as the original layer. Below, we provide two examples of how to customize MoEs.
81+
82+
```python
83+
# 0. Our implementation based on https://github.com/davidmrau/mixture-of-experts
84+
# Supported routing levels: node/instance/problem
85+
# Supported routing methods: input_choice/expert_choice/soft_moe/random (only for node/instance gating levels)
86+
from MOELayer import MoE
87+
moe_layer = MoE(input_size={INPUT_DIM}, output_size={OUTPUT_DIM}, hidden_size={HIDDEN_DIM},
88+
num_experts={NUM_EXPERTS}, k=2, T=1.0, noisy_gating=True,
89+
routing_level="node", routing_method="input_choice", moe_model="MLP")
90+
91+
# 1. tutel - https://github.com/microsoft/tutel
92+
from tutel import moe as tutel_moe
93+
moe_layer = tutel_moe.moe_layer(
94+
gate_type={'type': 'top', 'k': 2},
95+
model_dim={INPUT_DIM},
96+
experts={'type': 'ffn', 'count_per_node': {NUM_EXPERTS},
97+
'hidden_size_per_expert': {HIDDEN_DIM},
98+
'activation_fn': lambda x: torch.nn.functional.relu(x)},
99+
)
100+
```
101+
102+
103+
## Citation
104+
105+
```tex
106+
@inproceedings{zhou2024mvmoe,
107+
title ={MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts},
108+
author ={Jianan Zhou and Zhiguang Cao and Yaoxin Wu and Wen Song and Yining Ma and Jie Zhang and Chi Xu},
109+
booktitle ={International Conference on Machine Learning},
110+
year ={2024}
111+
}
112+
```
113+
114+
## Acknowledgments
115+
116+
* [ICML 2024 Review](https://github.com/RoyalSkye/Routing-MVMoE/blob/main/assets/Reviews_ICML24.md)
117+
* https://github.com/yd-kwon/POMO
118+
* https://github.com/davidmrau/mixture-of-experts

‎Reviews_ICML24.md ‎assets/Reviews_ICML24.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ We would like to thank the anonymous reviewers and (S)ACs of ICML 2024 for their
66

77
### Meta Review by Area Chair
88

9-
XXX
9+
This work represents a step towards cross-problem generalization in neural combinatorial optimization and thus the AC recommends acceptance. This is the first work to apply mixture-of-experts models to vehicle routing problems. The reviewers agree that the paper is well-written and the experiments are extensive. A weakness of this work is that mixture-of-experts yields marginal improvement upon a multi-task learning baseline (gap improvement of 0.1-0.5%). Furthermore, the proposed hierarchical gating mechanism does not appear to be advantageous beyond the default node-level input-choice gating. There is also concern about the limited scalability of the methods and the lack of discussion about other types of generalization in VRP, which was partially addressed by the authors' rebuttal.
1010

1111
----
1212

‎assets/mvmoe.png

149 KB
Loading

‎baselines/OR-Tools_baseline.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -338,8 +338,8 @@ def solve_or_tools_log(directory, name, depot, loc, demand, capacity, route_limi
338338
assignment = routing.SolveWithParameters(search_parameters)
339339
duration = time.time() - start
340340
if routing.status() not in [1, 2]:
341-
print(">> OR-Tools failed to solve instance - Solver status: {}".format(routing.status()))
342-
exit(0)
341+
print(">> OR-Tools failed to solve instance {} - Solver status: {}".format(name, routing.status()))
342+
return None, None, duration
343343
cost, route = print_solution(data, manager, routing, assignment, problem=problem, log_file=open(log_filename, 'w')) # route does not include the first and last node (i.e., depot)
344344
print("\n".join(["{}".format(r) for r in ([data['depot']] + route + [data['depot']])]), file=open(tour_filename, 'w'))
345345
save_dataset((route, duration), output_filename, disable_print=True)

‎test.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,8 @@ def args2dict(args):
3232
parser.add_argument('--problem', type=str, default="ALL", choices=["ALL", "CVRP", "OVRP", "VRPB", "VRPL", "VRPTW", "OVRPTW",
3333
"OVRPB", "OVRPL", "VRPBL", "VRPBTW", "VRPLTW",
3434
"OVRPBL", "OVRPBTW", "OVRPLTW", "VRPBLTW", "OVRPBLTW"])
35-
parser.add_argument('--problem_size', type=int, default=50)
36-
parser.add_argument('--pomo_size', type=int, default=50, help="the number of start node, should <= problem size")
35+
parser.add_argument('--problem_size', type=int, default=100)
36+
parser.add_argument('--pomo_size', type=int, default=100, help="the number of start node, should <= problem size")
3737

3838
# model_params
3939
parser.add_argument('--model_type', type=str, default="MOE_LIGHT", choices=["SINGLE", "MTL", "MOE", "MOE_LIGHT"])

‎train.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -31,8 +31,8 @@ def args2dict(args):
3131
parser.add_argument('--problem', type=str, default="Train_ALL", choices=["Train_ALL", "CVRP", "OVRP", "VRPB", "VRPL", "VRPTW", "OVRPTW",
3232
"OVRPB", "OVRPL", "VRPBL", "VRPBTW", "VRPLTW",
3333
"OVRPBL", "OVRPBTW", "OVRPLTW", "VRPBLTW", "OVRPBLTW"])
34-
parser.add_argument('--problem_size', type=int, default=50)
35-
parser.add_argument('--pomo_size', type=int, default=50, help="the number of start node, should <= problem size")
34+
parser.add_argument('--problem_size', type=int, default=100)
35+
parser.add_argument('--pomo_size', type=int, default=100, help="the number of start node, should <= problem size")
3636

3737
# model_params
3838
parser.add_argument('--model_type', type=str, default="MOE_LIGHT", choices=["SINGLE", "MTL", "MOE", "MOE_LIGHT"])

0 commit comments

Comments
 (0)