SynthArenav0.4.0
Navigation
  • Home
  • Stocks
  • Benchmarks
  • Runs
  • Leaderboard
  • Algorithms
Menus
    • How It Works
    • Benchmarks
    • Metrics
  • Why This Exists?
  • Changelog
  • Roadmap
  • Submit Results
  1. Home
  2. Leaderboard
mkt-lin-500
Market
500 targets with linear ground truth routes of variable length with all leaves in buyables. Part of the Procrustes suite.
Total Targets
500
Stock
ASKCOS Buyables Stock
Acceptable Routes
Available
Show Top-K Metrics:
Overall Metrics
Model performance comparison for ASKCOS Buyables Stock
Actions
ASKCOS
v2.0.0
94.4%
6.2%
14.8%
16.2%
245.7 min
$2.92
AiZynthFinder MCTS
v4.4.0
97.6%
17.8%
33.0%
35.8%
51.0 min
$0.15
AiZynthFinder MCTS (High)
v4.4.0
98.4%
17.4%
30.4%
33.4%
173.2 min
$0.52
AiZynthFinder Retro*
v4.4.0
98.0%
10.4%
22.2%
28.6%
219.2 min
$0.65
AiZynthFinder Retro* (High)
v4.4.0
99.0%
9.2%
20.6%
25.4%
859.1 min
$2.56
DMS Explorer XL
v1.1.3
97.2%
31.4%
49.8%
55.4%
121.2 min
$2.61
Retro*
v1.0.0
99.8%
22.2%
22.2%
22.2%
4.2 min
$0.01
Retro* (High)
v1.0.0
99.8%
22.2%
22.2%
22.2%
5.2 min
$0.02
SynPlanner MCTS Rollout
v1.3.2
73.6%
11.6%
18.4%
21.0%
106.8 min
$0.32
SynPlanner MCTS Val
v1.3.2
71.4%
12.6%
18.4%
20.6%
169.1 min
$0.50
SynPlanner NMCS
v1.3.2
71.4%
12.0%
19.0%
21.2%
509.0 min
$1.51
Syntheseus LocalRetro
v0.7.0
94.0%
8.6%
15.2%
18.8%
99.4 min
$0.30
Efficiency Frontier
Visualizing the trade-off between performance and resources.
Solvability
ASKCOS
98.0%
96.0%
96.0%
95.0%
87.0%
AiZynthFinder MCTS
100.0%
98.0%
99.0%
96.0%
95.0%
AiZynthFinder MCTS (High)
100.0%
99.0%
99.0%
98.0%
96.0%
AiZynthFinder Retro*
99.0%
99.0%
100.0%
98.0%
94.0%
AiZynthFinder Retro* (High)
100.0%
99.0%
100.0%
99.0%
97.0%
DMS Explorer XL
98.0%
99.0%
98.0%
96.0%
95.0%
Retro*
100.0%
100.0%
100.0%
99.0%
100.0%
Retro* (High)
100.0%
100.0%
100.0%
99.0%
100.0%
SynPlanner MCTS Rollout
70.0%
85.0%
74.0%
72.0%
67.0%
SynPlanner MCTS Val
81.0%
89.0%
77.0%
62.0%
48.0%
SynPlanner NMCS
76.0%
85.0%
77.0%
63.0%
56.0%
Syntheseus LocalRetro
95.0%
98.0%
96.0%
96.0%
85.0%
Top-1
ASKCOS
13.0%
8.0%
4.0%
4.0%
2.0%
AiZynthFinder MCTS
49.0%
12.0%
13.0%
9.0%
6.0%
AiZynthFinder MCTS (High)
50.0%
13.0%
11.0%
9.0%
4.0%
AiZynthFinder Retro*
26.0%
10.0%
5.0%
8.0%
3.0%
AiZynthFinder Retro* (High)
25.0%
7.0%
5.0%
5.0%
4.0%
DMS Explorer XL
39.0%
28.0%
27.0%
32.0%
31.0%
Retro*
40.0%
22.0%
17.0%
15.0%
17.0%
Retro* (High)
40.0%
22.0%
17.0%
15.0%
17.0%
SynPlanner MCTS Rollout
28.0%
17.0%
6.0%
7.0%
0.0%
SynPlanner MCTS Val
31.0%
18.0%
8.0%
6.0%
0.0%
SynPlanner NMCS
29.0%
20.0%
5.0%
5.0%
1.0%
Syntheseus LocalRetro
22.0%
9.0%
6.0%
3.0%
3.0%
Top-5
ASKCOS
31.0%
18.0%
10.0%
10.0%
5.0%
AiZynthFinder MCTS
77.0%
38.0%
25.0%
16.0%
9.0%
AiZynthFinder MCTS (High)
72.0%
36.0%
22.0%
14.0%
8.0%
AiZynthFinder Retro*
55.0%
20.0%
15.0%
14.0%
7.0%
AiZynthFinder Retro* (High)
56.0%
19.0%
14.0%
8.0%
6.0%
DMS Explorer XL
62.0%
51.0%
45.0%
44.0%
47.0%
Retro*
40.0%
22.0%
17.0%
15.0%
17.0%
Retro* (High)
40.0%
22.0%
17.0%
15.0%
17.0%
SynPlanner MCTS Rollout
39.0%
31.0%
9.0%
9.0%
4.0%
SynPlanner MCTS Val
47.0%
27.0%
9.0%
8.0%
1.0%
SynPlanner NMCS
41.0%
30.0%
13.0%
8.0%
3.0%
Syntheseus LocalRetro
33.0%
18.0%
11.0%
8.0%
6.0%
Top-10
ASKCOS
33.0%
20.0%
11.0%
10.0%
7.0%
AiZynthFinder MCTS
81.0%
43.0%
28.0%
18.0%
9.0%
AiZynthFinder MCTS (High)
78.0%
41.0%
23.0%
16.0%
9.0%
AiZynthFinder Retro*
68.0%
32.0%
19.0%
16.0%
8.0%
AiZynthFinder Retro* (High)
66.0%
27.0%
15.0%
13.0%
6.0%
DMS Explorer XL
67.0%
55.0%
54.0%
51.0%
50.0%
Retro*
40.0%
22.0%
17.0%
15.0%
17.0%
Retro* (High)
40.0%
22.0%
17.0%
15.0%
17.0%
SynPlanner MCTS Rollout
42.0%
32.0%
15.0%
11.0%
5.0%
SynPlanner MCTS Val
50.0%
31.0%
12.0%
9.0%
1.0%
SynPlanner NMCS
43.0%
33.0%
15.0%
11.0%
4.0%
Syntheseus LocalRetro
36.0%
24.0%
14.0%
12.0%
8.0%
Efficiency Frontier
Visualizing the trade-off between performance and computational resources. Ideal models are in the top-left.
X-Axis:
Y-Axis: