SynthArenav0.5.0
Navigation
  • Home
  • Stocks
  • Benchmarks
  • Runs
  • Leaderboard
  • Algorithms
Menus
    • How It Works
    • Benchmarks
    • Metrics
  • Why This Exists?
  • Changelog
  • Roadmap
  • Submit Results

From isChemist

Essays and software that make better scientific questions possible.

NewsletterStatus
  1. Home
  2. Leaderboard
ref-cnv-400
Reference
Stratified set of 400 convergent routes (100 each for lengths 2-5).
Total Targets
400
Stock
n5-stock
Acceptable Routes
Available
Show Top-K Metrics:
Overall Metrics
Model performance comparison for Solv-0[n5-stock]
Actions
ASKCOS
v2.0.0
67.5%
22.0%
0.0%
0.3%
0.3%
-
202.1 min
$2.41
AiZynthFinder MCTS
v4.4.0
100.0%
81.3%
17.3%
25.8%
27.8%
-
61.1 min
$0.18
AiZynthFinder MCTS (High)
v4.4.0
100.0%
87.3%
19.0%
29.3%
32.5%
-
197.4 min
$0.59
AiZynthFinder Retro*
v4.4.0
100.0%
87.5%
8.0%
16.3%
21.0%
-
319.2 min
$0.95
AiZynthFinder Retro* (High)
v4.4.0
100.0%
94.0%
5.8%
12.5%
15.3%
-
1024.8 min
$3.05
DMS Explorer XL
v1.1.3
100.0%
77.5%
37.3%
47.0%
47.8%
-
131.9 min
$2.84
Retro*
v0.1.0
96.5%
96.5%
31.8%
31.8%
31.8%
-
12.0 min
$0.04
Retro* (High)
v0.1.0
99.5%
99.5%
31.8%
31.8%
31.8%
-
16.7 min
$0.05
SynPlanner MCTS Rollout
v1.3.2
46.5%
46.3%
3.8%
5.3%
5.5%
-
113.6 min
$0.34
SynPlanner MCTS Val
v1.3.2
27.0%
26.8%
3.0%
5.5%
6.5%
-
198.3 min
$0.59
SynPlanner NMCS
v1.3.2
31.3%
31.0%
5.5%
8.5%
9.0%
-
407.4 min
$1.21
Syntheseus LocalRetro
v0.7.0
74.8%
74.8%
6.0%
11.3%
13.3%
-
104.3 min
$0.31
Efficiency Frontier
Visualizing the trade-off between performance and computational resources. Ideal models are in the top-left.
X-Axis:
Y-Axis:
Tier-0 valid
ASKCOS
81.0%
62.0%
63.0%
64.0%
AiZynthFinder MCTS
100.0%
100.0%
100.0%
100.0%
AiZynthFinder MCTS (High)
100.0%
100.0%
100.0%
100.0%
AiZynthFinder Retro*
100.0%
100.0%
100.0%
100.0%
AiZynthFinder Retro* (High)
100.0%
100.0%
100.0%
100.0%
DMS Explorer XL
100.0%
100.0%
100.0%
100.0%
Retro*
97.0%
98.0%
93.0%
98.0%
Retro* (High)
100.0%
100.0%
98.0%
100.0%
SynPlanner MCTS Rollout
60.0%
45.0%
43.0%
38.0%
SynPlanner MCTS Val
55.0%
27.0%
17.0%
9.0%
SynPlanner NMCS
51.0%
30.0%
24.0%
20.0%
Syntheseus LocalRetro
84.0%
84.0%
65.0%
66.0%
Solv-0[n5-stock]
ASKCOS
31.0%
22.0%
21.0%
14.0%
AiZynthFinder MCTS
93.0%
87.0%
69.0%
76.0%
AiZynthFinder MCTS (High)
98.0%
91.0%
80.0%
80.0%
AiZynthFinder Retro*
97.0%
90.0%
83.0%
80.0%
AiZynthFinder Retro* (High)
99.0%
95.0%
90.0%
92.0%
DMS Explorer XL
86.0%
72.0%
77.0%
75.0%
Retro*
97.0%
98.0%
93.0%
98.0%
Retro* (High)
100.0%
100.0%
98.0%
100.0%
SynPlanner MCTS Rollout
60.0%
44.0%
43.0%
38.0%
SynPlanner MCTS Val
55.0%
27.0%
16.0%
9.0%
SynPlanner NMCS
51.0%
30.0%
24.0%
19.0%
Syntheseus LocalRetro
84.0%
84.0%
65.0%
66.0%
Top-1
ASKCOS
0.0%
0.0%
0.0%
0.0%
AiZynthFinder MCTS
45.0%
14.0%
6.0%
4.0%
AiZynthFinder MCTS (High)
53.0%
13.0%
6.0%
4.0%
AiZynthFinder Retro*
21.0%
5.0%
5.0%
1.0%
AiZynthFinder Retro* (High)
17.0%
2.0%
4.0%
0.0%
DMS Explorer XL
39.0%
31.0%
42.0%
37.0%
Retro*
52.0%
27.0%
20.0%
28.0%
Retro* (High)
52.0%
28.0%
20.0%
27.0%
SynPlanner MCTS Rollout
11.0%
3.0%
1.0%
0.0%
SynPlanner MCTS Val
11.0%
1.0%
0.0%
0.0%
SynPlanner NMCS
17.0%
4.0%
1.0%
0.0%
Syntheseus LocalRetro
15.0%
7.0%
1.0%
1.0%
Top-5
ASKCOS
1.0%
0.0%
0.0%
0.0%
AiZynthFinder MCTS
57.0%
25.0%
11.0%
10.0%
AiZynthFinder MCTS (High)
72.0%
29.0%
12.0%
4.0%
AiZynthFinder Retro*
37.0%
11.0%
11.0%
6.0%
AiZynthFinder Retro* (High)
27.0%
11.0%
9.0%
3.0%
DMS Explorer XL
56.0%
42.0%
46.0%
44.0%
Retro*
52.0%
27.0%
20.0%
28.0%
Retro* (High)
52.0%
28.0%
20.0%
27.0%
SynPlanner MCTS Rollout
15.0%
3.0%
3.0%
0.0%
SynPlanner MCTS Val
21.0%
1.0%
0.0%
0.0%
SynPlanner NMCS
25.0%
5.0%
4.0%
0.0%
Syntheseus LocalRetro
27.0%
12.0%
2.0%
4.0%
Top-10
ASKCOS
1.0%
0.0%
0.0%
0.0%
AiZynthFinder MCTS
61.0%
28.0%
12.0%
10.0%
AiZynthFinder MCTS (High)
78.0%
34.0%
13.0%
5.0%
AiZynthFinder Retro*
44.0%
17.0%
15.0%
8.0%
AiZynthFinder Retro* (High)
35.0%
12.0%
11.0%
3.0%
DMS Explorer XL
57.0%
42.0%
47.0%
45.0%
Retro*
52.0%
27.0%
20.0%
28.0%
Retro* (High)
52.0%
28.0%
20.0%
27.0%
SynPlanner MCTS Rollout
16.0%
3.0%
3.0%
0.0%
SynPlanner MCTS Val
25.0%
1.0%
0.0%
0.0%
SynPlanner NMCS
26.0%
6.0%
4.0%
0.0%
Syntheseus LocalRetro
28.0%
14.0%
6.0%
5.0%