SynthArenav0.5.0
Navigation
  • Home
  • Stocks
  • Benchmarks
  • Runs
  • Leaderboard
  • Algorithms
Menus
    • How It Works
    • Benchmarks
    • Metrics
  • Why This Exists?
  • Changelog
  • Roadmap
  • Submit Results

From isChemist

Essays and software that make better scientific questions possible.

NewsletterStatus
  1. Home
  2. Leaderboard
ref-lin-600
Reference
Stratified set of 600 linear routes (100 each for lengths 2-7).
Total Targets
600
Stock
n5-stock
Acceptable Routes
Available
Show Top-K Metrics:
Overall Metrics
Model performance comparison for Solv-0[n5-stock]
Actions
ASKCOS
v2.0.0
72.7%
35.5%
0.7%
2.3%
2.7%
-
300.9 min
$3.58
AiZynthFinder MCTS
v4.4.0
100.0%
82.8%
10.2%
20.5%
21.5%
-
82.1 min
$0.24
AiZynthFinder MCTS (High)
v4.4.0
100.0%
88.8%
9.7%
19.2%
21.5%
-
250.9 min
$0.75
AiZynthFinder Retro*
v4.4.0
100.0%
88.2%
7.3%
17.5%
21.7%
-
294.1 min
$0.87
AiZynthFinder Retro* (High)
v4.4.0
100.0%
95.0%
6.2%
13.5%
16.5%
-
1099.2 min
$3.27
DMS Explorer XL
v1.1.3
100.0%
76.8%
31.5%
43.7%
45.2%
-
185.2 min
$3.98
Retro*
v0.1.0
95.8%
95.8%
25.5%
25.5%
25.5%
-
16.6 min
$0.05
Retro* (High)
v0.1.0
97.8%
97.8%
25.8%
25.8%
25.8%
-
34.6 min
$0.10
SynPlanner MCTS Rollout
v1.3.2
55.8%
55.2%
9.3%
12.2%
12.8%
-
150.9 min
$0.45
SynPlanner MCTS Val
v1.3.2
42.0%
42.0%
8.3%
11.3%
12.3%
-
237.6 min
$0.71
SynPlanner NMCS
v1.3.2
47.2%
46.8%
9.2%
12.0%
12.8%
-
610.6 min
$1.82
Syntheseus LocalRetro
v0.7.0
67.2%
67.2%
5.0%
10.8%
13.0%
-
154.2 min
$0.46
Efficiency Frontier
Visualizing the trade-off between performance and resources.
Tier-0 valid
ASKCOS
82.0%
77.0%
79.0%
74.0%
66.0%
58.0%
AiZynthFinder MCTS
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
AiZynthFinder MCTS (High)
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
AiZynthFinder Retro*
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
AiZynthFinder Retro* (High)
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
DMS Explorer XL
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
Retro*
99.0%
99.0%
96.0%
95.0%
95.0%
91.0%
Retro* (High)
99.0%
100.0%
98.0%
97.0%
98.0%
95.0%
SynPlanner MCTS Rollout
73.0%
60.0%
63.0%
58.0%
44.0%
37.0%
SynPlanner MCTS Val
73.0%
56.0%
51.0%
39.0%
23.0%
10.0%
SynPlanner NMCS
75.0%
59.0%
59.0%
39.0%
33.0%
18.0%
Syntheseus LocalRetro
79.0%
69.0%
73.0%
72.0%
62.0%
48.0%
Solv-0[n5-stock]
ASKCOS
53.0%
46.0%
38.0%
34.0%
26.0%
16.0%
AiZynthFinder MCTS
96.0%
92.0%
81.0%
85.0%
76.0%
67.0%
AiZynthFinder MCTS (High)
98.0%
95.0%
87.0%
88.0%
86.0%
79.0%
AiZynthFinder Retro*
98.0%
94.0%
89.0%
87.0%
85.0%
76.0%
AiZynthFinder Retro* (High)
98.0%
99.0%
95.0%
95.0%
94.0%
89.0%
DMS Explorer XL
94.0%
80.0%
73.0%
82.0%
65.0%
67.0%
Retro*
99.0%
99.0%
96.0%
95.0%
95.0%
91.0%
Retro* (High)
99.0%
100.0%
98.0%
97.0%
98.0%
95.0%
SynPlanner MCTS Rollout
73.0%
60.0%
62.0%
57.0%
43.0%
36.0%
SynPlanner MCTS Val
73.0%
56.0%
51.0%
39.0%
23.0%
10.0%
SynPlanner NMCS
74.0%
59.0%
59.0%
39.0%
32.0%
18.0%
Syntheseus LocalRetro
79.0%
69.0%
73.0%
72.0%
62.0%
48.0%
Top-1
ASKCOS
3.0%
0.0%
0.0%
1.0%
0.0%
0.0%
AiZynthFinder MCTS
43.0%
12.0%
3.0%
3.0%
0.0%
0.0%
AiZynthFinder MCTS (High)
43.0%
13.0%
1.0%
1.0%
0.0%
0.0%
AiZynthFinder Retro*
31.0%
9.0%
3.0%
1.0%
0.0%
0.0%
AiZynthFinder Retro* (High)
28.0%
6.0%
2.0%
1.0%
0.0%
0.0%
DMS Explorer XL
49.0%
30.0%
26.0%
37.0%
26.0%
21.0%
Retro*
49.0%
36.0%
23.0%
20.0%
14.0%
11.0%
Retro* (High)
48.0%
37.0%
23.0%
21.0%
14.0%
12.0%
SynPlanner MCTS Rollout
37.0%
12.0%
5.0%
2.0%
0.0%
0.0%
SynPlanner MCTS Val
39.0%
11.0%
0.0%
0.0%
0.0%
0.0%
SynPlanner NMCS
37.0%
12.0%
6.0%
0.0%
0.0%
0.0%
Syntheseus LocalRetro
17.0%
8.0%
3.0%
2.0%
0.0%
0.0%
Top-5
ASKCOS
7.0%
3.0%
3.0%
1.0%
0.0%
0.0%
AiZynthFinder MCTS
64.0%
35.0%
13.0%
9.0%
2.0%
0.0%
AiZynthFinder MCTS (High)
69.0%
32.0%
10.0%
4.0%
0.0%
0.0%
AiZynthFinder Retro*
57.0%
27.0%
10.0%
7.0%
4.0%
0.0%
AiZynthFinder Retro* (High)
52.0%
16.0%
7.0%
5.0%
1.0%
0.0%
DMS Explorer XL
64.0%
53.0%
36.0%
45.0%
31.0%
33.0%
Retro*
49.0%
36.0%
23.0%
20.0%
14.0%
11.0%
Retro* (High)
48.0%
37.0%
23.0%
21.0%
14.0%
12.0%
SynPlanner MCTS Rollout
44.0%
17.0%
8.0%
3.0%
1.0%
0.0%
SynPlanner MCTS Val
46.0%
20.0%
2.0%
0.0%
0.0%
0.0%
SynPlanner NMCS
41.0%
22.0%
9.0%
0.0%
0.0%
0.0%
Syntheseus LocalRetro
25.0%
18.0%
12.0%
5.0%
4.0%
1.0%
Top-10
ASKCOS
8.0%
4.0%
3.0%
1.0%
0.0%
0.0%
AiZynthFinder MCTS
66.0%
38.0%
14.0%
9.0%
2.0%
0.0%
AiZynthFinder MCTS (High)
73.0%
39.0%
12.0%
5.0%
0.0%
0.0%
AiZynthFinder Retro*
65.0%
36.0%
12.0%
10.0%
7.0%
0.0%
AiZynthFinder Retro* (High)
58.0%
27.0%
8.0%
5.0%
1.0%
0.0%
DMS Explorer XL
66.0%
53.0%
39.0%
46.0%
33.0%
34.0%
Retro*
49.0%
36.0%
23.0%
20.0%
14.0%
11.0%
Retro* (High)
48.0%
37.0%
23.0%
21.0%
14.0%
12.0%
SynPlanner MCTS Rollout
45.0%
19.0%
9.0%
3.0%
1.0%
0.0%
SynPlanner MCTS Val
49.0%
23.0%
2.0%
0.0%
0.0%
0.0%
SynPlanner NMCS
42.0%
24.0%
11.0%
0.0%
0.0%
0.0%
Syntheseus LocalRetro
29.0%
22.0%
13.0%
7.0%
6.0%
1.0%
Efficiency Frontier
Visualizing the trade-off between performance and computational resources. Ideal models are in the top-left.
X-Axis:
Y-Axis: