SynthArenav0.5.0
Navigation
  • Home
  • Stocks
  • Benchmarks
  • Runs
  • Leaderboard
  • Algorithms
Menus
    • How It Works
    • Benchmarks
    • Metrics
  • Why This Exists?
  • Changelog
  • Roadmap
  • Submit Results

From isChemist

Essays and software that make better scientific questions possible.

NewsletterStatus
  1. Home
  2. Leaderboard
mkt-cnv-160
Market
Stratified set of 160 convergent routes (40 each for lengths 2-5) that are solvable with buyables stock set.
Total Targets
160
Stock
buyables-stock
Acceptable Routes
Available
Show Top-K Metrics:
Overall Metrics
Model performance comparison for Solv-0[buyables-stock]
Actions
ASKCOS
v2.0.0
93.1%
93.1%
6.9%
15.0%
19.4%
80.0 min
$0.95
AiZynthFinder MCTS
v4.4.0
100.0%
98.1%
21.3%
38.1%
41.3%
17.8 min
$0.05
AiZynthFinder MCTS (High)
v4.4.0
100.0%
99.4%
20.6%
35.6%
38.8%
68.8 min
$0.20
AiZynthFinder Retro*
v4.4.0
100.0%
98.8%
8.1%
20.0%
27.5%
87.2 min
$0.26
AiZynthFinder Retro* (High)
v4.4.0
100.0%
99.4%
6.9%
20.0%
23.8%
327.6 min
$0.97
DMS Explorer XL
v1.1.3
100.0%
96.3%
36.9%
56.9%
58.1%
47.6 min
$1.02
Retro*
v0.1.0
98.8%
98.8%
34.4%
34.4%
34.4%
2.2 min
$0.01
Retro* (High)
v0.1.0
100.0%
100.0%
35.0%
35.0%
35.0%
3.1 min
$0.01
SynPlanner MCTS Rollout
v1.3.2
70.6%
70.0%
11.9%
20.6%
23.1%
38.8 min
$0.12
SynPlanner MCTS Val
v1.3.2
60.0%
59.4%
10.6%
16.3%
20.6%
59.7 min
$0.18
SynPlanner NMCS
v1.3.2
63.1%
63.1%
12.5%
17.5%
20.6%
161.1 min
$0.48
Syntheseus LocalRetro
v0.7.0
95.6%
95.6%
8.8%
16.9%
21.3%
42.1 min
$0.13
Efficiency Frontier
Visualizing the trade-off between performance and resources.
Tier-0 valid
ASKCOS
97.5%
92.5%
87.5%
95.0%
AiZynthFinder MCTS
100.0%
100.0%
100.0%
100.0%
AiZynthFinder MCTS (High)
100.0%
100.0%
100.0%
100.0%
AiZynthFinder Retro*
100.0%
100.0%
100.0%
100.0%
AiZynthFinder Retro* (High)
100.0%
100.0%
100.0%
100.0%
DMS Explorer XL
100.0%
100.0%
100.0%
100.0%
Retro*
97.5%
100.0%
97.5%
100.0%
Retro* (High)
100.0%
100.0%
100.0%
100.0%
SynPlanner MCTS Rollout
77.5%
77.5%
62.5%
65.0%
SynPlanner MCTS Val
77.5%
72.5%
45.0%
45.0%
SynPlanner NMCS
77.5%
70.0%
55.0%
50.0%
Syntheseus LocalRetro
100.0%
97.5%
87.5%
97.5%
Solv-0[buyables-stock]
ASKCOS
97.5%
92.5%
87.5%
95.0%
AiZynthFinder MCTS
100.0%
100.0%
92.5%
100.0%
AiZynthFinder MCTS (High)
100.0%
100.0%
97.5%
100.0%
AiZynthFinder Retro*
100.0%
100.0%
95.0%
100.0%
AiZynthFinder Retro* (High)
100.0%
100.0%
97.5%
100.0%
DMS Explorer XL
100.0%
100.0%
92.5%
92.5%
Retro*
97.5%
100.0%
97.5%
100.0%
Retro* (High)
100.0%
100.0%
100.0%
100.0%
SynPlanner MCTS Rollout
75.0%
77.5%
62.5%
65.0%
SynPlanner MCTS Val
75.0%
72.5%
45.0%
45.0%
SynPlanner NMCS
77.5%
70.0%
55.0%
50.0%
Syntheseus LocalRetro
100.0%
97.5%
87.5%
97.5%
Top-1
ASKCOS
12.5%
5.0%
5.0%
5.0%
AiZynthFinder MCTS
45.0%
20.0%
10.0%
10.0%
AiZynthFinder MCTS (High)
45.0%
20.0%
10.0%
7.5%
AiZynthFinder Retro*
20.0%
5.0%
2.5%
5.0%
AiZynthFinder Retro* (High)
12.5%
10.0%
2.5%
2.5%
DMS Explorer XL
50.0%
35.0%
20.0%
42.5%
Retro*
55.0%
37.5%
25.0%
20.0%
Retro* (High)
55.0%
37.5%
27.5%
20.0%
SynPlanner MCTS Rollout
27.5%
10.0%
7.5%
2.5%
SynPlanner MCTS Val
27.5%
10.0%
5.0%
0.0%
SynPlanner NMCS
30.0%
10.0%
7.5%
2.5%
Syntheseus LocalRetro
17.5%
7.5%
7.5%
2.5%
Top-5
ASKCOS
27.5%
15.0%
5.0%
12.5%
AiZynthFinder MCTS
72.5%
40.0%
25.0%
15.0%
AiZynthFinder MCTS (High)
80.0%
27.5%
22.5%
12.5%
AiZynthFinder Retro*
55.0%
7.5%
7.5%
10.0%
AiZynthFinder Retro* (High)
45.0%
17.5%
10.0%
7.5%
DMS Explorer XL
75.0%
55.0%
45.0%
52.5%
Retro*
55.0%
37.5%
25.0%
20.0%
Retro* (High)
55.0%
37.5%
27.5%
20.0%
SynPlanner MCTS Rollout
37.5%
27.5%
10.0%
7.5%
SynPlanner MCTS Val
37.5%
17.5%
7.5%
2.5%
SynPlanner NMCS
35.0%
22.5%
7.5%
5.0%
Syntheseus LocalRetro
40.0%
10.0%
10.0%
7.5%
Top-10
ASKCOS
37.5%
20.0%
5.0%
15.0%
AiZynthFinder MCTS
75.0%
50.0%
25.0%
15.0%
AiZynthFinder MCTS (High)
80.0%
37.5%
25.0%
12.5%
AiZynthFinder Retro*
65.0%
17.5%
12.5%
15.0%
AiZynthFinder Retro* (High)
50.0%
20.0%
12.5%
12.5%
DMS Explorer XL
75.0%
60.0%
45.0%
52.5%
Retro*
55.0%
37.5%
25.0%
20.0%
Retro* (High)
55.0%
37.5%
27.5%
20.0%
SynPlanner MCTS Rollout
40.0%
32.5%
10.0%
10.0%
SynPlanner MCTS Val
42.5%
27.5%
10.0%
2.5%
SynPlanner NMCS
40.0%
27.5%
10.0%
5.0%
Syntheseus LocalRetro
42.5%
22.5%
10.0%
10.0%
Efficiency Frontier
Visualizing the trade-off between performance and computational resources. Ideal models are in the top-left.
X-Axis:
Y-Axis: