SynthArenav0.5.0
Navigation
  • Home
  • Stocks
  • Benchmarks
  • Runs
  • Leaderboard
  • Algorithms
Menus
    • How It Works
    • Benchmarks
    • Metrics
  • Why This Exists?
  • Changelog
  • Roadmap
  • Submit Results

From isChemist

Essays and software that make better scientific questions possible.

NewsletterStatus
  1. Home
  2. Leaderboard
ref-lng-84
Reference
84 targets with extra long (8-10 steps) ground truth routes.
Total Targets
84
Stock
n1-n5-stock
Acceptable Routes
Available
Show Top-K Metrics:
Overall Metrics
Model performance comparison for Solv-0[n1-n5-stock]
Actions
ASKCOS
v2.0.0
44.0%
15.5%
0.0%
0.0%
0.0%
-
42.7 min
$0.51
AiZynthFinder MCTS
v4.4.0
100.0%
46.4%
0.0%
0.0%
0.0%
-
12.5 min
$0.04
AiZynthFinder MCTS (High)
v4.4.0
100.0%
59.5%
0.0%
0.0%
0.0%
-
38.1 min
$0.11
AiZynthFinder Retro*
v4.4.0
100.0%
61.9%
0.0%
0.0%
0.0%
-
44.4 min
$0.13
AiZynthFinder Retro* (High)
v4.4.0
100.0%
76.2%
0.0%
0.0%
0.0%
-
187.9 min
$0.56
DMS Explorer XL
v1.1.3
100.0%
72.6%
45.2%
50.0%
50.0%
-
43.8 min
$0.94
Retro*
v0.1.0
86.9%
86.9%
8.3%
8.3%
8.3%
-
5.4 min
$0.02
Retro* (High)
v0.1.0
94.0%
94.0%
8.3%
8.3%
8.3%
-
12.1 min
$0.04
SynPlanner MCTS Rollout
v1.3.2
33.3%
33.3%
0.0%
0.0%
0.0%
-
23.2 min
$0.07
SynPlanner MCTS Val
v1.3.2
9.5%
9.5%
0.0%
0.0%
0.0%
-
38.6 min
$0.11
SynPlanner NMCS
v1.3.2
11.9%
11.9%
0.0%
0.0%
0.0%
-
85.2 min
$0.25
Syntheseus LocalRetro
v0.7.0
36.9%
36.9%
0.0%
0.0%
0.0%
-
23.2 min
$0.07
Efficiency Frontier
Visualizing the trade-off between performance and resources.
Tier-0 valid
ASKCOS
48.3%
36.4%
25.0%
AiZynthFinder MCTS
100.0%
100.0%
100.0%
AiZynthFinder MCTS (High)
100.0%
100.0%
100.0%
AiZynthFinder Retro*
100.0%
100.0%
100.0%
AiZynthFinder Retro* (High)
100.0%
100.0%
100.0%
DMS Explorer XL
100.0%
100.0%
100.0%
Retro*
87.9%
90.9%
50.0%
Retro* (High)
93.1%
100.0%
75.0%
SynPlanner MCTS Rollout
34.5%
31.8%
25.0%
SynPlanner MCTS Val
13.8%
0.0%
0.0%
SynPlanner NMCS
17.2%
0.0%
0.0%
Syntheseus LocalRetro
36.2%
36.4%
50.0%
Solv-0[n1-n5-stock]
ASKCOS
19.0%
9.1%
0.0%
AiZynthFinder MCTS
51.7%
36.4%
25.0%
AiZynthFinder MCTS (High)
67.2%
45.5%
25.0%
AiZynthFinder Retro*
70.7%
45.5%
25.0%
AiZynthFinder Retro* (High)
82.8%
63.6%
50.0%
DMS Explorer XL
72.4%
81.8%
25.0%
Retro*
87.9%
90.9%
50.0%
Retro* (High)
93.1%
100.0%
75.0%
SynPlanner MCTS Rollout
34.5%
31.8%
25.0%
SynPlanner MCTS Val
13.8%
0.0%
0.0%
SynPlanner NMCS
17.2%
0.0%
0.0%
Syntheseus LocalRetro
36.2%
36.4%
50.0%
Top-1
ASKCOS
0.0%
0.0%
0.0%
AiZynthFinder MCTS
0.0%
0.0%
0.0%
AiZynthFinder MCTS (High)
0.0%
0.0%
0.0%
AiZynthFinder Retro*
0.0%
0.0%
0.0%
AiZynthFinder Retro* (High)
0.0%
0.0%
0.0%
DMS Explorer XL
37.9%
68.2%
25.0%
Retro*
6.9%
13.6%
0.0%
Retro* (High)
6.9%
13.6%
0.0%
SynPlanner MCTS Rollout
0.0%
0.0%
0.0%
SynPlanner MCTS Val
0.0%
0.0%
0.0%
SynPlanner NMCS
0.0%
0.0%
0.0%
Syntheseus LocalRetro
0.0%
0.0%
0.0%
Top-5
ASKCOS
0.0%
0.0%
0.0%
AiZynthFinder MCTS
0.0%
0.0%
0.0%
AiZynthFinder MCTS (High)
0.0%
0.0%
0.0%
AiZynthFinder Retro*
0.0%
0.0%
0.0%
AiZynthFinder Retro* (High)
0.0%
0.0%
0.0%
DMS Explorer XL
44.8%
68.2%
25.0%
Retro*
6.9%
13.6%
0.0%
Retro* (High)
6.9%
13.6%
0.0%
SynPlanner MCTS Rollout
0.0%
0.0%
0.0%
SynPlanner MCTS Val
0.0%
0.0%
0.0%
SynPlanner NMCS
0.0%
0.0%
0.0%
Syntheseus LocalRetro
0.0%
0.0%
0.0%
Top-10
ASKCOS
0.0%
0.0%
0.0%
AiZynthFinder MCTS
0.0%
0.0%
0.0%
AiZynthFinder MCTS (High)
0.0%
0.0%
0.0%
AiZynthFinder Retro*
0.0%
0.0%
0.0%
AiZynthFinder Retro* (High)
0.0%
0.0%
0.0%
DMS Explorer XL
44.8%
68.2%
25.0%
Retro*
6.9%
13.6%
0.0%
Retro* (High)
6.9%
13.6%
0.0%
SynPlanner MCTS Rollout
0.0%
0.0%
0.0%
SynPlanner MCTS Val
0.0%
0.0%
0.0%
SynPlanner NMCS
0.0%
0.0%
0.0%
Syntheseus LocalRetro
0.0%
0.0%
0.0%
Efficiency Frontier
Visualizing the trade-off between performance and computational resources. Ideal models are in the top-left.
X-Axis:
Y-Axis: