Leaderboard | SynthArena by isChemist Group

ref-lng-84

Reference

84 targets with extra long ground truth routes

Benchmark:

CuratedDeveloper

Total Targets

84

Stock

n1+n5 Stock

Acceptable Routes

Available

Show Top-K Metrics:

Overall Metrics

Model performance comparison for n1+n5 Stock

										Actions
AiZynthFinder MCTS	`v4.4.0`	46.4%	0.0%	0.0%	0.0%		12.5 min	$0.04
AiZynthFinder MCTS (High)	`v4.4.0`	59.5%	0.0%	0.0%	0.0%		38.1 min	$0.11
AiZynthFinder Retro*	`v4.4.0`	61.9%	0.0%	0.0%	0.0%		44.4 min	$0.13
AiZynthFinder Retro* (High)	`v4.4.0`	76.2%	0.0%	0.0%	0.0%		187.9 min	$0.56
DMS Explorer XL	`v1.1.3`	72.6%	45.2%	50.0%	50.0%		43.8 min	$0.94
Retro*	`v1.0.0`	86.9%	8.3%	8.3%	8.3%		5.4 min	$0.02
Retro* (High)	`v1.0.0`	94.0%	8.3%	8.3%	8.3%		12.1 min	$0.04
SynPlanner MCTS Rollout	`v1.2.0`	0.0%	0.0%	0.0%	0.0%		35.0 min	$0.10
SynPlanner MCTS Val	`v1.2.0`	0.0%	0.0%	0.0%	0.0%		35.5 min	$0.11
SynPlanner NMCS	`v1.3.2`	11.9%	0.0%	0.0%	0.0%	-	85.2 min	$0.25
Syntheseus LocalRetro	`v0.7.0`	36.9%	0.0%	0.0%	0.0%		23.2 min	$0.07

Solvability


AiZynthFinder MCTS	51.7%	36.4%	25.0%
AiZynthFinder MCTS (High)	67.2%	45.5%	25.0%
AiZynthFinder Retro*	70.7%	45.5%	25.0%
AiZynthFinder Retro* (High)	82.8%	63.6%	50.0%
DMS Explorer XL	72.4%	81.8%	25.0%
Retro*	87.9%	90.9%	50.0%
Retro* (High)	93.1%	100.0%	75.0%
SynPlanner MCTS Rollout	0.0%	0.0%	0.0%
SynPlanner MCTS Val	0.0%	0.0%	0.0%
SynPlanner NMCS	17.2%	0.0%	0.0%
Syntheseus LocalRetro	36.2%	36.4%	50.0%

Top-1


AiZynthFinder MCTS	0.0%	0.0%	0.0%
AiZynthFinder MCTS (High)	0.0%	0.0%	0.0%
AiZynthFinder Retro*	0.0%	0.0%	0.0%
AiZynthFinder Retro* (High)	0.0%	0.0%	0.0%
DMS Explorer XL	37.9%	68.2%	25.0%
Retro*	6.9%	13.6%	0.0%
Retro* (High)	6.9%	13.6%	0.0%
SynPlanner MCTS Rollout	0.0%	0.0%	0.0%
SynPlanner MCTS Val	0.0%	0.0%	0.0%
SynPlanner NMCS	0.0%	0.0%	0.0%
Syntheseus LocalRetro	0.0%	0.0%	0.0%

Top-5


AiZynthFinder MCTS	0.0%	0.0%	0.0%
AiZynthFinder MCTS (High)	0.0%	0.0%	0.0%
AiZynthFinder Retro*	0.0%	0.0%	0.0%
AiZynthFinder Retro* (High)	0.0%	0.0%	0.0%
DMS Explorer XL	44.8%	68.2%	25.0%
Retro*	6.9%	13.6%	0.0%
Retro* (High)	6.9%	13.6%	0.0%
SynPlanner MCTS Rollout	0.0%	0.0%	0.0%
SynPlanner MCTS Val	0.0%	0.0%	0.0%
SynPlanner NMCS	0.0%	0.0%	0.0%
Syntheseus LocalRetro	0.0%	0.0%	0.0%

Top-10


AiZynthFinder MCTS	0.0%	0.0%	0.0%
AiZynthFinder MCTS (High)	0.0%	0.0%	0.0%
AiZynthFinder Retro*	0.0%	0.0%	0.0%
AiZynthFinder Retro* (High)	0.0%	0.0%	0.0%
DMS Explorer XL	44.8%	68.2%	25.0%
Retro*	6.9%	13.6%	0.0%
Retro* (High)	6.9%	13.6%	0.0%
SynPlanner MCTS Rollout	0.0%	0.0%	0.0%
SynPlanner MCTS Val	0.0%	0.0%	0.0%
SynPlanner NMCS	0.0%	0.0%	0.0%
Syntheseus LocalRetro	0.0%	0.0%	0.0%