Leaderboard | SynthArena by isChemist Group

mkt-lin-500-single-gt

General

500 targets with linear ground truth routes of variable length with all leaves in buyables. Part of the Procrustes suite.

Benchmark:

CuratedDeveloper

Total Targets

500

Stock

ASKCOS Buyables Stock

Acceptable Routes

Available

Show Top-K Metrics:

Overall Metrics

Model performance comparison for ASKCOS Buyables Stock

									Actions
ASKCOS	`v2.0.0`	94.4%	0.8%	3.4%	3.8%	-	-
AiZynthFinder MCTS	`v4.4.0`	97.6%	4.2%	12.6%	14.0%	-	-
AiZynthFinder MCTS (High)	`v4.4.0`	98.4%	4.4%	10.4%	11.6%	-	-
AiZynthFinder Retro*	`v4.4.0`	98.0%	1.6%	6.4%	9.0%	-	-
AiZynthFinder Retro* (High)	`v4.4.0`	99.0%	1.2%	5.0%	7.2%	-	-
DMS Explorer XL	`v1.1.3`	97.2%	27.6%	43.8%	50.2%	-	-
Retro*	`v1.0.0`	99.8%	9.0%	9.0%	9.0%	-	-
Retro* (High)	`v1.0.0`	99.8%	9.0%	9.0%	9.0%	-	-
SynPlanner MCTS Rollout	`v1.2.0`	3.6%	0.2%	0.2%	0.2%	-	-
SynPlanner MCTS Val	`v1.2.0`	2.0%	0.2%	0.2%	0.2%	-	-
Syntheseus LocalRetro	`v0.7.0`	94.0%	1.0%	4.0%	5.0%	-	-

Solvability


ASKCOS	98.0%	96.0%	96.0%	95.0%	87.0%
AiZynthFinder MCTS	100.0%	98.0%	99.0%	96.0%	95.0%
AiZynthFinder MCTS (High)	100.0%	99.0%	99.0%	98.0%	96.0%
AiZynthFinder Retro*	99.0%	99.0%	100.0%	98.0%	94.0%
AiZynthFinder Retro* (High)	100.0%	99.0%	100.0%	99.0%	97.0%
DMS Explorer XL	98.0%	99.0%	98.0%	96.0%	95.0%
Retro*	100.0%	100.0%	100.0%	99.0%	100.0%
Retro* (High)	100.0%	100.0%	100.0%	99.0%	100.0%
SynPlanner MCTS Rollout	10.0%	5.0%	0.0%	1.0%	2.0%
SynPlanner MCTS Val	8.0%	1.0%	0.0%	1.0%	0.0%
Syntheseus LocalRetro	95.0%	98.0%	96.0%	96.0%	85.0%

Top-1


ASKCOS	3.0%	1.0%	0.0%	0.0%	0.0%
AiZynthFinder MCTS	20.0%	0.0%	1.0%	0.0%	0.0%
AiZynthFinder MCTS (High)	21.0%	1.0%	0.0%	0.0%	0.0%
AiZynthFinder Retro*	6.0%	2.0%	0.0%	0.0%	0.0%
AiZynthFinder Retro* (High)	5.0%	1.0%	0.0%	0.0%	0.0%
DMS Explorer XL	39.0%	22.0%	23.0%	26.0%	28.0%
Retro*	15.0%	11.0%	9.0%	5.0%	5.0%
Retro* (High)	15.0%	11.0%	9.0%	5.0%	5.0%
SynPlanner MCTS Rollout	1.0%	0.0%	0.0%	0.0%	0.0%
SynPlanner MCTS Val	1.0%	0.0%	0.0%	0.0%	0.0%
Syntheseus LocalRetro	5.0%	0.0%	0.0%	0.0%	0.0%

Top-5


ASKCOS	9.0%	6.0%	2.0%	0.0%	0.0%
AiZynthFinder MCTS	42.0%	11.0%	7.0%	2.0%	1.0%
AiZynthFinder MCTS (High)	36.0%	11.0%	4.0%	0.0%	1.0%
AiZynthFinder Retro*	22.0%	7.0%	2.0%	1.0%	0.0%
AiZynthFinder Retro* (High)	21.0%	4.0%	0.0%	0.0%	0.0%
DMS Explorer XL	61.0%	43.0%	36.0%	37.0%	42.0%
Retro*	15.0%	11.0%	9.0%	5.0%	5.0%
Retro* (High)	15.0%	11.0%	9.0%	5.0%	5.0%
SynPlanner MCTS Rollout	1.0%	0.0%	0.0%	0.0%	0.0%
SynPlanner MCTS Val	1.0%	0.0%	0.0%	0.0%	0.0%
Syntheseus LocalRetro	15.0%	3.0%	1.0%	1.0%	0.0%

Top-10


ASKCOS	10.0%	7.0%	2.0%	0.0%	0.0%
AiZynthFinder MCTS	44.0%	14.0%	9.0%	2.0%	1.0%
AiZynthFinder MCTS (High)	41.0%	11.0%	4.0%	1.0%	1.0%
AiZynthFinder Retro*	32.0%	9.0%	3.0%	1.0%	0.0%
AiZynthFinder Retro* (High)	29.0%	7.0%	0.0%	0.0%	0.0%
DMS Explorer XL	66.0%	50.0%	47.0%	43.0%	45.0%
Retro*	15.0%	11.0%	9.0%	5.0%	5.0%
Retro* (High)	15.0%	11.0%	9.0%	5.0%	5.0%
SynPlanner MCTS Rollout	1.0%	0.0%	0.0%	0.0%	0.0%
SynPlanner MCTS Val	1.0%	0.0%	0.0%	0.0%	0.0%
Syntheseus LocalRetro	18.0%	5.0%	1.0%	1.0%	0.0%