LFORLA
An AI agent evaluation system and organization that maintains benchmarks such as JS 3D Builder, emphasizing fixed scoring criteria, server-side static analysis and comparability across models.
Timeline (last 90 days)
The LFORLA evaluation mechanism contains four rules: fixed prompts, a fixed judging model, scoring split by dimension, and public verbatim answer records.
On the current Team Recruitment (Oracle) leaderboard, GLM 5.2 (LFORLA's own submission) scores 78.0.
The complete results and verbatim responses of Team Recruitment (Oracle) can be viewed at lforla.org.
JS 3D Builder (Three.js) is a benchmark whose task is, starting from written architectural specifications, to procedurally generate 3D building code using Three.js, scored by server-side static analysis, without executing the code or using an LLM judge.
The author of the ranking benchmark submitted his own model GLM 5.2 to his own benchmark, which constitutes a conflict of interest.
The Function Recovery dimension uses a binary threshold of 0.75, where a token similarity score of 0.75 or above counts as passing.
Nemotron 3 Ultra (via opencode) scored 43.49 on this benchmark.
The reverse engineering benchmark task in LFORLA is to recover the C source code of a function from a stripped binary.
Token similarity does not equal semantic correctness; a submission may have high token overlap while still being structurally incorrect.
Model selection should compare along dimensions that match the use case rather than looking only at overall rankings; complete source code reconstruction requires looking at Completeness and Function Recovery, while token-level fidelity requires looking at Code Similarity.
5 more timeline events
All Facts (13)
The LFORLA evaluation mechanism contains four rules: fixed prompts, a fixed judging model, scoring split by dimension, and public verbatim answer records.
50%UnverifiedOn the current Team Recruitment (Oracle) leaderboard, GLM 5.2 (LFORLA's own submission) scores 78.0.
50%UnverifiedThe complete results and verbatim responses of Team Recruitment (Oracle) can be viewed at lforla.org.
50%UnverifiedJS 3D Builder (Three.js) is a benchmark whose task is, starting from written architectural specifications, to procedurally generate 3D building code using Three.js, scored by server-side static analysis, without executing the code or using an LLM judge.
50%UnverifiedThe author of the ranking benchmark submitted his own model GLM 5.2 to his own benchmark, which constitutes a conflict of interest.
50%UnverifiedThe Function Recovery dimension uses a binary threshold of 0.75, where a token similarity score of 0.75 or above counts as passing.
50%UnverifiedThe reverse engineering benchmark task in LFORLA is to recover the C source code of a function from a stripped binary.
50%UnverifiedModel selection should compare along dimensions that match the use case rather than looking only at overall rankings; complete source code reconstruction requires looking at Completeness and Function Recovery, while token-level fidelity requires looking at Code Similarity.
50%UnverifiedToken similarity does not equal semantic correctness; a submission may have high token overlap while still being structurally incorrect.
50%UnverifiedIn this benchmark, every model receives exactly the same prompt and binary file, and models may choose between tool-assisted and tool-free modes.
50%UnverifiedThis benchmark uses deterministic token similarity comparison for scoring, with reference answers stored only on the server side and never shown to models.
50%UnverifiedThe Code Similarity metric in this benchmark consists of 60% bag-of-words cosine similarity plus 40% Longest Common Subsequence ratio.
50%UnverifiedThe 60/40 cosine and LCS mixing ratio is a subjective engineering choice made by the benchmark designers and has no theoretically optimal solution.
50%