Fiorillo v0.5
An open-source medical AI model with four billion parameters, fine-tuned from Qwen3-4B-Base. It specializes in interpreting evidence from medical randomized controlled trials (RCTs) and is capable of outputting calibrated probabilities for each candidate answer.
Timeline (last 90 days)
Fiorillo v0.5 is an open-source model with only 4 billion parameters, specifically designed for the task of interpreting evidence from medical randomized controlled trials (RCTs).
Fiorillo v0.5 outputs probabilities for each candidate answer, addressing a three-way classification question on the Evidence Inference 2.0 benchmark: whether an intervention significantly increases, significantly decreases, or does not significantly change a given outcome measure relative to a control group.
The primary expert module of Fiorillo v0.5 reads randomized trial articles truncated to 6,144 tokens.
Fiorillo v0.5 uses Qwen3-4B-Base as its backbone, with a low-rank adapter (LoRA) and a dedicated decision head stacked on top, fine-tuned exclusively for the EI task.
The EI training set contains 2,657 articles, of which the researchers used only the 1,431 whose licenses explicitly permit reuse.
Training with the same recipe using only license-clean articles resulted in an accuracy drop of 0.0123 (95% interval 0.0034 to 0.0207).
The researchers pre-registered four release criteria on the Open Science Framework (OSF), and the model would only be released if all four were passed.
On a test set of 333 articles and 1,218 prompts, Fiorillo v0.5 achieved an Expected Calibration Error (ECE) of 0.0168, well below the preset upper limit of 0.05.
Fiorillo v0.5's log loss is 0.8603 lower than the prior baseline and 0.1829 lower than the Gemma model reading the same input.
Fiorillo v0.5 achieves a macro-F1 of 0.9248, compared to the baseline's 0.8668.
5 more timeline events
All Facts (15)
For high-reliability scenarios such as evidence-based medicine, a small model that honestly expresses uncertainty may be more practical than an overconfident large model—this represents a tradeoff.
50%UnverifiedFiorillo v0.5 outputs probabilities for each candidate answer, addressing a three-way classification question on the Evidence Inference 2.0 benchmark: whether an intervention significantly increases, significantly decreases, or does not significantly change a given outcome measure relative to a control group.
50%UnverifiedThe primary expert module of Fiorillo v0.5 reads randomized trial articles truncated to 6,144 tokens.
50%UnverifiedFiorillo v0.5 uses Qwen3-4B-Base as its backbone, with a low-rank adapter (LoRA) and a dedicated decision head stacked on top, fine-tuned exclusively for the EI task.
50%UnverifiedThe EI training set contains 2,657 articles, of which the researchers used only the 1,431 whose licenses explicitly permit reuse.
50%UnverifiedTraining with the same recipe using only license-clean articles resulted in an accuracy drop of 0.0123 (95% interval 0.0034 to 0.0207).
50%UnverifiedThe researchers pre-registered four release criteria on the Open Science Framework (OSF), and the model would only be released if all four were passed.
50%UnverifiedOn a test set of 333 articles and 1,218 prompts, Fiorillo v0.5 achieved an Expected Calibration Error (ECE) of 0.0168, well below the preset upper limit of 0.05.
50%UnverifiedFiorillo v0.5's log loss is 0.8603 lower than the prior baseline and 0.1829 lower than the Gemma model reading the same input.
50%UnverifiedFiorillo v0.5 achieves a macro-F1 of 0.9248, compared to the baseline's 0.8668.
50%UnverifiedWhen no article is provided at all, Fiorillo v0.5's macro-F1 drops sharply to 0.4384, indicating that the model relies on the body text.
50%UnverifiedProviding only the title raises Fiorillo v0.5's F1 by 0.0939, suggesting there may be some degree of answer leakage from the title.
50%UnverifiedWhen the intervention group and control group are swapped, Fiorillo v0.5's directional answers are reversed 0.6652 of the time, indicating that the model understands causal directionality.
50%UnverifiedFiorillo v0.5 is an open-source model with only 4 billion parameters, specifically designed for the task of interpreting evidence from medical randomized controlled trials (RCTs).
50%UnverifiedFiorillo v0.5 is released under the Apache License 2.0 with DOI 10.57967/hf/10722.
50%