Foyer
A self-hosted personal email inbox agent system focused on routing actions for each message, such as type classification, urgency judgment, and reply decisions, accompanied by a measurement protocol centered on human-labeled review.
Timeline (last 90 days)
The production keyword rule achieves only 28% accuracy on the message type field, while the majority-class baseline is 58%.
Trained on about 200 labels, the TF-IDF+logistic regression baseline achieves the highest accuracy on all four fields—type, urgency, whether a reply is needed, and whether it is noise—at 0.79, 0.74, 0.83, and 0.90 respectively.
At the default threshold, the most accurate system ignored 18% of emails requiring action, while the best LLM's ignore rate was only 4%.
Foyer is a self-hosted personal inbox agent accompanied by a measurement protocol (stratified sampling, frozen snapshots, thread-grouped splits, pre-registered hypotheses, and human labels introduced within the agent review loop).
The study compared nine systems on 739 real Italian/English emails (231 unbiased samples and 500 adversarial samples), including keyword rules, TF-IDF+logistic regression, local typed decision models (Tev1, Nimble), a hosted decision model (Nimble v4-max), and hosted generative LLMs (Qwen2.5-7B, Gemma 4 26B/31B).
All Facts (5)
The production keyword rule achieves only 28% accuracy on the message type field, while the majority-class baseline is 58%.
50%UnverifiedTrained on about 200 labels, the TF-IDF+logistic regression baseline achieves the highest accuracy on all four fields—type, urgency, whether a reply is needed, and whether it is noise—at 0.79, 0.74, 0.83, and 0.90 respectively.
50%UnverifiedAt the default threshold, the most accurate system ignored 18% of emails requiring action, while the best LLM's ignore rate was only 4%.
50%UnverifiedFoyer is a self-hosted personal inbox agent accompanied by a measurement protocol (stratified sampling, frozen snapshots, thread-grouped splits, pre-registered hypotheses, and human labels introduced within the agent review loop).
50%UnverifiedThe study compared nine systems on 739 real Italian/English emails (231 unbiased samples and 500 adversarial samples), including keyword rules, TF-IDF+logistic regression, local typed decision models (Tev1, Nimble), a hosted decision model (Nimble v4-max), and hosted generative LLMs (Qwen2.5-7B, Gemma 4 26B/31B).
50%