AutoKon
An AI platform for the construction industry that manages contractor payment processes through AI visual inspection of on-site photographs.
Timeline (last 90 days)
In AutoKon, when agreement between automated and human review drops, the cause is almost always the rubric letting two reasonable reviewers score the same photo differently, not the model getting worse.
In AutoKon's architecture, WhatsApp messages are received and initially parsed by Botpress's conversational flow, after which n8n triggers the subsequent evaluation gate calls, status updates and reviewer notifications.
AutoKon's AI doesn't approve anything; it clears obviously complete, obviously well-supported evidence from the reviewer queue and flags what needs human handling.
Claude Opus Vision can describe a construction site (such as walls already plastered, roof trusses in place, floor leveling completed), but occasionally gives equally confident descriptions of completely different photos.
AutoKon places an evaluation stage, in which a second LLM acts as reviewer, before AI-generated descriptions enter payment decisions, scoring the first model's output against the image and the claim, and only items that pass can proceed.
In AutoKon, the evaluation stage sits in the payment path; if the evaluation stage goes down, payments wait.
AutoKon's evaluation reviewer prompt is extremely narrow, asking only 'is every claim here supported by this image' rather than assessing whether the description is fluent.
In high-stakes scenarios such as payments, LLM-as-judge review prompts should be narrowed as much as possible to verifiable specific claims, and the reviewer's reliability should be calibrated with human labels.
The goal of the AutoKon pipeline is Cohen's κ ≥ 0.80 between automated review and human labels.
AutoKon runs a five-role photo-evidence approval workflow, with on-site submissions delivered via WhatsApp through n8n and Botpress flows.
2 more timeline events
All Facts (12)
In AutoKon, when agreement between automated and human review drops, the cause is almost always the rubric letting two reasonable reviewers score the same photo differently, not the model getting worse.
50%UnverifiedIn AutoKon's architecture, WhatsApp messages are received and initially parsed by Botpress's conversational flow, after which n8n triggers the subsequent evaluation gate calls, status updates and reviewer notifications.
50%UnverifiedAutoKon's AI doesn't approve anything; it clears obviously complete, obviously well-supported evidence from the reviewer queue and flags what needs human handling.
50%UnverifiedClaude Opus Vision can describe a construction site (such as walls already plastered, roof trusses in place, floor leveling completed), but occasionally gives equally confident descriptions of completely different photos.
50%UnverifiedAutoKon places an evaluation stage, in which a second LLM acts as reviewer, before AI-generated descriptions enter payment decisions, scoring the first model's output against the image and the claim, and only items that pass can proceed.
50%UnverifiedIn AutoKon, the evaluation stage sits in the payment path; if the evaluation stage goes down, payments wait.
50%UnverifiedAutoKon's evaluation reviewer prompt is extremely narrow, asking only 'is every claim here supported by this image' rather than assessing whether the description is fluent.
50%UnverifiedIn high-stakes scenarios such as payments, LLM-as-judge review prompts should be narrowed as much as possible to verifiable specific claims, and the reviewer's reliability should be calibrated with human labels.
50%UnverifiedThe goal of the AutoKon pipeline is Cohen's κ ≥ 0.80 between automated review and human labels.
50%UnverifiedAutoKon runs a five-role photo-evidence approval workflow, with on-site submissions delivered via WhatsApp through n8n and Botpress flows.
50%UnverifiedThe author of this article, Faishal Wahiduddin, founding CTO and sole engineer of AutoKon, explicitly states that for privacy and commercial reasons he will not share the dataset size, the original prompt text, the number of labels, or the per-round scores.
50%UnverifiedOn the construction platform AutoKon, contractor payments are released only after on-site photos pass an AI check, which uses Claude Opus Vision together with Pinecone retrieval context to describe the photo content, after which an LLM-as-judge pipeline in Arize AX scores the descriptions, and only once judge agreement reaches κ ≥ 0.80 can it be used for payment decisions.
50%