Running AI Models Locally on Your Phone: Hands-On with the Open-Source Observer Agent

Running local AI models on your phone to monitor the real world sounds like a geek's toy, but a developer shared a project called Observer on Reddit's LocalLLaMA community that turns this idea into an open-source app. It supports iOS and Android, with all inference done on-device without relying on the cloud.

What can it monitor?

Developer Roy listed a set of quite everyday scenarios in the post: automatically taking a photo and sending a notification when a stranger approaches the vehicle; pushing a Telegram message with an image when a bird lands in the nest; directly calling to remind when the washing machine enters the rinse cycle; sending a notification after the dog finishes eating; calling to let you know when a particular machine at the gym becomes free.

What these scenarios have in common: none of them require complex visual understanding—they all boil down to "detecting whether a certain state has occurred." By continuously observing with the phone camera and using a lightweight model to make judgments, a large number of similar needs can be covered. The author uses "pretty much anything" to describe the scope of application. While slightly exaggerated, the direction is right—shifting monitoring logic from fixed rules to natural language descriptions genuinely lowers the barrier considerably.

Model selection: small enough to run on old phones

Observer's default configuration uses OneJev-0.8b as the decision model and qwen-0.8b as the LLM watcher. Both models are at the 0.8B parameter level, which is exactly the key to running the project on old phones. The author specifically mentions that idle old phones at home are better used as monitoring nodes than left gathering dust.

This trade-off is worth noting. Models at the 0.8B level have limited capability for complex reasoning, but they're sufficient for judgments like "is there a bird in this image." What truly gives Observer practical value is that it matches model capability boundaries with task complexity, rather than blindly pursuing larger models. For on-device AI applications, this pragmatic design approach is more instructive than parameter scale.

What does 0.8B parameters mean

0.8B (800 million parameters) is extremely small by today's language model standards. For reference, models that run smoothly on phones typically fall in the 1B–3B range, while mainstream cloud models have hundreds of billions of parameters. Parameter count directly determines memory usage and inference latency: a 0.8B model after 4-bit quantization is about 0.5GB in size, well within the available memory of an ordinary phone, with single inference latency in the hundreds of milliseconds.

The cost is limited reasoning depth. Such models struggle with multi-step logic, long-context understanding, or fine-grained visual descriptions, but perform reliably on "binary classification" judgments (present/absent, yes/no, whether a state has changed). This is exactly why Observer's choice of it as the decision model makes sense—the task itself is narrowed down enough that model scale is no longer the bottleneck.

Division of labor between OneJev and qwen-0.8b

The two models mentioned serve different roles: the decision model is responsible for judging whether the current frame or state should "trigger," while the LLM watcher is closer to a scheduling/understanding layer that converts the user's natural language description into executable observation conditions. This "judgment model + language model" combination is fairly common in on-device agent projects. The advantage is that heavy judgment and heavy understanding can be separated, each handled by the smallest possible model, rather than forcing a single large model to do everything.

Division of labor between device and cloud

The project's selling point is "100% local," meaning video streams and the inference process never leave the device. This is a clear advantage in privacy-sensitive scenarios—for needs like home monitoring and in-car monitoring, users often don't want footage uploaded to third-party servers.

However, based on the usage described by the author, the notification step still relies on external services like Telegram. In other words, the device handles perception and judgment locally, while the cloud only handles message delivery. This hybrid architecture strikes a balance between privacy and convenience, and also explains why the author offers both "send image to Telegram" and "direct phone call" notification methods: the former suits scenarios with network access that need visual evidence, while the latter pursues stronger reach.

On-device inference and the privacy boundary

"On-device" means the model runs directly on the user's device, with the inference process not passing through remote servers. It differs subtly from "local deployment": the latter often refers to running models on your own servers, where data still leaves the device. The real value of on-device inference is that data never leaves the device—raw video streams are never uploaded, and third-party services at most receive a text notification or a screenshot confirmed by the user.

This determines where the privacy boundary lies: what Observer guarantees is that "perception data doesn't leave the device," not that "no data ever goes online." Understanding this is important, because the notification step still sends results (sometimes with images) to Telegram. Users with extremely high privacy requirements need to further disable image attachments or switch to local notifications to make the entire chain fully closed-loop.

Open source and dual-platform coverage

The project is released as open source, with code hosted on GitHub, and is listed on both the App Store and Google Play. For an individual developer's project, simultaneous dual-platform coverage is uncommon, and it also lowers the barrier for ordinary users to try it—no need to compile it yourself; just install and experience it.

The author candidly admits in the post that the demo video quality is mediocre, because they wanted to get the demo out to the community as quickly as possible. This honesty actually makes the project seem more credible: it's more like a tool in active iteration than an over-polished product. The author also proactively solicits use cases from the community, hoping to improve the agent's capabilities—consistent with the common collaborative path of open-source projects.

Who is this kind of tool for

Observer in its current form is better suited to technical users willing to tinker: they need to understand what the model can and cannot do, and accept the potential instability of an early-stage project. But the direction it demonstrates—replacing dedicated monitoring hardware with a phone plus local small models—is appealing to a broader user base.

If you have an idle phone on hand and happen to have a need to "watch for something to happen," this project is worth a try. More importantly, it provides a sample of how on-device AI can be deployed: not pursuing general intelligence, but placing a small model into a specific scenario to solve a specific problem.

Typical trade-offs of on-device AI applications

Cramming large models into phones is not the mainstream way on-device AI is deployed. The more common path is "small model + narrow task + closed-loop scenario": the model doesn't need to be general, it just needs to output stably on a specific input distribution. Observer is exactly a sample of this approach—it abandons general visual understanding and compresses the problem into state detection, giving small models usability even on old hardware.

Common constraints for such applications include: power consumption and heat (continuously calling the camera and running inference drains battery significantly), background running permissions (iOS has strict restrictions on continuous camera use), and false-positive rate control. These determine that on-device monitoring tools are currently better suited to "short-duration, on-demand, attended" scenarios rather than 7×24 unattended security replacements. Understanding these constraints is necessary to form reasonable expectations for projects like Observer.

One-line Summary

Open-source app Observer uses local small models on phones to enable natural language-described state monitoring, supporting both platforms.

Paragraph Summary

Observer is an open-source, 100% on-device mobile AI monitoring app released by a developer on Reddit's LocalLLaMA community, supporting both iOS and Android. Users describe the state they want to monitor in natural language; the app continuously observes through the phone camera and uses on-device models to judge whether to trigger. Typical scenarios include strangers approaching a vehicle, a bird landing in a nest, a washing machine entering the rinse cycle, a pet finishing its meal, and a gym machine becoming free. The default configuration uses OneJev-0.8b as the decision model and qwen-0.8b as the LLM watcher; the 0.8B parameter scale enables it to run on old phones. Perception and inference stay entirely on-device, while notifications are delivered via Telegram or phone calls, forming a compromise between privacy and convenience. The project is open-sourced on GitHub and listed on app stores, and the author is soliciting use cases to iterate on the agent's capabilities.