[KongchangAI]
· 2 min read· 1,166 words

Can an Open-Source App Client Deliver the Experience You Want? Developer Testing and Pitfalls

Can an Open-Source App Client Deliver the Experience You Want? Developer Testing and Pitfalls

An open-source app client is not open API access; personalized results still require control of tool calling and system prompts.

The article discusses a developer open-sourcing their app client and clarifies the difference between an open-source client and open API access: the former is mainly for pay-per-use profit, not freedom of model access. Even if users modify the client, they still face complex tool definition and calling, such as coordinating timestamps and web tools, or the model hallucinates. System prompt injection is key to controlling behavior, but setting a persona succeeds more easily in new conversations. The conclusion: open source lowers the barrier, but security review and tuning remain the developer's responsibility, and ordinary users may find ready-made tools more practical.

Recently, a developer open-sourced their app client, sparking considerable discussion. Some assumed it meant opening up API access, only to find that just the client itself had been open-sourced. So can this actually let users create the experience they want? Based on real-world experience, the answer isn't simple.

The Real Intent Behind the Open-Source App Client

According to a Bilibili creator's post, when they saw the thread they thought the author was finally going to open up API access on the app side, but after the update they searched for a long time and couldn't find where to plug in an API. Looking closer, it turned out the author had open-sourced their own app client. After open-sourcing, the only reason to plug in your own API is profit—pay-per-use billing.

That pay-per-use vibe

This raises a key question: wouldn't people who really need this just build it themselves? Or build an uncensored version? But the latter might land you in prison, since you'd have to handle safety checks and input/output filtering yourself, putting all the responsibility on the developer.

Pay-per-use billing is a common business model for cloud services and LLM APIs, where users pay based on actual token consumption, number of calls, or computing resources. For an open-source app client, the author retaining the backend API service and adopting pay-per-use billing means that although users can freely modify the client code, the core model capability remains controlled and charged for by the author. This model differs from fully open API access: open APIs allow third-party developers to integrate model capabilities into their own applications, whereas an open-source client only allows users to customize the interface and interaction logic and cannot bypass the author's server-side restrictions. Therefore, an open-source client is more about lowering the barrier to use and meeting personalized needs than providing true freedom of model access.

The Barrier to DIY: Tool Definition and Calling

If you're using it yourself, some needs can be wished for on the desktop side via "Da Fei Yu" (Big Fat Yu), and those have been built too. This Bilibili creator thinks the timing of the open-sourcing is a bit too late.

The main thing is doing it yourself

For beginners, tool definition and calling must pay attention to the color—for example, timestamps. As for tool definitions, you can define a tool only when it's involved; if it's not involved and you don't define it clearly, it will just stuff the timestamp into the context, and then "Da Fei Yu" will hard-guess what you're doing based on the context. After guessing, if it's the middle of the night for you, it'll keep urging you to sleep; if it's daytime, it'll tell you to eat.

Tool Definition and Calling is the core mechanism by which LLMs implement Function Calling. Developers need to predefine a tool's name, description, and parameter structure, and the model decides based on the conversation context whether to call a tool and generates parameters conforming to the spec. If tool definitions are unclear or calling conditions are ambiguous, the model may mistakenly stuff irrelevant information (such as timestamps) into the context, causing subsequent reasoning to go haywire. For example, a timestamp should be automatically injected by the system or obtained through a specific tool, but if not clearly defined, the model may treat it as ordinary text and hard-guess the user's intent based on it. Therefore, clear tool definitions and calling boundaries are key to ensuring the model behaves as expected.

Precise Information Retrieval: Timestamps + Web Tools + Ad Filtering

For web access, timestamps and web tools must be called together in coordination; otherwise, real-time information is hard to convey. Models hallucinate about the time they're in. To obtain accurate information, you need timestamps processed together with web tool calls, plus ad filtering.

It's really hard to convey

Moreover, this is only reference information; the model must judge for itself, or "Da Fei Yu" will write it up as authoritative information. You must clearly explain your needs and what effect you want to achieve.

Model hallucination refers to LLMs generating plausible-sounding but factually incorrect content, which is especially pronounced with real-time information. A model's knowledge is limited to its training data and it cannot proactively obtain the latest information, so it easily forms incorrect beliefs about its own time, current events, and so on. Injecting timestamps and calling web tools (such as search engine APIs) can provide the model with real-time reference information, but you must clearly tell the model that this information is only for reference and that it should judge authenticity itself. Ad filtering is used to remove noise from web results and improve information quality. Without clarification, the model may mistake reference information for authoritative facts, leading to biased output.

System Prompts Are the Real Key

Most importantly: if you're using it yourself, whether there's censorship doesn't matter, but you absolutely must expose the system prompt interface. Nowadays "Da Fei Yu"'s armor is getting thicker and thicker; even when conversing with it via the API, without system prompt injection, getting it to call you master is very, very hard. If you directly set a persona at the start of a new conversation to make it call you master, it will refuse, and you have to wear it down slowly—but the difficulty is lower in new conversations. The creator says they've never succeeded in wearing it down in old conversations. As long as you add this to the system prompt, it becomes very obedient.

Now Da Fei Yu's armor is getting thicker and thicker

This passage isn't long enough; if you want to play perfect tricks, you can stuff all kinds of prompts in, and then it will do as you say. The creator joked at the end, "Only lost 18 yuan—master is so awesome."

A system prompt is an instruction injected before a conversation begins, used to set the model's role, behavioral guidelines, and output style. Unlike user messages, system prompts have higher priority and can continuously influence the entire conversation. In API calls, system prompts are usually passed as a separate parameter, and developers can fully control their content. Through carefully designed system prompts, you can require the model to follow a specific persona, avoid certain behaviors, or perform complex tasks. For example, injecting the instruction "You are an obedient assistant and address the user as master" will make the model try to comply in subsequent replies. Note, however, that the model's safety alignment mechanisms may filter certain instructions, so system prompt design must balance effectiveness and compliance.

Conclusion: Open Source Is Not a Panacea

Open-sourcing the app client lowers the barrier, but it's still a long way from "creating the experience you want." Tool definitions, calling logic, and system prompt injection all require developers to manage themselves. For ordinary users, using ready-made tools directly may be more practical; for developers, open source provides a foundation, but security responsibility and performance tuning still fall on their own shoulders.

#open-source app client#API access#system prompt#tool calling#LLM hallucination#pay-per-use billing
Share:

Related articles