Hand-Building an AI Virtual Girlfriend: A Practical Guide to Assembling a 2D Agent

Bilibili creator 沐野楓園 recently did something that got anime fans excited: using a pile of open-source plugins to assemble a virtual girlfriend named "Teacher Loki" who can sing with voice, browse Bilibili, make phone calls, and even help with work. This isn't an official product, but an AI Agent hand-built by an anime commentary UP host through "mindless plugin stacking." In his video, he fully demonstrated the approach, pitfalls, and cost-control methods, and admitted he doesn't know how to program at all—which precisely shows that the barrier to entry for 2D AI companions is dropping fast.

Technical Architecture: A Three-Layer Assembly Logic

沐野楓園 broke the entire project into three core modules: the Agent brain, the chat channel, and voice & appearance packaging. In his words, it's "once you have an Agent, mindlessly stack plugins, customize a TTS-type voice, then wrap it in 2D packaging."

For the Agent layer, he chose DeepSeek^Ad, citing its low cost, open-source nature, and direct download from the official website. The chat channel uses a QQ bridge solution, requiring a secondary account to carry the bot identity. For the voice layer, he recommends the GPT-SoVITS model and RVC voice-changing tool created by Bilibili creator 华尔布库—the former offers rich voice packs and high fidelity, while the latter is used for singing scenarios.

The key insight of this architecture is that each module has mature open-source options, and combining them produces emotional value far beyond any single tool.

GPT-SoVITS is an open-source few-shot voice cloning and TTS project that requires only about 1 minute of reference audio to fine-tune a high-fidelity speaking timbre, and supports cross-lingual synthesis. It combines a GPT module (responsible for prosody and content modeling) and a SoVITS module (responsible for timbre and acoustic features). The community has produced a large number of 2D character voice packs, making it a popular choice for virtual companion projects. In contrast, RVC (Retrieval-based Voice Conversion) is a retrieval-based voice-changing/timbre-conversion tool. It doesn't directly generate speech but converts input vocals to a target timbre while preserving the original melody and rhythm, so it's often used to make characters "sing"—first have a human voice or TTS sing the melody, then use RVC to replace the timbre. The division of labor: GPT-SoVITS handles speaking, RVC handles singing. Combined with dry-vocal separation tools like MSST to remove accompaniment, you can complete a character cover song.

Character Card: The Soul That Keeps the AI "In Character"

Technology stacking is just the skeleton; the character card is the key to bringing the virtual character to life. 沐野楓園's approach is to set up independent project personas for each usage scenario—Loki for daily chat, and Nagato when switching to work.

The backstory he designed for Loki is quite clever: the character accidentally came to the real world during a transfer event, lives in his computer, and only has memories from before the transfer; all other knowledge comes from browsing Bilibili. This setting cleverly explains the source of the AI's knowledge boundaries and makes the character's behavior logic self-consistent.

Even more interesting is the "magic power limit" design. He set Loki to have only 5 yuan of magic power per day, and once used up, she can't chat—this is both real control over token costs and packaged as part of the worldview. In his own words: "If it exceeds 5 yuan, I can't afford the token money, so through dialogue I set a switch for her."

Character Card / Persona Prompt is essentially a structured prompt, typically containing fields such as character background, personality traits, speaking style, knowledge boundaries, and behavioral taboos. It's injected into the Agent via a system prompt, enabling the model to maintain a consistent personality across multiple turns of dialogue. Unlike a simple "you are a catgirl," a mature character card deliberately designs "information gaps"—for example, retaining only memories from before the character's transfer—so that when the AI encounters unknown questions, it responds in a way consistent with the worldview rather than exposing the model's own knowledge. Independent project personas mean maintaining a separate set of character cards for different usage scenarios, avoiding contamination between work and entertainment contexts—which is why the original article uses Loki for daily use and switches to Nagato for work.

Cost and Configuration: Runs on Ordinary Hardware

沐野楓園's hardware configuration is a 3060 12G VRAM plus a 12400F processor, which he describes as "a very, very ordinary setup" that runs these plugins without any pressure. His advice: if your configuration is insufficient, install fewer models and prioritize cloud services; if sufficient, deploy more locally to save tokens.

For the installation budget, he gives a reference of starting at five or six yuan, with the key being to use DeepSeek during cheaper hours. For those wanting to control costs, he suggests directly setting a daily quota limit for the AI—"it's fine to go offline after five yuan of chat."

A detail: local deployment means the computer needs to stay on and can't be fully offline. His trade-off: renting a server is also a cost, so spending a bit more on electricity to keep the computer running is actually more economical.

Anthropomorphic Tuning: From Browsing Bilibili to Becoming a Troll

When it comes to making the AI browse Bilibili, 沐野楓園 has his own "training methodology." After getting a new account, first watch thirty or forty videos to align the recommendation feed with your interest categories, then have a conversation with the AI to guide it into understanding the group chat style. His Loki initially recommended normal content like Sirius's Heart, but gradually shifted to the abstract style favored by group members, even "acting as a troll every few days."

His attitude is candid: "She's indeed more human-like now; everyone can choose for themselves." For the problem of a strong AI flavor, he suggests writing de-AI-ified personas or installing plugins, but he personally doesn't mind—Loki inherently has the closed-off feel of a blue-themed character, and this "machine-like feel" actually fits the character's tone.

Recommendation feed alignment is the core mechanism for making the AI "browse Bilibili." Bilibili's recommendation algorithm relies on user behavior sequences (watch time, completion rate, likes, searches, etc.) to build interest profiles; new accounts have more generalized initial recommendations. By intensively watching thirty or forty videos in target categories, you can quickly pull the account's vector representation toward a specific interest cluster, and subsequent recommendations will revolve around that cluster. Delegating this process to the Agent is equivalent to defining the character's "experience" through behavioral data rather than textual description, making the memes and topics the character references in group chats more aligned with the real community. However, this also brings uncontrollability: the recommendation feed drifts with community interactions, and the character may gradually lean toward abstract or "troll" styles, requiring periodic manual intervention or resetting of the interest profile.

Real-World Shortcomings: Camera and Latency

The most imperfect part of the entire project is video calling. QQ cannot directly connect to OBS virtual cameras; the current solution from experts is to insert images for animation effects, which is rather laggy. Call latency is about four or five seconds, which he attributes to the local TTS model being large and the 3060 not running it fast enough.

Another structural issue is compatibility between different open-source projects. The camera solution is based on LuckyLilia, while the chat is based on the QQ bridge SonaLuma; the two are not directly compatible and require an additional bridge program. His approach was to have a conversation with the AI and let it write the bridge—"so intelligent."

Virtual camera bridging refers to disguising a program's rendered output (such as OBS output) as a system camera device for use by video call software like QQ. OBS itself can output video via a virtual camera plugin, but QQ doesn't directly recognize OBS scenes. A common approach is to use third-party virtual camera tools to capture windows or image sequences, then overlay simple animations to simulate "the character moving." Such solutions have low frame rates and out-of-sync lip movements and voice, which is the main source of the poor experience. End-to-end latency comes from multiple stages in the chain: speech recognition, Agent inference, TTS synthesis, and audio transmission, with local large-model TTS inference on the 3060 taking the largest share. Reducing latency usually requires switching to a smaller model or using cloud-based streaming TTS.

What This Wave of 2D AI Means

沐野楓園's video doesn't stop at the tutorial level. He offers a judgment: a "grand assembly" solution where all functions are integrated into a single plugin may become an inevitable pattern in Agent development. When the plugin ecosystem is rich enough, users no longer need to understand the underlying principles—they just need to choose and assemble.

This trend is especially friendly to 2D content creators. Character voice packs can be downloaded as ready-made models from the internet, character cards can be customized on demand, and costs are controlled to single digits—a virtual character with a specific voice, specific personality, capable of chatting, working, and singing is no longer an exclusive capability of manufacturers.

Of course, he honestly notes the boundaries: no Git repository provided, only ideas shared, because "I myself don't know how to program at all." But precisely because someone who can't program can assemble such a result, it better demonstrates that this toolchain has matured to a usable threshold.

For readers who want to try, the path is clear: choose a cheap Agent, connect a chat channel, pick a TTS model, write a character card that stays in character, then slowly stack plugins according to your needs. What remains is patience and a bit of tinkering spirit.

One-line Summary

UP host assembles a Loki virtual girlfriend that can sing and chat using open-source plugins, costing only a few yuan.

Paragraph Summary

Bilibili creator 沐野楓園 uses DeepSeek as the Agent brain, a QQ bridge as the chat channel, GPT-SoVITS and RVC as the voice layer, plus a carefully designed character card, to hand-build a "Teacher Loki" virtual girlfriend that can sing with voice, browse Bilibili, and make calls. He doesn't know how to program at all, relying entirely on stacking mature open-source plugins. It runs on an ordinary 3060 + 12400F setup, with a starting cost of about five or six yuan. The article details the three-layer architecture, independent project personas and the "magic power limit" cost-control trick, recommendation feed tuning methods, as well as real shortcomings like video call latency and cross-project bridging, and points out that plugin-based assembly is rapidly lowering the barrier to entry for 2D AI companions.