The Complete Evolution of a Quantitative Trading System: From Multi-User Isolation to AI Trading Agent in Practice

Introduction

Building a truly production-ready quantitative trading system goes far beyond writing a few strategy scripts. From data isolation and audit logging, to a trading experience system, AI Agent automated decision-making, and unified LLM call management — each layer is a critical step in pushing the system from "toy" to "production-grade."

This article walks through a developer's multi-iteration journey of evolving their quantitative trading system, covering the design philosophy and implementation details of core modules including user isolation, strategy analysis, trading experience accumulation, AI Agent decision-making, and an LLM gateway.

Multi-User Isolation & Audit Logging: The Foundation of a Production-Grade System

Resource Ownership Model Design

In the system's early days, all users shared the same data — clearly unacceptable in a multi-user scenario. To solve this, resources were divided into two categories:

• User-exclusive resources (workspaces): Strategies, simulation accounts, Agent runtime configurations, backtest records, etc., isolated by adding a user_id field. Queries filter by user ID, ensuring no interference between users.
• System-shared resources: News, events, signals, candlestick data, and other market data shared across all users.

The rationale is straightforward — market data is objective and doesn't need to be stored redundantly, while strategies and trading records are highly personal and must be strictly isolated.

The Necessity of Audit Logging

In a multi-user environment, operations on shared resources need to be traceable. The system introduced an audit logging module that records "which user did what operation at what time," especially for sensitive operations like deleting signals or news. This is standard for enterprise-grade systems and a critical tool for quickly pinpointing responsibility when issues arise.

Meanwhile, the frontend menu was reorganized from three sub-menus into: Dashboard, Data Center, Strategy Center, Trading Center, AI Agent Analysis Center, and System Settings. Module responsibilities became much clearer, avoiding the problem of a single menu carrying too many functions.

Strategy Analysis & Multi-Account Simulation: Data-Driven Strategy Validation

Strategy-Level Data Analysis

The system previously lacked the ability to quantitatively analyze strategy performance. The new strategy analysis module provides two levels of statistics:

• Global statistics: Aggregates total returns and average win rates across all strategies, automatically identifying the top performers.
• Per-strategy details: Statistics and filtering on each strategy's return curve and individual trade records.

With this data support, users can make decisions based on numbers rather than gut feelings.

Closed-Loop Design for Multi-Account Simulation

A single simulation account can't meet the need to validate multiple strategies simultaneously. The system now supports multi-account creation, allowing users to assign independent simulation accounts to different strategies. In the run management interface, users can specify which strategy drives which account for simulated trading, with data isolation maintained across accounts.

This forms a complete strategy validation loop:

Create Strategy → Bind Simulation Account → Run Simulated Trading → Analyze Strategy Performance → Optimize Strategy

Trading Experience System: Teaching AI to Review and Grow Like a Human Trader

Design Philosophy

This is the most intellectually profound part of the entire iteration. The core insight is: the key capability of human discretionary trading comes from the continuous cycle of trading, reviewing, and accumulating experience. To achieve truly effective automated trading, this "experience accumulation" philosophy must be encoded into the system.

Experience Data Structure

Each trading experience entry contains the following key fields:

• Type: Strategy pattern, event pattern, risk, market environment, signal, etc.
• Title & Summary: Specific description of the experience
• Tags & Associations: For retrieval and correlation
• Status: Active / Invalidated
• Score (Confidence): The reliability level of the experience

Dynamic Confidence Adjustment Mechanism

This introduces a feedback mechanism similar to reinforcement learning:

• When the AI makes a correct trade based on a certain experience, that experience's confidence increases
• When the AI makes an incorrect trade based on a certain experience, that experience's confidence decreases
• When confidence drops below a certain threshold, the experience is automatically marked as invalidated
• Users can also manually mark experiences as invalidated

This design gives the system the ability to self-evolve — good experiences are increasingly trusted, while poor experiences are gradually phased out.

Trading Agent: An Automated Closed Loop from Signal to Decision

The Agent's Decision-Making Process

The trading Agent sits at a critical position in the entire trading pipeline. The complete flow is:

News generates signals → Strategy filtering → Agent evaluation → Trade execution

During the evaluation phase, the Agent considers multiple dimensions:

1. Experience library: References historically accumulated trading experiences
2. Current positions: Determines whether exposure exceeds healthy levels (risk control)
3. Signal quality: Assesses the reliability of the signal itself
4. Risk level: Directly rejects high-risk signals

Real-World Performance

From the Agent's runtime logs, we can see it rejected a large number of signals — even some with high scores were firmly rejected because positions already exceeded healthy levels. High-risk or low-quality signals were likewise filtered out directly. Only signals that passed the comprehensive evaluation were executed.

This "better to miss than to make a mistake" conservative approach is precisely the hallmark of a mature trader. Looking at account performance, the Agent's returns outperformed other randomly-traded accounts, providing initial validation of the experience system + Agent decision-making approach.

Reinforcement Learning-Style Workflow

After completing a trade, the Agent feeds back to modify relevant experiences in the experience library — forming a complete reinforcement learning loop:

Experience guides decisions → Decisions produce results → Results feed back to refine experience → Better experience guides better decisions

This closed-loop mechanism is what fundamentally distinguishes the trading Agent from a simple rule engine.

LLM Gateway: Unified Management of AI Calls

Why an LLM Gateway Is Needed

The system has numerous scenarios that call LLM APIs (news analysis, signal generation, Agent decision-making, experience summarization, etc.), and each call has a cost. The LLM gateway's core responsibilities include:

• Request rate control: Prevents runaway costs from excessive calls
• Token consumption tracking: Precisely tracks token usage per request
• Cache hits: Avoids redundant calls for identical requests, saving money
• Logging: Including request model, Request ID, cache hit status, token consumption, etc.
• Cost estimation: Tracks today's spending and projected costs

The frontend's AI Operations Center page provides an intuitive monitoring dashboard, giving users a clear view of LLM usage at a glance. For any production system that relies on large language models, this kind of unified management layer is indispensable.

Conclusion & Reflections

After multiple iterations, this quantitative trading system has evolved from a simple strategy backtesting tool into a complete platform with user isolation, audit tracking, multi-account simulation, AI experience accumulation, automated Agent decision-making, and LLM cost management.

The most noteworthy design concept is the trading experience system — it systematically encodes the human trader's cognitive process of "review, accumulate, and grow," and combined with the dynamic confidence adjustment mechanism, gives the AI Agent the ability to continuously evolve. This approach is applicable not only to quantitative trading but serves as a reference for any scenario requiring AI to make ongoing decisions.

A truly production-ready AI trading system requires far more than just model capabilities — it demands engineered system design and continuous iterative refinement. From user isolation to experience accumulation, from Agent decision-making to cost management, every module answers the same question: How do you make an AI system run reliably in a real-world environment?