Phoenix
An LLM tracing and evaluation tool developed by Arize, featuring native OpenTelemetry (OTel) tracing capabilities. It is licensed under the Elastic License 2.0, meaning its source code is visible but it is not truly open source in the conventional sense.
Core Facts
Phoenix is open-sourced by Arize AI and focuses on experiment tracking and evaluation for LLM applications, complementing Arize's commercial ML monitoring platform.
Testing tool-calling accuracy requires attention to three levels: tool selection accuracy, parameter extraction accuracy, and tool chain orchestration rationality, of which parameter extraction is often the most error-prone.
Timeline (last 90 days)
Open-source tools such as LangSmith and Phoenix provide structured tracing capabilities, whose core value is organizing the execution chain into a queryable, comparable structure.
Common tool-calling evaluation frameworks break tool failures into two major categories: "wrong tool selected" and "correct tool selected but wrong parameters," with parameter errors further divided into structure and value validation.
The Phoenix framework provides a separate tool selection evaluator.
Clean-room development is an engineering method that isolates "reading the original" from "writing the copy"; Phoenix's use of it in 1984 to rebuild the IBM PC BIOS is the landmark case-law background.
Phoenix uses the Elastic-2.0 license, which is a source-available rather than OSI-certified open-source license.
The author recommends Langfuse, Opik, MLflow, Phoenix, and AcruxCore as tools with native LangChain integration.
Evaluations with Phoenix 101, hosted by Laurie Voss, will start from evaluation fundamentals in Turing Hall.
2D simulators (such as FARSITE and Phoenix) embed validated physical equations, and their outputs naturally carry accurate combustion dynamics labels.
During the real task run, the execution order was: Phoenix starts first, then Hermes begins working, and Relay exports trace data to Phoenix in real time.
The trace structure displayed in Phoenix is essentially a tree of Spans, where each Span represents a discrete operation and records the start time, end time, input parameters, and return values.
12 more timeline events
All Facts (20)
Phoenix is open-sourced by Arize AI and focuses on experiment tracking and evaluation for LLM applications, complementing Arize's commercial ML monitoring platform.
65%VerifiedTesting tool-calling accuracy requires attention to three levels: tool selection accuracy, parameter extraction accuracy, and tool chain orchestration rationality, of which parameter extraction is often the most error-prone.
65%UnverifiedLangSmith、Phoenix、Helicone等专门的LLM监控平台已经出现，反映业界对AI可观测性的重视
85%UnverifiedThe AMD R7-H255 is manufactured using TSMC's 4nm process.
60%UnverifiedOpen-source tools such as LangSmith and Phoenix provide structured tracing capabilities, whose core value is organizing the execution chain into a queryable, comparable structure.
50%UnverifiedThe Phoenix framework provides a separate tool selection evaluator.
50%UnverifiedCommon tool-calling evaluation frameworks break tool failures into two major categories: "wrong tool selected" and "correct tool selected but wrong parameters," with parameter errors further divided into structure and value validation.
50%UnverifiedClean-room development is an engineering method that isolates "reading the original" from "writing the copy"; Phoenix's use of it in 1984 to rebuild the IBM PC BIOS is the landmark case-law background.
50%UnverifiedThe author recommends Langfuse, Opik, MLflow, Phoenix, and AcruxCore as tools with native LangChain integration.
50%UnverifiedPhoenix uses the Elastic-2.0 license, which is a source-available rather than OSI-certified open-source license.
50%Unverified2D simulators (such as FARSITE and Phoenix) embed validated physical equations, and their outputs naturally carry accurate combustion dynamics labels.
50%UnverifiedTraces in Phoenix are presented turn by turn, corresponding to each round of the agent's think-act loop (in ReAct mode, one reasoning step plus one tool call).
50%UnverifiedThe trace structure displayed in Phoenix is essentially a tree of Spans, where each Span represents a discrete operation and records the start time, end time, input parameters, and return values.
50%UnverifiedDuring the real task run, the execution order was: Phoenix starts first, then Hermes begins working, and Relay exports trace data to Phoenix in real time.
50%UnverifiedPatrick Moorhead is an NVIDIA technical marketing engineer who demonstrated using NeMo Relay to perform chain tracing on the Hermes agent and visualize it in Phoenix.
50%UnverifiedPlug is the foundational abstraction for Phoenix's request processing pipeline, essentially being a function or module that receives a %Plug.Conn{} struct and returns a struct of the same type.
50%Unverifiedconn.assigns is the standard mechanism for passing data across Plugs and Controllers within the same request lifecycle.
50%UnverifiedSupabase's Realtime service is built using the Elixir language and the Phoenix framework, listening to PostgreSQL WAL changes and pushing them to clients via WebSocket.
50%UnverifiedLangSmith、Langfuse、Phoenix（Arize AI）等可观测性工具提供从请求到响应的全链路追踪能力
50%Unverified业界常用RAG可观测性工具包括LangSmith、Langfuse、Phoenix（Arize AI），提供全链路追踪能力
50%