Together Launches Post-Training Service: Customize Open-Source Models with SFT and RL

Title

Together Launches Post-Training Service: Customize Open-Source Models with SFT and RL

Meta Description

Together AI launches its Post-Training service, enabling SFT and RL to customize open-source models into proprietary models. Customers define data, rewards, environments, and training loops, while the platform provides the underlying training infrastructure.

Article Content

The capability boundaries of open-source models are being broken by post-training. Together AI recently launched its Together Post-Training service, allowing developers to transform open-source models into fully owned custom models through supervised fine-tuning (SFT) or reinforcement learning (RL). This move turns post-training from internal engineering work done by a handful of teams into a managed infrastructure capability.

Post-Training as a Service: Together Completes the Model Customization Puzzle

Together's official tweet states clearly: "Turn open models into custom models you own, with SFT or RL." This sentence highlights two core aspects of the service: on the training method side, it supports supervised fine-tuning and reinforcement learning; on the ownership side, it emphasizes that custom models belong to the customer.

For most teams, fine-tuning an open-source model is not difficult. The hard part is running the training loop, reward modeling, environment setup, and compute scheduling reliably. Together defines this part as "hard infrastructure," with the platform responsible for the underlying operation and tuned for quality and speed. Customers only need to define the data, rewards, environments, and the training loop itself.

This division of labor effectively splits post-training into two layers: the upper layer is business-related data and objective definition, and the lower layer is engineering and compute. Together chooses to handle only the lower layer, leaving control of the upper layer to customers, which distinguishes it from fully managed automatic fine-tuning services.

SFT and RL: The Division of Labor Between Two Post-Training Paths

SFT and RL are the two main lines of post-training today, and their applicable scenarios differ. SFT uses labeled input-output pairs for supervised learning, suitable for enabling models to master specific formats, domain knowledge, or conversational styles. The training process is relatively stable, and data quality directly determines the outcome. RL optimizes model behavior through reward signals, making it better suited for tasks that require exploration and trial and error, such as improving reasoning ability, agent decision-making, or preference alignment.

Together supports both, meaning customers can complete the process from supervised fine-tuning to reinforcement learning on the same platform. The three terms mentioned in the tweet—"rewards," "environments," and "training loop"—are exactly the key components of RL post-training: the reward function defines the optimization direction, the environment provides interaction feedback, and the training loop determines how data is consumed. Leaving these three to customer definition effectively opens up the entire strategy space of post-training.

Supervised fine-tuning (SFT) continues gradient descent on top of a pretrained model using paired input-output samples, enabling the model to imitate the behavior patterns in the labeled data. It is essentially still maximum likelihood training, so it is highly sensitive to data distribution: formats or knowledge absent from the data are difficult for SFT to learn from scratch. A common practice is to construct thousands to tens of thousands of high-quality samples and use a relatively low learning rate to avoid catastrophic forgetting.

Reinforcement learning (RL) does not directly tell the model the correct answer. Instead, it defines a reward function, lets the model sample multiple trajectories in the environment, and adjusts the policy according to the level of return. RLHF, DPO, and GRPO all belong to this family. The advantage of RL is that it can optimize objectives that are hard to express through labeling (such as the quality of reasoning steps or whether tool calls are correct), at the cost of difficult reward design, high training variance, and high compute consumption. In engineering practice, SFT is usually used for cold start first, followed by RL fine-tuning, which is also the practical reason Together provides both paths.

What Ownership of a Custom Model Means

"models you own" is a phrase in the tweet that is easy to overlook but worth examining closely. Under the API calling model, model capabilities are controlled by the vendor, and customers can only influence them through prompts and a small number of parameters. Post-training services change this relationship: the weights trained by customers with their own data belong to the customers.

This is especially important for teams with data moats. Domain-specific labeled data, internal business rules, and interaction records in a particular style are themselves competitive assets. Using this data for post-training while retaining model ownership is more controllable than handing the data to a third party for black-box fine-tuning. Together's positioning is therefore not to sell model capabilities, but to sell infrastructure that turns data into model capabilities.

The Practical Threshold of Platformized Post-Training

Launching the service is only the first step; actual results depend on several variables. Data preparation remains the heaviest work: SFT requires high-quality input-output pairs, and RL requires computable reward functions and a stable interaction environment, none of which the platform can do on the customer's behalf. The tweet puts "You define the data" first, which also implies the workload on the customer side.

Another variable is training reproducibility. Post-training involves a large number of hyperparameters and reward design choices. Whether the infrastructure provided by the platform allows customers to clearly track experiments and compare different configurations directly affects iteration efficiency. Together emphasizes "tuned for quality and speed," but the trade-off between quality and speed differs across tasks, and actual performance needs to be verified in specific scenarios.

From an industry trend perspective, post-training is moving from a research phase to a product phase. Open-source models provide the starting point, data provides differentiation, and training infrastructure determines who can combine the two faster. Together's move to make post-training a managed service lowers the engineering threshold, but the threshold for data and objective definition remains on the customer side.

For teams that have already accumulated domain data and want to escape the capability limits of general-purpose models, this type of service offers an option between "building your own training stack" and "only using APIs." Whether it is worth adopting depends on the depth of data assets and the strength of the need for model control.

Training reproducibility depends on several engineering elements: fixed data versions and preprocessing scripts, complete records of random seeds and hyperparameter configurations, periodic checkpoint saving, and version management of reward function code. Without these, when two training runs produce very different results, the team cannot determine whether the cause is data issues, reward design issues, or randomness. If a managed platform can provide an experiment tracking dashboard and configuration snapshots, it can significantly reduce the cost of such troubleshooting; otherwise, customers still need to build their own logging and comparison workflows.

One-line Summary

Together launches a managed post-training service supporting SFT and RL, with custom models owned by customers.

Paragraph Summary

Together AI has launched the Together Post-Training service, enabling developers to turn open-source models into custom models they own through supervised fine-tuning (SFT) or reinforcement learning (RL). The platform handles underlying infrastructure such as the training loop, reward modeling, environment setup, and compute scheduling, and is tuned for quality and speed; customers define the data, reward functions, interaction environments, and training strategies. The article distinguishes the applicable scenarios of SFT and RL, noting that SFT is suitable for learning formats and domain knowledge, while RL is suitable for reasoning, agent decision-making, and preference alignment. The ownership statement means that weights trained by customers with their own data belong to the customers, which is especially important for teams with data moats. The article also notes that data preparation and objective definition remain the main thresholds on the customer side, and that training reproducibility and the quality-speed trade-off need to be verified in specific scenarios.