How X finds feed candidates: sources before ranking | Gyo
How X finds posts for your feed
Before filtering and ranking, X has to find candidates for a reader. Following, interaction history, retrieval services and cached posts offer different routes in.
Gyo
You open X and find a post from someone you have never followed. Before the system could decide whether you would like it, something had to put that post in front of the ranker.
That earlier step is retrieval: finding a set of candidates for a particular reader and request. In the Phoenix pipeline examined here, nine registered sources offer different ways into that set. Which ones run depends on the request, its history, cached results and configuration. Being retrieved earns a post a place for further processing, not an impression.
The first article started with ranking weights. The second followed the candidates removed before that ranking stage. Both leave a question further upstream: where did those candidates come from?
A feed request starts with the reader
It helps to picture one person opening the app. The request has a viewer ID and some context. Before fetching posts, the pipeline gathers more information about that viewer.
The registered query hydrators include followed accounts, blocked and muted accounts, subscriptions, action sequences, engagement signals and cached candidates. Other components can supply topics, demographics, location and recent search queries. Each has its own implementation and enable conditions; registration alone does not mean every field is available on every request.
“Hydration” is the code’s term for filling in data. There are two different jobs to keep apart. Query hydration fills in what the system knows about the request and reader. Candidate hydration, later, fills in what it needs to know about the posts it found.
For example, the retrieval-sequence hydrator asks an aggregation service for the viewer’s action sequence and stores the returned sequence and columnar bytes. The source using those fields does not need to assemble that history itself.
That changes the question being asked. We are following a request that looks for posts for this reader. A post’s public interaction counts are only part of the information that might be involved.
How does X find posts from accounts I do not follow?
The examined Phoenix pipeline registers retrieval paths using engagement-derived seeds, an aggregated action sequence and a cached popular-post list, alongside followed-account retrieval. Which sources run depends on the request and configuration. A direct follow is not required for every retrieval path.
Does X use all nine candidate sources on every request?
No. The runner calls enabled sources. Topic requests, in-network-only mode, available engagement signals, cached candidates and feature switches affect eligibility. When has_cached_posts is true, the eight other sources disable themselves and CachedPostsSource supplies the loaded candidates.
Does entering the candidate pool guarantee an impression?
No. The collected candidates still pass through hydration, eligibility filters, scorers, selection and later processing. Retrieval is a route into further consideration, not a reserved screen slot.
Does every X post get a fixed initial test audience?
The source implementations examined here do not establish a universal initial audience. Their result limits and thresholds apply to retrieval methods and viewer requests, not a guaranteed number of impressions assigned to each post.
Registered source
Route into the candidate pool
ThunderSource
Requests in-network posts using the viewer’s followed-account list
PopularPostsSource
Reads a cached popular-post list and excludes explicit seen IDs
TweetMixerSource
Requests recommendations from the Tweet Mixer service
SimclustersSource
Uses posts in the viewer’s engagement signals as similarity-search seeds
PhoenixSource
Requests retrieval candidates using an aggregated action sequence and viewer context
PhoenixTopicsSource
Makes a topic-scoped retrieval request
PhoenixMOESource
Makes a separately configured Phoenix retrieval request
SidSource
Sends engagement-derived seed IDs to the SID retrieval service
CachedPostsSource
Returns candidates already loaded into the query from cache
This is a list for one candidate pipeline, not an inventory of every way content can appear anywhere on X. It also is not a nine-way split of the screen. A source can be disabled, return nothing, fail, or overlap with another source. The request collects candidates before the later filters and ranking steps decide what to keep.
The distinctions become easier to understand if we follow a few of those routes.
Following someone gives the system a retrieval input
ThunderSource builds an in-network request from the viewer ID and followed-account IDs. It also passes explicit seen-post exclusions, a configured result limit and an algorithm setting. Depending on a switch and client availability, it calls one of two client paths for that request.
The returned records become candidates with post IDs, author IDs and, where present, reply, conversation and repost relationships. In a general recommendation request, they receive a ForYouInNetwork served type. In an in-network-only request, the type is RankedFollowing.
So a follow relationship has a concrete role before ranking: it is an input to a service retrieving in-network posts. It is not just a multiplier added to a score at the end.
It still does not reserve a slot for every post from every followed account. The request has a limit, the service chooses what to return, and the candidates continue through the rest of the pipeline. “I follow this author” and “this particular post must appear” are different claims.
There is a second distinction worth making. A candidate’s retrieval source and its in-network flag are separate fields. The in-network hydrator sets that flag by comparing the candidate’s author with the viewer and the followed-ID set. We should not assume that anything found through another retrieval service must be from an unfollowed author.
Your earlier interactions can become starting points
Suppose the viewer recently interacted with several posts. A retrieval method can use those posts as seeds: start from their IDs and look for other candidates associated with them.
SimclustersSource has a particularly readable version of this path. Its helper gathers IDs from the query’s explicit and implicit engagement-signal maps, orders signals by engagement time with the newest first, and removes repeated post IDs.
Those maps are populated earlier. The explicit-signal hydrator handles categories including favorites, reposts, replies, bookmarks, shares and the viewer’s own original posts. There is also a separate implicit-signal hydrator. What reaches either map depends on its data and switches.
The SimClusters source drops seed posts older than 90 days, then looks for candidates for the remaining seeds. It checks an in-process cache, then a shared cache, and sends misses to the similarity service. The request specifies embedding types and cosine similarity; it does not send a hashtag to a text search box.
Returned candidates must have a similarity score above 0.5 in this implementation. The source interleaves the per-seed lists, removes repeated post IDs, caps the combined output at 800, fills in core post data, and keeps candidates with a nonzero author ID.
Those numbers refer to different things. Ninety days limits seed age. The 0.5 threshold applies to a similarity result. Eight hundred caps this source’s combined output before its author-ID check. None is the number of posts the reader will see, and the similarity score is not a probability of liking a post.
The source also sends a separate maximum age for the candidates it wants back. Its published default is 48 hours. An older seed can therefore lead the search toward newer posts. The 90-day seed window does not extend the life of a candidate past the age check discussed in the second article.
Here is a small example of the interleaving, using letters as invented post IDs. One seed returns A, B and C; another returns D, A and E. The helper takes turns across the lists and skips duplicate IDs, producing A, D, B, E, C. This illustrates how the local merge works; it is not a trace of a real reader’s feed.
The practical connection is to the reader’s history. A candidate may be found through a post the reader interacted with, even when they have never followed its author. The wrapper tells us how seeds and responses are handled. It does not expose the whole embedding-training process or establish that asking people to bookmark your post will increase its reach.
Phoenix makes a different kind of retrieval request
PhoenixSource requires the aggregated retrieval sequence described earlier. It sends that sequence, its columnar representation, result-limit parameters and context to the Phoenix retrieval client.
The context-building functions can assemble fields such as client country and language, followed topics and starter packs, demographic and location values, installed-app data and recent search queries. Some values are optional or defaulted. Seeing a field in the request builder establishes that it can be sent on this path; it does not tell us its influence inside the remote retrieval model.
The source can also pass seen-post exclusions. When that option is enabled, it combines seen and served IDs, adds impression IDs when Bloom-filter entries are absent, and supplies available Bloom filters separately. It sorts and deduplicates the explicit exclusion IDs before sending them.
That means history checks need not begin at the 21-filter array from the second article. The retrieval request itself can ask for known posts to be excluded. Other sources do some local filtering too: Tweet Mixer, for example, checks the age of returned candidates before handing them back to the shared runner.
Phoenix’s response can identify different datasets. The wrapper maps HomeCold and ZeroFav results to distinct served types, with a general Phoenix type as the fallback. This shows explicit handling for those returned categories. It does not establish a guaranteed audience allocation for every new or zero-like post.
The conversion helper also preserves a retrieval score, a cluster label and a position within the served type. A post can therefore arrive at the later pipeline with a score already attached to its source record. That score must not be confused with the final ranking calculation discussed in the first article.
Topic requests and alternate retrieval paths
PhoenixTopicsSource is enabled for a non-bulk topic request, provided it is neither in-network-only nor using cached posts. It resolves the requested topic IDs and filtering mode, then sends a topic-scoped retrieval request. The general Phoenix source has a different topic condition: it allows non-topic or bulk-topic requests.
That is request routing. It does not show that adding a hashtag to a post places it into a corresponding retrieval pool. As we saw with topic filtering in the second article, internal topic IDs need their own evidence.
The remaining alternatives have their own conditions. PhoenixMOESource calls a separately configured Phoenix cluster and is suppressed when the SID source feature flag is on, alongside its other checks. That flag check is not a check that SID actually returned candidates.
SidSource uses engagement-derived seed IDs, with configurable seed and result limits. Its request includes prefix-depth settings; its response records shared-prefix depth as a retrieval-source score. The wrapper then hydrates core data and removes candidates whose author ID remains zero. The full SID indexing and retrieval mechanism is not explained by this caller alone.
Finally, PopularPostsSource reads a cached popular-post list, takes up to its configured limit, and removes explicit seen IDs. TweetMixerSource delegates a recommendation request to another service, passing client context and seen-ID exclusions. Their names describe entry points; the source wrappers should not be mistaken for complete implementations of the services behind them.
The published defaults enable general Phoenix, SimClusters and Popular Posts, while disabling Tweet Mixer, Phoenix MOE and SID. Those are snapshot defaults. Request conditions, feature-switch matching and other overrides still determine execution.
A cached request takes a different route
The ninth source makes the conditional structure especially clear.
When enabled, the cached-posts query hydrator looks up a cache key incorporating the viewer, topic IDs, in-network-only mode and video-exclusion setting. It decodes the payload into candidates and sets has_cached_posts when there are at least 500.
All eight other registered sources check that flag and disable themselves when it is true. CachedPostsSource does the opposite: it enables on the flag and returns the cached candidates already in the query.
Simply finding some cached records is not enough to select this route. In this implementation, 499 decoded candidates leave the flag false; 500 set it true. That is a branch threshold for a cached candidate set, not a reader’s page size or an author’s distribution quota.
Nor does a cache hit mean the shared pipeline immediately returns those posts to the screen. Candidate hydration, filtering, scoring and selection are still stages in the runner; individual components may themselves skip work on cached requests. We would need to follow those enable conditions to describe exactly what is reused.
The lists meet before the filters
The shared source runner selects enabled sources, runs them concurrently and waits for their results. It then appends successful result lists in source registration order. A result returned earlier in wall-clock time does not jump ahead merely because its service was faster. A source error contributes no list to this collection; it is not converted here into a post-level penalty.
The collection is not a fresh global sort of all retrieval scores. Different services can supply different kinds of scores, and the runner simply concatenates their returned candidates.
This ordering connects directly to the second article. If the same post ID arrives from Thunder and Phoenix, Thunder’s candidate appears first because Thunder is registered first. The later duplicate filter keeps that first occurrence and merges retrieval-source records from the duplicate. The post has not earned two screen slots, and the later source’s attribution is not discarded.
The pipeline also registers 15 candidate hydrators. They cover such fields as core post data, reply ancestors, quotes, media, subscription requirements, block relationships, topics and engagement counts. Some sources have already filled in selected fields internally, as SimClusters and SID do for core data. Hydration is not confined to one universal database lookup.
The stage runner then passes the hydrated collection through pre-scoring filters, scorers, selection and later processing. This is where the three articles join: retrieval finds possibilities, filters remove candidates, and ranking compares those that remain.
What this explains about being discovered
For the reader who opened the app at the start, an unfamiliar author is no mystery. The registered sources include paths based on engagement-derived seeds, action-sequence retrieval and a popular-post list, alongside followed-account retrieval. A direct follow is not the only input through which a post can be found.
For an author, the useful distinction is between being available somewhere in the system and being retrieved for one particular request. If a post never enters that request’s candidate set, improving its hypothetical score at a later stage cannot change that request’s result. The later scorer only works on what reaches it.
But low impressions do not reveal which step failed. The post might not have been retrieved, might have failed a check, might have ranked lower, or might have been removed later. The source list expands the investigation; it does not diagnose an account from its view count.
It also gives no defensible recipe for a universal “first batch” of viewers. Retrieval limits belong to services and requests. Similarity thresholds belong to particular retrieval methods. Neither establishes that every post receives the same initial test audience before being promoted or stopped.
Following the path is more useful than guessing that batch size. First ask how a post could have been found for this reader. Then ask whether it survived the eligibility checks. Only after that does the ranking calculation become the relevant explanation.
There is still a question inside that calculation: how does the system estimate what the reader will do? The next part of the series follows the predictions to which those weights are applied.
Source and verification notes
Source-checked on November 3, 2026. Links are pinned to revision 9106e9f, the same snapshot used by the earlier articles. Upstream HEAD was revision b220448 when checked. A targeted comparison found no changes between those revisions in the source-registration block, the nine source implementations, the cited query and candidate hydrators, the response/context helper or the shared source runner discussed here.
This review follows the public callers, helpers and execution path. It does not run X’s production services, reconstruct their indexes or inspect a private serving trace. The interleaving example uses invented IDs and was checked against the published helper. Snapshot defaults and local constants are described with their scope, rather than presented as a complete production configuration.
