Varun Srinivasan on Farcaster

Varun Srinivasan pfp

Varun Srinivasan

A quick history of the Farcaster feed. Back in 2021, the first version of the feed just showed you every post, starting with the newest. There wasn't a way to unfollow! This was ok because we were just a 100 or so weird internet people who liked hanging out together. We added follows soon after, when someone started annoying someone else. This hurt new users quite a bit because they started with an empty feed and had to do a lot of work to make it good. So we picked the most active users each week and suggested they follow them when signing up. It was built in an incredibly simple way. Every time you loaded your app, we'd run a sql query to fetch the latest casts, hydrate the links and images and serve you a feed. At some point, we moved this into a background job so it would run every 30 seconds, and slow down if you stopped checking the app.

11 replies

29 recasts

235 reactions

Varun Srinivasan pfp

Varun Srinivasan

We added replies soon after, and had them show up in the feed. The network was still really small and slow at this time so we added "reply bumping". If a post got replied to, it would get bumped all the way to the top of your feed, showing you the newest reply. This made the feed feel a lot more dynamic and active. People quickly figured out that the best way to get engagement was to ask feel good questions. You'd get a lot of replies which kept you at the top of the feed. This was the first "algo gaming" and we had to put some limits on how often things got bumped up. The original feed architecture also stopped working when we got to a few hundred users. We switched to a fan-out model where a user’s feed was regenerated only when someone they followed casted. This had the nice property that every feed generation was useful — it would be guaranteed to add something new to the top of your feed. This new architecture introduced caching problems with counts. The feed would only regenrate if they were new posts, but people were liking the old posts frequently. These counts wouldn't update and then the user would get annoyed because they'd click into a post and see a different count. We solved this by adding periodic regeneration so you wouldn’t go a really long time without updating the counts. We also added a client side solution — if you loaded a conversation page and discovered a new cast count, your local cache would be updated which would override whatever the server response was.

3 replies

6 recasts

65 reactions

Varun Srinivasan pfp

Varun Srinivasan

People started complaining a lot about missing posts. You’d often see a quote cast and go “hey i never saw that cast that was quoted and I follow the author”. It happened to me a lot with Vitalik’s casts in particular — i’d often see a quote cast of it two days later and always miss the original. People thought the “algo feed” was to blame but when we dug in, it turned out to be the opposite problem. When I logged on in the morning I’d have about 100 unread casts. I’d read like 20-30 of them and then go do something else. When I come back there’s probably another 20 unread casts which I might catch up on. But the problem is that the casts I missed earlier are now buried under a bunch of read casts. It’s extremely unlikely that I’d go digging in to find them again. We introduced a “Since you’ve been gone” feed to solve this. If you were gone more than 8 hours you’d see a special feed on your next load. Instead of being reverse-chron, this feed would take all the casts in the last 8 hours and then order them by engagement. So if you had a fine time to catch up on what you missed at least you’d see the most interesting stuff first.

1 reply

0 recast

24 reactions

Varun Srinivasan pfp

Varun Srinivasan

The number of complaints of missing posts started creeping up again. The “since you’ve been gone” feed was a reasonable bandaid but wasn’t good enough. The main problem is that it was ephemeral and so you’d click on something and your feed would refresh and you’d lose it. Then all your good unread casts are buried underneath many read casts and you don’t really want to go searching for them again, so you just leave the app. The first solution to the problem was keeping track of what casts you’d viewed. We’d split your feed into two reverse chron chunks — the first would be unread casts and then second would be read casts. The way we tracked views was if more than 60% of a cast is on your screen for more than a second, the client would report it as viewed. The second solution was to introduce a ranking step that we called “author affinity”. We’d calculate how much you liked someones casts relative to how much they casted. So someone who cast 10 times a day but got 5 likes from you would rank much higher than someone who cast 100 times a day but got 10 likes. We’d reorder casts based on how much engagement you had with these people so that the people you cared about the most would show up first if there was a lot of unread stuff. A side effect here was that ranking often mean that a bunch of stuff from the same author would show up together. The most common case was @dwr.eth overwhelming people’s feeds. So we introduced an ordering step called author separation which would sort through the feed and space out casts from the same author by 5 spaces after everything else was done.

3 replies

0 recast

29 reactions

Varun Srinivasan pfp

Varun Srinivasan

This covers maybe the first 2 years of the Farcaster feed. If you're interested let me know, and I'll write up the rest sometime.

24 replies

0 recast

49 reactions

Pichi pfp

This was super cool! I love learning the lore and history but most importantly, understanding your thought process and what’s shaped the app into the Farcaster we know today.

0 reply

0 recast

6 reactions

Jacek.degen.eth 🎩 pfp

Jacek.degen.eth 🎩

Interested in a longer write-up, particularly curious about the time when channels launched, how that influenced your decisions, and your thoughts on a profile-based feed centered on people you follow versus a feed focused more on communities and topics you're interested in.

0 reply

0 recast

3 reactions

July pfp

These are please keep going

0 reply

0 recast

1 reaction

jtgi pfp

Interested. Great read.

0 reply

0 recast

1 reaction

Nick T pfp

yes please. this is alfa

0 reply

0 recast

0 reaction

Danica Swanson pfp

Great read and very eye-opening to learn about the trade-offs and your thought process. I'm bookmarking this for future reference and I'd love to read the more recent history too, especially as it pertains to the team's decision-making about channels.

0 reply

0 recast

0 reaction

rish pfp

Yeah interested. Also funny to remember back these feed mini eras

0 reply

0 recast

0 reaction

♾️ infinityadvisoryllc.com pfp

♾️ infinityadvisoryllc.com

0 reply

0 recast

0 reaction

pugson pfp

0 reply

0 recast

0 reaction

Stephan pfp

Thanks for sharing, interested in a write up

0 reply

0 recast

0 reaction

vrypan |--o--| pfp

Always interested in these posts, @v!

0 reply

0 recast

0 reaction

Adam pfp

Appreciate you demystifying the process and sharing the challenges you and @dwr.eth encountered along the way. Truly fascinating. I know you have a lot on your plate, but even if this was a weekly series I'm positive it would be a much anticipated read for those who are interested in the intricacies that make up the process. I know I'd love to see it.

0 reply

0 recast

1 reaction

Chris Carlson pfp

@chrislarsc.eth

Yes, curious to know when you changed to an ML feed. And let us know number of DAUs at each phase :)

0 reply

0 recast

0 reaction

Toluwanimi Ajisafe pfp

Toluwanimi Ajisafe

Please tell me more🤩🤩🤩 I love history

0 reply

0 recast

0 reaction

Tokenized Human pfp

Tokenized Human

@tokenizedhuman

This was great, thank you for taking the time to write it up. I have no idea how this stuff works so it's cool to hear about it. Would love to hear more.

0 reply

0 recast

0 reaction

M4NU33⌐◨-◨🎩 pfp

M4NU33⌐◨-◨🎩

Farcaster's story 🫡

0 reply

0 recast

0 reaction

D-wayñe 😇💜 pfp

D-wayñe 😇💜

I’d love to read more on this

0 reply

0 recast

0 reaction

0xZara.eth🎩 pfp

0 reply

0 recast

0 reaction

Jerry-d 🍖👽🎩 ↑ pfp

Jerry-d 🍖👽🎩 ↑

I’m very interested! You realize you’re writing part of your future biography when you encapsulate these historical updates for Farcaster, don’t you? 🤗

0 reply

0 recast

0 reaction