Farcaster Devs

it's not going to be viable for most farcaster clients to index the entire network up front in the future - quite soon probably

we're going to see a handful of key infrastructure providers like neynar emerge that provide data for the vast majority of clients

the alternative is that clients will have to index "missing parts" of the network on-demand when new users sign up for their service. this is what opencast does to keep costs reasonable

this begs the question: if everything is going through a handful of infra providers anyway, does the level of data redundancy that farcaster hubs provide *and everyone is paying for* make sense?

tbh i think we'd be lucky if we avoid total consolidation and end up with even a handful of popular infra providers

if you are trying to build a fully featured Farcaster client and run the data infrastructure yourself: good luck! it’s a slog.

after indexing, you will have to build recommendations to beat churn. algofeed. then clustering. and AI. then hourly incremental DAG tasks and data pipelines. 

all under 200ms latency 👍

why would someone need (or want) to run their own data infra?

You do whatever you want with your own infra

easiest way to build on farcaster @ neynar.com | /neynar

Building @recaster-fc and other farcaster tools

@neynar doesn't get you there? 

i'm curious to know what the gaps are

Oh neynar indexer as a service may help with that, but i think you still need maintain the database

A usecase someone ask for: get notification when someone unfollowed me.

gotcha, right, maybe i'm confused about the initial cast

1. a lot of folks prefer to be as decentralized as possible, so this is following those efforts

2. you run your own infrastructure to keep data as close to your application as possible (you can collocate a hub and indexer within the same VM)

3. you want to own your ecosystem if Warpcast/Neynar is down

Neynar and Warpcast being down/not talking to each other came at the right time 👍

wait was Warpcast propagating to your hub at this time? 

Afaik they weren’t even writing to their own ones so it wasn’t just Neynar <> Warpcast issue, right?

building farcaster infra @ /neynar 🪐 | casting @ /rish

Warpcast might not be publishing to hubs atm, incl. their own hubs 

Check the Warpcast cast url on explorer.neynar.com to confirm 

If you use another client, the cast hash should show up as available on all hubs, incl. Warpcast hubs

If you see any examples otherwise, reply to this cast; that’s helpful info

messages were likely from other clients like supercast and wildcard that run on neynar hubs and those were being propagated as expected (we didn't have any downtime for the instance you're pointing out)
 
i'm not refuting any arguments against running your own infra :)

also didn't mean to saunter into this debate late to argue lol 

just happened to check notifications late on the neynar account lol https://warpcast.com/jrf/0x7db50134 and was curious what downtime on our side you were referring to

messages were likely from other clients like supercast and wildcard that run on neynar hubs and those were being propagated as expected (we didn't have any downtime for the instance you're pointing out)
 
i'm not refuting any arguments against running your own infra :) 

https://warpcast.com/rish/0x4803f307

we were definitely getting some messages (not sure the #),  though my point was being able to write to the hub within your our own ecosystem — messages are still able to be signed locally without other hubs if you have the keys