Content
@
https://opensea.io/collection/neynar-1
0 reply
0 recast
0 reaction
artlu 🎩
@artlu
strong opinion, weakly held: there are always better choices than Replicator - clients do not need complex joins - 2 Hubs handle concurrency as well as 1 Postgres constantly drip-fed by Replicator - tested APIs > rewriting logic in SQL - backfill is too complex: distinct state layers in Hub, bullmq, redis, postgres
3 replies
1 recast
1 reaction
alex
@alexgrover
Any client of even basic complexity will need joins IMO
1 reply
0 recast
1 reaction
artlu 🎩
@artlu
Do you have an example of the type of join needed by a FC client? My hypothesis: simple joins can be handled in python/JS/TS without a DB. Neynar has APIs for the more common joins that a client might need. Airstack has APIs for joins that involve off-Hub data.
1 reply
0 recast
0 reaction
alex
@alexgrover
For example, the hot/new/top filters on Flink. Neynar is using a replicator so I consider that to be the same thing as running your own.
1 reply
0 recast
0 reaction
alex
@alexgrover
In memory joins will work until you have a dataset larger than a few thousand records, which I think you’ll run up on for a feed algo depending on the parameters. I’m excited about the potential of an in-hub DB from @haardikkk https://warpcast.com/haardikkk/0x6cd113a2
2 replies
0 recast
2 reactions
artlu 🎩
@artlu
I thought 10k in memory would be enough for usability, and easy enough for a server to handle. I under-considered 2 things: - return casts by channel isn't in the APIs (yet?) - client may not always want to show "latest N results" but also "next N results on page Y" which is more efficient in a db query than memory
1 reply
0 recast
0 reaction
alex
@alexgrover
Depends on the use case IMO! For example “top casts by day/month/year/all time” will require much more than 10k. I think there’s always going to be some edge case feature that’s not possible through the hub APIs
1 reply
0 recast
0 reaction
alex
@alexgrover
If you don’t need the extra infra by all means keep things simple, but it’s a case by case decision rather than a blanket prescription and I think as clients get more complex the DB will become necessary more of the time
0 reply
0 recast
1 reaction
