🦒
@srijan.eth
IMPORTANT — this morning a bug was encountered on @noiceapp which had led to sending out ~20k unintended tips some older farcaster posts got tipped by mistake due to how we recently changed our event ingestion infra. around 750 users were affected with ~$650 worth tips sent out. why/how it happened: this got triggered after us internally migrating farcaster event ingestion infra from neynar's webhooks to their kafka based stream for reliability reasons. tl;dr: • farcaster emits reaction.deleted events when users hit storage limits • our system misinterpreted these as reaction.created due to missing prior context • that triggered real tips on older casts as our idempotency logic didn’t catch it what we’ve done: • fixed the event type parsing and strengthened idempotency checks • reimbursed all affected users with 2× USDC of the USD value in the tokens originally deducted • personally following up in DMs of affected users with a quick rundown issue is patched, reimbursements are complete and we’ve added safeguards to prevent this in the future. appreciate who helped debug this 🙏
22 replies
8 recasts
137 reactions
Kasra Rahjerdi
@jc4p
this is a way of handling stuff like this that i haven’t seen anyone else do, i feel like most would sweep this under a rug, major props to you for helping nudge us all in the direction of legitimacy!
1 reply
0 recast
12 reactions
Jordan
@ruminations
yeah, and he's like this consistently - same w/the leaderboard farming, saw an issue, transparently collaborated w/community to find a solution
0 reply
0 recast
1 reaction