Carra Wu

@carra

291 Following

453 Followers

Carra Wu pfp

As an easter egg, Raju also "trash talks" opponents as the game odds shift, prompting Gemini for an appropriately confident quip, and outputting audibly. He's entering his teenage era :)

1 reply

0 recast

0 reaction

Carra Wu pfp

I hadn't done a hackathon in >5 years, and it was jaw dropping to see how powerful student engineers have become in the age of AI. It's humbling. Many thanks to the organizers for facilitating such a fun weekend, to for the last minute H100s, and to this awesome team! Now back to my day job...

0 reply

0 recast

0 reaction

Carra Wu pfp

We're very proud of Raju, his abilities, and how much we've been able to learn during this project. We went from knowing nothing about controlling robots with ML to being fully capable of modifying, initializing, training, and using them for specific functions. Devpost: https://devpost.com/software/raju-sassy-and-prodigal-chess-playing-robot

1 reply

0 recast

0 reaction

Carra Wu pfp

On the controls side, we knew Raju would struggle with identifying the target piece amongst 32 options on the board. After brainstorming, we decided to train a model with masked input frames that indicated the target tiles, leaving the transitions to be learned along with the pickup and placement routines.

1 reply

0 recast

0 reaction

Carra Wu pfp

This was a big risk, as we didn't have a precedent to guarantee success or a general model to tune from. Seeing Raju target pieces was a big payoff. Both of our custom solutions worked better than any alternative on the internet for the timeframe we were given.

1 reply

0 recast

0 reaction

Carra Wu pfp

The other big challenge we encountered was getting Raju to transport pieces from one tile to another. Mechanically, the base SO-ARM100 gripper struggled to hold the rounded chess pieces consistently. Redesigned gripper mandibles with centering indents and rubber solved this issue.

1 reply

0 recast

0 reaction

Carra Wu pfp

Building Raju came with a lot of challenges. First was interpreting the board state. We were unable find an out-of-the-box VLM or LLM capable of reliably parsing board state from the camera view(s) we'd chosen for training. Initially, we built an OpenCV system that identified pre/post move image diffs, but it struggled with the lack of contrast between pieces on their respective colored squares. In the end, we took a few dozen photos of various board states, labeled them, and trained a YOLO model to recognize the pieces. This solution was accurate once integrated with an algorithm that accounted for the previous board state.

1 reply

0 recast

0 reaction

Carra Wu pfp

This process involves masking the input frames (and the video frames used for training) with an overlay that indicates where the piece should be picked and placed.

1 reply

0 recast

0 reaction

Carra Wu pfp

Raju interprets board states using a fine-tuned YOLO object detection model, selects optimal moves via Stockfish, and then performs physical piece movements using an ACT policy trained to generally move a chess piece from a start square to a target square with an SO-ARM100 from LeRobot

1 reply

0 recast

0 reaction

Carra Wu pfp

On Saturday morning I met some cracked student engineers on the sidewalk and offered them a place to crash in NYC. 24 hrs later, we won the biggest robot arm hackathon ever. Meet Raju: an autonomous chess-playing robot (ELO >2500) inspired by GM Gukesh Dommaraju, who earned his title at age 12. Like Gukesh, Raju is poised for greatness at a young age, having just learned to move pieces yesterday.

1 reply

0 recast

3 reactions

Carra Wu pfp

Vibe coding Sundays!! 💖

0 reply

0 recast

4 reactions

Brenner pfp

Today's Vibe code project in collab with @carra: Send screenshot of a DM convo to Al, my assistant, and have it schedule a cal invite from it And we got it working!! cc @willc

4 replies

3 recasts

42 reactions

Mason Hall pfp

Availability Cascades seem like the outcome of gain-of-function research on the most potent combination of mimetic and memetic theory ever designed. Memetics = ideas are Memes (Dawkins/Dennett) Mimetics = human desire to emulate (Girard) Great read. https://pmarca.substack.com/p/availability-cascades-run-the-world

0 reply

2 recasts

4 reactions

a16zcrypto pfp

CSX NYC Day 1 w/ @guy @carra @michaelblau @jasonr @anna @mason & the team.

1 reply

2 recasts

14 reactions

lauren connolly pfp

lauren connolly

u had to be there 🪭🌀❣️ @chase @carra

0 reply

0 recast

8 reactions

Jason Rosenthal pfp

Jason Rosenthal

I’m excited to announce the 21 startups we’ve selected to join our Crypto Startup Accelerator (CSX) Fall 2024 cohort, which starts this week. Over the next 8 weeks, they’ll get firsthand lessons from top crypto founders, receive guidance and support from @a16zcrypto’s investing and operating teams, and build connections with each other and the broader a16z network. The CSX Fall 2024 companies are coming to NYC from 11 different countries, and they’re building on every level of the crypto stack, including AI, DeFi, infrastructure, and consumer applications such as photo sharing and mapping. We’re thrilled to partner with each of these CSX companies on their journey to product market fit. https://a16zcrypto.com/posts/article/announcing-the-csx-fall-2024-cohort/

7 replies

32 recasts

137 reactions

Michael Blau pfp

Very excited to kick off CSX NYC this week!!! Check out all the teams in the cohort 👇

0 reply

3 recasts

24 reactions

Ross Shuel pfp

For anyone building at the application layer: Today we introduce cash flows for application tokens. Application tokens — which correspond to smart contract protocols deploying services on top of blockchains that relay rights in “distributed businesses” — have historically lacked economic models connecting the tokens to the cash flows of the underlying businesses they are embedded within. There are three long-standing challenges to connecting cash flows to app tokens. 1. Governance 2. Value Distribution 3. Regulated Activity

1 reply

3 recasts

4 reactions

Carra Wu pfp

https://open.spotify.com/track/6AI3ezQ4o3HUoP6Dhudph3?si=xF1qRpv8Qk-KnHJc80ZK9A

1 reply

1 recast

6 reactions

Carra Wu pfp

https://open.spotify.com/track/1UP7PXne1lIYZB5G5aiRGL

0 reply

0 recast

1 reaction