potentially dumb question: if you trained GPT-4 on all published scientific papers, would it be able to identify which outcomes or findings were repeated across the corpus? yes, right?

building @privy | surfing @ venice, ca | eating @ gjusta / gjelina | “lucky me, lucky mud” 🤠

Hyperliquid

Trad biotech founder, web3 dabbler, full-tilt bozo

Yes let me know what you think.
this is a problem I’m interested in and lot of people have tried to tackle it, but there’s no clear leader by mainstream adoption

wow, awesome. gonna try this out! will let you know what I think.

Tried it out years ago, but scite is prob a decent version of what you describe.
As others are saying, it will never be that clean (eg “this is a true fact”), but scite flags citations, differentiating support vs conflict vs mention & relative strength.

Meta problem is much of literature is trash tho…

scite.ai

DeSci • Don’t trust, verify science Causality.network • MuseMatrix.xyz DeSci Fellowship • DeSci Mic Host • ethChicago.xyz • KB3

That sounds awesome! Someone recently inputted their research article asking for a lit review & said that it came back with a lot of the same questions the peer reviews had.

If you’re looking for collaborators I know a few awesome AI folks in DeSci or publication platforms that value the code base if you want intros

ah, thank you! would love to try this with GPT-4 and see if making some sort of guiding parameters for methodology could help.

it don’t think it will ever replace literature reviews, but could make the first 60% effort more efficient.

Based on consensus in DeSci currently, GPT3 (at least, unsure if this is validated on GPT4), is able to effectively pull out findings and could in theory aggregate them but cannot effectively discern between good experimental design which could result in lots of inaccurate data being included in this compilation.

The only problem is externality pricing.

Building https://negationgame.com

So this was a problem tackled by Content Mine, which started as a literature scraping repo and sort of evangelizing org. Based out of UK because law said scraping copy-written material was chill.

I wonder if the fact that scientific papers are covered under copyright has something to do with the relative un-insightfulness of GPTs.

If we lived in a world where they were part of the corpus I'm certain GPTs would be more informed

product at @splits | building hq trivia for poker

You could get a very good idea of this through embeddings. Could find themes and directional results

One thing you could do is generate embeddings for all of them and then search based on that.

would tell you which papers discuss the same subject and where, but not necessarily similar findings AFAIK.

weird internet kid. word rotator. yapping on /okbanger. playing with @wannanaplabs. curating in /cameron | dev + vet + HBS. 
Cameronarmstrong.org 🫡

You could then use those answers and implement a search to find all the studies that contain them to count them that way

(Disclaimer I’m just a monkey with a keyboard and an internet connection)

Sort of:

It’s not like search where you can find all papers with a fuzzy matched set of criteria and count the “similar results”

It’s more like you could autocomplete question : answer pairs and the answers that show up most often in the training set would be more likely to appear

However you could then..

✵ Create ✵ Share ✵ Remix ✵

Experimental 2D/3D artist. Data science Pythonista. He/They. Nerdy++; (Nearly) all work CC-BY/CC0/FOSS

https://mxvoid.com

tl;dr version—finding general patterns, yes. The deeper statistical analysis you're implying, no.

According to GPT-4, nah.
 https://i.imgur.com/JrQqrFm.png

I did some research on industrial standards, the summaries were great. However when asking for specific references it returned quite some which I could not find at all…. Wrong but for what reason so the question is will it come up with results to be trusted..