Connor McCormick ☀️ pfp
Connor McCormick ☀️
@nor
Wow, in 2010 Peter Norvig talks about the problem of word segmentation, and you can totally see how from this single page of code we get LLMs 15 years later https://vimeo.com/4725365?#t=1755.052
2 replies
1 recast
7 reactions

t pfp
t
@tudorizer
was this different than stemming used on full-text search? will watch the video soon
1 reply
0 recast
0 reaction

Connor McCormick ☀️ pfp
Connor McCormick ☀️
@nor
stemming?
1 reply
0 recast
0 reaction

t pfp
t
@tudorizer
https://zilliz.com/ai-faq/how-does-stemming-improve-fulltext-search
1 reply
0 recast
0 reaction

Connor McCormick ☀️ pfp
Connor McCormick ☀️
@nor
seems to precede stemming
1 reply
0 recast
1 reaction

t pfp
t
@tudorizer
as with most comp sci terms, the history is long and started in the 70s one way or another https://en.wikipedia.org/wiki/Stemming
1 reply
0 recast
1 reaction