Alternative Hypothesis: loneliness destroys innovation. If you don't have a bestie to talk about your research, having an "AI companion" does help with reducing friction. Ultimately collaboration has other parts of research that no AI interaction can bare, including emotional support and one and a while "lover's quarrel".
P.S. the news of Replika's AI progressing towards erotic behavior implies academically oriented AI will regress towards uninspiring "paper mill" behavior when left unattended.
"This wasn't actually hard, once I started in earnest" - as a terminal procrastinator, I find the same thing. Inevitably, once I finally just sit down and START, it's much easier than I expected. You'd think that after enough instances I'd internalize this enough that it would lower the getting-started barrier, but alas, no such luck.
"will some tasks always be outside the reach of LLMs no matter how much they're trained" - this is the question I'm most interested in. I think it may be linked to the bullshit-detecting question. I'm certainly no expert, but my intuition is that current LLMs (and other ML models) are exceedingly effective pattern-generalizers. We've figured out a way to, after exposing the machine to enough examples of patterns, somehow encode the common basis of those patterns into the network such that we can then turn it around and get it to produce more patterns. The problem is, at this point that's all they are - patterns. There's no distinction between "real" and "not-real" the way that humans (usually) have. Hence why ChatGPT will happily produce a blogpost about how scented candles can stop WiFi sniffing attacks: https://twitter.com/benedictevans/status/1601035547900018688
It's possible that this limitation could be overcome, but I think it will take more than just "bigger models." Maybe someone can combine a LLM with some sort of "memory", just a big database facts, and it can use that as a starting point instead of trying to generate its patterns from whole cloth, or something like that.
Alternative Hypothesis: loneliness destroys innovation. If you don't have a bestie to talk about your research, having an "AI companion" does help with reducing friction. Ultimately collaboration has other parts of research that no AI interaction can bare, including emotional support and one and a while "lover's quarrel".
P.S. the news of Replika's AI progressing towards erotic behavior implies academically oriented AI will regress towards uninspiring "paper mill" behavior when left unattended.
"This wasn't actually hard, once I started in earnest" - as a terminal procrastinator, I find the same thing. Inevitably, once I finally just sit down and START, it's much easier than I expected. You'd think that after enough instances I'd internalize this enough that it would lower the getting-started barrier, but alas, no such luck.
"will some tasks always be outside the reach of LLMs no matter how much they're trained" - this is the question I'm most interested in. I think it may be linked to the bullshit-detecting question. I'm certainly no expert, but my intuition is that current LLMs (and other ML models) are exceedingly effective pattern-generalizers. We've figured out a way to, after exposing the machine to enough examples of patterns, somehow encode the common basis of those patterns into the network such that we can then turn it around and get it to produce more patterns. The problem is, at this point that's all they are - patterns. There's no distinction between "real" and "not-real" the way that humans (usually) have. Hence why ChatGPT will happily produce a blogpost about how scented candles can stop WiFi sniffing attacks: https://twitter.com/benedictevans/status/1601035547900018688
It's possible that this limitation could be overcome, but I think it will take more than just "bigger models." Maybe someone can combine a LLM with some sort of "memory", just a big database facts, and it can use that as a starting point instead of trying to generate its patterns from whole cloth, or something like that.