I have bookmarked your post for reading, but i have a tangential concern. Like the scale argument feels a lot like throwing say billions upon billions of say PDP-11s to see what sticks. What stuck was time-sharing. Algorithms will always trump scale. Maybe you would agree and i feel you do
This is a very fair argument. Better algorithms will always win over time.
But I think the decision to "throw billions upon billions of GPUs" has already been made today. Whether that's the right decision, only time will tell.
But until then, there are still some "small pivots" in software and hardware that I think can help - that's why the ENIAC story seemed interesting to me.
One of the first things I noticed at my tour of UPenn was the ENIAC display in the Moore building. Really looking forward to the next parts!
Wow that’s great, it’s one thing on my bucket list!
I have bookmarked your post for reading, but i have a tangential concern. Like the scale argument feels a lot like throwing say billions upon billions of say PDP-11s to see what sticks. What stuck was time-sharing. Algorithms will always trump scale. Maybe you would agree and i feel you do
This is a very fair argument. Better algorithms will always win over time.
But I think the decision to "throw billions upon billions of GPUs" has already been made today. Whether that's the right decision, only time will tell.
But until then, there are still some "small pivots" in software and hardware that I think can help - that's why the ENIAC story seemed interesting to me.