Thanks, Daniel, for this stimulating interview. I appreciate Frankle’s deep commitment to empiricism. This interview offers a lot of practical insight for grad students about project and paper design. Build off the work of others. Be prepared to defend your alteration of original setups and variables. Don’t rely too heavily on theory. Follow the evidence. I would have loved to hear Frankle talk more about his empiricism in the context of question and topic selection. This matter seems to go beyond the scope of pure empiricism and opens up into larger philosophical, political, social, and economic concerns. The call for scholars to be more reflective or critical is very much in order, but the criteria that drives such critical processes is still very much in flux. For example, given the rapid state of change in the field, does it truly make sense to shift all research to one single kind of model when 3 to 5 years from now we don’t really have any idea what the files will look like? That said, I appreciate Frankle’s excitement for these new models, and I share his belief that they will continue to play a large role in the AI of the near future!
> I would have loved to hear Frankle talk more about his empiricism in the context of question and topic selection.
I should have asked him more about this!
> given the rapid state of change in the field, does it truly make sense to shift all research to one single kind of model when 3 to 5 years from now we don’t really have any idea what the files will look like?
I spoke to someone recently who told me, "when someone makes a boatload of money, they suddenly feel incredibly vindicated in their commitments" (or something to the same effect). Not that I think Jonathan suffers from anything like money going to his head! But I am with you that it's important to maintain a breadth of directions in the field—I think, whether more people work on them or not, we'll probably explore a lot of the limits of scaling language models, for instance. I do love your broader point that the criteria driving critical processes are in flux.
Thanks, Daniel, for this stimulating interview. I appreciate Frankle’s deep commitment to empiricism. This interview offers a lot of practical insight for grad students about project and paper design. Build off the work of others. Be prepared to defend your alteration of original setups and variables. Don’t rely too heavily on theory. Follow the evidence. I would have loved to hear Frankle talk more about his empiricism in the context of question and topic selection. This matter seems to go beyond the scope of pure empiricism and opens up into larger philosophical, political, social, and economic concerns. The call for scholars to be more reflective or critical is very much in order, but the criteria that drives such critical processes is still very much in flux. For example, given the rapid state of change in the field, does it truly make sense to shift all research to one single kind of model when 3 to 5 years from now we don’t really have any idea what the files will look like? That said, I appreciate Frankle’s excitement for these new models, and I share his belief that they will continue to play a large role in the AI of the near future!
> I would have loved to hear Frankle talk more about his empiricism in the context of question and topic selection.
I should have asked him more about this!
> given the rapid state of change in the field, does it truly make sense to shift all research to one single kind of model when 3 to 5 years from now we don’t really have any idea what the files will look like?
I spoke to someone recently who told me, "when someone makes a boatload of money, they suddenly feel incredibly vindicated in their commitments" (or something to the same effect). Not that I think Jonathan suffers from anything like money going to his head! But I am with you that it's important to maintain a breadth of directions in the field—I think, whether more people work on them or not, we'll probably explore a lot of the limits of scaling language models, for instance. I do love your broader point that the criteria driving critical processes are in flux.