RSS · Spotify · Apple Podcasts · Pocket Casts
"Now everyone is starting to solve robustness without having a benchmark that shows that robustness is a problem... There's many, many anecdotal reports of these problems - and on deployed systems, things that really daily affect people! [...] I would just like to have benchmark, things where I can test algorithms on this."
"It's this very counter-intuitive problem where throwing away data points is a form of regularization. [...] You throw away things you already know."
"Information theory for machine learning is mostly useless in its current form. Let me elaborate on this. I do think that it can be fixed, though, and would be totally fine spending five years of my life to do that. [...] All of these information theoretic tools that they use, like for example entropy, really make sense only in discrete spaces. Continuous entropy does not make sense in most cases."
"I'm tired of seeing papers that have an introduction about A, an experiment about B, and a theory about C."
Thanks to Tessa Hall for editing the podcast.