Attention
conservation notice: Links to forbiddingly-technical scientific papers
and lecture notes, about obscure corners of academia you don't care about, and
whose only connecting logic is having come to the attention of someone with all
the discernment and taste of a magpie (who's been taught elementary probability
theory).
Or whatever the heck it is I study these days.
(I
did promise that
this series would be intermittent.) In no particular order.
- Modibo K. Camara, "Computationally Tractable Choice" [PDF]
- I'll quote the abstract in full:
I incorporate computational constraints into decision theory in order to capture how cognitive limitations affect behavior. I impose an axiom of computational tractability that only rules out behaviors that are thought to be fundamentally hard. I use this framework to better understand common behavioral heuristics: if choices are tractable and consistent with the expected utility axioms, then they are observationally equivalent to forms of choice bracketing. Then I show that a computationally-constrained decisionmaker can be objectively better off if she is willing to use heuristics that would not appear rational to an outside observer.
- If you like seeing SATISFIABILITY reduced to decision-theoretic optimization problems,
this is the paper for you. I enjoyed this partly out of technical interest,
and partly to see Simon and Lindblom's heuristic arguments from the 1950s
rigorously validated.
- One last remark: the slippage of "rationality" in the last sentence of the abstract is fascinating. We started by wanting to define "rational behavior" as being about effectively adapting means to ends; we had an intuition, inherited from 18th century philosophy, that calculating the expectation values in terms of rat orgasm equivalents would be a good way to adapt means to ends; we re-defined "rational behavior" as "acting as though one were calculating and then maximizing an expected number of rat orgasm equivalents"; now it turns out that that is provably an inferior way of adapting means to ends, and we have to worry about what it says about rationality. There's something very wrong with
this picture!
- (Thanks to Suresh Naidu for sharing this paper with me.)
- Carlos Fernández-Loría and Foster Provost, "Causal Decision Making and Causal Effect Estimation Are Not the Same... and Why It Matters", arxiv:2104.04103
- To make an (admirably simple) argument even simpler: Think of decision-making as a classification problem, rather than estimation. If your classifier mis-estimates \( \mathbb{P}\left( Y|X=x \right) \), but you're nonetheless on the correct side of 1/2 (or whatever your optimal boundary might be), it doesn't matter for classification accuracy! So if you over-estimate the benefits of treatment for those you decide to treat, well, you're still treating them...
- Ira Globus-Harris, Michael Kearns, Aaron Roth, "Beyond the Frontier: Fairness Without Privacy Loss", arxiv:2201.10408
- My comments got long enough to go elsewhere.
- Hrayr Harutyunyan, Maxim Raginsky, Greg Ver Steeg, Aram Galstyan, "Information-theoretic generalization bounds for black-box learning algorithms", arxiv:2110.01584
- I was very excited to read this --- look at the authors! --- and it did not disappoint. It's a lovely paper which both makes a lot of sense at the conceptual level and gives decent, calculable bounds for realistic situations. I'd love to teach this in my learning-theory class, even though I'd have to cut other stuff to make room for the information-theoretic background.
- Adityanarayanan Radhakrishnan, Karren Yang, Mikhail Belkin, Caroline Uhler, "Memorization in Overparameterized Autoencoders", arxiv:1810.10333
- I was blown away when Uhler demonstrated some of the results in a talk here, and the paper did not disappoint.
- Mikhail Belkin, "Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation", arxiv:2105.14368
- Further to the theme.
- Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel, "Extracting Training Data from Large Language Mode