Idea Machines
Professor Michael Strevens discusses the line betwee...
more
Jan 18 2022 1h 3m
Chapter 1 59 sec
Chapter 2 59 sec
of cognitive scienceChapter 3 22 sec
knowledge about the worldChapter 4 11 sec
Ben: Where did this idea of the, this, the sort of conceptual framework that you came up with come from?Chapter 5 24 sec
Michael: Well, there is an interesting origin story, or at least it's interesting in a, in a nerdy kind of way. So it was interested in an actually teaching the, like what philosophers call that logic of confirmation, how, how evidence supports or undermines theoriesChapter 6 59 sec
look back on it and think of as being a little bit naive and clueless. And I had at some point in trying to make this stuff appealing in the right sort of way to my students so that they would see it it's really worth paying attentionChapter 7 59 sec
it was kind of puzzling me why it would be that there would, there would be this very kind of objective standard for something counting is evidence that nevertheless offered you more or less, no help in deciding what the evidence was actually telling youChapter 8 35 sec
like a game where everyone agrees on the rules and where people could be feeling more comfortable about the validity and importance of what they were doingChapter 9 23 sec
Ben: I love that. I feel in a way you did is like you like step one, sort of synthesized Coon and popper, and then went like one step beyond them. It's, it's this thing where I'm sure you'd go this, this, the concept that whenever you have like two, two theories that seem equally rightChapter 10 20 sec
contradictory, that demand is like that, that is a place where, you know, you need more theory, right? Because like, you look at popper and it's like, oh yeah, that seems, that seems right. But then there's you look at Kuhn and you're like, oh, that seems right. And then you're like, wait a minute. Because like, they sort of can't both live in the broom withoutChapter 11 38 sec
Michael: adding somethingChapter 12 44 sec
that process, if you kind of take a few steps back, has the look of pop and science in the sense that, in the sense that scientists, but now unwittingly rather than with their critical faculties, fully engaged and wittingly are, are taking the theory to a point where it just cannot be sustained anymore in the face of the evidenceChapter 13 14 sec
Ben: So it's like, like you could almost phrase Coon as like systemic pop area. Isn't right. To like no individual scientist is trying to do reputation, but then you have like the system eventuallyChapter 14 1 sec
refutes. And that like, that is what the paradigm shiftChapter 15 1 sec
Michael: is. That's exactly right. OhChapter 16 32 sec
Ben: that's fastChapter 17 22 sec
Michael: you know, in a way that, and that's completely fine, you know, people to drive a car, you don't know how the engine worksChapter 18 59 sec
And in fact the best drivers may not have very much mechanical understanding at all. And it's fine for scientists to be a part of the system and do what the system requires of them without really grasping how it works most of the time. 1, 1, 1 way it becomes important is when people startChapter 19 58 sec
results that are designed more to be published and cited in advance of scientist's career than to be the most useful, a summary of research. And then you, and you get time to simply then even worse, choosing their research direction, less out of curiosity, or the sense that they can really do something valuable for the world at large then because they see a narrower and shorter term opportunity to make their own nameChapter 20
BenChapter 21 6 sec
Yeah. Okay. That makes a lot of sense. It's like, what are the, what are the actual, like core pieces that, that drive the engine?Chapter 22 20 sec
Michael: So that's the practical, that's the practical side of the answer to your question. You might, people should care. I thing it's a fascinating story. I mean, I love these kinds of storiesChapter 23 31 sec
Ben: yeah, no, I mean, yeah. I think that I like that there are some people who just like, think it's fascinating and it's like also justChapter 24 53 sec
like sort of this idea of like P hacking, you're actually seeing, you're seeing what you like th the, the mechanisms that you described in the book in play where it sort of, it used to be that looking at P values was like, like having a good P value was considered sufficient evidence, but then we like now see that, like, having that sufficient P value doesn't, isn't actually predictiveChapter 25 5 sec
Michael: exactly. That's exactly right. So the, the whole, the significance testing is one of these, it's aChapter 26 59 sec
particular kind of instanciation of the sort of broadest set of rules. We, this whole rule based approach to science where you set up things. So that it's very clear what counts as, as publishable evidence, you have to have a statistically significant results in that P P value testing and stuff is the, is the most widespread of kind of way of thinking about statistical significanceChapter 27 59 sec
look at those rules and say, well, let me seeChapter 28 10 sec
very clear objective rulesChapter 29 48 sec
Ben: Yeah. And then, so it's like, so, so that game, that game went out of whack, but then sort of like there's. The broader metagame that is like that that's the, the point of the consistent thingChapter 30 23 sec
one that came to mind was like a lot of evolutionary arguments were sort of, because it's based on something that is lot like is in the past there there's sort of no way to. To gather additional evidence. W would you say that, like, it's actually, you have a fairly strict bound on what counts as science?Chapter 31 35 sec
Michael: It is, it is strict, but I think it's, it's not my, it's not in any way. My formulation, this is the way science really is now. It's okay. The point of sciences to is to develop theories and models and so on, and then to empirically test them. And a part of that activity is just developing the theories and modelsChapter 32 59 sec
unscientific, but then that the question of course immediately comes upChapter 33 36 sec
question that's come up. Well, maybe we are being too strict. Maybe we, if we could, we would encourage the creation of more useful, interesting illuminating explanatorily powerful models and theoriesChapter 34 8 sec
Ben: And, and, and your argument would be that like for the ladder, that is well don'tChapter 35 3 sec
Michael: rush in, I would say, you know, think carefully before you do itChapter 36 9 sec
Ben: No, I mean, I find that that very another, another place where I felt like your frameworkChapter 37 58 sec
I'm not quite sure what the right word is. Like sort of like there was, there was some friction was, is with especially with the the, the Taconic principle of needing to find like, sort of like very minute differences between what the theory would predictChapter 38
Michael: YeahChapter 39 59 sec
noChapter 40 59 sec
even, to make these much less specific predictions, you still need a lot of information and computing power, but I think most, most science of complex systems hinges on hinges on relaxing the, the demands for, forChapter 41 16 sec
move in that direction and conforms to the modelChapter 42 42 sec
Ben: another. Yeah. The, the thing that I guess that I'm, I'm sort of hung up on is if you want, like, if you relax the specificity of the predictions that you demand it makes it harder than to sort of compare to compare theories, right?Chapter 43 5 sec
to get evidence, you need to re lacks specificity it makes it then harder toChapter 44 53 sec
Michael: theories. No, that's very true. So before you, before you demand, is that theories explain why things fall to the floor when dropped then? Good. Einstein let's go. Aristotle looks. Exactly. Yeah. And one reason physics has been able to make so much progress is that the model, all Sara, the models are simple enough that we can make these very precise predictions that distinguish among theoriesChapter 45 59 sec
That the underlying processes are in some sense, fairly uncontroversial. And the hard part is finding the right kind of model to put them together in a way that is much simpler than they're actually put together in reality, but that still captures enough of those underlying processes to make good predictionsChapter 46 34 sec
very true. Yeah. But typically that complex systems kinds of models are rather more specific than that. I mean, usually they're too specific and they give you, they, they, they say something very precise that doesn't actually happen. Right. And what you're doing is you're trying to bring that, that particular prediction closer to what really happensChapter 47 24 sec
Ben: Yeah. But that makes sense. And so sort of to like another sort of track is like what do you think about like theory free?Chapter 48 5 sec
incredibly accurate predictions, like sort of, how does that square with, with the iron ruleChapter 49 53 sec
Michael: in your mind? That's a great question. So when I formulate the iron Roy, I build the notion of explanation into itChapter 50 59 sec
you say, okay, we now have we now have we've now finished doing the science of economics because we've found out how to build these neural networks that predict the economy, even though we have no idea how they work. Right. I mean, I don't think so. I don't think that's really satisfying because it's not providing us with the kind of knowledge that science is working towards, but I can imagine someone saying, well, maybe that's all we're ever going to getChapter 51 24 sec
but scientists have always have always played around with theories that seem to get the right answer for reasons that they don't fully comprehendChapter 52 34 sec
Ben: It's like, I sort of think of it in terms of. Almost like compression where the thing that is great about explanatory theories is that it compresses all, it just takes all the evidence and it sort of like just reduces the dimension drasticallyChapter 53 11 sec
explosion of I don't know, like of whatever is doing the explaining. Right?Chapter 54 47 sec
Michael: although it may be with these very complicated systems that even in an explanatory model is incredibly uncompressed. Yeah, exactly. Inflated. So we just have to, I mean, I think it's, it's kind of amazing. This is one of my other interests is the degree to which it's possible to build simple models of complicated systems and still get something out of them, not precise predictions about, about, about what's going to happen to particular components in the systemChapter 55 59 sec
about making predictions that are real predictions, but but a bit more qualitative, you know, will. Well one of the very first uses of these models was to answer the question of whether just generally killing a lot of the animals in an ecosystem will lead the the prey populations to increase relatively speaking or decreaseChapter 56 30 sec
is, I mean, here you are modeling this ocean. That's full of many, many different species of fish. And yet you just have a few differential equations. I mean, that look complicated, but the amount of compression is unbelievable. And the fact that you get anything sensible out of it at all is truly amazingChapter 57 28 sec
Ben: Or, or, or developed sort of a, an entirely different way of attacking those kinds of systemsChapter 58 15 sec
to to, to Titanic sort of like going down to really more and more detailChapter 59 43 sec
Michael: When you modeling, I mean, festival, they might be new modeling machinery and new kinds of mathematics that make it possible to compress things that were previously incompressible, but it may just be, I mean, we look at you look at a complicated system, like the, like in an ecosystem or the weather or something like thatChapter 60 59 sec
around and it's different fish being eaten, but still the overall number of each species being eaten is about the same, you know, it kind of all evens out in the end and that's what makes the compression possible. But if that's not the case, if, if these small changes make differences to the kinds of things we're trying to predict people, of course often associate this with the metaphor of the butterfly effectChapter 61 20 sec
And yet the idea that the rate of inflation is going to depend on this decision by somebody walking down the aisles of a supermarket in higher, that just doesn't seem right. It does seem that things kind of cancel out that these small effects mostly just get drowned out or they, they kind of shift things around without changing their high-level qualitative patentsChapter 62 38 sec
Ben: I mean, this is the diversion, but I feel like that that sort of like touches right on, like, do you believe in, in like the forces theory of history, more like the great man theory of history, right? And then it's like, and people make arguments both ways. And so I think that. And we just haven't haven't figured that outChapter 63 47 sec
not just like science machinesChapter 64 11 sec
Michael: there? Well, it's true that the, the metaphor of the coral reef doesn't doesn't capture that aspect of science. It's very true. So I think on the one hand that what's what is captured by the metaphor is the idea that theChapter 65 59 sec
the, what science leaves behind in terms of, of evidence that can is, is, is interpreted a new every generationChapter 66 59 sec
which, which are not for them to create a new, but rather just to kind of learn how to just have a use various instruments, how to use various statistical techniques actually. And so there's this continuity to the knowledge let's, as I say, not captured at all by the reef metaphor, both of those things are going, are going onChapter 67 7 sec
a data trove than, as, than a kind of a body of established knowledgeChapter 68 28 sec
Ben: then I think. Question is, is it's like, if, what counts as evidence changes and all you are getting is this data trove of things that people previously thought counted as evidence, right? Like, so you know, it's like, they all, all the things that were like, like thrown out and not included in the paper doesn't like that make it sort of harder to reinterpret itChapter 69 22 sec
Michael: Well, there's, I mean, yeah. The standards for counselors, evidence, I think of as being unchanging and that's an important part of the story here. So it's being passed on, it's supposed to be evidence now of course, some of it, some of it will turn out to be the result of faulty measurements, all these suspicious, some of that even outright fraud, perhapsChapter 70 59 sec
why you wouldn't want to just kind of take it for granted and they get that, that side of things is not really captured by the reef metaphor either. Yeah. But I think that the important thing that is captured by the metaphor is this idea that the, what, what's the thing that really is the heritage of science in terms of theory and evidence, is that evidence itself?Chapter 71 5 sec
trustworthy or untrustworthy as the evidence we're getting today. And there it is just recorded in the animals of scienceChapter 72 13 sec
Ben: So it's much more like the, the thing that's important is the, like the, the process and the filtering mechanism, then the, the, the specific artifacts that yeahChapter 73 39 sec
Michael: Come out, I'll make me part of what I'm getting at with that metaphor is the scientists have scientists produce the evidence. They have their, an interpretation of that evidence, but then they retireChapter 74 7 sec
somewhat skeptical eye, but not too skeptical. And that's the, that's the, the real treasure house yeah. OfChapter 75 51 sec
Ben: science and something that I was, I was wondering, it's like, you, you make this, this really, you have a sentence that you described, you say a non event such as sciences non-rival happens, so to speak almost everywhereChapter 76 15 sec
science consistently failed to arrive and ask, like, is there, is there something else like some other kind of like like intellectual machinery that also that has not arrived. Did you think, like, is it possible to look for that?Chapter 77
Michael: Oh, you meanChapter 78 4 sec
Ben: now? Yeah. Or like, like, or could someone have predicted science in the past? Like inChapter 79 37 sec
Michael: the past? I, I mean, okay. I mean, clearly there were a lot of things, highly motivated inside. Why is thinkers. Yeah. Who I assume I'd have loved to sell the question of say configuration of the solar system, you have that with these various models floating around for thousands of yearsChapter 80 59 sec
sun rotates around the earth and the inner planets rotate around the sun was also well knownChapter 81 59 sec
Did they think, did they themselves think that something was missing or did they think they had what they needed?Chapter 82 28 sec
not seem to have, sort of see a gap to see the need, for example, for precise, qualitative experiments or, or, or even the point of doing them. YeahChapter 83 27 sec
Ben: it was just it just makes me wonder w w some, some period in the future, we will look back at us and say like, oh, that thing, rightChapter 84 2 sec
Michael: Yeah. Well, yeah, I'm a philosopher. And we in, inChapter 85 59 sec
philosophy, it's still, it's still much like it was with AristotleChapter 86 36 sec
And in effect scientific techniques to determine what to, to adjudicate among philosophical theories, mainly by throwing away most of the theories as meaningless and insufficiently connected to empirical factsChapter 87 22 sec
Ben: I, I do want to be respectful of your time, the like 1, 1, 1 last thing I'd love to ask about is like, do you think that and, and you, you talked about this a bit in the book, is that, do you think that the way that we communicate science has become almost too sterile. And sort of one of my, my going concernsChapter 88 59 sec
is this the way in which everybody has become like super, super specializedChapter 89 59 sec
Michael: YeahChapter 90 59 sec
publications, which try to do this job. That's trueChapter 91 59 sec
iron ruleChapter 92 59 sec
building at a bomb, it's usually something, something horrendous or a a device, the device for the guns and bombers and so on that would allow thatChapter 93 32 sec
it's a very, it's a very attractive period in the history of human thought. When you go back and look at some of the things people were writing in the late forties and fifties, Computers, how the mind works. And so on. And I think some of that is coming out from this, this kind of almost scrambling process that that happened when, when these very specific kind of military engineering problems are solved by throwing people together who never normally would have talked to one anotherChapter 94 11 sec
Ben: I have a friend who described this as a serious context of use is it is a thing. And it's, I, I mean, I'm, I'm incredibly biased towards looking at that period. Okay. ButChapter 95 2 sec
Michael: I guess it's connected to what you're doingChapter 96 11 sec
Ben: Absolutely. Is I do you know whoChapter 97 56 sec
so I'm like, I agree with you very strongly. And it is it's. I find, I always find that fascinating because I feel like there's, there's like this. I mean, there's this paradigm that sort of got implemented after world war II, where do you think like, oh, like theory leads to applied science leads to leads to technology, but you actually see all these, these places where like, trying to do a thing makes you realize a new theoryChapter 98 2 sec
Michael: problem to solve. It's not just the caseChapter 99 10 sec
of kind of enjoying kind of chatting about what you each do. And then just going back to the thing you were doing beforeChapter 100 22 sec
Ben: though, because the incentives in that situation sort of like now fall outside of the iron rule right. Where it's like, it's like, you don't care. Like you don't care about like, I mean, I guess to some extent you could argue like the thing needs to workChapter 101 17 sec
Michael: correct. That's true. But, you know, but I think as you're about to say, engineering is not science and it's not it's the own rule is not overseeing engineering. It's the it's engineering is about making things that work and then about producing evidence for, or against various ideasChapter 102 7 sec
Ben: but then it can sort of like, I guess it can like spark those ideas that people then sort of like take, IChapter 103 24 sec
was like, I mean, in my head, it's all of this, like I think of what would I call like phenomena based cycles where like, there's, there's like this big, like cyclical movement where like you discover this like phenomena and then you like, can theorize it and you use that theory to then do like, I dunno, like build better microscopes, which then let you make new observations, which let you discover new phenomenaChapter 104 34 sec
Michael: It's really difficult to tell where things are going. Yeah. I think the discovery of plate tectonics is another good example of this sea, of these, all of these scientists doing things that, that certainly not looking into the possible mechanisms for continental drift, right. But instead, getting interested for their own personal reasons and doing things that don't sound very exciting, like measuring the magnet, the measuring the ways that the orientation of the magnetic field has changed over past historyChapter 105 53 sec
of the iron molecules or whatever, and the lock and, you know, it's, I mean, it's not, it's not completely uninteresting, but in itself it sounds like a kind of respectable, but probably fairly dull sideline and geologyChapter 106 5 sec
Ben: I, I truly do think that like, moving forward knowledge involves like being almost likeChapter 107 15 sec
irresponsible, right?Chapter 108 43 sec
Michael: the reason that sort of thing happens is cause a bunch of people decide they're interested inChapter 109
That'sChapter 110 54 sec
Ben: actually one of the, one of my concerns about trying to do science with, with like no networks is. How many times do you see someone just go like, huh, that's funny. And like, like so far you can't like computersChapter 111 3 sec
Michael: Yeah. This is another one of those cases thoughChapter 112 29 sec
Michael: Yeah. This is another one of those cases thoughChapter 113 6 sec
Ben: And maybe perhaps like the, like the, the distraction and like saying, oh, that's funny. It's like the natural state of human affairsChapter 114 11 sec
Michael: Well, I think so. I think if we, we would all be like Aristotle and it turns out it was better for science fair, actually a little bit less curious and yeahChapter 115 10 sec
Ben: one could almost say that like the, the iron rule, like w w would you say it's accurate that like the iron rule is absolutely. But soChapter 116 21 sec
is breaking in the sense that like, like if, if like somehow there, like you could enforce that, like every single person only obeyed it all the time science, like we, we actually, we make serendipitous discoveriesChapter 117 37 sec
Michael: time. All right. Put it a little bit differently. Cause I see the rule list is not so much, it's not so much a rural for life. And for thinking is for, for sort of publishing activityChapter 118 30 sec
they, they, they thought about the, the point of their lives as, as kind of investigators of natureChapter 119 28 sec
Ben: seems like an excellent place to endChapter 120
