Yes, it's a pretentious title. But I think you'll see what I mean by the end of this notebook.
The scientific method is powerful. Very powerful. At its most basic level, it ensures consistency and documentation. Both of those things enable all other scientific advancement. Being right isn't the focus; being more right tomorrow compared to yesterday is.
Let's dive right into an example to illustrate the archetypical science experiment:
We have a cell culture. We apply drug A on the cell culture. We then measure the levels of B in the cell culture. Simple.
In the ideal scientific setup, you have an independent variable and a dependent variable. You're trying to test if there's a directional relationship between the independent variable and the dependent variable.
We have two variables, A and B. A is something we have full control over, specifically in the dosing. That is out independent variable. B is what we're observing and we hypothesize that it is related to A.
Basically, we're trying to take many measures of B for various causally selected A and seeing if the resulting data exhibits a relationship that looks like it could be collected solely from chance + measurement error.
What we're doing is below:
There are many, many other variables, but we want these to be kept constant. If any of these other variables changes for whatever reason, then we have a confound. That's why having cell cultures be from the same source, stored at the same temperature, stored for the same time, having the same person handle them, having experiments in the same environment, etc. are all important. If we have those things, we can reasonably assume that all other variables are constant. The reasonably assume is critical; even the hardcore scientist hand waves some assumptions into the picture that are obviously false.
I can be honest: I've had a very tough time figuring out what people mean by hypothesis for a long time. A real long time.
It only started making sense after I took a graduate course in signal detection and estimation. There, a hypothesis was an explicit equation that outlines how a variable is distributed. Simple, elegant.
Then I returned to science and realized the way hypotheses were actually used. They were used to set up a problem, figure out the experimental variables, and then completely ignored. Null-hypothesis testing, in fact, doesn't actually test explicit hypotheses of the phenomenon beind studied; it has an explicit hypothesis of the null or what the noise will look like.
We don't know the distribution of the noise. Measuring that is a job for the engineer of the device/method used to measure the underlying latent variable. In our case, if we have a lot of samples, then we don't care because the central limit theorem tells us that the noise becomes normally distributed with a sufficient sample size. We can then assume the probability distribution of seeing B values from A due to pure chance/noise is a closed-form normal distribution.
So, let's look at our two variables again.
Now let's see it in its bigger context.
This will be important in section 3; we don't have to limit ourselves to independent/dependent variable testing. Doing so enables us to use old statistical tools, which is useful, but our statistical tools have improved by lightyears.
Engineering and clinical medicine do not do well with the scientific method. They don't do well because science requires precise manipulation and control of variables. In a strict sense, it requires precise control of all variables besides the independent variable and dependent variable. It also benefits strongly from less-noisy measurement of the dependent variable. Let's see that below.
Experimental design is adjusting the situation and practicalities to fit the standard statistical tools and tests. In today's day and age, it's absurd to just limit ourselves to this.
In contrast, engineering design is more broad. It's more about outlining a system, like a force diagram, as identifying each possible relationship, then letting the data affirm the model constructed. If you make a manipulation, make a prediction, and that prediction is very close to being true, then the model you constructed has some element of truth to it. If that happens again, then you're even more confident. In two trials, you can be confident in your model.
As an example: let's say I tell you I know how to contruct an engine. I go away for 3 months, come back to you, and show you an engine that takes a car 1 mile. If you respond with "sure, but that's just an n of 1" then I hate you and we're not friends anymore.
When that engine works, it shows that I had some knowledge that reflected reality.
I'll split this out into a separate discussion (still in the MinS philosophy) but it's central to the question of "what is science". After really considering this question, I realized I'm not at all a scientist. I don't expect to ever know truth using the scientific method, just little tiny slices of a massive dimensional space.