Karl
Popper is the man who defined science for my generation and probably
several more. His book
The Logic of Scientific Discovery was published
before I was born (1934) and translated by him into English in the late
1950s. I read it in the early 1960's.
Popper defined science as
the cumulative testing of falsifiable hypotheses. So all scientific
statements must be made in a falsifiable form. Yes: "Ice transforms to
water at 0 degrees C". No: "Global climate is changing".
Any
experiment with humans must be double blind with a control group. Double
blind means the person doing the experiment can not know which subjects
are in the control group and which aren't. The control group must be as
identical as possible to the test group.
I have spent 50 years
designing research experiments. It is always difficult and requires
imagination. I rarely see good research done anywhere.
I want to
report one of my most unusual experiments. This is from an
earlier blog:
In 1985 I debated Dr. Peter Sherrill, who later became one of my
closest friends, over the issue of classic random sample versus a Kinsey
100% group sample. The debate was public, in a California Public
Utilities Commission meeting room. I was on one end of a ten-foot table
representing Public Advocates and Peter was at the other end
representing Field Research and Pacific Bell.
Two experts on
survey research brought together to resolve a thorny issue of public
concern.
In 1984, some Mexican labor activists reported to Robert
Gnaizda (Public Advocates) that farm workers in Salinas had monthly
phone bills of $32, with every feature from call forwarding to three-way
calling. None of these farm workers could speak English.
Gnaizda phoned around the state and found that the same phenomenon was
occurring in Chinese and Vietnamese communities.
What was
happening was later proved in utility hearings. Pacific Bell sales
people, who had strong prize incentives for signing up customers, would
sign up a customer for the maximum phone package of $32 anytime the
sales person reached a person who didn’t speak English. Don’t speak
English … you just bought $32 a month of technical phone services.
The
public debate was between Dr. Sherrill (photo on left) and me over the issue of
measuring the extent of the marketing abuse damage. How many people had
been signed up for services without knowing what was going on?
Dr.
Sherrill made the argument that classic random phone sampling with
multi-lingual interviewers would find the correct answer. I argued that
the most reliable method would be a 100% in-person sample of ethnic
groups. The survey money would be paid to the ethnic organizations for
100% turnout and participation of their members. This is the Kinsey
sampling method.
We were both persuasive or maybe we both
weren’t. Pacific Bell was ordered to do both kinds of sampling to find
how many people had been cheated and how much money had been falsely
taken from non-English speakers.
Dr. Sherrill (Peter) and I
designed and agreed upon the questionnaire.
The day arrived when
the computer printouts from both surveys were ready. Peter and I sat
down in a Corey Canapary & Galanis Research office (they had done my
ethnic group survey) near Union Square. We looked at the responses to
the first question. Then the second question, then all the questions on
the first page, then we rapidly went through the entire printout.
The
answers were the same from the two distinctly different survey methods.
I don’t mean the same in a statistical sense, I mean the same in the
sense of being one or two points different on every single question.
Peter
and I were stunned. As professionals we reverted to the term “robust
data” meaning the truth will out regardless of the method. But we were
still stunned at the nearly identical outcomes.
This research in
1985 was probably the most careful, thorough and reliable proof that the
Kinsey research method was excellent. Superb.
Peter and my
research results were never publish in a survey research journal.
Oh
yes, Pacific Bell had to pay $60 million in refunds and $16 million in
penalties.
-----------------------------
Top photo Popper, bottom photo Kinsey.
Recent Comments