No Such Thing As An Opinion!

July 31, 2011

Are my readers more intelligent than average, or less? Part One


Are my readers more intelligent than average, or less intelligent than average?

You'd probably like to get right to the results, right?  Well, I need to first provide some context! (And a bit of anticipation, hopefully!)

On Saturday, Fox News reported:

"An old geek adage has become much more believable as a new study has found evidence that Internet Explorer users are generally dumber than Chrome and Firefox users.

A company called AptiQuant, a self-proclaimed 'world leader in the field of online psychometric testing,' published the results of an online study that tested the IQs of users and grouped the results according to which browser respondents used.

The study found that users of Chrome, Firefox, Safari and Opera were all slightly above average in IQ test results, but Microsoft Internet Explorer users tended to be lower on the IQ scale.

These results aren't surprising because browsers such as Chrome, Firefox and Opera are generally used by professionals and other assorted 'geeks,' while Internet Explorer is the default option on Windows computers and is mostly used by the inexperienced and those who don't even know there are other options."

"Opera users will be happy to know they averaged the highest IQ scores of all the browsers. Firefox, Chrome and Safari scores were relatively similar to each other, but lower than that of Opera."


Although I don't find this to be particularly surprising, I do find it interesting, and I was curious to see the actual study results myself! 

Again, Fox News reported that Chrome, Firefox, Safari and Opera users have IQs that are just "slightly" above those of Internet Explorer (IE) users.  However, they also stated that Opera users had the highest IQ of all.

Fox News
' characterization makes it sound as if Opera users' scores are barely ahead of the others'.  Right?

So I wondered:  Is it possible that Fox News doesn't characterize the study results well?  Just how much of an IQ difference does exist between the users?


Did Fox News report the news correctly? 

I decided to investigate.  I went to the source itself.  Here's what I found:

1)  First, Fox News' summary of the study results does match the summary provided in the original report.

In the introduction, the company that conducted the study (AptiQuant Psychometric Consulting Co.) states:

"There was no significant difference in the IQ scores between individuals using Google Chrome, Mozilla Firefox and Appleā€™s Safari; however, it was on an average higher than IE users. Individuals using Opera, Camino and IE with Chrome Frame scored a little higher on an average than others. These data support the hypothesis that the IQ score and the choice of web browser are related."

So the summary describes the IQ differences as being "on an average higher" (why "on an average", not "on average"? Isn't that an error of theirs?) and "a little higher on an average than others".

But at this point, I still didn't know just how big the actual IQ gap was. I kept looking.


2)  The report includes a chart of the actual IQ results (see page 4-the exact IQ scores aren't provided, so I will have to estimate the actual IQ score visually.  My estimates are not likely to be off by more than two to three points).


The IQ Scores Of Internet Browser Users, By Browser


Here are the average IQ scores of users of the following browsers:

128:  Opera
126:  Camino
123:  IE with Chrome Frame
113:  Safari
111:  Chrome
107:  Firefox
  96:  IE 8
  90:  IE 9
  86:  IE 7
  83:  IE 6 


Time To Be Surprised

Wow.  Wow!

Do you see why I'm surprised?  Do you see why those scores don't jive with the report's written conclusion?

Those results cover a huge range of IQ scores.  The difference between a 128 score and an 83 score is absolutely huge, in statistical terms (I will demonstrate in just a minute)!

So why does AptiQuant state "individuals using Opera, Camino and IE with Chrome Frame scored a little higher on an average than others"? (My emphasis).

A little higher?  Are 128, 126 and 123 IQs only a little higher than IQs of 83, 86, 90, 96, 107, 111, 113?

Am I living in the twilight zone?  Those are huge differences!

The average of the first comparison group (128, 126 and 123) is 125.7.  The 125.7 combined average compares to the average of the second group:  98.

That's a 27.7 point difference.


Here's How Huge The IQ Difference Is

Just how large is 27.7 points?  To answer, I'll have to get just a bit technical. Bear with me.

27.7 points = 1.85 standard deviations (27.7 / 15).

Look at
this chart.  It is a standard bell curve distribution (IQ tests typically display in a bell curve shaped distribution, so this chart should be accurate for use).

The distance between each horizontal line is one standard deviation.  For example, the chart shows that 34.1% of the population scores between the bell curve's midpoint and one standard deviation to the right.

Is my point yet obvious to you?  If only one standard deviation can cover a large range of 34.1% of the population, then 1.85 standard deviations can cover an extremely large range of people, approaching 68.2% of the population!

Again, remember the context.  AptiQuant stated "individuals using Opera, Camino and IE with Chrome Frame scored a little higher on an average than others." (My emphasis).  Does that seem accurate?

Now, to be more precise, depending on where the range of scores falls among the bell curve, one standard deviation refers to different proportions of the population, not necessarily a proportion of 34.1% (as you can see, the third and fourth standard deviations above average include only about 2.2% of the population.  However, that range of scores is absolutely mind boggling in the sense that someone at the start of the second standard deviation would have an IQ as rare as approximately 2 in 100, versus someone at the start of the fourth standard deviation, who would have an IQ that is so rare that only one person in many millions, probably many billions, has that score).


Here's How Large The IQ Difference Is, Exactly!

So, let's get back to the study results.  Remember, AptiQuant said that a score of 125.7 was only a "little higher" than 98.

Let's look at the chart, in order to visualize where the range of scores fall on the curve.

The start point is 98, just to the left of the center of the curve.  Now move two standard deviation points to the right.  The end point of the range, 125.7, is slightly to the left of that second standard deviation.

Look how much of the population that range covers!  That covers about 46.34% of the population!

So, AptiQuant is apparently claiming that a difference in IQ spanning 46.34% of the population is only a "little" difference?

That's shocking!  I would consider a "little" difference to be one that spans, say, 3-5% of the population.  A "moderate" one to cover maybe 6 -19%, and a large one to cover maybe 20%+.


Other Explanations

Is it possible that an aspect of the test itself means that the test results aren't comparable to the traditional test results I'm familiar with?  Is it possible that the 27.7 point difference isn't huge?

No.  That's not possible.  The test has the typical standard deviation of fifteen, and the test itself is a Weschler test, one of, if not the most, famous IQ tests in the world:

"The age-appropriate Wechsler Adult Intelligence Scale (IV) test was given. Mazes, an optional subtest, was omitted from the WISC-iV. Verbal IQ (VIQ), Performance IQ (PIQ) and Full Scale IQ (FSIQ) scores were calculated. IQ scores have a population mean of 100 and a standard deviation of 15."

Now, people who don't know much about IQ scores might think that the difference between 125.7 and 98 isn't large.  To them, a large gap might be, say, a 200 score versus a 100 score.


What Should Be Expected Of The Study's Designers?

But wouldn't you think that a company running an IQ related study would be familiar with the magnitude of IQ differences?

I would, because I would think that people who may have majored in study design would have come across IQ distributions at some point during their education.  I partly base my assumption on my own personal experience with such topics.

But perhaps such people encountered statistics courses that didn't happen to plot an IQ distribution, and have never really taken a good look at an IQ distribution.


Very unlikely, I would think, but at least plausible?  Especially plausible if the study creator doesn't
specialize in IQ related studies, right?

So, what's the background of AptiQuant?

Their full name is:

"AptiQuant Psychometric Consulting Co."

Dictionary.com defines psychometrics as: 

"The branch of psychology that deals with the design, administration, and interpretation of quantitative tests for the measurement of psychological variables such as intelligence, aptitude, and personality traits. Also called psychometry
."

and also

"1. the branch of psychology concerned with the design and use of psychological tests
2. the application of statistical and mathematical techniques to psychological testing"

Wikipedia defines it as: 

"Psychometrics is the field of study concerned with the theory and technique of psychological measurement, which includes the measurement of knowledge, abilities, attitudes, and personality traits, and educational measurement. The field is primarily concerned with the construction and validation of measurement instruments, such as questionnaires, tests, and personality assessments."

So, not surprisingly, psychometrics is a psychology related field.  What are the odds that someone could be educated in psychology and not come across an IQ distribution?

What more do I know about the company?  The report states: 

"About AptiQuant
 
AptiQuant was established in 2006 in Vancouver, Canada by psychologist and business graduate, Leonard Howard. Over the past few years, AptiQuant has become a world leader in the field of online psychometric testing. The company designs, scientifically validates, and publishes a comprehensive range of psychometric tests for the recruitment, career guidance, career management, and staff development markets."

A "world leader"?  I'm surprised.

Now listen, perhaps they
are a world leader in at least one respect, and perhaps their work is normally excellent.  I haven't seen their other work, so I can't judge it.  But I can judge the work that I have seen.

I'm just shocked that their summary of results would claim that an IQ average of about 125.7 is only a "little higher" than an average IQ average of about 98.

It is certainly true that such a characterization is
far off the mark!


Is Fox News Wrong Too?

You might also consider my analysis to be a critique of Fox News, since Fox News's summary didn't match the actual source data.  Right?


However, I'll give them the benefit of the doubt, more than the benefit of the doubt, since Fox News's characterization did match the summary provided by the study's designer.

After all, shouldn't
Fox News be able to safely assume that the study designer's summary is likely to be accurate?


What About The IQ Of My Readers?

Now to the point you're waiting for!

Well, almost there...see Part Two on Monday August 1!



HTML Comment Box is loading comments...

 

 

Make a Free Website with Yola.