I am going to ignore my first instinct (which is to rip into your polling methods and results) and opt for the constuctive suggestion route.
Clearly, coming up with meaningful results for a poll such as your "Reader's Choice Awards" is difficult. Based on your forum traffic, there are almost as many biases as there are readers. So, what do you do?
The answer may have its roots in your first attempt at statistically valid results. You say you chose a cross section from your reader database in order to get such results. The problem (as I see it) with this cross section, is it has database administrators voting on word processors while programmers vote on network OS's. I'll admit that there is some crossover for these people/products, but more often than not I am willing to bet that people voting outside of their own expertise base their selections on familiarity, and not always hands on familiarity.
How do you fix this? First you have to decide what the poll is really for. Are you trying to gather useful information that your readers can apply to their own decisions? Are you trying to start a controversy to spur interest? Are you trying to keep your readers involved? Are you just trying to let everyone have a little fun? Can you do two (or more) of these things at once?
Why not conduct multiple polls? Use your reader database and find a cross section of people familiar with (making use of) particular product categories. Send them category specific ballots and ask them for their choices. As part of the ballot, determine their a level of familiarity with each product. Here are some suggestions for grouping.
1.) software/hardware in production use
2.) software/hardware in pre-production testing
3.) software/hardware evaluation
4.) product specification evaluation
5.) advertising
6.) None
as well as
use daily/weekly/monthly/annually/once in a blue moon
and
length of time used/evaluated/been aware of
Do something similar with your general readership. Maybe you only ask for their familiarity with their actual picks. Maybe you make some answers optional. Maybe you let them opt to be included in an 'expert' group by completing an evaluation. Maybe you add some of the respondenets to your 'expert' groups when their familiarity warrants it.
Compile, rank (as you do your product evaluations), and publish all results. Yes, everything. Use the web, and present generalized and specific views. Offer some drill down options. Make some tab delimited files available for download. Put the 'expert's' opinions next to those of the 'masses'. This way readers that disagree with your ranking methods (or just your rankings) can apply their own methods. And everyone who cares can see what the results are based on.
So, does this sound like a plan (or at least the start of one)?
For questions or comments, email the Forums Editor