4.
Jul 5, 2007 12:40 PM

in response to:
runcorp
Re: Security suites, antivirus apps, firewall software, antispyware, and others in the same ilk: These things must be tested in batches, and the same batch, because they must be tested using the same "frozen" batch of viruses or threats. Testing with the same threats allows us to evaluate them on a level playing field. We certainly look at usability, features, etc., but the most important thing with these products is protection performance.
We can't test them as they get updated, else we'd have to retest every single one of them, every time any of them them revs. We try to pick a time during the year in which a critical mass of them has been updated, and then we test everything we can get our hands on. That's why the last batch you saw was from last year. We're working on a spyware feature for our October 2007 issue; it ought to go online in late August. Sorry that doesn't help you now, but I think that, when you do see those results, that you can be confident in our conclusions.
On the freshness quotient of other reviews: As with the apps above, we try to pick a time when we can find a critical mass of newly updated products, especially if our tests require batch subjective testing, as in the case with televisions for example. (Almost any TV looks great on its own; it's not until you see them side by side, all color-calibrated, all playing the same content, that you see differences).
But if there's something you'd like to see a review of, by all means, let me know. I'll tell you whether we might consider it, or if it's already in progress.