[PCWorld]

Post your comments for How We Test here

[Rick09]

where are your tools that i used for years cant find a link on your new site?

[tammitv10]

I just viewed the Tmobile MDA specs on your site and saw inaccurate information:1. The MDA does have a Bluetooth connectivity option.2. The MDA does have email/Exchange connectivity options w/ActiveSync.3. The MDA does have music playback capability.4. The MDA does have voice memo recorder capability (voice note).The MDA does have a built in speaker in addition to headphone jack.

[wwalker]

For items like graphics cards, printers, scanners, etc. I see no indication that they will even install on Windows XP x64 or the soon to be released Windows Vista x64. Without info for supported operating systems your reviews are next to useless.

[encycloman]

Point and shoot cameras are not being evaluated for the total speed - time that elapses after one clicks the button and when the shutter goes off - a VERY FRUSTRATING item if one is trying to take pics of active children or pets!Any plans on including that in any future reviews?

[SBrumbelow]

One of the most important testing issues to me is, "How do you select the brands, models, and configurations of products to test?"Look at HDTVs for example. Your competitor, PC Magazine, has given its Editor's Choice of 32 inch LCDs to a model from Sharp. PCWorld has apparently not tested any models from that manufacturer.This question certainly applies to them as well since you have tested brands and models that they have not evaluated.Why do these situations occur?

[AlanStafford]

A couple answers, in reverse-chron order (some of your posts are little old, but for the benefit of all):SBrumbelow: We have tested Sharp televisions--several, in fact. Here's a link to a 37-inch LCD TVs chart, with a Sharp at #3 [url]http://www.pcworld.com/article/id,127742-page,1/article.html_. We're almost done testing a new 32-inch Sharp set (the Aquos LC-32D50U); it's review should go up on the site in a couple weeks. It would have been up sooner, but we must test TVs in batches, rather than one at a time, and we didn't have room for a batch of them until recently. PC Mag has the same problem--its review cycle just hit on that type of TV first (this time).But to the bigger question you pose: We select products based freshness, whether we can get a product from a vendor, whether it's relevant to our readership (for example, we don't test mainframes or most software that is purchased via seat licenses), and...boy, lots of things. We're always trying to speed up our reviews process, and as part of that we're trying to speed up when they post to the web. But our resources aren't infinite--meaning, we can't test every new product the moment it comes out. We certainly try to post reviews of high-profile products as quickly as possible, but a new TV doesn't cause as much mass hysteria as the Wii does, for example. We try to decide which products merit coverage, and which products merit rushing to the site.Add to that the fact that TVs and many other products must be tested in batches (many TVs look great until you put them side-by-side with other sets), and that often means we must wait for a mini-critical mass of new sets. encycloman: We don't lab-test cameras' shutter response time because it's not testable. If we had our performance analyst click a stop watch at the same time as he presses the shutter release, then presses it again when it takes a picture....just think of all the possibilities for variance. If he's faster at clicking the stopwatch one time, that throws off the result; if he's slower at pressing the shutter, that screws it up too. Cameras have different lenses; that can affect focusing and metering, which will affect shutter speed. In short, you can't assign a (reliable) number to shutter response. But we do try to make note of it in our subjective evaluations.wwalker: We do report support for several operating systems (including Linux), but finding out about Vista support has been very difficult, and may continue to be difficult until the OS actually ships. I understand about wanting to know about XP x64; let me ponder that one further.tammitv10: I scanned the text and the test report associated with that review; it does say that the MDA has Bluetooth (but as with many phones, you can't sync with a PC, meaning that you can't use the phone as a modem for your laptop, which is a major drag). As for the Exchange connectivity, I think that means that you can't access Exchange e-mail while mobile (i.e., without syncing). I will have to check with the editors involved about the music playback and voice memoing, but the text does say "Music playback sounded good through the bundled stereo earbuds, but tinny through the handset's speaker." It could be that the chart data is incorrect.Rick09: I'm not sure which tools you're asking about--if you're still monitoring, could you elaborate?

[rjw64]

I have a new maxstor III 200gb - great to save to, but have you tried to restore a file? I did and finally had to call support - and I still had a problem putting it back in the right place!

[mcbarker]

[quote name='AlanStafford']> > > > We don't lab-test cameras' shutter response time because it's not testable. If we had our performance analyst click a stop watch at the same time as he presses the shutter release, then presses it again when it takes a picture....just think of all the possibilities for variance. If he's faster at clicking the stopwatch one time, that throws off the result; if he's slower at pressing the shutter, that screws it up too. Cameras have different lenses; that can affect focusing and metering, which will affect shutter speed. In short, you can't assign a (reliable) number to shutter response. But we do try to make note of it in our subjective evaluations.I brought the subject of PC World camera testing up in another thread, and I think that the above response reinforces my assertion that PCW isn't equipped to accurately test cameras, and therefore, should leave it to photography magazines which are equipped to do such testing. Contrary to your assertion, a camera's "shutter response time", or "shutter lag" as it is known, can indeed be measured (and has been in the past by Popular Photography & Imaging magazine), but certainly not by an "analyst" holding a stop watch. That's like slowing down a flying jet plane so that the passengers can jump aboard. The speed is generally measured in fractions of a millisecond by equipment which excludes the human factor, and is infinitely more sophisticated than a stopwatch. The shutter lag on most newer cameras is generally so insignificant that is usually no longer included in professional reviews, but there are still a few cheap point and shoot units, and also some which include an image stabilization feature, which do suffer significant shutter lag.Does anyone remember Stereo Review magazine? They too changed format (and name), and tried to be an "everything for everybody" magazine, testing, in addition to audio equipment, cell phones, cameras, computer systems, DVDs, CDs, and other electrical gadgets du jour (does this sound like today's PCW?). Subscribers complained loudly, and cancelled their subscriptions in droves. I haven't seen that magazine on any newsstands anywhere in the last couple of years. This also seems to be the direction that PC World is headed, unfortunately.

[AlanStafford]

[quote:f90098eb30] The shutter lag on most newer cameras is generally so insignificant that is usually no longer included in professional reviews, but there are still a few cheap point and shoot units, and also some which include an image stabilization feature, which do suffer significant shutter lag.[/quote:f90098eb30] That, I think, reinforces my point. Why should we (or Popular Photography) spend many thousands of dollars on test equipment to test something that has become a non-issue? (Although I agree that some cameras still have a problem with this). We've spent gobs of money over the years on test equipment; much of it becomes obsolete very quickly, and it's painful to see such costly items gathering dust on a shelf. And hey, if the problem is so egregious to be noticeable, we will note it in our review and knock its score down.I don't know about this specialized test equipment, but I still doubt its capabilities. Camera viewfinders and metering systems are different; I know that when we set up our test shots, it can be tough to get models to capture the same shot. That is, we might have to zoom out with one camera to capture the object in the lower left corner, which makes the object in another part of the frame a bit farther away. As part of a subjective review (i.e., live judges looking at images), that's ok--those judges can recognize the difference and adjust. But I'm just not confident that test equipment can do the same.However, if you could cite the equipment used for this testing, I'd be happy to look into it.

[mcbarker]

[quote:cb9881d2b8]I don't know about this specialized test equipment, but I still doubt its capabilities. Camera viewfinders and metering systems are different; I know that when we set up our test shots, it can be tough to get models to capture the same shot. That is, we might have to zoom out with one camera to capture the object in the lower left corner, which makes the object in another part of the frame a bit farther away. As part of a subjective review (i.e., live judges looking at images), that's ok--those judges can recognize the difference and adjust. But I'm just not confident that test equipment can do the same.However, if you could cite the equipment used for this testing, I'd be happy to look into it.[/quote:cb9881d2b8]I don't know specifically which equipment they use, but I would assume that it's of the same nature of much of the equipment I use in my companies QA test lab, which is programmed to run one or more tests on a unit when plugged into that unit's computer. Given that digital cameras are basically computers with a lens and a sensor, I don't imagine that it would be difficult to measure the response time (or time difference) of the shutter button making electrical contact with the camera's computer versus the instant the sensor starts receiving an electrical impulse (starts recording the image). It would be possible to do this while the camera is connected through its communications port, to a computer system which contains a program written specifically to monitor the time difference between the two points of contact. Pinpointing the cause (poor autofocus system, image stabilization lag, cheap electronics, etc) would be almost impossible though.

[dglevy]

Frankly, these ratings are rather suspect. At best, they may help many, but they don't help power users like myself. What I want to see is the raw data, so I can weight the results according to my own needs. For example, the hard drive test results have no specific numbers under performance. All you give is a performance rating from 0 to 100. What good does that do me? If one drive is 74 and another is 100, all that tells me is that one is faster than the other, but not by how much. The difference could be huge or it could be insignificant.What's worse, the online performance results for hard drives has even less information than the magazine itself. At least, in the magazine, you give the raw numbers of the copy and search tests.If your tests are really reliable and scientific, then why don't you make the data available?

[AlanStafford]

We do provide breakout test results for many topics; for example, look at this page on printer tests:[url]http://www.pcworld.com/product/compare?origurl=%2Farticle%2Fid%2C126368-page%2C1%2Farticle.html&charttitle=Top10InkjetPrinters&chartdate=TueJan1716%3A00%3A00PST2006&prodid=29590&prodid=29594These are tested speeds, not vendor-supplied specs.However, I do see that our hard drive test results aren't broken out; I'll check into why that is.

[dglevy]

Thanks, Alan, for your quick reply. Hopefully your inquiry will yield some answers soon, since I have to buy two hard drives within the next couple of weeks!

[AlanStafford]

OK, good news: We fixed our hard drive charts to break out the results of individual tests:[url]http://www.pcworld.com/article/id,123680-page,1/article.html_However, while the compare function works for internal hard drives, it's not working yet for NAS drives. We're checking into it.Also, the external hard drives chart doesn't have the breakouts yet, because we're just about to publish a new chart. When it goes live--in about a week, I think--the results will be broken out.As always, please let me know about any concerns you have with our reviews or testing methodology.

[dglevy]

Thanks, Alan, for the quick change! Very impressive that PC World saw my point and acted so fast on it...It looks like your methodology changed recently, so one still can't compare easily between drives test several months ago and drives tested more recently. But I think there was enough data there for me to make my decision. I think the data is pretty strong in supporting the Samsung Spinpoint, performance-wise. And it seems that PC World is a bit ahead of the pack on spotting it. Anandtech and storagereview.com have not tested it yet, as far as I can tell.Sincerely,David LevyWashington, DC

[talee16]

Just curious. What is the BIOS version used for Gateway GM5632E in your test?

[nepper]

While I found the monitor reviews interesting, I was dissapointed that they were only reviewed at native resolution. At work a couple of years ago, we deployed hundreds of IBM 17" LCD displays which looked great at native resolution, but were atrocious at any other resolution. Native resolution was just too small for the vast majority of users in the company. As a consequence, our company returned all of them, and went with a different monitor.
So, the last monitor I bought, I compared several of them at multiple resolutions, and only found one that looked good at most.
BOTTOM LINE: In our experience, there is a significant percentage of people that don't use native resolution.
best regards,
Neale

[kschroeder]

Is WOrldBench 6 Beta still available for download anywhere other than BitTorrent sites? Every link I have found at PCWorld.com leads to a "404" page. We're ready to buy, but I don't want to get WB 5.0 if 6 is available with more up-to-date software/applications in it.

[Renerib]

How do you test the battery life of cameras?