Yahoo!, Google, and Microsoft all argued during Congressional hearings in February that despite their capitulations to Chinese government censorship demands, the Chinese user is still better off because they are there than if they pulled out of China. Evidence is mounting that while Google and Microsoft may have a leg to stand on in that regard, one is hard-pressed to find any meaningful evidence that the Chinese user gains any better access to information or stronger privacy by using Yahoo! than if he or she were to use a domestic Chinese competitor.
Reporters Without Borders reports that Yahoo! is the "clear worst offender" in search engine censorship tests. RSF has been test-searching politically sensitive words on Yahoo Chinese search, Google.cn, MSN Beta Chinese search and their Chinese competitor, Baidu, then comparing which services yield the greatest amount of censorship, and who censors least. I've been playing around with those search engines myself lately and I agree with RSF's conclusion:
The testing threw up significant variations in the level of filtering. While yahoo.cn censors results as strictly as baidu.cn, search engines google.cn and the beta version of msn.cn let through more information from sources that are not authorized by the authorities.
RSF conducted searches on all 4 engines using 6 politically sensitive keywords: “6-4” (4 June, date of the Tiananmen Square massacre), “Falungong”, "Tibet Independence”, “Democracy”, “Human rights” and “press freedom.” I've been testing a much longer list of Chinese-language search terms plus a list of URL's for something I'm working on which will hopefully get published in a month or so. I can't scoop that report due to contractual agreements but I think it is necessary now to offer some of my own views on RSF's analysis, which says:
While Microsoft has just said it does not operate censorship, Reporters Without Borders found that the Chinese version of its search engine displays similar results to those of google.cn, which admits to filtering its content. Searches using a "subversive" key word display on average 83% of pro-Beijing websites on google.cn, against 78% on msn.cn. By contrast, the same type of request on an uncensored search engine, like google.com, produces only 28% of pro-Beijing sources of information. However, Microsoft like Google appears not to filter content by blocking certain keywords but by refusing to include sites considered illegal by the authorities.
My own testing shows that pattern of results to be generally true... but not completely. The MSN service appears to be changing by the day. MSN does seem to block keywords - at least in some cases. For instance, last week when I did a search on MSN Chinese beta for "Tiananmen Massacre" I got a page full of results but also a message disclosing that the results had been filtered. Today when I tried again I got nothing, and no message admitting to filtering the results. Here is the screenshot (click to enlarge):
It would seem pretty hard to get such a clean non-result unless the keyword is filtered as well as all the politically sensitive URL's that MSN's employees can anticipate.
Now here is my Google.cn search on "Tiananmen Massacre":
Lots of results from human rights and dissident websites do not appear because those URL's have been delisted from Google.cn (as is very apparent if you compare this with the same search on Google.com). Even so, websites containing discussion of the Tiananmen massacre still manage to come through on Google.cn...and if you scroll down to the bottom of that same page you get the standard warning message that some results have been removed in accordance with local laws, regulations etc.:
RSF's analysis continues:
The press freedom organisation is particularly shocked by the scale of censorship on yahoo.cn. first because the search results on “subversive” key words are 97% pro-Beijing. It is therefore censoring more than its Chinese competitor Baidu. Above all, the organisation was able to show that requests using certain terms, such as 6-4 (4 June, date of the Tiananmen Square massacre), or "Tibet independence", temporarily blocked the search tool. If you type in one of these terms on the search tool, first you receive an error message.
This is consistent with my experience using politically sensitive keywords with Yahoo! China and Baidu: a comparison of 25 search terms and 25 URL's across all 4 search engines finds Yahoo! China and Baidu to censor at very similar levels, making it fair to claim that Yahoo! China gives the Chinese user no more information and sometimes even less than Baidu. Here is what I get when I search "Tiananmen Massacre" on Yahoo China:
Note that I get this result sitting at my computer here in Cambridge, MA, USA, using my Speakeasy DSL connection, without any use of proxies or any other means to access the web via a Chinese ISP. Same exact result, of course, when I do the same search on Baidu.
RSF then goes on to describe what happens when you do a new search:
If you then go back to make a new request, even with a neutral key word, yahoo.cn refuses to respond. It takes one hour before the service can be used again. This method is not used by any other foreign search tools; only Baidu uses the same technique.
It is important to point out something here for the sake of fairness. It appears at least from where I sit that the shut-down of further service is not being done by Yahoo! but at the ISP level in China. Because sitting here in the U.S., right after the browser error I got for searching "Tiananmen Massacre" I was still able to immediately conduct another search for "Taishi Village" (the site of political unrest last summer) and got results:
They are clearly very censored results, as the same search on regular Yahoo.com will attest, but results nonetheless. Baidu also came through, with what appeared from looking at the URL's to be censorship at a similar level.
This issue of who is responsible for the loss of service problem, however, does not detract from RSF's clear and in my view correct point, which is that the Chinese user is probably no better off with Yahoo! being in China than if it ceased operating there.
If Yahoo! is going to keep on like this, they might as well just change the name of their Chinese service to that of their Chinese partner Alibaba, which now runs Yahoo! China anyway. Then at least they'd be a bit more honest with Chinese users about what they really are.
More seriously though, the biggest concern is that according to my sources in the industry, Google and MSN are feeling a lot of pressure from Chinese authorities to be just like Yahoo! China. Employees of certain competitors do not hesitate to tattle on Google and MSN in order to gain competitive advantage. So the point is this: Unless companies band together and push back against Chinese government censorship pressure, it's going to become a very rapid race to the bottom.