China censorship: Yahoo, Google and Microsoft compared

Yahoo!, Google, and Microsoft all argued during Congressional hearings in February that despite their capitulations to Chinese government censorship demands, the Chinese user is still better off because they are there than if they pulled out of China. Evidence is mounting that while Google and Microsoft may have a leg to stand on in that regard, one is hard-pressed to find any meaningful evidence that the Chinese user gains any better access to information or stronger privacy by using Yahoo! than if he or she were to use a domestic Chinese competitor.

Reporters Without Borders reports that Yahoo! is the "clear worst offender" in search engine censorship tests. RSF has been test-searching politically sensitive words on Yahoo Chinese search, Google.cn, MSN Beta Chinese search and their Chinese competitor, Baidu, then comparing which services yield the greatest amount of censorship, and who censors least. I've been playing around with those search engines myself lately and I agree with RSF's conclusion:

The testing threw up significant variations in the level of filtering. While yahoo.cn censors results as strictly as baidu.cn, search engines google.cn and the beta version of msn.cn let through more information from sources that are not authorized by the authorities.

RSF conducted searches on all 4 engines using 6 politically sensitive keywords: “6-4” (4 June, date of the Tiananmen Square massacre), “Falungong”, "Tibet Independence”, “Democracy”, “Human rights” and “press freedom.”  I've been testing a much longer list of Chinese-language search terms plus a list of URL's for something I'm working on which will hopefully get published in a month or so. I can't scoop that report due to contractual agreements but I think it is necessary now to offer some of my own views on RSF's analysis, which says:

While Microsoft has just said it does not operate censorship, Reporters Without Borders found that the Chinese version of its search engine displays similar results to those of google.cn, which admits to filtering its content. Searches using a "subversive" key word display on average 83% of pro-Beijing websites on google.cn, against 78% on msn.cn. By contrast, the same type of request on an uncensored search engine, like google.com, produces only 28% of pro-Beijing sources of information. However, Microsoft like Google appears not to filter content by blocking certain keywords but by refusing to include sites considered illegal by the authorities.

My own testing shows that pattern of results to be generally true... but not completely. The MSN service appears to be changing by the day. MSN does seem to block keywords - at least in some cases. For instance, last week when I did a search on MSN Chinese beta for "Tiananmen Massacre" I got a page full of results but also a message disclosing that the results had been filtered. Today when I tried again I got nothing, and no message admitting to filtering the results. Here is the screenshot (click to enlarge):

It would seem pretty hard to get such a clean non-result unless the keyword is filtered as well as all the politically sensitive URL's that MSN's employees can anticipate.

Now here is my Google.cn search on "Tiananmen Massacre":

Lots of results from human rights and dissident websites do not appear because those URL's have been delisted from Google.cn (as is very apparent if you compare this with the same search on Google.com). Even so, websites containing discussion of the Tiananmen massacre still manage to come through on Google.cn...and if you scroll down to the bottom of that same page you get the standard warning message that some results have been removed in accordance with local laws, regulations etc.:

RSF's analysis continues:

The press freedom organisation is particularly shocked by the scale of censorship on yahoo.cn. first because the search results on “subversive” key words are 97% pro-Beijing. It is therefore censoring more than its Chinese competitor Baidu. Above all, the organisation was able to show that requests using certain terms, such as 6-4 (4 June, date of the Tiananmen Square massacre), or "Tibet independence", temporarily blocked the search tool. If you type in one of these terms on the search tool, first you receive an error message.

This is consistent with my experience using politically sensitive keywords with Yahoo! China and Baidu: a comparison of 25 search terms and 25 URL's across all 4 search engines finds Yahoo! China and Baidu to censor at very similar levels, making it fair to claim that Yahoo! China gives the Chinese user no more information and sometimes even less than Baidu. Here is what I get when I search "Tiananmen Massacre" on Yahoo China:

Note that I get this result sitting at my computer here in Cambridge, MA, USA, using my Speakeasy DSL connection, without any use of proxies or any other means to access the web via a Chinese ISP.  Same exact result, of course, when I do the same search on Baidu.

RSF then goes on to describe what happens when you do a new search:

If you then go back to make a new request, even with a neutral key word, yahoo.cn refuses to respond. It takes one hour before the service can be used again. This method is not used by any other foreign search tools; only Baidu uses the same technique.

It is important to point out something here for the sake of fairness. It appears at least from where I sit that the shut-down of further service is not being done by Yahoo! but at the ISP level in China. Because sitting here in the U.S., right after the browser error I got for searching "Tiananmen Massacre" I was still able to immediately conduct another search for "Taishi Village" (the site of political unrest last summer) and got results:

They are clearly very censored results, as the same search on regular Yahoo.com will attest, but results nonetheless. Baidu also came through, with what appeared from looking at the URL's to be censorship at a similar level.

This issue of who is responsible for the loss of service problem, however, does not detract from RSF's clear and in my view correct point, which is that the Chinese user is probably no better off with Yahoo! being in China than if it ceased operating there.

If Yahoo! is going to keep on like this, they might as well just change the name of their Chinese service to that of their Chinese partner Alibaba, which now runs Yahoo! China anyway. Then at least they'd be a bit more honest with Chinese users about what they really are.

More seriously though, the biggest concern is that according to my sources in the industry, Google and MSN are feeling a lot of pressure from Chinese authorities to be just like Yahoo! China. Employees of certain competitors do not hesitate to tattle on Google and MSN in order to gain competitive advantage. So the point is this: Unless companies band together and push back against Chinese government censorship pressure, it's going to become a very rapid race to the bottom.

Comments

[Fons Tuinstra]

I'm not sure whether these scores really mean anything. Last month Dutch blogs took the number of Google-hits as an indication for the online popularity of journalists. I checked my own name and scored more than a million hits, suggesting I would be the second most popular journalist in the Netherlands: something I would consider to be rather overstated. I have rechecked then every week, and my popularity went up to 1.8 million. Since then it started to go down, to less than 400,000 now. No clue how it works, but I would be very careful in using these figures.

[bobby fletcher]

Just to note a while back when rconversation, simonworld, sun-bin were discussing Taishi, I found plenty of info on Baidu and Yahoo.cn:

http://www.baidu.com/s?wd=%C2%C0%B0%EE%C1%D0&cl=3

http://search.cn.yahoo.com/search?stype=&p=%C2%C0%B0%EE%C1%D0&scch=on&ei=gb

http://www.baidu.com/s?ie=gb2312&bs=%CC%AB%CA%AF+%D1%E0%C4%CF+%B0%AC%CF%FE%C3%F7&sr=&z=&wd=%CC%AB%CA%AF+%D1%E0%C4%CF+%B0%AC%CF%FE%C3%F7&ct=0&cl=3&f=8

[antipasto]

On Slashdot, recently, they discussed how China is lobbying for what I see amounts to their political model as a WIFI protocol.

It won't happen, and unfortunately for them, both kinds of IP, internet procotol, and intellectual property, have been able to transcend larger walls.

Business and governments both rely on people, so... will the customer always be right? Or no? Which customer? The User, or the Government? Or is Yahoo actually China's customer?

How exactly does China expect this market to work? It's beyond belief.

[mahathir_fan]

I laughed out loud when I read this posting. Still can't get over the Tianment incident can't you?

I have seen that PBS documentary on the incident and there was no massaccre. The eyewitness accounts by those reporters were completely false and did not agree with the video footages. The guy said they shot the ambulance, but video clearly showed it was an auto accident and crowds weren't running away so there was no shot fired.

[Chris Langdon]

Google, Yahoo and MSN censor
their ads in America. I was not allowed to advertise my websites,
www.ncjusticefraud.com, and www.chinaisevil.com on Googol, Yahoo or MSN. Googol retaliated against me by de-listing my NC site altogether, while MSN consistently ranked it highly (#8-#14), when doing an appropariate search If you go to my China site and click on the page named "Googlegag" you can read all the lurid details...It has been reported in the Sydney Morning herald that Sergey Brin has admitted that Google's decision re Chinese censorship was a mistake. If so, why doesn't Google allow my ads?
Sincerely, Chris Langdon, in the Google Archipelago

[bobby fletcher]

Chris, I checked out your site, and IMHO Google et al are correct to not touch your site with a 10 foot pole.

China is no more, or less evil than America. I mean are they dropping depleted uranium dirty bombs like we are? Are they proliferating nuclear weapon like what we are doing with India?

[Keir]

I agree. I checked out the site and it'sonly one page going over how the Chinese have killed millions (the US infamously doesn't keep track of how many Iraqis it kills)using Mao and events 50+ years ago as if they are relevant today. I'm sympathetic to criticism of the CCP which I loathe, but to do so in a lazy way (boycott the US because the Republican Party in power today enslaved blacks as national policy) reminds me of Lennon's warning that if you just go around using "pictures of Chairman Mao, you ain't going to make it with anyone anyhow."

[Keir]

Actually, to write things like " I have no evidence that Red China is supporting the rebels, but it is highly probable, almost certain, that they are" without any evidence to attack a government is highly offensive; to do so to an individual would get you sued. To argue that it is probable that a capitalist (I would argue fascist) CCP would help create instability along its borders to aid rebels simply because they call themselves 'Maoist' is ludicrous.

[chris langdon]

Dear Commie Lovers,
All governments are evil, it's just that China's is much more evil than others. Bobby Fletcher should realize that China still reveres Mao, even though he killed between 40 to 70 million people. Even the Russians had enough sense to condemn Stalin. Why would China stir up trouble on its border? It's been doing just that ever since the Communists took over, by invading Korea, Tibet, India and Vietnam.
Also, China keeps the North Korean government afloat, helping them build nuclear weapons while a million N. Koreans have starved to death. What about that Bobby Fletcher? Why do so called "liberals" like to make excuses for murderers, simply because they kill in the name of
equality for the masses?

Chris Langdon

[ffxiv gil]

What about that Bobby Fletcher? Why do so called "liberals" like to make excuses for murderers, simply because they kill in the name of
equality for the masses?