常见中文分词开源项目

中文分词和TF-IDF-月光博客

　　中文分词(Chinese Word Segmentation)指的是将一个汉字序列切分成一个一个单独的词。中文分词是文本挖掘的基础，对于输入的一段中文，成功的进行中文分词，可以达到电脑自动识别语句含义的效果。

　　TF-IDF（term frequency–inverse document frequency）是一种用于信息搜索和信息挖掘的常用加权技术。在搜索、文献分类和其他相关领域有广泛的应用。

　　TF-IDF的主要思想是，如果某个词或短语在一篇文章中出现的频率TF高，并且在其他文章中很少出现，则认为此词或者短语具有很好的类别区分能力，适合用来分类。TF词频(Term Frequency)指的是某一个给定的词语在该文件中出现的次数。IDF反文档频率(Inverse Document Frequency)的主要思想是：如果包含词条的文档越少，IDF越大，则说明词条具有很好的类别区分能力。

　　使用TF*IDF可以计算某个关键字在某篇文章里面的重要性，因而识别这篇文章的主要含义，实现计算机读懂文章的功能。

　　常见中文分词开源项目：

　　SCWS：

　　Hightman开发的一套基于词频词典的机械中文分词引擎，它能将一整段的汉字基本正确的切分成词。采用的是采集的词频词典，并辅以一定的专有名称，人名，地名，数字年代等规则识别来达到基本分词，经小范围测试大概准确率在 90% ~ 95% 之间，已能基本满足一些小型搜索引擎、关键字提取等场合运用。45Kb左右的文本切词时间是0.026秒， 换算完毕大概是 1.5MB文本/秒。支持PHP4he PHP 5。

　　ICTCLAS：　　

　　这可是最早的中文开源分词项目之一，ICTCLAS在国内973专家组组织的评测中活动获得了第一名，在第一届国际中文处理研究机构SigHan组织的评测中都获得了多项第一名。ICTCLAS3.0分词速度单机996KB/s，分词精度98.45%，API不超过200KB，各种词典数据压缩后不到3M.ICTCLAS全部采用C/C++编写，支持Linux、FreeBSD及Windows系列操作系统，支持C/C++、C#、Delphi、Java等主流的开发语言。

　　庖丁解牛分词：

　　Java 提供lucence 接口，仅支持Java语言。

　　CC-CEDICT：

　　一个中文词典开源项目，提供一份以汉语拼音为中文辅助的汉英辞典，截至2009年2月8日，已收录82712个单词。其词典可以用于中文分词使用，而且不存在版权问题。Chrome中文版就是使用的这个词典进行中文分词的。

说真话说假话？

令人"心神不宁"的网络-月光博客

　　著名学者钱理群先生曾在一次讲座上说，人说话应该有底线，这些底线依此是：一，力图说真话；二，不能说真话则应保持沉默；三，无权保持沉默而不得不说假话时则不应伤害他人。钱先生补充说，无权保持沉默而不得不说假话，若这种假话并未伤害他人，这种人可成为奴隶，尚有可同情处；但若为伤害他人而说假话，则只能称奴才，不可原谅。

How To Run Windows Apps In Linux

How To Run Windows Apps In Linux - Washingtonpost.com

Keir Thomas, PC World
PC World
Saturday, June 20, 2009 12:19 AM

Wine is an open source project that, on the face of it, seems to offer something wondrous: the ability to run Windows applications under Linux (or any other open source OS). It does this by attempting to recreate the Windows API layer in open source.

Finally! A chance to run those Windows programs for which there are no open source analogs: Quicken, or Flash CS. And all those games!

Sadly, the reality is a little more complicated. Yes, Wine is pretty good at running some Windows software and games. But to get the most out of it, you need to hack. Often, getting Wine to deliver is a black art, and not for novices.

Luckily, there are several ready-made installation wizards that help get around problems. These are scripts written by people who have done the hacking for you. There are four projects in particular that are worthy of mention: CrossOver, Cedega, Wine-Doors, and PlayOnLinux.

CrossOver comes in Game and standard versions, and is a proprietary and commercial project created by the same developers behind Wine. As such, it's simply the best choice for a fuss-free life. It'll cost you, but some of the money goes into sponsoring Wine developers.

Cedega is another proprietary and commercial project, this time primarily for gamers. By most accounts, it's the best choice if you simply want to play games, and the Cedega developers hack the Wine code significantly for better results. It will cost you, however, and works on a subscription basis rather than one-off licensing fee.

Wine-Doors and PlayOnLinux are both open source, and free-of-charge, so you may as well try each to find which suits your needs best. I take a quick look at both below. These instructions are for Ubuntu 9.04.

Preparation

Before installing any Wine application, it's a good idea to install the Microsoft Core Web fonts. These are the usual font suspects in a typical Windows install (Arial, Times New Roman etc.) and many Windows applications expect to find them in place. Things will probably look odd without them.

You can install them by either copying the .ttf files from a Windows installation, or by installing the msttcorefonts package. Fonts that are not supplied as part of this package, but which some applications might need, include Tahoma and MS Sans. You can find these around the web if you search. You can install them by creating a folder called .fonts in your/home directory and placing them there.

If you want to import directly from a Windows installation, you'll find the fonts in the C:\windows\fonts directory. Copy all the .ttf files to a USB stick, create a new folder in your/home directory called .fonts (note the period before the word fonts!), and place them there. And that's it. The fonts will now be available in all your applications.

PlayOnLinux

To install PlayOnLinux, you'll need to add a new repository and install a software package from it. Open a terminal window (Applications, Accessories, Terminal) and type the following:

Once the program has installed, you'll find it on the Applications, Games menu (it's worth mentioning that PlayOnLinux is ? as its name suggests--primarily designed to allow the installation of Windows games, but it also features scripts for many popular applications).

When the program is run for the first time, you'll have to complete a setup wizard. Click Forward to work your way through. This will download the latest versions of the install scripts from the PlayOnLinux server.

Once things have finished, click the Install button on the toolbar. In the Window that appears, click a category on the left relating to the type of software you want to install, and then select the application on the right. Most free-of-charge applications, like Internet Explorer, can be downloaded on demand, but you'll need the install CD for commercial applications.

Once you've made your choice, click the Apply button. The wizard will then start and walk you through installation, including making any necessary downloads. Bear in mind that, eventually, the program's own installation wizard may start, and you'll have to work through this as usual too. Don't close the PlayOnLinux's wizard, however! Leave it open until it finishes of its own accord.

Often a specific older version of Wine, known to work with the application, will need to be downloaded. This will happen automatically.

Once installed, the Windows program will be found on the Applications> Wine menu. Note that there's a slight issue with Ubuntu 9.04 (Jaunty) whereby desktop shortcuts created by third-party applications don't work. To fix this, you need to make them executable. This can be done by using the chmod command. For example, the following will fix a desktop shortcut for the Safari Web browser once it's been installed:

All desktop shortcuts created in this way have a .desktop file extension, and this should help you identify them.

***safari.jpg

Wine-Doors

To install Wine-Doors, you can once again add a new software repository. Open System> Administration> Software sources, and click the Third Party Tab in the window that appears. Click the Add button and type (or cut and paste) the following line:

Click the Close button and refresh the repository listings when prompted. Then use Synaptic to install the wine-doors package.

(Note that you can install the GPG key if you wish, which will avoid any harmless errors about unsigned packages.)

Once Wine-Doors has installed, you'll find it on the Applications> Wine menu, which is where you'll also find any Windows applications you install in future.

When Wine-Doors starts for the first time, you'll need to fill in your name and company details. These will be used when installing Windows applications. You can optionally tick the ?I Have a Windows License? button. This is needed in order to install certain Windows system components that facilitate the smooth running of Windows applications, as well as certain Windows add-ons whose EULAs demand a Windows license. Obviously, you shouldn't tick this box if you don't actually have a Windows license (i.e. a Windows installation on one of your computers).

When you click the Proceed button, Wine-Doors will update itself with the latest list of installation scripts. It's worth mentioning that hosting these scripts isn't cheap, and recently they had to be taken offline because of a funds shortfall

Once it's done, you'll be shown a list of applications and games. Simply click the install link alongside any in the list you wish to install, and then click the Apply button. Installation will then proceed, including the download of any necessary add-on software.

With any Wine installation, you'll find your applications in the directory of your folder (i.e. ). This is effectively a recreation of the Windows file system.

China Disables Some Google Functions

China Disables Some Google Functions - NYTimes.com

By EDWARD WONG

Published: June 19, 2009

BEIJING — The Chinese government disabled some search functions on the Chinese-language Web site of Google Friday, saying the site was linking too often to pornographic and vulgar content. Government officials met with managers of the Chinese operations of Google on Thursday afternoon to warn them that the company would be punished if the Web site was not purged of the offending material, according to a report Friday by Xinhua, the state news agency.

Earlier on Thursday, a government-supported Internet watchdog group, the China Internet Illegal Information Reporting Center, had criticized the search engine for its erotic content and threatened punishment from the government. The group had said that Google had already been warned twice, in January and April, about its content.

On Friday evening, it appeared that the associative-word feature of the Web site had been disabled. That is the function that displays a drop-down menu of words related to a search word that is typed into the search engine. The previous evening, reporters on China Central Television, the state television network, showed how typing in the Chinese word for son, erzi, could pull up associated terms that have lewd connotations.

State news organizations also reported that the ability to use Google's Chinese site to search overseas Web sites was supposed to be disabled, but that feature was still working on Friday evening.

Google released a statement saying it was making greater efforts to clean up its Chinese Web site. "We have been continually working to deal with pornographic content, and material that is harmful to children, on the Web in China," the statement said.

Though Google dominates the search engine market in the United States, it is struggling here to unseat Baidu, which has long been the most popular Chinese search engine.

Recent efforts by the Chinese government to limit access to the Internet have been met with outrage by Chinese computer users. The strongest reaction has been to the government's plan to force computer makers to install Internet censorship software on all computers sold in China after July 1. Critics say the software, called Green Dam-Youth Escort, could be used to censor Web sites with content deemed politically unacceptable, even though the government says the main use of the software will be to block access to pornography.

Computer experts also say the software can make a computer vulnerable to infiltration by hackers. This week, developers said they had found fixes to the problems. But on Friday, J. Alex Halderman, a computer science professor at the University of Michigan, said that a patched version of Green Dam had a security problem that was just as serious as the original one.

In a paper posted on the Internet, Mr. Halderman said it had taken him and his research team only one hour to find the new problem.

"This suggests that the security problems in Green Dam will be harder to fix than the government has suggested," he said in an e-mail message. "It's probably going to be impossible to make the software safe enough ahead of the July 1 deadline."

Andrew Jacobs contributed reporting, and Huang Yuanxi contributed research.

How to Install Tuxonice in Ubuntu

How to Install Tuxonice in Ubuntu | Ubuntu Geek

"TuxOnIce is most easily described as the Linux equivalentof Windows' hibernate functionality, but better. It saves the contents of memory to disk and powers down. When the computer is started up again, it reloads the contents and the user can continue from where they left off. No documents need to be reloaded or applications reopened and the process is much faster than a normal shutdown and start up." – tuxonice.net

Tuxonice is an alternative way to hibernate your machine than the standard ubuntu power manager way. It generally claims to be more configurable and flexible than the standard method, with faster hibernate and resume times.

How to Fix a Running Toilet

How to Fix a Running Toilet - wikiHow

Replace the flapper. If the toilet stops filling and then starts again intermittently, you have a slow leak. Try this to be certain. Place a dye tablet or a few drops of food coloring in the tank. Your local hardware store may have free dye tablets for this purpose. If, after an hour or two without flushing, you see this dye in the bowl, you have a slow leak, a small amount of water running into the bowl. The most common cause of slow leaks is a leaky flapper. Over time, this inexpensive rubber part decays, or minerals build up on it. Cleaning may work to remove mineral buildup, but it's usually best just to replace the whole part. There are a few standard kinds. Take your old one with you to the hardware store if you want the comparison to get the right size.

Close this valve by turning it clockwise, as you would a faucet.

• To perform the replacement, simply close the valve sticking out of the wall under your toilet tank.
• Flush the toilet. If the valve is completely closed, the tank will not refill and you will not hear water running after the tank empties.
• Pop the old flapper off its hinges, disconnect it from the chain, and pop the new one into place.
• Run a finger carefully around the rim where the flapper seats. Remove any uneven buildup of minerals that might cause a leak.
• Don't forget to open the valve all the way when you're ready for water again.
• Try flushing a few times to make sure the chain is the right length for the new flapper. It should open when you push the handle and then drop closed all the way when the tank empties. You may have to trim and adjust the chain by trial and error. Also, make sure that the flapper aligns properly with the opening.

A Gateway to Your Local Newspaper

A Gateway to Your Local Newspaper

Pick a U.S. state, pick a Canadian city, pick an international country or city and find the local newspaper home page. Great way to keep informed of news in small towns as well as large cities around the world.