123
 123

Tip: 看不到本站引用 Flickr 的图片? 下载 Firefox Access Flickr 插件 | AD: 订阅 DBA notes --

2008-09-05 Fri

15:32 New patches, new builds (22574 Bytes) » MySQL Performance Blog

We made new patches, improved previous and want to announce new builds for 5.0.62, 5.0.67 and 5.1.26 versions. One of biggest changes we separated releases of 5.0 into two branches.
First, just "-percona" release is more stable and contains only stable and proven on many installation patches. Second is "-percona-highperf" release, which contains experimental patches that can improve performance significantly. On this stage "percona-highperf" contains patches from Yasufumi Kinoshita (more info on Maximum performance of OLTP benchmark is not so scalable on multi-cpu.
Most important patch "split_buf_pool_mutex" to separate InnoDB buffer->pool mutex into several, and it allows to get 1.5x - 2.5x improvements in DBT2 benchmarks on 8cores boxes (benchmark results is coming and will be published)

For all three versions we added SHOW PATCHES command, which is based on Jeremy Cole's patch

SQL:
  1. mysql> SHOW patches;
  2. +--------------------------------------------------+--------------------------------------------------------+---------+----------------------------+---------+---------------------------------------+
  3. | File                                             | Name                                                   | Version | Author                     | License | Comment                               |
  4. +--------------------------------------------------+--------------------------------------------------------+---------+----------------------------+---------+---------------------------------------+
  5. | control_flush_and_merge_and_read.patch           | InnoDB patch TO control INSERT buffer AND flushing     | 1.0     | Yasufumi Kinoshita         | BSD     |                                       |
  6. | mysqld_safe_syslog.patch                         | Patch allows redirect output of error.log TO syslog-ng | 1.0     | Percona <info@percona.com> | GPL     | Ported FROM Debian                    |
  7. | microslow_innodb.patch                           | Extended statistics IN slow.log                        | 1.0     | Percona <info@percona.com> | GPL     |                                       |
  8. | split_buf_pool_mutex_fixed_optimistic_safe.patch | InnoDB patch TO fix buffer pool scalability            | 1.0     | Yasufumi Kinoshita         | BSD     |                                       |
  9. | control_io-threads.patch                         | InnoDB patch TO control count of IO threads            | 1.0     | Yasufumi Kinoshita         | BSD     |                                       |
  10. | userstats.patch                                  | SHOW USER/TABLE/INDEX statistics                       | 1.0     | Google                     | GPL     | Added INFORMATION_SCHEMA.*_STATISTICS |
  11. | show_patches.patch                               | SHOW PATCHES                                           | 1.0     | Jeremy Cole                | N/A     |                                       |
  12. +--------------------------------------------------+--------------------------------------------------------+---------+----------------------------+---------+---------------------------------------+

Next, we added tables USER_STATISTICS, INDEX_STATISTICS, TABLE_STATISTICS to INFORMATION_SCHEMA

SQL:
  1. mysql> SELECT * FROM information_schema.table_statistics;
  2. +-------------------------+-----------+--------------+----------------------+
  3. | TABLE_NAME              | ROWS_READ | ROWS_CHANGED | ROWS_CHANGED_INDEXES |
  4. +-------------------------+-----------+--------------+----------------------+
  5. | art86.forum86           |    469084 |           54 |                  162 |
  6. | mysql.db                |        36 |            0 |                    0 |
  7. | art57.link_out57        |  59336163 |        82549 |               577843 |
  8. | art116.author116        |   1921596 |         1581 |                 3162 |
  9. | art54.link_out54        |  36879450 |        23830 |               166810 |
  10. | art104.author104        |   1100203 |         2572 |                 5144 |
  11. | art107.forum107         |    820296 |           56 |                  168 |
  12. | art104.forum104         |    771121 |           63 |                  189 |
  13. | art111.article111       |  44100332 |        16411 |               164110 |
  14. | art93.thread_stats93    |    442440 |         9495 |                18990 |
  15. ...

For 5.0-percona-highperf release we added:

  • split_buf_pool_mutex_fixed_optimistic_safe.patch | InnoDB patch to fix buffer pool scalability
  • control_io-threads.patch | InnoDB patch to control count of IO threads
  • control_flush_and_merge_and_read.patch | InnoDB patch to control insert buffer and flushing

5.1 release
Contains build-in InnoDB plugin, so there is no more needs to download InnoDB separately, install it and run mysql with option --skip-innodb.
We think InnoDB plugin is stable enough to use it with MySQL 5.1 RC, at least stable to name whole MySQL-5.1-percona release as RC.

Along with previously mentioned patches, to 5.1 we added new column TIME_MS to
INFORMATION_SCHEMA.PROCESSLIST which contains time of query execution with microseconds granularity.

SQL:
  1. mysql> SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST;
  2. +----+------+-----------+------+---------+------+-----------+----------------------------------------------+---------+
  3. | ID | USER | HOST      | DB   | COMMAND | TIME | STATE     | INFO                                         | TIME_MS |
  4. +----+------+-----------+------+---------+------+-----------+----------------------------------------------+---------+
  5. 1 | root | localhost | NULL | Query   |    0 | executing | SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST |   0.607 |
  6. +----+------+-----------+------+---------+------+-----------+----------------------------------------------+---------+
  7. 1 row IN SET (0.00 sec)

On our release page
you can find links to RPMS for RedHat 4/5, binary tar.gz, source tar.gz (source contains source code with applied patches, so you can just download it and compile) and just raw patches.


Entry posted by Vadim | 2 comments

Add to: delicious | digg | reddit | netscape | Google Bookmarks

13:12 Friday round-up (2098 Bytes) » Red Hat Magazine

Here’s a few things we spotted on the web this week. As always, if you see something neat, let us know.

11:31 防火墙阻止PATCH压缩包解压 (476 Bytes) » yangtingkun
最近连续两次碰到了这个问题。在Windows环境中,利用RAR展开Oracle的补丁压缩文件时,会出现下面的错误。! f:p4505133_10105_winnt.zip: 无法打开 Disk1setup.exe 另一个程序正在使用此文件,进程无法访问。关闭瑞星的防火墙后,重新进行解压问题不在产生。一直在使用瑞星,但是最近才发现了这个问题,可能时瑞星处理EXE文件的策略发生了一些变化导致了这个问题吧。
09:49 Log Buffer #113: A Carnival of the Vanities for DBAs (8285 Bytes) » Pythian Group Blog » Log Buffer

This is the 113th edition of Log Buffer, the weekly review of database blogs.

Sheeri Cabral gets things going this week with her coverage of this rumour: Monty Widenius Leaves Sun/MySQL. If it turns out to be true, that seems like bad–or at least sad–news for MySQL, but Sheeri’s take is mostly an optimistic one.

MySQL’s Kaj Arno responds to the rumours on Monty resigning: “First, it’s a rumour.  . . .  Second, Monty’s resignation has been a possible outcome already since years before the Sun acquisition.  . . .  I can neither confirm nor deny the rumour.”

On CNET’s The Open Road, Matt Asay says. “Monty has done the right thing with his dissent. He has taken it outside the company, as Arjen Lentz, MySQL’s twenty-fifth employee, did before him. Arjen continues to be both a promoter and critic of MySQL, but is able to do so publicly without the constraints of an employee agreement. I assume Monty will do the same, and rightly so.” That’s not all the response out there, and there will certainly be still more as this develops.

In non-Monty blogs, Brook Johnson of Database Science asks, can a timestamp be slower than a datetime? (more…)

These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Slashdot
  • Google
  • del.icio.us
  • Facebook
  • bodytext
  • Technorati
  • TwitThis
  • Reddit
  • Spurl
  • De.lirio.us
  • Furl
  • blogmarks
  • Ma.gnolia
  • E-mail this story to a friend!
09:17 行为经济学读书笔记(1) (2114 Bytes) » 淘宝数据仓库团队

——感谢Deaton所作的文献综述。

最初的函数形式得益于Stone的研究,为孤立的单个商品建立对数需求函数:

log(qi) = αi + ei·log(x) + Σeik·log(pk)

这里ei是总支出弹性,eik是第k个价格对第i种需求的交叉价格弹性。因为观察数据的限制,必须把解释变量的数目保持在最低限度(自由度)。交叉价格弹性包含了收入效应和替代效应,Stone通过Slutsky方程将ei分解为ei = eik* - eiwk,

这里eik*是补偿交叉价格弹性,wk是预算份额。

将两式合并,用总价格指数的对数log(P)近似替代Σwk·log(pk),并加上齐次性限制Σeik*= 0得到

log(qi) = αi + ei·log(x/P) + Σeik*·log(pk/P)

这里Σ的范围被限制在相近的替代品和互补品(排除不相关商品之间的零替代)。

为了保存自由度,支出弹性ei首先是由预算研究来估计的。最后,为了解释口味的持续变化,Stone引入一个时间趋势,比如说θit,并取一阶差分以最小化残差的序列相关的效应:

Δ[log(qi) - ei·log(x/P)] =θi + Σeik*·Δlog(pk/P)

06:30 A new book on APEX coming out... (5657 Bytes) » The Tom Kyte Blog

There is a new APEX book coming out soon - I wrote a foreword for it.  There has been some discussion of it on asktom - and I thought I'd publish the foreword here:

I consider myself a pragmatic person – one that uses the right tools for a job, one that employs the most straightforward and easy way to accomplish a task.  To that end, I’ve been a great supporter and fan of Oracle’s Application Express (APEX) from before the day it was introduced.  I say “before the day” because I’ve had the honor and pleasure of using APEX long before it was released to the public at large – my website, http://asktom.oracle.com/, is one of the first websites ever built with the software that was to become known as APEX.

APEX is one of the most pragmatic database development tools I know of.  It does one thing and one thing well – it rapidly implements fully functional database applications – applications that are used to predominantly access, display and modify information stored in the database (you know, the important applications out there).  It facilitates using the database and it’s feature set to the fullest – allowing you to implement some rather complex applications, with as little work (code) as possible.  It is possible to build extremely scalable applications with a huge user base (http://metalink.oracle.com/ for example is built with APEX).  It is possible to build extremely functional applications, with seriously powerful user interfaces (APEX itself is written in APEX, proof of this).  It is easy to build applications rapidly, the current version of http://asktom.oracle.com was developed in a matter of days by two developers – in their spare time, it was not a full time job.

While it all sounds wonderful and easy so far, APEX is a rather sophisticated tool with many bits of functionality and a large degree of control over how the generated application will look and feel.  To fully utilize the power of APEX – one needs to have a guide, a mentor show them how to do so; very much akin to what I do with people regarding the Oracle database.

This book – Oracle Application Express – is that guide, the authors – Scott Spendolini and John Scott – are those mentors.  The book walks you through the steps you need to understand after you’ve installed and started using APEX, to go beyond the sample applications.  Covering diverse topics such as “Using the database features to full advantage” (one of my favorite topics) to “SQL Injection Attacks” – what they are and how to avoid them in APEX – to “Printing”; you’ll find many real world issues you will be faced with explained, demystified and solved in this book.

For example, Chapter 5 “Data Security” covers a wide breadth of topics regarding securing your database application.  There is a section on URL injection issues – what they are, how they are exploited, why you care about them and finally how to protect yourself from them.  There is a section on Session State Protection – following the same format –what it is, how it is exploited, why you care and finally how to protect yourself.  The same mentoring occurs with data level access where the authors introduce how to use Virtual Private Database, a core database feature – not really an APEX feature, to protect your data from unauthorized access.  Lastly, a critical application feature – Auditing – is discussed in depth using the “what it is, why it is, why you care and then how to do it” approach.  Whilst some of the content in this chapter is not specific to APEX, it is needed to give you a holistic view to building database applications – which is what this book is about.

This book covers not just the nitty gritty details of building a secure application, it covers all you need to build database applications with APEX.  When they are done with security, the authors move onto other necessary topics such as how to perform screen layout and application screen navigation, how to integrate reports and charts, how to integrate web services – enabling you to perform application integration – in an APEX environment, and much more.

If you are an APEX developer just starting out, or an APEX developer with experience under their belt but want to learn more about the environment you are using – this book is for you.  It describes from start to finish how to build a secure, functional, scalable application using the APEX application development environment.

Thomas Kyte
http://asktom.oracle.com/

05:11 VLDB 2008 proceedings, Oracle optimizer plan stability, adaptive cursor sharing and SecureFiles (1 Bytes) » Tanel Poder's blog: Core IT for geeks and pros
02:43 浪潮之巅 第十章 短暂的春秋——与机会失之交臂的公司(六) (10536 Bytes) » Fenng's shared items in Google Reader
发表者:Google(谷歌)研究员 吴军

第一节 太阳公司

1.1 昔日的辉煌

1.2 错失良机

1.3 历史的回放

第二节 Novell 公司

第三节 网景公司(Netscape)

第四节 Real Networks

当苹果公司的 iPod 以高科技精品的面貌上市,并风靡全球时,便有行家指出 iPod 其实并不是什么高科技新品,而是一个翻版的 MP3 播放器。MP3 音乐和播放器大家都很熟悉,它们在 iPod 出现前好几年就有了,而且最初搞 MP3 音乐的公司也不是苹果。

MP3 是当今在互联网上传播音乐的最通用的媒体格式。它的历史十分“悠久”,可以追述到 1979 年 AT&T 贝尔实验室搞的一些语音压缩算法。到了 1991 年,德国弗劳恩霍夫(Fraunhofer)应用研究所和 AT&T 贝尔实验室基于上述研究成果,提出了一种将高保真激光唱盘质量的音乐做成有损失压缩的音频压缩标准 Audio Layer 3,简称 MP3 。采用 MP3 数据格式的音乐质量比激光唱盘的质量要差一些,但是数据文件大小小了一个数量级。在互联网兴起以前,这种音频压缩方法并没有广泛应用。1995 年,弗劳恩霍夫基于 MP3 格式推出了在 Windows 操作系统下运行的世界上第一个 MP3 播放器 WinPlay3。当互联网兴起后,大家发现音乐经过压缩后,在互联网上传播成为可能。在 MP3 以后,又出现了各种各样的原理相同的音频文件压缩方法。

1995 年,微软的一位高管罗伯特•格雷斯从微软出来,创办了 Real Networks 公司。它一方面开发为互联网服务的通用音频和视频播放器,另一方面为媒体公司,比如 NBC 提供将节目放到互联网上的服务。该公司的播放器是跨平台的,能支持所有现有的音频和视频压缩格式,并且做到一边下载,一边播放(而不是下载完了再播放)。它还可以根据网路速度的快慢调整音频和视频质量(网路越快,质量越高)。

Real Networks 早期非常成功,创办的当年就推出了它的 1.0 播放器,并且在互联网上转播了NBA的篮球比赛。在接下来的两年里,Real Networks 每半年就推出一些新产品和服务。但是,到了 90 年代末,随着微软在播放器市场上地位的加强,Real Networks 的发展越来越受到制约。和微软的 IE 一样,其媒体播放器 Media Player 也是随 Windows 操作系统免费提供的。很快,微软的 Media Player 就超过了 RealPlayer,一举夺得市场占有率第一,这里面盗版的 Windows也为微软抢夺播放器的市场份额做出了“巨大的贡献”。有了免费的播放器,很少有人会花四十美元下载一个功能差不多的 RealPlayer 了(Real Networks 通常把它旧的播放器让用户免费下载,但是它的功能不全,它最新的播放器在美国总是要 $39.99,至今如此)。当然,由于专利的限制,有些格式的媒体不能用微软的 MediaPlayer 播放,必须用 RealPlayer 付费版播放,但是,如果哪家媒体公司选择了这种格式,观众和听众数必然少而又少,久而久之,这种不能通用的媒体格式便自然而然地趋于淘汰。到 2000 年,网络上绝大多数媒体都采用微软播放器可以播放的格式。这样,微软利用免费捆绑的 MediaPlayer,控制了广大用户计算机上的播放器,进而渐渐控制了互联网上的媒体文件格式。

微软从 1998 年起在播放器上挑战 Real Networks,到 2000 年前后便夺取了 RealPlayer 一半的市场份额(Nelson 和 Comsore 等第三方市场研究公司发布的播放器市场的占有率数据大相径庭,我们这里只能给出一个大致估计。),到了 2002 年,两家公司便平分秋色了。两年后,RealPlayer 的占有率只有微软 MediaPlayer 的一半不到。以后逐年减少,到去年占不到播放器市场的20%。

   &nbsp


Real Networks 源于微软又被微软打败。虽然它的境遇和网景公司有些相似,但是,两家公司失败的原因不尽相同。网景公司是被动地死守浏览器市场,但是在微软捆绑的打击下节节败退,最后无险可守,从此从人们的视线中消失了。Real Networks 也许因为是微软自己人办的公司,谙熟微软的竞争之道,所以一开始就不断在找退路,它在做播放器的同时,提供了多项基于互联网的广播和电视服务。Real Networks 通过向互联网用户收费订阅费(Subscription Fee)挣钱,同时在广播和电视中插播一些广告。除此之外,它还有音乐付费收听和视频付费收看等等。比如在网上听歌每月十三美元,网上玩游戏每月十美元。由于 Real Networks 比较早进入服务市场,同时服务市场又一直是微软的弱项,它在失去了播放器软件市场后,终于守住了网络音频和视频服务的市场。服务性收入在 Real Networks 的营业额中份额,最初是空白,但是逐年增加,去年 2007 年达到了 83%。(见下图)

   &nbsp


靠着付费服务的收入,Real Networks 挺过了从 2000 年到 2002年的互联网最艰难的时期,得以生存下来。但是 Real Networks 的营业额始终没有回到 2000 年的水平。华尔街对它已经很不看好。它的股价从 2000 年初的 90 多美元降到 2001 年 3 美元的最低点,虽然现在恢复到 6 美元,也不过是当年的零头而已。今天,RealPlayer 退出了一流公司的行列,但是,在互联网历史上,Real Networks 的贡献不可磨灭。它使得音乐在互联网上广泛传播(当然这里面 Napster 的贡献也很大)。在网络泡沫破碎前(2000年),音乐占整个互联网流量的第二位。

其实,Real Networks 有一个更好的办法来抵御微软的竞争,就是走后来苹果 iPod 的道路,推出自己的便携式媒体播放器。也许是因为 Real Networks 源于微软,不知不觉地采用了卖软件的商业模式,而没有想到做一个专门的类似 iPod 的消费电子产品。也许是因为当 Napster 打输了和唱片公司的官司后,Real Networks 看不到网上音乐市场的前景,便把精力集中在将新闻等电视节目搬到网络上。遗憾的是,美国广大观众至今仍然习惯于在大屏幕电视上看新闻而不是到计算机的小屏幕上看豆腐块大的视频。至于听歌,大家还是习惯将音乐下载到自己的 PC 机、iPod和手机上听,而不是通过互联网整天连到 Real Networks 的服务器上听。因此 Real Networks 的订户数量一直涨不上去,它的商业模式便如同一条死胡同,越走越窄。

Real Networks 在和微软竞争中幸存下来,已属不易。而它的前景却依然堪忧。首先,由于苹果 iPod 的热卖和它的免费的 iTunes 在 PC 机播放器市场上的强势地位,RealPlayer 的市场份额以更快的速度下降。早晚有一天 Real Networks 将不得不把它完整版的播放器免费提供出去。 iTunes 下载业务还将蚕食掉 Real Networks 已所剩不多的付费下载。其次,YouTube 的免费服务对 Real Networks 的付费订阅冲击很大(其实,Real Networks 自己成为 YouTube 并不难,但是要求 Real Networks 在 2000 年时推出一个类似 YouTube 的网站不太现实,因为那时候宽带上网并不普及。Real Networks 在开始时没有采用YouTube的在线服务方式,那么以后它就很难更改其商业模式了。我们在上一章提到,创业的大环境常常决定了一个公司的成败发展,Real Networks 和 YouTube 失之交臂就说明了这一点。)

Real Networks 作为早期互联网音乐、视频播放器和服务业的霸主,本来最有资格得到苹果现在在便携式媒体播放器的统治地位。如果 Real Networks 能看到并做到这一点,它和微软的竞争成败都无伤大局。但是,它错失了那次、也是唯一的一次不太容易把握的良机,从此便将领导音乐时尚潮流的使命拱手让给了苹果公司。这也许是一流的苹果公司和二流的 Real Networks 之间的差距。

结束语

从太阳公司到 Real Networks,都在短短的几年到十几年时间里走过了从无到有,再到顶点然后迅速衰落的过程。虽然它们最终败在了微软的手下,但是它们的衰退很大程度上是自己的原因( Novell 自身的原因少一些)。如果它们能找到真正适合自己发展的方向,它们就能在与微软的竞争中立于不败之地,或者竞争的成败甚至都不是很重要。这些公司对我们这个世界作出了巨大的贡献,它们改变了我们的生活方式。另一方面,这些曾经辉煌的公司薪尽火传。今天苹果公司的 iPod 和 iTunes 就有 Real Networks 播放器和在线下载的影子。今天 Google 的云计算在某种程度上实现了当年 Novell(和 Oracle)的网络 PC 。网景的浏览器已经是历史,但今天蓬勃发展起来的火狐(Firefox)可以说是它的翻版。历史不会简单的重复,失败是成功之母,IT 行业的精英们不仅没有再犯前人的错误,而且做得比前人更好。从这两个方面讲,我们都应该感谢这些只有短暂光辉的公司。
01:39 PL/SQL Naming (1 Bytes) » Oracle Scratchpad
01:22 READ ONLY account to Oracle database (564 Bytes) » 玉面飞龙的BLOG
看来数据库中一些read only的用户也可以捣乱。 read only的权限可以执行select for update,虽然没有权限执行实际的DML,但是可以lock表或行。 如果给恶意或者初级的数据库用户使用,也会给系统带来安全和稳定的隐患。 SQL> create user readonly_user identified by readonly_user; User created. SQL> grant create session to readonly_user; Grant succeeded. SQL> grant select any table to readonly_user; Grant succeeded. SQL> conn / Connected. SQL> select * from tab; TNAME [...]

2008-09-04 Thu

22:08 耐用品的难题 » 淘宝数据仓库团队
21:43 什么是逻辑 » Ricky's Blog on Testing and RAC
16:24 How quickly you should expect to see bugs fixed » MySQL Performance Blog
15:08 Video: oVirt, part 1 » Red Hat Magazine
13:46 三言二拍:“大”淘宝战略 » Fenng's shared items in Google Reader
12:02 9i远端导出报错ORA-12571 » yangtingkun
12:02 9i远端导出报错ORA-12571 » yangtingkun
08:13 B-Tree Index Internals: Rebuilding The Truth » 玉面飞龙的BLOG
07:45 闲话招行的安全 » Fenng's shared items in Google Reader
07:23 在 AIX 平台下编译安装 Sendmail » Fenng's shared items in Google Reader
07:07 香港的旺角 梦幻的城市 » Oracle Life
04:29 IBM将SSD纳入存储领域战略要点 » Fenng's shared items in Google Reader
02:24 Trouble-shooting » Oracle Scratchpad
02:00 在 AIX 5.3 和 6.1 中使用 Veritas Volume Manager (VxVM) V5 管理逻辑卷 » developerWorks 中国 : 技术文章 , 教程 AIX
02:00 在 AIX 平台下编译安装 Sendmail » developerWorks 中国 : 技术文章 , 教程 AIX
01:37 淘宝将与阿里妈妈合并 » Fenng's shared items in Google Reader
00:30 支付宝面向企业用户产品细化 » Fenng's shared items in Google Reader

2008-09-03 Wed

23:43 Google Chrome遭遇SPA崩溃 » OracleBlog.cn
23:27 闲话招行的安全 » Fenng's shared items in Google Reader
22:59 排版的规则 » Fenng's shared items in Google Reader
20:40 網站搬家的最佳處理方法(轉換網址) » Fenng's shared items in Google Reader
20:14 Janky, Debian Upgrades and You » Fenng's shared items in Google Reader
16:27 MySQLPlus: A Non-Blocking MySQL Driver for Ruby 1.8 and 1.9 » Fenng's shared items in Google Reader
14:09 ANALYZE: MyISAM vs Innodb » MySQL Performance Blog
07:00 Google Chrome! » Chanel [K]
06:41 MySQL and Drizzle Developers: Upgrade to Bazaar 1.6.1 Now » Fenng's shared items in Google Reader
00:29 Announcing the AWS Start-Up Challenge - Win $100,000 in Prizes » Fenng's shared items in Google Reader

2008-09-02 Tue

23:49 因为信任,所以简单 --专访支付宝架构师团队 (2) » Fenng's shared items in Google Reader
22:55 因为信任,所以简单 --专访支付宝架构师团队 (2) » Fenng's shared items in Google Reader