Tip: 看不到本站引用 Flickr 的图片? 下载 Firefox Access Flickr 插件 | AD: 订阅 DBA notes -- ![]()
2010-03-06 Sat
Upgrading to 11g Release Grid Infrastructure? You probably want to read on…
Oracle 11g Release 2 Grid Infrastructure has been dramatically redesigned compare to 10g and 11gR1 Clusterware. Coming with impressive set of new features, Grid Infrastructure also uses much more memory. While RAM is rather inexpensive these days, it does pose an inconvenience in some scenarios. Particularly, for sand-box type installations that I use all the time for my own tests and demonstrations. For production upgrades, you need to be aware of and plan for increased memory usage.
I’ve been able to easily run a 2 node 10g RAC cluster on my MacBook with 4 GB of RAM allocating less than 1 GB of RAM to each virtual machine. That was even enough for a mini database instance with a very small memory footprint. Oracle 11g Release 1 was pretty much the same except maybe the database instance itself required a bit more memory but one node could still fit within 1 GB of RAM.
In 11gR2, bare-bone Grid Infrastructure processes alone consume 10+ times more memory (11.2.0.1 on 32 bit Linux to be precise):
[gorby@cheese1 ~]$ ps -eo pid,%mem,rss,user,cmd --sort=rsz --cols 100 | grep -e '^ *PID' -e grid -e ohasd | grep -v grep PID %MEM RSS USER CMD 3614 0.0 1080 root /bin/sh /etc/init.d/init.ohasd run 4322 0.2 3368 oracle /nfs/11.2.0/grid/opmn/bin/ons -d 4323 0.4 5164 oracle /nfs/11.2.0/grid/opmn/bin/ons -d 4117 0.6 7860 root /nfs/11.2.0/grid/bin/oclskd.bin 3830 0.6 8788 oracle /nfs/11.2.0/grid/bin/gipcd.bin 5048 0.7 8992 oracle /nfs/11.2.0/grid/bin/tnslsnr LISTENER -inherit 4167 0.7 10052 oracle /nfs/11.2.0/grid/bin/evmlogger.bin -o /nfs/11.2.0/grid/evm/log/evmlogger.i 3969 0.9 12412 oracle /nfs/11.2.0/grid/bin/diskmon.bin -d -f 3860 0.9 12736 oracle /nfs/11.2.0/grid/bin/mdnsd.bin 4067 1.1 14648 root /nfs/11.2.0/grid/bin/octssd.bin reboot 5016 1.2 15860 root /nfs/11.2.0/grid/bin/orarootagent.bin 3956 1.3 16964 root /nfs/11.2.0/grid/bin/orarootagent.bin 4292 1.4 17984 oracle /nfs/11.2.0/grid/bin/oraagent.bin 3874 1.5 20112 oracle /nfs/11.2.0/grid/bin/gpnpd.bin 3817 1.5 20300 oracle /nfs/11.2.0/grid/bin/oraagent.bin 4083 1.8 23700 oracle /nfs/11.2.0/grid/bin/evmd.bin 4372 2.4 31548 oracle /nfs/11.2.0/grid/jdk/jre//bin/java -Doracle.supercluster.cluster.server=eo 3564 3.2 41532 root /nfs/11.2.0/grid/bin/ohasd.bin reboot 4081 3.5 44932 root /nfs/11.2.0/grid/bin/crsd.bin reboot 3906 18.6 239428 root /nfs/11.2.0/grid/bin/cssdagent 3887 18.6 239444 root /nfs/11.2.0/grid/bin/cssdmonitor 3924 20.1 258564 oracle /nfs/11.2.0/grid/bin/ocssd.bin
The second column above gives you amount of resident memory in KB for processes related to Grid Infrastructure. As you can cleanly see, processes of CSS components consume well above 700MB! In total we can account for 1 GB. (those calculations are flawed — see below)
Compare that with 10g (10.2.0.3 on 32 bit Linux) — bare-bone Clusterware processes consume only 60MB:
[oracle@lh1 ~]$ ps -eo pid,%mem,rss,user,cmd --sort=rsz --cols 100 | grep -e '^ *PID' -e nfs -e crs -e css -e evm | grep -v grep PID %MEM RSS USER CMD 6524 0.0 348 oracle /nfs1/oracle/oracle/product/10.2.0/crs/opmn/bin/ons -d 4892 0.1 992 oracle /bin/sh -c cd /nfs1/oracle/oracle/product/10.2.0/crs/log/lh1/cssd/oclsomon; 3262 0.1 1072 root /bin/sh /etc/init.d/init.evmd run 3506 0.1 1100 root /bin/sh /etc/init.d/init.crsd run 4575 0.1 1116 root /bin/su -l oracle -c sh -c 'ulimit -c unlimited; cd /nfs1/oracle/oracle/pro 4890 0.1 1120 root /bin/su -l oracle -c /bin/sh -c 'cd /nfs1/oracle/oracle/product/10.2.0/crs/ 4664 0.1 1180 root /bin/sh /etc/init.d/init.cssd oclsomon 3263 0.1 1188 root /bin/sh /etc/init.d/init.cssd fatal 4677 0.1 1188 root /bin/sh /etc/init.d/init.cssd daemon 6525 0.5 4792 oracle /nfs1/oracle/oracle/product/10.2.0/crs/opmn/bin/ons -d 4922 0.6 5224 oracle /nfs1/oracle/oracle/product/10.2.0/crs/bin/oclsomon.bin 5915 0.7 6280 oracle /nfs1/oracle/oracle/product/10.2.0/crs/bin/evmlogger.bin -o /nfs1/oracle/or 4576 1.1 9312 oracle /nfs1/oracle/oracle/product/10.2.0/crs/bin/evmd.bin 5018 1.1 9428 oracle /nfs1/oracle/oracle/product/10.2.0/crs/bin/ocssd.bin 4606 2.0 16712 root /nfs1/oracle/oracle/product/10.2.0/crs/bin/crsd.bin reboot
The memory usage above is a bit overstated. There are some shared memory accounted multiple times. I could use Smaps interface to get better per process statistics. For example, you could see that 3 of the “top offenders” (CSS binaries) have about 40MB of shared libraries each:
[root@cheese1 ~]# ./smaps.pl 3924 | head
VMSIZE: 258576 kb
RSS: 258564 kb total
39164 kb shared
5180 kb private clean
214220 kb private dirty
PRIVATE MAPPINGS
vmsize rss clean rss dirty file
15052 kb 0 kb 15052 kb
12016 kb 0 kb 12016 kb
11184 kb 0 kb 11184 kb
[root@cheese1 ~]# ./smaps.pl 3887 | head
VMSIZE: 239456 kb
RSS: 239444 kb total
40096 kb shared
6200 kb private clean
193148 kb private dirty
PRIVATE MAPPINGS
vmsize rss clean rss dirty file
14624 kb 0 kb 14624 kb
10240 kb 0 kb 10240 kb
10240 kb 0 kb 10240 kb
[root@cheese1 ~]# ./smaps.pl 3906 | head
VMSIZE: 239440 kb
RSS: 239428 kb total
40096 kb shared
6200 kb private clean
193132 kb private dirty
PRIVATE MAPPINGS
vmsize rss clean rss dirty file
14624 kb 0 kb 14624 kb
10240 kb 0 kb 10240 kb
10240 kb 0 kb 10240 kb
[root@cheese1 ~]#
One way to get a practical number is to check system memory usage with and without Grid Infrastructure running — the difference is about 750MB (see the “free” column of the second row).
[root@cheese1 ~]# free
total used free shared buffers cached
Mem: 1283040 1131584 151456 0 18504 295668
-/+ buffers/cache: 817412 465628
Swap: 655328 76 655252
[root@cheese1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cheese1'
...
...
CRS-4133: Oracle High Availability Services has been stopped.
[root@cheese1 ~]# free
total used free shared buffers cached
Mem: 1283040 397144 885896 0 18640 316632
-/+ buffers/cache: 61872 1221168
Swap: 655328 76 655252
[root@cheese1 ~]# ps -eo pid,%mem,rss,user,cmd --sort=rsz --cols 100 | grep -e '^ *PID' -e grid -e ohasd | grep -v grep
PID %MEM RSS USER CMD
3614 0.0 1084 root /bin/sh /etc/init.d/init.ohasd run
I don’t have 11gR1 test cluster handy so I can’t check 100% but Oracle 11g Release 1 Clusterware is not much different from 10g so memory usage must be similar.
The lesson is that if you are upgrading your Oracle RAC Cluster to 11gR2 from 10g or 11gR1, then you have to account for additional 700MB memory for Grid Infrastructure alone on each node. Note that, this doesn’t take into account higher memory usage of the database instances themselves.
在过去的6年间,我在北京做过不少公开的活动,将我有限的经验与知识分享出去,这些活动每次都有很多新的面孔出现,有些成为我的朋友,有些真的就踏上了DBA之路,这些积极的促进是我一直乐于去做这样活动的主要原因。
可是做一点"公益"的事情并不容易,直到最近,当我觉得可以解决这其中的所有问题之后,这一切成为可能。
ACOUG 正是基于这样一件事情我们发起的一个站点,ACOUG的含义是All China Oracle User Group,这个名字太过大了,存粹是为了定义这样一个域名而得来的,所以请不要计较我们最终能够涉及到多少产品与内容,我们只是想尽力去做一件想了很久而一直没有做的事情。
所以要做一个用户组,是因为在几次去美国期间,在OOW上见到各个国家或地区的Oracle用户组组织了非常多的活动,而国内最活跃的是论坛,ITPUB.NET是首屈一指的,但是国内没有类似国外的用户组的组织。CNOUG也是一个非常不错的地方,但是CNOUG实际上是另外一个ITPUB,以论坛为主的社区,而社区也正是一个中国的特色。
ACOUG,其实开始我想也许成为BJOUG可能会更为确切,因为我们的初始活动可能只能在北京开始尝试。只是后来借鉴了一个印度的新闻组域名,得来了ACOUG的名字。
ACOUG,首先我们并不会做论坛,ITPUB在Oracle技术方面,在国内已经无可超越,技术问题的探讨那里仍然是最为合适的地方,我们想做的是一些地面活动,一直以来,我想搭建一个平台,那些热爱技术的、有一技之长的朋友,可以到这个平台上来演讲、分享、交流,我们通过这个平台提供一个分享和展现的机会,如果你乐于分享,并且享受过程,那这里就是你的舞台;而对于那些刚刚开始学习Oracle技术,又想接触和深入了解一下这个圈子的朋友,在这个平台下,我们为你准备好了位置,你可以来聆听和探讨。
ACOUG,这个站点的存在意义,最终我们设计将这个站点做成一个"技术博客",我们希望邀请那些国内数据库领域的爱好者们加入,会有一些简单的规则,大家可以定期提供一些自己深入研究的技术文章,将这个站点作为一个统一的展现平台聚合起来,这个作用和国外的很多站点类似。这样的一个想法,没有什么新奇之处,我们只是希望,能够在朋友们的支持下,做一点新的事情、做一点有益的事情,基于爱好,基于那一直存在的理想主义。
这个想法,在年初和Kamus提起时,两个人达成了惊人的一致,就连想到的ACOUG的名字都是同样的,然后Kamus尝试用不同的程序搭建了网站的后台,最后或者说暂时我们选择了WP。Kamus在这件事情上投入了很多精力,感谢他!
搭建这个平台,我想,其实北京有众多的技术高手,如果每个人都能够分享一个主题,那么每两个月搞一次活动,我想应该是不难的,而以前一直存在的场地、费用问题,现在有我的恩墨科技做一点支持,靠朋友们的一点赞助,以及来自Oracle官方的一些支持,这些问题不再存在。
有时候我们不去做一件事情的借口是忙,事实上不是,我想即便我自己每个月抽出一两天,花费1~2小时去分享一个技术主题是非常容易的,唯一的借口是我们的懒惰。另外一个问题是,很多人不愿意为没有收益的事情付出,而我的观点是,有付出一定是有回报的,虽然这个回报可能在很久之后。而我愿意做这件事,更主要的是如前所述,在赤裸裸的为了追逐金钱而努力之外,我们还一直保有一点理想主义,我相信大家都有。如果缺少一个组织者,那我们愿意来试试看。
在我们这个消息传出去之后,我在美国认识的一位ACE Director - Ameed Taylor 已经发来邮件说愿意到现场来支持我们的活动,我说给点时间我们尝试一下,如果ACOUG能够发展下去,我希望有机会请更多的人来北京,或者在外地开展活动。
最后,我想,也许会有人质疑我们关于"公益"的动机,那么是的,我心里面窃窃的以为,如果我们搞的成功,那么也许我们这个舞台可以让一些具有真正技术的人凸显出来,所以乐于分享的人,也许会有些意想不到的收获,当然这些活动的组织者,也许有机会在更广泛的领域为业者所知,我说的有付出就一定有回报,某种程度上与此有关。而如果组织有价值,活动有价值,其他的交给时间吧。
在我的设想里,以后,我不是主要的演讲者了,越来越多的年轻人、技术专家涌现出来,如果你们愿意,这里有你的位置,当然我们目前还无法给出报酬。
第一次活动我们定在了3月20日,时间有点紧张,我的意思是,给我们一点压力,让这件事情尽快动起来,有开始才有未来,我跟Kamus说,行动有时候比思想更重要。
好了,这个活动,对于参与者,也许唯一的要求是一个详尽的注册流程,我们希望,我们的每一次活动所抵达的地方和人群都能够被记录下来,这是对我们这些尝试的鼓励和纪念。
关于ACOUG的网站,由于我们都不是专业的人士,如果有朋友碰巧懂得PHP或WP运用得好,我们诚挚的邀请志愿者加入这个站点的维护工作,谢谢。
就写这么多,让我们动起来吧!
-The End-
相关文章|Related Articles
- 2009 北京 - 继续暂住在我自己的房子里
- 我的个人Eygle新闻组 - 欢迎读者、朋友加入
- 结束《深入解析Oracle》技术访谈交流会
- 北飞的候鸟-Kamus北京庭审现场
- 一场即将到来的费解官司-声援Kamus
评论数量(7)|Add Comments
本文网址:http://www.eygle.com/archives/2010/03/acoug_starting.html
有感于国外Oracle用户组的发达,我跟eygle从今年开始尝试组建中国的Oracle用户组,All China Oracle User Group。我们的站点是acoug.org。
我们希望能够为全中国的Oracle技术爱好者提供一个交流与活动的平台,我们希望能够激发更多人对于Oracle技术以及相关知识的兴趣,我们希望沉浸在这些技术中的人员获得更多的乐趣。这是我跟eygle一直以来的一个理想,但愿我们能为此付出并且有所回报。
我们认识到中国目前掌握Oracle技术的人员广度和深度都跟美国、欧洲甚或是印度存在着较大差距,这需要我们更有热情的去付出,但愿在不久的将来ACOUG在全球Oracle用户组中占据一席之地。
ACOUG计划定期举行线下活动,而线下活动也将是ACOUG未来工作的重点,邀请著名的业内人士(不仅限于国内)来进行主题演讲,并围绕相关主题进行广泛的讨论从而使更多人获益。
eygle的文章有关于ACOUG更多的阐述,请猛击这里。
预计在这个月的3月20日(周六)开展ACOUG第一次线下活动,详细的地点、场地以及演讲主题还未确定,但是请有兴趣的朋友时刻关注ACOUG网站,欢迎参加。
2010-03-05 Fri
看到Kamus对SQLULDR2的留言后, 破有感触. 人们应当比较关注, 他们想要的功能用起来方便是否, 关键并不在于功能的多少. 而SQLULDR2的众多的命令行选项, 也确实有些让人发晕, 包括我自已.
为了方便大多数人使用, 简化了SQLULDR2的命令行帮助, 简化到如下所示.
SQL*UnLoader: Fast Oracle Text Unloader (GZIP), Release 3.0.1
(@) Copyright Lou Fangxin (AnySQL.net) 2004 - 2010, all rights reserved.
Usage: SQLULDR2 keyword=value [,keyword=value,...]
Valid Keywords:
user = username/password@tnsname
sql = SQL file name
query = select statement
field = separator string between fields
record = separator string between records
read = set DB_FILE_MULTIBLOCK_READ_COUNT at session level
sort = set SORT_AREA_SIZE at session level (UNIT:MB)
hash = set HASH_AREA_SIZE at session level (UNIT:MB)
array = array fetch size
rows = print progress for every given rows (default, 1000000)
file = output file name(default: uldrdata.txt)
log = log file name, prefix with + to append mode
text = output type (MYSQL, CSV, MYSQLINS, ORACLEINS, FORM, SEARCH).
parfile = read command option from parameter file
for field and record, you can use '0x' to specify hex character code,
\r=0x0d \n=0x0a |=0x7c ,=0x2c, \t=0x09, :=0x3a, #=0x23, "=0x22 '=0x27
对于专家而言, 可以用如下方式得到以前全部的命令行选项.
sqluldr2 help=yes
通过引入一个TEXT选项, 来针对不同格式的导出进行相关选项的设置, 不仅方便了大家使用, 也可以对SQLULDR2的功能有一个很直接的了解, 例如SQLULDR2可以导出数据给MySQL用, 或导出成Excel可以打开的标准CSV文件, 或是生成MySQL和ORACLE上的INSERT语句, 也可以按列显示记录, 或为一些特殊的搜索程序生成数据源.
再次感谢Kamus的好建议, 今年是支付宝的用户体验年, 应当从用户角度进行反思.
Relative Posts:
Is it Friday already? Where does the time go? Lots of stuff going on this week–here’s a few of the things that I found interesting.
Operating Systems
Russia Today-TV announced the existence of “Red Star”, the new OS developed in North Korea and based on Linux. I found this by way of Slashdot of course, citing the source as The Korea herald. According to the article it looks very much like the Windows UI, and features a “My Country” icon that allows connection to Korea’s closed internet-like network and the Woori office application. Slightly more information can be found there in the article N. Korea develops own OS.
The upcoming Ubuntu 10.04 (Lucid Lynx) has had the third alpha version released. The alpha versions are not for the faint of heart, and I have had to put in several bug reports myself already. It’s just part of the process though and will hopefully result in a rock solid release next month. The full release announcement can be found on the mailing list: see Lucid Alpha 3 released. A rundown of some of the things you can expect is on The H Online in Canonical releases Ubuntu 10.04 LTS Alpha 3.
Ars Technica has more on an item mentioned last week about the future of Open Solaris. OpenSolaris not dead; might not get all new Solaris features The exact details of which features may be excluded are not clear from the article but I did not get a chance to dig into some of the source material referenced so you may find more by clicking through them.
Internet
Opera 10.5 is out and Ars Technica has a detailed review of the new features, performance improvements and UI changes. There’s a lot of detail in the article, I suggest you read it for yourself. Hands on: Opera 10.50 makes impressive performance gains is the place to go. I think I’ll give the new Opera a try myself.
Local company makes good! I mentioned last week the takedown of the Waledac botnet, this week the Mariposa botnet suffered a similar fate. Ottawa’s own Defence Intelligence detected this botnet in 2009 and started an investigation that ended with the dismantling of the botnet by Spanish authorities. This is a much more important takedown than Waledac because Mariposa actually put sensitive and financial information at risk. You can read more of this story in Spanish arrests mark the end of dangerous botnet. Ottawa, represent!
Virtualization
Microsoft has released new drivers to improve the performance of Red Hat Enterprise Linux systems running under Hyper-V. The new drivers include new network and storage drivers, but still only support a single virtual CPU. These drivers are not officially included in, or supported by Red Hat, but they are fully support by Microsoft. Read Microsoft releases Hyper-V drivers for RHEL for more, and you can check out Microsoft’s side of the story on the technet blog in Red Hat Enterprise Linux and Hyper-V.
How Server Virtualization Impacts Storage. The title says it all. Penned by Heidi Biggar from Hitachi it has a good perspective on current industry trends and growth. We all know that consolidation methods, like virtualization, add bottlenecks on resources like CPU, memory and storage because they are all shared by many instances. Find out what Heidi has to say on the subject.
Data Center
Data Center Knowledge is reporting that the demand for data centers has been growing continually and no change is in sight for 2010. We’re certainly seeing the trend here with many companies making the push for full, real disaster recovery sites. There seems to be a real shortage of data center space and new ones are popping up slowly, so if you have lots of cash this may be a good business to get into. But I digress. The full story is in Strong Data Center Demand Seen for 2010.
Security
The H Online has a good article about the US government’s new cyber security policy. I haven’t had a chance to review the whole document as yet, but I have it bookmarked for later reading. US government publishes parts of its cyber security directive is where to go, which also links the source document.
Hardware
Tom’s Hardware has an interesting Op-Ed piece on the past, present and future of the 3D graphics industry. See Opinion: AMD, Intel, And Nvidia In The Next Ten Years for 10 pages of goodness.
Allyn Malventano has a very detailed review of Western Digital’s first foray into the SSD market. The new 256GB offering performed well through the tests but is still priced a bit too high for the market. The full review is at PC Perspective, see Western Digital SiliconEdge Blue 256GB SSD Review – WD enters the SSD market. You can find the Western Digital product information page on their site.
Well that about wraps it up for this edition. Comments are always welcome. Until next time, may your days be productive and the levels of administratium be non-toxic.
The 181st edition of Log Buffer has been published by Gary Myers on his Sydney Oracle Lab.
Having recently moved his blog, Gary approached the Log Buffer coordinator to volunteer for an edition because he knows that, with LB being a popular and established destination in the database blogoshphere, it would help him broadcast his new blog and welcome readers to it. You can do it too–simply send an email to the Log Buffer coordinator.
Here’s Gary’s Log Buffer #181.
So mothers keep your hackers at home
Don’t let them journey all alone
Tell them this world is full of danger
And to shun the repositories of strangers
– The Tag Set of Strangers,
(with apologies to) Nick Cave and the Bad Seeds
One of the things I love about Git is how I can add branches from remote repositories in mine at will without fearing messing up anything. The remote branches will not clash with mine, even if they share the same names, because they are referenced as repository/branch. However, as for anything else, you can still poke yourself in the eye if you try hard enough:
$ git remote bob $ git checkout -b bob/baroque Switched to a new branch 'bob/baroque' $ git fetch bob From ../bob * [new branch] baroque -> bob/baroque $ git checkout bob/baroque warning: refname 'bob/baroque' is ambiguous. Already on 'bob/baroque'
Here, I created a local branch called bob/baroque, which will end up having the same name as the branch baroque imported from Bob’s repository. Confusing, but not the end of the world. I can still see the different branches with gitk and access the local and remote branches via git checkout remotes/bob/baroque and git checkout remotes/bob/baroque.[1] The lesson to take from this, of course, is simply not to use slashes in branch names and sidestep the whole issue.
[1] The fiendish-minded reader probably wonders at this point what would happen if I were to create another local branch called remotes/bob/baroque. I would deserve to be shot, that’s what would happen.
With tags, surprisingly, the matter is much more prickly. Not only are tags not kept in the ”namespace” of their repository of origin, but git fetch has a very dangerous default behavior:
$ git log -n 1 somework
commit 483d008c6207554236232fef4e8cd22cfb4b9bb8
Author: Yanick Champoux <yanick@babyl.dyndns.org>
Date: Wed Mar 3 21:14:43 2010 -0500
some work on my repo
$ git fetch --tags bob
From ../b
- [tag update] somework -> somework
$ git log -n 1 somework
commit 5f7f8eddd2d44e359fe8bc0d1a2f1642d073cad9
Author: Yanick Champoux <yanick@babyl.dyndns.org>
Date: Wed Mar 3 21:15:25 2010 -0500
some work from Bob
Yup, that’s right, if there is a conflict, fetch --tags will silently clobber the local tags with their remote counterparts. Hope you remember all the commits you painstakingly tagged in the last six months… This behavior is so mind-bogglingly dangerous that, to this day, I wonder if I’m not missing something obvious.
Now, granted, it’s fairly rare to import tags from remote repositories. But there are instances, like when adding the gitpan history of your module to your repository, where it’s relevant. In those cases, for your own sake, make sure that the remote tags won’t clash with yours. Even better, don’t use --tags at all. Instead, do something akin to:
$ git ls-remote --tags bob | \ perl -nae'$F[1]=~s#refs/tags#bob#; `git tag $F[1] $F[0]`'
This will name remote tags using the repository/tag convention. Also, if the tag already exist, Git will complain and it won’t be clobbered by the new version.
Last week has featured two of the most interesting posts about MongoDB: first coming from Mathias Meyer (@roidrage) ☞ offline investigation of MongoDB and the second, a set of notes from running MongoDB in production published on ☞ Boxed Iced blog.
If you are interested in getting started with MongoDB, I’d encourage you take the time to go through Mathias’ post which covers the following aspects (I’ve also included a couple of comments)
-
collections and capped collections
Note: I couldn’t really understand the usage of namespaces and the implication on indexes
- data format
-
references
Note: I’d also strongly suggest taking a look at MongoDB documentation on ☞ schema design for more details
- indexes
- updates
- querying
-
durability
Note: we have covered before the MongoDB durability tradeoff in much detail
- replication
- caching
- backup
- storage
-
concurrency
Note: I’d really appreciate more details on this topic as it is not completely clear if all access (both read and writes) is serialized or just writes are serialized (or not?); also the impact on indexes is not clear either.
- memory
- GridFS
-
protocol access
Note: We argued before that access protocols are extremely important. MongoDB is one of the NoSQL solutions using a proprietary protocols and tries to “compensate” for that with tons of libraries
-
sharding
Note: probably biased, but I still wait for the moment MongoDB sharding would become at least beta.
MapReduce support seems to be missing from Mathias notes, but luckily we have that covered for you: MongoDB MapReduce tutorial.
While keeping in mind that some of these features are not unique to MongoDB and can be found in other systems, you should be ready to cross check your app requirements with the lessons learned by the guys at Boxed Ice:
-
namespace limits
We split our customers across (currently) 3 MongoDB databases because there is a namespace limit of 24,000 per database. This is essentially the number of collections + number of indexes.
-
initial sync/replication of large databases
Our databases are very large and it takes about 48-72 hours to fully sync all our current data onto a new slave in a different DC (via a site-to-site VPN for security). During this time you’re at risk because the slave is not up to date.
-
initial sync “slows” things
When doing a fresh sync from a master to a slave, we have observed a “slowdown” in our application response times.
-
index creation blocks
However, if you have an existing collection and create a new index on it then that process will block the database until the index is created.
-
efficiency of reclaiming diskspace
We have found that there is a massive discrepancy between a master and a freshly copied slave.
Even if not every application will have to deal with the size Boxed Ice is dealing, I couldn’t stop noticing that parts of the process of scaling MongoDB were really painful. Or as Sergio Bossa (@sbtourist) put it in ☞ one of the comments:
Anyways, it seems indeed you had almost the same problems you would had with a MySQL solution:
- Huge data to deal with.
- Manual sharding.
- Sync/replication delays.
So why didn’t you evaluate to switch to a more “large-scale” nosql solution like Cassandra or Riak?
Last but not least, drop me a note if you are planning to use or already using MongoDB in production and you’d like to share your experience with the NoSQL community
本文主要翻译自Jonathan Lewis的Blog Entry Log File Switch, 最后关于checkpoint部分,我添加了部分我自己的注释(以斜体字显示).
日志文件切换
几天前,我发布一个简单的查询语句,它通过查询v$log_history视图展示日志文件切换之间的时间间隔.如果你运行这个脚本,并且认为你的系统的日志文件切换频率不合适,那么你该如何处理呢?
如果时间间隔太短(由于日志文件频繁切换引发的checkpoint动作可能会导致DBWR进程过于活跃),可以通过新增新的更大的日志文件并删除旧的日志文件来解决.
如果切换的时间间隔太久,或者是时间间隔波动太大,而你希望使日志切换更加有规律,那么可以使用参数archive_lag_target(在Oracle 9i中引入)来为日志文件切换设置一个超时值(单位为秒).如果日志文件没有在指定的时间间隔内发生切换,那么系统会强制其切换日志,并且归档对应的已经使用的在线重做日志.
在此之前,人们一般使用类似于cron或者dbms_job一类的工具来实现(通过执行一条”alter system switch logfile“命令).从Oracle 9i开始,利用这个参数来实现就简单多了.
附录
- Instance Recovery Checkpoint
- Media Recovery Checkpoint
- Thread Checkpoint
- Interval Checkpoint
- Tablespace Checkpoint
- PQ Tablespace Checkpoint
- Close Database Checkpoint
- Incremental Checkpoint
- Local Database Checkpoint
- Global Database Checkpoint
- Object Reuse Checkpoint
- Object Checkpoint
实例恢复触发的检查点
介质恢复触发的检查点
单个instance关闭以及log file switch触发的检查点
log_checkpoint_timeout参数超时触发的检查点
表空间online/offline/read only/read write操作对应的检查点
不清楚具体原因
常规的检查点动作
对应于alter system checkpoint
对应于alter system checkpoint all命令的检查点.
对应于truncate table操作的检查点
对应于drop table的检查点操作
Oracle支持多种不同类型的checkpoint,这个列表会随着Oracle版本的不同而有所不同.Oracle 10.2.0.3的对应的checkpoint看似有如下这些:
在这些中间,我认为日志文件切换时引发的checkpoint可能是ThreadCheckpoint.
No related posts.
2010-03-04 Thu
AnySQL.net
Oracle & Starcraft
Give you some color to see see!
Oracle Scratchpad
Oracle Life
Chanel [K]
Oracle Security Blog
MySQL Performance Blog
The Tom Kyte Blog
Delicious/Fenng/oracle
O'Reilly Databases
Red Hat Magazine
车东[Blog^2]
blue_prince
玉面飞龙的BLOG
木匠 Creative and Flexible
生活帮-LifeBang
Hey!! Sky!
dba on unix
Brotherxiao's Home
jametong's shared items in Google Reader
DBA Tools
Inside the Oracle Optimizer - Removing the black magic
DBA@Taobao
存储部落
OracleBlog.cn
知道分子
支付宝官方 Blog - 支付志
木匠的天空 Oracle Architect and Developer
Hello DBA
OS与Oracle
Cary Millsap
Guy Harrison's main page
eagle's home
dbthink
DBA Notes
OracleDBA Blog---三少个人涂鸦地!
The Pythian Blog
myNoSQL
