123
 123

Tip: 看不到本站引用 Flickr 的图片? 下载 Firefox Access Flickr 插件 | AD: 订阅 DBA notes --

2010-06-11 Fri

18:33 世界杯开幕了 (268 Bytes) » OracleDBA Blog---三少个人涂鸦地!

世界杯开幕了,相信很多人会去看世界杯。
可是偶,对足球一窍不通,只能看别人狂欢了。
或许这一个月,是很多人的盛宴,我就可以好好工作了,嘿嘿,希望晚上不要被看球的人吵醒。

14:08 Installing Oracle 11gR2 Enterprise Edition on Ubuntu 10.04 (Lucid Lynx) (8822 Bytes) » The Pythian Blog

I recently reformatted my laptop with the latest Ubuntu LTS release, 10.04, aka Lucid Lynx. Since I like to have a native client installation as well as a portable sandbox server, I decided to install the latest version of Oracle EE, 11.2.0.1.

Rather than re-invent the wheel, I’m going to direct you to the previous Oracle-on-Ubuntu post by my colleague Augusto Bott. Many of the directions there hold true here (even with 32-bit vs 64-bit), with a few exceptions.

Download the Software
First and foremost, download the 32-or-64-bit installer files (there are 2 of them) from OTN.

Installing Pre-requisite Packages
Install these software packages on your Ubuntu 10.04 system:

$ sudo apt-get install unzip build-essential x11-utils rpm ksh lsb-rpm libaio1

Ubuntu 10.04 comes with libstdc++6 installed. However, Oracle 11gR2 requires libstdc++5. If you do not install libstdc++5, you will see errors as described in this OTN thread. The fix, as described in that thread, is to download and manually shoehorn the libstdc++5 library files onto Ubuntu 10.04:

$ wget http://mirrors.kernel.org/ubuntu/pool/universe/g/gcc-3.3/libstdc++5_3.3.6-17ubuntu1_amd64.deb
$ dpkg-deb -x libstdc++5_3.3.6-17ubuntu1_amd64.deb ia64-libs
$ sudo cp ia64-libs/usr/lib/libstdc++.so.5.0.7 /usr/lib64/
$ cd /usr/lib64/
$ sudo ln -s libstdc++.so.5.0.7 libstdc++.so.5

$ wget http://security.ubuntu.com/ubuntu/pool/universe/i/ia32-libs/ia32-libs_2.7ubuntu6.1_amd64.deb
$ dpkg-deb -x ia32-libs_2.7ubuntu6.1_amd64.deb ia32-libs
$ sudo cp ia32-libs/usr/lib32/libstdc++.so.5.0.7 /usr/lib32/
$ cd /usr/lib32
$ sudo ln -s libstdc++.so.5.0.7 libstdc++.so.5

Now we’re ready to move on.

Create User and Groups
First create the oinstall and dba groups:

$ sudo su -
# addgroup oinstall
# addgroup dba

Then create the oracle user and assign it to those groups:

# useradd -g oinstall -G dba -p password -d /home/oracle -s /bin/bash oracle
# mkdir /home/oracle
# chown -R oracle:dba /home/oracle

Now we create some symbolic links to give the Ubuntu system a more “Red Hat-ish layout,” as Augusto put it:

# ln -s /usr/bin/awk /bin/awk
# ln -s /usr/bin/rpm /bin/rpm
# ln -s /usr/bin/basename /bin/basename
# mkdir /etc/rc.d
# for i in 0 1 2 3 4 5 6 S ; do ln -s /etc/rc$i.d /etc/rc.d/rc$i.d ; done
# mkdir -p /u01/app/oracle
# chown -R oracle:dba /u01

Then we update some sysctl parameters by editing /etc/sysctl.conf. It is wisest to backup this file first and then add these lines to the end of /etc/sysctl.conf:

fs.file-max = 6815744
fs.aio-max-nr = 1048576
kernel.shmall = 2097152
kernel.shmmax = 2147483648
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 1048576
net.core.wmem_max = 1048576
net.ipv4.ip_local_port_range = 9000 65535

Note that these values are different from what was needed in 11gR1.

Now we update some limits for the oracle software owner by adding these lines to the end of /etc/security/limits.conf:

oracle soft nproc 2047
oracle hard nproc 16383
oracle soft nofile 1023
oracle hard nofile 65535

Again, it is best to backup the file first.

Now we activate the new settings from /etc/sysctl.conf with this command:

# sysctl -p

You should see the new settings in the output of that command.

Unpack and Install the Software
From here we go into the directory containing our two 11gR2 database zip files. I’m using the 64-bit edition, so my files are named linux.x64_11gR2_databaseXof2.zip, where X is 1 or 2. I unzip both of these files, which creates a “database” subdirectory:

$ unzip linux.x64_11gR2_database_1of2.zip
$ unzip linux.x64_11gR2_database_2of2.zip

Each of these commands will produce a large volume of output to the terminal as it lists each file being unpacked. Once this is done, your directory should look something like this:

$ ls -lh
total 2.2G
drwxr-xr-x 8 seiler seiler 4.0K 2009-08-20 14:34 database
-rw-r--r-- 1 seiler seiler 1.2G 2010-03-09 11:33 linux.x64_11gR2_database_1of2.zip
-rw-r--r-- 1 seiler seiler 1.1G 2010-03-09 12:38 linux.x64_11gR2_database_2of2.zip

Now you simply cd into the database directory and run the runInstaller program:

$ cd database
$ ./runInstaller

This will launch the Oracle Universal Installer, or OUI, program. The rest is pretty straight forward. Since this is just a sandbox, I chose not to provide any email info and declined to receive updates. I chose to first install the software only, which went perfectly well. OUI will complain about missing packages, since it is checking for RPMs. We can safely ignore these and proceed with installation.

Once installation is done I set these variables in my bash environment:

export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
export ORACLE_SID=orcl
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$PATH
export EDITOR=/usr/bin/vi

Then some quick tests to verify installation:

$ sqlplus /nolog

SQL*Plus: Release 11.2.0.1.0 Production on Thu Jun 10 23:46:42 2010

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

SQL> quit
$ /u01/app/oracle/product/11.2.0/dbhome_1/OPatch/opatch lsinventory
Invoking OPatch 11.1.0.6.6

Oracle Interim Patch Installer version 11.1.0.6.6
Copyright (c) 2009, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/11.2.0/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 11.1.0.6.6
OUI version       : 11.2.0.1.0
OUI location      : /u01/app/oracle/product/11.2.0/dbhome_1/oui
Log file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2010-06-10_23-46-53PM.log

Patch history file: /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch_history.txt

Lsinventory Output file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2010-06-10_23-46-53PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1): 

Oracle Database 11g                                                  11.2.0.1.0
There are 1 products installed in this Oracle Home.

There are no Interim patches installed in this Oracle Home.

--------------------------------------------------------------------------------

OPatch succeeded.

Everything looks hunky-dory. Let’s create an instance.

I launched dbca to create an Oracle 11gR2 instance. The one problem I encountered here was that, at the end of dbca prompts, the “Confirmation” dialog window was blank. Other blog posts on the internet suggested blindly clicking around until you magically hit the “OK” button, but more often than not I would hit the “Cancel” button. Turns out this is a not-so-uncommon problem with Java Swing and Gnome’s visual effects. The solution for Gnome users is to disable Visual Effects in the Appearance preferences before launching dbca. If you are using a lightweight desktop environment such as LXDE, you shouldn’t have this problem.

And that’s that. The rest is nothing new to those who have done installations before. You can optionally configure TNS names or Listener with the netca tool, or log into your new instance and enjoy.

Special thanks to Augusto Bott for not only authoring the previous Oracle-on-Ubuntu articles but also for suggesting the Gnome Visual Effects conflict!

08:49 世博会又一个传说 (255 Bytes) » 玉面飞龙的BLOG

以前我说过,如果你讨厌一个人,恨一个人,你就告诉他去上海的豫园去吃小吃。
现在我又可以说,如果你恨一个人,讨厌一个人,你就告诉他去上海看世博会,就说有很多好看的。

08:38 ORA-23481 in logical standby (15058 Bytes) » 玉面飞龙的BLOG

这个问题是在作Online redef测试的时候碰见的,如果在primary执行START_REDEF_TABLE的时候,写错了目标表的名字,可能导致在logical standby上报这个错误然后导致apply停掉。并且该错误无法被skip。貌似bug.

SQL> create table large(id number primary key,text varchar(32) ) tablespace users;
Table created.

SQL> insert into large select object_id,object_name from dba_objects where object_id is not null;
72631 rows created.

SQL> create table TEST_NEW(id number primary key,text varchar(32) ) tablespace users2;
Table created.

SQL> exec dbms_redefinition.can_redef_table ( USER,’TEST’, DBMS_REDEFINITION.CONS_USE_PK) ;
PL/SQL procedure successfully completed.

然后执行START_REDEF_TABLE;如下敲错成“-”,自然会在primary上提示表large-new不存在。

exec dbms_redefinition.START_REDEF_TABLE ( USER, ‘LARGE’, ‘LARGE-NEW’, NULL, DBMS_REDEFINITION.CONS_USE_PK);

然后会在logical standby上碰见ORA-26808错误。

ORA-26808: Apply process AS00 died unexpectedly.
ORA-23481: unexpected name string “LARGE-NEW”


下面是logical standby上的日志,

Then on logical standby, when it parse “LARGE-NEW”, apply process died and report ORA-23481: unexpected name string “LARGE-NEW”.

————–alter.log on logical standby ———————

ALTER DATABASE START LOGICAL STANDBY APPLY IMMEDIATE
ALTER DATABASE START LOGICAL STANDBY APPLY (oradb)
with optional part
IMMEDIATE
Attempt to start background Logical Standby process
LOGSTDBY parameters set by user:
LOGSTDBY MAX_EVENTS_RECORDED = 2000000000
LOGSTDBY RECORD_UNSUPPORTED_OPERATIONS = TRUE
LOGSTDBY LOG_AUTO_DELETE = FALSE
LOGSTDBY PRESERVE_COMMIT_ORDER = FALSE
Thu Jun 03 08:26:11 2010
LSP0 started with pid=30, OS id=9363
Completed: ALTER DATABASE START LOGICAL STANDBY APPLY IMMEDIATE
LOGMINER: Parameters summary for session# = 5
LOGMINER: Number of processes = 3, Transaction Chunk Size = 201
LOGMINER: Memory Size = 30M, Checkpoint interval = 150M
LOGMINER: SpillScn 7930843, ResetLogScn 754488
LOGMINER: summary for session# = 5
LOGMINER: StartScn: 0 (0×0000.00000000)
LOGMINER: EndScn: 0 (0×0000.00000000)
LOGMINER: HighConsumedScn: 7930824 (0×0000.007903c8)
LOGMINER: session_flag 0×1
Thu Jun 03 08:26:12 2010
LOGMINER: session#=5, builder MS01 pid=32 OS id=9367 sid=34 started
Thu Jun 03 08:26:13 2010
LOGMINER: session#=5, reader MS00 pid=31 OS id=9365 sid=38 started
LOGMINER: Begin mining logfile for session 5 thread 1 sequence 321, /home/oracle/app/oradb/oradata/oradb/arch2/1_321_696990990.dbf
Thu Jun 03 08:26:13 2010
LSP2 started with pid=36, OS id=9371
Thu Jun 03 08:26:14 2010
LOGMINER: session#=5, preparer MS02 pid=35 OS id=9369 sid=51 started
Thu Jun 03 08:26:15 2010
LOGSTDBY Analyzer process AS00 started with server id=0 pid=37 OS id=9373
Thu Jun 03 08:26:15 2010
LOGSTDBY Apply process AS01 started with server id=1 pid=38 OS id=9375
Thu Jun 03 08:26:15 2010
LOGSTDBY Apply process AS03 started with server id=3 pid=36 OS id=9379
Thu Jun 03 08:26:15 2010
LOGSTDBY Apply process AS02 started with server id=2 pid=39 OS id=9377
Thu Jun 03 08:26:15 2010
LOGSTDBY Apply process AS04 started with server id=4 pid=42 OS id=9381
Thu Jun 03 08:26:16 2010
LOGSTDBY Apply process AS05 started with server id=5 pid=44 OS id=9383
LOGSTDBY Analyzer process AS00 server id=0 pid=37 OS id=9373 stopped
Errors in file /home/oracle/app/oradb/diag/rdbms/standby/oradb/trace/oradb_lsp0_9363.trc:
ORA-26808: Apply process AS00 died unexpectedly.
ORA-23481: unexpected name string “LARGE-NEW”
LOGSTDBY Apply process AS01 server id=1 pid=38 OS id=9375 stopped
LOGSTDBY Apply process AS02 server id=2 pid=39 OS id=9377 stopped
LOGSTDBY Apply process AS03 server id=3 pid=36 OS id=9379 stopped
LOGSTDBY Apply process AS04 server id=4 pid=42 OS id=9381 stopped
LOGSTDBY Apply process AS05 server id=5 pid=44 OS id=9383 stopped
Errors in file /home/oracle/app/oradb/diag/rdbms/standby/oradb/trace/oradb_as00_9373.trc:
ORA-23481: unexpected name string “LARGE-NEW”
LOGMINER: session#=5, reader MS00 pid=31 OS id=9365 sid=38 stopped
LOGMINER: session#=5, builder MS01 pid=32 OS id=9367 sid=34 stopped
LOGMINER: session#=5, preparer MS02 pid=35 OS id=9369 sid=51 stopped
LOGSTDBY status: ORA-16222: automatic Logical Standby retry of last action
LOGMINER: Parameters summary for session# = 5
LOGMINER: Number of processes = 3, Transaction Chunk Size = 201
LOGMINER: Memory Size = 30M, Checkpoint interval = 150M
LOGMINER: SpillScn 7930843, ResetLogScn 754488
LOGMINER: summary for session# = 5
LOGMINER: StartScn: 0 (0×0000.00000000)
LOGMINER: EndScn: 0 (0×0000.00000000)
LOGMINER: HighConsumedScn: 7930824 (0×0000.007903c8)
LOGMINER: session_flag 0×1
Thu Jun 03 08:26:18 2010
LOGMINER: session#=5, reader MS00 pid=31 OS id=9385 sid=38 started
Thu Jun 03 08:26:19 2010
LOGMINER: session#=5, builder MS01 pid=32 OS id=9387 sid=51 started
Thu Jun 03 08:26:19 2010
LOGMINER: session#=5, preparer MS02 pid=35 OS id=9389 sid=43 started
LOGMINER: Begin mining logfile for session 5 thread 1 sequence 321, /home/oracle/app/oradb/oradata/oradb/arch2/1_321_696990990.dbf
Thu Jun 03 08:26:20 2010
LOGSTDBY Apply process AS01 started with server id=1 pid=37 OS id=9393
Thu Jun 03 08:26:20 2010
LOGSTDBY Apply process AS02 started with server id=2 pid=38 OS id=9395
Thu Jun 03 08:26:20 2010
LOGSTDBY Apply process AS03 started with server id=3 pid=39 OS id=9397
Thu Jun 03 08:26:20 2010
LOGSTDBY Analyzer process AS00 started with server id=0 pid=36 OS id=9391
Thu Jun 03 08:26:20 2010
LOGSTDBY Apply process AS04 started with server id=4 pid=42 OS id=9399
Thu Jun 03 08:26:20 2010
LOGSTDBY Apply process AS05 started with server id=5 pid=44 OS id=9401
LOGSTDBY Analyzer process AS00 server id=0 pid=36 OS id=9391 stopped
Thu Jun 03 08:26:21 2010
Errors in file /home/oracle/app/oradb/diag/rdbms/standby/oradb/trace/oradb_lsp0_9363.trc:
ORA-26808: Apply process AS00 died unexpectedly.
ORA-23481: unexpected name string “LARGE-NEW”
LOGSTDBY Apply process AS01 server id=1 pid=37 OS id=9393 stopped
LOGSTDBY Apply process AS02 server id=2 pid=38 OS id=9395 stopped
LOGSTDBY Apply process AS03 server id=3 pid=39 OS id=9397 stopped
LOGSTDBY Apply process AS04 server id=4 pid=42 OS id=9399 stopped
LOGSTDBY Apply process AS05 server id=5 pid=44 OS id=9401 stopped
Errors in file /home/oracle/app/oradb/diag/rdbms/standby/oradb/trace/oradb_as00_9391.trc:
ORA-23481: unexpected name string “LARGE-NEW”
LOGMINER: session#=5, reader MS00 pid=31 OS id=9385 sid=38 stopped
LOGMINER: session#=5, builder MS01 pid=32 OS id=9387 sid=51 stopped
LOGMINER: session#=5, preparer MS02 pid=35 OS id=9389 sid=43 stopped
———————————————————————————————–

cat /home/oracle/app/oradb/diag/rdbms/standby/oradb/trace/oradb_as00_9391.trc

*** 2010-06-03 08:26:22.065
ORA-23481: unexpected name string “LARGE-NEW”
OPIRIP: Uncaught error 447. Error stack:
ORA-00447: fatal error in background process
ORA-23481: unexpected name string “LARGE-NEW”

—————————————————————————————————

cat /home/oracle/app/oradb/diag/rdbms/standby/oradb/trace/oradb_lsp0_9363.trc

*** 2010-06-03 08:26:21.980
dbkedDefDump(): Starting a non-incident diagnostic dump (flags=0×0, level=0, mask=0×0)
—– Error Stack Dump —–
ORA-26808: Apply process AS00 died unexpectedly.
ORA-23481: unexpected name string “LARGE-NEW”
KNACDMP: *******************************************************
KNACDMP: Dumping apply coordinator’s context at bfc99c94
KNACDMP: Apply Engine # 0
KNACDMP: Apply Engine name
KNACDMP: Coordinator’s Watermarks ——————————
KNACDMP: Apply High Watermark = 0000007903c8 (SCN=0×0000.007903c8)
KNACDMP: Apply Low Watermark = 0000007903c8 (SCN=0×0000.007903c8)
KNACDMP: Recovery Low Watermark = 0000007903c8 (SCN=0×0000.007903c8)
KNACDMP: Fetch Low Watermark = (SCN=0×0000.00000000)
KNACDMP: Fetch Low Watermark Time = 0
KNACDMP: Oldest SCN = (SCN=0×0000.00000000)
KNACDMP: Oldest XID =
KNACDMP: Oldest Create Time = 0
KNACDMP: Last replicant syncpoint SCN = 0×0000.00000000
KNACDMP: Last syncpoint at primary SCN = 0×0000.00000000
KNACDMP: First partition max pos = 000000799b61 (SCN=0×0000.00799b61)
KNACDMP: Last partition max pos = 000000799b61 (SCN=0×0000.00799b61)
KNACDMP: Last processed = 0000007903d7 (SCN=0×0000.007903d7)
KNACDMP: Conservative pos = 0000007903d7 (SCN=0×0000.007903d7)
KNACDMP: Recovery start pos = (SCN=0×0000.00000000)
KNACDMP: Recovery high watermark = 0000007903c8 (SCN=0×0000.007903c8)
KNACDMP: Spill LWM = (SCN=0×0000.00000000)
KNACDMP: Spill LWM Create Time = 0
KNACDMP: Coordinator’s constants ——————————-
KNACDMP: number of apply slaves = 5
KNACDMP: safety level (K) = 1
KNACDMP: max txns in memory = 1200
KNACDMP: max constraints per table = 653
KNACDMP: hash table size (in entries) = 1000000
KNACDMP: Coordinator’s intervals ——————————-
KNACDMP: syncpoint interval (ms) = 4294967295
KNACDMP: write low watermark interval(ms)= 1
KNACDMP: Coordinator’s timers/counters ————————-
KNACDMP: current time = 1275524781
KNACDMP: low watermark timer = 0
KNACDMP: syncpoint timer = 1275524780
KNACDMP: txnbufsize timer = 1275524780
KNACDMP: Coordinator’s txn counts ————————-
KNACDMP: total txns applied = 0
KNACDMP: number of unassigned txns = 0
KNACDMP: avg number of unassigned txns = 0.00
KNACDMP: total applied at last plwm write= 0
KNACDMP: apply prog. entries below plwm = 0
KNACDMP: total unassigned lcrs = 0
KNACDMP: Coordinator’s State/Flags —————————–
KNACDMP: Coordinator’s State = KNACST_APPLY_UNTIL_END
KNACDMP: Coordinator’s Flags = 0×8200
KNACDMP: Slave counts ——————————————
KNACDMP: number of reserved slaves = 1
KNACDMP: number of admin slaves = 0
KNACDMP: number of slaves in wait cmt = 0
KNACDMP: number of slaves suspended = 0
KNACDMP: number of safe slaves = 0
KNACDMP: avg number of idle slaves = 0.00
KNACDMP: number of slaves initializing = 0
KNACDMP: number of slaves terminating = 0
KNACDMP: Slave Lists ——————————————-
KNACDMP: Dumping All Slaves :-
Slave id = 0, State = 8, Flags = 0, Not Assigned
Slave id = 1, State = 0, Flags = 0, Not Assigned
Slave id = 2, State = 0, Flags = 0, Not Assigned
Slave id = 3, State = 0, Flags = 0, Not Assigned
Slave id = 4, State = 0, Flags = 0, Not Assigned
Slave id = 5, State = 0, Flags = 0, Not Assigned
KNACDMP: End dumping all slaves
KNACDMP: syncdep slaves = { }
KNACDMP: cont chunk slaves = { }
KNACDMP: cont slaves = { }
KNACDMP: exec txn slaves = { }
KNACDMP: Active slaves (0) = { }
KNACDMP: Idle slaves (5) = { 1 2 3 4 5 }
KNACDMP: Txn Lists ———————————————
KNACDMP: Dumping all txns :-
KNACDMP: End dumping all txns.
KNACDMP: Complete txns = { ** NO UNASS ** }
KNACDMP: Unassigned txns = { }
KNACDMP: *******************************************************

*** 2010-06-03 08:26:25.156
Warning: Apply error received: ORA-26714: User error encountered while applying. Clearing.
=====================

事件视图DBA_LOGSTDBY_EVENTS中没有记录任何有用的信息。
如下等各种方法都无法skip该错误, SQL apply进程无法启动。

exec dbms_logstdby.SKIP_ERROR(’TABLE’,'BINZHANG’,'%’,null);
exec dbms_logstdby.SKIP_ERROR(’NON_SCHEMA_DDL’,'BINZHANG’,'LARGE_NEW’,null);
exec dbms_logstdby.SKIP_ERROR(’SCHEMA_DLL’,'BINZHANG’,'LARGE_NEW’,null);
EXECUTE DBMS_LOGSTDBY.SKIP (stmt => ‘DML’, schema_name => ‘BINZHANG’, object_name => ‘%’, proc_name => null);
EXECUTE DBMS_LOGSTDBY.SKIP (stmt => ‘SCHEMA_DLL’, schema_name => ‘BINZHANG’, object_name => ‘%’, proc_name => null);
EXECUTE DBMS_LOGSTDBY.SKIP (stmt => ‘NON_SCHEMA_DDL’, schema_name => ‘BINZHANG’, object_name => ‘%’, proc_name => null);

测试环境:oracle11gR2 redhat5.

08:24 Hadoop and Complex Data Processing Workflows with Cascading (2450 Bytes) » myNoSQL

Understanding the basic concepts behind MapReduce is not a very difficult task, but those using extensively MapReduce tasks inside Hadoop are already facing new challenges like:

  • how can you run multiple map and/or reduce phases in your data processing?
  • how can you better coordinate the data processing execution flow for more complex scenarios?
  • how can you perform additional work between map/reduce phases?

Addressing these new challenges is the goal of the ☞ Cascading project:

Cascading is a feature rich API for defining and executing complex, scale-free, and fault tolerant data processing workflows on a Hadoop cluster.

Christopher Curtin’s slides embedded below are offering a good overview of what can be achieved using Cascading (starting with slide 20).

03:17 Twitter, Redis and Gizzard (1580 Bytes) » myNoSQL

Just one day after mentioning Giazzard as a possible solution for partitioning data in CouchDB, I’m hearing that Twitter is experimenting with Redis and uses ☞ Gizzard for data partitioning. Code for this experiment is available on ☞ GitHub under the name Haplocheirus[1]

References

  1. Just in case you are wondering what Haplocheirus means, it is a ☞ dinosaur.  ()
03:09 Recommendation Engines with Neo4j (1831 Bytes) » myNoSQL

Even if a bit hidden behind the Rails code, the slides embedded below and these ☞ code snippets are using the RESTful access to Neo4j to build a recommendation engine in a network of StarWars heroes:

You Might Also Like: Implementing User Recommendations in Rails

Probably obvious, but graph databases are definitely a good tool for building recommendation engines.

2010-06-10 Thu

19:11 How can I change the default logging interval for the “sar” monitoring utility? (449 Bytes) » DBA@SKY-MOBI
[From] Red Hat Knowledgebase http://kbase.redhat.com/faq/docs/DOC-5406 The sar utility is called by cron every 10 minutes to put a snapshot of system activity in the /var/log/saDD file (where “DD” is the two-digit day of the month). To change this value, edit the /etc/cron.d/sysstat file. The file is in standard cron format. The default file is below: # run system activity accounting tool every 10 minutes */10 * * * * [...]
15:23 MongoDB NoSQL Ecosystem News & Links (3210 Bytes) » myNoSQL
12:48 Purge Thread Spiral of Death (3352 Bytes) » MySQL Performance Blog

I just wrote a large post on reasons for innodb main tablespace excessive growth and I thought it would make sense to explain briefly of why it is so frequently you have purge not being the problem at all and when out of no where you can see purge thread being unable to keep up and undo table space explodes and performance drops down. Here is what happens.

When you have typical OLTP system with small transactions your UNDO space is small and it fits in buffer pool. In fact most of the changes do not need to go to the disk at all – the space for undo space is allocated, used and freed without ever needing to go to the disk.

Now when you have spike in writes or long running transactions which increases your undo space size it may be evicted from buffer pool and stored on disk. This is when problems often starts to happen. Now instead of purge thread simply operating in memory it has to perform IO which slows it down dramatically and makes it unable to handle amount of changes coming in.

The solution to this problem may range from pacing the load (which is helpful for batch job operations as it can be controlled), using innodb_max_purge_lag or enable separate purge thread (or threads) via innodb_use_purge_thread if you’re running Percona Server.


Entry posted by peter | No comment

Add to: delicious | digg | reddit | netscape | Google Bookmarks

12:34 招聘 » OS与Oracle
12:27 Reasons for run-away main Innodb Tablespace » MySQL Performance Blog
08:01 Scaling CouchDB » myNoSQL
05:07 Logical standby notes » 玉面飞龙的BLOG

2010-06-09 Wed