当前在线人数16637
首页 - 分类讨论区 - 电脑网络 - 数据库版 -阅读文章
未名交友
[更多]
[更多]
文章阅读:Re: question on large tables (>=800 million records, 10 G
[同主题阅读] [版面: 数据库] [作者:wyr] , 2007年01月20日09:49:38
wyr
进入未名形象秀
我的博客
[上篇] [下篇] [同主题上篇] [同主题下篇]

发信人: wyr (遗忘小资), 信区: Database
标  题: Re: question on large tables (>=800 million records, 10 G b
发信站: BBS 未名空间站 (Sat Jan 20 10:12:43 2007), 转信

customized hash algorithm to help you partition your data based on the
features  of your query. I do not know how mysql implement their algorithm.
Here is my 2 cents based on my understanding of Teradata ..

If you have a primary index (unique or not unique), you starting from trying
to distribute your data evenly into several segmentatoins using these
columns(I am assuming your query condition is primarily based on these
columns). let say 10 .
Then based on your PK, you build a hash algorithm to make the permutatoin of
all your PK columns map to 10 bucket .
For each bucket, try to build your btree  within the bucket.
Your will have 10 copies of your partition search module which take care
only 1 parition.

Now you have a structure which contains 10 partitions and each partition has
its own btree as reference to its own data. and 10 worker thread

A query coming and access a specific record can go through your hash
algorithm to determin whick partition it will go to and then turned to a
woker thread to search a smaller index file.

If an aggregation query comes in, which involes data cross multiple
paritions, then, depends on where condition in the query, you may be able to
determin what are those paritions you need to send your request.

starting from here, using parallelism to make your application run faster. n
(logn) or what so ever.. if you can divide linearly by Node. if your n is
not infinitly.. then it is still runnable.
【 在 babycry (babycry) 的大作中提到: 】
: Question # 1:
: Why build a customized B-tree/Hash table ?
: How is it different from the B-tree implementation in a database server?
: Why the B-tree/Hash table implemented in mysql server is NOT good ?
: How can a customized B-tree/Hash table benefit ?
: Somebody cannot drive a car from Boston to S.F. in one hour
: does not necesserily mean you can do it if you drive by yourself.
: Question # 2:
: How upgrading hardware will make the application faster ...
: say from 5 minutes per query to 1 minute per query ?
: ...................


--

※ 来源:·BBS 未名空间站 mitbbs.com·[FROM: 70.244.]

[上篇] [下篇] [同主题上篇] [同主题下篇]
[转寄] [转贴] [回信给作者] [修改文章] [删除文章] [同主题阅读] [从此处展开] [返回版面] [快速返回] [收藏] [举报]
 
回复文章
标题:
内 容:

未名交友
将您的链接放在这儿

友情链接


 

Site Map - Contact Us - Terms and Conditions - Privacy Policy

版权所有,未名空间(mitbbs.com),since 1996