首页 > 技术相关, 数据库 > Redis实现之虚拟存储系统(VM)实现(四)–线程式虚存(Threaded VM)交换

Redis实现之虚拟存储系统(VM)实现(四)–线程式虚存(Threaded VM)交换

2011年8月6日 发表评论 阅读评论

非阻塞虚存的几种实现选择

由于阻塞式虚存实现有各种缺点,因此,Redis实现了非阻塞式的虚存系统。实现非阻塞式虚存系统有以下几种方法:

  1. 最显然的方法,就是把整个redis做成多线程的,每个数据库操作都是自动分配到某个线程(或者fork一个新的线程),这样就不存在阻塞的问题。但是这对redis不是一个好的解决方案,因为为了实现提高操作速度,redis没有锁,保证原子性是通过顺序执行来保证的,因此整个redis引入多线程要增加锁,导致速度变慢(也许多核上能快点),并且极大增加代码量(现在redis只有1万行代码)。
  2. 通过非阻塞式IO实现,由于redis是基于事件循环的(event-loop based),可以采用非阻塞IO。但redis的作者基于以下两点把这个否了:①非阻塞文件操作不像sockets,会导致很多不兼容的问题,这和操作系统紧密相关,要做很多和操作系统相关的工作。②另外一个原因就是,在虚存中,文件IO操作仅仅是一部分,还有很多事件花在数据的编解码上面(编解码成交换文件swap file)。
  3. 于是就剩下最后一种方法,通过增加IO线程来完成数据交换。

下面,具体介绍第三种实现,即IO线程:

IO线程

IO线程的设计目标依重要性排列如下:

  • 实现简单:尽量减少数据冲突(race)的可能性;用简单的锁;VM的代码要能够和Redis的其他代码分离(个人感觉这符合设计正交性原则)。
  • 高性能,对于客户端访问数据不会造成锁(等待)。
  • IO线程要能够编解码交换文件(针对第二种实现的缺点2)。

鉴于以上目标,Redis 的非阻塞VM实现的方式为,实现了一个redis主线程(实际和客户端交互的线程,处理请求),以及一些通过一个简单的信号量(mutex)和任务队列进行通信的IO进程。简单的说,主线程将IO请求交给IO线程进行后台处理(通过将一个(IO)任务结构交给server.io_newjobs队列)。假如没有IO进程,则新建一个。有些IO进程完成IO任务后,结果将发送到server.io_processed队列。IO进程将通过UNIX管道给主进程一个信号,以便告知已经完成,可以继续新的任务。

iojob的数据结构如下:

 typedef struct iojob {
    int type;   /* Request type, REDIS_IOJOB_* */
    redisDb *db;/* Redis database */
    robj *key;  /* This I/O request is about swapping this key */
    robj *val;  /* the value to swap for REDIS_IOREQ_*_SWAP, otherwise this
                 * field is populated by the I/O thread for REDIS_IOREQ_LOAD. */
    off_t page; /* Swap page where to read/write the object */
    off_t pages; /* Swap pages needed to save object. PREPARE_SWAP return val */
    int canceled; /* True if this command was canceled by blocking side of VM */
    pthread_t thread; /* ID of the thread processing this entry */
} iojob;
 

IO进程仅仅能处理三种类型的任务(通过上面的type域定义):

  • REDIS_IOJOB_LOAD:将给定key的对象从交换空间load回主存。
  • REDIS_IOJOB_PREPARE_SWAP:计算将val指向的对象交换到交换空间所需要的页数,计算的结果放在page域。
  • REDIS_IOJOB_DO_SWAP:将val指向的对象交换到交换空间。

主线程除了以上三项工作不做以外,其他的工作都是主线程完成的,比如通过算法找到需要交换出去的对象之类的。



本文作者: Sigma    在新浪微博关注SigmaSigmaWeibo    RSS订阅本博客
本文链接: http://www.sigma.me/2011/08/06/redis-vm-implement-threaded-vm.html
本博客采用知识共享署名—非商业性-禁止演绎使用3.0协议进行许可,转载请保留作者和原文链接。

  1. 2011年8月6日15:54 | #1

    为毛你推荐的广告是“第二代 让男人省事,让女人不省人事”。。。

    感觉有必要把技术类blog和生活/评论类blog分开,这样有利于提高流量。至少要分到不同的tags下,方便分别订阅。

  2. 2011年8月6日16:06 | #2

    @grapeot

    哈哈,肯定是百度更懂中国人,知道你的喜好~

    话说google pagerank就和你博客的主题有关,要是大杂烩,pr值很低,并且google adsense匹配也不好,容易封号

    现在我在这个博客基本不写生活类等不关痛痒的东西,否则好不容易搞来的订阅量又下去了(之前经常出现我没发一篇文章,订阅量瞬间降了几个,之后又慢慢升上来,现在基本不会了)

    小炫耀一下,我的这个博客pr值已经是4是4是4啦

  3. 2011年8月6日16:47 | #3

    查了一下哥的才2。。。主要的确没多少人链过去。。。

  4. 2011年8月6日17:55 | #4

    有道理,如何选一个大家感兴趣但能写(或者能写得好看)的人又不多的领域就是学问了。这就是重要的东西。

  5. 2011年8月6日17:58 | #5

    @grapeot

    嗯,感觉很多时候只能挖小的,之后慢慢积累,豆瓣就是这样发起来的

  6. 2011年8月6日16:44 | #6

    牛逼。。。4了。。。

    我的意思不是生活类,而是大家都能看得懂或者受众面广的东西。这个redis的原理是不是太专业了?

  7. 2011年8月6日16:55 | #7

    @grapeot

    大家都能看懂的,大众的,肯定很多人写,权重就低了,哥现在主要还是要靠搜索引擎,权重很重要

    只要我写过的,都得排前面才行,否则就白写了

    (其实sigma很多人搜,我排名也比较前,但是没转化成流量,主要搜索出来的结果都是我这个博客的介绍,而不是介绍sigma这个词的文章)

  8. 2019年1月9日11:52 | #8

    Thanks so much for providing individuals with an exceptionally remarkable chance to read from this website. It can be so lovely and full of a good time for me personally and my office colleagues to visit your site on the least thrice in a week to read the fresh items you have. And definitely, I’m just at all times happy with your surprising suggestions served by you. Certain two tips in this article are absolutely the most effective we have ever had.

  9. 2019年1月10日08:42 | #9

    I definitely wanted to send a comment in order to say thanks to you for those great steps you are writing on this website. My particularly long internet search has at the end been compensated with reliable strategies to go over with my companions. I ‘d assert that we visitors actually are truly lucky to dwell in a really good site with many special professionals with beneficial plans. I feel truly fortunate to have discovered the website page and look forward to plenty of more awesome minutes reading here. Thank you again for a lot of things.

  10. 2019年1月11日04:47 | #10

    I simply wanted to appreciate you again. I’m not certain the things I could possibly have gone through in the absence of these creative concepts shown by you about that area of interest. It had become the frightening circumstance in my opinion, however , taking note of this skilled avenue you handled it made me to leap for fulfillment. Now i am happier for this support as well as have high hopes you find out what a great job your are getting into training people by way of your web site. Probably you haven’t encountered all of us.

  11. 2019年1月12日00:21 | #11

    I just wanted to write down a small word in order to express gratitude to you for some of the great techniques you are writing on this site. My extensive internet research has at the end of the day been rewarded with really good tips to talk about with my neighbours. I would say that most of us site visitors are very endowed to live in a useful site with very many brilliant individuals with useful opinions. I feel extremely grateful to have used your entire website page and look forward to really more excellent times reading here. Thank you again for a lot of things.

  12. 2019年1月12日16:31 | #12

    I wanted to send you a little bit of word to help thank you so much yet again about the striking methods you have shown in this case. It’s wonderfully generous with you to give unhampered all that numerous people could possibly have offered for sale for an e book in order to make some dough for themselves, certainly considering that you could possibly have tried it in case you considered necessary. These tactics in addition served as a good way to be aware that someone else have the identical zeal just like my personal own to figure out very much more with regard to this condition. I am sure there are a lot more fun instances ahead for individuals that look over your blog.

  13. 2019年1月12日23:53 | #13

    Thanks so much for giving everyone an extraordinarily wonderful opportunity to check tips from this blog. It’s usually very superb and jam-packed with a lot of fun for me and my office friends to search your website a minimum of thrice in a week to learn the new issues you have got. And lastly, I’m just always impressed with your outstanding solutions served by you. Certain 4 facts on this page are ultimately the most impressive I’ve ever had.

  14. 2019年1月13日11:37 | #14

    I wish to voice my respect for your generosity giving support to women who absolutely need help on the matter. Your real dedication to passing the message up and down had become exceedingly practical and have surely allowed those just like me to reach their targets. Your own warm and friendly help and advice implies a great deal a person like me and especially to my mates. Warm regards; from each one of us.

  15. 2019年1月14日07:07 | #15

    A lot of thanks for all your valuable hard work on this blog. Debby takes pleasure in managing investigations and it is simple to grasp why. Almost all learn all about the dynamic manner you make efficient tips and tricks on your blog and in addition welcome contribution from other ones about this article so my simple princess is truly being taught a great deal. Enjoy the rest of the year. Your performing a remarkable job.

  16. 2019年1月14日16:29 | #16

    I wanted to write a brief note so as to say thanks to you for those superb tips and tricks you are sharing here. My considerable internet lookup has finally been honored with excellent details to go over with my great friends. I would express that many of us visitors actually are rather fortunate to exist in a remarkable network with very many outstanding people with useful techniques. I feel extremely happy to have seen your webpage and look forward to plenty of more fabulous moments reading here. Thanks once again for all the details.

  17. 2019年1月14日16:39 | #17

    Thanks a lot for providing individuals with remarkably memorable possiblity to read from this website. It’s usually very cool and also packed with a good time for me personally and my office fellow workers to visit your website particularly three times in a week to study the newest tips you will have. And of course, we are at all times astounded with the amazing methods served by you. Certain two tips in this post are completely the simplest we have all ever had.

  18. 2019年1月15日03:14 | #18

    I precisely had to thank you so much again. I’m not certain the things I could possibly have undertaken without the actual tactics provided by you about my industry. It absolutely was an absolute difficult problem in my opinion, nevertheless looking at the well-written style you handled that forced me to weep over gladness. Extremely happier for the information and in addition believe you recognize what a great job that you’re undertaking instructing the rest thru your web blog. I know that you have never got to know all of us.

  19. 2019年1月15日10:51 | #19

    I precisely needed to appreciate you again. I do not know the things that I could possibly have taken care of without these information revealed by you concerning this area. It became the intimidating circumstance in my position, but being able to see a new expert technique you handled that took me to leap with fulfillment. Now i’m happier for your support as well as wish you really know what a great job you’re doing teaching most people all through your blog. I know that you’ve never got to know all of us.

  20. 2019年1月15日20:07 | #20

    Thank you for all your valuable hard work on this website. My mum take interest in carrying out investigation and it is simple to grasp why. My partner and i notice all regarding the lively form you render sensible tips via the website and in addition increase response from visitors on this area of interest plus our own simple princess is without a doubt studying a great deal. Take advantage of the rest of the new year. You’re carrying out a good job.

  21. 2019年1月16日05:04 | #21

    Thanks a lot for providing individuals with an extremely special possiblity to discover important secrets from this website. It is always so sweet and jam-packed with amusement for me personally and my office acquaintances to search your blog minimum three times in 7 days to find out the newest guidance you have. And of course, I’m actually astounded with the fabulous principles served by you. Some 1 tips in this posting are indeed the finest we have all had.

  22. 2019年1月16日12:46 | #22

    I precisely wished to say thanks again. I do not know the things that I would have undertaken without the actual information documented by you directly on that question. It was a very frightful scenario in my position, however , being able to view the very well-written style you resolved that forced me to leap over joy. I will be thankful for your information and then pray you find out what a great job that you are carrying out teaching people today thru your blog. I am certain you have never come across all of us.

  23. 2019年1月16日23:47 | #23

    I in addition to my pals were studying the nice key points on your web blog while quickly I got an awful suspicion I had not expressed respect to the web site owner for those tips. My people had been totally passionate to read all of them and now have clearly been taking advantage of those things. Many thanks for turning out to be considerably considerate as well as for deciding upon variety of remarkable ideas most people are really eager to know about. My personal honest regret for not expressing appreciation to sooner.

  24. 2019年1月17日07:55 | #24

    Thank you for each of your work on this web page. Betty really likes making time for research and it’s really obvious why. My partner and i know all concerning the dynamic ways you convey sensible guidelines by means of your web blog and as well inspire response from other people on the subject so our favorite princess is truly being taught so much. Enjoy the remaining portion of the new year. You are always carrying out a splendid job.

  25. 2019年1月17日16:25 | #25

    I enjoy you because of all your valuable work on this web page. Kim delights in going through investigations and it’s really simple to grasp why. Many of us learn all concerning the dynamic form you give worthwhile guidelines by means of the web blog and as well as inspire participation from other individuals on the issue so my daughter is undoubtedly discovering a lot. Enjoy the remaining portion of the new year. You have been performing a useful job.

  26. 2019年1月18日00:21 | #26

    I would like to express my thanks to you for rescuing me from this situation. As a result of looking throughout the search engines and seeing suggestions which were not powerful, I was thinking my entire life was well over. Being alive devoid of the answers to the problems you’ve fixed all through this report is a crucial case, and ones which could have badly affected my entire career if I hadn’t noticed your website. That understanding and kindness in taking care of all the details was crucial. I am not sure what I would’ve done if I hadn’t discovered such a subject like this. I’m able to at this moment look ahead to my future. Thanks a lot very much for the specialized and result oriented guide. I won’t think twice to suggest the website to anybody who desires care on this situation.

  27. 2019年1月18日02:41 | #27

    I precisely had to appreciate you all over again. I am not sure the things that I would have done without these tricks shown by you directly on such a concern. It seemed to be a frightening dilemma for me personally, nevertheless being able to see this skilled way you dealt with it forced me to jump over contentment. I will be happier for this work and even hope that you comprehend what a great job your are undertaking educating others with the aid of your blog. Most likely you haven’t encountered all of us.

  28. 2019年1月18日09:42 | #28

    I have to show some appreciation to this writer for bailing me out of this type of condition. As a result of scouting through the world wide web and meeting solutions that were not productive, I thought my entire life was over. Existing without the solutions to the issues you’ve resolved as a result of your article content is a serious case, as well as the kind which could have badly affected my entire career if I hadn’t come across your website. Your primary expertise and kindness in controlling every item was valuable. I’m not sure what I would have done if I had not discovered such a step like this. I can now look forward to my future. Thanks a lot very much for your skilled and sensible guide. I won’t hesitate to refer your web site to anyone who will need counselling about this situation.

  29. 2019年1月18日17:31 | #29

    Thank you for all your valuable effort on this web page. Kate takes pleasure in making time for investigations and it’s really simple to grasp why. Most people learn all concerning the compelling way you convey priceless guidance by means of your website and even welcome response from some other people on the matter then our own girl has been becoming educated a lot. Take advantage of the remaining portion of the year. Your doing a first class job.

  30. 2019年1月19日03:06 | #30

    I enjoy you because of all of your hard work on this blog. My aunt really loves managing internet research and it is obvious why. A lot of people know all relating to the dynamic mode you render great tactics via your blog and in addition increase contribution from others about this idea and my princess is truly studying a lot. Take advantage of the rest of the year. You’re the one carrying out a dazzling job.

  31. 2019年1月19日09:40 | #31

    My wife and i have been now fortunate that Jordan managed to carry out his inquiry through the entire ideas he was given while using the weblog. It’s not at all simplistic just to be giving away secrets and techniques which often a number of people have been trying to sell. We know we now have the writer to be grateful to because of that. The type of illustrations you made, the simple website navigation, the friendships you will aid to create – it’s all fantastic, and it’s making our son and our family know that the concept is thrilling, which is highly vital. Many thanks for all the pieces!

  32. 2019年1月19日18:25 | #32

    I wish to express some thanks to the writer just for rescuing me from this particular dilemma. Just after browsing through the the web and coming across solutions which were not helpful, I was thinking my life was well over. Being alive without the presence of solutions to the difficulties you have resolved by means of your article content is a critical case, and the kind which could have adversely affected my career if I hadn’t noticed your website. Your primary talents and kindness in handling a lot of stuff was important. I’m not sure what I would’ve done if I hadn’t encountered such a subject like this. I’m able to at this time look forward to my future. Thanks for your time so much for this impressive and sensible help. I will not think twice to propose your web sites to anybody who should have assistance on this area.

  33. 2019年1月20日02:31 | #33

    I do consider all of the ideas you’ve presented to your post.
    They’re very convincing and can definitely work.
    Nonetheless, the posts are too brief for novices. May just you please prolong them a little from subsequent
    time? Thanks for the post.

  34. 2019年1月20日03:49 | #34

    I needed to compose you that tiny word just to say thank you again for your spectacular advice you have shown in this article. It was really generous with people like you to supply publicly precisely what a number of us would have made available as an electronic book in making some dough for their own end, notably since you could possibly have tried it in case you desired. Those solutions additionally acted to be a good way to be sure that someone else have the same dreams much like my personal own to grasp whole lot more with regards to this issue. I think there are some more pleasant situations up front for people who find out your blog post.

  35. 2019年1月20日06:42 | #35

    My spouse and i got really cheerful that Michael could finish up his studies out of the precious recommendations he obtained when using the weblog. It’s not at all simplistic just to choose to be giving for free thoughts that many people could have been trying to sell. And now we figure out we have got the blog owner to give thanks to for that. The most important illustrations you have made, the easy website navigation, the friendships your site make it easier to engender – it’s got mostly sensational, and it’s really making our son in addition to the family reason why the subject is fun, and that’s pretty vital. Thank you for the whole thing!

  36. 2019年1月20日10:43 | #36

    My wife and i got absolutely thrilled when Louis managed to finish up his investigation out of the precious recommendations he grabbed while using the blog. It is now and again perplexing to just happen to be giving away guides some people have been making money from. And now we know we need the website owner to thank for this. The most important explanations you’ve made, the simple blog menu, the friendships you will make it easier to promote – it’s many remarkable, and it’s really facilitating our son in addition to our family reckon that that article is satisfying, and that is wonderfully fundamental. Thank you for everything!

  37. 2019年1月20日14:23 | #37

    Magnificent goods from you, man. I have understand your stuff previous
    to and you’re just extremely magnificent.
    I really like what you’ve acquired here, really
    like what you are saying and the way in which you say it.
    You make it entertaining and you still care for to keep
    it smart. I can’t wait to read far more from you.
    This is actually a terrific web site.

  38. 2019年1月20日20:02 | #38

    I in addition to my buddies happened to be taking note of the best solutions on your website then all of a sudden developed an awful suspicion I never thanked you for those techniques. All of the young men were definitely as a consequence very interested to study all of them and have very much been loving them. I appreciate you for indeed being so helpful as well as for obtaining certain nice themes millions of individuals are really wanting to be aware of. My sincere apologies for not expressing appreciation to you sooner.

  39. 2019年1月21日04:39 | #39

    My spouse and i have been peaceful that Emmanuel could finish off his analysis from the ideas he discovered through your site. It is now and again perplexing to simply find yourself making a gift of key points that many others have been selling. We figure out we’ve got the blog owner to be grateful to because of that. The type of illustrations you have made, the straightforward website menu, the relationships you will assist to promote – it’s got everything extraordinary, and it is making our son and our family reason why this content is entertaining, which is extremely indispensable. Thank you for all!

  40. 2019年1月21日16:30 | #40

    Hi, I do think this is a great website. I stumbledupon it 😉 I
    will come back yet again since I book-marked
    it. Money and freedom is the greatest way to change,
    may you be rich and continue to help others.

  41. 2019年1月22日05:12 | #41

    I simply wanted to write a small note in order to express gratitude to you for some of the magnificent recommendations you are sharing at this site. My time-consuming internet investigation has now been honored with professional points to exchange with my relatives. I would claim that we readers actually are extremely endowed to be in a useful network with so many outstanding people with great hints. I feel truly fortunate to have used the site and look forward to so many more amazing minutes reading here. Thanks once more for everything.

  42. 2019年1月22日22:55 | #42

    I happen to be writing to let you be aware of of the magnificent discovery my cousin’s daughter enjoyed going through yuor web blog. She realized some things, not to mention what it’s like to have an awesome coaching mindset to make men and women smoothly know precisely a variety of complex issues. You truly exceeded our expected results. Thank you for providing the practical, healthy, explanatory and also easy thoughts on your topic to Tanya.

  1. 本文目前尚无任何 trackbacks 和 pingbacks.

无觅相关文章插件,快速提升流量