欢迎来到麦多课文档分享! | 帮助中心 海量文档,免费浏览,给你所需,享你所想!
麦多课文档分享
全部分类
  • 标准规范>
  • 教学课件>
  • 考试资料>
  • 办公文档>
  • 学术论文>
  • 行业资料>
  • 易语言源码>
  • ImageVerifierCode 换一换
    首页 麦多课文档分享 > 资源分类 > PPT文档下载
    分享到微信 分享到微博 分享到QQ空间

    A New DMA Registration Strategyfor Pinning-Based High .ppt

    • 资源ID:373167       资源大小:237.50KB        全文页数:12页
    • 资源格式: PPT        下载积分:2000积分
    快捷下载 游客一键下载
    账号登录下载
    微信登录下载
    二维码
    微信扫一扫登录
    下载资源需要2000积分(如需开发票,请勿充值!)
    邮箱/手机:
    温馨提示:
    如需开发票,请勿充值!快捷下载时,用户名和密码都是您填写的邮箱或者手机号,方便查询和重复下载(系统自动生成)。
    如需开发票,请勿充值!如填写123,账号就是123,密码也是123。
    支付方式: 支付宝扫码支付    微信扫码支付   
    验证码:   换一换

    加入VIP,交流精品资源
     
    账号:
    密码:
    验证码:   换一换
      忘记密码?
        
    友情提示
    2、PDF文件下载后,可能会被浏览器默认打开,此种情况可以点击浏览器菜单,保存网页到桌面,就可以正常下载了。
    3、本站不支持迅雷下载,请使用电脑自带的IE浏览器,或者360浏览器、谷歌浏览器下载即可。
    4、本站资源下载后的文档和图纸-无水印,预览文档经过压缩,下载后原文更清晰。
    5、试题试卷类文档,如果标题没有明确说明有答案则都视为没有答案,请知晓。

    A New DMA Registration Strategyfor Pinning-Based High .ppt

    1、A New DMA Registration Strategy for Pinning-Based High Performance Networks,Dan Bonachea & Christian Bell U.C. Berkeley and LBNL bonachea,csbellcs.berkeley.edu http:/www.cs.berkeley.edu/bonachea/gasnet,This work is part of the UPC and Titanium projects, funded in part by the DOE, NSF and DOD,Problem

    2、 Motivation: Client,Global-address space (GAS) languages UPC, Titanium, Co-Array Fortran, etc. Large globally-shared memory areas w/language support for direct access to remote memory Total remotely accessible memory size limited only by VM space Working set of memory being touched likely to fit in

    3、physical mem App performance tends to be sensitive to the latency & CPU overhead for small operations Implications for communication layer (GASNet) Want low-latency and low-overhead for non-blocking small puts/gets (think 8 bytes) Want high-bandwidth, zero-copy msgs for large transfers zero-copy: ge

    4、t higher bandwidth AND avoid CPU overheads Ideally all communication should be fully one-sided one-sided: dont interrupt remote host CPU - hurts remote compute performance and increases round-trip latency,Problem Motivation: Hardware,Pinning-based NICs (e.g. Myrinet, Infiniband) Provide one-sided RD

    5、MA transfer support, but Memory must be explicitly registered ahead of time Requires explicit action by the host CPU on both sides Tell the OS to pin virtual memory page (kernel call) Register fixed virtual/physical mapping w/NIC (PCI transaction) Memory registration can be expensive! Especially on

    6、Myrinet - average is 40 microsec to register one page, 6 milliseconds to deregister one page Costs primarily due to preventing race conditions with pending messages that could compromise system memory protection Want to reduce the frequency of registration operations and the need for two-sided synch

    7、ronization Reducing cost of a single registration operation is also important, but orthogonal to this research,Memory Registration Approaches,Hardware-Based (e.g. Quadrics) Zero-copy, One-sided, Full memory space accessible, No handshaking or bookkeeping in software Hardware complexity and price, Ke

    8、rnel modifications Pin Everything - pin pages at startup or when allocated Zero-copy, One-sided (no handshaking) Total usage limited physical memory, may require a custom allocator Bounce Buffers - stream data through pre-pinned bufs No registration cost at runtime, Full memory space accessible Two-

    9、sided, mem copy costs (CPU consumption - increases CPU overhead, prevents comm. overlap), Messaging overhead (metadata and handshaking) Rendezvous - round-trip message to pin remote pages Zero-copy, Full memory space accessible, Only handshaking synchronous Two-sided, Registration costs paid on ever

    10、y operation (very bad on Myrinet) Firehose - our algorithm Zero-copy, One-sided (common case), Full memory space accessible, Only handshaking is synchronous, Registration costs amortized Messaging overhead (metadata and handshaking) on miss (uncommon case),Basic Idea: A Hybrid Approach,Firehose - A

    11、distributed strategy for handling registration Get the benefits of Pin-Everything in the common case Revert to Rendezvous-like behavior for the uncommon case Allow remote nodes to control and cache registration ops Each node sets aside M bytes of physical memory for registration purposes (some reaso

    12、nable fraction of phys mem) Guarantee F = physical pages to every remote node, which has control over where theyre mapped in virtual mem When a remote page is already mapped, can freely use one-sided RDMA on it (a hit) - exploits temporal and physical locality Send Rendezvous-like synchronization me

    13、ssages to change mappings when a needed remote page not mapped (a miss) Also cache local memory registration to amortize pinning costs,Firehose: Conceptual Diagram,Runtime snapshot of two nodes (A and C) mapping their firehoses to a third node (B),A and C can freely “pour“ data through their firehos

    14、es using RDMA to/from anywhere in the buckets they map on B Refcounts used to track number of attached firehoses (or local pins) Support lazy deregistration for buckets w/ refcount = 0 using a victim FIFO with a fixed max length (MAXVICTIM) to avoid re-pinning costs,Firehose: Implementation Details,

    15、Implemented on Myrinet/GM as part of a GASNet impl. Fully non-blocking (even for firehose misses) and supports multi-threaded clients - also need refcounts on firehoses to prevent races Use active messages to perform remote Firehose operations Currently only one-sided for puts, because GM lacks RDMA

    16、 gets For now, gets implemented as an active message and a firehose put Physical memory consumption never exceeds M+MAXVICTIM (both tunable parameters) - may be much less, based on access pattern Data structures used: Local bucket table: bucket virtual addr = bucket ref count Bucket Victim FIFO: lin

    17、ks buckets w/ refcount = 0 Firehose table: (remote node id, bucket virtual addr) = firehose ref count All bookkeeping operations are O(1) Overhead for all metadata lookups/modifications for a put 1s,Application Benchmarks,Simple kernels written in Titanium - just want a realistic access pattern 2 no

    18、des, Dual PIII-866MHz, 1GB RAM, Myrinet PCI64C, 33MHz/64bit PCI bus Rendezvous includes caching of locally-pinned pages Rendezvous no-unpin not a robust strategy for GAS lang, shown for comparison Firehose misses are rare, and even misses often hit in victim cache Avg time to move firehose on remote

    19、 node is 5 s and 14 s (pin takes 40 s) Firehose never needed to unpin anything in this case (total mem sz phys mem),Performance Results: Peak Bandwidth,Peak bandwidth - puts to same location with increasing message sz Firehose beats Rendezvous no-unpin by eliminating round-trip handshaking msgs Fire

    20、hose gets 100% hit rate - fully one-sided/zero-copy transfers,Performance Results: Put Bandwidth,64 KB puts, randomly distributed over increasing working set size Rendezvous no-unpin is “cheating“ beyond 450 MB Note graceful degradation of Firehose beyond 450 MB working set,Performance Results: Roun

    21、dtrip Put Latency,8-byte puts, randomly distributed over increasing working set size Rendezvous no-unpin is “cheating“ beyond 450 MB Rendezvous with-unpin not shown: is about 6000 microseconds,Conclusions,Firehose algorithm is a good registration strategy for GAS languages on pinning-based networks

    22、Get the performance benefits of Pin-Everything approach (without the drawbacks) in the common case and degrade to Rendezvous-like behavior for the uncommon case Exposes one-sided, zero-copy RDMA as common case Amortizes cost of registration and synchronization over multiple operations, and avoids co

    23、st of repinning recently used pages Cost of handshaking and registration negligible when working set fits in physical memory, and degrades gracefully beyond Future Work Use Firehose for an Infiniband-GASNet implementation Make MAXVICTIM adaptive for better scaling when access pattern unbalanced (bucket “stealing“) avoid unpin-repin costs Extend to RDMA gets in GM2 (soon to be released) http:/www.cs.berkeley.edu/bonachea/gasnet,


    注意事项

    本文(A New DMA Registration Strategyfor Pinning-Based High .ppt)为本站会员(bowdiet140)主动上传,麦多课文档分享仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对上载内容本身不做任何修改或编辑。 若此文所含内容侵犯了您的版权或隐私,请立即通知麦多课文档分享(点击联系客服),我们立即给予删除!




    关于我们 - 网站声明 - 网站地图 - 资源地图 - 友情链接 - 网站客服 - 联系我们

    copyright@ 2008-2019 麦多课文库(www.mydoc123.com)网站版权所有
    备案/许可证编号:苏ICP备17064731号-1 

    收起
    展开