cross worldhttps://yohiro.games/Firefly6.16.8https://github.com/CuteLeaf/Firefly2026年9月27日 22:39:16https://yohiro.games/posts/sakurai/https://yohiro.games/posts/sakurai/Fri, 05 Apr 2024 00:00:00 GMT<section><h2>介绍<a href="#介绍"><span>#</span></a></h2><p><a href="https://www.youtube.com/@sora_sakurai_jp" target="_blank">樱井政博的游戏开发方法</a>（桜井政博のゲーム作るには）是星之卡比、任天堂明星大乱斗的游戏制作人樱井政博在油管的频道，主要发布了一系列游戏开发的建议以及方法论。我一直在断断续续地看，有时很受启发，但是这样看无异于走马观花，于是在这里总结归档一下，并在频道更新时，进行相应的更新。</p><p>我参考的版本是 B 站由大乱斗吧主光波大侠上传的版本，传送门点<a href="https://space.bilibili.com/5449804/channel/collectiondetail?sid=663948" target="_blank">这里</a>。</p><p>视频分为多个不同的大类，各类别具有不同的编号，我整理了一下，大致分类如下：</p> <table><thead><tr><th>编号</th><th>类别</th></tr></thead><tbody><tr><td>A*</td><td>职业素养</td></tr><tr><td>B*</td><td>游戏性</td></tr><tr><td>C*</td><td>策划 / 游戏设计</td></tr><tr><td>D*</td><td>游戏机制</td></tr><tr><td>E*</td><td>团队运营</td></tr><tr><td>F*</td><td>图形画面</td></tr><tr><td>G*</td><td>动画表现</td></tr><tr><td>H*</td><td>特效表现</td></tr><tr><td>I*</td><td>音乐/音效</td></tr><tr><td>J*</td><td>游戏界面</td></tr><tr><td>K*</td><td>程序/技术</td></tr><tr><td>L*</td><td>市场/宣发</td></tr><tr><td>M*</td><td>开发轶事</td></tr><tr><td>N*</td><td>开发理念</td></tr><tr><td>Z*</td><td>新闻</td></tr></tbody></table><p>这个系列将提取每期视频中有价值的观点、内容，当作记录。</p></section> <section><h2>A：职业素养<a href="#a职业素养"><span>#</span></a></h2><section><h3>A01：游戏开发不是游戏<a href="#a01游戏开发不是游戏"><span>#</span></a></h3><ul> <li>开发者需要回应玩家的需求</li> <li>无论开发怎样的游戏，目标始终是让玩家体会到乐趣（Entertain Players）</li> </ul></section><section><h3>A02：演讲要精简<a href="#a02演讲要精简"><span>#</span></a></h3><ul> <li>Presentation are all about speed！单刀直入，丢掉冗余的介绍，一切从简</li> </ul></section><section><h3>A03：给玩家交代<a href="#a03给玩家交代"><span>#</span></a></h3><ul> <li>以方便开发为优先而非满足玩家实际需求时——“如何向玩家交代？”</li> <li>玩家不会在意开发者做什么，更注重游戏质量</li> <li>开发碰到问题时，也不要忽略玩家视角</li> <li>总之，Player comes first</li> </ul></section><section><h3>A04：开发中的需求变更<a href="#a04开发中的需求变更"><span>#</span></a></h3><ul> <li>最应该避免的是因为游戏不够有趣而被迫变更需求的情况，因此在企划阶段就应为游戏赋予乐趣</li> <li>有问题及时反馈，“我就知道”这种话毫无帮助，玩家没办法解决问题</li> <li>将解决方案摆到台面上，团队共同选择出当前最优解</li> <li>应收集尽可能多的信息后再进行决策</li> </ul></section><section><h3>A05：企划书格式<a href="#a05企划书格式"><span>#</span></a></h3><ul> <li>企划书内容太多不是问题，可以通过调整讲解的节奏来调整，使观看者有看漫画一样的感觉</li> <li>大乱斗 SP 的企划大纲：概述 / 对战 / 角色 / 场地 / 画面 / 音乐 / 模式 / 网络 / 新角色</li> </ul></section><section><h3>A06：企划定稿技巧<a href="#a06企划定稿技巧"><span>#</span></a></h3><ul> <li>想法分为多级标题，内容要精简，要搭配图片来具象化想要表达的理念</li> <li>立项原因不宜冗长，玩家不会看这个</li> <li>趁着兴奋上头的时候多写点</li> <li>企划阶段并不是全部，在开发过程中不断打磨，也能为游戏赋予趣味</li> </ul></section><section><h3>A07：困难一时，游戏长存<a href="#a07困难一时游戏长存"><span>#</span></a></h3><ul> <li>见标题</li> </ul></section><section><h3>A08：游戏总监要有个性<a href="#a08游戏总监要有个性"><span>#</span></a></h3><ul> <li>可以从与其他人不同的角度看待问题的敏锐观察力</li> <li>要<strong>学会表达</strong>自己的创意和想法</li> </ul></section><section><h3>A09：Just Do It<a href="#a09just-do-it"><span>#</span></a></h3><ul> <li>解决拖延症的办法：干就完了！</li> <li>没有动力也不要胡思乱想，没有意义</li> <li>先做着，做着做着就有动力了</li> <li>任务图表很有用，但是可以后期补，现在干就完事了</li> </ul></section><section><h3>A10：如何坚持自己的专栏<a href="#a10如何坚持自己的专栏"><span>#</span></a></h3><ul> <li>旺盛的表达欲</li> <li>不占用太多时间</li> <li>没有压力</li> <li>越是重复性强的工作，越应该学着放松自己，这样才能享受工作</li> </ul></section><section><h3>A11：不要依赖 Plan B<a href="#a11不要依赖-plan-b"><span>#</span></a></h3><ul> <li>着眼大局，选择自己认为最好的方案</li> <li><strong>完善</strong>一个方案，不行再以此为基础调整方向</li> <li>以理想状态为目标展开工作</li> <li>总之，深度优先！</li> </ul></section><section><h3>A12：明确目标受众<a href="#a12明确目标受众"><span>#</span></a></h3><ul> <li>随着人的成长，价值观逐渐成形，会形成各个分支从而产生了不同类型的玩家</li> <li>年轻人的价值观仍未完全成型，因此更能接受新鲜事物，拥有最广阔的市场</li> <li>在各个分支选择目标受众时，不要仅凭好恶，应自上而下的审视市场状况</li> </ul></section><section><h3>A13：多输入，多输出<a href="#a13多输入多输出"><span>#</span></a></h3><ul> <li>让输出内容变成一种习惯</li> <li>光说不练假把式，光练不说傻把式</li> </ul></section><section><h3>A14：闷声发大财<a href="#a14闷声发大财"><span>#</span></a></h3><ul> <li>保持不与外界产生联系的高压状态</li> <li>优先独立完成企划，再去与团队进行头脑风暴</li> <li>想象自己整了个大活（褒义）他人的反应和玩家的反响</li> </ul></section><section><h3>A15：真理并不总是掌握在大多数人手中<a href="#a15真理并不总是掌握在大多数人手中"><span>#</span></a></h3><ul> <li>服从多数达成的决策大多比较稳妥，也就是中庸</li> <li>对创意性项目有大局考量的人适合把控方向，所以有中立视角能负责任和吸引火力的总监是项目的灵魂人物</li> <li>团队内成员最好对问题和各种解决方案透明化</li> </ul></section><section><h3>A16：暗示的力量<a href="#a16暗示的力量"><span>#</span></a></h3><ul> <li>比起推着自己走向特定的目标，<strong>暗示自己要逃离现状</strong>可能更有效</li> </ul></section><section><h3>A17：坚信概念的指引<a href="#a17坚信概念的指引"><span>#</span></a></h3><ul> <li>概念设计可以赋予游戏吸引力、指导开发方向</li> <li>缺少核心概念之前不应冒进</li> <li>纸上谈兵终有不足，计划外的要素常会成为乐趣的源泉</li> </ul></section><section><h3>A18：竞争与收获<a href="#a18竞争与收获"><span>#</span></a></h3><ul> <li>没啥干货，主要阐述时下游戏开发者面临的竞争和玩家的收获</li> </ul></section><section><h3>A19：说出来就有可能实现<a href="#a19说出来就有可能实现"><span>#</span></a></h3><ul> <li>态度改变生活</li> <li>赢家总是能够<strong>付诸行动</strong>的人</li> </ul></section><section><h3>A20：信息茧房<a href="#a20信息茧房"><span>#</span></a></h3><ul> <li>信息茧房搭建了一个杜绝外界观点和视角的封闭世界，封闭太久会使人无法发现真正的问题所在</li> <li>制作游戏的时候，有时朝着自己觉得“不平衡”的方向或许更好呢，至少比随大流要强</li> </ul></section></section> <section><h2>B：游戏性<a href="#b游戏性"><span>#</span></a></h2><section><h3>B01：风险与回报<a href="#b01风险与回报"><span>#</span></a></h3><ul> <li>将游戏性定义为<strong>博弈</strong>（push and pull）或<strong>风险与回报</strong>（risk and reward），类似格斗游戏中的立回</li> <li>而控制风险并获得回报的方式是玩家层面的策略，策略也是一种游戏性</li> <li>控制风险的机制是游戏平衡的关键</li> <li>发挥玩家的主观能动性控制 / 减少风险并获得回报，这就是策略的意义</li> <li>游戏性是游戏的特征之一，还有其他令玩家乐在其中的体验形式</li> <li>通过降低游戏的复杂度来吸引更为大众化的玩家</li> </ul></section><section><h3>B02：施加压力和释放压力<a href="#b02施加压力和释放压力"><span>#</span></a></h3><ul> <li>如何给玩家<strong>施加压力</strong>，如何让玩家<strong>释放压力</strong>是游戏是否有趣的核心</li> <li>压力本身没有乐趣，但积压到一定程度将其释放，可以带来巨大的满足感</li> <li>施加压力包括设置敌人、障碍或解密要素等，总之是要阻碍玩家的游戏进程</li> <li>在玩家通过一个挑战后，应给予奖励，为玩家提供更有效的处理 / 缓解压力的手段（经验值，技能点，装备等）</li> <li>玩家推进流程、自主选择流程本身也是一种奖励</li> <li>不需要直接模仿其他游戏使如何处理压力的，更应该研究一下为何如此设计，这样更能促进自己在开发中进行创新</li> </ul></section><section><h3>B03：RPG 的游戏性<a href="#b03rpg-的游戏性"><span>#</span></a></h3><ul> <li>RPG 的核心游戏性元素是探索和战斗</li> <li>ARPG 需要考虑动作场面的平衡</li> <li>玩家先会根据敌人的外观特征和攻击方式决定战斗策略</li> <li>最好也不要让玩家考虑太多</li> </ul></section><section><h3>B04：游戏性之外的乐趣<a href="#b04游戏性之外的乐趣"><span>#</span></a></h3><ul> <li>操作本身带来的乐趣 <ul> <li>设计者需要对手感拥有敏锐的感觉以及为之调优的能力</li> </ul> </li> <li>冒险游戏和视觉小说 <ul> <li>通常不重视风险回报，重点在于剧情推进。也存在以此为基础从而模糊了类型的作品，比如逆转裁判</li> </ul> </li> <li>电影式 / 剧情为主的游戏 <ul> <li>剧情演出可以成为促使玩家推进流程直面危机的驱动力</li> <li>鲜明的角色和时间观有助于加深玩家对游戏的印象，从而创造出玩家与游戏的内在联系</li> </ul> </li> <li>Licensed Games（指 IP 衍生游戏，如各种漫改、动画改）</li> <li>基于现实世界的模拟游戏 <ul> <li>在屏幕上体验真实世界，越贴近现实，玩家越能感受到比实际呈现在眼前的更多的信心和更深层次的体验</li> </ul> </li> <li>建造类游戏 <ul> <li>建造类游戏的乐趣源于创造，但配合机制带来了风险与回报，就能获得更多乐趣（如建造抵御进攻，冒险收集材料等）</li> </ul> </li> <li>音游 <ul> <li>节奏本身就是一大乐趣</li> <li>根据设计的机制不同，带来了博弈，也可以套用风险回报理论</li> </ul> </li> <li>体育游戏 <ul> <li>体育本身就具有游戏性和观赏性</li> </ul> </li> </ul></section><section><h3>B05：“恶魔之缶”的机制<a href="#b05恶魔之缶的机制"><span>#</span></a></h3><ul> <li>实际上是讲一种有玩家自己选择难度的机制。战斗前投入“恶魔之缶”中的赌注越多，难度越高，收益越大</li> <li>但是高难度带来的高收益会促使玩家选择更高的难度，就无法保证轻度玩家的体验</li> </ul></section><section><h3>B06：允许玩家合法作弊<a href="#b06允许玩家合法作弊"><span>#</span></a></h3><ul> <li>玩家最小化损失和最大化收益的过程就是策略</li> <li>允许玩家作弊，就是说使玩家在不承担任何损失的情况下取得一定的收益</li> <li>作弊不失为一种乐趣，而且令人印象深刻</li> <li>S / L 大法也是一种乐趣</li> </ul></section><section><h3>B07：飞行操作<a href="#b07飞行操作"><span>#</span></a></h3><ul> <li>允许飞行就要允许坠落，允许加速就要允许减速</li> <li>动作附带风险，并成功将其克服是爽点的来源</li> <li>如果角色可以飞行，应使玩家与周遭环境的交互应做到绝对精确</li> <li>为作品增加机制时，也要考虑与之<strong>相反的机制</strong></li> </ul></section><section><h3>B08：需要设置敌人吗？<a href="#b08需要设置敌人吗"><span>#</span></a></h3><ul> <li>给玩家<strong>施加压力</strong>，玩家<strong>释放压力</strong>，玩家获得奖励变得更强，从而<strong>成长</strong>，玩家对后续展开保持期待，并获得满足感</li> <li>玩家应对后续关卡更加得心应手，从而<strong>推进</strong>进度，得以更进一步的探索游戏</li> <li>释放压力、成长和推进时让游戏真正变得好玩的要素，但没有压力，就很难做到这三者</li> <li>压力的呈现有多种形式，并不一定非要通过击杀敌人和游戏机制的设计</li> </ul></section><section><h3>B09：优先考虑奖励要素<a href="#b09优先考虑奖励要素"><span>#</span></a></h3><ul> <li>现在的游戏奖励通常表现为强化、金钱、经验值、技能点等用于提升角色的资源，或是各类收集品、消耗品等等</li> <li>即使没有奖励，进度推进本身也是一种奖励，不过单是推进度已经很难算是激励因素了</li> <li>一旦游戏的框架搭建完成，就应该立刻去思考游戏中的奖励机制</li> <li>游戏必须做到让玩家觉得能从整个过程中收益，且最终获得的物品极具价值</li> </ul></section><section><h3>B10：对战型游戏一定要很复杂吗？<a href="#b10对战型游戏一定要很复杂吗"><span>#</span></a></h3><ul> <li>想让游戏系统经得起长期推敲的话，机制的复杂度是必要的</li> <li>游戏变得复杂不可避免，但不要把习惯当作理所当然</li> </ul></section><section><h3>B11：下落型益智游戏的游戏性<a href="#b11下落型益智游戏的游戏性"><span>#</span></a></h3><ul> <li>垒得越高，风险越大，消除后缓解风险</li> <li>想要一次性消除的方块越多，风险越大，收益也越大</li> <li>设计这类游戏不应只从消除起手，而应思考消除之所以有趣的原因</li> </ul></section><section><h3>B12：令人轻松的死亡<a href="#b12令人轻松的死亡"><span>#</span></a></h3><ul> <li>角色不作为强调对象，死亡作为游戏的机制，而不是可以强调角色本身</li> <li>游戏规则扫清了失望的情绪，比如较为迅速的重开，就减少了沮丧的时间</li> <li>击败对手或许并不是一种回报，如果目标是胜利，击败的对手无法终结掉对局，也就无法放松</li> <li>击败对手或被对手击败，如果这个瞬间的乐趣能打动玩家，那这就是最棒的消遣</li> <li>因此要仔细思考如何让玩家失利后依然继续游戏</li> <li>对战型游戏并非只有主宰对手这一种乐趣</li> </ul></section><section><h3>B13：难度增加但不只是增加<a href="#b13难度增加但不只是增加"><span>#</span></a></h3><ul> <li>玩家和敌人的强度成正比并不是一件好事，也就是说难度曲线不应是线性的</li> <li>可以以登山作为类比，坡度保持不变很是无聊，最好有峰有谷，让玩家克服一些挑战也让玩家有所休息</li> <li>在敌人不断出现的游戏中，提供一个让玩家轻松获得收益的地点是个不错的主意</li> <li>如果游戏中敌人会和玩家同步强化，玩家推进游戏的动力会大打折扣</li> <li>比较理想的结局方案是始终为有趣的对策保留开发空间</li> </ul></section><section><h3>B14：重试务必要快<a href="#b14重试务必要快"><span>#</span></a></h3><ul> <li>减少让玩家感到无所事事的时间</li> <li>重开够快就能提升玩家对难度和阻碍的容忍度</li> <li>只是推进游戏不足以使玩家一直保持兴趣，关键是要上手操作</li> <li>哪怕是加载界面的信息，总之一定要整点东西让玩家互动</li> <li>失败效果的演出可以浮夸，但应该保持简短</li> <li>重生保护</li> </ul></section><section><h3>B15：你的游戏是否值得让玩家不断反复重试<a href="#b15你的游戏是否值得让玩家不断反复重试"><span>#</span></a></h3><ul> <li>死掉就要重开的机制很适合独立游戏，主要是减少素材量</li> <li>为维持玩家推进游戏的驱动力，需要给玩家树立<strong>期待</strong></li> <li>如果吸引力不足或变化不足，玩家就不乐意重复游戏了</li> <li>不要以为自己能打通自己的游戏就万事大吉了，玩家更喜欢提升游玩量并获得新鲜的感受</li> <li>要试着去思考玩家反复游玩同一块内容的感受</li> </ul></section><section><h3>B16：2D 飞行射击游戏的游戏性<a href="#b162d-飞行射击游戏的游戏性"><span>#</span></a></h3><ul> <li>在画面末端射出的子弹速度慢容易打空，而在画面头部射出的子弹速度快、数量多</li> <li>炸弹在敌人、地面附近可以高速发射，反之则慢得多</li> <li>远离 boss 降低风险，发射的子弹少，从而回报下降。而贴脸输出则可以高速开火</li> <li>画面中间容易捡到的道具威力一般，接近地面、障碍物的往往为威力强大</li> <li>要思考游戏中玩家需要接受哪些缺陷，又能获得怎样的优势</li> <li>游戏性也不是一切，不必死磕</li> </ul></section><section><h3>B17：游戏性与公众性的平衡<a href="#b17游戏性与公众性的平衡"><span>#</span></a></h3><ul> <li>过度游戏性也许会削减受众的广度</li> <li>过分复杂的规则会使玩家难以上手</li> <li>有时就是要舍弃掉提升游戏性的机会</li> <li>风险增加的同时，应伴随着克服困难时为玩家带来的更强的愉悦感</li> <li>不同的人群对难易度的感知存在差异，设定目标群体要依据游戏设计和企划而定</li> </ul></section><section><h3>B18：升龙拳的指令<a href="#b18升龙拳的指令"><span>#</span></a></h3><ul> <li>升龙拳的指令（→ ↓ ↘ A），→ 代表靠近对手，风险增大，↓ 下蹲时无法防御空中攻击，↘ 无法摆出任何防御时风险最大</li> <li>升龙拳越早释放威力越大，起手有无敌帧，可以打掉各种攻击，出招时距对手越近，效果越好</li> <li>权衡输入指令的难度、输入时间、对手的速度以及未命中的风险，从而突出博弈感</li> <li>操作困难的指令如果强度太高，会区分使用得出和用不出招式的玩家，因此需要注重平衡性</li> </ul></section><section><h3>B19：线上排位机制的风险与不足<a href="#b19线上排位机制的风险与不足"><span>#</span></a></h3><ul> <li>排名的好处可以给玩家带来一种目标感</li> <li>匹配面太窄会使匹配时间变长，甚至匹配不上。匹配面太宽，玩家水平差距可能更大</li> <li>排名在给玩家以成就感这方面做得并不好</li> <li>模糊最强玩家的身份</li> </ul></section><section><h3>B20：玩乐本质<a href="#b20玩乐本质"><span>#</span></a></h3><ul> <li>竞争，体现为风险回报理论，博弈理论</li> <li>赌博，体现为各种随机要素</li> <li>模仿，模拟，体现为游戏信息量的密度和丰富度、画面表现 / 音效表现等</li> <li>眩晕，体现为抽象的感觉，如害怕、胜利的喜悦等</li> <li>工作生活之外的自我娱乐手段也许是“无意义”的，但正是这“无意义”的经历能帮助自己找到合适的生活方式</li> </ul></section><section><h3>B21：IP 授权改编游戏<a href="#b21ip-授权改编游戏"><span>#</span></a></h3><ul> <li>改编游戏受限于原作的设计，故事设定完全遵循原作无异于剧透</li> <li>改编游戏需要提供的是，对原作的还原度和服务好原作粉丝</li> <li>改编游戏作为 IP 衍生，需权衡产品价值和游戏乐趣</li> </ul></section><section><h3>B22：动作游戏的游戏性<a href="#b22动作游戏的游戏性"><span>#</span></a></h3><ul> <li>跳跃前离平台边缘越近就越能跳过更大的空当，但也更容易坠落</li> <li>冲刺会加快移动速度，但操作难度也变大，会消耗耐力或者撞到障碍物 / 敌人</li> <li>近战而非远程</li> <li>远程射击会消耗资源，且一般射程越远子弹越少</li> <li>闪避可以躲过攻击，但是占有了回合且具有硬直，但闪避或格挡后的攻击命中可以收获回报</li> <li>躲藏，躲藏时被发现的风险很低，但相应的收益也降低了，玩家也可以选择承担风险尝试暗杀</li> </ul></section><section><h3>B23：使模拟工作游戏变得有乐趣<a href="#b23使模拟工作游戏变得有乐趣"><span>#</span></a></h3><ul> <li>取材于现实世界，坚持完成并达成预期会激励玩家继续</li> <li>完成工作会有收益，收益是指工作，赚钱，购买新的工具，获得新的技能使工作更容易完成的游戏循环</li> <li>易于沉浸，前两者结合的协同感，使游戏没有明确的结束节点从而容易使人沉浸</li> <li>平衡工作收益与技能成长</li> </ul></section><section><h3>B24：尽量不要让玩家卡关<a href="#b24尽量不要让玩家卡关"><span>#</span></a></h3><ul> <li>玩家在胜利和失败之间摇摆时，会给玩家以最为强烈的宣泄感</li> <li>玩家的技能存在上限，难度过大可能会使玩家无法通关</li> <li>持续失利后提示降低难度，很伤自尊但好过通不了关</li> <li>允许玩家跳关也好过毫无进展</li> <li>无论游戏挑战多么好玩，但这些乐趣会在玩家放弃时化为泡影</li> <li>允许玩家随时更改难度有很大帮助</li> <li>一开始就上强度也有其乐趣，但不应该默认所有玩家都喜欢这样</li> </ul></section><section><h3>B25：常规策略和玩家自由的平衡<a href="#b25常规策略和玩家自由的平衡"><span>#</span></a></h3><ul> <li>强制性的策略与教程无异，最好不要教玩家玩游戏</li> <li>自由选择策略（装备 / Build）的博弈性也许不强，但也是一种策略</li> <li>允许玩家试错来找到合适的策略</li> </ul></section><section><h3>B26：战略游戏的游戏性<a href="#b26战略游戏的游戏性"><span>#</span></a></h3><ul> <li>如果战场上两支战力相当的队伍相遇，结果通常是势均力敌，否则游戏便无公平可言</li> <li>改变部署、配置从而对对方形成碾压是战略游戏最根本的游戏性</li> <li>攻击力和特性、能力上的差距可以增加游戏的深度</li> <li>在构建过程中为各单位加入独特性，从而构建有着明确优劣势与合理资源消耗的队伍，能使游戏性跟上一层楼</li> <li>战略游戏中有让玩家/敌人随机躲避掉攻击的机制，这种机制使得同样的战斗可以产生不同的结果，提高了玩家的专注度，但也有不公平之嫌</li> <li>棋类游戏的博弈不是一兵一卒的战斗力，而是攻击范围，这点在主打地图炮以及长距离攻击的战略游戏中也适用，关键在于占据有利位置。这类游戏的重点同样是击败势均力敌的对手</li> <li>需要玩家思索或者操作的越多，游戏的策略性和游戏性也就越强，因此需要平衡双方的优劣势</li> </ul></section></section> <section><h2>C：策划 / 游戏设计<a href="#c策划--游戏设计"><span>#</span></a></h2><section><h3>C01：帧率<a href="#c01帧率"><span>#</span></a></h3></section><section><h3>C02：让玩家尽早上手<a href="#c02让玩家尽早上手"><span>#</span></a></h3><ul> <li>过多的前置剧情动画会拖延玩家亲身体验游戏的节奏</li> <li>让玩家先参与游戏，然后再展开剧情，或者使用倒叙</li> <li>早期教程应作出乐趣来，最好不要让玩家明显感知到这是教程</li> </ul></section><section><h3>C03：看得见的反馈<a href="#c03看得见的反馈"><span>#</span></a></h3><ul> <li>给出可达的目标，让玩家体验完成目标的乐趣</li> </ul></section><section><h3>C04：如何进入游戏行业<a href="#c04如何进入游戏行业"><span>#</span></a></h3><ul> <li>尝试自己做游戏</li> <li>不妨从模仿开始，重要的是在这个过程中积累解决问题的经验</li> <li>做完给别人玩，收集玩家反馈</li> </ul></section><section><h3>C05：显示器的视野范围<a href="#c05显示器的视野范围"><span>#</span></a></h3><ul> <li>玩家在电视机前时，视野范围约为 30°</li> <li>平衡游戏画面中的目标要素在实际屏幕中所占比重</li> </ul></section><section><h3>C06：单个按键的潜力<a href="#c06单个按键的潜力"><span>#</span></a></h3><ul> <li>单键小游戏可分四种：快速连点、按键时机、反应时间、交替按键</li> <li>按键时机：在特定时间按下或释放按键</li> <li>反应时间：收到游戏提示时立刻按下按键</li> <li>交替按键：关注何时按下何时释放</li> <li>简单的操作往往能成为最优解</li> </ul></section><section><h3>C07：解析、分析与重建<a href="#c07解析分析与重建"><span>#</span></a></h3><ul> <li>分析一款游戏之所以有趣的原因，再将这些元素重构为自己独特的想法</li> <li>解析出游戏好玩的要素</li> <li>采用风险回报理论分析为何好玩</li> <li>尝试与其他元素融合，重建为新的玩法</li> </ul></section><section><h3>C08：跑图任务<a href="#c08跑图任务"><span>#</span></a></h3><ul> <li>给足奖励（重要或有用游戏物品或是有趣的剧情）</li> <li>提高跑图中获取乐趣的频率</li> <li>给玩家提供多种路线选择，比如危险的进路或是安全的远路</li> <li>达成目标后提供返回捷径</li> <li>营造悬念使玩家对后续展开产生期待，比如世界观的设定或是谜团</li> </ul></section><section><h3>C09：估算帧数<a href="#c09估算帧数"><span>#</span></a></h3><ul> <li>1·2·3· 估算帧数</li> <li>15 帧是很多接受指令到实际触发指令的时间，招式的速度感大致如此</li> <li>使用可动人偶模型帮助湍动将动作用视觉方式展现出来</li> <li>计算招式的帧数时长</li> </ul></section><section><h3>C10：让教程自然呈现<a href="#c10让教程自然呈现"><span>#</span></a></h3><ul> <li>让玩家面临切实的挑战，通过解决问题来学习玩法，也能让玩家更投入从而激发兴趣</li> <li>呈现方式有多种，在屏幕上直接显示操作指示、放到游戏道具上供玩家查看、直接显示在 UI 上等</li> <li>显示按键时，跨平台的影响</li> <li>教程不应拖慢游戏的节奏</li> </ul></section><section><h3>C11：夸赞玩家<a href="#c11夸赞玩家"><span>#</span></a></h3><ul> <li>大乱斗击飞时的喝彩</li> <li>玩家表现好的时候给予鼓励，并不是游戏内的物品奖励，奖励和流程是应得的</li> <li>在表现事物时要尽量避免只是单纯的呈现，要尝试匹配玩家体验</li> </ul></section><section><h3>C12：按键一定要有响应<a href="#c12按键一定要有响应"><span>#</span></a></h3><ul> <li>做出操作以及获得反馈（视觉、听觉或触觉等</li> <li>按下按键的反馈要易察觉</li> </ul></section><section><h3>C13：游戏写作<a href="#c13游戏写作"><span>#</span></a></h3><ul> <li>对于想爽玩游戏的玩家而言，剧情有时会拖节奏，如果动作游戏这么做会使玩家厌烦</li> <li>绝大多数情况，剧情的展开要以玩家操作的主要角色展开</li> <li>战斗、敌对情节中，主角必胜。因情节发展来否定玩家成果的做法可能会使玩家讨厌</li> <li>同伴不应该在剧情中死亡或者离队</li> <li>根据实际情况，上述四条可以无视，可能会惹恼部分玩家，但也因此会让人印象深刻</li> <li>现代游戏剧情中很重要的一点是，先让玩家上手玩，然后才是剧情，序章要避免大段的背景介绍</li> </ul></section><section><h3>C14：选边站队的乐趣<a href="#c14选边站队的乐趣"><span>#</span></a></h3><ul> <li>在双方决胜的比赛中，选择支持的一队，更容易让人投入其中</li> <li>任何能让人欢呼喝彩的，都有其积极意义</li> <li>如果观战可以下注，那么游戏本地化时需要考虑各国对于界定赌博的相关规则</li> <li>如果给玩家多个卡池选择，会比之用一个卡池随机抽取的效果要好</li> </ul></section><section><h3>C15: 为何会晕 3D<a href="#c15-为何会晕-3d"><span>#</span></a></h3><ul> <li>眼睛观察到的信息（环境、画面）和内耳接收到的（速度、摇晃）不匹配</li> <li>晕车的时候要多看看远方</li> <li>玩游戏的时候，速度本身不会发生变化，游戏画面的运动会有一种强烈的惯性感，惯性感和接收信息的差异会令人不舒服</li> <li>拉开视野是个避免晕 3D 的方法，因为拉大视野会告诉大脑没有东西在动</li> <li>游戏画面制作的柔和些，避免产生剧烈晃动，可以避免晕 3D</li> <li>驾驶游戏的第一视角，避免车内部的晃动也可以避免</li> <li>增加画面中的信息，间接告诉玩家画面不是真的在动，至于怎么加，需要权衡</li> </ul></section><section><h3>C16：大纲视图<a href="#c16大纲视图"><span>#</span></a></h3><ul> <li>整理界面层级 / 角色技能时，用来呈现需求会更直观</li> <li>个人用来梳理思路的好方法</li> </ul></section><section><h3>C17：奖励要素的设计<a href="#c17奖励要素的设计"><span>#</span></a></h3><ul> <li>制作游戏时就应该考虑给予玩家的奖励要素，因为不可能什么都给玩家，因此在立项阶段就应该考虑奖励要素，并思考以何种方式呈现</li> <li>最有效的奖励便是让玩家随着进度的推进而逐渐变强，如商店物品种类增加、更好的装备、经验值技能点、宝箱和掉落、素材炼成等</li> <li>不知道在下个区域有怎样有趣的新内容的期待，很能激发玩家继续游戏的欲望</li> <li>也可以引入使玩家失去得到的能力/物品的机制来扩展游戏深度</li> <li>精致的收集物也是很好的奖励要素，但是人工成本和时间成本都很高（大乱斗 SP 的贴纸、手办）</li> <li>成就/奖杯模式，呈现一个像是玩家游戏经历的博物馆或成就墙，现代游戏一般会接入相应平台的成就系统</li> </ul></section><section><h3>C18：避免对比<a href="#c18避免对比"><span>#</span></a></h3><ul> <li>没有参照物对比，理解新事物可能存在困难</li> <li>初期策划阶段很难准确描述一个作品，所以策划要对游戏的全貌有一定认知</li> <li>很多人会以现有的游戏来做你的项目的参照，但是企划中不应涉及太多相关现有游戏的要素，原因有二</li> <li>其一，项目如果和现有作品相似，完全可以玩原作。</li> <li>其二，对不同年龄段和不同经历的人而言，直接的对比会令人产生不同的理解</li> <li>同系列续作更应该用已有的作品来当作卖点，自己的企划书更应该突出自己的亮点</li> </ul></section><section><h3>C19：续作的开发<a href="#c19续作的开发"><span>#</span></a></h3><ul> <li>能成为系列的游戏，底层系统的开发一定足够完善</li> <li>开发续作可以在已有的底层系统上做改进、扩展原有系统或者增加游戏容量</li> <li>前作的质量会影响续作的销量</li> <li>最终要的是，让玩家相信续作比前作乐趣更多、体验更好</li> </ul></section><section><h3>C20：约定俗成的操作方式<a href="#c20约定俗成的操作方式"><span>#</span></a></h3><ul> <li>如果玩家对游玩方式非常熟悉，游戏可以省略各种解释（FPS）</li> <li>一般来说，开发者应避免照抄其他作品，但游戏业界也存在着“标准答案”</li> <li>玩家有以自己喜爱的方式游玩的自由，因此按键映射这种基本的功能是必要的</li> <li>但是操作如果严格标准化，或许会丢掉游戏独特的乐趣</li> </ul></section><section><h3>C21：自定义机制<a href="#c21自定义机制"><span>#</span></a></h3><ul> <li>为了做出好的成果，要预先想一下其他人对它的反应</li> <li>对于高度自定义的游戏，开发者不仅要想到最终的结果，还要考虑自定义流程和 UI 设计（内容和方式</li> <li>应当给予玩家自定义结果的提示</li> <li>解释清楚各组件的效果</li> <li>提示新的组件与现有组件的孰优孰劣</li> <li>各部分组件可能过多，因此组件要能在界面中可以轻松找到（分类/预设</li> <li>快速试验自定义效果</li> <li>设计酷炫</li> </ul></section><section><h3>C22：制作相仿的游戏<a href="#c22制作相仿的游戏"><span>#</span></a></h3><ul> <li>模仿某个游戏时，要时常考虑，玩家为何不去玩原作？因此也要具备一些原作所没有的独特魅力</li> <li>当拥有了更多独特有趣的点后，就有可能自成一派</li> </ul></section><section><h3>C23：分数的呈现形式<a href="#c23分数的呈现形式"><span>#</span></a></h3><ul> <li>现代的游戏很少有用分数来作为激励手段，但开发者依然要尽可能的让玩家取得高分后感到愉悦</li> <li>游戏之所以好玩正是因为有这种令人竭力追求的东西</li> <li>决定分数的变量可以是某种单位，如年龄、战斗力、偏差值、金钱等等</li> <li>最好不要直接使用得分</li> </ul></section><section><h3>C24：用参数来塑造角色特质<a href="#c24用参数来塑造角色特质"><span>#</span></a></h3><ul> <li>游戏角色应在游戏规则范围内，有一套相对合理的运行方式，关键在于为游戏创造出更多可能性</li> <li>无论是可操纵角色还是敌方角色，给角色增加<strong>独特性</strong>，可能添加策略性</li> </ul></section><section><h3>C25：对色觉障碍玩家的关怀措施<a href="#c25对色觉障碍玩家的关怀措施"><span>#</span></a></h3><ul> <li>如果“颜色”这一要素在游戏中占有很大比重，那么可能会使得色觉障碍玩家无法正常游玩</li> <li>可以使用 Photoshop 内置的色盲滤镜进行检查，对容易混淆的颜色可以用亮度加以区分</li> <li>场景内可互动物品也可能会难以分辨</li> <li>使用灰度滤镜检查场景内物品的明暗程度和前后景层次关系</li> </ul></section><section><h3>C26：电脑控制的角色<a href="#c26电脑控制的角色"><span>#</span></a></h3><ul> <li>电脑的反应如果超过真人，会有极强的不公平感</li> <li>参照目标数据信息进行决策</li> <li>当电脑处于优势时，简化电脑的逻辑，以平衡玩家</li> </ul></section><section><h3>C27：游戏内的过场动画<a href="#c27游戏内的过场动画"><span>#</span></a></h3><ul> <li>时间长的过场动画一定要有明确意义，毕竟在实际游戏中可能很难得到像看电影一样的微妙时刻</li> <li>快节奏的动作游戏中加入过场动画可以给玩家喘息的机会</li> <li>过场动画也可以作为一种奖励，但是在流媒体时代，重剧情游戏的过场动画给予的激励程度有限</li> <li>能跳过或快进对话的话，对玩家会友好许多</li> <li>玩家有跳过过场动画的权力和跳过 Credits 的权力</li> <li>在过场动画中加入 QTE 在当下已经不流行了，因此加入 QTE 时要慎重考虑</li> </ul></section><section><h3>C28：以大场面开局<a href="#c28以大场面开局"><span>#</span></a></h3><ul> <li>游戏开场的节奏不应拖沓，可以先将高潮部分的大场面端出来，再在后续讲走到这一步的过程</li> <li>玩家打开新游戏时会充满期待的探索一番，这时候再闲庭信步就不好了</li> <li>尽早将游戏的控制权交给玩家，在一开始就将玩家调动起来</li> <li>最好在故事的开头就把调子起高，假如因为游戏结构做不到这点，也要让开场的节奏紧凑一些</li> </ul></section></section> <section><h2>D：游戏机制<a href="#d游戏机制"><span>#</span></a></h2><section><h3>D01：画面定格<a href="#d01画面定格"><span>#</span></a></h3><ul> <li>画面定格可用于打击、击败 boss 和玩家落命时</li> <li>定格表现可让玩家知道上述事件发生时的具体状况，不会不明不白的死掉</li> <li>定格效果可以使玩家直观的感受到胜利与失败</li> <li>常用的表现形式有，停顿、镜头晃动和慢动作</li> <li>可以注意下 BGM 在这个过程中的处理方式</li> </ul></section><section><h3>D02：打造操作手感<a href="#d02打造操作手感"><span>#</span></a></h3><ul> <li>按下同样的按键，根据游戏的实际变现或设计机制存在手感上的“轻重”之分</li> <li>角色的移动速度、惯性、跳跃的物理、延迟、帧率都会影响手感</li> <li>即使按下相同的按键，画面上的呈现的结果是不同的，考虑到手感这一点，设计时就要考虑的额外的要素</li> <li>针对不同的平台的手柄，调试不同的敏感度参数</li> </ul></section><section><h3>D03：跳跃机制<a href="#d03跳跃机制"><span>#</span></a></h3><ul> <li>重点是要设定跳起后位于空中时落地方向的加速度，实际情况中还要考虑惯性，最大速度等，具体调试参数要试游戏的实际情况而定</li> <li>大乱斗SP 的角色存在一个起跳准备帧（windup），假如去掉该帧，会误判玩家的操作，因此需要使用起跳帧规避</li> <li>为角色提供较大的起跳速度，然后逐渐增大落地加速度，到达最高点后，落地加速度变为默认值，如此这般增加了滞空时间，从而为玩家增加了反应时间（动态的重力加速度？）</li> </ul></section><section><h3>D04：大乱斗 SP 的受击停止（HitStop）设计<a href="#d04大乱斗-sp-的受击停止hitstop设计"><span>#</span></a></h3><ul> <li>提高受击角色的晃动程度，出击角色只需轻微晃动</li> <li>不要移动 hitbox，容易使连招打空</li> <li>地面做水平晃动，空中可以自由晃动，避免可能产生的卡地板的问题</li> <li>晃动幅度的衰减，应依据晃动的幅度、持续的帧数来设置合适的衰减（阻尼）</li> <li>控制 hitstop 的数量，一般越强力的招式 hitstop 持续的帧数越多</li> <li>插入动作转化（伤害）帧。通常在 hitstop 时，受击者的动作不变。大乱斗sp 在受击帧中额外添加了不同的受击动画，而不是单纯地冻结动作，以4帧左右的动画过渡到受击动画</li> <li>Hitstop 期间持续使攻击方进行微小的位移。hitstop 的过程中，进攻方的动作幅度较低。在 hitstop 完成后，再恢复原有的动作幅度（针对攻击者的时间放缓？），应视不同的招式控制开启与否。此外，控制电气相关的招式延长 hitstop，多人对战时减少 hitstop</li> <li>晃动幅度视相机距离而定。相机较远时，晃动幅度应明显一些，使镜头的远近不影响玩家对打击的直观感受</li> </ul></section><section><h3>D05：3D 游戏的角色中心点<a href="#d053d-游戏的角色中心点"><span>#</span></a></h3><ul> <li>相机中心点跟随角色时，可能会难以分辨角色的运动，因此中心点应当不跟随角色，或者仅沿运动轴低幅度运动</li> <li>锁定视点适用于部分游戏，比如写实的第三人称竞速游戏</li> <li>对于提供更广视野的横向卷轴游戏，镜头会移动到玩家前方，更多展现玩家前方的场景</li> </ul></section><section><h3>D06：规避慢热的问题<a href="#d06规避慢热的问题"><span>#</span></a></h3><ul> <li>“慢”指的是游戏节奏，部分过于耗时间的东西、让玩家失去目标无所事事，没能真正让玩家去玩游戏而被浪费了的时间</li> <li>为了带了紧凑的游戏体验，要尽可能的减少加载时间</li> <li>剧情和过场动画可以让玩家稍作放松，但如果一些过场只是在浪费时间，那么砍掉也无妨，现在的游戏更倾向于短促紧凑的过场</li> <li>“让玩家等待是一种罪过！”</li> </ul></section><section><h3>D07：速度的单位<a href="#d07速度的单位"><span>#</span></a></h3><ul> <li>像素游戏可以用 Pixel/Frame，即每帧移动了多少个像素来表达速度</li> <li>3D 游戏的环境考虑了现实物件的比例，因此 3D 游戏中可使用现实的速度单位如 km/h</li> <li>根据实际游戏情况，还是可能会使用像是 米/帧，厘米/帧 之类</li> </ul></section><section><h3>D08：边界跌落行为<a href="#d08边界跌落行为"><span>#</span></a></h3><ul> <li>角色在边界直接跌落</li> <li>让角色因自身的移动而跌落，而非通过招式动作触发。当角色处于边界时，由于使用了具有朝边界方向位移的招式而跌落会使得招式输入直接取消，这点对动作游戏并不友好</li> <li>允许特定招式在使用过程中触发边界跌落。比如玩家朝边界闪避不会跌落，而使用朝边界方向冲刺的攻击招式会触发跌落</li> <li>有些 3D 游戏里会使玩家在边界运动时再次输入边界方向的操作指令而是角色掉落，也有些会在边界与运动方向相对平行调整玩家角色的运动方向，也有些游戏甚至会在玩家朝边界运动时自动触发跳跃行为</li> <li>开发人员务必不要在游戏试玩测试时忽略操作感</li> </ul></section><section><h3>D09：地图也是游戏屏幕<a href="#d09地图也是游戏屏幕"><span>#</span></a></h3><ul> <li>如果让玩家在赶路时过多关注小地图，而错过路上的风光和事件，未免因小失大</li> <li>在游戏画面上提供更多的信息量</li> <li>通过安排布置地形来引导玩家绕到有趣的地方</li> <li>开发者需要考量玩家在游玩时会看向画面的何处、注意力放在画面哪里，更重要的是预测玩家的所思所想</li> </ul></section><section><h3>D10：预料之外的结果<a href="#d10预料之外的结果"><span>#</span></a></h3><ul> <li>由于硬件环境不同，分发的版本可能会有不同的性能表现，可能会出现由于算力不同而使难度曲线发生变化的情况</li> </ul></section><section><h3>D11：摇杆快弹输入<a href="#d11摇杆快弹输入"><span>#</span></a></h3><ul> <li>摇杆的快弹可对应冲刺、闪避这种需要快速反应的行为</li> <li>但快弹的操作因人而异，需要提供调整灵敏度的选项</li> </ul></section><section><h3>D12：新手教程<a href="#d12新手教程"><span>#</span></a></h3><ul> <li>不论游戏有没有特别的操作方式，一段拆解游戏规则的教程都是必要的</li> <li>最糟糕的教程就是让玩家像阅读说明书一样阅读一大段文字</li> <li>略好一点的方案是让玩家在训练情景下学习如何操控</li> <li>还有一种常见的做法是把文字教程沿关卡中的道路一路放置过去，让玩家读到之后就可以立刻操作实践</li> </ul></section><section><h3>D13：用随机性为游戏增彩<a href="#d13用随机性为游戏增彩"><span>#</span></a></h3><ul> <li>游戏中的内容循环往复，适当添加随机性可让玩家不会很快就产生玩腻了的感觉</li> <li>比起一成不变的结果，改变概率就可以给玩家的决策添加趣味和新鲜感</li> <li>但是如果随机性成分过重，会使策略性大大降低，使得玩家的努力取得的成果小于运气取得的成果</li> <li>如果失败的原因是运气，会使玩家沮丧。需要让玩家在游戏中感知到一点，就是自己为自己的失利负责</li> <li>随机道具投放可以在一定程度上平衡实力差距</li> <li>可以以系统时间作为生成随机数的种子</li> </ul></section><section><h3>D14：让步/让分系统<a href="#d14让步让分系统"><span>#</span></a></h3><ul> <li>为了应对对局者水平差距过大的问题，部分游戏引入了让步系统</li> <li>大多数竞速游戏，对于排名靠后的玩家会有一定的速度加成。马里奥赛车中的大子弹头也是类似的机制，这种系统在休闲游戏中可以说是必备的</li> <li>水平高的玩家可以一路赢到底的游戏并不有趣，如果仅是休闲游玩，还是让大家都能获得乐趣更好</li> <li>在线对战另说</li> <li>会使场面更加混乱的随机性要素，如必杀、道具、场地等，很适合派对游戏</li> </ul></section><section><h3>D15：平均 = 平凡<a href="#d15平均--平凡"><span>#</span></a></h3><ul> <li>依照角色的强度来做平衡，会使角色的能力趋于稳定，但在避“短”的同时，使“长”也遭到削减，结果就是趋同</li> <li>可以通过“扬短”的方式来抵消优势，从而凸显角色特点</li> <li>大有不同的双方进行博弈将极大提升游戏的策略广度</li> <li>互联网是一个回音室，一旦有人发表或好或坏的评价，这种评价会形成一股势力，使相信的人更加深信不疑。但如此价值观上的差异会产生不同的选择，从而带来更丰富的体验</li> <li>如果去掉个性化的特质，那么游戏的广度也会失去意义</li> <li>中规中矩的东西难言魅力，平均和平凡是一码事</li> </ul></section><section><h3>D16：在线更新<a href="#d16在线更新"><span>#</span></a></h3><ul> <li>现代的游戏变得无比复杂，庞大的数据量会导致的不稳定因素比单纯的代码要多得多</li> <li>因为工期的原因，release 前的一段时间可能会有较多改动，每个改动都要重新评估游戏流程的话很费时间，因此开发过程中很难玩到最终的成品游戏</li> <li>市场是最好的漏洞测试人员，QA 阶段没法保证找到每一个 bug。对于极低概率触发的 bug，可能会有玩家发现并上传视频</li> <li>可以很便捷的调整平衡性和加入新特性，因此在线更新是一种红利</li> <li>但是站在玩家角度，如果玩家真的放弃一款游戏，那么更新也失去了意义，因此要尽可能的在发售时就能达到最佳状态</li> </ul></section></section> <section><h2>E：团队运营<a href="#e团队运营"><span>#</span></a></h2><section><h3>E01：信息共享<a href="#e01信息共享"><span>#</span></a></h3><ul> <li>在大项目做事，还难以窥得全貌的工作，是有一定压力的。因此在团队内与同事共享工作进度，可以排解压力。</li> <li>就改进沟通效率来说，有以下的三点建议 <ul> <li>每日报告，可以用线上的方式分享每日的进度、吐槽，让工作人员进行沟通交流，（感觉像是项目内部的朋友圈</li> <li>每日一图，每天发一张项目进度截图，有开发人员参与吐槽，也算是一种调剂心情的手段</li> <li>项目汇报，每隔几个月要求各开发团队汇报进度，内容包括画面、音效、特效、界面等，通过汇报可以让每位开发人员的进度变得公开，使每个人都能更宏观地了解整个项目，更能产生主人公意识，也可以及早避免开发走向死角</li> </ul> </li> </ul></section><section><h3>E02：岗位分工<a href="#e02岗位分工"><span>#</span></a></h3><ul> <li>总监，总监需要提出并整合游戏的设计思路，并拥有最终决策权，该职位需要监制当前项目的所有工作，并在必要时做出调整，如果这名总监还要构思设计游戏规则与体系，也可以称他为游戏设计师</li> <li>策划，策划需要将总监给出的设计思路写成下发给团队各个部门的设计文档或者说需求表，其中需要包含游戏编码所需的详细资料，此外，策划还可能会涉及到团队管理、平衡性调整、监制测试、有时候还要编写实现脚本等</li> <li>程序，程序要将需求表上的内容具体化为计算机可处理的指令，将各个子项目组的工作整合并予以实现</li> <li>3D 建模，使用图形工具来构建游戏中的物体，具体指角色、场景、敌人、道具等，通常团队中有很多建模师，但每个人的工作内容差别较大</li> <li>原画，绘制 2D 美术图，如场景概念图、角色立绘等</li> <li>动画设计，制作使建模活动起来的动画效果，不仅仅是制作拟真的动作，还要考虑夸张度、丰富性以及符合角色性格，还有细分的制作建模内部框架以实现动画效果的绑定师</li> <li>特效设计，制作视觉特效，与其他设计师的区别在于要对动画效果有一定的感性认识</li> <li>音频设计，可分为作曲、音效设计以及配音</li> <li>界面设计，UI 团队里也可以有独立的策划、美术以及程序</li> <li>运维，负责管理服务器，维护开发环境，为团队内的成员提供技术支持</li> <li>项目经理，把控进度，掌握、解决开发过程中的相关问题，分配人力资源，还要负责一些类似办公室主任一样的硬件采购的工作</li> <li>游戏测试，负责试玩游戏并指出游戏中的问题或者是需要调整的地方，反馈给策划团队进行评估</li> <li>漏洞测试，主要负责找 bug，并编写错误报告阐述如何复现 bug，然后由项目经历指派人员来解决问题，直到修复完毕</li> <li>影像制作，录制、剪辑实机画面</li> </ul></section><section><h3>E03：意见要及时反馈<a href="#e03意见要及时反馈"><span>#</span></a></h3><ul> <li>身怀不同技能、观点以及审美的人会注意到各不相同的问题</li> <li>假如对游戏的某个模块不满意，最不该做的就是在团队中保持沉默，然后在做完之后才说我早就觉得如何如何，不管想法是否会让项目发生改变，干等只会错失良机</li> <li>马后炮只会影响开发团队的心态，对项目本身也没有任何正面意义，因此有想法就要分享给团队，说不定就可以补救一些问题</li> <li>不要怕没人理睬，真正要做的是向负责对应部分的人清晰表达自己的想法，但大家提出质疑的时候，最好也想一想如何改进，并给出可能的解决方法</li> <li>总之就是不要只关注问题本身而忽略解决方案，如此便可营造一个富有建设性的开发氛围</li> <li>即使经过团队内讨论后，没能根据意见修改，但也可以听取大家的想法，可以帮助你更好的理解和接受</li> <li>确实有人单纯的不爱开口表达，但让大家听取自己的想法，这些思考就可能会切实影响游戏成品的表现，团队中的每个人都拥有影响游戏开发的能力，因此切忌沉默是金</li> </ul></section><section><h3>E04：10 人产出 7 人工<a href="#e0410-人产出-7-人工"><span>#</span></a></h3><ul> <li>让团队成员专注于自己的工作没有问题，但这样很难称为一个团队</li> <li>有能的员工在担任领导职务时，通常无法持续输出和以前相同的工作成果，老练的创作者去担任管理后，通常无法继续产出作品</li> <li>理想状态就是能干的员工称为团队的大脑，去决定开发方向并把控质量</li> <li>小型团队可以让成员专注自己的工作，但规模越大越要重视管理</li> <li>为使成员对所做项目有宏观认识，可以开团队会议，由各个小团队展示工作成果</li> </ul></section><section><h3>E05：一次性向所有人阐述观点<a href="#e05一次性向所有人阐述观点"><span>#</span></a></h3><ul> <li>线性的自上而下的传达任务，反馈的结果有可能差之千里</li> <li>企划、程序、建模、动画、特效等各团队领导者可以凑在一起开会讨论传达事项</li> <li>各团队在组内开会复盘，确保所有人步调一致，这样就不是所有人都能够明白总监所传达的内容</li> <li>这套管理方式可以让各个团队的进度变得清晰</li> <li>为保证效果，需要把会开得简明扼要</li> <li>开会的人可以买点零食，边吃边开会</li> </ul></section><section><h3>E06：每日一图<a href="#e06每日一图"><span>#</span></a></h3><ul> <li>与团队成员共享自己的开发日报</li> <li>在这个日报中置顶一张图，并延申来探讨若干个话题</li> <li>话题可以是项目的最新进展和当前的工作内容，也可以附上 check list 和要修复的内容</li> <li>也可以分享最近玩的游戏、事件和所思所得</li> <li>樱井桑这么做是因为他是受雇佣的外来者，这么做可以快速让团队成员了解他</li> <li>大型团队中每个人的工作只是开发全局的一小部分，这有可能会让成员失去队友作品的大局观，因此让成员看到他人、团队整体的进展成果还是有必要的</li> <li>意见版也很有用</li> </ul></section><section><h3>E07：Bug 测试<a href="#e07bug-测试"><span>#</span></a></h3><ul> <li>只有 bug 少，不存在没有 bug 的游戏</li> <li>哪怕是打补丁修复游戏漏洞，因为修好一个问题，可能会有其他问题出现，每修一次就要再过一遍别的地方</li> </ul></section><section><h3>E08：作为公司的一员制作游戏<a href="#e08作为公司的一员制作游戏"><span>#</span></a></h3><ul> <li>玩家觉得游戏没意思，可能有很多原因，卖相不好、标题平淡、缺少宣传、宣发时机等等都会影响最终得销售结果，但不管结果如何，职员依然能从游戏制作中获得薪水，所以也不全是坏处</li> <li>这个过程中，公司是承担游戏开发最大风险的个体</li> <li>公司要做的是平衡风险和回报，确保员工能专心于自身的工作</li> <li>上班是今日助人解忧，他日恩恩相抱，所以最好还是抱着积极的心态，尽量和他人一同协作吧</li> </ul></section><section><h3>E09：今日事今日毕<a href="#e09今日事今日毕"><span>#</span></a></h3><ul> <li>积压的邮件在当日回复，快速反应才能快速迭代</li> <li>没有做完的事情积压到以后会更加痛苦，因为第二天有第二天的工作</li> <li>可以给自己定一个时间，在这特定时间中，不管外界的纷扰，专心自己的工作</li> </ul></section><section><h3>E10：做决策要当机立断<a href="#e10做决策要当机立断"><span>#</span></a></h3><ul> <li>假设在开会讨论某项决定，但无法达成一致结果，也不要直接散会“下次再说”</li> <li>除开有关责任方不在场的情况，倘若有人可以做决定，务必在开完会后给个准话</li> <li>总监的每个决定都会引导开发团队走向长期的开发路径之中，做错一个决定就会浪费很长时间，所以慎重决策是可以理解的，但是实际情况往往是没有那么多让开发团队犹豫的时间</li> <li>迅速而坚定的决策对开发团队有许多好处，团队里的成员也应尽力促成决策的达成，毕竟下决策的人需要尽可能多的信息</li> <li>保持决策的灵活性，要给纠错留出余地</li> </ul></section><section><h3>E11：Bug 测试系统<a href="#e11bug-测试系统"><span>#</span></a></h3><ul> <li>保证 QA 测试最新的版本</li> <li>QA 测试时根据设计文档，将各个测试单元进行排列组合，或分成小组来检查特定部分的内容，以确保覆盖所有的测试内容</li> <li>找到 bug 并知道如何复现后，通过工单进行报告</li> <li>项目经历将 bug 指派修复人员，并设定优先级，通常从高到低为： <ul> <li>S：导致游戏卡死/崩溃，存在损坏存档可能的</li> <li>A：阻碍玩家推进游戏流程</li> <li>B/C：较低优先级可以暂时搁置，甚至可被有意搁置</li> </ul> </li> <li>修复完成后的工单返回给 QA 测试无误后，任务完成，工单转为 Fixed 归档</li> <li>大型项目仅靠人工测试无法涵盖尽可能多的情况，所以要使用脚本挂机测试、试玩测试，总之测试的过程中，找到的 bug 越多越好</li> <li>Bug 是无穷尽的，所以目标不是消灭所有 bug 而是<strong>快速定位，尽快修复</strong></li> </ul></section><section><h3>E12：永无止境的测试<a href="#e12永无止境的测试"><span>#</span></a></h3><ul> <li>对战型游戏，通过亲自与人对战来得出哪些角色强度过高，哪些强度过低或没有特色，然后进行调整</li> <li>能让每个角色都能自成一派，是最好的调整方案，让玩家的观点发生分歧也无妨，这也是乐趣的一部分。但看某一方面，游戏总是不平衡的，与其如此，不如让角色的个性更加鲜明</li> </ul></section><section><h3>E13：总监与制作人<a href="#e13总监与制作人"><span>#</span></a></h3><ul> <li>二者的职能在不同的公司或项目之间或许会存在差异</li> <li>总监对于游戏内容的几乎方方面面都拥有最终决策权，总监需要从理念上把控游戏的核心吸引力，规划游戏的走向，尽量让大家减少意见分歧和目标不明确的状况。如果把职能拆分开来，需要明确谁具有最终决策权</li> <li>制作人掌握的是团队管理以及市场宣发等方面，需要和其他部门协作来解决融资、管理以及开发团队的人力配置等问题，要精确计算以及对比实际开发成本和预期销售收入，以使团队的效益最大化</li> <li>制作人需要参与游戏的宣发环节，因为通常总监忙于开发团队内部的事务，因此制作人需要负责部分市场和公关的事务。视公司、项目而定，二者的职能区别并不是绝对的</li> <li>通常一个项目只有一个总监，而一个制作人可能会参与不同的项目。总监要对游戏的质量负责，制作人需要从公司层面管理项目进度。</li> </ul></section></section> <section><h2>F：图形画面<a href="#f图形画面"><span>#</span></a></h2><section><h3>F01：重视打光而非绘制资产<a href="#f01重视打光而非绘制资产"><span>#</span></a></h3><ul> <li>不要过分绘制素材，要表现出光影效果</li> <li>添加了光照和阴影的资产会更有区分度，在强光区域加上雾化效果后表现力会更强，大气雾运用得当可以给画面带来深度与距离感</li> <li>添加光源，色调多样化，加入氛围，其他后处理</li> <li>如果要求做一棵树，那就不能使只做棵树就收工，尽量不要只画出素材就结束工作，最重要的是要表达出素材环境里的光线反射</li> </ul></section><section><h3>F02：突出存在碰撞判定的区域<a href="#f02突出存在碰撞判定的区域"><span>#</span></a></h3><ul> <li>制作场景时需要重视存在碰撞体积的对象，而不是强调无碰撞体积的对象</li> <li>对于动作游戏，能让玩家清楚地明白哪些地方可以让角色站立活动是很重要的</li> <li>对于 3D 平台，如大乱斗，碰撞区域会变得难以分辨，因此需要把地形元素和非地形元素区分开来，避免背景喧宾夺主，分散玩家的专注度</li> <li>一种比较简单的方案是，将无碰撞体积判定的区域整体做淡化处理（颜色淡化、DOF），从而降低背景的存在感</li> </ul></section><section><h3>F03：分清画面主次<a href="#f03分清画面主次"><span>#</span></a></h3><ul> <li>开发者需要分辨作品中哪些元素能产生更正面的影响，哪些要素产生较少的影响，从而高效分配资源</li> <li>对游戏进行精简是一项基本技能，不应仅局限于图形层面</li> </ul></section><section><h3>F04：素材比例要精准<a href="#f04素材比例要精准"><span>#</span></a></h3><ul> <li>空气透视，环境光，光追，玩具相机效果和景深等画面效果会显著影响尺寸感</li> <li>素材的绘制尺寸应尽可能大，并且要从远处进行观察</li> <li>如果贴图的颜色过分饱和，环境光对其的影响也会变得有限（变平），写实的作品，饱和度就不宜太高，可在整体世界搭建完成后再调整这些细节</li> <li>过度高光会产生塑料感，不过限制光照面积和光照强度会带来截然不同的感受</li> <li>如果法线贴图对不上素材的尺寸，最终光照效果会很差</li> <li>比例缩小的模型颜色会更饱和，高光更强更广，受环境光的影响小，材质显得更平</li> </ul></section><section><h3>F05：监制图片二次处理<a href="#f05监制图片二次处理"><span>#</span></a></h3><ul> <li>监修图片很难用言语表达出想要的效果，因此有时直接改图比语言的描述更好用</li> <li>一个方便的调整方法是在 Photoshop 里将原画单独放到一个图层，拉高对比度，调节色调，用橡皮擦擦掉光线照到的调整过色调的那一层，最后再调整整体颜色</li> <li>玩家要关注的东西、主体、路线和敌人的对比度要比场景高，画面前后关系要明确</li> </ul></section><section><h3>F06：大乱斗 SP 的场景<a href="#f06大乱斗-sp-的场景"><span>#</span></a></h3><ul> <li>大乱斗 SP 的场景展示</li> </ul></section><section><h3>F07：像素画基础<a href="#f07像素画基础"><span>#</span></a></h3><ul> <li>奇数大小的长宽/直径会显得更居中</li> <li>对灰度像素图，添加比边缘更浅的颜色来软化边缘，添加第三种颜色来表示反射</li> </ul></section><section><h3>F08：统一视觉风格<a href="#f08统一视觉风格"><span>#</span></a></h3><ul> <li>贴图板，贴图板上有各种展示世界观、场景、角色、道具等元素的美术作品，大多数项目中不会有太多这类原画，并且这些作品通常由团队里最出色的美术创作而成</li> <li>团队以这些美术作品为基础进行建模，以及锚定光影，以此消除不同美术风格差异</li> </ul></section><section><h3>F09：统一角色画风的技巧<a href="#f09统一角色画风的技巧"><span>#</span></a></h3><ul> <li>大乱斗 SP 中的角色画风各异，因此有必要统一角色的画风</li> <li>降低饱和度，以便光照下产生的阴影与接收阴影的面层次分明</li> <li>制作观感相似的贴图。卡通角色通常使用单一颜色填充，现实风格的角色会使用更丰富和精细的贴图，因此对卡通风格的角色添加更多细节化处理和简化现实风格角色的材质设计来达到视觉的平衡</li> <li>调节头身比</li> <li>如有需要则细化</li> <li>以真实感方式来表达角色</li> </ul></section><section><h3>F10：画面会影响游戏性吗<a href="#f10画面会影响游戏性吗"><span>#</span></a></h3><ul> <li>游戏性可以通过画面而变得更加出众，游戏提供的乐趣与画面表现可以相互作用</li> <li>在游戏商店，通过实际画面能给玩家直观的感受，如果画面不吸引玩家，玩家就不会去玩这个游戏，因此画面是游戏乐趣的加成要素</li> </ul></section><section><h3>F11：监制大乱斗中的轶事<a href="#f11监制大乱斗中的轶事"><span>#</span></a></h3><ul> <li>介绍开发大乱斗 SP 过程中的反馈案例，视频内容比较全</li> <li>攻击技能看起来应该打到人的时候，在实际上并没有触发打击判定，这时候要从动画和碰撞检测两方面着手</li> <li>观感上看起来是可以触发打击判定的，降低持续碰撞检测的时间</li> <li>光触发因果律时在判定位置添加残影</li> </ul></section><section><h3>F12：如何呈现微缩感<a href="#f12如何呈现微缩感"><span>#</span></a></h3><ul> <li>找一张俯拍照，上下部分加上模糊效果模拟景深（这样会让照片看起来像是从近处拍出来的），拉高饱和度，同时调淡蓝色（减少空间透视感）</li> <li>浅景深是增强氛围感的方法，但除开尤其需要的场合，最好不要乱用，因为每个人对画面的关注焦点不同，去看向模糊的地方会引起不适</li> <li>高饱和度的色彩应该尽量避免出现在大型物体上，以体现空间透视感。相反，越近的物体饱和度会越来越高</li> <li>单纯地绘制材质贴图并不能成就连贯统一的视觉感受，应从整体考虑</li> </ul></section><section><h3>F13：处理显示器色彩差异<a href="#f13处理显示器色彩差异"><span>#</span></a></h3><ul> <li>虽然有测量校色的设备，但游戏画面会传输到普通的电视上显示，因此用处有限</li> <li>在大家都可以看到的电视上，根据照明条件校准颜色，但不同的显示器上表现的差异确实过大</li> <li>办公室的灯光泛黄，做出来的美术素材就有可能泛蓝，办公室的灯光应呈中性</li> </ul></section><section><h3>F14：动态模糊<a href="#f14动态模糊"><span>#</span></a></h3><ul> <li>中间帧添加模糊会显得更连贯，减少卡顿感</li> <li>可以在贴图中绘制，也可以交给算法，但在如今人工绘制的模糊效果会降低观感</li> <li>在游戏选项里，将控制模糊强度的选项交给玩家</li> </ul></section><section><h3>F15：让想象变得真实<a href="#f15让想象变得真实"><span>#</span></a></h3><ul> <li>制作素材时，不光要考虑素材的材质、光照，还要考虑质量、用途以及摆放位置</li> <li>在世界中以何种方式放置和呈现素材，所以要从大局出发思考如何摆放</li> </ul></section><section><h3>F16：成品最终呈现效果<a href="#f16成品最终呈现效果"><span>#</span></a></h3><ul> <li>现时代只把模型的面数提高已经不行了，要关注整体效果（其实主要是应用现代的图形技术和各种后处理）</li> </ul></section><section><h3>F17：如何打破一面墙<a href="#f17如何打破一面墙"><span>#</span></a></h3><ul> <li>断面不要直来直去显得平坦，要更加不规则、更加参差不齐</li> <li>破面整体不要显得太方正</li> <li>如果一个效果怎么改都改不到理想效果，那就予以接受然后继续前进吧</li> <li>想要修正偏离太多的工作，可能要照着原先反馈的完全相反的方式来修改</li> </ul></section><section><h3>F18：漫画风视觉效果<a href="#f18漫画风视觉效果"><span>#</span></a></h3><ul> <li>出现角色侧面形象时，嘴巴移动到脸的一侧，显得漫画感的同时还可以不用做下巴部分的动画</li> </ul></section></section> <section><h2>G：动画表现<a href="#g动画表现"><span>#</span></a></h2><section><h3>G01：动画参数文档<a href="#g01动画参数文档"><span>#</span></a></h3><ul> <li>动画清单，包含所有该角色在游戏中所需要的动画。在该阶段，所有动作均需要使用文字将其描述出来，同时要标注动画的持续时间以及触发打击判定的时机，需要在此时平衡好所有动画准确的触发以及持续时间</li> <li>可以循环使用的动画需要特殊标注</li> <li>使用可动人偶模型来与动画师沟通，摆好造型后，通过简单的拍照，裁剪出轮廓，拉高对比度，这样的流程就可以放入动画清单中</li> <li>非人型角色不适用人偶模型，通过详细的文字来描述</li> <li>反馈动画修改时，确保所有动画师都在场，通过分享知识和要点，可以避免犯下同样的错误，让大家共同学习进步</li> </ul></section><section><h3>G02：表现过了只是刚好正确<a href="#g02表现过了只是刚好正确"><span>#</span></a></h3><ul> <li>招式动画的设计一般不存在与现实动作“差得太远”的问题，反而更容易出现<strong>表现力不足</strong>的问题</li> <li>某些动作定格时，角色模型会显得扭曲，会被要求修改，但是修改后动画的冲击力可能会受到影响</li> <li>先突破自然的极限，然后在收敛一点</li> </ul></section><section><h3>G03: 攻击动画分解<a href="#g03-攻击动画分解"><span>#</span></a></h3><ul> <li>攻击动画可分为：待机、起手、攻击、收招</li> <li>待机动画有两种态：在地面和在空中</li> <li>起手动画负责引导，可以以第一帧为起手来做一些大幅度动作，快起手也能实现快速反馈，同时也鼓励对手快速闪避</li> <li>攻击动画判定瞬间，躯体活动幅度更大</li> <li>收招动画期价不接收玩家输入，因此要与待机动画做好区分，保证动画时间</li> <li>也可以添加取消帧，使收招动画不完全执行完毕也可以接收输入，执行其他操作</li> <li>攻击动画的文档中还需要提供招式打击判定帧，招式总帧数或者说到可取消帧之前的所有帧</li> </ul></section><section><h3>G04：起手动画<a href="#g04起手动画"><span>#</span></a></h3><ul> <li>按键输入时立刻改变角色姿势。通常游戏中会在待机到起手间穿插过渡动画以避免动作不连贯的问题，但在大乱斗 SP 中将动作的响应时间放到了更重要的位置，因为显示和网络都存在延迟，所以没有穿插过度动画</li> <li>起手动画并不是必要的</li> <li>触发打击判定前的引导过程可以掩盖掉网络延迟造成的不连续</li> <li>优秀的起手引导动画可以大大加强招式的视觉冲击感，所以起手动画并不是越快越好</li> </ul></section><section><h3>G05：攻击动画<a href="#g05攻击动画"><span>#</span></a></h3><ul> <li>通用规则是对攻击动画进行夸张化处理，帮助玩家分辨动作</li> <li>保持打击点位于居中位置，以避免角色朝向其他方向造成的打击不准确的问题</li> <li>动作不必可以做成对称的</li> <li>动作前倾时避免出现直接弯曲躯干，动作过渡中可以存在一定的扭曲姿势</li> <li>尽量在攻击命中后再出现 HitStop 效果，如果攻击幅度不够大就出现效果，会丢失一些打击感</li> <li>预览动作时从多个不同的角度观察，以确保各角度的表现效果都是令人满意的</li> <li>动作定格时要<strong>突出的角色剪影</strong>，四肢动作必须清晰，动画制作时应确能表现出闪光点</li> </ul></section><section><h3>G06：收招动画<a href="#g06收招动画"><span>#</span></a></h3><ul> <li>收招动画可能是持续时间较长的动画序列</li> <li>通常招式的响应时间越快越好，即能从起手动作快速过渡到攻击判定的招式</li> <li>为了体现游戏的博弈性，强力的招式应该带来较长的硬直、副作用</li> <li>在大乱斗中，常规招式的持续时间通常在一秒内，打击判定和触发判定前的起手阶段，二者通常占招式总过程的1/4到1/3，更有甚者，快起手的招式甚至不到整体的 1/5</li> <li>对于存在消除帧(Cancel Frame)的游戏，收招点就是可取消硬直的帧数</li> <li>综上，由于在动作的序列帧中占比较大，因此收招动画可以精确展现角色此前的动作</li> <li>在收招动画后，应平滑过渡到角色默认的待机动作，或者利用这段时间尽可能突出招式的力度，比如拉长持续时间，加大动作幅度或是引入取消硬直的帧数</li> </ul></section><section><h3>G07：真实感缺位时的夸张化处理<a href="#g07真实感缺位时的夸张化处理"><span>#</span></a></h3><ul> <li>当角色的多边形数量较少时，真实感会显得不足，因此越具真实感的动作反而越不真实。这种情况下需要夸张化的处理，动作夸张了，反而是刚刚好</li> <li>角色动作受影响的因素有很多，比如屏占比、视距、大小等等</li> <li>对于动作游戏，动画效果最好是能做到大开大合</li> <li>当动画缺少信息量时，用夸张化的动画来弥补</li> </ul></section><section><h3>G08：姿态建议<a href="#g08姿态建议"><span>#</span></a></h3><ul> <li>使用可动人偶凹造型</li> <li>避免对称或者说左右镜像，头部压低避免显得太正，挺胸</li> <li>实体、臀部、腿不要排到一条线上，双脚略微向外张开，距离镜头的距离不一样，角色整体稍微偏离重心避免杵在正中央</li> <li>角色身上的服装配饰要具备动感</li> </ul></section><section><h3>G09：扁平化缩放<a href="#g09扁平化缩放"><span>#</span></a></h3><ul> <li>使用拉伸缩放来让动画更生动，比如水平的拉伸带来纵向的压缩</li> <li>如果是针对角色的纵向压缩，那么可以在压缩时为水平方向添加体积，避免显得生硬</li> </ul></section><section><h3>G10：受击动画<a href="#g10受击动画"><span>#</span></a></h3><ul> <li>受击动画首帧会有 hit stop 效果</li> <li>通常受击动作与常规站立动作相似，可以对受击动作做夸张化处理</li> <li>由于角色总是会在受击时播放受击动画，所以当角色的 hit box 延展出来后（比如踢腿、刺拳），受击角色的受击动画与攻击角色可能会存在一定的距离，这能体现动作的冲击性。但当攻击者使用轻攻击时，受击角色的 hit box 会后撤归位，从而使连段的过程中战斗双方的距离越来越大，最终导致攻击方的轻攻击打空</li> <li>格斗游戏（大乱斗为例）中判定位置有上、中、下三种，攻击力度有小、中、大三种，因此共九种受击类型，因此每种受击类型都要单独制作动画</li> <li>受击动画要破坏角色的平衡感，受击时使角色的轮廓清晰可见</li> </ul></section><section><h3>G11：插帧<a href="#g11插帧"><span>#</span></a></h3><ul> <li>应尽快过渡到关键帧，过渡帧不用太多，帧数应投入到起手以及攻击动画本身上</li> <li>插帧可能会让动画的冲击力减弱</li> </ul></section><section><h3>G12：面部动画<a href="#g12面部动画"><span>#</span></a></h3><ul> <li>上下齿不完全咬合</li> </ul></section><section><h3>G13：镜像动画<a href="#g13镜像动画"><span>#</span></a></h3><ul> <li>格斗游戏的相机在角色侧面，因此为了更好的表现角色，需要做特殊处理（比如角色侧着脸跑步）</li> </ul></section><section><h3>G14 动画混合<a href="#g14-动画混合"><span>#</span></a></h3><ul> <li>过渡动画持续时间太长会降低动作的节奏感和冲击力，因此使用动画混合时要快和巧</li> </ul></section><section><h3>G15 手部动画细节<a href="#g15-手部动画细节"><span>#</span></a></h3><ul> <li>倾斜手指来做到自然的抓握</li> <li>手部的体积小，且手指的处理复杂，因此可以在优化时合并手指的动画</li> </ul></section><section><h3>G16 招式动画与打击判定<a href="#g16-招式动画与打击判定"><span>#</span></a></h3><ul> <li>打击判定并非直接判定到角色的拳头、武器上，而是覆盖了更大的范围</li> <li>快节奏的格斗游戏更需要如此，角色的空间位置变换更频繁，因此需要更宽泛的攻击判定</li> <li>弄清楚打击判定从何时开始，以及角色攻击判定生效前所处的姿势非常重要，这就是 hit stop 应当生效的时机。如果是最终结算的击杀，画面还会完全静止，这时招式的造型就显得更为重要</li> <li>比起严谨的碰撞，当武器挥过去以后再触发打击判定会比较好</li> <li>动画师应当留意打击判定生效的时机和位置</li> </ul></section><section><h3>骨骼动画的局限性<a href="#骨骼动画的局限性"><span>#</span></a></h3></section></section> <section><h2>H：特效表现<a href="#h特效表现"><span>#</span></a></h2></section> <section><h2>I：音乐 / 音效<a href="#i音乐--音效"><span>#</span></a></h2></section> <section><h2>J：游戏界面<a href="#j游戏界面"><span>#</span></a></h2></section> <section><h2>K：程序/技术<a href="#k程序技术"><span>#</span></a></h2></section> <section><h2>L：市场/宣发<a href="#l市场宣发"><span>#</span></a></h2></section> <section><h2>M：开发轶事<a href="#m开发轶事"><span>#</span></a></h2></section> <section><h2>N：开发理念<a href="#n开发理念"><span>#</span></a></h2></section>https://yohiro.games/posts/apply-dbscan-in-unreal/https://yohiro.games/posts/apply-dbscan-in-unreal/Thu, 15 Feb 2024 00:00:00 GMT<section><h2>介绍<a href="#介绍"><span>#</span></a></h2><p>在<a href="/posts/Clustering-DBSCAN">上一篇文章</a>中介绍了 DBSCAN 的原理，这里来记录一下在引擎中的应用。</p><p>我决定引入聚类算法是为了尝试在 UE 中将放置某种包围盒的步骤自动化，从而提高迭代效率。</p><p>我在场景中沿着 Nav Mesh 生成了采样点，采样点的数量为 4643 个，分布如下：</p><p></p><figure><img src="/assets/images/DBSCAN/Samples.png" alt="Samples" /><figcaption>Samples</figcaption></figure> <em>可以视作采样点的集合 D</em><p></p><p>在参照下图对其他聚类算法进行了一些对比后，选择了使用 DBSCAN。</p><p></p><figure><img src="https://scikit-learn.org/stable/_images/sphx_glr_plot_cluster_comparison_001.png" alt="cluster_comparison" /><figcaption>cluster_comparison</figcaption></figure> <em>各种聚类算法的效果对比</em><p></p><p>这个选择出于以下几个原因：</p><ol> <li>采样点是沿着 Nav Mesh 生成的，密度比较平均</li> <li>DBSCAN 的算法比较简单，易于实现</li> <li>游戏中的路径形成的样本集有可能是非凸的</li> <li>噪声多可以接受，身是可以通过去除噪声点而达到离线构建效率优化的目的</li> </ol><p>综上，我选择了 DBSCAN。</p></section> <section><h2>参数选择<a href="#参数选择"><span>#</span></a></h2><p>之前的文章中提到过参数选择的问题，<a href="https://cdn.aaai.org/KDD/1996/KDD96-037.pdf" target="_blank">原论文</a>中介绍了一种叫做 <strong>k-dist</strong> 的选择 Epsilon 和 MinPts 的方法。</p><section><h3>k-dist<a href="#k-dist"><span>#</span></a></h3><p><strong>k-dist</strong> 指的是样本中的点 P 到距它第 k 近的点的距离。通过排序后的 k-dist 图可以确定在哪个点（附近）距离变化速度大，从而了解密度分布的情况。</p><p>k-dist 方法是一种经验性的方法，试想这样一种情景：</p><p>设点 p 第 k 近的邻接点的距离为 d，那么对于半径为 d 的点 p 邻域内必包含至少 k+1 个点，这么说是因为邻域可能存在多个点到 p 的距离与 d 相同，对于大样本集，这种情况的概率较低，因此，可以认为点 p 的邻域 d 内包含 k+1 个点。</p><p>这时如果我们在样本集内选择任意一点 p，并将 Epsilon 设为上面提到的 d，将 MinPts 设为 k，则所有 k-dist 小于等于 d 的点都将成为核心点。</p><p>假如能够在最小的 Cluster 中找到最大的 k-dist 的值，便可确认以最小的 Cluster 的密度为参考的 Epsilon 阈值，这个值正是按照 k-dist 排序后的第一个拐点。</p><p></p><figure><img src="/assets/images/DBSCAN/k-distance.png" alt="k-dist" /><figcaption>k-dist</figcaption></figure> <em>以该拐点为界，左侧的点为噪声点，右侧的点为 Cluster 中的点</em><p></p><p>相对于该拐点，具有较大 k-dist 的值的点可以被认为是噪声点，所有其他较小的点则归于某个 Cluster。</p></section><section><h3>步骤<a href="#步骤"><span>#</span></a></h3><p>假如数据集中的数据为 <span><span>NN</span><span><span><span></span><span>N</span></span></span></span> 维数据：</p><ol> <li>选取 k 值，可以取 k 为 <span><span>2N2N</span><span><span><span></span><span>2</span><span>N</span></span></span></span> 或 <span><span>2N−12N-1</span><span><span><span></span><span>2</span><span>N</span><span></span><span>−</span><span></span></span><span><span></span><span>1</span></span></span></span></li> <li>对数据集中所有的点 p，计算距离点 p 第 k 近的点的距离（k-distance）</li> <li>依据 k-distance 的大小，将所有的点降序排序</li> <li>绘制 k-distance 图</li> <li>找到拐点位置的 k-dist 值，即为 Eps 的值。</li> <li>取 Minpts 为 k+1</li> </ol></section></section> <section><h2>实现参考<a href="#实现参考"><span>#</span></a></h2><p>这里我找到了 <a href="https://github.com/CallmeNezha/SimpleDBSCAN" target="_blank">SimpleDBSCAN</a>，一个轻量的 header-only 的 C++ 实现。SimpleDBSCAN 的实现中使用了 <a href="https://oi-wiki.org/ds/kdt/" target="_blank">kd 树</a>来做复杂的样本划分，以加速大样本的查询。</p></section> <section><h2>最终效果<a href="#最终效果"><span>#</span></a></h2><p>使用 DBSCAN 进行聚类的结果如下，其中 <strong>MinPts = 10</strong>, <strong>Epsilon = 450</strong>:</p> <table><thead><tr><th>聚类结果</th><th>噪声</th></tr></thead><tbody><tr><td><figure><img src="/assets/images/DBSCAN/MinPts%3D10_Eps%3D450.png" alt="MinPts = 10, Epsilon = 450" /><figcaption>MinPts = 10, Epsilon = 450</figcaption></figure></td><td><figure><img src="/assets/images/DBSCAN/MinPts%3D10_Eps%3D450_Noises.png" alt="Noises" /><figcaption>Noises</figcaption></figure></td></tr></tbody></table><p>就该结果而言，存在下面几个问题：</p><ol> <li>聚类可以进一步细分，比如最上面连成一大块的紫色可以进一步划分</li> <li>对于比较狭窄的地方，由于采样点不足，无法形成有效聚类，从而产生噪声</li> <li>对于比较宽阔的地方，由于采样点之间的距离比较大，无法形成有效聚类，从而产生噪声</li> </ol><p>在引擎中导出 k-distance 数据后，使用 pyplot 绘制图像如下：</p><p></p><figure><img src="/assets/images/DBSCAN/6dist.png" alt="k-distance" /><figcaption>k-distance</figcaption></figure><p></p><p>由于样本点数目较多，因此很难直观地看出拐点在哪里，这种情况下 k-dist 方法似乎不适用？从上图中可以得到的信息是，在大概 [280,380] 这个范围内 k-dist 变化较快，也就是说在 [280,380] 之外的区域即使调整 Eps 也不会有明显变化。</p><p></p><figure><img src="/assets/images/DBSCAN/6dist-scale.png" alt="slope" /><figcaption>slope</figcaption></figure><p></p><p>放大看了一下，最终我选择了 Eps = 288，下面是 <strong>MinPts = 6</strong>, <strong>Epsilon = 288</strong> 的结果：</p> <table><thead><tr><th>聚类结果</th><th>噪声</th></tr></thead><tbody><tr><td><figure><img src="/assets/images/DBSCAN/MinPts%3D6_Eps%3D288.png" alt="MinPts = 6, Epsilon = 288" /><figcaption>MinPts = 6, Epsilon = 288</figcaption></figure></td><td><figure><img src="/assets/images/DBSCAN/MinPts%3D6_Eps%3D288_Noises.png" alt="Noises" /><figcaption>Noises</figcaption></figure></td></tr></tbody></table><p>最终的结果差强人意，但基本满足了我的需求。</p><p>我也试了其他的参数，下面是 <strong>MinPts = 10</strong>, <strong>Epsilon = 350</strong> 的结果：</p><p></p><figure><img src="/assets/images/DBSCAN/MinPts%3D10_Eps%3D350.png" alt="MinPts = 10, Epsilon = 350" /><figcaption>MinPts = 10, Epsilon = 350</figcaption></figure> <em>MinPts = 10, Epsilon = 350 的划分结果</em><p></p><p>可以看到零碎的区域明显减少，因为随着 MinPts 和 Epsilon 的增大，划分 Cluster 的粒度逐渐变大。如果样本集中的密度相对平均，而又期望划分较大粒度的 Cluster，那么使用 k-dist 方法未必能直接获得结果，但是可以确定大致的范围，对样本的密度分布有一个直观的认识。</p></section> <section><h2>总结<a href="#总结"><span>#</span></a></h2><p>使用 k-dist 可以确定 DBSCAN 的参数的范围，但实际应用中最好还是搭配可视化的数据调试界面进行参数的调整。而且 k-dist 找的是适配最小的 Cluster 的一组参数，有些情况下，我们会期望划分的 Cluster 具有较大的粒度，这种情况下使用 DBSCAN 或者 k-dist 方法确定参数可能不是那么合适了，或许 <a href="https://scikit-learn.org/stable/modules/clustering.html#hdbscan" target="_blank">HDBSCAN</a> 和 <a href="https://zh.wikipedia.org/wiki/OPTICS" target="_blank">OPTICS</a> 可以获得更好的结果。</p></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://scikit-learn.org/stable/modules/clustering.html#dbscan" target="_blank">scikit-leran clustering</a></li> <li><a href="https://github.com/CallmeNezha/SimpleDBSCAN" target="_blank">SimpleDBSCAN</a></li> <li><a href="https://www.youtube.com/watch?v=C3r7tGRe2eI" target="_blank">DBSCAN Clustering Easily Explained with Implementation</a></li> <li><a href="https://blog.csdn.net/Cyrus_May/article/details/113504879" target="_blank">DBSCAN的参数选择及其应用于离群值检测</a></li> </ul></section>https://yohiro.games/posts/clustering-dbscan/https://yohiro.games/posts/clustering-dbscan/Thu, 01 Feb 2024 00:00:00 GMT<section><h2>介绍<a href="#介绍"><span>#</span></a></h2><p><strong>DBSCAN (Density-based spatial clustering of applications with noise)</strong> 是一个基于密度的聚类（Cluster）算法：给定空间里的一个点集，该算法能把附近的点分成一组（有很多相邻点的点），并标记出位于低密度区域的噪声点。</p><p>原版论文在<a href="https://cdn.aaai.org/KDD/1996/KDD96-037.pdf" target="_blank">这里</a></p></section> <section><h2>概念描述及参数<a href="#概念描述及参数"><span>#</span></a></h2> <table><thead><tr><th>标识</th><th>含义</th></tr></thead><tbody><tr><td><span><span>DD</span><span><span><span></span><span>D</span></span></span></span></td><td>样本集</td></tr><tr><td><span><span>PiP_i</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>i</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td><span><span>DD</span><span><span><span></span><span>D</span></span></span></span> 中的各样本点</td></tr><tr><td><span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> (Epsilon)</td><td><span><span>PiP_i</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>i</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 的邻域距离阈值</td></tr><tr><td><span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span></td><td><span><span>PiP_i</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>i</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 在 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 范围内样本的数量</td></tr></tbody></table><p></p><figure><img src="/assets/images/DBSCAN/Definition1.png" alt="definition_1" /><figcaption>definition_1</figcaption></figure> <em>核心点与边缘点（左）和直接密度可达（右）</em><p></p><p>核心点 (Core Points) : 满足条件，在 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 范围内，具有 <span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span> 个样本的样本点 <span><span>PiP_i</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>i</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>，称作核心点。</p><p>边缘点 (Border Points) : 在<strong>核心点</strong> <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 范围内，且在样本点 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 范围内的样本数量小于 <span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span>，称作边缘点</p><p>噪声点 (Noise Points/ Outlier) : 非可达点，即不在核心点的 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 范围内的样本点，称为噪声点</p><p>直接密度可达 (directly density-reachable) : 当 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 在除 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 以外的任意样本点 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 的邻域半径 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 范围内，<strong>且 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 是一个核心点</strong>，则称 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 可由 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 直接（密度）可达。</p><ul> <li>也就是说，核心点 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 的 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 范围内除 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 以外的点，都是由 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 直接可达的，同理，非核心点没有<strong>直接可达</strong>的点。</li> </ul><p></p><figure><img src="/assets/images/DBSCAN/Definition2.png" alt="definition_2" /><figcaption>definition_2</figcaption></figure> <em>可达性（左）和连接性（右）</em><p></p><p>密度可达 (density-reachable) : 对于样本中的点，存在路径 <span><span>P1P_1</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>,…,<span><span>PnP_n</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>n</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>，使 <span><span>P1P_1</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> = <span><span>PP</span><span><span><span></span><span>P</span></span></span></span>，<span><span>PnP_n</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span>n</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> = <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span>，如果该路径中任意一点 <span><span>Pi+1P_{i+1}</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span><span>i</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 可以由 <span><span>PiP_{i}</span><span><span><span></span><span><span>P</span><span><span><span><span><span><span></span><span><span><span>i</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 直接可达，那么称 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 是可从 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> （密度）可达的。</p><ul> <li>根据该定义，非核心点是可以由其他点可达的，但没有点是由非核心点可达的。</li> <li>参考上图，可达性是<strong>非对称的</strong>，<span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 到 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 可达不代表 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 到 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 可达。</li> </ul><p>密度相连 (density-connected) : 对于样本中的点，存在一个点 <span><span>OO</span><span><span><span></span><span>O</span></span></span></span> 使得 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 和 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 都是由 <span><span>OO</span><span><span><span></span><span>O</span></span></span></span> 可达的，则点 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 和点 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 被称为（密度）相连的，连接性是<strong>对称的</strong>。定义了连接性后，每个 Cluster 都符合两个性质：</p><ul> <li>同一个 Cluster 中的任意两点都是相连的。</li> <li>如果 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 是由 Cluster 里的点 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 可达的，那么 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 和 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 在同一个 Cluster 中。</li> </ul><section><h3>举例<a href="#举例"><span>#</span></a></h3><p></p><figure><img src="/assets/images/DBSCAN/DBSCAN-Illustration.png" alt="referenced from wikipedia" /><figcaption>referenced from wikipedia</figcaption></figure><p></p><p>上图中，<span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span> = 4，点 A 和其他红色点是核心点，因为它们的 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域（图中红色圆圈）里包含最少 4 个点（包括自己），由于它们之间相互相可达，它们形成了一个 Cluster。点 B 和点 C 不是核心点，但它们可由 A 经其他核心点可达，作为边缘点加入同一个 Cluster 。点 N 是噪声点，它既不是核心点，又不由其他点可达。</p></section><section><h3>距离函数<a href="#距离函数"><span>#</span></a></h3><p>这里的距离可以是任意空间内的距离函数，通常使用欧氏距离，在游戏开发中也可能会使用曼哈顿距离。为直观起见，本文中的距离均指的是欧氏距离。</p></section></section> <section><h2>算法<a href="#算法"><span>#</span></a></h2><p>为确定 Cluster, DBSCAN 算法首先从集合里任意一点 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 开始遍历所有密度可达点。然后根据 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 和 <span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span> 判断 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 是核心点还是边缘点。</p><p>如果 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 是核心点，该流程会产生一个 Cluster。</p><p>如果 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 是边缘点，那么没有任何点是从 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 密度可达的，DBSCAN 将会从样本集中选择下一个点重复该过程。</p><p>对于存在多个 Cluster 的情况，算法会根据规则进行合并，这个规则是：</p><ul> <li>Cluster 中，对所有 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 的密度可达的点 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 也属于该 Cluster</li> <li>Cluster 中，对所有 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 的密度相连的点 <span><span>QQ</span><span><span><span></span><span>Q</span></span></span></span> 也属于该 Cluster</li> </ul><p>确定了这些，就明确了 DBSCAN 的算法，简单来说，算法分为下面的步骤：</p><ol> <li>初始化样本集 <span><span>DD</span><span><span><span></span><span>D</span></span></span></span>，以及 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span>，<span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span></li> <li>从任意一点 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 开始查询 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域内的样本点</li> <li>如果 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域的点的数量大于等于 <span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span>，则创建一个 Cluster，否则标记 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 为噪声点（该噪声点也可能位于后续点形成的 Cluster 内）</li> <li>对除点 <span><span>PP</span><span><span><span></span><span>P</span></span></span></span> 外满足条件的 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域内的样本点，逐个查询这些点 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域内的样本点</li> <li>如果这些点 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域的点的数量大于等于 <span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span>，便将这些点加入此次创建的 Cluster 中</li> <li>从样本中其他点出发，重复步骤 2 直到遍历完成</li> </ol><p><a href="https://cdn.aaai.org/KDD/1996/KDD96-037.pdf" target="_blank">论文</a> 4.1 中提供了伪代码：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>DBSCAN(SetOfPoints, Eps, MinPts)</span></div></div><div><div><div>2</div></div><div><span>// SetOfPoints is UNCLASSIFIED</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>ClusterId := nextId(NOISE)</span></div></div><div><div><div>4</div></div><div><span> </span><span>FOR</span><span> i </span><span>FROM</span><span> </span><span>1</span><span> </span><span>TO</span><span> SetOfPoints.size </span><span>DO</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>Point := SetOfPoints.get(i)</span></div></div><div><div><div>6</div></div><div><span> </span><span>IF</span><span> Point.ClId = UNCLASSIFIED </span><span>THEN</span></div></div><div><div><div>7</div></div><div><span> </span><span>IF</span><span> ExpandCluster(SetOfPoints, Point, ClusterId, Eps, MinPts) </span><span>THEN</span></div></div><div><div><div>8</div></div><div><span><span> </span></span><span>ClusterId := nextId(ClusterId)</span></div></div><div><div><div>9</div></div><div><span> </span><span>END</span><span> </span><span>IF</span></div></div><div><div><div>10</div></div><div><span> </span><span>END</span><span> </span><span>IF</span></div></div><div><div><div>11</div></div><div><span> </span><span>END</span><span> </span><span>FOR</span></div></div><div><div><div>12</div></div><div><span>END</span><span> </span><span>// DBSCAN</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>以及创建聚类的 ExpandCluster 函数，经过该函数可以生成一个 Cluster：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>ExpandCluster(SetOfPoints, Point, ClId, Eps,MinPts) : </span><span>Boolean</span></div></div><div><div><div>2</div></div><div><span> </span><span>//</span></div></div><div><div><div>3</div></div><div><span> </span><span>// regionQuery 函数用于查询并返回以当前点 Point 为中心的 Eps 邻域中的点</span></div></div><div><div><div>4</div></div><div><span> </span><span>// seeds 是包含了从点 Point 出发的所有可达点的集合</span></div></div><div><div><div>5</div></div><div><span> </span><span>//</span></div></div><div><div><div>6</div></div><div><span><span> </span></span><span>seeds := SetOfPoints.regionQuery(Point, Eps)</span></div></div><div><div><div>7</div></div><div><span> </span><span>IF</span><span> seeds.size &lt; MinPts </span><span>THEN</span><span> </span><span>// no core point</span></div></div><div><div><div>8</div></div><div><span> </span><span>//</span></div></div><div><div><div>9</div></div><div><span> </span><span>// 对 Eps 邻域内的样本数量不足 MinPts 个的非核心点（边缘点），由于非核心点没有可达点，因此返回 false。</span></div></div><div><div><div>10</div></div><div><span> </span><span>// 这里虽然标记为 NOISE 但在后续步骤可能会改变该标记</span></div></div><div><div><div>11</div></div><div><span> </span><span>//</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>SetOfPoint.changeClId(Point, NOISE)</span></div></div><div><div><div>13</div></div><div><span><span> </span></span><span>RETURN </span><span>False</span></div></div><div><div><div>14</div></div><div><span> </span><span>ELSE</span><span> </span><span>// all points in seeds are density-reachable from Point</span></div></div><div><div><div>15</div></div><div><span> </span><span>//</span></div></div><div><div><div>16</div></div><div><span> </span><span>// 这里的点是除了初始点 Point 外所有满足 MinPts 和 Eps 的直接密度可达的点，</span></div></div><div><div><div>17</div></div><div><span> </span><span>// 因此先将这些点加入同一个 Cluster 中，并将初始点 Point 删除</span></div></div><div><div><div>18</div></div><div><span> </span><span>//</span></div></div><div><div><div>19</div></div><div><span><span> </span></span><span>SetOfpoints.changeClId(seeds, ClId)</span></div></div><div><div><div>20</div></div><div><span><span> </span></span><span>seeds.delete(Point)</span></div></div><div><div><div>21</div></div><div><span> </span><span>WHILE</span><span> seeds &lt;&gt; </span><span>Empty</span><span> </span><span>DO</span></div></div><div><div><div>22</div></div><div><span><span> </span></span><span>currentP := seeds.first()</span></div></div><div><div><div>23</div></div><div><span> </span><span>//</span></div></div><div><div><div>24</div></div><div><span> </span><span>// 从该 Cluster 中的其他点出发，查找其他的 Eps 邻域内可达的点</span></div></div><div><div><div>25</div></div><div><span> </span><span>//</span></div></div><div><div><div>26</div></div><div><span> </span><span>result</span><span> := SetOfpoints.regionQuery(currentP, Eps)</span></div></div><div><div><div>27</div></div><div><span> </span><span>IF</span><span> </span><span>result</span><span>.size &gt;= MinPts </span><span>THEN</span></div></div><div><div><div>28</div></div><div><span> </span><span>FOR</span><span> i </span><span>FROM</span><span> </span><span>1</span><span> </span><span>TO</span><span> </span><span>result</span><span>.size </span><span>DO</span></div></div><div><div><div>29</div></div><div><span><span> </span></span><span>resultP := </span><span>result</span><span>.get(i)</span></div></div><div><div><div>30</div></div><div><span> </span><span>//</span></div></div><div><div><div>31</div></div><div><span> </span><span>// 如果可达点暂未划分到其他 Cluster 中（包括此前被标记为 NOISE 的），便将该点加入到该 Cluster 中</span></div></div><div><div><div>32</div></div><div><span> </span><span>// 若此前未归类过，便将该可达点加入可达点的集合 seeds 中</span></div></div><div><div><div>33</div></div><div><span> </span><span>//</span></div></div><div><div><div>34</div></div><div><span> </span><span>IF</span><span> resultP.ClId </span><span>IN</span><span> </span><span>{UNCLASSIFIED, NOISE}</span><span> </span><span>THEN</span></div></div><div><div><div>35</div></div><div><span> </span><span>IF</span><span> resultP.ClId = UNCLASSIFIED </span><span>THEN</span></div></div><div><div><div>36</div></div><div><span><span> </span></span><span>seeds.append(resultP)</span></div></div><div><div><div>37</div></div><div><span> </span><span>END</span><span> </span><span>IF</span></div></div><div><div><div>38</div></div><div><span><span> </span></span><span>SetOfPoints.changeClId(resultP, ClId)</span></div></div><div><div><div>39</div></div><div><span> </span><span>END</span><span> </span><span>IF</span><span> </span><span>// UNCLASSIFIED or NOISE</span></div></div><div><div><div>40</div></div><div><span> </span><span>END</span><span> </span><span>FOR</span></div></div><div><div><div>41</div></div><div><span> </span><span>END</span><span> </span><span>IF</span><span> </span><span>// result.size &gt;= MinPts</span></div></div><div><div><div>42</div></div><div><span> </span><span>//</span></div></div><div><div><div>43</div></div><div><span> </span><span>// 查询完成后从将当前点从点集中删除</span></div></div><div><div><div>44</div></div><div><span> </span><span>//</span></div></div><div><div><div>45</div></div><div><span><span> </span></span><span>seeds.delete(currentP)</span></div></div><div><div><div>46</div></div><div><span> </span><span>END</span><span> </span><span>WHILE</span><span> </span><span>// seeds &lt;&gt; Empty</span></div></div><div><div><div>47</div></div><div><span><span> </span></span><span>RETURN </span><span>True</span></div></div><div><div><div>48</div></div><div><span> </span><span>END</span><span> </span><span>IF</span></div></div><div><div><div>49</div></div><div><span>END</span><span> </span><span>// ExpandCluster</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div></section> <section><h2>评估<a href="#评估"><span>#</span></a></h2><section><h3>参数<a href="#参数"><span>#</span></a></h3><p>参数的选择向来是个令人头大的问题，在对样本集缺少了解的情况下，很难选出合适的参数。论文中提到了一种叫做 <strong>k-distance</strong> 的选择 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 的方法，我将在应用时加以解释。但是即使如此，这个过程也是要不断迭代，最好在数据可视化时进行调整。</p><p>假如样本集中的点密度差异较大，也很难产生一个满足所有情况的聚类结果，因为全局的参数 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 和 <span><span>MinPtsMinPts</span><span><span><span></span><span>M</span><span>in</span><span>P</span><span>t</span><span>s</span></span></span></span> 只有一套。密度变化较大的情况，可以考虑使用 <a href="https://zh.wikipedia.org/wiki/OPTICS%E7%AE%97%E6%B3%95" target="_blank">OPTICS</a> 或者 <a href="https://scikit-learn.org/stable/modules/clustering.html#hdbscan" target="_blank">HDBSCAN</a>。</p></section><section><h3>稳定性<a href="#稳定性"><span>#</span></a></h3><p>DBSCAN 的结果是确定的，对于给定顺序的数据集来说，相同参数下生成的 Clusters 是相同的。然而，当相同数据的顺序不同时，生成的 Clusters 较之另一种顺序会有所不同。</p><p>首先，即使不同顺序的数据集的核心点是相同的，Clusters 的标签会取决于数据集中各采样点的顺序。其次，可达点被分配到哪个 Clusters 也是会受到数据顺序的影响的，比如一个边缘采样点位于一个分属于两个不同的 Clusters 的核心点的 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域内，这时该边缘点被分配到哪个 Cluster 中取决于哪一个 Cluster 先创建。</p><p>因此说 DBSCAN 是<code>不稳定</code>的。</p></section><section><h3>效率<a href="#效率"><span>#</span></a></h3><p>算法中遍历了所有的样本点，但实践中算法的时间消耗往往受到 regionQuery 函数的制约。</p><p>对于 regionQuery 函数，论文中提到可以使用 <a href="https://zh.wikipedia.org/wiki/R*%E6%A0%91" target="_blank">R* 树</a>来创建空间索引以提高效率，实践中也可以使用其他的空间划分方法，比如 kd 树。算法期望 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 邻域范围内的点的数量比起整个样本集的数量是一个较小的值，因此每次查询的时间复杂度为 <em>O(logn)</em>。由于对列表中的每个点的都要进行一次 regionQuery，因此 DBSCAN 的平均的时间复杂度为 <em>O(n * logn)</em>。</p><p>当 <span><span>ϵ\epsilon</span><span><span><span></span><span>ϵ</span></span></span></span> 较大，且 <span><span>DD</span><span><span><span></span><span>D</span></span></span></span> 数量也比较庞大时，regionQuery 建树的时间消耗会非常大。</p></section></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://zh.wikipedia.org/wiki/DBSCAN" target="_blank">DBSCAN</a></li> <li><a href="https://scikit-learn.org/stable/modules/clustering.html#dbscan" target="_blank">scikit-leran clustering</a></li> <li><a href="https://zhuanlan.zhihu.com/p/643338798" target="_blank">基于密度的聚类算法（1）——DBSCAN详解</a></li> <li><a href="https://zhuanlan.zhihu.com/p/104355127" target="_blank">常用聚类算法</a></li> <li><a href="https://www.youtube.com/watch?v=C3r7tGRe2eI" target="_blank">DBSCAN Clustering Easily Explained with Implementation</a></li> </ul></section>https://yohiro.games/posts/dlss/https://yohiro.games/posts/dlss/Thu, 21 Dec 2023 00:00:00 GMT<section><h2>Overview<a href="#overview"><span>#</span></a></h2><p>我们知道图片放大后，相比于原图片有新产生的像素点，这些像素点通常是通过各种插值产生的，因此，直接放大的图片的质量不佳，往往会显得很糊。大学时的计算机网络课程中曾经提到过奈奎斯特采样定理，说是要恢复失真的信号，采样频率应该大于二倍的最高频率。也就是说在二倍原始采样率的情况下，可以完整的恢复原始图片的质量。但事实是，在实时渲染领域，在一帧之内，由于 GPU 渲染带来的压力，通常没有更多的采样信息用于重建该帧的图像，因此，诞生了很多时域采样的算法用来重建图像。现在游戏界流行的各种 Super Resolution 大多是基于时域重建的原理来重构图像。目前游戏界流行的超分算法有 NVIDIA 的 DLSS，AMD 的 FSR 以及 Intel 的 XeSS，Unreal 里也有内建的 TSR。</p><p>DLSS 的亮点在于它是基于深度学习来重建图像的，比起其它获取时域信息来提高采样率的算法，它重建图像时对像素的评估的更准确。更多细节可以几年前知乎大佬文刀秋二的<a href="https://zhuanlan.zhihu.com/p/123642175" target="_blank">这篇文章</a>。</p><p>NVIDIA 为 Unreal 提供了 DLSS 插件，支持以下特性：</p><p>NVIDIA Deep Learning Supersampling Frame Generation (DLSS-FG) : 适用于 Geforce RTX 40 系显卡的帧生成技术。</p><p>NVIDIA Deep Learning Supersampling Super Resolution (DLSS-SR) : 超分变率的 DLSS，即通常我们玩家社区讨论的广义 DLSS。</p><p>NVIDIA Deep Learning Anti-Aliasing (DLAA) : 深度学习的反走样技术。</p><p>NVIDIA Image Scaling (NIS) : 一种可用于非 NVIDIA 产品的图像放大、锐化技术。</p><p>NVIDIA Ray Reconstruction (DLSS-RR) : 在密集光追场景中，通过在采样的光线间生成高质量像素从而增强画面质量的技术。</p><p>写这篇文章的时候，参考资料来自于 2023 年 5 月 Release 的 DLSS 3.1.13 的附带文档。</p></section> <section><h2>Unreal 支持<a href="#unreal-支持"><span>#</span></a></h2><p><a href="https://developer.nvidia.com/rtx/dlss/get-started#ue-requirements" target="_blank">这里</a>可以下载到 NVIDIA 官方提供的 Unreal Engine 插件。这个插件分为四个部分：</p><ul> <li>DLSS，提供超分/反走样以及即将到来的光线重建</li> <li>DLSSMoviePipelineSupport，为 MovieRenderQueue 出片提供支持</li> <li>NIS，提供图像拉伸的支持</li> <li>Streamline，提供 DLSS 帧生成功能以及 <a href="https://developer.nvidia.com/performance-rendering-tools/reflex" target="_blank">NVIDIA Reflex</a></li> </ul><blockquote><p>根据文档说明，DLSS Sharpen 已被弃用，将来会使用 NIS 单独处理锐化。如果需要兼容将来的版本，应提早规划。 {: .prompt-info }</p></blockquote><p>这篇文章将重点关注 DLSS-SR 和 DLSS-FG 在引擎里的应用以及注意事项。</p></section> <section><h2>DLSS-SR/RR<a href="#dlss-srrr"><span>#</span></a></h2><p>由于 DLSS 取代的是原引擎中 TAAU 的部分，因此要确保 TAAU 是启用了的，可以通过 r.TemporalAA.Upscaler 来检查。</p><p>在此基础上使用以下 Console Variable 开启 DLSS-SR:</p><ol> <li>r.NGX.Enable 1 (也可以通过命令行传参 -ngxenable 或 -ngxdisable 控制开启)</li> <li>r.NGX.DLSS.Enable 1</li> <li>r.ScreenPercentage 66.7 (可以是 50~99 的任意值)</li> </ol><blockquote><p>屏幕分辨率需设置在合理的范围内，对 DLSS-SR 通常是 50~67 这个区间，满分辨率会启用 DLAA。 {: .prompt-info }</p></blockquote><p>默认情况下，编辑器内的关卡视图是不会使用 DLSS 的，如果需要的话要去 Project Settings 的 DLSS 插件设置中勾上。</p><section><h3>Presets<a href="#presets"><span>#</span></a></h3><p>UE 的 DLSS 插件设置里有一个预设选项，枚举了 A~G 共七个种类，猜测是使用了不同类型的游戏去炼丹。</p><p></p><figure><img src="/assets/images/DLSS/presets-unreal.png" alt="presets in unreal" /><figcaption>presets in unreal</figcaption></figure> <em>unreal DLSS 插件中的 presets</em><p></p><p>按照 DLSS Programming Guide 的说法，在默认的 DLSS 配置下可以为画面提供不错的质量，但如果想要在此基础上进一步提升，可以考虑调整 DLSS 的预设，文档中描述了以下几种预设，对应 DLSS 插件中的 A 到 G：</p> <table><thead><tr><th>预设名称</th><th>模式</th><th>说明</th></tr></thead><tbody><tr><td>A</td><td>性能/平衡/画质</td><td>最早适用的预设，能够对抗不同帧间物体运动产生的拖影</td></tr><tr><td>B</td><td>性能最佳</td><td>同预设 A, 但是性能最佳模式</td></tr><tr><td>C</td><td>性能/平衡/画质</td><td>该预设将专注于当前帧的信息，倾向于快节奏的游戏</td></tr><tr><td>D</td><td>性能/平衡/画质</td><td>各模式下的默认预设，倾向于图像稳定的游戏</td></tr><tr><td>E</td><td>-</td><td>占位符，截至目前未使用</td></tr><tr><td>F</td><td>性能最佳</td><td>性能最佳模式以及 DLAA 模式下的默认预设</td></tr><tr><td>G</td><td>-</td><td>占位符，截至目前未使用</td></tr></tbody></table></section><section><h3>DLSS-RR<a href="#dlss-rr"><span>#</span></a></h3><p>如果是使用光追的工程，在使用 DLSS-RR 前需要将引擎的降噪器关掉。此外，在 EnableDLSSRR 函数中也对引擎配置的 CVars 进行了检查。</p><p>DLSS-RR 要检查下面的 CVars:</p><ol> <li>r.NGX.DLSS.denoisermode 1</li> <li>r.Lumen.Reflections.BilateralFilter 0 (运行时更改会触发断言，最好放在引擎的配置文件里)</li> <li>r.Lumen.Reflections.ScreenSpaceReconstruction 0</li> <li>r.lumen.Reflections.Temporal 0</li> <li>r.Shadow.Denoiser 0</li> </ol><p>可以关注 Log 中的内容，这些初始化时的检查会输出到 Log 中，详情可见 DLSSLibrary.cpp。</p></section><section><h3>GPU Debug<a href="#gpu-debug"><span>#</span></a></h3><p>因为使用了 NGX，所以 DLSS 可能会和一些 GPU Debug 工具（如 Renderdoc）不兼容，因此官方建议使用 NSight。</p></section><section><h3>DoF<a href="#dof"><span>#</span></a></h3><p>DLSS 在管线中的位置和原先 UE 中 TAA Upscale 的位置一样，DoF 效果与其他 SS 存在差异。为减少与原先 DoF 效果的差异，推荐将 DLSS 的等级调节为质量或最高质量。</p></section><section><h3>常见的画面瑕疵<a href="#常见的画面瑕疵"><span>#</span></a></h3><ol> <li> <p>Motion vector</p> <ul> <li>Motion vector 错误,可通过 Debug Overlay 进行检查</li> <li>Motion vector 必须为 16/32 位的浮点数，整型会丢弃亚像素信息</li> <li>Motion vector 没有考虑到动态物体的运动</li> </ul> </li> <li> <p>抖动</p> <p>引擎、DLSS 的抖动模式不匹配造成的画面的抖动。</p> </li> <li> <p>TAA/TSR 的影响</p> <p>TAA/TSR 通常会影响渲染时的 Depth/Velocity 以及 Jitter 方式。所以关闭掉 TAA/TSR 可能也会影响 DLSS 的表现。这是因为为减少时域的图像信息的失真，这类算法通常会添加相应的抖动以避免产生额外的画面瑕疵，DLSS 同样需要这些信息。</p> </li> <li> <p>曝光问题</p> <p>正确的曝光/预曝光，尤其是对动态的物体。</p> </li> <li> <p>剩余显存过低的问题</p> <p>在剩余显存过低，渲染资源没能在 GPU 上准备完毕时，DLSS 可能会崩溃。</p> </li> </ol></section></section> <section><h2>DLSS-FG<a href="#dlss-fg"><span>#</span></a></h2><p>对于 5.2 之前的版本，需要去 NVIDIA <a href="https://github.com/NvRTX/UnrealEngine/tree/dlss3/sl2-5.2-dlss-plugin" target="_blank">fork 的版本</a>去找相应的分支合并。对于历史版本还需要打补丁，这些可以参照目录中的相应文档，这里就不再赘述。</p><section><h3>硬件需求<a href="#硬件需求"><span>#</span></a></h3><ul> <li><code>硬件加速 GPU 计划</code>必须启用，详细的看参考<a href="https://devblogs.microsoft.com/directx/hardware-accelerated-gpu-scheduling/" target="_blank">这个</a></li> <li>需要在 NV 的 Ada 架构的 GPU 上运行，也就是 RTX40/RTX6000 系显卡</li> <li>驱动版本 536.99 或以上</li> </ul></section><section><h3>调试<a href="#调试"><span>#</span></a></h3><p>帧生成的插件设置里有个 Debug Overlay 选项，打开后可以看到 Streamline 的 Overlay。</p></section><section><h3>UI alpha<a href="#ui-alpha"><span>#</span></a></h3><p>帧生成可以同时提高 UI 的渲染质量，在 UI 完成渲染后，DLSS-FG 会提取 UI 的 Alpha 通道，UE 在默认情况下并不会清除 SceneColor 的 Alpha，因为不是所有的 UMG 材质的混合模式都输入 Alpha 还写入到像素中。但如果像是 console 窗口，在 UI 元素都渲染完后再次写 Alpha，这就会导致 console 窗口的 Alpha 不被清除，在不同的帧之间持续存在，从而影响生成的帧。</p><p>引擎没有内建的方法只清除 SceneColor 的 Alpha，因此 Streamline 插件在后处理完成之后添加了一个 pass 用于在 UI 完成渲染之前清除 SceneColor 的 Alpha，该 Pass 可由 <code>r.Streamline.ClearSceneColorAlpha</code> 开启。</p><p>最终传递给 Streamline 的 Alpha 只要在等于 0.0 时才会判定为 UI，这个阈值也可以通过 CVar <code>r.Streamline.TagUIColorAlphaThreshold</code> 来控制。</p></section><section><h3>Reflex<a href="#reflex"><span>#</span></a></h3><p>插件提供了 NVIDIA Reflex 的实现，它与 UE 现有的 Reflex 插件不完全兼容，该插件附带未修改的UE版本。因此文档建议禁用 UE 现有的 Reflex 插件，并使用 DLSS 帧生成插件中提供的 Reflex。</p><p>但是使用 UE Reflex 插件的蓝图功能的现有项目应该在添加 UE DLSS Frame Generation 插件后继续工作，因为 Reflex 插件的蓝图应该默认调用 DLSS Frame Generation 插件的模块化功能。</p></section><section><h3>编辑器<a href="#编辑器"><span>#</span></a></h3><p>帧生成目前只应用于打包版本以及编辑器中 PIE 时，因此在正常编辑时是没有帧生成效果的。</p></section><section><h3>VSync<a href="#vsync"><span>#</span></a></h3><p>文档建议在使用帧生成时关掉 VSync，启用该功能时可能会导致 VSync 的效果不正确。</p></section></section> <section><h2>其他细节<a href="#其他细节"><span>#</span></a></h2><section><h3>色彩空间<a href="#色彩空间"><span>#</span></a></h3><p>DLSS 可以处理 LDR/HDR，但对 LDR 有一些需要注意的点：</p><ul> <li>LDR 的色彩空间必须被映射到 0 到 1</li> <li>LDR 模式下，DLSS 使用低精度的 8 位色彩，输入给 DLSS 的 color buffer 需要是线性的编码，如 sRGB</li> <li>LDR 模式下，颜色的数据不能是在线性颜色空间的，如果在 LDR 模式下处理线性空间的颜色，DLSS 会产生 color banding/shifting 等各种视觉问题</li> </ul></section><section><h3>渲染资源的创建标记<a href="#渲染资源的创建标记"><span>#</span></a></h3><ul> <li> <p>传入 DLSS 的 input buffer，比如颜色、运动向量、深度还有历史帧、曝光等，需要时可读的资源状态，像是 SRV/Texture/SampleImage 这种，因为需要传入 ComputeShader。比如在 DX12 下，资源状态的标记应该为 D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE。</p> </li> <li> <p>同理，输出的 output buffer 的资源需要是 UAV（RWTexture/StorageImage），DX12 下的标记需要是 D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS。</p> </li> </ul></section><section><h3>Mipmap<a href="#mipmap"><span>#</span></a></h3><p>由于 DLSS 启用时，贴图是在显示分辨率而不是渲染分辨率下采样的，所以可以考虑使用 mip bias 提升整体的贴图质量，建议计算如下：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>DlssMipLevelBias</span><span> </span><span>=</span><span> </span><span>NativeBias</span><span> </span><span>+</span><span> </span><span>log2</span><span>(</span><span>Render</span><span> </span><span>XResolution</span><span> </span><span>/</span><span> </span><span>Display</span><span> </span><span>XResolution</span><span>) </span><span>-</span><span> </span><span>1.0</span><span> </span><span>+</span><span> </span><span>epsilon</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>如果贴图本身像素是高频、比较密集的，可能不太适合这种方法，最好将该贴图的 mip bias 保持原状。</p></section><section><h3>Motion Vector<a href="#motion-vector"><span>#</span></a></h3><p>DLSS 的关键点之一就是速度图，对每一个像素渲染其运动向量，通过该速度图实现前一帧和后一帧的映射。因此，低精度和错误的速度图通常会造成渲染错误。</p><p>DLSS 速度图的格式为 RG32_FLOAT 或 RG16_FLOAT。因此，如果引擎自定义了速度图的格式，传给 DLSS 前需要进行解码。</p><p>DLSS 计算运动向量的方式与 Unreal 有所不同，文档中给出了这样的 shader 代码用于将 Unreal 计算的运动向量转为 DLSS 可用的：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>Texture2D DepthTexture;</span></div></div><div><div><div>2</div></div><div><span>Texture2D VelocityTexture;</span></div></div><div><div><div>3</div></div><div> </div></div><div><div><div>4</div></div><div><span>float2 </span><span>UVToClip</span><span><span>(</span><span>float2 </span><span>UV</span><span>)</span></span></div></div><div><div><div>5</div></div><div><span>{</span></div></div><div><div><div>6</div></div><div><span> </span><span>return</span><span> </span><span>float2</span><span>(</span><span>UV</span><span>.</span><span>x</span><span> </span><span>*</span><span> </span><span>2</span><span> </span><span>-</span><span> </span><span>1</span><span>, </span><span>1</span><span> </span><span>-</span><span> </span><span>UV</span><span>.</span><span>y</span><span> </span><span>*</span><span> </span><span>2</span><span>);</span></div></div><div><div><div>7</div></div><div><span>}</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span>float2 </span><span>ClipToUV</span><span><span>(</span><span>float2 </span><span>ClipPos</span><span>)</span></span></div></div><div><div><div>10</div></div><div><span>{</span></div></div><div><div><div>11</div></div><div><span> </span><span>return</span><span> </span><span>float2</span><span>(</span><span>ClipPos</span><span>.</span><span>x</span><span> </span><span>*</span><span> </span><span>0.5</span><span> </span><span>+</span><span> </span><span>0.5</span><span>, </span><span>0.5</span><span> </span><span>-</span><span> </span><span>ClipPos</span><span>.</span><span>y</span><span> </span><span>*</span><span> </span><span>0.5</span><span>);</span></div></div><div><div><div>12</div></div><div><span>}</span></div></div><div><div><div>13</div></div><div> </div></div><div><div><div>14</div></div><div><span>float3 </span><span>HomogenousToEuclidean</span><span><span>(</span><span>float4 </span><span>V</span><span>)</span></span></div></div><div><div><div>15</div></div><div><span>{</span></div></div><div><div><div>16</div></div><div><span> </span><span>return</span><span> </span><span>V</span><span>.</span><span>xyz</span><span> </span><span>/</span><span> </span><span>V</span><span>.</span><span>w</span><span>;</span></div></div><div><div><div>17</div></div><div><span>}</span></div></div><div><div><div>18</div></div><div> </div></div><div><div><div>19</div></div><div><span>void</span><span> </span><span>VelocityResolvePixelShader</span><span>(</span></div></div><div><div><div>20</div></div><div><span><span> </span></span><span>float2 InUV : TEXCOORD0</span><span>,</span></div></div><div><div><div>21</div></div><div><span><span> </span></span><span>float4 SvPosition : SV_Position</span><span>,</span></div></div><div><div><div>22</div></div><div><span><span> </span></span><span>out float4 OutColor : SV_Target0 </span><span>)</span></div></div><div><div><div>23</div></div><div><span>{</span></div></div><div><div><div>24</div></div><div><span><span> </span></span><span>OutColor </span><span>=</span><span> </span><span>0</span><span>;</span></div></div><div><div><div>25</div></div><div><span><span> </span></span><span>float2 Velocity </span><span>=</span><span> </span><span>VelocityTexture</span><span>[</span><span>SvPosition</span><span>.</span><span>xy</span><span>].</span><span>xy</span><span>;</span></div></div><div><div><div>26</div></div><div><span> </span><span>float</span><span> Depth </span><span>=</span><span> </span><span>DepthTexture</span><span>[</span><span>SvPosition</span><span>.</span><span>xy</span><span>].</span><span>x</span><span>;</span></div></div><div><div><div>27</div></div><div> </div></div><div><div><div>28</div></div><div><span> </span><span>if</span><span> (</span><span>all</span><span>(</span><span>Velocity</span><span>.</span><span>xy</span><span> </span><span>&gt;</span><span> </span><span>0</span><span>))</span></div></div><div><div><div>29</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>30</div></div><div><span><span> </span></span><span>Velocity </span><span>=</span><span> </span><span>DecodeVelocityFromTexture</span><span>(Velocity);</span></div></div><div><div><div>31</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>32</div></div><div><span> </span><span>else</span></div></div><div><div><div>33</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>34</div></div><div><span><span> </span></span><span>float4 ClipPos;</span></div></div><div><div><div>35</div></div><div><span> </span><span>ClipPos</span><span>.</span><span>xy</span><span> </span><span>=</span><span> </span><span>SvPositionToScreenPosition</span><span>(</span><span>float4</span><span>(</span><span>SvPosition</span><span>.</span><span>xyz</span><span>, </span><span>1</span><span>)).</span><span>xy</span><span>;</span></div></div><div><div><div>36</div></div><div><span> </span><span>ClipPos</span><span>.</span><span>z</span><span> </span><span>=</span><span> Depth;</span></div></div><div><div><div>37</div></div><div><span> </span><span>ClipPos</span><span>.</span><span>w</span><span> </span><span>=</span><span> </span><span>1</span><span>;</span></div></div><div><div><div>38</div></div><div> </div></div><div><div><div>39</div></div><div><span><span> </span></span><span>float4 PrevClipPos </span><span>=</span><span> </span><span>mul</span><span>(ClipPos, </span><span>View</span><span>.</span><span>ClipToPrevClip</span><span>);</span></div></div><div><div><div>40</div></div><div> </div></div><div><div><div>41</div></div><div><span> </span><span>if</span><span> (</span><span>PrevClipPos</span><span>.</span><span>w</span><span> </span><span>&gt;</span><span> </span><span>0</span><span>)</span></div></div><div><div><div>42</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>43</div></div><div><span><span> </span></span><span>float2 PrevClip </span><span>=</span><span> </span><span>HomogenousToEuclidean</span><span>(PrevClipPos).</span><span>xy</span><span>;</span></div></div><div><div><div>44</div></div><div><span><span> </span></span><span>Velocity </span><span>=</span><span> </span><span>ClipPos</span><span>.</span><span>xy</span><span> </span><span>-</span><span> </span><span>PrevClip</span><span>.</span><span>xy</span><span>;</span></div></div><div><div><div>45</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>46</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>47</div></div><div> </div></div><div><div><div>48</div></div><div><span> </span><span>OutColor</span><span>.</span><span>xy</span><span> </span><span>=</span><span> Velocity </span><span>*</span><span> </span><span>float2</span><span>(</span><span>0.5</span><span>, </span><span>-</span><span>0.5</span><span>) </span><span>*</span><span> </span><span>View</span><span>.</span><span>ViewSizeAndInvSize</span><span>.</span><span>xy</span><span>;</span></div></div><div><div><div>49</div></div><div><span> </span><span>OutColor</span><span>.</span><span>xy</span><span> </span><span>=</span><span> </span><span>-</span><span>OutColor</span><span>.</span><span>xy</span><span>;</span></div></div><div><div><div>50</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div></section><section><h3>Conservative Rasterization<a href="#conservative-rasterization"><span>#</span></a></h3><p>如果画面中存在一些细小的对象，它的信息有可能会在渲染时丢失，比如使用较低分辨率渲染，或者因像素过少被光栅器丢弃。DLSS 可以重建这种丢失的细节，但前提是要有该对象的运动信息。对细小像素应用的 Conservative Rasterization^{<a href="#user-content-fn-ref1">1</a>}^{<a href="#user-content-fn-ref2">2</a>}，允许即使十分细微的像素也可以光栅化到图像上，借助于此，可以构建更多细节。</p></section><section><h3>亚像素抖动<a href="#亚像素抖动"><span>#</span></a></h3><p>为模拟高采样率，DLSS 通过应用亚像素抖动的方法来让渲染器提供的额外采样点。</p><p>3D 渲染中有很多可用的抖动方式，考虑到最终效果，抖动方式需要有比较好的像素覆盖率。NV 训练时，发现使用 <a href="https://en.wikipedia.org/wiki/Halton_sequence" target="_blank">Halton sequence</a> 可以取得最好的画面效果。Halton 序列是一种低差异序列，可以生成分布均匀的样本点。</p><p>Halton 序列在 Unreal 里地形上的草地、大气雾以及光照贴图等中都有应用。</p></section></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2></section> <section><h2>Footnotes<a href="#footnote-label"><span>#</span></a></h2> <ol> <li> <p><a href="https://learn.microsoft.com/en-us/windows/win32/direct3d12/conservative-rasterization" target="_blank">Direct3D 12 Conservative Rasterization</a> <a href="#user-content-fnref-ref1">↩</a></p> </li> <li> <p><a href="https://developer.nvidia.com/gpugems/gpugems2/part-v-image-oriented-computing/chapter-42-conservative-rasterization" target="_blank">Chapter 42. Conservative Rasterization</a> <a href="#user-content-fnref-ref2">↩</a></p> </li> </ol> </section>https://yohiro.games/posts/game/https://yohiro.games/posts/game/Tue, 28 Nov 2023 00:00:00 GMT<section><h2>Intro<a href="#intro"><span>#</span></a></h2><p>UE5 已经 release 很久了，为了解当时看起来很酷炫的特性在正式游戏中的应用情况，这里追踪了 Remnant2 以及 RoboCop 的渲染情况。为获得尽可能真实可信的数据，以下抓帧时的数据均在关闭了诸如 DLSS 之类的超分技术和动态分辨率，中画质 1080p 且关闭垂直同步的情况下获取。</p><p>硬件环境：</p><ul> <li>i7-10700 2.90GHz</li> <li>NVIDIA RTX 2060</li> </ul></section> <section><h2>Remnant2<a href="#remnant2"><span>#</span></a></h2><p>Remnant2 中对新特性的使用非常保守，仅开启了 Nanite。抓帧的位置是新游戏的出生点，具有较多的植被，部分数冠的顶点数非常多。在该场景的 BasePass 中可以看到，所有的 Draw 都来自于 Nanite。</p><p></p><figure><img src="/assets/images/UE5Game/Remnant2.png" alt="Remnant2" /><figcaption>Remnant2</figcaption></figure><p></p><hr /> <table><thead><tr><th>Pass</th><th>Number</th><th>GPU Time^{<a href="#user-content-fn-tip">1</a>}</th></tr></thead><tbody><tr><td>PrePass</td><td>357</td><td>3.96</td></tr><tr><td>BasePass</td><td>453</td><td>2.92</td></tr><tr><td>ShadowDepth</td><td>485</td><td>15.36</td></tr><tr><td>Translucent</td><td>60</td><td>0.12</td></tr></tbody></table><section><h3>Virtual Shadow Map<a href="#virtual-shadow-map"><span>#</span></a></h3><p>根据 Overview 中的表格，ShadowDepth 占用了最多的 GPU 时间，而且 PrePass 耗时也超过了 BasePass，可见整个场景的瓶颈在于<code>顶点数过多</code>。</p><p>在开启了 Nanite 的情况下，Remnant2 将启用 VSM 的选项放在了设置的图形菜单中，将开启与否的选择权交给了玩家。</p><p></p><figure><img src="/assets/images/UE5Game/Menu_VSM.png" alt="Menu_VSM" /><figcaption>Menu_VSM</figcaption></figure>{: .shadow .rounded-10 } <em>该选项就是开启 VSM</em><p></p><p>根据官方文档中所说^{<a href="#user-content-fn-vsm">2</a>}，Nanite几何体始终渲染到虚拟阴影贴图，是性能最高的选项，可提供最高质量。然而这个游戏中并没有默认开启 VSM。</p><p>关闭 VSM 后的 Shadow Map 是多个 1024 x 1024 的并到一个大的 Shadow Map 中，此处有七个投射阴影的光源。手动开启 VSM 后，发现渲染线程时间和 GPU 时间均获得了近 2 ms 左右的提升。VSM 的 PagePool 大小为 16384 x 1024。</p><p>从抓帧的数据来看，耗时较高的几个 DrawCall 都是来自于 ShadowDepth Pass 中，考虑下面的这个 DrawCall:</p><p></p><figure><img src="/assets/images/UE5Game/ShadowDepth_Hot.png" alt="ShadowDepth" /><figcaption>ShadowDepth</figcaption></figure>{: .shadow .rounded-10 } <em>某个 PLA 的 ShadowDepth</em><p></p><p>直接使用了 LOD 0 的 Mesh 去投射阴影，除了这个 DrawCall 外还有许多零碎的小物件，这些都增加了 ShadowDepth 的负荷。针对这种情况，通常的方案是使用代理网格或是低级的 LOD 来投射阴影。如果是因为 ShadowMap 的分辨率问题导致的 Cache miss 还要考虑 ShadowMap 的压缩。</p><p>Unreal 中提供了多种阴影方案，这个游戏中也使用了不止一种，这里仅考虑高顶点数造成瓶颈的问题。</p></section><section><h3>Nanite<a href="#nanite"><span>#</span></a></h3><p>开启与关闭 Nanite，Stats 信息中的 Prims 数量有着巨大的差别，这是因为开启 Nanite 后，GNumPrimitivesDrawnRHI 中没有统计 Nanite 的图元数量，如果需要观测 Nanite 的情况，需要使用 <code>r.Nanite.ShowStats</code> 来开启 Nanite 的 Stats 面板。</p><p>Nanite 效果最为明显的是树冠，下面的树的 Nanite Fallback Mesh 有 256,891 个顶点，在 PrePass, BasePass 以及 ShadowDepth 均有较高的耗时。</p><p></p><figure><img src="/assets/images/UE5Game/Foliage_Hot.png" alt="Tree" /><figcaption>Tree</figcaption></figure>{: .shadow .rounded-10 } <em>Nsight 中树的 Geometry</em><p></p></section><section><h3>Lumen<a href="#lumen"><span>#</span></a></h3><p>根据工作室采访时所说^{<a href="#user-content-fn-ref">3</a>}，Remnant2 中并没有使用 Lumen，通过导出的设置来看，Mesh 也没有生成 SDF。取而代之的是 Unreal 5.x 中的 Fallback 方案，即使用了 SSGI 的方式模拟全局光，而且是<code>半分辨率</code>的 SSGI。</p><p>严格来说，开启 GI 后，AO 的开启与否是需要讨论的。一方面是因为 GI 已经提供了具有真实感的光照和阴影，另一方面是因为当前的 GI 方案并不完善，总是存在没有捕捉到的地方，尤其是对 SSGI 而言。因此在角落、拐角、裂缝等处的 AO 依然是必要的。</p><p>所以 SSGI + DFAO + SSR 可以作为一个 Lumen 的替代方案。不过，上个月的 Unreal Fest Day^{<a href="#user-content-fn-tip1">4</a>} 中，Epic 的一位老哥给给出了这样的建议：</p><p></p><figure><img src="/assets/images/UE5Game/LumenFallback.png" alt="LumenFallback" /><figcaption>LumenFallback</figcaption></figure>{: .shadow .rounded-10 w=‘767’ h=‘390’ } <em>详情见油管视频 52min47s</em><p></p><p>即关闭 GI 用 AO + SSR 作为 Fallback 方案，回归原先 4.x 的管线。</p><hr /></section></section> <section><h2>RoboCop<a href="#robocop"><span>#</span></a></h2><p>RoboCop 使用了 UE5 所有的新特性，是非常标准的 5.x 的游戏。抓帧的位置同样来自游戏的出生点，场景是位于小巷中，因此顶点数不像前者那么多，地面的积水部分比较能反映 Lumen 的效果。</p><p></p><figure><img src="/assets/images/UE5Game/RoboCop.png" alt="RoboCop" /><figcaption>RoboCop</figcaption></figure><p></p><section><h3>Virtual Shadow Map<a href="#virtual-shadow-map-1"><span>#</span></a></h3><p>VSM 的 PhysicalPagePool 在引擎中默认的分辨率是 16384 x 4096，RoboCop 里使用的是 16384 x 2048，最大的 Page 限定在 512，Page 大小不够会造成 Page Pool Overflow 使阴影消失/变为块状，后台打开抓帧的 Renderdoc 时很容易可以重现这种情景。</p><p>此处有 7 处投射阴影的动态光源，因此在关闭 VSM 后绘制了 7 张 512 x 512 的 Shadow Map。在关闭 VSM 后，阴影质量被设为了最低，512 的分辨率对于 Shadow Map 来说非常低，但是抓帧的这个场景比较黑看不出来差别。</p><p>在显存比较吃紧的低端设备，可以关掉 VSM，回到 CSM。</p></section><section><h3>Nanite<a href="#nanite-1"><span>#</span></a></h3><p>除了目前尚不支持的对蒙皮网格、透明物件的渲染以及贴花以外，场景中的 Mesh 几乎都使用了 Nanite。但是不同于前者，在全局关掉 Nanite 的支持后，没有使用 Fallback Mesh 替代，应该是修改了 ProxyRenderMode。</p><p>而且这个游戏也允许走软光栅的（一般是应用了 WPO 的材质） Mesh 进入 Shadow Pass。</p></section><section><h3>Lumen<a href="#lumen-1"><span>#</span></a></h3><p>Lumen 的耗时在我的机器上是 5.4 ms 左右，按照 Epic 老哥的说法^{<a href="#user-content-fn-tip1">4</a>}，在 60 fps 下 Lumen 的 Budget 应该限制在 4 ms, 在 30 fps 下应该限制在 8 ms。</p><p>下面的反射使用了较为粗糙的追踪，应该是关掉了 TraceMeshSDFs。</p><p></p><figure><img src="/assets/images/UE5Game/ReflectionCompare.png" alt="Lumen_ReflectionCompare" /><figcaption>Lumen_ReflectionCompare</figcaption></figure>{: .shadow .rounded-10 }<p></p><p>在使用 GI 的情况下，可以考虑仅改变不同的反射方法进行性能分级。这里是两个不同的反射方法的效果对比（Lumen vs SSR）：</p><p></p><figure><img src="/assets/images/UE5Game/RoboCop_Reflection.png" alt="Comparsion of Lumen and SSR" /><figcaption>Comparsion of Lumen and SSR</figcaption></figure>{: .shadow .rounded-10 } <em>Comparsion of Lumen and SSR (1)</em><p></p><p></p><figure><img src="/assets/images/UE5Game/RoboCop_Reflection2.png" alt="Comparsion of Lumen and SSR" /><figcaption>Comparsion of Lumen and SSR</figcaption></figure>{: .shadow .rounded-10 } <em>Comparsion of Lumen and SSR (2)</em><p></p><p>RoboCop 的 DiffuseIndirect 进行了下采样，在中高画质下均开启了 Lumen，在低画质下关掉了 GI，也关掉了反射。</p></section></section> <section><h2>其他<a href="#其他"><span>#</span></a></h2><p>有些其他的发现，比如 RoboCop 中的角色皮肤使用了半分辨率的次表面散射。对于 SceneColor 的格式，Remnant2 使用了 RGBA，而 RoboCop 使用了 RGB + SeparateTranslucency。</p><p>对 RoboCop 有一点体验不佳的是，在第一次启动游戏构建 PSO 时没有任何提示页面，有一段很长时间的卡顿。XGP 版本的 Lies of P 则是另一个问题，每次启动都有重编 PSO 的界面，猜测应该是个 bug。</p></section> <section><h2>Summary<a href="#summary"><span>#</span></a></h2><p>UE5 的新特性的使用是建立在硬件规格的提升之上的，目前看来在 Nanite + Lumen + VSM + TSR/DLSS/FSR/XeSS 这样一个体系下，才能取得最佳效果。</p><p></p><figure><img src="/assets/images/UE5Game/5dotx.png" alt="5.x features" /><figcaption>5.x features</figcaption></figure> <em>图片来自 Unreal Fest Day 2 | Livestream 2 Pt. 2^{<a href="#user-content-fn-fest">5</a>}</em><p></p></section> <section><h2>References<a href="#references"><span>#</span></a></h2></section> <section><h2>Footnotes<a href="#footnote-label"><span>#</span></a></h2> <ol> <li> <p>GPU Time 来自 Nsight 中的数据，具体到某一 Pass 并不准确，相对大小可供参考。 <a href="#user-content-fnref-tip">↩</a></p> </li> <li> <p><a href="https://docs.unrealengine.com/5.3/zh-CN/virtual-shadow-maps-in-unreal-engine/" target="_blank">虚拟阴影贴图</a> <a href="#user-content-fnref-vsm">↩</a></p> </li> <li> <p><a href="https://wccftech.com/remnant-ii-is-powered-by-ue5-though-it-doesnt-use-lumen-yet/" target="_blank">Remnant II Is Powered by UE5, Though It Doesn’t Use Lumen Yet</a> <a href="#user-content-fnref-ref">↩</a></p> </li> <li> <p><a href="https://www.youtube.com/watch?v=Cb63bHkWkwk&amp;t=7701s" target="_blank">Unreal Fest Day 1 | Livestream 2</a> <a href="#user-content-fnref-tip1">↩</a> <a href="#user-content-fnref-tip1-2">↩²</a></p> </li> <li> <p><a href="https://www.youtube.com/watch?v=8eO2xdrDms8" target="_blank">Unreal Fest Day 2 | Livestream 2 Pt. 2</a> <a href="#user-content-fnref-fest">↩</a></p> </li> </ol> </section>https://yohiro.games/posts/blendshape-with-pose-wrangler/https://yohiro.games/posts/blendshape-with-pose-wrangler/Wed, 11 Oct 2023 00:00:00 GMT<blockquote><p>新版 Pose Wrangler 插件已经发布在了 Epic 商城中，新版插件添加了 Driver Bone 的导出模块，及其导入引擎的插件。可以<a href="https://www.unrealengine.com/marketplace/en-US/product/pose-driver-connect" target="_blank"><strong>点击这里</strong></a>获取新版插件。新版插件中对 BlendShape 的支持依然是 Experimental，具体的教程视频以及讨论串可以访问<a href="https://forums.unrealengine.com/t/tutorial-authoring-secondary-animation-in-maya-and-unreal-with-pose-driver-connect/1496699" target="_blank"><strong>这个帖子</strong></a> {: .prompt-warning }</p></blockquote> <section><h2>背景<a href="#背景"><span>#</span></a></h2><p>Pose Wrangler 是 EpicGames 提供的Maya开源工具，原始的仓库在<a href="https://github.com/chrisevans3d/poseWrangler" target="_blank"><strong>这里</strong></a>。也可以<a href="https://github.com/YohiroW/poseWrangler" target="_blank">获取个人 fork 的版本</a>，添加了对导出 fbx 以及 json 的支持。</p><p>使用 PoseWrangler 创建 Blendshape 有在 DCC 的准备步骤和导入引擎后的步骤，下面将从 Maya 和 UnrealEngine 两端进行说明。</p></section> <section><h2>Maya<a href="#maya"><span>#</span></a></h2><section><h3>界面<a href="#界面"><span>#</span></a></h3><p>根据文档，python 脚本编辑器内输入以下代码以打开插件界面，Driver</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>from</span><span> epic_pose_wrangler </span><span>import</span><span> main</span></div></div><div><div><div>2</div></div><div><span><span>pose_wrangler </span><span>=</span><span> main.</span><span>PoseWrangler</span><span>()</span></span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>可以保存到工具架，方便下次使用。</p></section><section><h3>创建解算器<a href="#创建解算器"><span>#</span></a></h3><p>首先需要在 PoseWrangler 中根据骨骼创建 RBF 解算器，RBF 是指 <em>Radial Based Function</em>，即<em>径向基函数</em>，在建模领域主要应用于肌肉、表情等形变方向。</p><ol> <li>点击 <kbd>Create Solver</kbd> 以创建解算器，初次创建的解算器会自动进入编辑状态</li> <li>如果没有进入编辑状态，点击 <kbd>Edit Solver</kbd> 以进入编辑状态（编辑完成需要手动点击 Finish Editing）</li> </ol><p><strong>Tips</strong>：可以通过导入预设的 json 文件，以减少工作量。</p></section><section><h3>创建 Pose<a href="#创建-pose"><span>#</span></a></h3><p>在创建完骨骼对应的解算器后，进入针对当前解算器的编辑状态。</p><ol> <li>在大纲视图中选中驱动骨骼后，点击 <kbd>Add Driver Transform</kbd></li> <li>点击 <kbd>AddPose</kbd> 以添加 Pose</li> <li>在 Maya 内的编辑视口内控制骨骼，调整为需要编辑的 Pose</li> <li>调整完成后，点击 <kbd>Update Pose</kbd> 更新 Pose</li> <li>修改 Pose 对应的名称</li> </ol><p><strong>Note</strong>: 在 Pose 列表为空时，添加驱动骨骼时将会创建名为 default 的默认 Pose，如果需要使用插件编辑 blendshape，那么该 Pose 需予以保留。v1.0 版本中的默认 Pose 的命名 base_pose 在 v2.0 中将不支持。</p></section><section><h3>创建 Blendshape<a href="#创建-blendshape"><span>#</span></a></h3><ol> <li>选中当前的解算器和需要修型的 Pose</li> <li>在大纲视图中选中需要添加 Blendshape 的 Mesh 后，点击 <kbd>Create Blendshape</kbd></li> <li>创建了 Blendshape 后会自动创建原 Mesh 的副本， 并且选中该副本进入编辑 Blendshape 的状态</li> <li>编辑完该 Pose 后，点击 <kbd>Finish Editing</kbd> 完成对 Blendshape 的编辑</li> </ol></section><section><h3>导出 fbx<a href="#导出-fbx"><span>#</span></a></h3><ol> <li>点击 <kbd>Bake Poses To Timeline</kbd> 将当前解算器所对应的 Pose 作为帧动画烘焙到时间轴上。该动画会在导入引擎后产生 Anim Sequence 文件，我们需要使用 Anim Sequence 文件来创建引擎内可使用的 Pose 资产。</li> <li>选中骨骼和几何体导出为 fbx，确保 Animation 和 Blendshape 能够被导出。</li> </ol></section></section> <section><h2>UnrealEngine<a href="#unrealengine"><span>#</span></a></h2><section><h3>导入 fbx<a href="#导入-fbx"><span>#</span></a></h3><ol> <li>在 fbx 的导入选项中，确保勾选 <code>Import morph targets</code></li> <li>如果需要将曲线导入骨骼中，要在 fbx 的导入选项中，确保勾选 <code>Add Curve Metadata to Skeleton</code></li> </ol><p>如果正确的话，此时可以在 Skeletal Mesh 的 <code>Morph Target Preview</code> 标签窗口下看到所有在 Maya 中创建的 Blendshape。</p></section><section><h3>准备 Pose 资产<a href="#准备-pose-资产"><span>#</span></a></h3><p>需要借助动画蓝图中的 <code>Pose Driver</code> 节点驱动 Blendshape，而使用 PoseDriver 节点需要在引擎内创建 Pose 资产。</p><ol> <li>首先需要确保，导入到引擎的动画序列中拥有动画曲线的信息</li> <li>在 ContentBrowser 里找到导入引擎中的动画序列 ，右键可以创建 Pose 类型的资产。以这种方式创建的 Pose 资产会将动画序列中的每一帧提取为一个 Pose ，其顺序与在 Maya 中 Pose Wrangler 的 Pose 列表一致</li> <li>按顺序输入 Pose 的名称即可</li> </ol></section><section><h3>动画蓝图<a href="#动画蓝图"><span>#</span></a></h3><p>测试用动画蓝图只包含两个节点</p><ul> <li>InputPose 用于指定当前的 Pose</li> <li>Pose Driver 将会根据 Pose 资产中定义的一系列 Pose 去驱动指定的骨骼</li> </ul><p>测试用的 Pose Driver 节点的参数较为简单:</p><ol> <li>指定 Pose 资产</li> <li>指定源骨骼为 Maya 中编辑的骨骼</li> <li>RBF 参数中的解算半径设为自动</li> <li>Pose Target 这里直接从 Pose 资产中拷贝</li> </ol></section><section><h3>预览<a href="#预览"><span>#</span></a></h3><p>将该动画蓝图作为 Skeletal Mesh 的后处理动画蓝图即可预览</p><p>效果如下：</p> <table><thead><tr><th>无 Blendshape</th><th>有 Blendshape</th></tr></thead><tbody><tr><td><figure><img src="/assets/images/blendshape/without_bs.png" alt="without blendshape" /><figcaption>without blendshape</figcaption></figure></td><td><figure><img src="/assets/images/blendshape/with_bs.png" alt="with blendshape" /><figcaption>with blendshape</figcaption></figure></td></tr></tbody></table></section></section> <section><h2>TroubleShooting<a href="#troubleshooting"><span>#</span></a></h2><ul> <li>确保当前所使用的 PoseWrangler 的版本是 2.x 版本，非 2.x 版本没有创建 blendshape 的选项</li> <li>需确保 Maya 中的各节点没有使用命名空间</li> <li>需确保骨骼和 Mesh 的原点都位于（0，0，0）</li> <li>创建的 Pose 时, 默认的 TPose 需更名为 <code>default</code></li> </ul></section>https://yohiro.games/posts/occlusion-in-unreal/https://yohiro.games/posts/occlusion-in-unreal/Wed, 05 Jul 2023 00:00:00 GMT<section><h2>概述<a href="#概述"><span>#</span></a></h2><p>最近需要利用 Occlusion Query 记录场景中可见物件的信息。为了验证方案的可行性，我决定先在开启 Occlusion Cull 的情况下，从引擎的 View.PrimitiveVisibilityMap 中回读 PrimitiveIndex 来确定可见物件。</p><p>然后发现 5.3 版本 SceneVisibility 这块变动挺大的，很多面熟的老函数不见了。查提交发现在引擎 5.3.2 版本中，可见性这块进行了重构，详情可见<a href="https://github.com/EpicGames/UnrealEngine/commit/9cd755694f97946ad0e84806250d9fdf428cefc7#diff-f521e57df7b2dd21cce113d087ba67ccadedb1b5c479916e2be97dfab6fd1caf" target="_blank">这个提交</a>。该提交将 Occlusion Query 任务分配到了异步的 TaskGraph 中，还暂时移除了 PrecomputedVisibility（应该是 bug？）。</p><p>最后把 Occlusion Cull 这块重新过了一遍，这里留个档记录一下。</p></section> <section><h2>Occlusion Cull<a href="#occlusion-cull"><span>#</span></a></h2><section><h3>简介<a href="#简介"><span>#</span></a></h3><p>Occlusion Query（遮挡查询） 或 Occlusion Cull（遮挡裁剪） 很多时候混为一谈，指的是对摄像机视锥内将被遮挡几何体剔除掉的技术，但是实际上是查询未必要剔除，剔除必然伴随着查询的。Query 是 GPU 提供的用于查询渲染资源状态的对象。通过 Query 我们可以查询到 Occlusion/ TimeStamp/ PSO状态 等。Occlusion Cull 是通过 Occlusion Query 是否有像素被绘制来确定是否被遮挡，从而决定几何体的可见性的。</p><p>一个简单的 Occlusion Cull 的大致流程如下：</p><ol> <li>创建一个 Query 用作请求查询</li> <li>禁止颜色写入，所有通道的 ColorMask 设为 false</li> <li>禁止深度写入</li> <li>通知 GPU 开始 Query，此时会重置可见像素的数量</li> <li>在一个 Depth Only 的 Pass 中渲染场景中物体的包围盒做深度测试</li> <li>结束 Query，停止记录可见像素的数量</li> <li>开启颜色写入（如有必要，也可开启深度写入）</li> <li>获取 Query 的结果，即记录的可见像素的数量</li> <li>根据可见像素的数量判断该物体是否绘制</li> </ol></section><section><h3>问题<a href="#问题"><span>#</span></a></h3><p>上述的 Occlusion Cull 流程带来了两个问题，分别是 CPU 与 GPU 的同步问题以及遮挡细节带来的准确度问题。</p><section><h4>同步<a href="#同步"><span>#</span></a></h4><p>CPU 和 GPU 的任务之间并不是同步进行的。比如，当 CPU 给 GPU 创建一个渲染指令时，CPU 不会在原地等待 GPU 的任务执行完成，而是继续去执行 CPU 上的其他任务，创建的渲染指令会被驱动程序发送给 GPU 的 CommandQueue，当这些命令执行完毕，便完成渲染。</p><p>但 Occlusion Query 的流程的问题在于，CPU 需要回读 GPU 端渲染包围盒的结果后才知道是否可见，从而进行裁剪。这意味者 CPU 必须在某一刻等待 GPU 完成查询，这个过程破坏了二者的并行性，从而大大降低了渲染效率。一个常用的解决方式是让 CPU 回读上一帧的 Query 结果，如果相机运动过快可能会出现错误的渲染，但由于 Occlusion Query 是基于包围盒进行裁剪，是一种保守的裁剪方式，因此影响不是非常明显。</p><p></p><figure><img src="/assets/images/OcclusionQuery/await.jpg" alt="sync" /><figcaption>sync</figcaption></figure> <em>CPU/GPU 以及 Occlusion Cull 的同步</em><p></p><p>之前玩对马岛时，在室内场景快速旋转相机，有时会出现白色的谜之闪烁，应该就是和 Occlusion Query 的同步有关。</p></section><section><h4>遮挡<a href="#遮挡"><span>#</span></a></h4><p>另一个带来的问题是使用包围盒来确认可见性带来的不精确的遮挡，考虑下图的这种情况： </p><figure><img src="/assets/images/OcclusionQuery/fig01.jpg" alt="Occlusion Issue" /><figcaption>Occlusion Issue</figcaption></figure><p></p><p>树的包围盒遮挡了车的包围盒，但是对于实际的像素而言，车并没有完全被树所遮挡。</p><p>一种解决方式是将一些几何体作为 Occluder 单独渲染，不经过遮挡剔除。渲染完成后，再针对较小的几何体做遮挡剔除。</p></section></section><section><h3>效率<a href="#效率"><span>#</span></a></h3><p>在虚幻的官方文档^{<a href="#user-content-fn-offcialdocument">1</a>}中，对引擎内各种裁剪方式进行了简单的介绍，在 Hardware Occlusion Queries 中提到：</p><blockquote><p>The cost of hardware occlusion scales with the number of queries performed on the GPU. Use Distance and Precomputed Visibility methods to reduce the number of queries performed each frame by the GPU.</p></blockquote><p>也就是说，在使用 Occlusion Query 时应该注意一个平衡：</p><p><em>通过 Occlusion Culling 裁剪获得的性能提升应该大于 Occlusion Test 产生的额外性能消耗，否则 Occlusion Query 的引入会对性能产生负面影响。</em></p></section></section> <section><h2>虚幻中的 Occlusion Query<a href="#虚幻中的-occlusion-query"><span>#</span></a></h2><p>UE 在 Occlusion Query 之前，在 FrustumCull 中完成了 Distance Cull 然后在后续完成了 Precomputed Visibility 的检查，引擎采用这些操作以减少 Occlusion Test 过程中产生的 Draw call。此外，如前文所说，为解决 Occlusion Query 产生的 CPU/GPU 同步点的问题，虚幻采用的也是回读历史帧查询结果的方式，通过获取到的历史帧结果，以最大限度的减少 Occlusion Test 的消耗。</p><section><h3>大致流程<a href="#大致流程"><span>#</span></a></h3><p>该流程参考基于 Unreal Engine 5.3.2 版本。</p><section><h4>准备渲染<a href="#准备渲染"><span>#</span></a></h4><p>在当前版本，重构后的 LaunchVisibilityTasks 替代了原先的 ComputeViewVisibility。Visibility 的计算被分解为一个个 FVisibilityViewPacket 分配到 TaskGraph 中异步处理。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>void</span><span><span> </span><span>FVisibilityTaskData</span><span>::</span></span><span>LaunchVisibilityTasks</span><span>()</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>// ...</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span><span> </span></span><span>// All task events are connected to prerequisites now and can be safely triggered.</span></div></div><div><div><div>6</div></div><div><span> </span><span>Tasks</span><span>.</span><span>BeginInitVisibility</span><span>.</span><span>Trigger</span><span>();</span></div></div><div><div><div>7</div></div><div><span> </span><span>Tasks</span><span>.</span><span>LightVisibility</span><span>.</span><span>Trigger</span><span>();</span></div></div><div><div><div>8</div></div><div><span> </span><span>Tasks</span><span>.</span><span>FrustumCull</span><span>.</span><span>Trigger</span><span>();</span></div></div><div><div><div>9</div></div><div><span> </span><span>Tasks</span><span>.</span><span>OcclusionCull</span><span>.</span><span>Trigger</span><span>();</span></div></div><div><div><div>10</div></div><div><span> </span><span>Tasks</span><span>.</span><span>ComputeRelevance</span><span>.</span><span>Trigger</span><span>();</span></div></div><div><div><div>11</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>每个 FVisibilityViewPacket 会先做 Frustum Cull 筛掉不在视锥内和被 Distance cull 裁掉的几何体，然后再根据 Precomputed Visibility 筛掉一部分。</p><p>然后在等待上一轮 OcclusionQuery 的结果返回后，创建新一轮的 OcclusionCullTask:</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span>FVisibilityViewPacket</span><span>::</span></span><span>FVisibilityViewPacket</span><span>(</span><span>FVisibilityTaskData</span><span>&amp;</span><span> </span><span>InTaskData</span><span>, </span><span>FScene</span><span>&amp;</span><span> </span><span>InScene</span><span>, </span><span>FViewInfo</span><span>&amp;</span><span> </span><span>InView</span><span>, </span><span>int32</span><span> </span><span>InViewIndex</span><span>)</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>// ...</span></div></div><div><div><div>4</div></div><div><span> </span><span>if</span><span> (</span><span>TaskConfig</span><span>.</span><span>Schedule</span><span> </span><span>==</span><span> EVisibilityTaskSchedule::</span><span>Parallel</span><span>)</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>6</div></div><div><span><span> </span></span><span>// Chain the frustum cull task to the relevance task since we only wait on relevance.</span></div></div><div><div><div>7</div></div><div><span> </span><span>Tasks</span><span>.</span><span>ComputeRelevance</span><span>.</span><span>AddPrerequisites</span><span>(</span><span>Tasks</span><span>.</span><span>FrustumCull</span><span>);</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span><span> </span></span><span>// Callback for when an occlusion command is queued from frustum culling.</span></div></div><div><div><div>10</div></div><div><span> </span><span>OcclusionCull</span><span>.</span><span>CommandPipe</span><span>.</span><span>SetCommandFunction</span><span>([</span><span>this</span><span>](</span><span>FPrimitiveRange</span><span><span> </span><span>PrimitiveRange</span><span>)</span></span></div></div><div><div><div>11</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>12</div></div><div><span> </span><span>UpdatePrimitiveFading</span><span>(</span><span>Scene</span><span>, </span><span>View</span><span>, </span><span>ViewState</span><span>, </span><span>PrimitiveRange</span><span>);</span></div></div><div><div><div>13</div></div><div> </div></div><div><div><div>14</div></div><div><span> </span><span>const</span><span> </span><span>int32</span><span> </span><span>NumCulledPrimitives</span><span> </span><span>=</span><span> </span><span>PrecomputedOcclusionCull</span><span>(</span><span>*</span><span>this</span><span>, </span><span>PrimitiveRange</span><span>);</span></div></div><div><div><div>15</div></div><div> </div></div><div><div><div>16</div></div><div><span> </span><span>if</span><span> (</span><span>OcclusionCull</span><span>.</span><span>ContextIfParallel</span><span>)</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>18</div></div><div><span> </span><span>OcclusionCull</span><span>.</span><span>ContextIfParallel</span><span>-&gt;</span><span>AddPrimitives</span><span>(</span><span>PrimitiveRange</span><span>);</span></div></div><div><div><div>19</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>20</div></div><div><span><span> </span></span><span>// ..</span></div></div><div><div><div>21</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>22</div></div><div><span>}</span></div></div><div><div><div>23</div></div><div> </div></div><div><div><div>24</div></div><div><span>void</span><span> FGPUOcclusionParallel::</span><span>AddPrimitives</span><span>(</span><span>FPrimitiveRange</span><span> </span><span>PrimitiveRange</span><span>)</span></div></div><div><div><div>25</div></div><div><span>{</span></div></div><div><div><div>26</div></div><div><span> </span><span>WaitForLastOcclusionQuery</span><span>();</span></div></div><div><div><div>27</div></div><div> </div></div><div><div><div>28</div></div><div><span> </span><span>for</span><span> (</span><span>FSceneSetBitIterator</span><span> </span><span>BitIt</span><span>(</span><span>View</span><span>.</span><span>PrimitiveVisibilityMap</span><span>, </span><span>PrimitiveRange</span><span>.</span><span>StartIndex</span><span>); </span><span>BitIt</span><span>.</span><span>GetIndex</span><span>() </span><span>&lt;</span><span> </span><span>PrimitiveRange</span><span>.</span><span>EndIndex</span><span>; </span><span>++</span><span>BitIt</span><span>)</span></div></div><div><div><div>29</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>30</div></div><div><span> </span><span>FGPUOcclusionParallelPacket</span><span>*</span><span> </span><span>Packet</span><span> </span><span>=</span><span> </span><span>Packets</span><span>.</span><span>Last</span><span>();</span></div></div><div><div><div>31</div></div><div> </div></div><div><div><div>32</div></div><div><span> </span><span>if</span><span> (</span><span>Packet</span><span>-&gt;</span><span>AddPrimitive</span><span>(</span><span>BitIt</span><span>.</span><span>GetIndex</span><span>()))</span></div></div><div><div><div>33</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>34</div></div><div><span> </span><span>if</span><span> (</span><span>Packet</span><span>-&gt;</span><span>IsFull</span><span>())</span></div></div><div><div><div>35</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>36</div></div><div><span> </span><span>Packet</span><span>-&gt;</span><span>LaunchOcclusionCullTask</span><span>();</span></div></div><div><div><div>37</div></div><div><span> </span><span>CreateOcclusionPacket</span><span>();</span></div></div><div><div><div>38</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>39</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>40</div></div><div><span> </span><span>else</span></div></div><div><div><div>41</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>42</div></div><div><span><span> </span></span><span>// The primitive will not be occluded, so accumulate a packet of primitives to send directly to the relevance pipe to reduce latency.</span></div></div><div><div><div>43</div></div><div><span> </span><span>NonOccludedPrimitives</span><span>.</span><span>Emplace</span><span>(</span><span>BitIt</span><span>.</span><span>GetIndex</span><span>());</span></div></div><div><div><div>44</div></div><div> </div></div><div><div><div>45</div></div><div><span> </span><span>if</span><span> (</span><span>NonOccludedPrimitives</span><span>.</span><span>Num</span><span>() </span><span>==</span><span> </span><span>MaxNonOccludedPrimitives</span><span>)</span></div></div><div><div><div>46</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>47</div></div><div><span> </span><span>ViewPacket</span><span>.</span><span>Relevance</span><span>.</span><span>CommandPipe</span><span>.</span><span>AddNumCommands</span><span>(</span><span>1</span><span>);</span></div></div><div><div><div>48</div></div><div><span> </span><span>ViewPacket</span><span>.</span><span>Relevance</span><span>.</span><span>CommandPipe</span><span>.</span><span>EnqueueCommand</span><span>(</span><span>MoveTemp</span><span>(</span><span>NonOccludedPrimitives</span><span>));</span></div></div><div><div><div>49</div></div><div><span> </span><span>NonOccludedPrimitives</span><span>.</span><span>Reserve</span><span>(</span><span>MaxNonOccludedPrimitives</span><span>);</span></div></div><div><div><div>50</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>51</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>52</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>53</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>在 LaunchOcclusionCullTask 后，FGPUOcclusionPacket::OcclusionCullPrimitive 中实现了具体的行为：</p><ol> <li> <p>首先判断是否有对应的 Occlusion History，若没有，直接创建一个</p> </li> <li> <p>如果有 Occlusion History 获取相应的 FRHIRenderQuery，根据 depth 的采样数量判断是否有像素绘制，从而判断是否被遮挡</p> </li> <li> <p>判断是否允许包围盒测试</p> <p>判断依据按优先级依次为：</p> <ul> <li>和摄像机原点的距离是否大于等于 <code>r.NeverOcclusionTestDistance</code> 设置的距离</li> <li>是否在近裁切面之后</li> <li>正交投影的情况下，是否位于相机投影体范围内（默认正交投影体是 XOY 平面 沿 Z 轴方向）</li> <li>其余情况下，外接球的半径是否小于 HALF_WORLD_MAX，关于该值的定义详见 EngineDefines.h</li> </ul> <p>如果包围盒测试未通过，则认定为未被遮挡，不会进入后续的 Occlusion Query 流程。</p> </li> <li> <p>通过包围盒测试后，首先更新用于记录上一次通过包围盒测试的帧数计数器。然后根据 Occlusion 的类型对 Bounds 进行注册，Occlusion 的类型以及优先级为</p> <ul> <li>Occlusion Feedback</li> <li>HZB Occlusion</li> <li>常规 Occlusion</li> </ul> </li> <li> <p>针对需要做常规 Occlusion 的几何体，根据 PrimitiveOcclusionHistory 里的遮挡情况进一步进行划分为：</p> <p>Grouped Occlusion : 历史帧结果是被遮挡的，会被视作不可见，将它们的 AABB 合为一批进行 Query，进行粗略的查询</p> <p>Individual Occlusion : 没有被遮挡的，需要使用单独的遮挡做查询</p> <p>在一个 Vertex Buffer 中 Grouped Occlusion 最大的 Batch 数不超过 16，Individual Occlusion 只有 1：</p> <div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>/**</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>* Initialization constructor.</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>* </span><span>@param</span><span> </span><span>InView</span><span> - copy to init with</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>*/</span></div></div><div><div><div>5</div></div><div><span><span>FViewInfo</span><span>::</span></span><span>FViewInfo</span><span>(</span><span>const</span><span> </span><span>FSceneView</span><span>*</span><span> </span><span>InView</span><span>)</span></div></div><div><div><div>6</div></div><div><span><span> </span></span><span>: </span><span>FSceneView</span><span>(</span><span>*</span><span>InView</span><span>)</span></div></div><div><div><div>7</div></div><div><span><span> </span></span><span>, </span><span>IndividualOcclusionQueries</span><span>((</span><span>FSceneViewState</span><span>*</span><span>)</span><span>InView</span><span>-&gt;</span><span>State</span><span>,</span><span>1</span><span>)</span></div></div><div><div><div>8</div></div><div><span><span> </span></span><span>, </span><span>GroupedOcclusionQueries</span><span>((</span><span>FSceneViewState</span><span>*</span><span>)</span><span>InView</span><span>-&gt;</span><span>State</span><span>,FOcclusionQueryBatcher::</span><span>OccludedPrimitiveQueryBatchSize</span><span>)</span></div></div><div><div><div>9</div></div><div><span><span> </span></span><span>, </span><span>CustomVisibilityQuery</span><span>(</span><span>nullptr</span><span>)</span></div></div><div><div><div>10</div></div><div><span>{</span></div></div><div><div><div>11</div></div><div><span> </span><span>Init</span><span>();</span></div></div><div><div><div>12</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div> <div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>/** The maximum number of consecutive previously occluded primitives which will be combined into a single occlusion query. */</span></div></div><div><div><div>2</div></div><div><span>enum</span><span> { </span><span>OccludedPrimitiveQueryBatchSize</span><span> </span><span>=</span><span> </span><span>16</span><span> };</span></div></div></code></pre><div><div></div><div></div></div></figure></div> </li> <li> <p>划分完成后通过 BatchPrimitive 进行合批，同时将创建的 Query 注册到 PrimitiveOcclusionHistory中</p> </li> <li> <p>更新 PrimitiveOcclusionHistory</p> <p>刷新 PrimitiveOcclusionHistory 的<code>上次更新时间（LastConsideredTime）</code>，上次更新帧数以及上帧是否被遮挡等信息。其中<code>上次更新时间</code>是 FSceneViewState::TrimOcclusionHistory 清除旧 Queries 的重要依据。</p> <p>这一步主要是为了判断历史帧的结果是否有效，是否需要丢弃重新创建 Query。</p> </li> <li> <p>处理具有多个 Query 的几何体</p> <p>对于 HISM 这种有多个子 Bounds 的几何体，判断是否接受这一轮 Query 的结果。如果所有的子 Bounds 都被遮挡，直接标记它为不可见。</p> </li> </ol></section><section><h4>渲染时<a href="#渲染时"><span>#</span></a></h4><p>渲染时的入口可以在 DeferredShadingRenderer::RenderOcclusion。</p><ol> <li> <p>首先看是否 Depth Target 是否需要 down sample，然后对每个 View 去发起 Occlusion Query。</p> <p>调用 AllocateOcclusionTests 发起 Query 时，会先将失效的 Occlusion 结果通过 TrimOcclusionHistory 进行清除。</p> <div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>// Clear primitives which haven't been visible recently out of the occlusion history, and reset old pending occlusion queries.</span></div></div><div><div><div>2</div></div><div><span>ViewState</span><span>-&gt;</span><span>TrimOcclusionHistory</span><span>(</span><span>ViewFamily</span><span>.</span><span>Time</span><span>.</span><span>GetRealTimeSeconds</span><span>(), </span><span>ViewFamily</span><span>.</span><span>Time</span><span>.</span><span>GetRealTimeSeconds</span><span>() </span><span>-</span><span> </span><span>GEngine</span><span>-&gt;</span><span>PrimitiveProbablyVisibleTime</span><span>, </span><span>ViewFamily</span><span>.</span><span>Time</span><span>.</span><span>GetRealTimeSeconds</span><span>(), </span><span>ViewState</span><span>-&gt;</span><span>OcclusionFrameCounter</span><span>);</span></div></div></code></pre><div><div></div><div></div></div></figure></div> <p>清除的条件会参考引擎配置中的一个参数，叫做<code>潜在可见时间（PrimitiveProbablyVisibleTime）</code>，指的是明确在上一次可见后过多久可以认为该几何体依然可见，引擎默认是 8.0。</p> <p>如果在 <code>LastConsideredTime</code> 后过去了 <code>PrimitiveProbablyVisibleTime</code> 这么久，那么该几何体将从 Occlusion 的历史帧中被移除。</p> <p>需要注意的一点是，不光 BasePass，其他的渲染特性，如 Shadow，Light，Planar Reflection 等也要在这里创建 Query，也就是说会逐 View 的去创建 QueryArray。</p> </li> <li> <p>然后 VS 合批后在 BeginOcclusionTests 中执行绘制，在 FenceOcclusionTests 中执行同步。</p> <p>禁止写颜色和写深度：</p> <div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>FGraphicsPipelineStateInitializer</span><span> </span><span>GraphicsPSOInit</span><span>;</span></div></div><div><div><div>2</div></div><div><span>RHICmdList</span><span>.</span><span>ApplyCachedRenderTargets</span><span>(</span><span>GraphicsPSOInit</span><span>);</span></div></div><div><div><div>3</div></div><div><span>GraphicsPSOInit</span><span>.</span><span>PrimitiveType</span><span> </span><span>=</span><span> </span><span>PT_TriangleList</span><span>;</span></div></div><div><div><div>4</div></div><div><span>GraphicsPSOInit</span><span>.</span><span>BlendState</span><span> </span><span>=</span><span> TStaticBlendStateWriteMask&lt;</span><span>CW_NONE</span><span>, </span><span>CW_NONE</span><span>, </span><span>CW_NONE</span><span>, </span><span>CW_NONE</span><span>, </span><span>CW_NONE</span><span>, </span><span>CW_NONE</span><span>, </span><span>CW_NONE</span><span>, </span><span>CW_NONE</span><span>&gt;::</span><span>GetRHI</span><span>();</span></div></div><div><div><div>5</div></div><div><span>// Depth tests, no depth writes, no color writes, opaque</span></div></div><div><div><div>6</div></div><div><span>GraphicsPSOInit</span><span>.</span><span>DepthStencilState</span><span> </span><span>=</span><span> TStaticDepthStencilState&lt;</span><span>false</span><span>, </span><span>CF_DepthNearOrEqual</span><span>&gt;::</span><span>GetRHI</span><span>();</span></div></div><div><div><div>7</div></div><div><span>GraphicsPSOInit</span><span>.</span><span>BoundShaderState</span><span>.</span><span>VertexDeclarationRHI</span><span> </span><span>=</span><span> </span><span>GetVertexDeclarationFVector3</span><span>();</span></div></div></code></pre><div><div></div><div></div></div></figure></div> <p>整个 Query 过程只需要 AABB 与像素无关，可以不绑 PS 使用固定的 PSO：</p> <div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>// Lookup the vertex shader.</span></div></div><div><div><div>2</div></div><div><span>TShaderMapRef</span><span>&lt;</span><span>FOcclusionQueryVS</span><span>&gt; </span><span>VertexShader</span><span>(</span><span>View</span><span><span>.</span><span>ShaderMap</span><span>);</span></span></div></div><div><div><div>3</div></div><div><span>GraphicsPSOInit</span><span>.</span><span>BoundShaderState</span><span>.</span><span>VertexShaderRHI</span><span> </span><span>=</span><span> </span><span>VertexShader</span><span>.</span><span>GetVertexShader</span><span>();</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span>if</span><span> (</span><span>View</span><span>.</span><span>Family</span><span>-&gt;</span><span>EngineShowFlags</span><span>.</span><span>OcclusionMeshes</span><span>)</span></div></div><div><div><div>6</div></div><div><span>{</span></div></div><div><div><div>7</div></div><div><span> </span><span>TShaderMapRef</span><span>&lt;</span><span>FOcclusionQueryPS</span><span>&gt; </span><span>PixelShader</span><span>(</span><span>View</span><span>.</span><span>ShaderMap</span><span>);</span></div></div><div><div><div>8</div></div><div><span> </span><span>GraphicsPSOInit</span><span>.</span><span>BoundShaderState</span><span>.</span><span>PixelShaderRHI</span><span> </span><span>=</span><span> </span><span>PixelShader</span><span>.</span><span>GetPixelShader</span><span>();</span></div></div><div><div><div>9</div></div><div><span> </span><span>GraphicsPSOInit</span><span>.</span><span>BlendState</span><span> </span><span>=</span><span> TStaticBlendState&lt;</span><span>CW_RGBA</span><span>&gt;::</span><span>GetRHI</span><span>();</span></div></div><div><div><div>10</div></div><div><span>}</span></div></div><div><div><div>11</div></div><div> </div></div><div><div><div>12</div></div><div><span>SetGraphicsPipelineState</span><span>(</span><span>RHICmdList</span><span>, </span><span>GraphicsPSOInit</span><span>, </span><span>0</span><span>);</span></div></div></code></pre><div><div></div><div></div></div></figure></div> </li> </ol><p>几个关键的 Code path 是 RenderOcclusion -&gt; AllocateOcclusionTests -&gt; TrimOcclusionHistory -&gt; BeginOcclusionTests -&gt; FenceOcclusionTests。</p></section></section><section><h3>其他<a href="#其他"><span>#</span></a></h3><ul> <li> <p>使用 down sample 的 depth target 去绘制 Occlusion</p> <p>使用 <code>r.DownsampledOcclusionQueries</code> 来开启/关闭使用半分辨率的 Depth Target 来渲染包围盒。</p> <div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>const</span><span> </span><span>auto</span><span> </span><span>RenderOcclusionLambda</span><span> </span><span>=</span><span> [</span><span>&amp;</span><span>]()</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>RDG_GPU_STAT_SCOPE</span><span>(</span><span>GraphBuilder</span><span>, </span><span>RenderOcclusion</span><span>);</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span> </span><span>const</span><span> </span><span>int32</span><span> </span><span>AsyncComputeMode</span><span> </span><span>=</span><span> </span><span>CVarSceneDepthHZBAsyncCompute</span><span>.</span><span>GetValueOnRenderThread</span><span>();</span></div></div><div><div><div>6</div></div><div><span> </span><span>bool</span><span> </span><span>bAsyncCompute</span><span> </span><span>=</span><span> </span><span>AsyncComputeMode</span><span> </span><span>!=</span><span> </span><span>0</span><span>;</span></div></div><div><div><div>7</div></div><div> </div></div><div><div><div>8</div></div><div><span> </span><span>FBuildHZBAsyncComputeParams</span><span> </span><span>AsyncComputeParams</span><span> </span><span>=</span><span> {};</span></div></div><div><div><div>9</div></div><div><span> </span><span>if</span><span> (</span><span>AsyncComputeMode</span><span> </span><span>==</span><span> </span><span>2</span><span>)</span></div></div><div><div><div>10</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>11</div></div><div><span> </span><span>AsyncComputeParams</span><span>.</span><span>Prerequisite</span><span> </span><span>=</span><span> </span><span>ComputeLightGridOutput</span><span>.</span><span>CompactLinksPass</span><span>;</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>13</div></div><div> </div></div><div><div><div>14</div></div><div><span> </span><span>RenderOcclusion</span><span>(</span><span>GraphBuilder</span><span>, </span><span>SceneTextures</span><span>, </span><span>bIsOcclusionTesting</span><span>,</span></div></div><div><div><div>15</div></div><div><span> </span><span>bAsyncCompute</span><span> </span><span>?</span><span> </span><span>&amp;</span><span>AsyncComputeParams</span><span> </span><span>:</span><span> </span><span>nullptr</span><span>);</span></div></div><div><div><div>16</div></div><div> </div></div><div><div><div>17</div></div><div><span> </span><span>CompositionLighting</span><span>.</span><span>ProcessAfterOcclusion</span><span>(</span><span>GraphBuilder</span><span>);</span></div></div><div><div><div>18</div></div><div><span>};</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div> </li> <li> <p>缓存的 Occlusion 历史帧不超过 4 帧</p> <p>每个 FPrimitiveOcclusionHistory 都持有一个相对应的 FPrimitiveComponentId，并记录了其过去几帧（不大于 4）的 Query 结果。</p> <div><div><div><figure><figcaption><span>ScenePrivate.h</span><span>展开</span><span>收起</span></figcaption><pre><code><div><div><div>1</div></div><div><span>//</span></div></div><div><div><div>2</div></div><div><span>//</span></div></div><div><div><div>3</div></div><div><span>/** The occlusion query which contains the primitive's pending occlusion results. */</span></div></div><div><div><div>4</div></div><div><span>FRHIRenderQuery</span><span>*</span><span> </span><span>PendingOcclusionQuery</span><span>[FOcclusionQueryHelpers::</span><span>MaxBufferedOcclusionFrames</span><span>];</span></div></div><div><div><div>5</div></div><div><span>uint32</span><span> </span><span>PendingOcclusionQueryFrames</span><span>[FOcclusionQueryHelpers::</span><span>MaxBufferedOcclusionFrames</span><span>];</span></div></div><div><div><div>6</div></div><div> </div></div><div><div><div>7</div></div><div><span>//</span></div></div><div><div><div>8</div></div><div><span>// ScenePrivateBase.h</span></div></div><div><div><div>9</div></div><div><span>//</span></div></div><div><div><div>10</div></div><div><span>enum</span></div></div><div><div><div>11</div></div><div><span>{</span></div></div><div><div><div>12</div></div><div><span> </span><span>MaxBufferedOcclusionFrames</span><span> </span><span>=</span><span> </span><span>4</span></div></div><div><div><div>13</div></div><div><span>};</span></div></div><div><div><div>14</div></div><div> </div></div><div><div><div>15</div></div><div><span>// get the system-wide number of frames of buffered occlusion queries.</span></div></div><div><div><div>16</div></div><div><span>static</span><span> </span><span>int32</span><span> </span><span>GetNumBufferedFrames</span><span>(ERHIFeatureLevel::</span><span>Type</span><span><span> </span><span>FeatureLevel</span><span>);</span></span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div> </li> <li> <p>历史帧结果，固定每 6 帧清理一次</p> <div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>void</span><span><span> </span><span>FSceneViewState</span><span>::</span></span><span>TrimOcclusionHistory</span><span>(</span><span>float</span><span><span> </span><span>CurrentTime</span><span>, </span></span><span>float</span><span><span> </span><span>MinHistoryTime</span><span>, </span></span><span>float</span><span><span> </span><span>MinQueryTime</span><span>, </span></span><span>int32</span><span><span> </span><span>FrameNumber</span><span>)</span></span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>// Only trim every few frames, since stale entries won't cause problems</span></div></div><div><div><div>4</div></div><div><span> </span><span>if</span><span> (</span><span>FrameNumber</span><span> </span><span>%</span><span> </span><span>6</span><span> </span><span>==</span><span> </span><span>0</span><span>)</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>6</div></div><div><span> </span><span>int32</span><span> </span><span>NumBufferedFrames</span><span> </span><span>=</span><span> FOcclusionQueryHelpers::</span><span>GetNumBufferedFrames</span><span>(</span><span>GetFeatureLevel</span><span>());</span></div></div><div><div><div>7</div></div><div> </div></div><div><div><div>8</div></div><div><span> </span><span>for</span><span>(TSet&lt;</span><span>FPrimitiveOcclusionHistory</span><span>,</span><span>FPrimitiveOcclusionHistoryKeyFuncs</span><span>&gt;::</span><span>TIterator</span><span> </span><span>PrimitiveIt</span><span>(</span><span>Occlusion</span><span>.</span><span>PrimitiveOcclusionHistorySet</span><span>);</span></div></div><div><div><div>9</div></div><div><span> </span><span>PrimitiveIt</span><span>;</span></div></div><div><div><div>10</div></div><div><span> </span><span>++</span><span>PrimitiveIt</span></div></div><div><div><div>11</div></div><div><span><span> </span></span><span>)</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>13</div></div><div><span><span> </span></span><span>// If the primitive hasn't been considered for visibility recently, remove its history from the set.</span></div></div><div><div><div>14</div></div><div><span> </span><span>if</span><span> (</span><span>PrimitiveIt</span><span>-&gt;</span><span>LastConsideredTime</span><span> </span><span>&lt;</span><span> </span><span>MinHistoryTime</span><span> </span><span>||</span><span> </span><span>PrimitiveIt</span><span>-&gt;</span><span>LastConsideredTime</span><span> </span><span>&gt;</span><span> </span><span>CurrentTime</span><span>)</span></div></div><div><div><div>15</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>16</div></div><div><span> </span><span>PrimitiveIt</span><span>.</span><span>RemoveCurrent</span><span>();</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>18</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>19</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>20</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div> </li> <li> <p>SceneProxy 的 AcceptOcclusionResults 接口 这个虚函数默认没有实现，目前引擎中只有 HISM 和 VirtualHeightFieldMesh 实现了这个接口。</p> <p>AcceptOcclusionResults 能够提供当帧的 Occlusion 结果。对于 HISM 来说，它内部维护了一个 ViewId 和 FFoliageOcclusionResults 的 Map，如果 SubOcclusionQueries 与 HISM 的 FFoliageOcclusionResults 不匹配的时候，根据 AcceptOcclusionResults 的结果重建该 Map。可能因为存在这种 Clustering 概念的几何体集合，所以引擎的代码里才引入了 SubQueries 这个概念。</p> </li> <li> <p>Nanite mesh 走 GPU 剔除的 pass，如果使用 FreezeRendering 调试会发现 nanite mesh 不会被裁剪。此外有些渲染特性的开启也会导致相应的 primitive 不会被遮挡裁剪剔除掉，具体的可以查看相应的 SceneProxy 的 CanBeOccluded 函数。</p> </li> <li> <p>Mobility 为静态（static 以及 Stationary）和纯动态的组件，Occlusion Query 的处理不同，笼统地说，动态组件的查询会更精确，因为动态的 primitive 不会使用 grouped，将作为独立的 primitive 去查询。</p> </li> </ul></section></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://developer.nvidia.com/gpugems/gpugems/part-v-performance-and-practicalities/chapter-29-efficient-occlusion-culling" target="_blank">chapter-29-efficient-occlusion-culling</a></li> <li><a href="https://castle-engine.io/occlusion_culling?page=/occlusion_query" target="_blank">Castle Game Engine - Occlusion Culling</a></li> <li><a href="https://www.khronos.org/opengl/wiki/Query_Object#Occlusion_queries" target="_blank">Query Object</a></li> <li><a href="https://developer.nvidia.com/gpugems/gpugems2/part-i-geometric-complexity/chapter-6-hardware-occlusion-queries-made-useful" target="_blank">Chapter 6. Hardware Occlusion Queries Made Useful</a></li> <li><a href="https://zhuanlan.zhihu.com/p/66407205" target="_blank">剔除：从软件到硬件</a></li> <li><a href="https://registry.khronos.org/vulkan/specs/1.3-extensions/html/vkspec.html#queries-occlusion" target="_blank">18.3. Occlusion Queries</a></li> </ul></section> <section><h2>Footnotes<a href="#footnote-label"><span>#</span></a></h2> <ol> <li> <p><a href="https://docs.unrealengine.com/5.3/en-US/visibility-and-occlusion-culling-in-unreal-engine/" target="_blank">Visibility and Occlusion Culling</a> <a href="#user-content-fnref-offcialdocument">↩</a></p> </li> </ol> </section>https://yohiro.games/posts/world-building-guide/https://yohiro.games/posts/world-building-guide/Fri, 09 Jun 2023 00:00:00 GMT<section><h2><a href="https://docs.unrealengine.com/5.1/en-US/world-partition-in-unreal-engine/" target="_blank"><strong>World Partition</strong></a><a href="#world-partition"><span>#</span></a></h2><section><h3>实践<a href="#实践"><span>#</span></a></h3><section><h4>Grid setup<a href="#grid-setup"><span>#</span></a></h4><ul> <li>以单独的 Grid 开始构建，最终的成品中包含的 Grid 越少越好。</li> <li>应基于 Gameplay、物理行为、Actor 数量等来规划 Grid 的具体的尺寸和加载范围，并在规模变化时进行 profiling，再次调整。</li> <li>考虑 HLOD</li> </ul></section><section><h4>使用 Level Instance 和 Packed Level Actor<a href="#使用-level-instance-和-packed-level-actor"><span>#</span></a></h4></section><section><h4>使用 StreamingSourceComponent<a href="#使用-streamingsourcecomponent"><span>#</span></a></h4><p>在 5.1 以后的版本中，PlayerController 已经是一个 StreamingSourceComponent。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>class</span><span> ENGINE_API </span><span>APlayerController</span><span> : </span><span>public</span><span> </span><span>AController</span><span>,</span></div></div><div><div><div>2</div></div><div><span> </span><span>public</span><span> </span><span>IWorldPartitionStreamingSourceProvider</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>这个组件可以附加在任何 Actor 上，可以用于一些无缝 CutScene 的预加载。</p></section></section><section><h3>问题<a href="#问题"><span>#</span></a></h3><section><h4>创建许多新 Grid 以及运行时的 Data Layer<a href="#创建许多新-grid-以及运行时的-data-layer"><span>#</span></a></h4><ul> <li> <p>每个 Grid 的每一个小格子都至少包含一个 Actor，产生一个 StreamingLevel。此外，每个 Data Layer 也会创建一个 StreamingLevel。任何行为产生的 StreamingLevel 都会对性能有所影响，例如： - 根据资产类型创建多个 Grid 或 Data Layer，像是在一个森林的场景中创建树木的 Grid、FX 的 Grid、灯光的 Grid、大石头小石头的。 - 仅使用 Data Layer 用作编辑目的而不在运行时改变其状态。</p> </li> <li> <p>需要平衡 Grid/Data Layer 的规模和 Streaming 的效率 - 通常为一些特殊内容添加 Grid/Data Layer 的行为，不会造成太大的性能影响，可以放心使用。</p> </li> </ul></section><section><h4>场景/Data Layer 的组织<a href="#场景data-layer-的组织"><span>#</span></a></h4></section><section><h4>Actor 的包围盒<a href="#actor-的包围盒"><span>#</span></a></h4></section></section><section><h3>限制<a href="#限制"><span>#</span></a></h3><section><h4>Grid 隶属于 Persistent 关卡<a href="#grid-隶属于-persistent-关卡"><span>#</span></a></h4><p>目前仅支持一个 Persistent 关卡中包含多个 Grid。</p></section><section><h4>2D Streaming<a href="#2d-streaming"><span>#</span></a></h4><p>当前版本的 Streaming 是 2D 视角的，不支持垂直空间的 Streaming。根据 World Partition 的 <a href="https://udn.unrealengine.com/s/article/World-Building-Features-Roadmap" target="_blank">Roadmap</a>，将在 5.4plus 版本中添加 3D 区块的支持。</p></section><section><h4>重新配置区块 vs 分割 Actor<a href="#重新配置区块-vs-分割-actor"><span>#</span></a></h4><p>分割 Actor，例如 Landscape、Foliage 时，不能超过区块大小的限制，否则导致该 Actor（Landscape/Foliage）在每次 Streaming grid 重新配置后无法更新。当前版本的引擎内提供了命令行重新创建这类 Actor，需要时可以调用。</p></section><section><h4>编辑器内需要手动加载<a href="#编辑器内需要手动加载"><span>#</span></a></h4><p>当前版本中并不支持根据相机的位置、朝向自动加载区块，需要设计者在 World Partition 编辑器内选择加载。相较于 Nanite 和 Texture 的 Streaming，World Partition 在编辑器内的加载不受 Pool/Buffer 数量的限制，因此会导致内存耗尽从而崩溃。</p></section><section><h4>运行时限制<a href="#运行时限制"><span>#</span></a></h4><ul> <li>运行时无法创建/修改 grid，生成 streaming 的行为法中在 PIE 和 Cook 时。</li> <li>不支持运行时关卡的创建/注入行为 - 修改 Game Feature 插件可以往指定的关卡添加 Actor - 修改 Level Instance 的代码也是可行的</li> <li>Spawn 的 Actor 是位于 Persistent 关卡中的，子 Actor 的关卡位置取决于其父节点的加载关卡</li> <li>Persistent 关卡中的 Actor 引用其他 Spatially Loaded 的 Actor，引擎禁止该行为</li> </ul></section><section><h4>GC<a href="#gc"><span>#</span></a></h4><p>为了分散 GC 的压力，已卸载的内容会在项目配置中定义的固定时间片后进行 GC，或是当指定数量的 cell 被卸载后进行 GC。</p></section><section><h4>One File Per Actor（OFPA）<a href="#one-file-per-actorofpa"><span>#</span></a></h4><p>WP 与 OFPA 密不可分，因此需要考虑产生的额外的 actor 文件数量所带来的问题，还有 source control 所面临的问题。</p></section><section><h4>Hot reload<a href="#hot-reload"><span>#</span></a></h4><p>当 Sync/Revert 一个资产时，引用到该资产的 Actor，如果位于 WP 中，hot reload 不会对这个 Actor 起效。WP 关卡必须关闭然后重新加载 Actor。</p></section><section><h4>服务器端的 Streaming<a href="#服务器端的-streaming"><span>#</span></a></h4><p>当前版本不支持，根据 <a href="https://udn.unrealengine.com/s/article/World-Building-Features-Roadmap" target="_blank"><strong>Roadmap</strong></a> 显示，将在 5.3 版本中添加该功能（Experimental），5.4plus 版本中可供产品使用。</p></section><section><h4>光照<a href="#光照"><span>#</span></a></h4><p>仅支持动态光照，不允许烘培静态光。</p></section></section><section><h3>用例<a href="#用例"><span>#</span></a></h3><section><h4>堡垒之夜 第四章<a href="#堡垒之夜-第四章"><span>#</span></a></h4> <table><thead><tr><th>项目</th><th>数值</th></tr></thead><tbody><tr><td>地图大小</td><td><code>2km x 2km</code></td></tr><tr><td>Actor 数量</td><td><code>约 100k</code></td></tr><tr><td>Cell 大小</td><td><code>128m</code></td></tr><tr><td>加载范围</td><td><code>256m（加载距离随平台和性能等级改变）</code></td></tr></tbody></table><ul> <li> <p>2 HLOD setup - 建筑物的 HLOD（两层） - HLOD 0 为支持破坏功能，HLOD0 的 mesh 被合并。256m cell，加载范围 512m，spatially loaded - HLOD 1 简化的 mesh，512m cell，2048m load range，spatially loaded - 树木的 HLOD（一层） - 使用 imposter 的 Instanced layer , always loaded</p> </li> <li> <p>Level Instances: - 所有的 POI</p> </li> <li> <p>Data layer： - 4 个 初生岛屿和团队战大厅 - 1 个 用于赛季变换/特殊事件</p> </li> <li> <p>服务器端： - 全部加载 - Server streaming 在 PIE 下使用，用于提升开发者的迭代效率</p> </li> <li> <p>Landscape: - Always loaded</p> </li> <li> <p>Packaed Level Actors: - 无</p> </li> </ul></section><section><h4>黑客帝国 demo<a href="#黑客帝国-demo"><span>#</span></a></h4> <table><thead><tr><th>项目</th><th>数值</th></tr></thead><tbody><tr><td>地图大小</td><td><code>4km x 4km</code></td></tr><tr><td>Actor 数量</td><td><code>约 107k</code></td></tr><tr><td>Cell 大小</td><td><code>128m</code></td></tr><tr><td>加载范围</td><td><code>128m</code></td></tr></tbody></table><ul> <li>HLOD setup - HLOD 0 启用 Nanite 的 Instance layer，256m cell，加载范围 768m，spatially loaded - HLOD 1 简化的 mesh，256m cell，always loaded</li> <li>使用 houdini 过程化构建，同时使用 Rule Processor 插件 - 所有的 POI</li> <li>Packaed Level Actors: - PLA 和使用了 Instanced static mesh 过程化生成的 Actor - 该做法应用于所有的建筑物、街道、小物件</li> <li>Data layer（运行时 35/ 编辑器 21）： - 用于 cut scenee 和 gameplay - 用于屋顶和路面下的资产</li> <li>Landscape: - 无</li> </ul></section><section><h4>古代山谷 demo<a href="#古代山谷-demo"><span>#</span></a></h4> <table><thead><tr><th>项目</th><th>数值</th></tr></thead><tbody><tr><td>地图大小</td><td><code>2km x 2km</code></td></tr><tr><td>Actor 数量</td><td><code>约 14k</code></td></tr><tr><td>Cell 大小</td><td><code>64m</code></td></tr><tr><td>加载范围</td><td><code>64m</code></td></tr></tbody></table><ul> <li>HLOD setup - HLOD Instance layer，always loaded</li> <li>Packaed Level Actors: - 用于组合岩石</li> <li>Data layer（运行时 2/ 编辑器 1）</li> <li>Landscape: - 无</li> </ul></section></section><section><h3>调试<a href="#调试"><span>#</span></a></h3><ul> <li> <p>常用 CVars</p> <p>更多的可以执行 DumpConsoleCommands，将所有命令 dump 出来，wp 相关的 debug 指令太多，下面仅列出一些常用的。</p> </li> </ul> <table><thead><tr><th>命令</th><th>说明</th></tr></thead><tbody><tr><td>wp.Runtime.ToggleDrawRuntimeHash2D or 3D</td><td>streaming grid 可视化</td></tr><tr><td>wp.Runtime.OverrideRuntimeSpatialHashLoadingRange -grid=[index] -range=[DesiredValue]</td><td>覆写指定 grid 的加载距离</td></tr><tr><td>wp.runtime.hlod</td><td>显示 HLOD</td></tr><tr><td>wp.Runtime.RuntimeSpatialHashUseAlignedGridLevels</td><td></td></tr><tr><td>wp.Runtime.RuntimeSpatialHashSnapNonAlignedGridLevelsToLowerLevels</td><td></td></tr><tr><td>wp.Runtime.RuntimeSpatialHashPlaceSmallActorsUsingLocation</td><td></td></tr></tbody></table></section></section> <section><h2><a href="https://docs.unrealengine.com/5.1/en-US/one-file-per-actor-in-unreal-engine/" target="_blank"><strong>One File Per Actor</strong></a><a href="#one-file-per-actor"><span>#</span></a></h2><p>One File Per Actor(OFPA)</p><section><h3>实践<a href="#实践-1"><span>#</span></a></h3><section><h4>Epic 强建议 WP 的使用者使用引擎内部的版本管理<a href="#epic-强建议-wp-的使用者使用引擎内部的版本管理"><span>#</span></a></h4><p>原因如下：</p><ol> <li>WP 所引用的外部 Actor 文件以 GUID 命名，因此在外部文件夹中或是版本控制软件中难以识别。而在引擎的版本控制视图下，可以看到 Actor 的名称、类型以及路径，允许使用者对这些文件进行过滤、排序。</li> <li>Uncontrolled CL 可以追踪所有本地非只读的文件，这一功能在外部的版本控制（如 Perforce）中不可用。</li> <li>在提交时存在数据验证阶段，而且这里的验证可以根据项目需要进行扩展。</li> <li>在引擎内通过 CL 导航至 Actor 或其他内容，比起切换到外部的版本控制软件，体验上更加丝滑。</li> </ol></section><section><h4>使用 Uncontrolled CL（ChangeList）<a href="#使用-uncontrolled-clchangelist"><span>#</span></a></h4><ul> <li>使用 Uncontrolled CL 可以在不打开版本控制软件的情况下看到你编辑的、设为可写的或是添加的资产。</li> <li>能使 workspace 的使用情况更加清晰，避免出现不受版本控制的文件的影响。</li> <li>当修改内容准备完毕，将 Uncontrolled CL 移到版本控制的 CL。</li> <li>程序同学应该使用 Uncontrolled CL 进行调试、测试以避免版本控制对 binary 的锁定（比如 check out）。</li> </ul></section><section><h4>创建自定义的提交验证<a href="#创建自定义的提交验证"><span>#</span></a></h4><p>提交验证可以通过代码进行扩展，关键的验证如 CL 中的文件缺失、资产引用问题，这些已集成到引擎中。</p></section></section><section><h3>问题<a href="#问题-1"><span>#</span></a></h3><section><h4>需要使用者去管理个人文件<a href="#需要使用者去管理个人文件"><span>#</span></a></h4><p>大多数使用 OFPA 的人并不想管理一个又大又长的 Changelist，并且担忧与同事并行工作会带来问题。</p></section><section><h4>使用 P4V 管理外部 Actor<a href="#使用-p4v-管理外部-actor"><span>#</span></a></h4><p>如上文实践中所述，使用 WP 的过程中应该尽量避免使用 P4 提交。</p></section><section><h4>分支/合并/创建副本/文件数量<a href="#分支合并创建副本文件数量"><span>#</span></a></h4><ul> <li>OFPA 会给服务器端的加载、同步以及创建分支添加很大的压力，需要 DM 多加注意。</li> <li>对大型工作室，应考虑代理、edge server 等方式分散负载。</li> </ul></section></section><section><h3>限制<a href="#限制-1"><span>#</span></a></h3><section><h4>GUID 只能在 UE 里转为可读的名称<a href="#guid-只能在-ue-里转为可读的名称"><span>#</span></a></h4></section><section><h4>启用 OFPA vs 不启用 OFPA<a href="#启用-ofpa-vs-不启用-ofpa"><span>#</span></a></h4><p>对于启用和不启用 OFPA 的 Level Instances，他们的 Streaming 行为有所不同：</p><p>启用 OFPA： : 在 PIE/Cook 生成流送数据时，Level Instance 中的内容会被分散开放到 Persistent 关卡，并被指定到相应的 Grid 中。而 Level Instance 本身在运行时不存在。</p><p>不启用 OFPA : 在 PIE/Cook 生成流送数据时，Level Instance 会被保留，而且会被当作一个 Actor 在包含该 Level Instance 的 grid cell level 被加载的同时 Stream in。</p></section><section><h4>访问未加载 Actor<a href="#访问未加载-actor"><span>#</span></a></h4><p>当一些工具需要获取 Actor 的 GUID 时，如果所在 World 没有加载，GUID 信息/Actor 的描述信息是取不到的（以前不也是这样吗？）</p></section><section><h4>锁定<a href="#锁定"><span>#</span></a></h4><p>这里的锁定，指的是版本控制中的锁定，是指阻止用户编辑一个没有权限的文件（一般是被其他人 CheckOut 的文件）的行为。当前没有实现 Actor 锁定。</p></section><section><h4>观察文件状态<a href="#观察文件状态"><span>#</span></a></h4><p>相较于 5.0 版本，5.1 在获取文件状态方面做出了一些提升。用户确认是否可以对资产做出修改的一种方式是：在场景大纲下启用版本控制列，观察是否存在未保存的文件与版本控制内的文件有冲突。</p></section></section></section> <section><h2><a href="https://docs.unrealengine.com/5.1/en-US/level-instancing-in-unreal-engine/" target="_blank"><strong>Level Instances 和 Packed Level Actors(PLA)</strong></a><a href="#level-instances-和-packed-level-actorspla"><span>#</span></a></h2><p>Level Instance 和 Packed Level Actor 是 5.x 版本中实现 Level Instance 的两种方式，前者是 Level 级别的粒度，后者是 Actor 级别的粒度。</p><section><h3>Level Instance<a href="#level-instance"><span>#</span></a></h3><p>创建子关卡的 Actor 的集合，与 4.x 版本中基于关卡的 workflow 类似。当使用 World Partition 时，Level instance 应该启用 OFPA，因为启用 OFPA 后，引擎会在生成 Streaming 数据时将 Level instance 中的内容分离到 Persistent 关卡中的 Streaming grid。如果 Level instance 不启用 OFPA，那么它会被当作一个独立的 Streaming level，在规模较大较复杂的场景中会导致性能和 Streaming 的问题。</p><section><h4>特性<a href="#特性"><span>#</span></a></h4><ul> <li>具有可嵌套的层级结构</li> <li>同一 World 中现支持多个 Level instance</li> <li>对非 OFPA 关卡支持 Level streaming</li> <li>对关卡整体设置的编辑</li> <li>Level instance 内的所有 Actor 都支持 Data layer</li> </ul></section><section><h4>适用场景<a href="#适用场景"><span>#</span></a></h4><p>POI、房屋、内部陈设、建筑物的地板、村庄、Gameplay 相关配置等。</p></section></section><section><h3>Packed Level Actor<a href="#packed-level-actor"><span>#</span></a></h3><p>一种由<code>Static mesh</code>（包括 ISM/HISM）合并而来的<code>Blueprint actor</code>，Static mesh 会被替换为链接到<code>Packed level actor</code>的<code>Packed level blueprint</code>。简单来说 PLA 是由许多 ISM/HISM 组件构成的 Actor。PLA 不可覆盖不可脚本话，它会在每次更新时重新创建，所以应应用于静态物件。</p><section><h4>特性<a href="#特性-1"><span>#</span></a></h4></section><section><h4>适用场景<a href="#适用场景-1"><span>#</span></a></h4><p>静态建筑物、模型复用率高的拼合密度较高的大型物件等。</p></section></section><section><h3>问题<a href="#问题-2"><span>#</span></a></h3><section><h4>创建具有 level instance 的 world scale layer<a href="#创建具有-level-instance-的-world-scale-layer"><span>#</span></a></h4><p>由于编辑器中加载 level instance 的过程是阻塞、非异步的，使用非常大尺寸的 level instance 的去嵌套其他所有的 POI 往往会造成加载问题。必要时在所需的位置放置较小的 level instance，使用 Editor only 的 Data layer 是个比较好的选择。</p></section><section><h4>使用非 OFPA 的 level instance<a href="#使用非-ofpa-的-level-instance"><span>#</span></a></h4><p>World 中每个没有启用 OFPA 的 level instance 都会创建一个独立的 streaming level，这会导致 streaming 问题和性能问题。</p></section><section><h4>PLA 过大<a href="#pla-过大"><span>#</span></a></h4><p>当 PLA 的大小超过了 streaming cell 的大小，会引发一些性能问题、内存问题，正常情况下 PLA 的大小不应超过 streaming cell 的大小</p></section></section><section><h3>限制<a href="#限制-2"><span>#</span></a></h3><section><h4>无法覆盖<a href="#无法覆盖"><span>#</span></a></h4><ul> <li>对 level instance，所有编辑的结果会直接应用到关卡的源文件上。</li> <li>对 Packed level actor，每次更新 PLA 时，PLA 都会重新构建，所以当覆盖一个属性时会触发该机制从而无法覆盖。</li> </ul></section><section><h4>不支持关卡蓝图<a href="#不支持关卡蓝图"><span>#</span></a></h4><p>由于 level instance 在 streaming 期间会被分离为 persistent 关卡中的各个资产，所以无法执行单一关卡的脚本。</p></section><section><h4>level instance 中 actor 的引用<a href="#level-instance-中-actor-的引用"><span>#</span></a></h4><p>无法从高级别的 actor 引用 level instance 内的子 actor</p></section><section><h4>编辑器内无异步加载/部分加载<a href="#编辑器内无异步加载部分加载"><span>#</span></a></h4><ul> <li>在编辑器中加载 level instance 会加载其所有的内容，整个过程是非异步的。</li> <li>PLA 会被当作单独的 Actor 加载。</li> </ul></section><section><h4>非编辑模式下无法获取子 Actor<a href="#非编辑模式下无法获取子-actor"><span>#</span></a></h4><ul> <li>目前的版本中，level instance 的子 Actor 无法被选中、编辑、拷贝、查看属性等</li> <li>这会给一些操作带来不便，比如在主 world 编辑地形时，尝试隐藏部分 level instance 的内容，或是复制其他 level instance 粘贴到当前编辑的关卡中。</li> </ul></section></section><section><h3>用例<a href="#用例-1"><span>#</span></a></h3><section><h4>堡垒之夜 第四章<a href="#堡垒之夜-第四章-1"><span>#</span></a></h4><ul> <li>场景中所有的主要位置，如建筑物、房屋、POI 这种粒度的关卡，都由 level instance 构建。</li> <li>大多数情况下 level instance 被当作独立的关卡。</li> <li>没有使用 PLA，因为堡垒之夜的 gameplay 决定了，建筑物可以破坏、交互，因此没有使用 PLA。</li> </ul></section><section><h4>黑客帝国 demo<a href="#黑客帝国-demo-1"><span>#</span></a></h4><ul> <li>PLA 用于除程序化生成的建筑物外其他的主要建筑物。</li> <li>PLA 还用于创建预制的屋顶</li> <li>所有过程化生成的建筑物、道路、屋顶、部分物件都由 ISM 构成（等同于 PLA·</li> <li>只有最基础的关卡保有逐模块的碰撞。</li> <li>产品的最后阶段，PCG 流程停止了，PLA 最终也被转换为了 ISM actor。这么做都是为了能够手动修改物件，但该过程是不可逆的。</li> </ul></section><section><h4>古代山谷<a href="#古代山谷"><span>#</span></a></h4><ul> <li>PLA 扩展为可支持碰撞合并，以优化 streaming level 时在物理模块和 AddToWorld 时的消耗。</li> </ul></section></section></section> <section><h2><a href="https://docs.unrealengine.com/5.1/en-US/world-partition---data-layers-in-unreal-engine/" target="_blank"><strong>Data Layers</strong></a><a href="#data-layers"><span>#</span></a></h2><p>Data layer 允许在运行时/编辑时限定数据加载的条件。Actor 和 World Partition 决定了 streaming 的逻辑，Data Layer 则像是一个过滤器，用于决定哪些关卡需要加载。</p><section><h3>运行时 Data layer<a href="#运行时-data-layer"><span>#</span></a></h3><ul> <li>用于处理不同情景</li> <li>管理任务、游戏进度、事件等各种特定的数据</li> <li>HLOD 支持，创建的 HLOD 的状态会和 Data layer 的状态一同变化同时也是编辑器的 Data layer</li> <li>运行时具有三种状态： - Unloaded（unloaded and not visible） - Loaded（loaded and not visible） - Activated（loaded and visible）</li> </ul></section><section><h3>编辑器 Data layer<a href="#编辑器-data-layer"><span>#</span></a></h3><ul> <li>用于在编辑时组织内容</li> <li>为了更方便的编辑，数据相对独立</li> <li>预览运行时 Data layer 的内容</li> <li>存在仅编辑器可见的 Data layer，在 Cook 版本和 PIE 中不可见</li> <li>编辑器 Data layer 的状态： - IsInitiallyVisible（加载 world 时，是否默认可见） - IsInitiallyLoaded（加载 world 时，是否默认加载） - Loaded - Visible</li> </ul></section><section><h3>实践<a href="#实践-2"><span>#</span></a></h3><section><h4>使用仅编辑器可见的 Data layer 来分离数据<a href="#使用仅编辑器可见的-data-layer-来分离数据"><span>#</span></a></h4><p>使用编辑器 Data layer 可以将指定的数据如 gameplay sequence 和 cinematic 数据和其他的数据分离开。</p></section><section><h4>预加载<a href="#预加载"><span>#</span></a></h4></section><section><h4>Data layer 的负责人<a href="#data-layer-的负责人"><span>#</span></a></h4><p>对于项目的 Data layer，最好有技术人员制定它的结构，有条件的话，可以预先定义 data layer 的资产以匹配项目的结构和目标，比如任务/事件/游戏进程/工作类型等。</p></section><section><h4>优化<a href="#优化"><span>#</span></a></h4><p>通过 Data layer，可以在特定的 Sequence 或 gameplay 中移除不需要的关卡，也可以针对不同的平台进行优化，从而节省内存，提升性能。</p></section></section><section><h3>问题<a href="#问题-3"><span>#</span></a></h3><section><h4>Data layer 的规模<a href="#data-layer-的规模"><span>#</span></a></h4><ul> <li>运行时 Data layer 的创建应该被重点照顾，因为如 WP 一节所述，每一个 cell 中的 actor 对应的运行时 Data layer 都会创建一个新的 Streaming level。过多使用运行时 data layer 会严重影响 streaming 的效率。</li> <li>不过使用运行时 Data layer 来处理特定的内容，比如每个任务或是某个主题的场景，是比较安全的行为，对全局的 streaming 效率没有太大影响。</li> </ul></section><section><h4>Actor 上多个 Data layer 的组合<a href="#actor-上多个-data-layer-的组合"><span>#</span></a></h4></section><section><h4>Data layer 和 Streaming 的混淆<a href="#data-layer-和-streaming-的混淆"><span>#</span></a></h4></section><section><h4>使用 Data layer 加载太多东西<a href="#使用-data-layer-加载太多东西"><span>#</span></a></h4></section></section><section><h3>限制<a href="#限制-3"><span>#</span></a></h3><section><h4></h4></section><section><h4><a href="#-1"><span>#</span></a></h4></section></section><section><h3><a href="#-2"><span>#</span></a></h3></section><section><h3>用例<a href="#用例-2"><span>#</span></a></h3><section><h4>堡垒之夜 第四章<a href="#堡垒之夜-第四章-2"><span>#</span></a></h4><ul> <li>4 个运行时 Data layer，用于大厅/初始小岛</li> <li>1 个运行时 Data layer，用于赛季变动以及特殊事件</li> </ul></section><section><h4>黑客帝国 demo<a href="#黑客帝国-demo-2"><span>#</span></a></h4><ul> <li>35 个运行时 Data layer</li> <li>32 个编辑器 Data layer - Sequence 以及 Gameplay 内容的 streaming 和卸载 - Sequence 特定的优化 - 多个仅编辑器的 Data layer 用于 PCG 生成</li> </ul></section><section><h4>古代山谷 demo<a href="#古代山谷-demo-1"><span>#</span></a></h4><p>2 个运行时 Data layer，1 个 Editor only</p></section></section></section> <section><h2><a href="https://docs.unrealengine.com/5.1/en-US/world-partition---hierarchical-level-of-detail-in-unreal-engine/" target="_blank"><strong>HLOD</strong></a><a href="#hlod"><span>#</span></a></h2><p>WorldPartition HLOD 与原先 UE4.x 中的 HLOD 不同在于，WorldPartition HLOD 不是和单独的关卡相关联的，而是由 Grid 生成的。初此之外与原先的 HLOD 概念基本类似，都是 Actor 聚合所生成的低模代理。</p><section><h3>实践<a href="#实践-3"><span>#</span></a></h3><section><h4>预先定义 HLOD 的类型和层级<a href="#预先定义-hlod-的类型和层级"><span>#</span></a></h4><p>在概念阶段定义拥有最好的视觉效果的 HLOD 的 streaming、层级和类型，可以从运行平台、运行平台的内存和性能、Instancing、Nanite、场景密度、场景尺寸、WP 的 Grid 大小、资产类型等方面出发考虑 HLOD 的优化。</p></section><section><h4>在合适的距离评估 HLOD<a href="#在合适的距离评估-hlod"><span>#</span></a></h4><p>如果使用 Merged 或 Simplified 类型，近距离的 HLOD 会显得非常潦草，因为 HLOD 被设计为用作远距离时的低模代理。启用 Nanite 的 Instance 层会获得不错的效果。</p></section></section></section>https://yohiro.games/posts/nanite/https://yohiro.games/posts/nanite/Thu, 16 Mar 2023 00:00:00 GMT<section><h2>概述<a href="#概述"><span>#</span></a></h2><p>本文是两篇 Nanite 技术分享，<a href="https://advances.realtimerendering.com/s2021/Karis_Nanite_SIGGRAPH_Advances_2021_final.pdf" target="_blank">Nanite - A Deep Dive</a> 以及 <a href="https://highperformancegraphics.org/slides22/Journey_to_Nanite.pdf" target="_blank">Journey to Nanite</a> 的记录。</p><hr /></section> <section><h2>方案选择<a href="#方案选择"><span>#</span></a></h2><p>受 virtual texture 启发，Epic 想要一套 virtual geometry 技术用来减少面数、DrawCall 和内存对美术团队的限制。为此，他们尝试了各种方案。</p><section><h3>Voxel<a href="#voxel"><span>#</span></a></h3><p>首先使用体素化的方法来描述 mesh，但是对于一个 2M 的 mesh，使用体素方法重新采样到 narrow band SDF 后达到了 13M，而这甚至是去除了空白区域的稀疏存储。</p><p>这是因为体素化的采样是一种均匀的重采样方式，而均匀的重采样意味着细节的丢失，这对于硬表面模型来说很不友好。因为这种采样方式不是自适应的，在平滑区域和边缘区域采样方式是相同的。</p><p>而且即使使用高分辨率的存储，重建的模型的效果也不好，再加上这种做法可能会影响整个工作流，因此该放弃了方案。</p></section><section><h3>Subdivision Surface<a href="#subdivision-surface"><span>#</span></a></h3><p>细分表面看起来可以无限细分，非常适合近距离展示模型，但需要美术来创建一个 base cage 模型，来构建基本的形状信息。然而 base cage 的精度依然比游戏中使用的模型精度要高，这使得美术的工作变复杂了，因此放弃该方案。</p></section><section><h3>Displacement Map<a href="#displacement-map"><span>#</span></a></h3><p>然后又想到了使用顶点置换的方式，通过 displacement map，低模的顶点数可以很少，而且烘焙的方式也很成熟。但是 Displacement Map 对于多层的链条状模型存在限制，而且该方法和 Subdivision Surface 一样，只能增加细节，对于模型退化时存在一个下限。</p></section><section><h3>Point Cloud<a href="#point-cloud"><span>#</span></a></h3><p>点云的做法可能会引入巨量的 overdraw，缺少连续性信息时，很难确定相邻的点之间的洞是否需要填充。或许机器学习的相关算法比较成熟后，可以获得不错的应用。</p></section><section><h3>Triangles<a href="#triangles"><span>#</span></a></h3><p>最终发现还是三角形比较靠谱，它通用、高效。</p><hr /></section></section> <section><h2>GPU 管线<a href="#gpu-管线"><span>#</span></a></h2><p>在 UE 4.22 版本中，渲染管线经过了一次重构，具体可以参考<a href="https://dev.epicgames.com/documentation/en-us/unreal-engine/mesh-drawing-pipeline?application_version=4.27" target="_blank">官方文档</a>，以及<a href="https://www.youtube.com/watch?v=qx1c190aGhs" target="_blank">该 Talk</a>。在这次重构后，渲染器变为了 <strong>Retain 模式</strong>，场景信息可以被存储到显存中，并在不同的帧间保持不变，当场景发生变化时，会触发相应数据的增量更新。</p><p>此外由于 Nanite 的资源会存储于一个单一的资源结构中，所以不需要 bindless 便可以访问其中的任一资源。</p><p>以上构成了 GPU 管线的必要条件，在一次 dispatch 中就可以获得整个视图的可见性信息，一次 indirect draw 就可以对深度中所有的三角形进行光栅化。</p><section><h3>剔除<a href="#剔除"><span>#</span></a></h3><p>Nanite 的剔除是基于 Cluster 来进行的。一个 Cluster 就是一组三角形，最多为 128 个，Nanite 会为每个 cluster 构建一个 bounds，裁剪时会根据该 bounds 进行裁剪。</p><p>剔除的方式有视锥剔除和遮挡剔除。</p><p>Nanite 的遮挡剔除借由 <a href="https://blog.selfshadow.com/publications/practical-visibility/" target="_blank">HZB</a> 来完成。根据 cluster 的 bounds 来计算得到一个屏幕控件的矩形，然后使用最低的 mip 来判断是否遮挡。</p><p>使用 HZB 的剔除，需要先构建 HZB，通常游戏中会使用上一帧的深度信息，但这种做法无法保证剔除的精度，不过上一帧可见的，当前帧大概率可见。因此，Nanite 引入了一种双 Pass 的遮挡剔除，具体做法是：</p><ul> <li>绘制上一帧的可见内容</li> <li>根据上一帧的可见性构建 HZB</li> <li>对当前帧使用上一步构建的 HZB 进行剔除</li> <li>使用上一步剔除的结果构建一个新的 HZB</li> </ul></section><section><h3>可见性与材质解耦<a href="#可见性与材质解耦"><span>#</span></a></h3><p>Nanite 实现了像素的可见性与材质的解耦，这么做是因为开发团队想要消除在光栅化过程中 Shader 的切换、材质的 evaluate 造成的 overdraw 以及低效的 quad 绘制。</p><p>有一些 object space 的方法可以实现这一点，比如使用 <a href="https://zh.wikipedia.org/wiki/Reyes%E6%B8%B2%E6%9F%93%E6%9E%B6%E6%9E%84" target="_blank">REYES 渲染架构</a>或使用 Texture Space。</p><p>但是 object space 的方法可能会造成很多 overshade，所以回归到 Deferred Material 的方法，为此引入了 Visibility Buffer。</p></section><section><h3>Visibility Buffer<a href="#visibility-buffer"><span>#</span></a></h3><p>首先将深度信息，Instance ID，Triangle ID 等数据写入 Visibility Buffer。</p><p>接下来在渲染材质时，会先加载 visibility buffer 然后加载三角形数据。随后将当前像素的三角形变换到屏幕空间，并推导出像素的重心坐标，利用它来加载、插值。</p><p>在延迟渲染的路径下，材质的评估只是将数据写入 GBuffer 然后再进行着色。在这种方式下，所有不透明物体可以通过一个 DrawCall 完成绘制，CPU 端的消耗不再受限于物体的数量，材质的绘制与 Shader 相关而非物体数量。</p><p>到此为止，渲染速度有所提升，但消耗依然与实例与面片数目正相关。</p><p>最优情况是可见的面片的数量与屏幕像素数量相近，但在实践中很难实现这一点：渲染几何体的消耗与屏幕分辨率正相关，而非场景的复杂度。</p><p>Nanite 使用 Cluster 来组织场景，就是希望每帧渲染的 cluster 数目基本保持稳定，从而<strong>实现渲染消耗与场景复杂度的解耦</strong>。</p><hr /></section></section> <section><h2>LOD<a href="#lod"><span>#</span></a></h2><p>Nanite 的 LOD 是基于 cluster 的划分，父节点的 cluster 是子节点 cluster 的简化版本。</p><p>在运行时，对 cluster 来做切割来获得相应粒度的 cluster，切割的依据是 cluster 在屏幕空间投影的误差。父节点的绘制结果与当前节点绘制结果差异不大时，才考虑切换为下一级 LOD。</p><section><h4>LOD cracks<a href="#lod-cracks"><span>#</span></a></h4><p>如果 LOD 的切换只考虑当前 cluster 而不考虑邻域的 cluster，就会在切换时出现缝隙导致破面。直观的解决方法有两个：</p><ul> <li>简化时锁定与邻接 cluster 的共享边</li> <li>使独立的 cluster 始终与边缘匹配</li> </ul><p>但是锁边的方式会使得各 cluster 的边不再平滑。Nanite 的做法是在 build 时将 cluster 分组，同一个 cluster group 中的永远在同一级 LOD，这样一来就不存在独立的 cluster 了，也就不会产生 cracks。对组与组之间切换可能出现的问题，可以对组应用锁边，为此研究了不少方案，感兴趣的话可以查看 <a href="https://d-nb.info/997062789/34" target="_blank">Multiresolution structures for interactive visualization of very large 3D datasets</a> 和 <a href="https://vcg.isti.cnr.it/Publications/2005/CGGMPS05/Slide_BatchedMT_Vis05.pdf" target="_blank">Batched Multi Triangulations</a> 这两篇论文。</p></section><section><h3>Streaming<a href="#streaming"><span>#</span></a></h3><p>这种 LOD 策略能够匹配 virtual geometry 的需求，不需要将所有的 cluster 一次性全加载到内存中，只需要将切割下来的 cluster 加载即可，不需要渲染的 cluster 可以被置换出去。就像 VT 一样根据需要来请求数据。</p><hr /></section></section> <section><h2>Build<a href="#build"><span>#</span></a></h2><p>首先构建叶子节点 cluster，一个 cluster 包含 128 个三角形。</p><p>然后对每一级 LOD：</p><ul> <li>分组以消除共享边</li> <li>合并三角形</li> <li>将三角形的数量简化至原有数量的一半</li> <li>将简化后的三角形继续划分为 cluster</li> </ul><p>重复该过程，直到 cluster 只剩下一个。</p><p><img src="/assets/images/Nanite/build-operation.png" alt="" /> <em>构建示例</em></p><p>注意到构建完成后，最后形成的数据结构不是树而是一个 DAG。</p><section><h3>Group 的选择<a href="#group-的选择"><span>#</span></a></h3><p>具有较多共享边的 Cluster 应归于一个组中。当具有的外轮廓的边越少，被锁定的边越少，最终简化时受到的制约就越少。</p><p>这种问题在 CS 领域被称为 <strong>Graph Partitioning</strong>，它会将一个图拆成多个划分，并保证 一个划分到另一个划分的权重是最小的。Nanite 的 cluster 就对应 Graph 中的节点， Cluster group 对应一个划分，共享边的数量就对应划分的权重。</p><p>Nanite 构建时使用了 METIS 库来做划分。</p></section><section><h3>初次构建 cluster<a href="#初次构建-cluster"><span>#</span></a></h3><p>构建 cluster 时需要考虑多方面的因素：</p><ul> <li>为了使剔除的效率尽可能高，cluster 的 bounds 应尽量小</li> <li><strong>为了保证光栅化的效率，每个 cluster 的三角面的数量应该接近 128</strong></li> <li>为了保证 primitive shader 地效率，每个 cluster 的顶点应该存在上限</li> <li><strong>为了 nanite mesh 更自然地退化，cluster 外轮廓的边数应尽量少</strong></li> </ul><p>这些要求很难完全满足，因此<strong>三角形的数量</strong>以及<strong>外轮廓边数</strong>被优先考虑。</p></section><section><h3>模型简化<a href="#模型简化"><span>#</span></a></h3><p>模型简化使用的是传统的边缘坍缩算法，误差简化是由 Quadric Error Metric （QEM）得来。</p><p>相邻的两级 LOD 会计算一个estimate error，运行时会根据这个值投影到屏幕空间所产生的像素误差来判断是否需要切换 LOD。</p></section></section> <section><h2>无缝 LOD 切换<a href="#无缝-lod-切换"><span>#</span></a></h2><p>简单来说，Nanite 会将模型简化时保存的 estimate error 投影到屏幕空间，然后通对型到相机的距离和角度来评估具体使用哪一级 LOD。</p><p>切换 LOD 时，同一个 Group 中的所有 cluster 都需要进行同样的 LOD 选择，这个做法就是为了避免出现上面提到的破面问题。为此，同一组中的 cluster 需要存储相同的误差以及 sphere bounds。</p><p>选择 LOD 的过程实际上就是对 Cluster 结构的 DAG 进行裁剪的过程。为了能在 GPU 中高效执行，需要考虑并行化。</p><p>DAG 会在什么时候发生裁剪呢？或者说 LOD 会在什么时候进行切换呢？</p><p>当父一级的误差较大，而当前节点的误差较小，小于我们设定的误差阈值时，这时父、子节点评估误差的结果不同，此时发生切换。这种说法只有在 DAG 中存在一条分割线时成立，这就要求 DAG 中从根节点到叶子节点的路径的误差评估结果必须是单调的。这一点不会在运行时处理，会在 Nanite 构建时完成。</p><p>运行时再加上 TAA，subpixel 的变化会被进一步平滑。</p><section><h3>分层 LOD 选择<a href="#分层-lod-选择"><span>#</span></a></h3><p>一个场景中会有许多的 cluster 对它们逐个遍历消耗会很大，而其中绝大多数是不可见的，因此考虑添加一个层次结构，逐步细化。</p><ul> <li>当 <strong>ParentError &gt; threshold &amp;&amp; ClusterError &lt;= threshold</strong> 时<strong>渲染</strong></li> <li>当 <strong>ParentError &lt;= threshold || ClusterError &gt; threshold</strong> 时<strong>裁剪</strong></li> </ul><p>Nanite 通过构建一个 8 节点的 BVH 来实现 LOD 的分层。BVH 的剔除是一个多 Pass 的剔除，每个 Pass 处理一层 BVH。实践中为了充分利用缓存机制，往往希望处理完父节点立刻处理其子节点，但是目前 GPU 不支持这种调度。</p><p>但是 GPU 可以对之前的线程进行重用，并指派新的工作。</p></section><section><h3>双 Pass 遮挡剔除<a href="#双-pass-遮挡剔除"><span>#</span></a></h3><p>在通过 BVH 来选择 LOD cluster 时，可见性检查同时也在进行。但是再引入了 LOD 切换后，原先的流程需要一定的修改。</p><p>由于每帧的 LOD 选择不同导致 cluster group 会流入流出，每帧的 cluster 数据可能会不同。所以我们应该检查当前选择的 clusters 是否在上一帧可见。</p><p>然后流程就变为了这样：</p><ul> <li>用上一帧的 transform 来做 HZB 测试</li> <li>渲染可见的几何体并保存不可见的几何体列表</li> <li>从 depth buffer 构建当帧 HZB，并使用不可见几何体列表执行 HZB 测试</li> <li>渲染前一帧不可见但当前帧可见的几何体</li> <li>重新构建该帧的 HZB 留给下一帧使用</li> </ul><p><img src="/assets/images/Nanite/cull-dataflow.webp" alt="" /> <em>Nanite 的双 Pass 剔除</em></p><hr /></section></section> <section><h2>光栅化<a href="#光栅化"><span>#</span></a></h2><p>接下来要讨论的一个问题是光栅化的尺度。</p><p>小三角形对传统的光栅化性能有较大影响。光栅器会划分出较大的 tile，然后再将该 tile 划分为 4x4 的小 tile，最终以 2x2 的像素的 quad 输出。</p><p>现代 GPU 的并行是针对的像素而不是三角形。</p><section><h3>软光栅<a href="#软光栅"><span>#</span></a></h3><p>放弃传统光栅意味着放弃硬件单元的支持，也失去了光栅化单元以及深度测试的硬件支持，但是 depth buffer 又是必然需要的。</p><p>Nanite 的软光栅通过 64 位的原子操作 <a href="https://learn.microsoft.com/zh-cn/windows/win32/direct3dhlsl/interlockedmax" target="_blank">InterlockedMax</a> 完成，从高位到低位分别是：</p> <table><thead><tr><th>Depth</th><th>可见 cluster 的索引</th><th>三角形索引</th></tr></thead><tbody><tr><td>30</td><td>24</td><td>7</td></tr></tbody></table><p>软光栅运行在一个 thread group 为 128 的 compute shader 上，分两个阶段：</p><ul> <li>第一阶段处理顶点，每个顶点对应一个线程，如果 cluster 中的顶点数大于 128，那么可以再启动一个线程组。</li> <li>第二阶段处理面片，每个三角形对应一个线程，先读取顶点属性插值然后再光栅化。</li> </ul><p>这里有一段简化处理面片的代码以供参考:</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>for</span><span> (</span><span>uint</span><span> y = MinPixel.y; y&lt; MaxPixel.y; x++)</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>float</span><span> CX0 = CY0;</span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span> CX1 = CY1;</span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span> CX2 = CY2;</span></div></div><div><div><div>6</div></div><div><span> </span><span>float</span><span> ZX = ZY;</span></div></div><div><div><div>7</div></div><div> </div></div><div><div><div>8</div></div><div><span> </span><span>for</span><span> (</span><span>uint</span><span> x = MinPixel.x; x &lt; MaxPixel.x; x++)</span></div></div><div><div><div>9</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>10</div></div><div><span> </span><span>if</span><span>( </span><span>min3</span><span>(CX0, CX1, CX2) &gt;= </span><span>0</span><span> )</span></div></div><div><div><div>11</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>12</div></div><div><span> </span><span>WritePixel</span><span>(PixelValue, </span><span>uint2</span><span>(x,y), ZX);</span></div></div><div><div><div>13</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>14</div></div><div> </div></div><div><div><div>15</div></div><div><span><span> </span></span><span>CX0 -= Edge01.y;</span></div></div><div><div><div>16</div></div><div><span><span> </span></span><span>CX1 -= Edge12.y;</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>CX2 -= Edge20.y;</span></div></div><div><div><div>18</div></div><div><span><span> </span></span><span>ZX += GradZ.x;</span></div></div><div><div><div>19</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>20</div></div><div> </div></div><div><div><div>21</div></div><div><span><span> </span></span><span>CX0 += Edge01.x;</span></div></div><div><div><div>22</div></div><div><span><span> </span></span><span>CX1 += Edge12.x;</span></div></div><div><div><div>23</div></div><div><span><span> </span></span><span>CX2 += Edge20.x;</span></div></div><div><div><div>24</div></div><div><span><span> </span></span><span>ZX += GradZ.y;</span></div></div><div><div><div>25</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>这里虽然有循环的嵌套，但是由于覆盖的像素少，因此消耗并不高。</p></section><section><h3>硬光栅<a href="#硬光栅"><span>#</span></a></h3><p>大尺寸的面片依然走硬件光栅，这样性能会更好。使用硬光栅还是软光栅需要逐 cluster 评估，Nanite 将像素数小于 32 的三角形面片定义为小三角形，对于这种三角形使用软光栅。</p><p>这么做带来的问题是，上面代码中的迭代次数的增多，但有效像素占比下降。那么不对整个矩形进行光栅化，而是用扫描线的方法又如何呢？不对 X 轴方向上的像素逐个测试是否覆盖三角形，直接解出 X 轴方向上的像素数量就可以了。</p><p>按照这个方法对上面的代码进行优化后：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float3</span><span> Edge012 = { Edge01.y, Edge12.y, Edge20.y };</span></div></div><div><div><div>2</div></div><div><span>bool3</span><span> bOpenEdge = Edge012 &lt; </span><span>0</span><span>;</span></div></div><div><div><div>3</div></div><div><span>float3</span><span> InvEdge012 = Edge012 == ? 1e8: </span><span>rcp</span><span>( Edge012 );</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span>for</span><span> (</span><span>uint</span><span> y = MinPixel.y; y&lt; MaxPixel.y; x++)</span></div></div><div><div><div>6</div></div><div><span>{</span></div></div><div><div><div>7</div></div><div><span> </span><span>float3</span><span> CrossX = </span><span>float3</span><span>( CY0, CY1, CY2 ) * InvEdge012;</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span> </span><span>float3</span><span> MinX = bOpenEdge? CrossX: </span><span>0</span><span>;</span></div></div><div><div><div>10</div></div><div><span> </span><span>float3</span><span> MaxX = bOpenEdge? MaxPixel.x - MinPixel.x : CrossX;</span></div></div><div><div><div>11</div></div><div> </div></div><div><div><div>12</div></div><div><span> </span><span>float</span><span> x0 = </span><span>cell</span><span>( </span><span>max</span><span>(MinX.x, MinX.y, MinX.z) );</span></div></div><div><div><div>13</div></div><div><span> </span><span>float</span><span> x1 = </span><span>min3</span><span>( MaxX.x, MaxX.y, MaxX.z );</span></div></div><div><div><div>14</div></div><div><span> </span><span>float</span><span> ZX = ZY + GradZ.x * x0;</span></div></div><div><div><div>15</div></div><div> </div></div><div><div><div>16</div></div><div><span><span> </span></span><span>x0 += MinPixel.x;</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>x1 += MinPixel.x;</span></div></div><div><div><div>18</div></div><div> </div></div><div><div><div>19</div></div><div><span> </span><span>for</span><span> (</span><span>float</span><span> x = x0; x &lt;= x1; x++)</span></div></div><div><div><div>20</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>21</div></div><div><span> </span><span>WritePixel</span><span>(PixelValue, </span><span>uint2</span><span>(x,y), ZX);</span></div></div><div><div><div>22</div></div><div><span><span> </span></span><span>ZX += GradZ.x;</span></div></div><div><div><div>23</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>24</div></div><div> </div></div><div><div><div>25</div></div><div><span><span> </span></span><span>...</span></div></div><div><div><div>26</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>当一个 wave（wavefront？）中 X 方向的像素数大于 4，将会使用扫描线光栅器。</p></section><section><h3>Overdraw<a href="#overdraw"><span>#</span></a></h3><p>Nanite 没有逐三角形的裁剪，就像是 EarlyZ test 没有硬件支持。HZB 测试也是针对 cluster 中的几何体，而没有细致到像素级别，因此 overdraw 带来的问题不容小觑。</p><p>当表面互相覆盖的距离很小，小于 cluster 的边界时，这种紧密的重叠会导致 Nanite 的 overdraw。此外，如果几何体表面布满孔洞也会影响光栅化效率。</p><p>Mesh 稀疏的地方通常会存在大量大的三角形面片，但这并不会降低 overdraw。大的三角形对应大的 cluster，cluster 越大，剔除的粒度就越粗糙。这意味着随着三角形的增大，像素的 overdraw 也会随之上升。这就是说渲染的三角形较多时，运行效率反而越高。</p><p>这与传统的光栅化流程确实大相径庭，难怪原文中会说:</p><blockquote><p>The reliance on previous frame depth for occlusion culling is one of Nanite’s biggest deficiencies.</p></blockquote></section><section><h3>Tiny instances<a href="#tiny-instances"><span>#</span></a></h3><p>如果物件实例在屏幕空间占用的像素本身就很少呢？</p><p>最直观的方法时直接根据 screen size 把它剔除掉，但如果该实例是各个几何体的组成结构，那么直接剔除掉可能会导致整个几何体消失不见。</p><p>所以在某个时间点，需要将实例合并。比如远处的地形需要流出并替换为低模的代理网格。因为即使 Nanite 可以调整渲染消耗，但是有限的内存并不允许这么做。</p><p>因此在远处，应尽可能的将实例合并为代理。</p><p>在合并之前的过渡区域，Nanite 也提供 visibility buffer imposters 作为补充。该做法与常规的 imposter 类似，但是存储的是 visbility buffer 中的 depth 和三角形 ID。</p><hr /></section></section> <section><h2>Nanite 材质<a href="#nanite-材质"><span>#</span></a></h2><p>在解码了 visibility buffer 后可以获取到 depth, visible cluster index 和 triangle index。接下来就可以准备评估材质了，这一步骤可简略的描述为：</p><ul> <li>加载 VisBuffer</li> <li>加载 Visible Cluster 获取到 Instance Id 和 Cluster Id</li> <li>加载 instance 的变换信息</li> <li>加载三角形顶点的索引</li> <li>加载三角形顶点的位置</li> <li>将三角形的顶点变换到屏幕空间</li> <li>推到出三角形像素的重心坐标</li> <li>加载属性并进行插值</li> </ul><section><h3>材质 ID<a href="#材质-id"><span>#</span></a></h3><p>在解码 vis buffer 后可以通过 visible cluster index 获得 instance id 和 cluster id，在这之后可以：</p><ul> <li>通过 cluster id 和 triangle id 获得 material slot id</li> <li>通过 instance id 和 material slot id 获得 material id</li> </ul><p>这样一来理论上可以通过一个全屏的 quad 来绘制出所有的材质，但实际上的做法是对每个独立的材质来绘制全屏的 quad，渲染时跳过那些未被当前材质覆盖的像素。由于 Nanite 是 GPU 驱动的，CPU 并不知道材质的可见性，所以对每个材质都做全屏的像素检查会非常低效，因此需要对材质像素进行剔除。</p></section><section><h3>材质剔除<a href="#材质剔除"><span>#</span></a></h3><p>剔除材质时最先想到的是使用 stencil，但是这就要在每次材质渲染时重置一次，因此选择使用 depth test，将 material id 看作是 depth 来做剔除。</p><p>用一个 CS 可以输出标准深度和材质深度，当输出标准深度时，也可以一并输出 HTILE 以对 HZB 加速。</p><p>最终对所有的材质绘制一个全屏的 quad，将材质的深度写入到 quad 的 Z 中，并将深度测试设为等于，便可以仅绘制相应材质的像素了。</p><p>但依然有更好的方法：将全屏的 quad 划分为 8x4 的 tile 进行粗粒度的剔除，并将这个信息写入一个 32 位的 mask 中。如果一个 tile 中不包含相应的材质，则为 mask 中相应的 tile 设置标记，并在 VS 中将 x 的值写为 NaN 来跳过该 tile。</p></section><section><h3>UV 差分<a href="#uv-差分"><span>#</span></a></h3><p>Nanite 的软光栅的单位是也是 quad，但相比传统的三角形光栅，nanite 的光栅并非是 2x2 像素的 quad，而是一个较大的 quad 它会包含多个三角形，因此能减少小面片造成的 overdraw。</p><p>为避免混淆，下面我会称 nanite 的光栅化以一个 tile 为基本光栅单位。</p><p>不过，这样一来，一次光栅化流程中可能会包含 depth 不连续处、UV seam，甚至不同的物件的像素。</p><p>为了确保像素的连续，nanite 的做法是做链式求导，传统光栅是将 2x2 的 quad 作为基本单位，而 nanite 为确保像素连续，需要根据原本 tile 再次微分。如果无法通过该解析方法来获取差分信息，将会回退到相邻像素的差分法来模拟微分结果。有了微分的结果，在采样时可以指定梯度来进一步采样贴图。</p><p>以上的操作是在材质通过 HLSL translator 转化时实现的，所以对于用户层是感知不到的。</p><hr /></section></section> <section><h2>性能数据<a href="#性能数据"><span>#</span></a></h2><p>Main pass:</p> <table><thead><tr><th>指标</th><th>数量</th></tr></thead><tbody><tr><td>Instances pre-cull</td><td>896322</td></tr><tr><td>Instances post-cull</td><td>3668</td></tr><tr><td>Cluster node visits</td><td>39274</td></tr><tr><td>Cluster candidates</td><td>1536794</td></tr><tr><td>Visible clusters SW</td><td>184828</td></tr><tr><td>Visible clusters HW</td><td>6686</td></tr></tbody></table><p>Post pass:</p> <table><thead><tr><th>指标</th><th>数量</th></tr></thead><tbody><tr><td>Instances pre-cull</td><td>102804</td></tr><tr><td>Instances post-cull</td><td>365</td></tr><tr><td>Cluster node visits</td><td>19139</td></tr><tr><td>Cluster candidates</td><td>458805</td></tr><tr><td>Visible clusters SW</td><td>7370</td></tr><tr><td>Visible clusters HW</td><td>536</td></tr></tbody></table><p>Total rasterized:</p> <table><thead><tr><th>指标</th><th>数量</th></tr></thead><tbody><tr><td>Clusters</td><td>199,420</td></tr><tr><td>Triangles</td><td>25,041,711</td></tr><tr><td>Vertices</td><td>19,851,262</td></tr></tbody></table><p>可见，通过 Nanite LOD 将原本数以亿计的面片减少到了 2500 万左右。</p><p>在古代山谷的 demo 中使用了动态分辨率并通过时域上采样到 4K 分辨率，上采样前的分辨率平均约为 1400p。Vis buffer 的绘制消耗了约 2.5 ms，将 vis buffer 中的信息输出到 GBuffer 的消耗约为 2 ms.由于 nanite 由 GPU 驱动，整个过程中基本上没有 CPU 消耗。</p><hr /></section> <section><h2>Nanite 阴影<a href="#nanite-阴影"><span>#</span></a></h2><p>实际的渲染中并不是只有 main pass 才绘制多边形，绘制阴影时也需要。但是绘制阴影时需要如此多的细节吗？</p><p>也许间接光不需要，但是物件的阴影还是需要的。高精度几何体和使用法线贴图的几何体最大的区别就是<strong>自阴影的细节</strong>。</p><p>由于光追在对海量几何体数量的支持上仍存在问题，因此目前还是使用传统的光栅化阴影方案。这种情况下需要严格限制灯光数量，否则阴影的消耗会因 Nanite 而迅速增长。</p><p>考虑到 Nanite 处理的都是静态网格体，而场景里的灯光大部分也是静态的，因此可以考虑通过缓存来减少阴影的计算消耗，那么很自然地想到使用 Virtual Shadow Map。</p><section><h3>VSM<a href="#vsm"><span>#</span></a></h3><p>使用的 VSM 的分辨率为 16k，这是为了匹配屏幕像素。上面说过，一个 cluster 的三角形数量会控制在 128 个左右，因此 VSM 的每个 page 的大小也为 128，因此虚拟化的阴影贴图 mip0 包含 128 个页，也就是 16k 的分辨率。</p><p>绘制 VSM 时大致流程如下：</p><ul> <li>对屏幕空间的每个像素</li> <li>对影响到当前像素的每个灯光</li> <li>将该像素投影到阴影空间</li> <li>通过屏幕空间像素和阴影贴图的纹素是一比一的原则来选择 mip level</li> <li>标记该 mip level 中的 page 为 needed</li> </ul><p>VSM 依赖于缓存功能，大多数情况下只有移动的物体和相机移动时视锥的边缘需要重绘。</p></section><section><h3>Multi-view<a href="#multi-view"><span>#</span></a></h3><p>Nanite 的绘制具有相当多的流程，因此它的性能消耗受制于同步。因此 nanite 的调用应尽量少，试想假如一盏灯绘制一次阴影便要调用一次 nanite，最终的性能消耗会相当爆炸。</p><p>因此 nanite 管线中实现了多视图的绘制而非只输出一个视图的信息。这能使得 nanite 能够一次性为场景中所有的灯光生成虚拟化的 shadow map 的 mip。</p></section><section><h3>阴影的裁剪<a href="#阴影的裁剪"><span>#</span></a></h3><p>阴影的裁剪也会走 HZB，不过额外添加了一个测试。那就是将 cluster/instance 投影到阴影空间，然后判断投影到的 shadow page 是否被标记为 needed，如果没有标记便剔除掉。</p><p>对于软光栅管线，由于软光栅处理的都是很小的 cluster，因此可以认为该 cluster 中的内容不会跨多个 page，也就是软光栅流程中一个 cluster 对应一个 page。</p><p>但硬件光栅处理的 cluster 的粒度较大，很可能会一个 cluster 占多个 page。对此引入了 VSM 逐像素的虚拟页表到物理页表的转换。</p></section><section><h3>Shadow LOD<a href="#shadow-lod"><span>#</span></a></h3><p>与 Nanite 的 LOD 选择一样，nanite 阴影的 LOD 也是根据一个像素单位的误差来评估 LOD 的。上面说了，屏幕空间像素和阴影贴图的纹素是一比一的，这就使得阴影的绘制是与屏幕分辨率相关而非场景复杂度相关。</p><p>但是这不代表绘制到 shadow map 中的三角形与 main pass 中的完全一模一样，这就导致了几何体上可能会出现奇怪的自阴影（就像光追下的 nanite proxy），为此会在屏幕空间做一个短距离的 trace 来修复这个问题，所以理论上，可以选择提升 LOD 的误差来来使 LOD 选择更激进，从而提高渲染效率。</p><hr /></section></section> <section><h2>Streaming<a href="#streaming-1"><span>#</span></a></h2><p>在提到 Nanite 的 LOD 切换时有提到，如果当前渲染的 mesh 的精度与目标精度有差距时，就会让 CPU 加载更高精度的数据进来，且切换时是以 cluster group 为单位进行的，以确保 LOD 切换是无缝的。</p><p>因此在 streaming 时也要完成一整个 group 的加载才可以显示。</p><p>不像 VT 我们可以确定每个 page 的大小，nanite 中的 cluster 具有可变的顶点数、属性等等。同时为避免内存碎片，也应该固定流入单位的大小，因此必须能够适配可变数量的几何信息。</p><section><h3>分页<a href="#分页"><span>#</span></a></h3><p>Nanite 会将 cluster group 存入固定大小的 page 中，存入时会考虑数据的邻接关系和 DAG 中的层级关系，来确保运行时要加载的 page 数是最小的。</p><p>第一个页会作为 root 页驻留在 GPU 中，保存着尽可能多的 DAG 顶层信息。Root 页总是驻留在 GPU 中，意味者总是有东西需要渲染的。驻留页通过 ByteAddressBuffer 的形式在 GPU 中。可是 cluster group 中包含的数据量是很大的，如果直接以此填充 page 会有很多空间被浪费。</p><p>既然如此那就继续细分，将每个 group 以 cluster 为粒度参考，分成多个 parts，然后将它们填入到不同的 page 中以使内存紧凑。这样当需要加载这些 cluster 对应的 group 时就索引它们所在的 page 并加载。当这些 parts 全部加载完成，那么该 group 才能视作加载完成。</p></section><section><h3>评估<a href="#评估"><span>#</span></a></h3><p>VT 通过 UV 梯度可以得知需要加载哪些 page，对于 nanite 则需要对 DAG 进行层级遍历后才能得知加载哪些。</p><p>当前裁剪的层级节点需要包含的数据比我们流入所需要的更多，因为在遍历时，我们需要评估是否已经绘制了已经驻留在内存中的节点。不过这些层级信息只是一些 meta 信息，数据量并不大，所以可以全部加载进来。</p><p>有了全部的层级信息后，就可以跳过已加载的节点，进行完整的遍历，因此通常新加载的物体可以很快找到需要的 LOD 以及对应的 Page。</p><p>层级信息也可以 streaming，这个过程是分多帧进行的。</p><p>IO 延迟会导致更对可见的 mesh popping 的现象。</p></section><section><h3>流送请求<a href="#流送请求"><span>#</span></a></h3><p>流送请求实际上在层级 cluster culling 时就已经发生了。</p><p>请求的内容为评估完加载优先级的一系列 pages，具体的评判标准是 LOD 误差。评判时并不会只考虑主视图，同时也会考虑其他视图，比如阴影。</p><p>CPU 也会异步回读请求，它会为 DAG 添加缺失的依赖内容，并为优先级最高的 page 发送 IO 请求，然后将低优先级的 page 移出以腾出空间。</p><p>最后，当 IO 请求准备完毕，page 数据会被 GPU 加载并据此更新 GPU 端的数据结构，更新加载/卸载的 page，更新完成加载的 group 以及标记 cluster 为叶节点等。</p><hr /></section></section> <section><h2>Nanite 压缩<a href="#nanite-压缩"><span>#</span></a></h2><p>Nanite 有两种压缩几何体的表达方式，它们包含的数据相同，但根据使用方式有不同的优化方向。</p><p>面向内存 : 这种方式用于渲染代码中，在光栅化、材质评估时会使用。适用于需要实时解码和随机访问场合，为的是更好地控制 streaming pool，以使 cache 的利用率更高。</p><p>面向磁盘 : 这种方式用于 streaming 过程中，streaming 相较于 rendering 是低频的操作，因此该方式不适合需要随机访问的场合，但也因此可以添加更高级的特性，比如使用基于 byte 的 <a href="https://zh.wikipedia.org/wiki/LZ77%E4%B8%8ELZ78" target="_blank">LZ 压缩</a>，为的是让磁盘占用更小。</p><section><h3>内存数据<a href="#内存数据"><span>#</span></a></h3><section><h4>量化与编码<a href="#量化与编码"><span>#</span></a></h4><p>对于面向内存的方式，首先 Nanite 会根据美术同学的调整和一些启发式调整进行一次全局量化保存位置信息和各种属性，在保存为 cluster 后，会根据 cluster 中的 min/max 范围转换到本地坐标系下，可视为局部数据。</p><p>由于 cluster 的 min/max 会变，因此局部数据的存储位数会根据具体范围变化，每个 cluster 需要适配不同的顶点格式，不过 cluster 中的各顶点使用的格式是相同的。</p><p>该方式是直接用于实时渲染的，因此必须使用 GPU 上的 bitstream reader 来解码。</p><p>如果不同 cluster 使用不同的格式，不连续的量化会导致 cluster 间出现接缝。即使能够处理单个物件内部的接缝，放到具体场景中的摆放方式是无法预测的，此外，由于 nanite 是鼓励多用实例化的方法的，所以场景内多实例的摆放方式是无法避免的。为解决这一问题，这些物件需要全局尺度下的量化。</p><p>Nanite 的做法是添加了一个在本地坐标系原点中心的步长，然后由用户设定的值作为该步长的指数进行计算。该步长不是归一化的，对相同量化级别，平移/旋转也是步长的整数倍，从而使两个相邻物体能更好的保持一致。实践中，该做法只能保证叶子节点间的衔接，对于更上层的 LOD cluster，由于 group 可能会不一致，也就是相邻的两个物件使用了不同的 LOD 而导致接缝问题，但由于叶子节点才是距离相机最近的，其他的上层 LOD 距离相机更远，因此有接缝也不很明显。</p></section><section><h4>三角面和其他属性<a href="#三角面和其他属性"><span>#</span></a></h4><p>三角面的索引会进行简单的调整，使第一个顶点的索引是最小的。而 cluster 的顶点数存在上限，因此使用 7 bit 可以保存第一个索引。其他的两个索引则保存为相对于第一个索引的偏移，nanite 保证该偏移不超过 32，因此使用 5 bit 可以保存其余两个索引，共计 17 bit。</p><p>UV 坐标可能会存在跳变，在编码时，会考虑消除掉 UV 范围中最大的 gap 来实现对 UV 的压缩。</p><p>法线会直接用八面体坐标来表达。切线不存储，会在使用时通过 uv 坐标的梯度计算得出，实际上就是法线空间的 U 和 V，这种方法与屏幕空间微分来获取切线空间类似，不过 nanite 没有直接使用求导，而是重用材质 pass 中重新插值和贴图 LOD 采样所使用的三角形 delta。</p></section><section><h4>材质表<a href="#材质表"><span>#</span></a></h4><p>一个 mesh 可能会包含多个材质。为使材质表尽可能小，nanite 本想要使每个 cluster 仅包含一个材质，但这个做法有太多限制。所以最好将材质信息逐三角形保存，按材质进行排序，并存储每个材质对应的三角形的范围。</p><p>大多数 cluster 的材质数量不超过 3 个，这些会被记录在一个 32 bit 的表中。超过 3 个的 cluster 会被记录在一个 64 bit 的变长表中。</p></section></section><section><h3>磁盘数据<a href="#磁盘数据"><span>#</span></a></h3><p>会优先在支持硬件 LZ 解压的平台上使用硬件解压，在做该分享时，只支持 console 平台。</p><p>目前 nanite 的压缩是将现有数据转码为可压缩的格式，而不是重新设计一套格式来支持压缩，因为开发者认为支持硬件解压的会越来越多。</p><p>LZ 压缩提供了字符串匹配和乱序编码的支持，且压缩的数据主要是 transform，因此转码过程可在 GPU 端进行，以充分利用并行性，此外由于 nanite 在 GPU 端就包含了数据，无需从 CPU 端复制，也能减少不少消耗。</p><section><h4>LZ 优化<a href="#lz-优化"><span>#</span></a></h4><p>首先是将数据按 byte 对齐，这会增加原始的数据大小，但是可以减少压缩后的数据大小。</p><p>然后是将数据按类型重新排序，以最小化数据偏移的宽度。</p><p>Nanite 倾向于选择较小的字节值，这么做是为了更好的 LZ 编码效果。</p></section><section><h4>删除重复顶点<a href="#删除重复顶点"><span>#</span></a></h4><p>在 build 时产生了大量的冗余数据，由于 cluster 是独立的，各相邻的 cluster 之间的共享边存在重复的顶点。这些冗余顶点在 LZ 压缩的过程中通常无法识别，因为相同的顶点在不同的 cluster 中编码不同，而且 cluster 的父级的数据只有在 GPU 端才可见，因此冗余的部分要用引用的方式来存储。由于在引用的数据可能没有加载到内存中，可以使用父节点的数据作为 fallback。</p><p>通常有 30% 的顶点会被当做引用来存储。</p></section><section><h4>拓扑编码<a href="#拓扑编码"><span>#</span></a></h4><p>磁盘编码使用了更紧凑的拓扑，不再是记录每个三角形的三个顶点作为索引，而是用带状三角形序列，这样在每次新加一个三角形时只要单独添加一个顶点索引就可以。该三角形带越长需要的平均索引数就越少。</p><p>在记录顶点时要添加一个 bit 位 IsLeft 来指示新增的面片的位置关系。</p><p>编码时还会对首次使用到的顶点来排序，当第一次引用到某个顶点，它的索引将正好是目前遍历过的顶点的总数，因此不需要显式存储索引列表。但对非第一次顶点的引用仍需显式存储，nanite builder 保证这些引用与出现过的最大顶点索引之间的偏移不会超过 5 bit。</p><p>最终的编码结构是一系列 bit mask 和偏移值，它们用于指示何时重置条带、何时向左或向右移动以及是否需要应用偏移。</p> <table><thead><tr><th>IsReset</th><th>IsLeft</th><th>IsRef</th><th>Ref Value</th></tr></thead><tbody><tr><td>1</td><td>0</td><td>0</td><td>-</td></tr><tr><td>0</td><td>0</td><td>1</td><td>6</td></tr><tr><td>0</td><td>1</td><td>0</td><td>-</td></tr></tbody></table><p>假如首次引用的顶点索引为 v44，cluster 中出现的最大顶点索引为 v50，那么记录的偏移值就是 6。</p></section></section><section><h3>Demo 中的内存表现<a href="#demo-中的内存表现"><span>#</span></a></h3><p><a href="benchmark.png"></a></p><p>在压缩到磁盘后占用的内存只有 4.61GB 是原始数据的大概 18%，因此估算出每百万三角形占大概 10.9MB 的磁盘空间。不过按照<a href="https://dev.epicgames.com/documentation/en-us/unreal-engine/nanite-virtualized-geometry-in-unreal-engine?utm_source=chatgpt.com#data-size" target="_blank">官方文档</a>的说法现在每百万三角形的磁盘占用约为 13.8MB。</p><hr /></section></section> <section><h2>将来的工作<a href="#将来的工作"><span>#</span></a></h2><p>虽然 nanite 可以用于绝大多数情况了，但依然有些内容需要支持：</p><ul> <li>透明材质和 Mask 材质</li> <li>非刚性几何体的变形</li> <li>具有动画的几何体</li> </ul><p>此外还有光追、地形、细分等，对于绘制像树木、植被等聚合体的效果和性能消耗也并不理想，将会在未来持续更新。</p></section>https://yohiro.games/posts/generate-obb-via-pca/https://yohiro.games/posts/generate-obb-via-pca/Wed, 23 Nov 2022 00:00:00 GMT<section><h2>背景<a href="#背景"><span>#</span></a></h2><p>在游戏开发中，常见的包围盒有<em>球形包围盒</em>、<em>轴对齐包围盒（AABB）</em>、<em>有向包围盒（OBB）</em>、<em>K-DOP</em> 以及<em>凸包</em>等，包围盒通常用于判断可见性以及碰撞检测等领域。其中，AABB 和 OBB 的应用最为广泛。</p><p></p><figure><img src="/assets/images/OBB/bounding-box.png" alt="bounding-box" /><figcaption>bounding-box</figcaption></figure> <em>不同类型的包围盒</em><p></p><p>球形包围盒 : 球形包围盒是包含其所属对象的最小球体。一般用于较粗粒度的碰撞检测。</p><p>AABB (<strong>A</strong>xis-<strong>A</strong>ligned <strong>B</strong>ounding <strong>B</strong>ox) : 轴对齐包围盒（AABB） 是包含其所属对象的且各边平行于各坐标轴的最小六面体。它不会相对于轴进行旋转，因此可以仅通过 Min 和 Max 来进行定义，实现以及计算非常简单。但也由于不进行旋转，所以空间冗余很大。</p><p>OBB (<strong>O</strong>riented <strong>B</strong>ounding <strong>B</strong>ox) : 定向包围盒（OBB）是包含其所属对象的且相对于坐标轴方向任意的最小的长方体。相较于 AABB，OBB 的空间利用率更高，但求交的时候也会变得复杂。</p><p>K-DOP (<strong>D</strong>iscrete <strong>O</strong>riented <strong>P</strong>olytope) : 离散定向多面体 （DOP）是包含其所属对象的二维空间的凸多边形或者三维空间的凸多面体，它是一组无限远的定向平面移动到与物体相交而得到，于是 DOP 就是这些平面相交平面所生成的凸多面体。从 K 个平面构建的 DOP 称为 k-DOP。</p><p>凸包 (Convex Hull) : 凸包是包容物体的最小凸体。如果物体是有限个点的集合，那么凸包就是一个多面体，实际上它是包容多面体的最小立体。</p><p>这篇文章主要用于阐述如何使用主成分分析（<strong>P</strong>rincipal <strong>C</strong>omponents <strong>A</strong>nalysis）方法获得 OBB 的基向量，然后据此进一步创建 OBB。</p></section> <section><h2>PCA 介绍以及原理<a href="#pca-介绍以及原理"><span>#</span></a></h2><p>想象一下在二维平面上有一系列的点，我们需要找到这些点分布最为零散的两个方向，且有尽可能多的点（的投影）在这两个方向上。这里我们要找的这<code>两个方向</code>实际上就是主成分，也是 OBB 的基向量。</p><section><h3>介绍<a href="#介绍"><span>#</span></a></h3><p>主成分分析是一种统计分析、简化数据集的方法。它利用正交变换来对一系列可能相关的变量的观测值进行线性变换，从而投影为一系列线性不相关变量的值，这些<em>不相关变量</em>称为<em>主成分</em>。简单来讲，PCA 解决了这样一个问题：</p><blockquote><p>如果我们有一组N维向量，现在要将其降到K维（K小于N），那么我们应该如何选择K个基才能最大程度保留原有的信息？</p></blockquote><p></p><figure><img src="/assets/images/OBB/PCA.png" alt="PCA" /><figcaption>PCA</figcaption></figure> <em>一个高斯分布的主成分分析。黑色的两个向量是此分布的协方差矩阵的特征向量，其长度为对应的特征值之平方根，并以分布的平均值为原点。</em><p></p></section><section><h3>原理<a href="#原理"><span>#</span></a></h3><p>PCA 中的主成分实际上指的是，使数据中的点的投影数量最多的向量，这个向量被称作主成分。</p><p>那么如何找到主成分呢？这里来回顾一下数理统计和线性代数的内容。</p><p>在统计学里，**方差（Variance）**用于描述数据的离散程度，**协方差（Covariance）**用于描述数据的相关程度。方差越大，数据越离散；协方差越大，样本的相关性越强。</p><p></p><figure><img src="/assets/images/OBB/Covariance_trends.png" alt="Covariance Trends" /><figcaption>Covariance Trends</figcaption></figure><p></p><p>对于一维的数据，只需要找出方差最大的方向即可，这个方向就是一维数据的主成分，但对于更高维度的数据就需要引入协方差来解决。</p><p>对期望为 <span><span>E(X)E(X)</span><span><span><span></span><span>E</span><span>(</span><span>X</span><span>)</span></span></span></span> 和 <span><span>E(Y)E(Y)</span><span><span><span></span><span>E</span><span>(</span><span>Y</span><span>)</span></span></span></span> 的随机变量 <span><span>XX</span><span><span><span></span><span>X</span></span></span></span> 和 <span><span>YY</span><span><span><span></span><span>Y</span></span></span></span> 的协方差 <span><span>Cov(X,Y)Cov(X,Y)</span><span><span><span></span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Y</span><span>)</span></span></span></span> 定义如下：</p><span><span><span>Cov(X,Y)=E[(X−E(X))(Y−E(Y))]=E(XY)−E(X)E(Y)Cov(X,Y) = E[(X-E(X))(Y-E(Y))] = E(XY) - E(X)E(Y)</span><span><span><span></span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Y</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span>E</span><span>[(</span><span>X</span><span></span><span>−</span><span></span></span><span><span></span><span>E</span><span>(</span><span>X</span><span>))</span><span>(</span><span>Y</span><span></span><span>−</span><span></span></span><span><span></span><span>E</span><span>(</span><span>Y</span><span>))]</span><span></span><span>=</span><span></span></span><span><span></span><span>E</span><span>(</span><span>X</span><span>Y</span><span>)</span><span></span><span>−</span><span></span></span><span><span></span><span>E</span><span>(</span><span>X</span><span>)</span><span>E</span><span>(</span><span>Y</span><span>)</span></span></span></span></span><p>从上式可知，<span><span>Cov(X,Y)=Cov(Y,X)Cov(X,Y) = Cov(Y,X)</span><span><span><span></span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Y</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span>C</span><span>o</span><span>v</span><span>(</span><span>Y</span><span>,</span><span></span><span>X</span><span>)</span></span></span></span>。</p><p>接下来要通过协方差来确定线性无关的方向，也就是独立的方向作为主成分。</p><p>参考右图，当 <span><span>Cov(X,Y)&gt;0Cov(X,Y) &gt; 0</span><span><span><span></span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Y</span><span>)</span><span></span><span>&gt;</span><span></span></span><span><span></span><span>0</span></span></span></span> 时，两变量<em>线性正相关</em>；当 <span><span>Cov(X,Y)&lt;0Cov(X,Y) &lt; 0</span><span><span><span></span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Y</span><span>)</span><span></span><span>&lt;</span><span></span></span><span><span></span><span>0</span></span></span></span> 时，两变量<em>线性负相关</em>。 而当 <span><span>Cov(X,Y)==0Cov(X,Y) == 0</span><span><span><span></span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Y</span><span>)</span><span></span><span>==</span><span></span></span><span><span></span><span>0</span></span></span></span> 时，两变量<em>线性无关</em>。</p><p>假设我们的数据是三维的，通过计算各维度数据的协方差后，可以得到这样的一个<em>实对称矩阵</em>，也就是<strong>协方差矩阵</strong>：</p><span><span><span>[Cov(X,X)Cov(X,Y)Cov(X,Z)Cov(Y,X)Cov(Y,Y)Cov(Y,Z)Cov(Z,X)Cov(Z,Y)Cov(Z,Z)]\begin{bmatrix} Cov(X,X) &amp; Cov(X,Y) &amp; Cov(X,Z) \\ Cov(Y,X) &amp; Cov(Y,Y) &amp; Cov(Y,Z) \\ Cov(Z,X) &amp; Cov(Z,Y) &amp; Cov(Z,Z) \\ \end{bmatrix}</span><span><span><span></span><span><span><span><span><span><span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span><span><span><span><span><span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>X</span><span>)</span></span></span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>Y</span><span>,</span><span></span><span>X</span><span>)</span></span></span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>Z</span><span>,</span><span></span><span>X</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span><span></span><span><span><span><span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Y</span><span>)</span></span></span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>Y</span><span>,</span><span></span><span>Y</span><span>)</span></span></span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>Z</span><span>,</span><span></span><span>Y</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span><span></span><span><span><span><span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>X</span><span>,</span><span></span><span>Z</span><span>)</span></span></span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>Y</span><span>,</span><span></span><span>Z</span><span>)</span></span></span><span><span></span><span><span>C</span><span>o</span><span>v</span><span>(</span><span>Z</span><span>,</span><span></span><span>Z</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span><span><span><span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span></span><p>主对角线的元素是变量与其自身的协方差，代表该变量的方差，非对角线上的元素代表协方差。</p><p>要找线性无关的方向，就要使协方差等于零，就需要令协方差矩阵的非对角线位置的值为零。因此，我们需要对协方差矩阵做变换，使非对角线上的元素化为零, 也就是将协方差矩阵<strong>对角化</strong>。</p><p>由于协方差矩阵是一个实对称矩阵，因此它具有以下性质：</p><ul> <li>实对称矩阵的不同特征值所对应的特征向量是正交的</li> <li>n 阶实对称矩阵必可对角化</li> <li>k 重特征值必有 k 个线性无关的特征向量</li> </ul><p>也就是说，对于三维向量所构成的协方差矩阵，必然存在三个正交的特征向量，而这三个特征向量正是 OBB 的基向量，也就是 OBB 本地坐标系的轴。</p><p>那么接下来的问题就是如何将协方差矩阵对角化，求得特征向量。</p><p>求解特征向量，可以参考 <a href="https://zh.wikipedia.org/wiki/%E9%9B%85%E5%8F%AF%E6%AF%94%E6%B3%95" target="_blank">Jacobi 迭代法</a>，也可以调用其他三方数学库，如 Eigen。</p><p>通过以上步骤便获得了主成分，也就是 OBB 的基向量，更详细的数学解释可以阅读<a href="http://blog.codinglabs.org/articles/pca-tutorial.html" target="_blank">这篇文章</a>。</p></section></section> <section><h2>算法<a href="#算法"><span>#</span></a></h2><ol> <li>获得模型的顶点作为输入样本集</li> <li>对样本集中的每一个样本点，计算协方差矩阵</li> <li>求解协方差矩阵的特征向量</li> <li>对样本集中的每一个样本点，计算样本点在特征向量方向上的投影距离，以获得 OBB 在各轴向上的最大和最小值</li> <li>将上一步得到的 OBB 坐标系下的数据转换到世界坐标系下</li> </ol><p>伪代码如下：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>void</span><span> </span><span>PCA</span><span>(</span><span>samples</span><span>, </span><span>x</span><span>, </span><span>y</span><span>, </span><span>z</span><span>)</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>// 期望</span></div></div><div><div><div>4</div></div><div><span> </span><span>vec3</span><span> </span><span>center</span><span> </span><span>=</span><span> </span><span>0</span></div></div><div><div><div>5</div></div><div><span> </span><span>for</span><span> </span><span>p</span><span> </span><span>in</span><span> </span><span>samples</span><span>:</span></div></div><div><div><div>6</div></div><div><span> </span><span>center</span><span> </span><span>+=</span><span> </span><span>p</span></div></div><div><div><div>7</div></div><div><span> </span><span>center</span><span> </span><span>/=</span><span> </span><span>samples</span><span>.</span><span>size</span><span>()</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span><span> </span></span><span>// 协方差矩阵</span></div></div><div><div><div>10</div></div><div><span> </span><span>matrix3x3</span><span> </span><span>Covmatrix</span></div></div><div><div><div>11</div></div><div><span> </span><span>for</span><span> </span><span>p</span><span> </span><span>in</span><span> </span><span>samples</span><span>:</span></div></div><div><div><div>12</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>0</span><span>][</span><span>0</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>x</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>x</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>x</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>x</span><span>)</span></div></div><div><div><div>13</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>1</span><span>][</span><span>0</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>x</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>x</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>y</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>y</span><span>)</span></div></div><div><div><div>14</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>2</span><span>][</span><span>0</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>x</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>x</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>z</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>z</span><span>)</span></div></div><div><div><div>15</div></div><div> </div></div><div><div><div>16</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>0</span><span>][</span><span>1</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>y</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>y</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>x</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>x</span><span>)</span></div></div><div><div><div>17</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>1</span><span>][</span><span>1</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>y</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>y</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>y</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>y</span><span>)</span></div></div><div><div><div>18</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>2</span><span>][</span><span>1</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>y</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>y</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>z</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>z</span><span>)</span></div></div><div><div><div>19</div></div><div> </div></div><div><div><div>20</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>0</span><span>][</span><span>2</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>z</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>z</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>x</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>x</span><span>)</span></div></div><div><div><div>21</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>1</span><span>][</span><span>2</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>z</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>z</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>y</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>y</span><span>)</span></div></div><div><div><div>22</div></div><div><span> </span><span>Covmatrix</span><span>[</span><span>2</span><span>][</span><span>2</span><span>] </span><span>+=</span><span> (</span><span>p</span><span>.</span><span>z</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>z</span><span>) </span><span>*</span><span> (</span><span>p</span><span>.</span><span>z</span><span> </span><span>-</span><span> </span><span>center</span><span>.</span><span>z</span><span>)</span></div></div><div><div><div>23</div></div><div> </div></div><div><div><div>24</div></div><div><span><span> </span></span><span>// 将协方差矩阵归一化</span></div></div><div><div><div>25</div></div><div><span> </span><span>for</span><span> </span><span>elem</span><span> </span><span>in</span><span> </span><span>Covmatrix</span></div></div><div><div><div>26</div></div><div><span> </span><span>elem</span><span> </span><span>/=</span><span> </span><span>samples</span><span>.</span><span>size</span><span>()</span></div></div><div><div><div>27</div></div><div> </div></div><div><div><div>28</div></div><div><span><span> </span></span><span>// 求解协方差矩阵的特征向量矩阵</span></div></div><div><div><div>29</div></div><div><span> </span><span>matrix3x3</span><span> </span><span>eigenVectors</span><span> </span><span>=</span><span> </span><span>computeEigenVectors</span><span>(</span><span>Covmatrix</span><span>)</span></div></div><div><div><div>30</div></div><div><span> </span><span>vec3</span><span> </span><span>lambda0</span><span> </span><span>=</span><span> </span><span>eigenVectors</span><span>.</span><span>getColVector</span><span>(</span><span>0</span><span>).</span><span>normalize</span><span>()</span></div></div><div><div><div>31</div></div><div><span> </span><span>vec3</span><span> </span><span>lambda1</span><span> </span><span>=</span><span> </span><span>eigenVectors</span><span>.</span><span>getColVector</span><span>(</span><span>1</span><span>).</span><span>normalize</span><span>()</span></div></div><div><div><div>32</div></div><div><span> </span><span>vec3</span><span> </span><span>lambda2</span><span> </span><span>=</span><span> </span><span>eigenVectors</span><span>.</span><span>getColVector</span><span>(</span><span>2</span><span>).</span><span>normalize</span><span>()</span></div></div><div><div><div>33</div></div><div> </div></div><div><div><div>34</div></div><div><span><span> </span></span><span>// 获得 OBB 坐标系下的各轴向上的最大和最小值</span></div></div><div><div><div>35</div></div><div><span> </span><span>vec3</span><span> </span><span>min</span><span>,</span><span>max</span></div></div><div><div><div>36</div></div><div><span> </span><span>for</span><span> </span><span>p</span><span> </span><span>in</span><span> </span><span>samples</span><span>:</span></div></div><div><div><div>37</div></div><div><span> </span><span>vec3</span><span> </span><span>local</span><span> </span><span>=</span><span> </span><span>p</span><span> </span><span>-</span><span> </span><span>center</span></div></div><div><div><div>38</div></div><div><span> </span><span>local</span><span> </span><span>=</span><span> </span><span>vec3</span><span>(</span></div></div><div><div><div>39</div></div><div><span> </span><span>dot</span><span>(</span><span>local</span><span>, </span><span>lambda0</span><span>),</span></div></div><div><div><div>40</div></div><div><span> </span><span>dot</span><span>(</span><span>local</span><span>, </span><span>lambda1</span><span>),</span></div></div><div><div><div>41</div></div><div><span> </span><span>dot</span><span>(</span><span>local</span><span>, </span><span>lambda2</span><span>)</span></div></div><div><div><div>42</div></div><div><span><span> </span></span><span>)</span></div></div><div><div><div>43</div></div><div> </div></div><div><div><div>44</div></div><div><span> </span><span>min</span><span>.</span><span>x</span><span> </span><span>=</span><span> </span><span>min</span><span>(</span><span>local</span><span>.</span><span>x</span><span>, </span><span>min</span><span>.</span><span>x</span><span>)</span></div></div><div><div><div>45</div></div><div><span> </span><span>min</span><span>.</span><span>y</span><span> </span><span>=</span><span> </span><span>min</span><span>(</span><span>local</span><span>.</span><span>y</span><span>, </span><span>min</span><span>.</span><span>y</span><span>)</span></div></div><div><div><div>46</div></div><div><span> </span><span>min</span><span>.</span><span>z</span><span> </span><span>=</span><span> </span><span>min</span><span>(</span><span>local</span><span>.</span><span>z</span><span>, </span><span>min</span><span>.</span><span>z</span><span>)</span></div></div><div><div><div>47</div></div><div> </div></div><div><div><div>48</div></div><div><span> </span><span>max</span><span>.</span><span>x</span><span> </span><span>=</span><span> </span><span>max</span><span>(</span><span>local</span><span>.</span><span>x</span><span>, </span><span>max</span><span>.</span><span>x</span><span>)</span></div></div><div><div><div>49</div></div><div><span> </span><span>max</span><span>.</span><span>y</span><span> </span><span>=</span><span> </span><span>max</span><span>(</span><span>local</span><span>.</span><span>y</span><span>, </span><span>max</span><span>.</span><span>y</span><span>)</span></div></div><div><div><div>50</div></div><div><span> </span><span>max</span><span>.</span><span>z</span><span> </span><span>=</span><span> </span><span>max</span><span>(</span><span>local</span><span>.</span><span>z</span><span>, </span><span>max</span><span>.</span><span>z</span><span>)</span></div></div><div><div><div>51</div></div><div> </div></div><div><div><div>52</div></div><div><span> </span><span>vec3</span><span> </span><span>localCenter</span><span> </span><span>=</span><span> (</span><span>min</span><span> </span><span>+</span><span> </span><span>max</span><span>) </span><span>*</span><span> </span><span>0.5</span></div></div><div><div><div>53</div></div><div><span> </span><span>vec3</span><span> </span><span>localExtent</span><span> </span><span>=</span><span> (</span><span>max</span><span> </span><span>-</span><span> </span><span>min</span><span>) </span><span>*</span><span> </span><span>0.5</span></div></div><div><div><div>54</div></div><div> </div></div><div><div><div>55</div></div><div><span><span> </span></span><span>// 变换到世界坐标系下</span></div></div><div><div><div>56</div></div><div><span> </span><span>vec3</span><span> </span><span>worldCenter</span><span> </span><span>=</span><span> </span><span>center</span><span> </span><span>+</span><span> </span><span>lambda0</span><span> </span><span>*</span><span> </span><span>localCenter</span><span>.</span><span>x</span><span> </span><span>+</span><span> </span><span>lambda1</span><span> </span><span>*</span><span> </span><span>localCenter</span><span>.</span><span>y</span><span> </span><span>+</span><span> </span><span>lambda2</span><span> </span><span>*</span><span> </span><span>localCenter</span><span>.</span><span>z</span></div></div><div><div><div>57</div></div><div><span> </span><span>x</span><span> </span><span>=</span><span> </span><span>lambda0</span><span> </span><span>*</span><span> </span><span>localExtent</span><span>.</span><span>x</span></div></div><div><div><div>58</div></div><div><span> </span><span>y</span><span> </span><span>=</span><span> </span><span>lambda1</span><span> </span><span>*</span><span> </span><span>localExtent</span><span>.</span><span>y</span></div></div><div><div><div>59</div></div><div><span> </span><span>z</span><span> </span><span>=</span><span> </span><span>lambda2</span><span> </span><span>*</span><span> </span><span>localExtent</span><span>.</span><span>z</span></div></div><div><div><div>60</div></div><div> </div></div><div><div><div>61</div></div><div><span><span> </span></span><span>...</span></div></div><div><div><div>62</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div></section> <section><h2>其他<a href="#其他"><span>#</span></a></h2><p>我的目的是通过 PCA 方法找到 OBB 的基，到这一步我已经达成目标了。但如果是使用 PCA 做数据降维，需要在上述取得特征向量后，将原始数据转换到以特征向量为基的数据空间内。假设我们要取 N 维数据，那么需要将特征向量按照其特征值的大小降序排列为矩阵，取前 N 行构成矩阵 P。那么，对于原数据 X 降至 N 维后的数据便是：</p><span><span><span>Y=PXY = PX</span><span><span><span></span><span>Y</span><span></span><span>=</span><span></span></span><span><span></span><span>P</span><span>X</span></span></span></span></span></section> <section><h2>总结<a href="#总结"><span>#</span></a></h2><p>PCA 本质上是将方差最大的方向作为数据的主要特征（基），从几何方面看，它能够选择数据最离散的方向构建无关的正交的向量，正是因为这点，我们可以使用该算法得到 OBB。而在数理统计方面，它可以作为一种简化数据集的方法来处理复杂数据，比如用在人脸识别方面。</p><p>但 PCA 假定主成分分布在正交的方向上，对于生成 OBB 没什么问题，但对于在非正交方向上方差较大的数据集，效果会很有限。</p></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://zh.wikipedia.org/wiki/%E5%8C%85%E5%9B%B4%E4%BD%93" target="_blank">包围体</a></li> <li><a href="https://zh.wikipedia.org/wiki/%E4%B8%BB%E6%88%90%E5%88%86%E5%88%86%E6%9E%90" target="_blank">主成分分析</a></li> <li><a href="https://en.wikipedia.org/wiki/Covariance_matrix" target="_blank">Covariance matrix</a></li> <li><a href="http://blog.codinglabs.org/articles/pca-tutorial.html" target="_blank">PCA的数学原理</a></li> <li><a href="https://zhuanlan.zhihu.com/p/523291781" target="_blank">【引擎研发】OBB生成与可视化</a></li> <li><a href="https://blog.csdn.net/qing101hua/article/details/53100112" target="_blank">如何生成OBB（OrientedboundingBox）方向包围盒</a></li> <li><a href="https://zh.wikipedia.org/wiki/%E9%9B%85%E5%8F%AF%E6%AF%94%E6%B3%95" target="_blank">雅可比法</a></li> <li><a href="https://hewjunwei.wordpress.com/2013/01/26/obb-generation-via-principal-component-analysis/" target="_blank">OBB generation via Principal Component Analysis</a></li> <li><a href="https://zhuanlan.zhihu.com/p/32412043" target="_blank">PCA主成分分析学习总结</a></li> </ul></section>https://yohiro.games/posts/shaderassembly/https://yohiro.games/posts/shaderassembly/Mon, 17 Oct 2022 00:00:00 GMT<section><h2>前言<a href="#前言"><span>#</span></a></h2><p>在尝试逆向工程时，通常会在截帧的 GPU 信息中看到着色器的汇编代码。为了更好理解 GPU 的工作原理，如何加载数据、选择分支等，同时更好进行 Shader 调试，就个人而言有必要对 shader 汇编有一定的了解。</p><p>这篇文章的原文来自<a href="https://interplayoflight.wordpress.com/2021/04/18/how-to-read-shader-assembly" target="_blank">这里</a>，原文基于 AMD GCN 架构下的着色器汇编代码来进行了简单拆解。了解后，可以将该方法应用到其他 GPU 架构的汇编上。</p><p>本文的各种指令在 <a href="https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/instinct-mi200-cdna2-instruction-set-architecture.pdf" target="_blank">“AMD Instinct MI200” Instruction Set Architecture</a> 中有更准确的描述。</p><p>本篇使用的工具是<a href="https://shader-playground.timjones.io/" target="_blank">Shader Playground</a></p></section> <section><h2>背景<a href="#背景"><span>#</span></a></h2><p>现代 Shader 会由像是 GLSL、HLSL 这类着色器语言来编写。编写完成后，会使用 fxc 或是 dxc 这种工具将着色器代码转换成与 GPU 无关的形式，这里称作中间语言。然后 GPU 的驱动会将该<strong>中间语言</strong> (IL, Intermediate Language) 转换为在特定 GPU 上执行的<strong>着色器汇编</strong>（ISA）。这里，我们更侧重于关注着色器汇编，因为它更能代表实际执行的代码。</p><p>比如下面是一个简单的将输入的像素乘以一个数的 Shader：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>struct</span><span> PSInput</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>float4</span><span><span> color : </span><span>COLOR</span><span>;</span></span></div></div><div><div><div>4</div></div><div><span>};</span></div></div><div><div><div>5</div></div><div> </div></div><div><div><div>6</div></div><div><span>cbuffer</span><span> cbData</span></div></div><div><div><div>7</div></div><div><span>{</span></div></div><div><div><div>8</div></div><div><span> </span><span>float4</span><span> data;</span></div></div><div><div><div>9</div></div><div><span>}</span></div></div><div><div><div>10</div></div><div> </div></div><div><div><div>11</div></div><div><span>float4</span><span> </span><span>PSMain</span><span>(PSInput input) : SV_TARGET</span></div></div><div><div><div>12</div></div><div><span>{</span></div></div><div><div><div>13</div></div><div><span> </span><span>return</span><span> data * input.color;</span></div></div><div><div><div>14</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>它的中间代码：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>il_ps_2_55</span></div></div><div><div><div>2</div></div><div><span>dcl_global_flags refactoringAllowed</span></div></div><div><div><div>3</div></div><div><span>dcl_cb cb0[1]</span></div></div><div><div><div>4</div></div><div><span>dcl_input_generic_interp(linear) v0</span></div></div><div><div><div>5</div></div><div><span>dcl_output_generic o0</span></div></div><div><div><div>6</div></div><div><span>mul_ieee r4096, v0, cb0[0]</span></div></div><div><div><div>7</div></div><div><span>mov o0, r4096</span></div></div><div><div><div>8</div></div><div><span>ret_dyn</span></div></div><div><div><div>9</div></div><div><span>end</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>AMD GCN 架构下的着色器汇编:</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>s_mov_b32 m0, s8</span></div></div><div><div><div>2</div></div><div><span>s_buffer_load_dwordx4 s[0:3], s[4:7], 0x00</span></div></div><div><div><div>3</div></div><div><span>v_interp_p1_f32 v2, v0, attr0.x</span></div></div><div><div><div>4</div></div><div><span>v_interp_p2_f32 v2, v1, attr0.x</span></div></div><div><div><div>5</div></div><div><span>v_interp_p1_f32 v3, v0, attr0.y</span></div></div><div><div><div>6</div></div><div><span>v_interp_p2_f32 v3, v1, attr0.y</span></div></div><div><div><div>7</div></div><div><span>v_interp_p1_f32 v4, v0, attr0.z</span></div></div><div><div><div>8</div></div><div><span>v_interp_p2_f32 v4, v1, attr0.z</span></div></div><div><div><div>9</div></div><div><span>v_interp_p1_f32 v0, v0, attr0.w</span></div></div><div><div><div>10</div></div><div><span>v_interp_p2_f32 v0, v1, attr0.w</span></div></div><div><div><div>11</div></div><div><span>s_waitcnt lgkmcnt(0)</span></div></div><div><div><div>12</div></div><div><span>v_mul_f32 v1, s0, v2</span></div></div><div><div><div>13</div></div><div><span>v_mul_f32 v2, s1, v3</span></div></div><div><div><div>14</div></div><div><span>v_mul_f32 v3, s2, v4</span></div></div><div><div><div>15</div></div><div><span>v_mul_f32 v0, s3, v0</span></div></div><div><div><div>16</div></div><div><span>v_cvt_pkrtz_f16_f32 v1, v1, v2</span></div></div><div><div><div>17</div></div><div><span>v_cvt_pkrtz_f16_f32 v0, v3, v0</span></div></div><div><div><div>18</div></div><div><span>exp mrt0, v1, v1, v0, v0 done compr vm</span></div></div><div><div><div>19</div></div><div><span>s_endpgm</span></div></div><div><div><div>20</div></div><div><span>end</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>这段 Shader 代码过于简单了，没有代表性，让我们稍微添加一点东西。</p><p>考虑下面的 Shader 代码，该代码中包含了插值、采样、读取 buffer、基础运算、分支执行的操作：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>struct</span><span> PSInput</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>float2</span><span><span> uv : </span><span>TEXCOORD</span><span>;</span></span></div></div><div><div><div>4</div></div><div><span>};</span></div></div><div><div><div>5</div></div><div> </div></div><div><div><div>6</div></div><div><span>cbuffer</span><span> cbData</span></div></div><div><div><div>7</div></div><div><span>{</span></div></div><div><div><div>8</div></div><div><span> </span><span>float4</span><span> data;</span></div></div><div><div><div>9</div></div><div><span>}</span></div></div><div><div><div>10</div></div><div> </div></div><div><div><div>11</div></div><div><span>Texture2D</span><span>&lt;</span><span>float4</span><span>&gt; tex;</span></div></div><div><div><div>12</div></div><div><span>SamplerState</span><span> samplerLinear;</span></div></div><div><div><div>13</div></div><div> </div></div><div><div><div>14</div></div><div><span>float4</span><span> </span><span>PSMain</span><span>(PSInput input) : SV_TARGET</span></div></div><div><div><div>15</div></div><div><span>{</span></div></div><div><div><div>16</div></div><div><span> </span><span>float4</span><span> result = tex.</span><span>Sample</span><span>(samplerLinear, input.uv);</span></div></div><div><div><div>17</div></div><div> </div></div><div><div><div>18</div></div><div><span> </span><span>float</span><span> factor = data.x * data.y;</span></div></div><div><div><div>19</div></div><div> </div></div><div><div><div>20</div></div><div><span> </span><span>if</span><span>( factor &gt; </span><span>0</span><span> )</span></div></div><div><div><div>21</div></div><div><span> </span><span>return</span><span> data.z * result;</span></div></div><div><div><div>22</div></div><div><span> </span><span>else</span></div></div><div><div><div>23</div></div><div><span> </span><span>return</span><span> data.w * result;</span></div></div><div><div><div>24</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>下面的是上面的 Shader 生成的汇编：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>s_mov_b32 m0, s20</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>s_mov_b64 s[22:23], exec</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>s_wqm_b64 exec, exec</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>v_interp_p1_f32 v2, v0, attr0.x</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>v_interp_p2_f32 v2, v1, attr0.x</span></div></div><div><div><div>6</div></div><div><span><span> </span></span><span>v_interp_p1_f32 v3, v0, attr0.y</span></div></div><div><div><div>7</div></div><div><span><span> </span></span><span>v_interp_p2_f32 v3, v1, attr0.y</span></div></div><div><div><div>8</div></div><div><span><span> </span></span><span>s_and_b64 exec, exec, s[22:23]</span></div></div><div><div><div>9</div></div><div><span><span> </span></span><span>image_sample v[0:3], v[2:4], s[4:11], s[12:15] dmask:0xf</span></div></div><div><div><div>10</div></div><div><span><span> </span></span><span>s_buffer_load_dwordx4 s[0:3], s[16:19], 0x00</span></div></div><div><div><div>11</div></div><div><span><span> </span></span><span>s_waitcnt lgkmcnt(0)</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>v_mov_b32 v4, s1</span></div></div><div><div><div>13</div></div><div><span><span> </span></span><span>v_mul_f32 v4, s0, v4</span></div></div><div><div><div>14</div></div><div><span><span> </span></span><span>v_cmp_lt_f32 vcc, 0, v4</span></div></div><div><div><div>15</div></div><div><span><span> </span></span><span>s_cbranch_vccz label_0017</span></div></div><div><div><div>16</div></div><div><span><span> </span></span><span>s_waitcnt vmcnt(0)</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>v_mul_f32 v0, s2, v0</span></div></div><div><div><div>18</div></div><div><span><span> </span></span><span>v_mul_f32 v1, s2, v1</span></div></div><div><div><div>19</div></div><div><span><span> </span></span><span>v_mul_f32 v2, s2, v2</span></div></div><div><div><div>20</div></div><div><span><span> </span></span><span>v_mul_f32 v3, s2, v3</span></div></div><div><div><div>21</div></div><div><span><span> </span></span><span>s_branch label_001C</span></div></div><div><div><div>22</div></div><div><span>label_0017:</span></div></div><div><div><div>23</div></div><div><span><span> </span></span><span>s_waitcnt vmcnt(0)</span></div></div><div><div><div>24</div></div><div><span><span> </span></span><span>v_mul_f32 v0, s3, v0</span></div></div><div><div><div>25</div></div><div><span><span> </span></span><span>v_mul_f32 v1, s3, v1</span></div></div><div><div><div>26</div></div><div><span><span> </span></span><span>v_mul_f32 v2, s3, v2</span></div></div><div><div><div>27</div></div><div><span><span> </span></span><span>v_mul_f32 v3, s3, v3</span></div></div><div><div><div>28</div></div><div><span>label_001C:</span></div></div><div><div><div>29</div></div><div><span><span> </span></span><span>s_mov_b64 exec, s[22:23]</span></div></div><div><div><div>30</div></div><div><span><span> </span></span><span>v_cvt_pkrtz_f16_f32 v0, v0, v1</span></div></div><div><div><div>31</div></div><div><span><span> </span></span><span>v_cvt_pkrtz_f16_f32 v1, v2, v3</span></div></div><div><div><div>32</div></div><div><span><span> </span></span><span>exp mrt0, v0, v0, v1, v1 done compr vm</span></div></div><div><div><div>33</div></div><div><span><span> </span></span><span>s_endpgm</span></div></div><div><div><div>34</div></div><div><span>end</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>直接看可能看不懂，现在忽略一些必要代码，看一些比较明显的（由于 markdown 不好渲染，所以用了 diff 的形式呈现）：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float4 result = tex.Sample(samplerLinear, input.uv);</span></div></div><div><div><div>2</div></div><div><span>image_sample v[0:3], v[2:4], s[4:11], s[12:15] dmask:0xf</span></div></div><div><div><div>3</div></div><div> </div></div><div><div><div>4</div></div><div><span>float factor = data.x * data.y;</span></div></div><div><div><div>5</div></div><div><span>v_mov_b32 v4, s1</span></div></div><div><div><div>6</div></div><div><span>v_mul_f32 v4, s0, v4</span></div></div><div><div><div>7</div></div><div> </div></div><div><div><div>8</div></div><div><span>if( factor &gt; 0 )</span></div></div><div><div><div>9</div></div><div><span>v_cmp_lt_f32 vcc, 0, v4</span></div></div><div><div><div>10</div></div><div><span>s_cbranch_vccz label_0017</span></div></div><div><div><div>11</div></div><div> </div></div><div><div><div>12</div></div><div><span>data.z * result;</span></div></div><div><div><div>13</div></div><div><span>v_mul_f32 v0, s2, v0</span></div></div><div><div><div>14</div></div><div><span>v_mul_f32 v1, s2, v1</span></div></div><div><div><div>15</div></div><div><span>v_mul_f32 v2, s2, v2</span></div></div><div><div><div>16</div></div><div><span>v_mul_f32 v3, s2, v3</span></div></div><div><div><div>17</div></div><div> </div></div><div><div><div>18</div></div><div><span>data.w * result;</span></div></div><div><div><div>19</div></div><div><span>v_mul_f32 v0, s3, v0</span></div></div><div><div><div>20</div></div><div><span>v_mul_f32 v1, s3, v1</span></div></div><div><div><div>21</div></div><div><span>v_mul_f32 v2, s3, v2</span></div></div><div><div><div>22</div></div><div><span>v_mul_f32 v3, s3, v3ji</span></div></div><div><div><div>23</div></div><div> </div></div><div><div><div>24</div></div><div><span>return</span></div></div><div><div><div>25</div></div><div><span>exp mrt0, v0, v0, v1, v1 done compr vm</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>注意到几乎所有的指令都以 <strong>v_</strong> 或 <strong>s_</strong> 开头，比如 <strong>v_mov_b32</strong>，<strong>s_mov_b32</strong>。这里的 v 代表矢量 vector，s 代表标量 scalar。</p><p>根据 <a href="https://www.techpowerup.com/gpu-specs/docs/amd-gcn1-architecture.pdf" target="_blank">GCN 架构白皮书</a>中的描述，汇编指令会被计算单元（CU）加载为称作 wavefront 的工作项，这个所谓的工作项在不同的架构下的名称有所不同，但是可以笼统地当作处理像素的线程。它会并行地执行着色器指令。</p><p>GCN 架构的 CU 可以解码七种指令类型：</p><ul> <li>分支指令</li> <li>标量 ALU 或内存指令</li> <li>矢量 ALU 指令</li> <li>矢量内存指令</li> <li>本地数据共享指令</li> <li>全局内存或导出指令</li> <li>其他特殊指令</li> </ul><p>标量指令有两条 pipeline，一是用于处理分支和控制终端，二是在标量数据缓存中读取数据。每个 SIMD 内部有矢量 pipeline 用于加速浮点数的计算。</p><p>举例来说，当 GPU 从 Constant Buffer 中读取数据时，将会发出标量指令。当需要将一个像素的颜色乘一个值时，将会发出矢量指令。</p><p>在着色器汇编代码中，指令的右侧是指令的寄存器，寄存器在存储着色器指令在操作中使用到的数据。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>v_mul_f32 v0, s3, v0</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>该指令意为将每个 wavefront 的 v0 矢量寄存器中的值与 s3 标量寄存器中的值相乘，并将结果写入到矢量寄存器 v0 中。注意到上面的汇编中，也有形为 <code>s[22:23]</code> 的寄存器，意为寄存器范围，可用来存储更多数据。</p><p>在 AMD 的 GCN 指令集架构中，有多种不同的寄存器类型，具体信息可以直接访问<a href="https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/gcn3-instruction-set-architecture.pdf" target="_blank">该文档</a>。</p><p>在上面的指令中，同时也指定了数据类型。v_mul_<strong>f32</strong>表明该指令操作 32 位的浮点数，v_mov_<strong>b32</strong> 表明在寄存器加载 32 位无类型数值，v_cvt_pkrtz<strong>_f16_f32</strong> 表明是将 32 位的浮点数转为半精度的 16 位浮点数。</p><p>有了以上的背景知识后，可以尝试解读上面的着色器汇编代码了。</p></section> <section><h2>解读<a href="#解读"><span>#</span></a></h2><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>s_mov_b32 m0, s20</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>s_mov_b64 s[22:23], exec</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>s_wqm_b64 exec, exec</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>着色器代码正式执行前会设置数据地址。上面的着色器代码会在后续的操作中进行插值，因此会先将每个顶点的 UV 坐标的本地数据存储的偏移量从标量寄存器 S20 加载到 M0 寄存器中。</p><p>然后，<strong>exec</strong> 寄存器中的内容的副本放入了标量寄存器 s22，s23 中。关于 exec 寄存器，文档中的介绍如下：</p> <table><thead><tr><th>Abbrev.</th><th>Name</th><th>Size</th><th>Description</th></tr></thead><tbody><tr><td>EXEC</td><td>Execute Mask</td><td>64 bits</td><td>A bit mask with one bit per thread, which is applied to vector instructions and controls that threads execute and that ignore the instruction</td></tr></tbody></table><p>也就是说，exec 寄存器是一个掩码，每个 wavefront 线程占用一位，该掩码用来描述线程的执行状态。</p><p>由于该寄存器可用于所有 wavefront 因此它是个标量指令以 s 开头。</p><p>最后执行 <strong>s_wqm_b64</strong> 指令，它会检查掩码中每组的四个比特位，如果该组的四个位中任意的一位设置为一，那么该组的四个比特位都会设为一。在使用 Piexl Shader 渲染时，PS 始终以 2x2 的 quad 为单位来进行渲染。因此，该指令会用于确认哪些 quad 是处于激活状态。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>v_interp_p1_f32 v2, v0, attr0.x</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>v_interp_p2_f32 v2, v1, attr0.x</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>v_interp_p1_f32 v3, v0, attr0.y</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>v_interp_p2_f32 v3, v1, attr0.y</span></div></div></code></pre><div><div></div><div></div></div></figure></div><blockquote><p>Pixel shaders use LDS to read vertex parameter values; the pixel shader then interpolates them to find the per-pixel parameter values.</p><p>PS 使用 LDS 读取顶点参数值，然后通过对它们执行插值来找到各像素对应的值。</p></blockquote><p>接下来的部分使用 M0 寄存器中的偏移，对 Vertex Shader 提供的 uv 坐标插值。由于每个像素有自己的独立的 uv 值，因此使用矢量指令。注意到每个 uv 分量的插值都要使用两个指令。这两个指令执行时，GPU 会读取三角形顶点传入的参数中的 uv 值并进行插值。</p><p>这也说明在 GCN 架构的 GPU 中插值是发起生在着色器，而非使用专门的硬件单元执行。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>s_and_b64 exec, exec, s[22:23]</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>该命令相当于执行了一个按位与的操作，如果 wavefront 的某个线程没有激活则会跳过该线程。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>image_sample v[0:3], v[2:4], s[4:11], s[12:15] dmask:0xf</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p><strong>image_sample</strong> 是最基础的图片采样指令。sample 代表该指令可以通过 SamplerState 来指定纹理过滤方式。寄存器 v[0&lt;3&gt;] 代表存放结果的 4 个寄存器 v0 到 v3；v[2&lt;4&gt;] 包含插值的 uv 坐标 v2，v3；标量寄存器 s[4&lt;11&gt;] 表示用于存放纹理描述符的 8 个寄存器，该描述符通常是指向的纹理内存的地址；最后一个标量寄存器 s[12&lt;15&gt;] 表示纹理过滤器对象的描述符的 4 个寄存器，也就是 SamplerState 的内存地址。dmask 是一个 4 位的数据掩码，指定纹理采样时应处理的通道的数量，此处 0xf 表示要读取所有 4 个通道。即使没有 v 开头的前缀，image_sample 也是一个矢量指令。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>s_buffer_load_dwordx4 s[0:3], s[16:19], 0x00</span></div></div><div><div><div>2</div></div><div><span>s_waitcnt lgkmcnt(0)</span></div></div><div><div><div>3</div></div><div><span>v_mov_b32 v4, s1</span></div></div><div><div><div>4</div></div><div><span>v_mul_f32 v4, s0, v4</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>接下来从 Constant Buffer 中读取常量数据 Data。Constant Buffer 读取数据是所有 wavefront 的通用标量操作，因此使用标量指令 <strong>s_buffer_load_dwordx4</strong>。该指令会从 Constant Buffer 中读取四个 dword，第一个标量范围指定存储加载读取结果的四个标量寄存器 s0 到 s3，存储 fp32 的值；第二个范围指定 Constant Buffer 的描述符，也就是内存地址。</p><p>由于内存加载存在延迟，从 s_buffer_load 执行到返回数据给后续指令 <strong>v_mov_b32</strong> 之间会已经经过了若干个时钟周期，因此编译器在两个指令之间添加了 <strong>s_waitcnt</strong> 指令，等待所有的加载操作完成，参数 <strong>lgkmcnt</strong> 占 4 bits 表示等待的 LDS/GDS/Constant/Message，这里我们在等待 Constant Buffer 的数据的加载完成。完成加载后，接下来的移动指令 v_mov_b32 将寄存器 s1 中的数据加载到 v4 寄存器中，然后调用 v_mul_f32 将 v4 中的值直接与寄存器 s0 中的值相乘，并将结果保存到 v4 中。</p><p>上面的汇编表示在 GCN 架构中，标量的乘法必须将其中一个值移到矢量寄存器中。但并非所有的数据类型都是如此，如果 Constant Buffer 中的 Data 的数据类型由 float4 改为 uint4，那么这里的 v_mul_f32 就可以由 s_mul_i32 来完成。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>v_cmp_lt_f32 vcc, 0, v4</span></div></div><div><div><div>2</div></div><div><span>s_cbranch_vccz label_0017</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>接下来执行分支判断指令 <strong>v_cmp_lt_f32</strong>，LT 意为 <strong>L</strong>ess <strong>T</strong>han 判断 0 是否小于寄存器 v4 中的值 ，<strong>vcc</strong> 是一个 64 位的掩码，每个线程一个位，用于存储矢量比较的结果，值为 1 代表通过该比较，值为 0 代表未通过该比较。此处比较虽然是矢量的，也即每个相乘使用不同的值去比较，但 v4 中的值对于各个线程都是相同的。<strong>s_cbranch_vccz</strong> 指令会检查 VCC 的比较结果是否为零，为零意味着比较失败，跳转到 label 0017 处执行，否则则为 NOP，也就是 no operation 不执行任何操作。</p><p>另外，比较指令是矢量的，但是实际的分支指令是标量，也就是对所有的线程都是相同的操作。在 GCN 架构中，所有类型的分支指令都是如此，它们由标量单元进行处理。这种情况下，分支指令对 wavefront 中的每个线程就是要么全都执行要么全都不执行，也就是 factor.x 要么都小于 0，要么都大于等于 0，因为编译器是知道该值是来自 Costant Buffer 的标量，对所有线程来说都是相同的，因此对同一个 wavefront 中不存在分歧。</p><p>假如将比较的值改为每个线程不同的值：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float4</span><span> </span><span>PSMain</span><span>(PSInput input) : SV_TARGET</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>float4</span><span> result = tex.</span><span>Sample</span><span>(samplerLinear, input.uv);</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span> </span><span>// 使用从纹理中采样得到的值，该值对每个像素或者说线程都是不同的</span></div></div><div><div><div>6</div></div><div><span> </span><span>if</span><span>( result.x &gt; </span><span>0</span><span> )</span></div></div><div><div><div>7</div></div><div><span> </span><span>return</span><span> data.z * result;</span></div></div><div><div><div>8</div></div><div><span> </span><span>else</span></div></div><div><div><div>9</div></div><div><span> </span><span>return</span><span> data.w * result;</span></div></div><div><div><div>10</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>那么相应的 ISA 会变为：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>image_sample v[0:3], v[2:4], s[4:11], s[12:15] dmask:0xf</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>s_mov_b64 s[0:1], exec</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>s_waitcnt vmcnt(0)</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>v_cmpx_gt_f32 s[2:3], v0, 0</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>s_cbranch_execz label_0017</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>可以看到此时会使用 exec 中的掩码来确定哪些线程正在活动，并使用 <strong>v_cmpx_gt_f32</strong> 判断 v0 是否大于 0，比较失败的会返回 NOP，然后执行指令 <strong>s_cbranch_execz</strong>，该指令也是个分支判断的指令，与 s_cbranch_vccz 不同的是，该指令读取 exec 寄存器中的值来判断比较结果。</p><p>回到原先的 ISA 中，接下来有：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>s_waitcnt vmcnt(0)</span></div></div><div><div><div>2</div></div><div><span>v_mul_f32 v0, s2, v0</span></div></div><div><div><div>3</div></div><div><span>v_mul_f32 v1, s2, v1</span></div></div><div><div><div>4</div></div><div><span>v_mul_f32 v2, s2, v2</span></div></div><div><div><div>5</div></div><div><span>v_mul_f32 v3, s2, v3</span></div></div><div><div><div>6</div></div><div><span>s_branch label_001C</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>这是 if 条件为 true 的分支，也即 factor 大于 0 的分支。这里会将纹理采样的结果乘上来自 Constant Buffer 的常量值 data.z。当需要纹理采样值时，GPU 需要确保该结果读取完成，否则会有 undefined 行为。因此，编译器在这里添加了另一个等待指令 <strong>s_waitcnt</strong>，该指令后的 <strong>vmcnt</strong> 与上文提到的 lgkmcnt 不同，vmcnt 表示等待矢量内存返回。</p><p>这段代码最后，执行 <strong>s_branch</strong> 直接跳转到 label_001C 位置。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>label_0017:</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>s_waitcnt vmcnt(0)</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>v_mul_f32 v0, s3, v0</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>v_mul_f32 v1, s3, v1</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>v_mul_f32 v2, s3, v2</span></div></div><div><div><div>6</div></div><div><span><span> </span></span><span>v_mul_f32 v3, s3, v3</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>这是 else 分支所执行的代码，内容是将纹理采样的结果乘 data.w。同样，这里会使用等待指令以确保该内存读取结果可用。从这两段代码可以看出 v_mul_f32 可以将矢量与标量相乘。</p><p>一个 float4 的乘法运算对应了 ISA 中的四条指令，这和 GCN 的架构设计有关，</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>label_001C:</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>s_mov_b64 exec, s[22:23]</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>v_cvt_pkrtz_f16_f32 v0, v0, v1</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>v_cvt_pkrtz_f16_f32 v1, v2, v3</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>exp mrt0, v0, v0, v1, v1 done compr vm</span></div></div><div><div><div>6</div></div><div><span>s_endpgm</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>在示例的 ISA 的最后一部分，先是将之前存储在 s22 和 s23 中的掩码值转移到 exec 中，以确保已激活的线程能够正常取得计算结果。</p><p><strong>v_cvt_pkrtz_f16_f32</strong> 指令将两个 f32 的值转为 f16 并 pack 到一个 f32 中。对于 v0/v1 和 v2/v3 该指令各执行了一次，最终在 v0 和 v1 中保存了 float4 类型的计算结果。通过这种方式压缩了数据长度节省了带宽。</p><p><strong>exp</strong> 指令意为 export，对于 Pixel Shader，该指令会将计算的结果输出到绑定的 render target 上。由于 Pixel Shader 必须输出至少一个 Color/Depth/Null 的 Target，因此所有的 Pixel Shader 都包含一个 export 指令。这里的参数 <strong>mrt0</strong> 代表支持 mrt，我们的输出目标是多个 render target 中的第一个。紧接着的几个寄存器，done 表示这是该 Shader 的最后的输出，compr 代表数据为压缩的格式，vm 代表可以使用 exec 的掩码来告知 color buffer 中哪些像素是有效或无效。</p><p>最后的 <strong>s_endpgm</strong> 指令表示终止该 wavefront 的执行。到这里该 Shader 便执行完毕了。</p></section> <section><h2>最后<a href="#最后"><span>#</span></a></h2><p>上面解读的 Shader 的样例可能太简单，并不实用，但是通过这个过程可以更好理解 GPU 如何执行从而提高对 shading language 的敏感程度。比如，将上面示例中的代码稍微改一下，只使用一个 return 语句，它们的代码和 ISA 会变成：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float4</span><span> </span><span>PSMain</span><span>(PSInput input) : SV_TARGET</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>float4</span><span> result = tex.</span><span>Sample</span><span>(samplerLinear, input.uv);</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span> factor = data.x * data.y;</span></div></div><div><div><div>6</div></div><div> </div></div><div><div><div>7</div></div><div><span> </span><span>if</span><span>( factor &gt; </span><span>0</span><span> )</span></div></div><div><div><div>8</div></div><div><span><span> </span></span><span>result *= data.z;</span></div></div><div><div><div>9</div></div><div><span> </span><span>else</span></div></div><div><div><div>10</div></div><div><span><span> </span></span><span>result *= data.w;</span></div></div><div><div><div>11</div></div><div> </div></div><div><div><div>12</div></div><div><span> </span><span>return</span><span> result;</span></div></div><div><div><div>13</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>s_mov_b32 m0, s20</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>s_mov_b64 s[22:23], exec</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>s_wqm_b64 exec, exec</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>v_interp_p1_f32 v2, v0, attr0.x</span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>v_interp_p1_f32 v3, v0, attr0.y</span></div></div><div><div><div>6</div></div><div><span><span> </span></span><span>v_interp_p2_f32 v2, v1, attr0.x</span></div></div><div><div><div>7</div></div><div><span><span> </span></span><span>v_interp_p2_f32 v3, v1, attr0.y</span></div></div><div><div><div>8</div></div><div><span><span> </span></span><span>s_and_b64 exec, exec, s[22:23]</span></div></div><div><div><div>9</div></div><div><span><span> </span></span><span>image_sample v[0:3], v[2:4], s[4:11], s[12:15] dmask:0xf</span></div></div><div><div><div>10</div></div><div><span><span> </span></span><span>s_buffer_load_dwordx4 s[0:3], s[16:19], 0x00</span></div></div><div><div><div>11</div></div><div><span><span> </span></span><span>s_waitcnt lgkmcnt(0)</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>v_mov_b32 v4, s1</span></div></div><div><div><div>13</div></div><div><span><span> </span></span><span>v_mul_f32 v4, s0, v4</span></div></div><div><div><div>14</div></div><div><span><span> </span></span><span>v_cmp_lt_f32 vcc, 0, v4</span></div></div><div><div><div>15</div></div><div><span><span> </span></span><span>s_waitcnt vmcnt(0)</span></div></div><div><div><div>16</div></div><div><span><span> </span></span><span>v_mul_f32 v4, s2, v0</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>v_mul_f32 v5, s2, v1</span></div></div><div><div><div>18</div></div><div><span><span> </span></span><span>v_mul_f32 v6, s2, v2</span></div></div><div><div><div>19</div></div><div><span><span> </span></span><span>v_mul_f32 v7, s2, v3</span></div></div><div><div><div>20</div></div><div><span><span> </span></span><span>v_mul_f32 v0, s3, v0</span></div></div><div><div><div>21</div></div><div><span><span> </span></span><span>v_mul_f32 v1, s3, v1</span></div></div><div><div><div>22</div></div><div><span><span> </span></span><span>v_mul_f32 v2, s3, v2</span></div></div><div><div><div>23</div></div><div><span><span> </span></span><span>v_mul_f32 v3, s3, v3</span></div></div><div><div><div>24</div></div><div><span><span> </span></span><span>v_cndmask_b32 v0, v0, v4, vcc</span></div></div><div><div><div>25</div></div><div><span><span> </span></span><span>v_cndmask_b32 v1, v1, v5, vcc</span></div></div><div><div><div>26</div></div><div><span><span> </span></span><span>v_cndmask_b32 v2, v2, v6, vcc</span></div></div><div><div><div>27</div></div><div><span><span> </span></span><span>v_cndmask_b32 v3, v3, v7, vcc</span></div></div><div><div><div>28</div></div><div><span><span> </span></span><span>s_mov_b64 exec, s[22:23]</span></div></div><div><div><div>29</div></div><div><span><span> </span></span><span>v_cvt_pkrtz_f16_f32 v0, v0, v1</span></div></div><div><div><div>30</div></div><div><span><span> </span></span><span>v_cvt_pkrtz_f16_f32 v1, v2, v3</span></div></div><div><div><div>31</div></div><div><span><span> </span></span><span>exp mrt0, v0, v0, v1, v1 done compr vm</span></div></div><div><div><div>32</div></div><div><span><span> </span></span><span>s_endpgm</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>与原本的示例相比，分支指令完全被移除了，取而代之的是四个 <strong>v_cndmask_b32</strong> 指令，它们根据存储在 vcc 中的 <strong>v_cmp_lt_f32</strong> 的比较结果来选择输出的值。</p><p>虽然这种写法消除了分支指令的影响，但也引入了更多的矢量寄存器，不过多数情况下，这也是一种优化。</p></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://interplayoflight.wordpress.com/2021/04/18/how-to-read-shader-assembly/" target="_blank">How to read shader assembly – Interplay of Light</a></li> <li><a href="https://www.humus.name/Articles/Persson_LowLevelThinking.pdf" target="_blank">Low-Level Thinking in High-Level Shading Language</a></li> <li><a href="https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/gcn3-instruction-set-architecture.pdf" target="_blank">R1200 ISA</a></li> <li><a href="https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/instinct-mi200-cdna2-instruction-set-architecture.pdf" target="_blank">“AMD Instinct MI200” Instruction Set Architecture</a></li> </ul></section>https://yohiro.games/posts/deltacolorcompression/https://yohiro.games/posts/deltacolorcompression/Sun, 21 Aug 2022 00:00:00 GMT<section><h2>问题<a href="#问题"><span>#</span></a></h2><p>前段时间遇到一个平台相关问题，问题是这样：</p><p>在 UE 里使用了 SceneCapture 处理的一个低分辨率的 RT，经处理后要做 CopyToResolveTarget，而在这之后作为 Src 的 SceneCaptureRT 没有被拉伸到目标 RT 的大小。最后怀疑了会不会是 DX12 的 CopyTextureRegion 的问题，于是去查了相关资料，最后在一个 2020 年的 release note 里找到了这条信息：</p><blockquote><p>…其中问题在于，当在源纹理上启用DCC时，CopyTextureRegion() 将忽略源矩形并将整个纹理复制到目标中的同一子矩形。</p></blockquote><p>可以看到 CopyTextureRegion 对于 DCC 的支持不是很完善。在相应的 RT 上添加 TexCreate_DisableDCC 的 Flag 后问题确实可以解决问题，但还是想看看 DCC 究竟干了什么，为什么禁止了这个 Flag 问题可以解决。</p></section> <section><h2>Delta Color Compression<a href="#delta-color-compression"><span>#</span></a></h2><blockquote><p>除解压缩外，以下信息仍然适用于 AMD RDNA/RDNA2 架构。 {: .prompt-warning }</p></blockquote><p>带宽是一种珍贵的资源，虽然硬件在不断进步，但是游戏渲染的分辨率越来越高，DataBuffer 越来越大，在读写 RenderTarget 时占用了很多带宽。引入 <strong>D</strong>elta <strong>C</strong>olor <strong>C</strong>ompression，就是为了缓解带宽的压力。</p><p>DCC 是一种针对特定区域的无损压缩，通过数据一致性来减少所需的带宽。如同传统的压缩器，只不过是用于 3D 图形渲染，DCC 的思想关键在于它是逐 block 处理而不是逐像素处理，在一个 block 之内只有一个值是全精度存储，其余所存储的是相对于全精度值的增量，如果同一个 block 内使用了近似的色彩，则增量值可以使用比输入少得多的位来存储。</p><p>DCC 的压缩器在 Color block 內部，它允许图形以类似于 DepthStencil target 的方式压缩 Color target。实际的硬件实现中会根据实际数据和访问方式来自动调节 block 的大小以优化潜在的随机访问。</p><p>通常，我们读取数据的频率比写入数据的频率高得多。为了充分发扬节省带宽这一优势，着色器核心被赋予了读取新的压缩颜色以及所有现有压缩表面的能力。这允许在 Render to texture 中完全跳过解压缩操作，在资源状态发生变换时，比如 Render target 到 Texture 的过程中，Barrier transition 是 NoOp，从而不触发昂贵的解压缩。</p><section><h3>注意事项<a href="#注意事项"><span>#</span></a></h3><p>像 EarlyZ 一样，存在一些应用场景会使该优化失效，下面列出了一些要注意的事项，更完整的可以阅读 <a href="https://gpuopen.com/learn/dcc-overview/" target="_blank">Getting the Most Out of Delta Color Compression</a>。</p><ul> <li> <p>如无必要，不要把 RenderTarget 标记为 Shader 可读的资源</p> <p>Shader 可读的资源，比如 SRV，通常没有很好的被压缩。硬件中的 Shader 单元可以直接压缩数据，但是有一些另外的压缩选项只有在由 Shader core 读取数据的行为被禁止的情况下才可以进行。这一点对 MSAA 的 Depth target 影响最大，如果它不作为 SRV，它会取得较高的压缩质量。</p> </li> <li> <p>试着用 32 bit 的浮点深度格式，而不是 16 bit</p> <p>当作为 Shader 资源时，D32Fs 可能会比 D16s 被压缩的更小。它们的不同在于被分配的大小和解压时的带宽占用，然而解压本身并不是一个非常频繁的操作，除非在屏幕上很小的区域中存在很密集的 Mesh，这些 Mesh 具有很细小的三角形。D32Fs 也允许使用 Reverse-z 来添加近裁切面的精度，忽略掉解压地场景，D32Fs 对比 D16Fs 几乎没有消耗。</p> <p>在 GCN 架构的 AMD GPU 上，不存在真正的 24 bit 的 Depth target，24 位的深度会被当作忽略了 8 位精度的 32 位的深度来处理，因此 24 位到 32 位的切换几乎没有额外消耗。</p> </li> <li> <p>尽量写所有的颜色通道</p> <p>对于非压缩的数据，执行各通道独立的写入没有问题。但对压缩的数据，数据需要先被读取，解压，更新，最后再写回原先的通道中。为了更有效地压缩，最好同时完全覆写所有通道。</p> </li> </ul><p>不过这些注意事项也不是必定要遵守的，还是应该根据工程的实际情况来进行判断。</p><p>由于目前不是所有的 GPU 单元都可以读写压缩数据，所以有一些场景仍会导致 DCC 失效，从而触发解压缩，其中针对 <a href="https://learn.microsoft.com/zh-cn/windows/win32/direct3d12/user-mode-heap-synchronization" target="_blank">Copy engine</a> 的有这么两条：</p><blockquote><ul> <li>Generally once flagged as a copy source or target, we decompress first because we don’t know where it is going.</li> <li>Raw copies can be done between surfaces of <strong>the same type and size</strong>… if the driver knows that this is going to happen.</li> </ul></blockquote><p>回到文章一开始所说的 bug，CopyTextureRegion 会始终把具有 DCC 标记的渲染资源当作与目标大小一致的 RenderTarget 来 resolve。这么做是可能为了确保 DCC 起效，因为如果启用了 DCC 的源纹理与目标纹理不一致，那么源纹理无法经由 DCC 压缩器解压。但是这样的话我认为接口本身就不应该提供源矩形大小的参数，所以最终可能是为了兼容不同平台上 D3D12 的接口才做了如此处理，最终导致了 DCC flag 有无造成了这样的 bug。</p></section></section>https://yohiro.games/posts/cloth/https://yohiro.games/posts/cloth/Fri, 11 Mar 2022 00:00:00 GMT<section><h2>背景<a href="#背景"><span>#</span></a></h2><p>这段时间做了些布料模拟相关的工作，实际内容是对 <a href="https://github.com/pafuhana1213/KawaiiPhysics" target="_blank">KawaiiPhysics</a> 做了一些二次开发，使它满足美术表现效果。为什么要对 KawaiiPhysics 做二次开发呢？</p><p>美术比较习惯使用代理网格进行模拟工作流程，但是 UE5 里的 Chaos Cloth 始终达不到他们想要的效果，而且 Chaos 的开销还很大。于是拿来了 KawaiiPhysics，但 KawaiiPhysics 的效果实际上是针对单根骨骼，对相邻的骨骼缺少约束，更适合做短裙，中短发还有乳摇的效果。而我们项目里的衣服更多是具有长下摆、广袖的样式，这种情况下使用 KawaiiPhysics 又使得布料显得太 Q 弹。</p><p>也就是这样，对 KawaiiPhysics 做了一些魔改，在这期间学习到了些关于物理，布料模拟的东西，现记录如下。</p></section> <section><h2>Mass-Spring 模型<a href="#mass-spring-模型"><span>#</span></a></h2><p>布料模拟使用了经典的 <a href="https://graphicsinterface.org/wp-content/uploads/gi1995-17.pdf" target="_blank">Mass-Spring</a> 模型，顾名思义，也就是将一块布料当作一张分布有许多质点，且质点间由弹簧互相连接构成的网。</p><section><h3>质点<a href="#质点"><span>#</span></a></h3><p>假设模拟的布料是一个均匀分布了 M × N 个质点的矩形，其中的每个质点具有自己的质量、位置以及速度。</p></section><section><h3>弹簧<a href="#弹簧"><span>#</span></a></h3><p>对该系统内的质点，根据质点间连接方式的不同，将材质上每个质点所受到的弹性力分解为以下三种弹簧，对应三种约束（实际上都是距离约束）：</p><ul> <li>质点上下左右四个方向相连接的，用于模拟结构约束（Structural）</li> <li>质点斜向对角相连接的，用于模拟剪切约束（Shear）</li> <li>质点上下左右跨一个质点相连接的，用于模拟弯曲约束（Bend）</li> </ul><p></p><figure><img src="/assets/images/Cloth/MassSpring.jpg" alt="图1" /><figcaption>图1</figcaption></figure> <em>图1</em><p></p><p>上述三种约束对应到材质上有下面抵抗形变的作用：</p><ul> <li>结构约束用于抵抗布料在垂直方向和水平方向的拉伸，结构约束越强越不易被拉伸</li> <li>剪切约束用于抵抗扭曲，越强就越不会被剪切力所扭曲，更能在质点附近维持原有形状</li> <li>弯曲约束用于抵抗弯曲，越强就不易弯曲，使得材质在视觉上显得质地更硬</li> </ul><p><img src="/assets/images/Cloth/VisualMassSpring.png" alt="" /> <em>三种力对应三种不同的形变</em></p><p>参考图 1，将各质点编码，对第 [i,j] 个质点，与他相连的其他的弹簧的位置为：</p> <table><thead><tr><th>约束类型</th><th>相邻质点</th></tr></thead><tbody><tr><td>结构约束</td><td>[i, j+1]，[i, j-1]，[i+1, j]，[i-1, j]</td></tr><tr><td>剪切约束</td><td>[i+1, j+1]，[i+1, j-1]，[i-1, j-1]，[i-1, j+1]</td></tr><tr><td>弯曲约束</td><td>[i, j+2]，[i, j-2]，[i+2, j]，[i-2, j]</td></tr></tbody></table><p>可见对于单个质点而言，它需要周围 12 个质点的信息。</p><p>对于弹簧而言还有两个值得注意的变量，刚度（Stiffness）以及原始长度（Rest Length）。</p><p>上述的三种约束可以对应各自独立的刚度。对于我们常见的布料材质，通常剪切约束较弱，剪切约束的刚度对材质的视觉效果具有很大影响。此外，结构约束的刚度通常很大，而剪切约束和弯曲约束的刚度都很小^{<a href="#user-content-fn-tip">1</a>}。但这并不意味着这三种约束是相互独立的，比如剪切约束也可以分解为水平方向和垂直方向的约束，可见剪切约束也会一定程度上影响材质的拉伸。</p><p>而原始长度 Rest length 用于求解约束，计算某时刻两质点间的相对距离与原始长度 Rest length 的差值，从而确定约束力的大小，其距离约束可以这么表达：</p><span><span><span>Cdist(p0,p1)=∣p0−p1∣−drestC_{dist}(p_0, p_1) = |p_0-p_1| - d_{rest}</span><span><span><span></span><span><span>C</span><span><span><span><span><span><span></span><span><span><span>d</span><span>i</span><span>s</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span><span>p</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>,</span><span></span><span><span>p</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span>∣</span><span><span>p</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>−</span><span></span></span><span><span></span><span><span>p</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>∣</span><span></span><span>−</span><span></span></span><span><span></span><span><span>d</span><span><span><span><span><span><span></span><span><span><span>r</span><span>es</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span><p>上述计算弹力并求解约束的过程大致可由代码表达如下：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>void</span><span> </span><span>SolveConstraint</span><span>(</span><span>Point</span><span>&amp;</span><span><span> </span><span>P0</span><span>, </span></span><span>Point</span><span>&amp;</span><span><span> </span><span>P1</span><span>, </span></span><span>const</span><span> </span><span>float&amp;</span><span><span> </span><span>Stiffness</span><span>, </span></span><span>const</span><span> </span><span>float&amp;</span><span><span> </span><span>RestLength</span><span>)</span></span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>Vec3</span><span> </span><span>Vector</span><span> </span><span>=</span><span> </span><span>P1</span><span>.</span><span>Position</span><span> </span><span>-</span><span> </span><span>P0</span><span>.</span><span>Position</span><span>;</span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span> </span><span>Distance</span><span> </span><span>=</span><span> </span><span>Vector</span><span>.</span><span>Size</span><span>();</span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span> </span><span>Force</span><span> </span><span>=</span><span> </span><span>Stiffness</span><span> </span><span>*</span><span> (</span><span>Distance</span><span> </span><span>-</span><span> </span><span>RestLength</span><span>);</span><span> // Hooke's Law</span></div></div><div><div><div>6</div></div><div> </div></div><div><div><div>7</div></div><div><span> </span><span>Vec3</span><span> </span><span>Displacement</span><span> </span><span>=</span><span> </span><span>Vector</span><span>.</span><span>Normalize</span><span>() </span><span>*</span><span> </span><span>Force</span><span>;</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span> </span><span>float</span><span> </span><span>MassSum</span><span> </span><span>=</span><span> </span><span>P0</span><span>.</span><span>Mass</span><span> </span><span>+</span><span> </span><span>P1</span><span>.</span><span>Mass</span><span>;</span></div></div><div><div><div>10</div></div><div><span> </span><span>float</span><span> </span><span>M1Ratio</span><span> </span><span>=</span><span> </span><span>P1</span><span>.</span><span>Mass</span><span> </span><span>/</span><span> </span><span>MassSum</span><span>;</span></div></div><div><div><div>11</div></div><div> </div></div><div><div><div>12</div></div><div><span> </span><span>P0</span><span>.</span><span>Position</span><span> </span><span>+=</span><span> </span><span>Displacement</span><span> </span><span>*</span><span> </span><span>M1Ratio</span><span>;</span></div></div><div><div><div>13</div></div><div><span> </span><span>P1</span><span>.</span><span>Position</span><span> </span><span>-=</span><span> </span><span>Displacement</span><span> </span><span>*</span><span> (</span><span><span>1.0</span><span>f</span></span><span> </span><span>-</span><span> </span><span>M1Ratio</span><span>);</span></div></div><div><div><div>14</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>将质点所受到的弹性力记为 <span><span>FsF_s</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>，<span><span>FsF_s</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 可和其他外力一同用于计算质点的加速度。</p><p><span><span>FsF_s</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 满足<a href="https://zh.wikipedia.org/wiki/%E8%83%A1%E5%85%8B%E5%AE%9A%E5%BE%8B" target="_blank">胡克定律</a>，即对弹性系数为 <span><span>ksk_s</span><span><span><span></span><span><span>k</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 的材料，有 <span><span>Fs=−ks ΔxF_s = -k_s\, \Delta x</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>−</span><span><span>k</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>Δ</span><span>x</span></span></span></span></p></section><section><h3>其他<a href="#其他"><span>#</span></a></h3><p>阻尼（Damping） : 如果没有阻尼的限制，弹簧振子的将会无止境地弹下去，因此引入阻尼来模拟弹簧弹性渐弱，能量流失的过程。对材质阻尼系数为 <span><span>cdc_d</span><span><span><span></span><span><span>c</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 速度为 <span><span>vv</span><span><span><span></span><span>v</span></span></span></span> 的质点，阻尼力 <span><span>FdF_d</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 记为:</p><span><span><span>Fd=−cd vF_d = -c_d\,v</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>−</span><span><span>c</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>v</span></span></span></span></span><p>重力 : 重力通常作为一个全局常量而存在，不过也不排除部分游戏中有以重力为核心的玩法，在这里我们将重力记为 <span><span>FgF_g</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>g</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>。</p><p>其他外力 : 至于其他的外力，比较常见的是风力，这些外力可统一记作 <span><span>FeF_e</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>e</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>。</p></section><section><h3>模拟<a href="#模拟"><span>#</span></a></h3><p>对基于动力学的模拟方式，大致流程如下：</p><p>对于合力 <span><span>FF</span><span><span><span></span><span>F</span></span></span></span>，有</p><span><span><span>F=Fs+Fd+Fg+FeF = F_s + F_d + F_g + F_e</span><span><span><span></span><span>F</span><span></span><span>=</span><span></span></span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span></span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span></span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>g</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span></span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>e</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span><p>知道了质点的质量 <span><span>mm</span><span><span><span></span><span>m</span></span></span></span>，根据牛二，可以获得加速度 <span><span>aa</span><span><span><span></span><span>a</span></span></span></span></p><span><span><span>a=Fm=Fs+Fd+Fg+Fema = \frac{F}{m} = \frac{F_s + F_d + F_g + F_e}{m}</span><span><span><span></span><span>a</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>m</span></span></span><span><span></span><span></span></span><span><span></span><span><span>F</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>m</span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>F</span><span><span><span><span><span><span></span><span><span>s</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>g</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>e</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>有了加速度便可以根据 <span><span>tt</span><span><span><span></span><span>t</span></span></span></span> 时刻的速度和位置计算出 <span><span>t+1t+1</span><span><span><span></span><span>t</span><span></span><span>+</span><span></span></span><span><span></span><span>1</span></span></span></span> 时刻的速度和位置：</p><span><span><span>vt+1=vt+a tv_{t+1} = v_t + a\,t</span><span><span><span></span><span><span>v</span><span><span><span><span><span><span></span><span><span><span>t</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span>v</span><span><span><span><span><span><span></span><span><span>t</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span></span><span><span></span><span>a</span><span></span><span>t</span></span></span></span></span><span><span><span>xt+1=xt+vt+1 tx_{t+1} = x_t + v_{t+1}\,t</span><span><span><span></span><span><span>x</span><span><span><span><span><span><span></span><span><span><span>t</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span>x</span><span><span><span><span><span><span></span><span><span>t</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span></span><span><span></span><span><span>v</span><span><span><span><span><span><span></span><span><span><span>t</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>t</span></span></span></span></span><p>如此迭代，在计算位置和速度时应用上面提到的约束便可以完成基础的模拟逻辑了。</p></section></section> <section><h2>碰撞<a href="#碰撞"><span>#</span></a></h2><p>由于布料存在于场景中必然会与其他对象发生交互，这就带来了另一个让人抠脑壳的问题，也就是布料与其他对象的碰撞。当严格考虑现实世界的布料时，碰撞将会变得非常复杂，因为还要考虑在布料碰撞的过程中产生的折叠、褶皱等，因此这里暂不深入到这种细节，将着重于对<strong>角色-布料</strong>和<strong>布料-布料</strong>这两种碰撞的处理。</p><section><h3>布料-角色<a href="#布料-角色"><span>#</span></a></h3><p>对于布料-角色的碰撞，通常的做法是用一系列简单的形状拼合成人形当作角色的代理，但是简单碰撞的形状有限，碰撞凸包的形状会对布料的造型产生影响，弄不好的话会在碰撞代理的连接出产生很难看的葫芦形凹缝。</p><p>下面对马岛中所使用的人形碰撞：</p> <table><thead><tr><th>对马岛中的碰撞凸包示意</th><th>运行时</th></tr></thead><tbody><tr><td><figure><img src="/assets/images/Cloth/proxy.png" alt="PhysicalAsset" /><figcaption>PhysicalAsset</figcaption></figure></td><td><figure><img src="/assets/images/Cloth/TsushimaInGame.png" alt="PhysicalAssetInGame" /><figcaption>PhysicalAssetInGame</figcaption></figure></td></tr></tbody></table><p>在对布料进行建模时，在每个质点上会设定一个质点的半径来限制可运动的最大距离。通过参考质点半径以及碰撞的凸包的半径来确定如何校正质点的位置。这个校正过程可以简单的抽象为：检测碰撞→碰撞发生→根据穿深进行反方向校正，大多数布料发生碰撞的过程都遵循这个过程，从而避免穿模。</p><p><img src="/assets/images/Cloth/Chain.png" alt="" /> <em>可以看作粗粒度的布料的模型，球形的即是质点</em></p><p>根据穿深进行校正时有两种解算方式：基于物理和基于位置，区别在于是反方向施加力/冲量，还是施加位移。对于对于模拟的时间步长比较敏感，很容易出现在大时间步时弹出一个巨大的力的问题。因此游戏中布料的碰撞校正通常选择后者，而后者的做法就是 <strong>P</strong>osition <strong>B</strong>ased <strong>D</strong>ynamic 的思想，不强调施加力的过程，只要能根据最终施加的位移结果能够达成视觉上的正确，那我们就认为它是正确的。</p></section><section><h3>布料-布料<a href="#布料-布料"><span>#</span></a></h3><p>布料-布料碰撞可以分为两种，一种是布料材质与其自身的自碰撞，另一种是使用多层布料时，不同层的布料间的碰撞。</p><p>对布料的自碰撞，一个方法是与上次模拟的位置以质点半径做球形的 Sweep 检测来确定，确保质点间的碰撞距离保始终满足 <span><span>dcollision=min(2r,drest)d_{collision} = min(2r, d_{rest})</span><span><span><span></span><span><span>d</span><span><span><span><span><span><span></span><span><span><span>co</span><span>l</span><span>l</span><span>i</span><span>s</span><span>i</span><span>o</span><span>n</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>min</span><span>(</span><span>2</span><span>r</span><span>,</span><span></span><span><span>d</span><span><span><span><span><span><span></span><span><span><span>r</span><span>es</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span></span></span></span>，还有限制最大速度等。更多关于布料自碰撞的建议可以参考十分钟物理的<a href="https://www.youtube.com/watch?v=XY3dLpgOk4Q" target="_blank">这个视频</a>。</p><p>对于多层布料的碰撞，可以根据布料质点来构建数个 Quad 然后依据点和三角形的碰撞来处理不同层布料间的碰撞，确保外层布料的点落在内层布料形成的 Quads 外侧。但是这种做法稳定性一般，对于细窄小的飘带有可能会产生很鬼畜的抖动。</p><p>或者参考对马岛的方案，将外层布料参考内层布料做蒙皮，为外层布料添加一个值为质点半径的偏移：</p><p></p><figure><img src="/assets/images/Cloth/Layering.png" alt="Layering Cloth" /><figcaption>Layering Cloth</figcaption></figure> <em>对马岛中的多层布料</em><p></p><p>这种做法更像是让多层布料分别做球形的碰撞，再让外层布料向内移动一段距离，从而使衣服不会显得太臃肿。</p></section></section> <section><h2>其他补充<a href="#其他补充"><span>#</span></a></h2><section><h3>时间步长<a href="#时间步长"><span>#</span></a></h3><p>由于模拟的基础是弹簧，所以不可避免的存在拉伸。对于皮革、绳索这类较硬、并不需要太多拉伸的材质，效果并不是很好。时间步长较短时，会使得材料显得很硬，时间步长较大时，模拟的效果不稳定。</p><p>搜集了一番资料，我看到两种比较主流的方法：</p><p>一种是尝试使用其他的数值积分方法， 比如 <a href="https://www.techbrood.com/zh/news/webgl/%E7%B2%92%E5%AD%90%E8%BF%90%E5%8A%A8%E6%A8%A1%E6%8B%9F---verlet%E7%A7%AF%E5%88%86%E7%AE%97%E6%B3%95%E7%AE%80%E4%BB%8B.html" target="_blank">Verlet 积分方法</a>。 Verlet 积分法是在游戏中用得比较多的用于求解牛顿运动方程的数值方法。Verlet 积分方法要解决的问题是，给定 <span><span>tt</span><span><span><span></span><span>t</span></span></span></span> 时刻的位置 <span><span>xx</span><span><span><span></span><span>x</span></span></span></span> 和速度 <span><span>vv</span><span><span><span></span><span>v</span></span></span></span>，得到 <span><span>t+Δtt+\Delta t</span><span><span><span></span><span>t</span><span></span><span>+</span><span></span></span><span><span></span><span>Δ</span><span>t</span></span></span></span> 时刻的位置 <span><span>x(t+Δt)x(t+\Delta t)</span><span><span><span></span><span>x</span><span>(</span><span>t</span><span></span><span>+</span><span></span></span><span><span></span><span>Δ</span><span>t</span><span>)</span></span></span></span> 和速度 <span><span>v(t+Δt)v(t+\Delta t)</span><span><span><span></span><span>v</span><span>(</span><span>t</span><span></span><span>+</span><span></span></span><span><span></span><span>Δ</span><span>t</span><span>)</span></span></span></span>。简单来说就是，Verlet 积分方法通过将 <span><span>v(t)v(t)</span><span><span><span></span><span>v</span><span>(</span><span>t</span><span>)</span></span></span></span> 以及 <span><span>x(t)x(t)</span><span><span><span></span><span>x</span><span>(</span><span>t</span><span>)</span></span></span></span> 做泰勒展开，然后获得新的迭代式：</p><span><span><span>x(t+Δt)=2 x(t)+x(t−Δt)+a(t) Δt2x(t+\Delta t) = 2\,x(t) + x(t-\Delta t) + a(t)\, \Delta t^2</span><span><span><span></span><span>x</span><span>(</span><span>t</span><span></span><span>+</span><span></span></span><span><span></span><span>Δ</span><span>t</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span>2</span><span></span><span>x</span><span>(</span><span>t</span><span>)</span><span></span><span>+</span><span></span></span><span><span></span><span>x</span><span>(</span><span>t</span><span></span><span>−</span><span></span></span><span><span></span><span>Δ</span><span>t</span><span>)</span><span></span><span>+</span><span></span></span><span><span></span><span>a</span><span>(</span><span>t</span><span>)</span><span></span><span>Δ</span><span><span>t</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span><span><span><span>v(t)=x(t+Δt)−x(t−Δt)2Δtv(t) = \frac{x(t+\Delta t)-x(t-\Delta t)}{2\Delta t}</span><span><span><span></span><span>v</span><span>(</span><span>t</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>2Δ</span><span>t</span></span></span><span><span></span><span></span></span><span><span></span><span><span>x</span><span>(</span><span>t</span><span></span><span>+</span><span></span><span>Δ</span><span>t</span><span>)</span><span></span><span>−</span><span></span><span>x</span><span>(</span><span>t</span><span></span><span>−</span><span></span><span>Δ</span><span>t</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>具体推导以及误差计算可以参考<a href="https://zh.wikipedia.org/wiki/%E9%9F%A6%E5%B0%94%E8%8E%B1%E7%A7%AF%E5%88%86%E6%B3%95" target="_blank">维基百科</a>。</p><p>另一种方法，便是使用 <strong>SupStep</strong>，即令时间步长划分为多个固定的子步长进行模拟。这个方法会引入额外的计算，但是也加大了模拟的准确度以及稳定性。</p><p>我添加了使用 SubStep 的方法，确实稳定性大大提高，但是否使用这种方法还需要根据游戏实际运行状况进行评估。</p></section><section><h3>PBD / XPBD<a href="#pbd--xpbd"><span>#</span></a></h3><p>之前在提到布料-角色碰撞时提到了 PBD（Position Based Dynamic），PBD 是一种基于位置来解碰撞问题的方式，区别于通常基于力的方式。</p><p>在 KawaiiPhysics 中操作的单位是骨骼，它是一个基于关节（Joint）的模型，而在它的模型之上使用物理学的方法添加碰撞、约束自然会变得麻烦，因此单纯使用位置来约束碰撞是更合适的方式。</p><p>PBD 最大的优势是稳定，发生碰撞时碰撞体不会向外飘。PBD 的核心理念见下图：</p><p></p><figure><img src="/assets/images/Cloth/PBD.png" alt="PBD overview" /><figcaption>PBD overview</figcaption></figure><p></p><p>根据上一次模拟的速度和时间计算新的位置，然后解算新的位置投影到约束上的点，最后根据投影到约束上点的位置来更新速度。想了解更多的话，这里有<a href="https://www.cnblogs.com/daiday/p/8037378.html" target="_blank">中译论文</a>，也可以看<a href="https://matthias-research.github.io/pages/publications/posBasedDyn.pdf" target="_blank">原版</a>。还有份更详细的资料在<a href="https://mmacklin.com/EG2015PBD.pdf" target="_blank">这里</a>。</p><p><a href="https://matthias-research.github.io/pages/publications/XPBD.pdf" target="_blank">XPBD</a> 则在 PBD 的基础上解决刚度与迭代次数耦合的问题。关于 XPBD 有一个<a href="https://matthias-research.github.io/pages/tenMinutePhysics/09-xpbd.pdf" target="_blank">简单的介绍</a>，作者总结了依据梯度求解约束的通用式子，作者油管频道的其他视频和文章也值得一看，毕竟作者就是这两篇论文的作者。</p><p>在对马岛的 GDC talk 中老哥提到一个 “giant hack”，就是对于弹力，完全<em>舍弃掉胡克定律</em>，在每次模拟时使弹簧向原始长度靠拢一点点，我觉得也是利用了 PBD 的思想去做计算。</p></section></section> <section><h2>参考资料<a href="#参考资料"><span>#</span></a></h2><ul> <li><a href="https://ics.uci.edu/~shz/courses/cs114/docs/proj3/index.html" target="_blank">CS114 Project 3 Simulation using Mass-Spring System</a></li> <li><a href="https://gdcvault.com/play/1027350/Blowing-from-the-West-Simulating" target="_blank">Blowing from the West: Simulating Wind in ‘Ghost of Tsushima’</a></li> <li><a href="https://gdcvault.com/play/1025083/Cloth-Self-Collision-with-Predictive" target="_blank">Cloth Self Collision with Predictive Contacts</a></li> <li><a href="https://matthias-research.github.io/pages/tenMinutePhysics/index.html" target="_blank">Ten Minute Physics</a></li> <li><a href="https://carmencincotti.com/2022-07-11/position-based-dynamics/" target="_blank">Position Based Dynamics</a></li> <li><a href="https://www.cnblogs.com/FlyingZiming/p/17293782.html" target="_blank">游戏模拟——Position based dynamics</a></li> </ul></section> <section><h2>Footnotes<a href="#footnote-label"><span>#</span></a></h2> <ol> <li> <p>(Synthesis Lectures on Visual Computing Computer Graphics, Animation, Computational Photography, and Imaging, #24) Tuur Stuyck_ Brian A. Barsky - Cloth Simulation for Computer Graphics-Morgan &amp; Clayp <a href="#user-content-fnref-tip">↩</a></p> </li> </ol> </section>https://yohiro.games/posts/scene-color-format/https://yohiro.games/posts/scene-color-format/Tue, 28 Sep 2021 00:00:00 GMT<section><h2>背景<a href="#背景"><span>#</span></a></h2><p>在目前帮忙的项目里，为了实现一个效果，长期在屏幕空间存在两个 Renderer，一个普通的 SceneRenderer 还有一个 SceneCaptureRenderer。</p><p>针对 SceneCaptureRenderer 做了一些裁剪的优化，目前看来没有什么问题，可是它呈现的内容少，我想让 SceneCaptureRenderer 的 SceneColorFormat 跟原本的 SceneRenderer 的 SceneColorFormat 区分开来。</p><p>当然不能直接改 SceneCapture 使用的 RT 格式，因为引擎的绘制过程中这两个 Renderer 用的是同一个 SceneColor，直接改会每帧都更新 RHI 不断地触发 Reallocating。</p><p>那就多加一个 RT 吧，可以用半分变率，内存也吃得消。于是开始调整这两个不同的 SceneColor 的格式，格式影响最大的一个是带宽的消耗，另一个是 Alpha 位数带来的问题。</p></section> <section><h2>带宽消耗<a href="#带宽消耗"><span>#</span></a></h2><p>用 Snapdragon Profiler 得到了这样的带宽消耗数据：</p> <table><thead><tr><th>SceneColorFormat</th><th>FP16</th><th>RGB111110</th><th>RGBA1010102</th><th>BGRA8888</th></tr></thead><tbody><tr><td>Bandwidth(R/W mega beats)</td><td>12</td><td>8</td><td>6</td><td>6</td></tr></tbody></table><p>基本上符合预期，在 Adreno 文档的 <a href="https://developer.qualcomm.com/sites/default/files/docs/adreno-gpu/snapdragon-game-toolkit/gdg/gpu/faq.html#what-is-the-performance-of-1010102-vs-111110-formats" target="_blank"><strong>FAQ</strong></a> 里也提到过：</p><blockquote><p><strong>What is the performance of 1010102 vs. 111110 formats?</strong></p><p>Both formats will perform better than FP16. There is a hardware “fast path” for 1010102 which will allow it to perform slightly better than 111110.</p></blockquote><p>这符合得到的数据，即 <code>BGRA8888 = RGBA1010102 &gt; RGB111110 &gt;&gt; FP16</code>。</p></section> <section><h2>后处理与 Alpha<a href="#后处理与-alpha"><span>#</span></a></h2><section><h3>Alpha<a href="#alpha"><span>#</span></a></h3><p>由于移动平台受限的带宽问题，大多数情况下使用 RGB111110 作为 SceneColorFormat。但是这个格式没有 Alpha 通道，而 UE4 的移动 Forward 管线中又将深度写到了 Alpha 里，导致在 Resolve 后在后处理 Pass 里拿不到深度。具体代码在 <em>SceneTexturesCommon.ush</em> 中：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>/** Returns DeviceZ which is the z value stored in the depth buffer. */</span></div></div><div><div><div>2</div></div><div><span>float</span><span> </span><span>LookupDeviceZ</span><span><span>(</span><span> float2 </span><span>ScreenUV</span><span> </span><span>)</span></span></div></div><div><div><div>3</div></div><div><span>{</span></div></div><div><div><div>4</div></div><div><span> </span><span>// ...</span></div></div><div><div><div>5</div></div><div> </div></div><div><div><div>6</div></div><div><span> </span><span>#if</span><span> </span><span>MOBILE_DEFERRED_SHADING</span></div></div><div><div><div>7</div></div><div><span> </span><span>return</span><span> </span><span>Texture2DSample</span><span>(</span><span>MobileSceneTextures</span><span>.</span><span>SceneDepthTexture</span><span>, </span><span>MobileSceneTextures</span><span>.</span><span>SceneDepthTextureSampler</span><span>, ScreenUV).</span><span>r</span><span>;</span></div></div><div><div><div>8</div></div><div><span> </span><span>#else</span></div></div><div><div><div>9</div></div><div><span> </span><span>// SceneDepth texture is not accessible during post-processing as we discard it at the end of mobile BasePass</span></div></div><div><div><div>10</div></div><div><span> </span><span>// instead fetch DeviceZ from SceneColor.A</span></div></div><div><div><div>11</div></div><div><span> </span><span>return</span><span> </span><span>Texture2DSample</span><span>(</span><span>MobileSceneTextures</span><span>.</span><span>SceneColorTexture</span><span>, </span><span>MobileSceneTextures</span><span>.</span><span>SceneColorTextureSampler</span><span>, ScreenUV).</span><span>a</span><span>;</span></div></div><div><div><div>12</div></div><div><span> </span><span>#endif</span></div></div><div><div><div>13</div></div><div> </div></div><div><div><div>14</div></div><div><span> </span><span>// ...</span></div></div><div><div><div>15</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>这就是为什么会有 bKeepDepthContent，通过 bKeepDepthContent 可以在后处理中访问丢掉的深度信息，从而解决没有 SceneColor.A 的问题。或者加一个 SceneDepthCopy 的 RT 用来拷贝深度，这样比较灵活。</p></section><section><h3>后处理<a href="#后处理"><span>#</span></a></h3><p>在 MobileHDR 的管线下，在处理折射效果，也就是 Distortion Pass 时也是放到后处理中的，从而可以直接使用后处理 Resolve 好的 SceneColor。</p><p>然后在渲染半透明的时候，还要在 MobileBasePass 里将 BlendState 设为 RGBA，否则半透明通道无法正确写入信息。</p></section></section>https://yohiro.games/posts/tiledmemoryandmemoryless/https://yohiro.games/posts/tiledmemoryandmemoryless/Tue, 04 May 2021 00:00:00 GMT<section><h2>背景<a href="#背景"><span>#</span></a></h2><p>移动端设备通常使用 <a href="https://zh.wikipedia.org/wiki/%E5%9F%BA%E4%BA%8E%E5%9B%BE%E5%9D%97%E6%B8%B2%E6%9F%93" target="_blank">Tiled Rendering</a>，与桌面 / Console 端 GPU 的立即渲染模式不同，Tiled Rendering 会将 Frame buffer 划分为多个矩形片，每个矩形片称为一个 Tile，以此来减少带宽的消耗。</p> <table><thead><tr><th>Immediate Rendering</th><th>Tiled Rendering</th></tr></thead><tbody><tr><td><figure><img src="https://developer.samsung.com/sd2_images/game/tech_GPUFramebuffer_06.gif" alt="IMR" /><figcaption>IMR</figcaption></figure></td><td><figure><img src="https://developer.samsung.com/sd2_images/game/tech_GPUFramebuffer_07.gif" alt="TBR" /><figcaption>TBR</figcaption></figure></td></tr><tr><td><figure><img src="https://community.arm.com/resized-image/__size/1040x0/__key/communityserver-blogs-components-weblogfiles/00-00-00-20-66/4403.model_2D00_imr.png" alt="IMR-DataFlow" /><figcaption>IMR-DataFlow</figcaption></figure></td><td><figure><img src="https://community.arm.com/resized-image/__size/1040x0/__key/communityserver-blogs-components-weblogfiles/00-00-00-20-66/6560.model_2D00_tbr.png" alt="TBR-DataFlow" /><figcaption>TBR-DataFlow</figcaption></figure></td></tr></tbody></table><p>表格左边是使用 linear cacheline 的立即模式的绘制过程。渲染时，图像上方显示内存中四条连续的 cacheline。每个 cacheline 上方都有一个长条矩形，用于指示与 cacheline 对应的像素在 frame buffer 中的位置：红色表示 ache line 已写入但 cache miss，绿色表示 cache 命中，较亮的颜色表示最近访问过的 cacheline。与 miss 的 cacheline 对应的 frame buffer 像素以紫色（Frame buffer）和白色（Depth buffer）显示。</p><p>可见，立即模式的渲染可能会以不可预测的方式访问内存，因为具体的顺序取决于三角形是如何提交给 GPU 的。</p><p>表格右边是使用 square cache tile 的 Tiled rendering 的绘制过程，上面的四个 cacheline 分别覆盖 frame buffer 和 depth buffer 中的一块矩形区域，然后紧接着是缓存的 frame buffer 区域和缓存的 depth buffer 区域。这里的 cacheline 所保存的像素数与使用 linear cache 的立即模式中相同。</p><p>在空间上接近的三角形，在提交的时间上一般也会比较接近，因此这种方式更好对缓存区域分组，从而提高缓存命中率。而且每个 tile 中会绘制更多的内容，相比立即模式，内存传输的频率有所降低，从而达到了减少带宽消耗的目的。可见，相较于立即模式，tiled rendering 拥有更好的<strong>局部性</strong>。</p></section> <section><h2>Tiled Memory<a href="#tiled-memory"><span>#</span></a></h2><p>简单来说，Tiled Memory 是 GPU 芯片上的内存，与通用显存（系统内存）相比，具有带宽较高、延迟较低、功耗较小的特点。</p><blockquote><p>Image layout transitions with VK_IMAGE_LAYOUT_UNDEFINED allow the implementation to discard the image subresource range, which can provide performance or power benefits. Tile-based architectures may be able to avoid flushing tile data to memory, and immediate style renderers may be able to achieve fast metadata clears to reinitialize frame buffer compression state, or similar.</p></blockquote><p>移动 GPU 一般都有 Tile Memory。 ， （还有一种与 Tile Memory 特性相反的系统内存。在移动设备中，CPU 和 GPU 通常共享此（物理）系统内存。）</p><p>Metal 或 Vulkan 可以显式声明一个 RenderPass 来决定是否将 Tile 内存移动到系统内存（如果下一个渲染过程中不需要 Tile 内存中某个渲染过程的输出，那么显式地声明您可以想要丢弃 tile 内存中的数据（Memory Less，不要将 tile 内存中的数据移动到显存）或在下一个渲染通道中按原样使用 tile 内存中的数据而不将其移动到显存（。您还可以节省内存带宽。）</p><p>然而，在 OpenGL ES 的情况下，没有 RenderPass 的概念，所以如果你想丢弃 Tile 内存中的数据而不将其移动到显存，可以使用 <code>glClear</code> 函数来通知 GPU 驱动程序这一点。但由于 OpenGL ES Spec 中没有有关 Tile 内存的信息，因此在调用 <code>glClear</code> 函数时是否执行此操作似乎因 GPU 驱动程序而异。</p><p>另外，在 MSAA 的情况下，存储像素值需要的内存是样本数量的 N 倍，但是 tile 内存的大小大于 MSAA 所需的内存大小，因此如果对 MSAA 采样数据可以存储在 tile 存储器中， tile 存储器存储 MSAA 的采样数据之后，可以在 tile 存储器中执行 MSAA 解析。</p><p>因此，使用 MSAA 时和不使用 MSAA 时，内存带宽使用量是相同的（通过在 tile 内存上执行 MSAA Resolve，从 tile 内存传输到显存的数据大小与使用 MSAA 时相同。因为 Tile 内存非常接近 GPU 核心，因此数据传输和访问速度比显存快得多。</p><p>Mali GPU 支持称为<a href="https://community.arm.com/arm-community-blogs/b/graphics-gaming-and-vr-blog/posts/pixel-local-storage-on-arm-mali-gpus" target="_blank">“像素本地存储”</a>的功能，这意味着在着色器阶段声明数据存储在 Tile Memory 中。声明为像素本地存储的类型的数据在渲染过程中仅存在于 tile 内存中，并且被丢弃而不是复制到显存（系统内存），从而防止不必要的内存带宽浪费。移动 GPU 通过基于 tile 的渲染对场景中的所有图元进行顶点着色，然后收集每个 tile （将渲染目标划分为一定尺寸（基于 Mali GPU 的 16x16）的 tile ）上要绘制的图元。在为要绘制的每个 tile 收集场景中（在几何工作集中）的所有原始三角形后，仅对屏幕上可见的三角形执行片段着色（丢弃被遮挡的片段）（隐藏表面移除）。 在移动 GPU 上，按如下方式完成：一次对场景中的所有三角形执行顶点着色，为每个 tile 收集数据，然后一次为每个 tile 执行片段着色。</p><p>这里，当对每个 tile 执行片段着色时，等于 tile 大小的渲染目标被存储在 tile 存储器中。 Pixel Local Storage 是程序员直接定义要存储在这个 Tile Memory 中的数据的地方，在移动 GPU 上执行延迟渲染时，如果在 G-buffer pass 和 Resolve pass 中使用 Pixel Local Storage，则存储 G-buffer。作为显存（系统内存），您可以通过将其存储在 Tile Memory 中并在 Resolve pass 中立即读取来节省内存带宽。此外，存储在 Tile Memory 中的 Pixel Local Storage 数据不会转移到显存（只是被丢弃），因此这里也可以减少内存使用量。</p></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://developer.qualcomm.com/sites/default/files/docs/adreno-gpu/snapdragon-game-toolkit/gdg/gpu/overview.html#tile-rendering" target="_blank">Adreno Game Developer Guides - Overview</a></li> <li><a href="https://community.arm.com/arm-community-blogs/b/graphics-gaming-and-vr-blog/posts/pixel-local-storage-on-arm-mali-gpus" target="_blank">Pixel Local Storage on ARM Mali GPUs</a></li> <li><a href="https://developer.samsung.com/galaxy-gamedev/resources/articles/gpu-framebuffer.html#Immediate-mode-rasterizers" target="_blank">GPU Framebuffer Memory: Understanding Tiling</a></li> <li><a href="https://developer.apple.com/documentation/metal/resource_fundamentals/choosing_a_resource_storage_mode_for_apple_gpus" target="_blank">Choosing a Resource Storage Mode for Apple GPUs</a></li> <li><a href="https://developer.apple.com/documentation/metal/mtlstoragemode/memoryless" target="_blank">MTLStorageMode.memoryless</a></li> </ul></section>https://yohiro.games/posts/uniform-buffer-alignment/https://yohiro.games/posts/uniform-buffer-alignment/Tue, 01 Dec 2020 00:00:00 GMT<section><h2>背景<a href="#背景"><span>#</span></a></h2><p>最近 UE4.21 的项目中遇到了一个 Bug，具体问题是这样的：</p><p>玩家可以蹲伏在草丛中进入隐匿状态，进入隐匿状态后，草丛有一个不透明度降低的效果。问题出在这种效果在有些平台存在，有些平台不存在。</p><p>后来经过抓帧、调试这样的一番探索，发现问题出在 uniform buffer 里。</p></section> <section><h2>问题<a href="#问题"><span>#</span></a></h2><p>真正的问题是，在某个 uniform buffer 之后，从 NSight 中观测到的数据出现了偏移。</p><p>UE4里的 uniform buffer 定义在 SceneView 里：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>// View uniform buffer member declarations</span></div></div><div><div><div>2</div></div><div><span>#define</span><span> </span><span>VIEW_UNIFORM_BUFFER_MEMBER_TABLE</span><span> </span><span>\</span></div></div><div><div><div>3</div></div><div><span> </span><span>VIEW_UNIFORM_BUFFER_MEMBER</span><span>(</span><span>FMatrix</span><span>, </span><span>TranslatedWorldToClip</span><span>) </span><span>\</span></div></div><div><div><div>4</div></div><div><span> </span><span>VIEW_UNIFORM_BUFFER_MEMBER</span><span>(</span><span>FMatrix</span><span>, </span><span>WorldToClip</span><span>) </span><span>\</span></div></div><div><div><div>5</div></div><div> </div></div><div><div><div>6</div></div><div><span><span> </span></span><span>....</span></div></div><div><div><div>7</div></div><div> </div></div><div><div><div>8</div></div><div><span> </span><span>VIEW_UNIFORM_BUFFER_MEMBER</span><span>(</span><span>FVector4</span><span>, </span><span>HairRenderInfo</span><span>) </span><span>\</span></div></div><div><div><div>9</div></div><div><span> </span><span>VIEW_UNIFORM_BUFFER_MEMBER</span><span>(</span><span>uint32</span><span>, </span><span>HairRenderInfoBits</span><span>) </span><span>\</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>UE4 里 GLSL 的 uniform buffer 默认使用的是 std140 的布局。在这个 uniform table 里，如果成员中有数组，便会产生 uniform 成员偏移的问题，而这些偏移仅出现在使用 GLSL 的平台上。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>ralloc_asprintf_append</span><span>(</span><span>buffer</span><span>, </span><span>"layout(std140) uniform </span><span>%s</span><span>\n</span><span>{</span><span>\n</span><span>"</span><span>, </span><span>block_name</span><span>);</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>因为这种布局会将数组中的元素，不论类型，都当做 <code>16 字节</code> 来对齐。</p><p>官方的原话是 <strong>The array stride (the bytes between array elements) is always rounded up to the size of a vec4 (ie: 16-bytes).</strong></p><p><img src="/assets/images/UniformTable/Layout.png" alt="" /></p><p>也就是说如果你在 uniform table 中声明了一个成员，比如：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>VIEW_UNIFORM_BUFFER_MEMBER_ARRAY</span><span>(</span><span>float</span><span>, </span><span>LightVolume</span><span>[</span><span>2</span><span>])</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>这段代码在 HLSL 和 GLSL 中内存的布局是不同的，在 HLSL 中和我们 C++ 中声明的长度一致。而在 GLSL 中，由于 layout 是 std140，所以这里的 LightVolume[2] 实际上占用了 32 byte，而不是 8 byte。</p><p>这就使得编译 shader 时，在加入了一些 padding 后，从 CPU 传到 GPU 的值有了偏移。</p></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://www.khronos.org/opengl/wiki/Interface_Block_(GLSL)" target="_blank"><strong>Interface Block (GLSL)</strong></a></li> </ul></section>https://yohiro.games/posts/profile-guided-optimization-in-unreal/https://yohiro.games/posts/profile-guided-optimization-in-unreal/Mon, 17 Aug 2020 00:00:00 GMT<section><h2>简介<a href="#简介"><span>#</span></a></h2><p><code>PGO(Profile Guided Optimization)</code>是一种基于 LLVM 的编译时优化，通过使用运行时收集的分析数据来指导编译器进行优化。截至 UE5.0 版本，Epic 对 PC、Android 以及各 Console 平台都进行了配置。网路上很少有 PGO 相关的文章，这篇文章旨在尝试 PGO 的使用流程。</p></section> <section><h2>流程<a href="#流程"><span>#</span></a></h2><p>按照 Epic 的流程，PGO 可以直接启用，也可以搭配<code>Gauntlet 自动测试框架</code>集成到自动测试流程中使用。关于 Gauntlet 框架的可参考官方文档 <a href="https://docs.unrealengine.com/4.26/zh-CN/TestingAndOptimization/Automation/Gauntlet/" target="_blank"><strong>Gauntlet 自动化框架</strong></a>。PGO 整体的大致流程如图：</p><p></p><figure><img src="/assets/images/PGOInUnreal/Progress2.png" alt="Progress2" /><figcaption>Progress2</figcaption></figure><p></p><section><h3>构建用于 PGO 的版本<a href="#构建用于-pgo-的版本"><span>#</span></a></h3><p>首先需要构建用于收集分析数据的 PGO 版本。添加命令行<code>-PGOProfile</code>以开启相应的宏：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>/* --- TargetRules.cs --- */</span></div></div><div><div><div>2</div></div><div><span>/// </span><span>&lt;</span><span>summary</span><span>&gt;</span></div></div><div><div><div>3</div></div><div><span>/// Whether to enable Profile Guided Optimization (PGO) instrumentation in this build.</span></div></div><div><div><div>4</div></div><div><span>/// </span><span>&lt;/</span><span>summary</span><span>&gt;</span></div></div><div><div><div>5</div></div><div><span>[</span><span>CommandLine</span><span>(</span><span>"-PGOProfile"</span><span>, </span><span>Value</span><span><span> </span><span>=</span><span> </span></span><span>"true"</span><span>)]</span></div></div><div><div><div>6</div></div><div><span>[</span><span>XmlConfigFile</span><span>(</span><span>Category</span><span><span> </span><span>=</span><span> </span></span><span>"BuildConfiguration"</span><span>)]</span></div></div><div><div><div>7</div></div><div><span>public</span><span> </span><span>bool</span><span><span> </span><span>bPGOProfile</span><span> </span><span>=</span><span> </span></span><span>true</span><span>;</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span>/* --- UEBuildTarget.cs --- */</span></div></div><div><div><div>10</div></div><div><span>if</span><span><span> (</span><span>Rules</span><span>.</span><span>bPGOProfile</span><span>)</span></span></div></div><div><div><div>11</div></div><div><span>{</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>GlobalCompileEnvironment</span><span>.</span><span>Definitions</span><span>.</span><span>Add</span><span>(</span><span>"ENABLE_PGO_PROFILE=1"</span><span>);</span></div></div><div><div><div>13</div></div><div><span>}</span></div></div><div><div><div>14</div></div><div><span>else</span></div></div><div><div><div>15</div></div><div><span>{</span></div></div><div><div><div>16</div></div><div><span><span> </span></span><span>GlobalCompileEnvironment</span><span>.</span><span>Definitions</span><span>.</span><span>Add</span><span>(</span><span>"ENABLE_PGO_PROFILE=0"</span><span>);</span></div></div><div><div><div>17</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>PGO 版本中，宏<code>ENABLE_PGO_PROFILE</code>应该被启用，否则不会输出 PGO 的临时文件：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>/* --- PlatformMisc.cpp --- */</span></div></div><div><div><div>2</div></div><div><span>#if</span><span> </span><span>ENABLE_PGO_PROFILE</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>// Write the PGO profiling file on a clean shutdown.</span></div></div><div><div><div>4</div></div><div><span> </span><span>extern</span><span> </span><span>void</span><span> </span><span>PGO_WriteFile</span><span>();</span></div></div><div><div><div>5</div></div><div><span> </span><span>PGO_WriteFile</span><span>();</span></div></div><div><div><div>6</div></div><div><span>#endif</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>可以通过在 build 时传入指定的参数<code>-PGOProfile</code>来控制是否开启<code>ENABLE_PGO_PROFILE</code>。默认情况下打开 PGOProfile 后也会打开 LTO，因此链接时间会变得非常长。</p></section><section><h3>运行时收集数据<a href="#运行时收集数据"><span>#</span></a></h3><p>接下来需要启动游戏正常游玩、正常退出。因为只有在<code>RequestExit()</code>时才会在<code>PGO_WriteFile</code>中调用<code>__llvm_profile_write_file</code>即写入扩展名为<code>*.profraw</code>的临时文件。 该临时文件的输出目录可由命令行参数指定，但由于实现的问题，在不同的平台中，命令行参数有所不同，在启动时不指定输出目录，便会将该文件写入到默认位置，详情可参照<code>PGO_GetOutputDirectory</code>函数。</p></section><section><h3>创建 Gauntlet 的测试用例&lt;可选&gt;<a href="#创建-gauntlet-的测试用例可选"><span>#</span></a></h3><p>如果需要集成到自动测试流程中，需要创建测试用例并添加该测试至 Gauntlet 的项目里。这个过程可以参考<code>Engine\Source\Programs\AutomationTool\Gauntlet\Unreal\Game</code>下的 Samples。 Gauntlet 中已经有一个 PGO 的测试节点<code>Gauntlet.UnrealPGONode.cs</code>，其中 PGOConfig 有下面几个参数，可通过命令行传入给 UAT，其中<code>ProfileOutputDirectory</code>是必需的。</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>/// </span><span>&lt;</span><span>summary</span><span>&gt;</span></div></div><div><div><div>2</div></div><div><span>/// Output directory to write the resulting profile data to.</span></div></div><div><div><div>3</div></div><div><span>/// </span><span>&lt;/</span><span>summary</span><span>&gt;</span></div></div><div><div><div>4</div></div><div><span>[</span><span>AutoParam</span><span>(</span><span>""</span><span>)]</span></div></div><div><div><div>5</div></div><div><span>public</span><span> </span><span>string</span><span><span> </span><span>ProfileOutputDirectory</span><span>;</span></span></div></div><div><div><div>6</div></div><div> </div></div><div><div><div>7</div></div><div><span>/// </span><span>&lt;</span><span>summary</span><span>&gt;</span></div></div><div><div><div>8</div></div><div><span>/// Directory to save periodic screenshots to whilst the PGO run is in progress.</span></div></div><div><div><div>9</div></div><div><span>/// </span><span>&lt;/</span><span>summary</span><span>&gt;</span></div></div><div><div><div>10</div></div><div><span>[</span><span>AutoParam</span><span>(</span><span>""</span><span>)]</span></div></div><div><div><div>11</div></div><div><span>public</span><span> </span><span>string</span><span><span> </span><span>ScreenshotDirectory</span><span>;</span></span></div></div><div><div><div>12</div></div><div> </div></div><div><div><div>13</div></div><div><span>[</span><span>AutoParam</span><span>(</span><span>""</span><span>)]</span></div></div><div><div><div>14</div></div><div><span>public</span><span> </span><span>string</span><span><span> </span><span>PGOAccountSandbox</span><span>;</span></span></div></div><div><div><div>15</div></div><div> </div></div><div><div><div>16</div></div><div><span>[</span><span>AutoParam</span><span>(</span><span>""</span><span>)]</span></div></div><div><div><div>17</div></div><div><span>public</span><span> </span><span>string</span><span><span> </span><span>PgcFilenamePrefix</span><span>;</span></span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>使用 UAT 运行指定的测试用例。可以加入到 bat 文件里，方便集成到 Jenkins 一类的 CI 里：</p><div><div><div><figure><figcaption><span></span><span>Terminal window</span><span>展开</span><span>收起</span></figcaption><pre><code><div><div><div>1</div></div><div><span>rem</span><span> path for RunUAT.bat</span></div></div><div><div><div>2</div></div><div><span>set</span><span> </span><span>UAT_PATH</span><span><span>=</span><span>RunUAT.bat</span></span></div></div><div><div><div>3</div></div><div><span>rem</span><span> project name</span></div></div><div><div><div>4</div></div><div><span>set</span><span> </span><span>PRJ_NAME</span><span><span>=</span><span>{ProjectName}</span></span></div></div><div><div><div>5</div></div><div><span>rem</span><span> staging path</span></div></div><div><div><div>6</div></div><div><span>set</span><span> </span><span>STAGING_DIR</span><span><span>=</span><span>{EngineRoot}\{ProjectName}\Saved\StagedBuilds</span></span></div></div><div><div><div>7</div></div><div><span>rem</span><span> test command</span></div></div><div><div><div>8</div></div><div><span>set</span><span> </span><span>TEST_CMD</span><span><span>=</span><span>RunUnreal</span></span></div></div><div><div><div>9</div></div><div><span>rem</span><span> test name</span></div></div><div><div><div>10</div></div><div><span>set</span><span> </span><span>TEST_NAME</span><span><span>=</span><span>PGOTest</span></span></div></div><div><div><div>11</div></div><div><span>rem</span><span> profdata output path</span></div></div><div><div><div>12</div></div><div><span>set</span><span> </span><span>PROFILE_OUTPUT_PATH</span><span><span>=</span><span>{ProjectName}\Saved\Automation\PGO\</span></span></div></div><div><div><div>13</div></div><div><span>rem</span><span> screenshot path</span></div></div><div><div><div>14</div></div><div><span>set</span><span> </span><span>SCREENSHOT_DIRECTORY</span><span><span>=</span><span>{ProjectName}\Saved\Automation\PGO\Screenshot\</span></span></div></div><div><div><div>15</div></div><div><span>rem</span><span> platform Name</span></div></div><div><div><div>16</div></div><div><span>set</span><span> </span><span>PLATFORM</span><span><span>=</span><span>Android</span></span></div></div><div><div><div>17</div></div><div><span>rem</span><span> build configuration</span></div></div><div><div><div>18</div></div><div><span>set</span><span> </span><span>CONFIG</span><span><span>=</span><span>Test</span></span></div></div><div><div><div>19</div></div><div> </div></div><div><div><div>20</div></div><div><span>rem</span><span> ********************* Start Gauntlet Test *********************</span></div></div><div><div><div>21</div></div><div><span>%UAT_PATH%</span><span> </span><span>%TEST_CMD%</span><span> -project=</span><span>%PRJ_NAME%</span><span> -platform=</span><span>%PLATFORM%</span><span> -configuration=</span><span>%CONFIG%</span><span> -build=</span><span>%STAGING_DIR%</span><span>\</span><span>%PLATFORM%</span><span> -test=</span><span>%TEST_NAME%</span><span> -ProfileOutputDirectory=</span><span>%PROFILE_OUTPUT_PATH%</span><span> -ScreenshotDirectory=</span><span>%SCREENSHOT_DIRECTORY%</span></div></div><div><div><div>22</div></div><div><span>rem</span><span> ********************* End Gauntlet Test *********************</span></div></div><div><div><div>23</div></div><div><span>pause</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div></section></section> <section><h2>注意事项<a href="#注意事项"><span>#</span></a></h2><ul> <li>PGO 本质上是编译器优化，因此随着优化等级的提高，有可能会暴露出代码中一些原本不存在的问题。</li> <li>随着版本的迭代，代码不断更新，原本 PGO 收集的数据在用于新版本的优化时的效果会大打折扣。</li> <li>Profile 数据会有额外的内存占用。</li> </ul></section> <section><h2>参考资料<a href="#参考资料"><span>#</span></a></h2><ul> <li><a href="https://qiita.com/donbutsu17/items/cd17d500a9fed143e061" target="_blank">Gauntlet Automation Framework</a> 介绍 Gauntlet 测试框架，可以搭配官方文档一起看</li> <li><a href="https://horugame.com/gauntlet-automated-testing-and-performance-metrics-in-ue4/" target="_blank">GAUNTLET AUTOMATED TESTING AND PERFORMANCE METRICS IN UE4</a> 早期版本中 Gauntlet，可以当作参考</li> <li><a href="https://www.docswell.com/s/EpicGamesJapan/ZEEL7Z-UE4_LargeScaleDevSQEX_Optimize#p31" target="_blank">実行速度の最適化のあれこれ</a> 介绍了基于 Sample 的 PGO</li> <li><a href="https://github.com/DaedalicEntertainment/ue4-test-automation" target="_blank">Daedalic Test Automation Plugin</a> Github 上一款开源的 UE 的自动测试插件，对 Gauntlet 也进行了封装</li> <li><a href="https://source.android.google.cn/devices/tech/perf/pgo" target="_blank">使用配置文件引导的优化 (PGO)</a> Android 项目中使用 PGO</li> </ul></section>https://yohiro.games/posts/directly-run-on-attached/https://yohiro.games/posts/directly-run-on-attached/Fri, 03 Jul 2020 00:00:00 GMT<section><h2>背景<a href="#背景"><span>#</span></a></h2><p>调试的过程中，有时会碰到要 Attach 到某个进程的情况。遇到这种情况，一般来说就是 Alt+D,P 呼出 AttachToProcess 界面，然后找到进程 Attach。今天发现一个方法，可以一键直接开启 Attach 到指定的进程，而且具有一定的扩展性。本着好东西大家一起分享的原则，现记录如下。</p></section> <section><h2>原理<a href="#原理"><span>#</span></a></h2><p>这个方法是利用 VS 的宏和库，将指令脚本化，类似 Excel 里的宏。通过引入VS的扩展 <strong>Visual Commander</strong> 执行相应的命令。</p></section> <section><h2>步骤<a href="#步骤"><span>#</span></a></h2><ol> <li> <p><code>VS 菜单栏-&gt; Extensions -&gt; Manage Extensions</code> 搜 Visual Commander（下称 VCmd），先把它下下来<br /> </p><figure><img src="/assets/images/DirectlyRunOnAttached/VisualCommander.png" alt="Vcmd_Extension" /><figcaption>Vcmd_Extension</figcaption></figure>{: .light .shadow .rounded-10}<p></p> </li> <li> <p>下载完成后，可在 Extensions 中找到 VCmd，找到 Commands，我们可以在这里添加自定义的指令<br /> </p><figure><img src="/assets/images/DirectlyRunOnAttached/VCMDWhere.png" alt="VCmd_Command_Intro" /><figcaption>VCmd_Command_Intro</figcaption></figure>{: .light .shadow .rounded-10}<p></p> </li> <li> <p>然后点击 Add 以添加编辑页面，语言可选 C# 或 VB,可以看到用了 DTE 接口。可以起个名字，方便管理<br /> </p><figure><img src="/assets/images/DirectlyRunOnAttached/Command.png" alt="VCmd_Command_View" /><figcaption>VCmd_Command_View</figcaption></figure>{: .light .shadow .rounded-10}<p></p> </li> <li> <p>要运行的内容，按照我们的需求，就是找到指定的进程并 Attach 上去。我这里 Attach 的是 AutomationTool</p> <div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>using</span><span> </span><span>EnvDTE</span><span>;</span></div></div><div><div><div>2</div></div><div><span>using</span><span> </span><span>EnvDTE80</span><span>;</span></div></div><div><div><div>3</div></div><div> </div></div><div><div><div>4</div></div><div><span>public</span><span> </span><span>class</span><span> </span><span>C</span><span> : </span><span>VisualCommanderExt</span><span>.</span><span>ICommand</span></div></div><div><div><div>5</div></div><div><span>{</span></div></div><div><div><div>6</div></div><div><span> </span><span>public</span><span> </span><span>void</span><span> </span><span>Run</span><span>(</span><span>EnvDTE80</span><span>.</span><span>DTE2</span><span><span> </span><span>DTE</span><span>, </span></span><span>Microsoft</span><span>.</span><span>VisualStudio</span><span>.</span><span>Shell</span><span>.</span><span>Package</span><span><span> </span><span>package</span><span>)</span></span></div></div><div><div><div>7</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>8</div></div><div><span> </span><span>foreach</span><span>(</span><span>Process2</span><span><span> </span><span>proc</span><span> </span></span><span>in</span><span><span> </span><span>DTE</span><span>.</span><span>Debugger</span><span>.</span><span>LocalProcesses</span><span>)</span></span></div></div><div><div><div>9</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>10</div></div><div><span> </span><span>if</span><span><span>(</span><span>proc</span><span>.</span><span>Name</span><span>.</span></span><span>ToString</span><span>().</span><span>Contains</span><span>(</span><span>"AutomationToolLauncher.exe"</span><span>))</span></div></div><div><div><div>11</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>proc</span><span>.</span><span>Attach2</span><span>(</span><span>""</span><span>);</span></div></div><div><div><div>13</div></div><div><span> </span><span>return</span><span>;</span></div></div><div><div><div>14</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>15</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>16</div></div><div><span><span> </span></span><span>System</span><span>.</span><span>Windows</span><span>.</span><span>MessageBox</span><span>.</span><span>Show</span><span>(</span><span>"AutomationToolLauncher not found."</span><span>);</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>18</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div> <p>记得保存。</p> </li> <li> <p>这个时候已经可以运行了，但是为了一键 Attach，我们得把它绑定到一个按键上 菜单栏找 <code>Tools -&gt; Customize -&gt; Commands</code> 标签下，选择 <code>ToolBar -&gt; Standard</code>，这里看个人习惯，我放到了工具栏上 Standard 栏，右侧 <code>Add Command</code> 找到要执行的 Command<br /> </p><figure><img src="/assets/images/DirectlyRunOnAttached/AddCommand.png" alt="Add_Customize_Command" /><figcaption>Add_Customize_Command</figcaption></figure><br /> Extensions 目录下找到相应的 Command，不记得是第几个可以在 VCmd 的 Command 编辑界面看看左上角<p></p> </li> <li> <p>然后 Preview 里面选中后可以移动到合适的位置或者 <code>Modify Selection</code> 改个喜欢的名字<br /> </p><figure><img src="/assets/images/DirectlyRunOnAttached/Customize.png" alt="Modify_Customize_Command" /><figcaption>Modify_Customize_Command</figcaption></figure>{: .light .shadow .rounded-10}<p></p> </li> <li> <p>当目标进程启动时，直接点我们添加的按钮，就可以 Attach 到了<br /> </p><figure><img src="/assets/images/DirectlyRunOnAttached/CustomizeAttachButton.png" alt="Customize_Button" /><figcaption>Customize_Button</figcaption></figure>{: .light .shadow .rounded-10}<p></p> </li> </ol></section> <section><h2>最后<a href="#最后"><span>#</span></a></h2><p>其他的一些功能和样例可以参考 Extensions -&gt; VCmd -&gt; Command Examples/ Extension Examples 或者前往 <a href="https://vlasovstudio.com/visual-commander/commands.html" target="_blank"><strong>visual-commander</strong></a></p></section>https://yohiro.games/posts/wavelane/https://yohiro.games/posts/wavelane/Tue, 18 Feb 2020 00:00:00 GMT<p>In shader programming, you often run into a problem where you want to iterate an array in memory over all pixels in a compute shader group (tile). Tiled deferred lighting is the most common case. 8x8 tile loops over a light list culled for that tile.</p> <p>Simplified HLSL code looks like this:</p> <div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>Buffer</span><span>&lt;</span><span>float4</span><span>&gt; lightDatas;</span></div></div><div><div><div>2</div></div><div><span>Texture2D</span><span>&lt;</span><span>uint2</span><span>&gt; lightStartCounts;</span></div></div><div><div><div>3</div></div><div><span>RWTexture2D</span><span>&lt;</span><span>float4</span><span>&gt; output;</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span>[</span><span>numthreads</span><span>(</span><span>8</span><span>, </span><span>8</span><span>, </span><span>1</span><span>)]</span></div></div><div><div><div>6</div></div><div><span>void</span><span> </span><span>CSMain</span><span>(</span><span>uint3</span><span> tid : SV_DispatchThreadID, </span><span>uint3</span><span> gid : SV_GroupID)</span></div></div><div><div><div>7</div></div><div><span>{</span></div></div><div><div><div>8</div></div><div><span> </span><span>uint2</span><span> lightStartCount = lightStartCounts[gid.xy];</span></div></div><div><div><div>9</div></div><div><span> </span><span>float4</span><span> lightAccumulator = </span><span>0</span><span>;</span></div></div><div><div><div>10</div></div><div> </div></div><div><div><div>11</div></div><div><span> </span><span>for</span><span> (</span><span>uint</span><span> i = </span><span>0</span><span>; i &lt; lightStartCount.y; ++i)</span></div></div><div><div><div>12</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>13</div></div><div><span> </span><span>uint</span><span> lightIndex = lightStartCount.x + i;</span></div></div><div><div><div>14</div></div><div> </div></div><div><div><div>15</div></div><div><span> </span><span>// In real impl light data would be position, radius, color, etc.</span></div></div><div><div><div>16</div></div><div><span> </span><span>float4</span><span> lightData = lightDatas[lightIndex]; </span><span>// 64-wide mem load. All lanes load from same address.</span></div></div><div><div><div>17</div></div><div><span><span> </span></span><span>lightAccumulator += lightData;</span></div></div><div><div><div>18</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>19</div></div><div> </div></div><div><div><div>20</div></div><div><span><span> </span></span><span>output[tid.xy] = lightAccumulator;</span></div></div><div><div><div>21</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div> <p>The problem with this code is that we load the same light data for each lane. If you would have used constant buffer instead of Buffer, the compiler would have generated special code utilizing GPU specific uniform load facility like constant buffer hardware (Nvidia) or scalar unit (AMD). But this can be problematic, since constant buffers have 64KB size limit, and constant buffer array elements must be float4/int4 (16 byte alignment). Not even enough space to hold 1080p tile light lists.</p> <p>Fortunately with SM 6.0 wave intrinsics we can do better. We can load 32 (Nvidia) or 64 (AMD) ligths at once using a single load instruction and then use WaveReadLaneAt to broadcast light data from one lane to all lanes, one lane at a time. This reduces the number of load instructions by 32x / 64x. Also there’s no register bloat, since each loaded lane has now unique data, instead of replicated value. Register file usage is as good as with AMD scalar loads.</p> <p>Optimized code looks like this:</p> <div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>[</span><span>numthreads</span><span>(</span><span>8</span><span>, </span><span>8</span><span>, </span><span>1</span><span>)]</span></div></div><div><div><div>2</div></div><div><span>void</span><span> </span><span>CSMainWave</span><span>(</span><span>uint3</span><span> tid : SV_DispatchThreadID, </span><span>uint3</span><span> gid : SV_GroupID, </span><span>uint</span><span> gix : SV_GroupIndex)</span></div></div><div><div><div>3</div></div><div><span>{</span></div></div><div><div><div>4</div></div><div><span> </span><span>const</span><span> </span><span>uint</span><span> WAVE_LANES = </span><span>WaveGetLaneCount</span><span>(); </span><span>// Usually 64 or 32. Compile time constant.</span></div></div><div><div><div>5</div></div><div> </div></div><div><div><div>6</div></div><div><span> </span><span>uint2</span><span> lightStartCount = lightStartCounts[gid.xy];</span></div></div><div><div><div>7</div></div><div><span> </span><span>float4</span><span> lightAccumulator = </span><span>0</span><span>;</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span> </span><span>for</span><span> (</span><span>uint</span><span> i = </span><span>0</span><span>; i &lt; </span><span>divRoundUp</span><span>(lightStartCount.y, WAVE_LANES); ++i)</span></div></div><div><div><div>10</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>11</div></div><div><span> </span><span>// Load 32/64 unique items at once in outer loop. 32x/64x reduction in mem loads.</span></div></div><div><div><div>12</div></div><div><span> </span><span>float4</span><span> lightData64 = lightDatas[lightStartCount.x + gix + i * WAVE_LANES];</span></div></div><div><div><div>13</div></div><div> </div></div><div><div><div>14</div></div><div><span> </span><span>uint</span><span> loopIters = </span><span>min</span><span>(WAVE_LANES, lightStartCount.y - WAVE_LANES * i);</span></div></div><div><div><div>15</div></div><div><span> </span><span>for</span><span> (</span><span>uint</span><span> i2 = </span><span>0</span><span>; i2 &lt; loopIters; ++i2)</span></div></div><div><div><div>16</div></div><div><span><span> </span></span><span>{</span></div></div><div><div><div>17</div></div><div><span> </span><span>// In real impl light data would be position, radius, color, etc.</span></div></div><div><div><div>18</div></div><div><span> </span><span>float4</span><span> lightData = </span><span>WaveReadLaneAt</span><span>(lightData64, i2); </span><span>// Broadcast current light to all lanes</span></div></div><div><div><div>19</div></div><div><span><span> </span></span><span>lightAccumulator += lightData;</span></div></div><div><div><div>20</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>21</div></div><div><span><span> </span></span><span>}</span></div></div><div><div><div>22</div></div><div> </div></div><div><div><div>23</div></div><div><span><span> </span></span><span>output[tid.xy] = lightAccumulator;</span></div></div><div><div><div>24</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div> <p>This code actually will beat scalar optimized code on AMD, since one vector load is faster than 64 scalar loads. Another advantage is that this optimization works for all buffer and texture types, not just for constant/raw/structured buffers.</p> <p>According to my GPU buffer benchmark (<a href="https://github.com/sebbbi/perftest" target="_blank">https://github.com/sebbbi/perftest</a>), new Nvidia Maxwell/Pascal/Volta/Turing drivers employ uniform address optimization for all buffer and texture types. I have seen 20x-27x performance increases for cases where all threads inside a loop load from uniform address. My guess is this Nvidia optimization employed in their new compiler is similar to the optimization I described above. I don’t have access to their PIX plugin to validate my claim. Maybe someone with access to that plugin can validate my assumption (if NDA is not a problem).</p> <p>Recent AMD drivers employ a similar optimization for global atomic add. This can be validated with publicly available tools such as RenderDoc. If all lanes add C or 0, the compiler first does WavePrefixCountBits (MBCNT) and then a single lane global atomic, reducing serialization to atomic counter by 64x.</p> <p>I hope that AMD also implements the optimization above to their shader compiler. In AMDs case, this optimization is actually super good, since in most cases AMD gets also full memory coalescing gain from this optimizationg, resulting in 4x load issue rate.</p> <p>Shader playground link: <a href="http://shader-playground.timjones.io/04cb6af828fe0e1a48d651e25d06cffb" target="_blank">http://shader-playground.timjones.io/04cb6af828fe0e1a48d651e25d06cffb</a></p> <section><h2>reference<a href="#reference"><span>#</span></a></h2><ul> <li><a href="https://gist.github.com/sebbbi/ba4415339b535d22fb18e2d824564ec4" target="_blank">FastUniformLoadWithWaveOps</a></li> <li><a href="https://gpuopen.com/learn/amd-gcn-assembly-cross-lane-operations/" target="_blank">AMD GCN Assembly: Cross-Lane Operations</a></li> </ul></section>https://yohiro.games/posts/approx-for-sincos/https://yohiro.games/posts/approx-for-sincos/Tue, 12 Nov 2019 00:00:00 GMT<p>对 GLSL 中缺失的 FP64 函数的近似，其他的信息可以参考<a href="https://outerra.blogspot.com/2014/05/double-precision-approximations-for-map.html" target="_blank"><strong>OpenGL中地图投影的双精度近似</strong></a></p> <div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>//sin approximation, error &lt; 5e-9</span></div></div><div><div><div>2</div></div><div><span>double</span><span> </span><span>sina_9</span><span>(</span><span>double</span><span><span> </span><span>x</span><span>)</span></span></div></div><div><div><div>3</div></div><div><span>{</span></div></div><div><div><div>4</div></div><div><span> </span><span>//minimax coefs for sin for 0..pi/2 range</span></div></div><div><div><div>5</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a3 </span><span>=</span><span> </span><span>-</span></span><span>1.666665709650470145824129400050267289858e-1LF</span><span>;</span></div></div><div><div><div>6</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a5 </span><span>=</span><span> </span></span><span>8.333017291562218127986291618761571373087e-3LF</span><span>;</span></div></div><div><div><div>7</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a7 </span><span>=</span><span> </span><span>-</span></span><span>1.980661520135080504411629636078917643846e-4LF</span><span>;</span></div></div><div><div><div>8</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a9 </span><span>=</span><span> </span></span><span>2.600054767890361277123254766503271638682e-6LF</span><span>;</span></div></div><div><div><div>9</div></div><div> </div></div><div><div><div>10</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> m_2_pi </span><span>=</span><span> </span></span><span>0.636619772367581343076LF</span><span>;</span></div></div><div><div><div>11</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> m_pi_2 </span><span>=</span><span> </span></span><span>1.57079632679489661923LF</span><span>;</span></div></div><div><div><div>12</div></div><div> </div></div><div><div><div>13</div></div><div><span> </span><span>double</span><span><span> y </span><span>=</span><span> </span></span><span>abs</span><span><span>(x </span><span>*</span><span> m_2_pi);</span></span></div></div><div><div><div>14</div></div><div><span> </span><span>double</span><span><span> q </span><span>=</span><span> </span></span><span>floor</span><span>(y);</span></div></div><div><div><div>15</div></div><div><span> </span><span>int</span><span><span> quadrant </span><span>=</span><span> </span></span><span>int</span><span>(q);</span></div></div><div><div><div>16</div></div><div> </div></div><div><div><div>17</div></div><div><span> </span><span>double</span><span><span> t </span><span>=</span><span> (quadrant </span><span>&amp;</span><span> </span></span><span>1</span><span><span>) </span><span>!=</span><span> </span></span><span>0</span><span><span> </span><span>?</span><span> </span></span><span>1</span><span><span> </span><span>-</span><span> y </span><span>+</span><span> q </span><span>:</span><span> y </span><span>-</span><span> q;</span></span></div></div><div><div><div>18</div></div><div><span><span> </span></span><span>t </span><span>*=</span><span> m_pi_2;</span></div></div><div><div><div>19</div></div><div> </div></div><div><div><div>20</div></div><div><span> </span><span>double</span><span><span> t2 </span><span>=</span><span> t </span><span>*</span><span> t;</span></span></div></div><div><div><div>21</div></div><div><span> </span><span>double</span><span><span> r </span><span>=</span><span> </span></span><span>fma</span><span>(</span><span>fma</span><span>(</span><span>fma</span><span>(</span><span>fma</span><span><span>(a9, t2, a7), t2, a5), t2, a3), t2</span><span>*</span><span>t, t);</span></span></div></div><div><div><div>22</div></div><div> </div></div><div><div><div>23</div></div><div><span><span> </span></span><span>r </span><span>=</span><span> x </span><span>&lt;</span><span> </span><span>0</span><span><span> </span><span>?</span><span> </span><span>-</span><span>r </span><span>:</span><span> r;</span></span></div></div><div><div><div>24</div></div><div> </div></div><div><div><div>25</div></div><div><span> </span><span>return</span><span><span> (quadrant </span><span>&amp;</span><span> </span></span><span>2</span><span><span>) </span><span>!=</span><span> </span></span><span>0</span><span><span> </span><span>?</span><span> </span><span>-</span><span>r </span><span>:</span><span> r;</span></span></div></div><div><div><div>26</div></div><div><span>}</span></div></div><div><div><div>27</div></div><div> </div></div><div><div><div>28</div></div><div><span>//sin approximation, error &lt; 2e-11</span></div></div><div><div><div>29</div></div><div><span>double</span><span> </span><span>sina_11</span><span>(</span><span>double</span><span><span> </span><span>x</span><span>)</span></span></div></div><div><div><div>30</div></div><div><span>{</span></div></div><div><div><div>31</div></div><div><span> </span><span>//minimax coefs for sin for 0..pi/2 range</span></div></div><div><div><div>32</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a3 </span><span>=</span><span> </span><span>-</span></span><span>1.666666660646699151540776973346659104119e-1LF</span><span>;</span></div></div><div><div><div>33</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a5 </span><span>=</span><span> </span></span><span>8.333330495671426021718370503012583606364e-3LF</span><span>;</span></div></div><div><div><div>34</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a7 </span><span>=</span><span> </span><span>-</span></span><span>1.984080403919620610590106573736892971297e-4LF</span><span>;</span></div></div><div><div><div>35</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> a9 </span><span>=</span><span> </span></span><span>2.752261885409148183683678902130857814965e-6LF</span><span>;</span></div></div><div><div><div>36</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> ab </span><span>=</span><span> </span><span>-</span></span><span>2.384669400943475552559273983214582409441e-8LF</span><span>;</span></div></div><div><div><div>37</div></div><div> </div></div><div><div><div>38</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> m_2_pi </span><span>=</span><span> </span></span><span>0.636619772367581343076LF</span><span>;</span></div></div><div><div><div>39</div></div><div><span> </span><span>const</span><span> </span><span>double</span><span><span> m_pi_2 </span><span>=</span><span> </span></span><span>1.57079632679489661923LF</span><span>;</span></div></div><div><div><div>40</div></div><div> </div></div><div><div><div>41</div></div><div><span> </span><span>double</span><span><span> y </span><span>=</span><span> </span></span><span>abs</span><span><span>(x </span><span>*</span><span> m_2_pi);</span></span></div></div><div><div><div>42</div></div><div><span> </span><span>double</span><span><span> q </span><span>=</span><span> </span></span><span>floor</span><span>(y);</span></div></div><div><div><div>43</div></div><div><span> </span><span>int</span><span><span> quadrant </span><span>=</span><span> </span></span><span>int</span><span>(q);</span></div></div><div><div><div>44</div></div><div> </div></div><div><div><div>45</div></div><div><span> </span><span>double</span><span><span> t </span><span>=</span><span> (quadrant </span><span>&amp;</span><span> </span></span><span>1</span><span><span>) </span><span>!=</span><span> </span></span><span>0</span><span><span> </span><span>?</span><span> </span></span><span>1</span><span><span> </span><span>-</span><span> y </span><span>+</span><span> q </span><span>:</span><span> y </span><span>-</span><span> q;</span></span></div></div><div><div><div>46</div></div><div><span><span> </span></span><span>t </span><span>*=</span><span> m_pi_2;</span></div></div><div><div><div>47</div></div><div> </div></div><div><div><div>48</div></div><div><span> </span><span>double</span><span><span> t2 </span><span>=</span><span> t </span><span>*</span><span> t;</span></span></div></div><div><div><div>49</div></div><div><span> </span><span>double</span><span><span> r </span><span>=</span><span> </span></span><span>fma</span><span>(</span><span>fma</span><span>(</span><span>fma</span><span>(</span><span>fma</span><span>(</span><span>fma</span><span>(ab, t2, a9), t2, a7), t2, a5), t2, a3),</span></div></div><div><div><div>50</div></div><div><span><span> </span></span><span>t2</span><span>*</span><span>t, t);</span></div></div><div><div><div>51</div></div><div> </div></div><div><div><div>52</div></div><div><span><span> </span></span><span>r </span><span>=</span><span> x </span><span>&lt;</span><span> </span><span>0</span><span><span> </span><span>?</span><span> </span><span>-</span><span>r </span><span>:</span><span> r;</span></span></div></div><div><div><div>53</div></div><div> </div></div><div><div><div>54</div></div><div><span> </span><span>return</span><span><span> (quadrant </span><span>&amp;</span><span> </span></span><span>2</span><span><span>) </span><span>!=</span><span> </span></span><span>0</span><span><span> </span><span>?</span><span> </span><span>-</span><span>r </span><span>:</span><span> r;</span></span></div></div><div><div><div>55</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div> <p>对于 Cosine 而言，平移 π/2 即可</p> <div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>//cos approximation, error &lt; 5e-9</span></div></div><div><div><div>2</div></div><div><span>double</span><span> </span><span>cosa_9</span><span>(</span><span>double</span><span><span> </span><span>x</span><span>)</span></span></div></div><div><div><div>3</div></div><div><span>{</span></div></div><div><div><div>4</div></div><div><span> </span><span>//sin(x + PI/2) = cos(x)</span></div></div><div><div><div>5</div></div><div><span> </span><span>return</span><span> </span><span>sina_9</span><span><span>(x </span><span>+</span><span> </span></span><span>DBL_LIT</span><span>(</span><span>1.57079632679489661923LF</span><span>));</span></div></div><div><div><div>6</div></div><div><span>}</span></div></div><div><div><div>7</div></div><div> </div></div><div><div><div>8</div></div><div><span>//cos approximation, error &lt; 2e-11</span></div></div><div><div><div>9</div></div><div><span>double</span><span> </span><span>cosa_11</span><span>(</span><span>double</span><span><span> </span><span>x</span><span>)</span></span></div></div><div><div><div>10</div></div><div><span>{</span></div></div><div><div><div>11</div></div><div><span> </span><span>//sin(x + PI/2) = cos(x)</span></div></div><div><div><div>12</div></div><div><span> </span><span>return</span><span> </span><span>sina_11</span><span><span>(x </span><span>+</span><span> </span></span><span>DBL_LIT</span><span>(</span><span>1.57079632679489661923LF</span><span>));</span></div></div><div><div><div>13</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div>https://yohiro.games/posts/filament-lighting-parti/https://yohiro.games/posts/filament-lighting-parti/Thu, 22 Aug 2019 00:00:00 GMT<p>本篇是 <a href="https://google.github.io/filament/Filament.html" target="_blank"><strong>Filament</strong></a> 的笔记，以及部分自己的理解。</p> <p>可以结合 Desktop 的渲染方式一起，看 Filament 的渲染为了更好地支持移动端，舍弃了哪些。</p> <p>也可以搭配 <a href="https://www.bilibili.com/video/BV1X7411F744/?share_source=copy_web&amp;vd_source=7a4dacf2c6974d860175d6d297f6d566" target="_blank">【GAMES101-现代计算机图形学入门-闫令琪】</a> 食用，风味更佳。</p> <section><h1>光照<a href="#光照"><span>#</span></a></h1><p>现在常用的引擎，如 Unity 以及 Unreal 等，对于光照的处理并不统一。</p><p>以 Unreal 为例，<strong>点光源</strong>亮度的单位是<em>流明（Lumen）</em>，用于表示的单位。但是<strong>方向光</strong>的亮度是一个没有具体单位的量，为了使点光源与 5000 lumen 的亮度相匹配，方向光的亮度需要调为 10。这种单位上的不统一，使得在场景中打光变得很不方便，也很难保持视觉一致性。</p><p>我们可以选择一个单位来统一光照，但又不方便迁移。例如在室外以亮度为 10 的方向光作为太阳光的亮度，所有其他的灯光都调整为以此为基准的相对亮度，在移动这些灯光到室内时，又会使得室内环境过亮。</p><p>这里先定义两种类型的光照：</p><p>直接光照 : 点光源、区域光和 photometric lights（光度学光源？）</p><p>间接光照 : 基于图像的照明(<strong>I</strong>mage <strong>B</strong>ased <strong>L</strong>ighting)</p><section><h2>单位<a href="#单位"><span>#</span></a></h2> <table><thead><tr><th>光度学术语名</th><th>符号</th><th>单位</th></tr></thead><tbody><tr><td>光功率 (Luminous power)</td><td><span><span>Φ\Phi</span><span><span><span></span><span>Φ</span></span></span></span></td><td>流明 Lumen(<span><span>lmlm</span><span><span><span></span><span>l</span><span>m</span></span></span></span>)</td></tr><tr><td>光强 (Luminous intensity)</td><td><span><span>II</span><span><span><span></span><span>I</span></span></span></span></td><td>坎德拉 Candela (<span><span>cdcd</span><span><span><span></span><span>c</span><span>d</span></span></span></span>) or <span><span>lm/srlm/sr</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span>sr</span></span></span></span></td></tr><tr><td>光照度 (Illuminance)</td><td><span><span>EE</span><span><span><span></span><span>E</span></span></span></span></td><td>勒克斯 Lux (<span><span>lxlx</span><span><span><span></span><span>l</span><span>x</span></span></span></span>) or <span><span>lm/m2lm/m^2</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span><span>m</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></td></tr><tr><td>光亮度/发光率 (Luminance)</td><td><span><span>LL</span><span><span><span></span><span>L</span></span></span></span></td><td>尼特 Nit (<span><span>ntnt</span><span><span><span></span><span>n</span><span>t</span></span></span></span>) or <span><span>cd/m2cd/m^2</span><span><span><span></span><span>c</span><span>d</span><span>/</span><span><span>m</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></td></tr><tr><td>辐射功率 (Radiant power)</td><td><span><span>Φe\Phi_e</span><span><span><span></span><span><span>Φ</span><span><span><span><span><span><span></span><span><span>e</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>Watt (<span><span>WW</span><span><span><span></span><span>W</span></span></span></span>)</td></tr><tr><td>发光效能 (Luminous efficacy)</td><td><span><span>η\eta</span><span><span><span></span><span>η</span></span></span></span></td><td>Lumens per watt (<span><span>lm/Wlm/W</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span>W</span></span></span></span>)</td></tr><tr><td>发光效率 (Luminous efficiency)</td><td><span><span>VV</span><span><span><span></span><span>V</span></span></span></span></td><td>Percentage (%)</td></tr></tbody></table><p>下面是 Filament 中的灯光类型以及单位。</p> <table><thead><tr><th>灯光类型</th><th>单位</th></tr></thead><tbody><tr><td>方向光</td><td>照度 Lux (<span><span>lxlx</span><span><span><span></span><span>l</span><span>x</span></span></span></span>) or <span><span>lm/m2lm/m^2</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span><span>m</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></td></tr><tr><td>点光源</td><td>光功率 Lumen (<span><span>lmlm</span><span><span><span></span><span>l</span><span>m</span></span></span></span>)</td></tr><tr><td>聚光灯</td><td>光功率 Lumen (<span><span>lmlm</span><span><span><span></span><span>l</span><span>m</span></span></span></span>)</td></tr><tr><td>光度灯光</td><td>光强 Candela (<span><span>cdcd</span><span><span><span></span><span>c</span><span>d</span></span></span></span>) or <span><span>lm/srlm/sr</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span>sr</span></span></span></span></td></tr><tr><td>光度遮罩灯光</td><td>光功率 Lumen (<span><span>lmlm</span><span><span><span></span><span>l</span><span>m</span></span></span></span>)</td></tr><tr><td>区域光</td><td>光功率 Lumen (<span><span>lmlm</span><span><span><span></span><span>l</span><span>m</span></span></span></span>)</td></tr><tr><td>基于图像的灯光</td><td>光亮度/发光率 Nit (<span><span>ntnt</span><span><span><span></span><span>n</span><span>t</span></span></span></span>) or <span><span>cd/m2cd/m^2</span><span><span><span></span><span>c</span><span>d</span><span>/</span><span><span>m</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></td></tr></tbody></table><section><h3>关于辐射功率单位<a href="#关于辐射功率单位"><span>#</span></a></h3><p>艺术家可能习惯于通过功率来衡量灯光的亮度，因此我们应该允许用户使用功率单位来定义灯光的亮度。转换如下:</p><span><span><span>Φ=Φeη\Phi = \Phi_e \eta</span><span><span><span></span><span>Φ</span><span></span><span>=</span><span></span></span><span><span></span><span><span>Φ</span><span><span><span><span><span><span></span><span><span>e</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>η</span></span></span></span></span><p>这个等式中 <span><span>η\eta</span><span><span><span></span><span>η</span></span></span></span> 指的是发光效能，单位是流明每瓦 (<span><span>lm/Wlm/W</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span>W</span></span></span></span>) ，根据<a href="https://zh.wikipedia.org/wiki/%E7%99%BC%E5%85%89%E6%95%88%E8%83%BD" target="_blank">维基百科中的说法</a>，已知最大的发光效能为 <strong>683</strong> <span><span>lm/Wlm/W</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span>W</span></span></span></span> ，那么用发光效率 <span><span>VV</span><span><span><span></span><span>V</span></span></span></span> 来表示的话，就是这样：</p><span><span><span>Φ=Φe683×V\Phi = \Phi_e 683 \times V</span><span><span><span></span><span>Φ</span><span></span><span>=</span><span></span></span><span><span></span><span><span>Φ</span><span><span><span><span><span><span></span><span><span>e</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>683</span><span></span><span>×</span><span></span></span><span><span></span><span>V</span></span></span></span></span><p>使用各种类型灯的发光效率或发光效率将瓦特转换为流明</p> <table><thead><tr><th>光源类型</th><th>发光效能(<span><span>lm/Wlm/W</span><span><span><span></span><span>l</span><span>m</span><span>/</span><span>W</span></span></span></span>)</th><th>发光效率 (%)</th></tr></thead><tbody><tr><td>白炽灯</td><td>14-35</td><td>2-5%</td></tr><tr><td>LED 灯</td><td>28-100</td><td>4-15%</td></tr><tr><td>荧光灯</td><td>60-100</td><td>9-15%</td></tr></tbody></table><span><span><span>I=E⋅d2I = E \cdot d^2</span><span><span><span></span><span>I</span><span></span><span>=</span><span></span></span><span><span></span><span>E</span><span></span><span>⋅</span><span></span></span><span><span></span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span></section></section><section><h2>直接光照<a href="#直接光照"><span>#</span></a></h2><p>上面仅对不同的光源类型定义了单位，Filament 使用物理光单位，亮度的计算是在 shader 中进行的，因此所有的光照函数都要计算亮度 <span><span>LoutL_{out}</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>（也称<em>出射辐亮度</em>），该亮度值取决于光照度 <span><span>EE</span><span><span><span></span><span>E</span></span></span></span> 和 BSDF <span><span>f(v,l)f(v,l)</span><span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span></span></span></span></p><span><span><span>Lout=f(v,l)EL_{out} = f(v,l)E</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span>E</span></span></span></span></span><section><h3>方向光<a href="#方向光"><span>#</span></a></h3><p>方向光主要用于户外环境，用来模拟阳光/月光，并不存在于真实的世界中，但因为太阳/月亮的距离足够远，所以可以近似为平行的方向光。</p><p><img src="/assets/images/Filament/diagram_directional_light.png" alt="" /></p><p>这种光照对表面的漫反射效果不错，但对于镜面反射的效果并不好。在寒霜引擎中，将来自太阳的方向光模拟为圆盘状的区域光，作为镜面反射的补偿。</p><p>Filament 中方向光的单位是照度(<span><span>lxlx</span><span><span><span></span><span>l</span><span>x</span></span></span></span>)，有部分原因就是可以测量天空、阳光的具体照度值，从而简化 <span><span>LoutL_{out}</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 的计算。</p><span><span><span>Lout=f(v,l)E⊥&lt;N⋅L&gt;L_{out} = f(v,l) E_{\bot} \left&lt; N \cdot L \right&gt;</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span><span>E</span><span><span><span><span><span><span></span><span><span><span>⊥</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span><span>⟨</span><span>N</span><span></span><span>⋅</span><span></span><span>L</span><span>⟩</span></span></span></span></span></span><p>其中 <span><span>E⊥E_{\bot}</span><span><span><span></span><span><span>E</span><span><span><span><span><span><span></span><span><span><span>⊥</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 是光源对垂直于光源的表面的照度，下面是加州 3 月的晴朗天气下测得的天空和阳光的照度值（满月为 1<span><span>lxlx</span><span><span><span></span><span>l</span><span>x</span></span></span></span>）：</p> <table><thead><tr><th>光源</th><th>上午 10 点</th><th>中午 12 点</th><th>下午 5&lt;30&gt;</th></tr></thead><tbody><tr><td><span><span>Sky⊥Sky_{\bot}</span><span><span><span></span><span>S</span><span>k</span><span><span>y</span><span><span><span><span><span><span></span><span><span><span>⊥</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> + <span><span>Sun⊥Sun_{\bot}</span><span><span><span></span><span>S</span><span>u</span><span><span>n</span><span><span><span><span><span><span></span><span><span><span>⊥</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>120,000</td><td>130,000</td><td>90,000</td></tr><tr><td><span><span>Sky⊥Sky_{\bot}</span><span><span><span></span><span>S</span><span>k</span><span><span>y</span><span><span><span><span><span><span></span><span><span><span>⊥</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>20,000</td><td>25,000</td><td>9,000</td></tr><tr><td><span><span>Sun⊥Sun_{\bot}</span><span><span><span></span><span>S</span><span>u</span><span><span>n</span><span><span><span><span><span><span></span><span><span><span>⊥</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>100,000</td><td>105,000</td><td>81,000</td></tr></tbody></table><p>方向光的实现:</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>vec3</span><span><span> l </span><span>=</span><span> </span></span><span>normalize</span><span><span>(</span><span>-</span><span>lightDirection);</span></span></div></div><div><div><div>2</div></div><div><span>float</span><span><span> NoL </span><span>=</span><span> </span></span><span>clamp</span><span>(</span><span>dot</span><span><span>(</span><span>n</span><span>,</span><span> l</span><span>), </span></span><span>0.0</span><span>, </span><span>1.0</span><span>);</span></div></div><div><div><div>3</div></div><div> </div></div><div><div><div>4</div></div><div><span>// lightIntensity is the illuminance</span></div></div><div><div><div>5</div></div><div><span>// at perpendicular incidence in lux</span></div></div><div><div><div>6</div></div><div><span>float</span><span><span> illuminance </span><span>=</span><span> lightIntensity </span><span>*</span><span> NoL;</span></span></div></div><div><div><div>7</div></div><div><span>vec3</span><span><span> luminance </span><span>=</span><span> </span></span><span>BSDF</span><span><span>(</span><span>v</span><span>,</span><span> l</span><span>) </span><span>*</span><span> illuminance;</span></span></div></div></code></pre><div><div></div><div></div></div></figure></div></section><section><h3>精准光<a href="#精准光"><span>#</span></a></h3><p>一般的渲染引擎中支持两种精准光，点光源和聚光灯，精准光是指<em>从某一位置发光的光源</em>，这种光源的光线均匀地照射到各个方向。但它不是物理准确的，因为下面两个原因：</p><ol> <li>过于准确，且趋于无限小</li> <li>不遵循<a href="https://en.wikipedia.org/wiki/Inverse-square_law" target="_blank">平方反比原则</a></li> </ol><p>对于 1，可以引入区域光解决这一问题。对于 2 直接引入材质表面着色点到光源的距离 <span><span>dd</span><span><span><span></span><span>d</span></span></span></span> 来解决，如下。</p><span><span><span>E=Lin&lt;N⋅L&gt;=Id2&lt;N⋅L&gt;E = L_{in} \left&lt; N \cdot L \right&gt; = \frac{I}{d^2} \left&lt; N \cdot L \right&gt;</span><span><span><span></span><span>E</span><span></span><span>=</span><span></span></span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>in</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span><span>⟨</span><span>N</span><span></span><span>⋅</span><span></span><span>L</span><span>⟩</span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>I</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span><span>⟨</span><span>N</span><span></span><span>⋅</span><span></span><span>L</span><span>⟩</span></span></span></span></span></span><section><h4>点光源<a href="#点光源"><span>#</span></a></h4><p>点光源仅由空间中的位置来定义。</p><p><img src="/assets/images/Filament/diagram_point_light.png" alt="" /></p><p>点光源的单位是发光功率（<span><span>lmlm</span><span><span><span></span><span>l</span><span>m</span></span></span></span>），发光功率是通过光源立体角上的发光强度的积分计算的：</p><span><span><span>Φ=∫ΩIdl=∫02π∫0πIdθdϕ=4πII=Φ4π\begin{aligned} \Phi = \int_{\Omega} I dl = \int_{0}^{2\pi} \int_{0}^{\pi} I d\theta d\phi = 4 \pi I \\ I = \frac{\Phi}{4 \pi} \end{aligned}</span><span><span><span></span><span><span><span><span><span><span><span><span></span><span><span>Φ</span><span></span><span>=</span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span><span>Ω</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>I</span><span>d</span><span>l</span><span></span><span>=</span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span><span>0</span></span></span></span><span><span></span><span><span><span>2</span><span>π</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span><span>0</span></span></span></span><span><span></span><span><span><span>π</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>I</span><span>d</span><span>θ</span><span>d</span><span>ϕ</span><span></span><span>=</span><span></span><span>4</span><span>π</span><span>I</span></span></span><span><span></span><span><span>I</span><span></span><span>=</span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span>Φ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span></span><p>得到了发光功率，就可以很容易地计算出光照强度：</p><span><span><span>I=Φ4πI = \frac{\Phi}{4 \pi}</span><span><span><span></span><span>I</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span>Φ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>结合上面提到的 <span><span>Lout=f(v,l)EL_{out}=f(v,l)E</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span>E</span></span></span></span>，最终可以得到：</p><span><span><span>Lout=f(v,l)Φ4πd2&lt;N⋅L&gt;L_{out} = f(v,l) \frac{\Phi}{4 \pi d^2} \left&lt; N \cdot L \right&gt;</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>π</span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>Φ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span><span>⟨</span><span>N</span><span></span><span>⋅</span><span></span><span>L</span><span>⟩</span></span></span></span></span></span></section><section><h4>聚光灯<a href="#聚光灯"><span>#</span></a></h4><p>聚光灯由位置、方向和两个锥角 <span><span>θinner\theta_{inner}</span><span><span><span></span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>inn</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 和 <span><span>θouter\theta_{outer}</span><span><span><span></span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 来定义。一般的光照使用 <span><span>1d2\frac{1}{d^2}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 来定义距离的衰减，而对于聚光灯 <span><span>θinner\theta_{inner}</span><span><span><span></span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>inn</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 和 <span><span>θouter\theta_{outer}</span><span><span><span></span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 定义了聚光灯锥形区域两侧的角度衰减。</p><p><img src="/assets/images/Filament/diagram_spot_light.png" alt="" /></p><span><span><span>Φ=∫ΩIdl=∫02π∫0θouterIdθdϕ=2π(1−cosθouter2)II=Φ2π(1−cosθouter2)\begin{aligned} \Phi = \int_{\Omega} I dl = \int_{0}^{2\pi} \int_{0}^{\theta_{outer}} I d\theta d\phi = 2 \pi (1 - cos\frac{\theta_{outer}}{2})I\\ I = \frac{\Phi}{2 \pi (1 - cos\frac{\theta_{outer}}{2})} \end{aligned}</span><span><span><span></span><span><span><span><span><span><span><span><span></span><span><span>Φ</span><span></span><span>=</span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span><span>Ω</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>I</span><span>d</span><span>l</span><span></span><span>=</span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span><span>0</span></span></span></span><span><span></span><span><span><span>2</span><span>π</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span><span>0</span></span></span></span><span><span></span><span><span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>I</span><span>d</span><span>θ</span><span>d</span><span>ϕ</span><span></span><span>=</span><span></span><span>2</span><span>π</span><span>(</span><span>1</span><span></span><span>−</span><span></span><span>cos</span><span><span></span><span><span><span><span><span><span></span><span><span>2</span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>)</span><span>I</span></span></span><span><span></span><span><span>I</span><span></span><span>=</span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>2</span><span>π</span><span>(</span><span>1</span><span></span><span>−</span><span></span><span>cos</span><span><span></span><span><span><span><span><span><span></span><span><span><span>2</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>Φ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span></span><p>但在这里，聚光灯的圆锥与亮度耦合在一起，调整锥角的角度时，也会同时影响亮度。因此，需要将角度和亮度解耦以便灯光师调节：</p><span><span><span>Φ=πII=Φπ\begin{aligned} \Phi = \pi I \\ I = \frac{\Phi}{\pi} \\ \end{aligned}</span><span><span><span></span><span><span><span><span><span><span><span><span></span><span><span>Φ</span><span></span><span>=</span><span></span><span>π</span><span>I</span></span></span><span><span></span><span><span>I</span><span></span><span>=</span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span>Φ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span></span><p>聚光灯的两种评估方式：</p><p>从吸收光的角度 :</p><span><span><span>Lout=f(v,l)Φπd2&lt;N⋅L&gt;λ(l)L_{out} = f(v,l) \frac{\Phi}{\pi d^2} \left&lt; N \cdot L \right&gt; \lambda(l)</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span><span></span><span><span><span><span><span><span></span><span><span>π</span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>Φ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span><span>⟨</span><span>N</span><span></span><span>⋅</span><span></span><span>L</span><span>⟩</span></span><span></span><span>λ</span><span>(</span><span>l</span><span>)</span></span></span></span></span><p>从反射光的角度 :</p><span><span><span>Lout=f(v,l)Φ2π(1−cosθouter2)d2&lt;N⋅L&gt;λ(l)L_{out} = f(v,l) \frac{\Phi}{2 \pi (1 - cos\frac{\theta_{outer}}{2}) d^2} \left&lt; N \cdot L \right&gt; \lambda(l)</span><span><span><span></span><span><span>L</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span><span></span><span><span><span><span><span><span></span><span><span>2</span><span>π</span><span>(</span><span>1</span><span></span><span>−</span><span></span><span>cos</span><span><span></span><span><span><span><span><span><span></span><span><span><span>2</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>)</span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>Φ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span><span>⟨</span><span>N</span><span></span><span>⋅</span><span></span><span>L</span><span>⟩</span></span><span></span><span>λ</span><span>(</span><span>l</span><span>)</span></span></span></span></span><p>以上两式中的 <span><span>λ(l)\lambda(l)</span><span><span><span></span><span>λ</span><span>(</span><span>l</span><span>)</span></span></span></span> 项是聚光灯的角度衰减因子，描述如下：</p><span><span><span>λ(l)=l×spotDirection−cosθoutercosθinner−cosθouter\lambda(l) = \frac{l \times spotDirection - cos\theta_{outer}}{cos\theta_{inner} - cos\theta_{outer}}</span><span><span><span></span><span>λ</span><span>(</span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>cos</span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>inn</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>−</span><span></span><span>cos</span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>l</span><span></span><span>×</span><span></span><span>s</span><span>p</span><span>o</span><span>t</span><span>D</span><span>i</span><span>r</span><span>ec</span><span>t</span><span>i</span><span>o</span><span>n</span><span></span><span>−</span><span></span><span>cos</span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>o</span><span>u</span><span>t</span><span>er</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span></section><section><h4>衰减函数<a href="#衰减函数"><span>#</span></a></h4><p>光照的衰减需要正确地评估衰减因子，简单的 <span><span>1d2\frac{1}{d^2}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 项并不好使，因为：</p><ol> <li>当几何体与光源相接触，可能会有 <span><span>dd</span><span><span><span></span><span>d</span></span></span></span> = 0，从而导致除以 0。</li> <li>理论上每个光源的范围是无限远的，在有多光源的场合会导致 <span><span>1d2\frac{1}{d^2}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 永远不会为 0，如果要正确着色，需要对场景内每一个光源进行评估。</li> </ol><p>为解决上述问题 1，Filament 的解决方式是将点光源假设为一个小半径为 1 cm 的球形区域光。</p><span><span><span>E=Imax(d2,0.012)E = \frac{I}{max(d^2, {0.01}^2)}</span><span><span><span></span><span>E</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>ma</span><span>x</span><span>(</span><span><span>d</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>,</span><span></span><span><span><span>0.01</span></span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>I</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>为解决问题 2， 可以为每个光源引入一个影响半径的参数解决。通过引入这一变量可以展示给美术光源所影响到的范围，也能方便引擎渲染时剔除光源。</p><p>由 GLSL 实现的基于物理的精准光源参考：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> </span><span>getSquareFalloffAttenuation</span><span><span>(</span><span>vec3 </span><span>posToLight</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>lightInvRadius</span><span>)</span></span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>float</span><span><span> distanceSquare </span><span>=</span><span> </span></span><span>dot</span><span>(posToLight, posToLight);</span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span><span> factor </span><span>=</span><span> distanceSquare </span><span>*</span><span> lightInvRadius </span><span>*</span><span> lightInvRadius;</span></span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span><span> smoothFactor </span><span>=</span><span> </span></span><span>max</span><span>(</span><span>1.0</span><span><span> </span><span>-</span><span> factor </span><span>*</span><span> factor, </span></span><span>0.0</span><span>);</span></div></div><div><div><div>6</div></div><div><span> </span><span>return</span><span><span> (smoothFactor </span><span>*</span><span> smoothFactor) </span><span>/</span><span> </span></span><span>max</span><span>(distanceSquare, </span><span><span>1</span><span>e</span></span><span>-</span><span>4</span><span>);</span></div></div><div><div><div>7</div></div><div><span>}</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span>float</span><span> </span><span>getSpotAngleAttenuation</span><span><span>(</span><span>vec3 </span><span>l</span><span>,</span><span> vec3 </span><span>lightDir</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>innerAngle</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>outerAngle</span><span>)</span></span></div></div><div><div><div>10</div></div><div><span>{</span></div></div><div><div><div>11</div></div><div><span> </span><span>// 缩放和偏移计算可以在CPU端完成</span></div></div><div><div><div>12</div></div><div><span> </span><span>float</span><span><span> cosOuter </span><span>=</span><span> </span></span><span>cos</span><span>(outerAngle);</span></div></div><div><div><div>13</div></div><div><span> </span><span>float</span><span><span> spotScale </span><span>=</span><span> </span></span><span>1.0</span><span><span> </span><span>/</span><span> </span></span><span>max</span><span>(</span><span>cos</span><span><span>(innerAngle) </span><span>-</span><span> cosOuter, </span></span><span><span>1</span><span>e</span></span><span>-</span><span>4</span><span>)</span></div></div><div><div><div>14</div></div><div><span> </span><span>float</span><span><span> spotOffset </span><span>=</span><span> </span><span>-</span><span>cosOuter </span><span>*</span><span> spotScale</span></span></div></div><div><div><div>15</div></div><div> </div></div><div><div><div>16</div></div><div><span> </span><span>float</span><span><span> cd </span><span>=</span><span> </span></span><span>dot</span><span>(</span><span>normalize</span><span><span>(</span><span>-</span><span>lightDir), l);</span></span></div></div><div><div><div>17</div></div><div><span> </span><span>float</span><span><span> attenuation </span><span>=</span><span> </span></span><span>clamp</span><span><span>(cd </span><span>*</span><span> spotScale </span><span>+</span><span> spotOffset, </span></span><span>0.0</span><span>, </span><span>1.0</span><span>);</span></div></div><div><div><div>18</div></div><div><span> </span><span>return</span><span><span> attenuation </span><span>*</span><span> attenuation;</span></span></div></div><div><div><div>19</div></div><div><span>}</span></div></div><div><div><div>20</div></div><div> </div></div><div><div><div>21</div></div><div><span>vec3</span><span> </span><span>evaluatePunctualLight</span><span>()</span></div></div><div><div><div>22</div></div><div><span>{</span></div></div><div><div><div>23</div></div><div><span><span> </span></span><span>vec3 l </span><span>=</span><span> </span><span>normalize</span><span>(posToLight);</span></div></div><div><div><div>24</div></div><div><span> </span><span>float</span><span><span> NoL </span><span>=</span><span> </span></span><span>clamp</span><span>(</span><span>dot</span><span>(n, l), </span><span>0.0</span><span>, </span><span>1.0</span><span>);</span></div></div><div><div><div>25</div></div><div><span><span> </span></span><span>vec3 posToLight </span><span>=</span><span> lightPosition </span><span>-</span><span> worldPosition;</span></div></div><div><div><div>26</div></div><div> </div></div><div><div><div>27</div></div><div><span> </span><span>float</span><span> attenuation;</span></div></div><div><div><div>28</div></div><div><span><span> </span></span><span>attenuation </span><span>=</span><span> </span><span>getSquareFalloffAttenuation</span><span>(posToLight, lightInvRadius);</span></div></div><div><div><div>29</div></div><div><span><span> </span></span><span>attenuation </span><span>*=</span><span> </span><span>getSpotAngleAttenuation</span><span>(l, lightDir, innerAngle, outerAngle);</span></div></div><div><div><div>30</div></div><div> </div></div><div><div><div>31</div></div><div><span> </span><span>// 这里的光照强度的单位是坎德拉，可由光照功率转换而来</span></div></div><div><div><div>32</div></div><div><span><span> </span></span><span>vec3 luminance </span><span>=</span><span> (</span><span>BSDF</span><span><span>(v, l) </span><span>*</span><span> lightIntensity </span><span>*</span><span> attenuation </span><span>*</span><span> NoL) </span><span>*</span><span> lightColor;</span></span></div></div><div><div><div>33</div></div><div><span> </span><span>return</span><span> luminance;</span></div></div><div><div><div>34</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>以上的部分计算可在 CPU 端完成。</p></section></section><section><h3>光度学光源<a href="#光度学光源"><span>#</span></a></h3><p>TODO</p></section><section><h3>面光源<a href="#面光源"><span>#</span></a></h3><p>TODO</p></section><section><h3>光源参数<a href="#光源参数"><span>#</span></a></h3><p>Filement 的目标是使光源参数易于美工和开发人员使用. 为此需要直观方便，因此光源颜色(或色调)与光源强度相互独立，且光源的颜色定义为线性 RGB 颜色(或在外部工具/UI 中使用 sRGB)。</p> <table><thead><tr><th>参数</th><th>定义</th></tr></thead><tbody><tr><td>光源类型</td><td>平行光 Directional, 点光源 point, 聚光灯 spot, 面光源 area</td></tr><tr><td>方向</td><td>平行光、点光、聚光灯、面光的方向</td></tr><tr><td>光源颜色</td><td>发射出的光的颜色，指定为线性 RGB，在外部工具中可以 sRGB 或色温指定</td></tr><tr><td>光照强度</td><td>直观的反映为灯光的亮度，具体单位取决于光源类型</td></tr><tr><td>衰减半径</td><td>最大的影响距离</td></tr><tr><td>内锥角</td><td>聚光灯形成的内圆锥体的角度</td></tr><tr><td>外锥角</td><td>聚光灯形成的外圆锥体的角度</td></tr><tr><td>长度</td><td>面光源的长度, 用于创建线性或管状灯光</td></tr><tr><td>半径</td><td>面光源的半径, 用于创建球形或管状灯光</td></tr></tbody></table><section><h4>色温<a href="#色温"><span>#</span></a></h4><p>真实世界中的人造灯光通常由它们的色温来定义, 以开尔文(K)为单位. 光源的色温是理想黑体辐射体的温度, 此黑体辐射出的光的色调与光源相当. 为方便起见, 这些工具应该支持美工使用色温指定光源的色调(有意义的范围为1,000 K到12,500 K)。</p></section></section><section><h3>预曝光<a href="#预曝光"><span>#</span></a></h3></section></section><section><h2>IBL<a href="#ibl"><span>#</span></a></h2></section><section><h2>静态光照<a href="#静态光照"><span>#</span></a></h2></section><section><h2>透明/半透明光照<a href="#透明半透明光照"><span>#</span></a></h2></section><section><h2>遮挡<a href="#遮挡"><span>#</span></a></h2></section><section><h2>法线映射<a href="#法线映射"><span>#</span></a></h2></section><section><h2>体积效果<a href="#体积效果"><span>#</span></a></h2></section><section><h2>抗锯齿<a href="#抗锯齿"><span>#</span></a></h2></section><section><h2>后处理<a href="#后处理"><span>#</span></a></h2></section><section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="http://graphicrants.blogspot.com/2013/08/specular-brdf-reference.html" target="_blank">Specular BRDF Reference</a></li> <li><a href="https://www.unrealengine.com/en/blog/physically-based-shading-on-mobile" target="_blank">physically-based-shading-on-mobile</a></li> <li><a href="https://randomascii.wordpress.com/2012/09/09/game-developer-magazine-floating-point/" target="_blank">Game Developer Magazine Floating Point</a></li> <li><a href="https://media.contentapi.ea.com/content/dam/eacom/frostbite/files/s2014-pbs-frostbite-slides.pdf" target="_blank">Moving Frostbite to PBR</a></li> </ul></section></section>https://yohiro.games/posts/filament-materials-partii/https://yohiro.games/posts/filament-materials-partii/Mon, 15 Jul 2019 00:00:00 GMT<p>本篇是 <a href="https://google.github.io/filament/Filament.html" target="_blank"><strong>Filament</strong></a> 的笔记，以及部分自己的理解。</p> <p>可以结合 Desktop 的渲染方式一起，看 Filament 的渲染为了更好地支持移动端，舍弃了哪些。</p> <p>也可以搭配 <a href="https://www.bilibili.com/video/BV1X7411F744/?share_source=copy_web&amp;vd_source=7a4dacf2c6974d860175d6d297f6d566" target="_blank">【GAMES101-现代计算机图形学入门-闫令琪】</a> 食用，风味更佳。</p> <section><h2>透明涂层（Clear coat）模型<a href="#透明涂层clear-coat模型"><span>#</span></a></h2><p>这里把具有各向同性的单层材质称为<strong>标准材质</strong>，而标准材质上具有半透明薄膜涂层的，比如汽车油漆，漆木这类多层材料被称为<strong>透明涂层材质(Clear coat)</strong></p><p></p><figure><img src="/assets/images/Filament/material_clear_coat.png" alt="标准材料模型（左）和透明涂层模型（右）的比较" /><figcaption>标准材料模型（左）和透明涂层模型（右）的比较</figcaption></figure><p></p><p>由上面的介绍可以看出，Clear coat 材质具有两层表面，其中标准材质层称为 <em>Base layer</em>，透明涂层称作 <em>Clear coat layer</em>，透明涂层是具有各向同性的电介质，标准材质层可以是任何层（电介质或导体）。Clear coat 材质的基本模型如下：</p><p></p><figure><img src="/assets/images/Filament/diagram_clear_coat.png" alt="Clear coat 模型" /><figcaption>Clear coat 模型</figcaption></figure><p></p><p>实时渲染领域，不会模拟涂层间的反射和折射行为，入射光将穿过透明涂层，因此也会存在能量损失。</p><section><h3>Clear Coat Specular BRDF<a href="#clear-coat-specular-brdf"><span>#</span></a></h3><p>Clear coat 也会使用标准模型中同样的 Cook-Torrance 微表面 BRDF 进行建模。由于透明涂层是具有各向同性的电介质，具有较低的粗糙度，因此可以选择比较简单的 D，F，G。</p><p>由于标准模型中使用的分布函数项以及菲涅尔项的计算已经足够简单^{<a href="#user-content-fn-disney">1</a>}^{<a href="#user-content-fn-karis13a">2</a>}，所以这里仅考虑 Geomertry 项，在<a href="https://www.researchgate.net/publication/2378872_A_Microfacet_Based_Coupled_Specular-Matte_BRDF_Model_with_Importance_Sampling" target="_blank">这篇论文</a>^{<a href="#user-content-fn-kelemen01">3</a>}中描述了一种基于重要性采样的可以取代 SmithGGX 的几何阴影函数。</p><span><span><span>V(l,h)=14(l⋅h)2V(l,h) = \frac{1}{4({l}\cdot{h})^2}</span><span><span><span></span><span>V</span><span>(</span><span>l</span><span>,</span><span></span><span>h</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>(</span><span><span>l</span></span><span></span><span>⋅</span><span></span><span><span>h</span></span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>这个函数所描绘的 Mask Shadow 并不是基于物理的，但它足够简单，适合实时渲染。GLSL 中的实现也非常简单：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> </span><span>V_Kelemen</span><span>(</span><span>float</span><span><span> </span><span>LoH</span><span>) {</span></span></div></div><div><div><div>2</div></div><div><span> </span><span>return</span><span> </span><span>0.25</span><span><span> </span><span>/</span><span> (LoH </span><span>*</span><span> LoH);</span></span></div></div><div><div><div>3</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div></section><section><h3>关于 Fresnel 项<a href="#关于-fresnel-项"><span>#</span></a></h3><p>Specular BRDF 需要法向的反射率 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>，这里假定涂层的主要成分是聚氨酯，一种涂料和清漆中常见的化合物，在<em>空气-聚氨酯</em>的 IOR 为 1.5，因此可以参考上面提到的公式计算 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>：</p><span><span><span>f0=(1.5−1)2(1.5+1)2=0.04f_0 = \frac{(1.5 - 1)^2}{(1.5 + 1)^2} = 0.04</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>(</span><span>1.5</span><span></span><span>+</span><span></span><span>1</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>(</span><span>1.5</span><span></span><span>−</span><span></span><span>1</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>0.04</span></span></span></span></span><p>由于透明涂层的存在，必须考虑能量的损耗，</p><span><span><span>f(v,l)=fd(v,l)(1−Fc)+fr(v,l)(1−Fc)+fc(v,l)f(v,l) = f_d(v,l)(1- F_c)+f_r(v,l)(1-F_c)+f_c(v,l)</span><span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span>(</span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>c</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span><span></span><span>+</span><span></span></span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span>(</span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>c</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span><span></span><span>+</span><span></span></span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>c</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span></span></span></span></span><p>这里，<span><span>FcF_c</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span>c</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 是 clear coat 的 Fresnel 项，<span><span>fcf_c</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>c</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 是 clear coat BRDF。</p></section><section><h3>参数<a href="#参数"><span>#</span></a></h3><p>除了标准模型的 6 个参数（BaseColor, Roughness,Metallic,Reflectance，Emissive，Ambient occlusion）外，clear coat 添加了另外两个参数，</p> <table><thead><tr><th>参数</th><th>定义</th></tr></thead><tbody><tr><td>ClearCoat</td><td>透明涂层的强度，介于 0 到 1 之间。</td></tr><tr><td>ClearCoatRoughness</td><td>非物理真实的感知粗糙度，介于 0 到 1 之间。</td></tr></tbody></table><p><em>ClearCoatRoughness</em> 是重新映射的值，映射方法、clamp 的值域与标准模型的粗糙度一样，以开平方的方式再映射到线性空间。</p><p></p><figure><img src="/assets/images/Filament/material_clear_coat1.png" alt="ClearCoat" /><figcaption>ClearCoat</figcaption></figure> <em>ClearCoat 从 0 到 1 的变化，金属度 1.0，粗糙度 0.8</em><p></p><p></p><figure><img src="/assets/images/Filament/material_clear_coat2.png" alt="ClearCoatRoughness" /><figcaption>ClearCoatRoughness</figcaption></figure> <em>ClearCoatRoughness 从 0 到 1 的变化，金属度 1.0，粗糙度 0.8，ClearCoat 1.0</em><p></p><p>完成了再映射和参数化的、集成了 clear coat BRDF 的 GLSL 实现：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>void</span><span> </span><span>BRDF</span><span><span>(</span><span>...</span><span>)</span></span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>// compute Fd and Fr from standard model</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span> </span><span>// remapping and linearization of clear coat roughness</span></div></div><div><div><div>6</div></div><div><span><span> </span></span><span>clearCoatPerceptualRoughness </span><span>=</span><span> </span><span>clamp</span><span>(clearCoatPerceptualRoughness, </span><span>0.089</span><span>, </span><span>1.0</span><span>);</span></div></div><div><div><div>7</div></div><div><span><span> </span></span><span>clearCoatRoughness </span><span>=</span><span> clearCoatPerceptualRoughness </span><span>*</span><span> clearCoatPerceptualRoughness;</span></div></div><div><div><div>8</div></div><div> </div></div><div><div><div>9</div></div><div><span> </span><span>// clear coat BRDF</span></div></div><div><div><div>10</div></div><div><span> </span><span>float</span><span><span> Dc </span><span>=</span><span> </span></span><span>D_GGX</span><span>(clearCoatRoughness, NoH);</span></div></div><div><div><div>11</div></div><div><span> </span><span>float</span><span><span> Vc </span><span>=</span><span> </span></span><span>V_Kelemen</span><span>(clearCoatRoughness, LoH);</span></div></div><div><div><div>12</div></div><div><span> </span><span>float</span><span><span> Fc </span><span>=</span><span> </span></span><span>F_Schlick</span><span>(</span><span>0.04</span><span><span>, LoH) </span><span>*</span><span> clearCoat;</span></span><span> // clear coat strength</span></div></div><div><div><div>13</div></div><div><span> </span><span>float</span><span><span> Frc </span><span>=</span><span> (Dc </span><span>*</span><span> Vc) </span><span>*</span><span> Fc;</span></span></div></div><div><div><div>14</div></div><div> </div></div><div><div><div>15</div></div><div><span> </span><span>// account for energy loss in the base layer</span></div></div><div><div><div>16</div></div><div><span> </span><span>return</span><span><span> color </span><span>*</span><span> ((Fd </span><span>+</span><span> Fr </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>-</span><span> Fc)) </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>-</span><span> Fc) </span><span>+</span><span> Frc);</span></span></div></div><div><div><div>17</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div></section><section><h3>标准材质层<a href="#标准材质层"><span>#</span></a></h3><p>涂层的存在使得反射的 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 需要重新计算，因为原先的 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 是基于<em>空气-材质</em>层的，而标准材质层需要计算的是<em>材质-涂层</em>这一层。我们可以用 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 表示材质的折射率 <span><span>IORbaseIOR_{base}</span><span><span><span></span><span>I</span><span>O</span><span><span>R</span><span><span><span><span><span><span></span><span><span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 从而获得涂层的 <span><span>f0basef_{0base}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>0</span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>。</p><p>首先计算标准材质层的 IOR:</p><span><span><span>IORbase=1+f01−f0IOR_{base} = \frac{1+\sqrt{f_0}}{1-\sqrt{f_0}}</span><span><span><span></span><span>I</span><span>O</span><span><span>R</span><span><span><span><span><span><span></span><span><span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>1</span><span></span><span>−</span><span></span><span><span><span><span><span><span></span><span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span><span></span><span>+</span><span></span><span><span><span><span><span><span></span><span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>然后计算标准材质层的 <span><span>f0basef_{0base}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>0</span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>，其中 1.5 是涂层的折射率，</p><span><span><span>f0base=(IORbase−1.5IORbase+1.5)2f_{0base} = (\frac{IOR_{base}-1.5}{IOR_{base}+1.5})^2</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>0</span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>(</span><span><span></span><span><span><span><span><span><span></span><span><span>I</span><span>O</span><span><span>R</span><span><span><span><span><span><span></span><span><span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>1.5</span></span></span><span><span></span><span></span></span><span><span></span><span><span>I</span><span>O</span><span><span>R</span><span><span><span><span><span><span></span><span><span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>−</span><span></span><span>1.5</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span><p>涂层层的折射率是固定的，可以将上述的两个方程联立以简化：</p><span><span><span>f0base=(1−5f0)2(5−f0)2f_{0base} = \frac{(1-5\sqrt{f_0})^2}{(5-\sqrt{f_0})^2}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>0</span><span>ba</span><span>se</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>(</span><span>5</span><span></span><span>−</span><span></span><span><span><span><span><span><span></span><span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>(</span><span>1</span><span></span><span>−</span><span></span><span>5</span><span><span><span><span><span><span></span><span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>如果需要进一步的优化 clear coat 模型，可以将标准材质层的粗糙度从 ClearCoatRoughness 中分离出来。</p></section></section> <section><h2>各向异性模型<a href="#各向异性模型"><span>#</span></a></h2><p>上面所描述的材质模型均是针对各项同性表面，即表面各个方向的属性是一样的。但是像是拉丝金属这类材质，需要用各向异性模型进行表达：</p><p></p><figure><img src="/assets/images/Filament/material_anisotropic.png" alt="各向异性" /><figcaption>各向异性</figcaption></figure> <em>各向同性 vs 各向异性</em><p></p><section><h3>各项异性 specular BRDF<a href="#各项异性-specular-brdf"><span>#</span></a></h3><p>可以通过将描述各向同性的标准材质模型的 specular BRDF 的粗糙度分解为<strong>切线方向的粗糙度 <span><span>αt\alpha_{t}</span><span><span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></strong> 和<strong>副切线方向的粗糙度 <span><span>αb\alpha_{b}</span><span><span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>b</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></strong>，从而获得各向异性材质的 NDF</p><span><span><span>Daniso(h,α)=1παtαb1((t⋅hαt)2+(b⋅hαb)2+(n⋅h)2)2D_{aniso}(h,\alpha) = \frac{1}{\pi\alpha_{t}\alpha_{b}} \frac{1}{(( \frac{t \cdot h}{\alpha_{t}})^2 + ( \frac{b \cdot h}{\alpha_{b}})^2 +(n \cdot h)^2)^2}</span><span><span><span></span><span><span>D</span><span><span><span><span><span><span></span><span><span><span>ani</span><span>so</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>b</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span></span><span><span><span><span><span><span></span><span><span>((</span><span><span></span><span><span><span><span><span><span></span><span><span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>t</span><span>⋅</span><span>h</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>(</span><span><span></span><span><span><span><span><span><span></span><span><span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>b</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>b</span><span>⋅</span><span>h</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>h</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>但是这个 NDF 会引入两个额外参数。<a href="https://blog.selfshadow.com/publications/s2013-shading-course/rad/s2013_pbs_rad_slides.pdf" target="_blank"><strong>Neubelt13</strong></a>^{<a href="#user-content-fn-neubelt13">4</a>} 提出引入 anisotropy 参数，用该参数来表示 <span><span>αt\alpha_{t}</span><span><span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 和 <span><span>αb\alpha_{b}</span><span><span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span><span>b</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>，</p><span><span><span>αt=ααb=lerp(0,α,1−anisotropy)\begin{aligned} \alpha_t &amp;= \alpha \\ \alpha_b &amp;= lerp(0, \alpha, 1 - anisotropy) \end{aligned}</span><span><span><span></span><span><span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>t</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>b</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span><span><span><span><span><span></span><span><span></span><span></span><span>=</span><span></span><span>α</span></span></span><span><span></span><span><span></span><span></span><span>=</span><span></span><span>l</span><span>er</span><span>p</span><span>(</span><span>0</span><span>,</span><span></span><span>α</span><span>,</span><span></span><span>1</span><span></span><span>−</span><span></span><span>ani</span><span>so</span><span>t</span><span>r</span><span>o</span><span>p</span><span>y</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span></span><p><a href="https://media.disneyanimation.com/uploads/production/publication_asset/48/asset/s2012_pbs_disney_brdf_notes_v3.pdf" target="_blank"><strong>迪士尼的模型</strong></a>^{<a href="#user-content-fn-disney">1</a>}中定义的各向异性有较好的视觉效果，但也更为昂贵：</p><span><span><span>αt=α1−0.9×anisotropyαb=α1−0.9×anisotropy\begin{aligned} \alpha_t &amp;= \frac{\alpha}{\sqrt{1 - 0.9 \times anisotropy}} \\ \alpha_b &amp;= \alpha \sqrt{1 - 0.9 \times anisotropy} \end{aligned}</span><span><span><span></span><span><span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>t</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>b</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span><span><span><span><span><span></span><span><span></span><span></span><span>=</span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span><span><span><span><span></span><span><span>1</span><span></span><span>−</span><span></span><span>0.9</span><span></span><span>×</span><span></span><span>ani</span><span>so</span><span>t</span><span>r</span><span>o</span><span>p</span><span>y</span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>α</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span><span><span></span><span><span></span><span></span><span>=</span><span></span><span>α</span><span><span><span><span><span><span></span><span><span>1</span><span></span><span>−</span><span></span><span>0.9</span><span></span><span>×</span><span></span><span>ani</span><span>so</span><span>t</span><span>r</span><span>o</span><span>p</span><span>y</span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span></span><p>Filament 没有使用上面二者，而是选择了高光更为锐利的 <a href="https://blog.selfshadow.com/publications/s2017-shading-course/imageworks/s2017_pbs_imageworks_slides_v2.pdf" target="_blank"><strong>Kulla17</strong></a>^{<a href="#user-content-fn-kulla17">5</a>}:</p><span><span><span>αt=α×(1+anisotropy)αb=α×(1−anisotropy)\begin{aligned} \alpha_t &amp;= \alpha \times (1 + anisotropy) \\ \alpha_b &amp;= \alpha \times (1 - anisotropy) \end{aligned}</span><span><span><span></span><span><span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>t</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>b</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span><span><span><span><span><span></span><span><span></span><span></span><span>=</span><span></span><span>α</span><span></span><span>×</span><span></span><span>(</span><span>1</span><span></span><span>+</span><span></span><span>ani</span><span>so</span><span>t</span><span>r</span><span>o</span><span>p</span><span>y</span><span>)</span></span></span><span><span></span><span><span></span><span></span><span>=</span><span></span><span>α</span><span></span><span>×</span><span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span>ani</span><span>so</span><span>t</span><span>r</span><span>o</span><span>p</span><span>y</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span></span><p>由于法线贴图本身就需要切线和副切线数据，因此这两个参数可以很方便的获得到，下面是最终的实现：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span><span> at </span><span>=</span><span> </span></span><span>max</span><span><span>(roughness </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>+</span><span> anisotropy), </span></span><span>0.001</span><span>);</span></div></div><div><div><div>2</div></div><div><span>float</span><span><span> ab </span><span>=</span><span> </span></span><span>max</span><span><span>(roughness </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>-</span><span> anisotropy), </span></span><span>0.001</span><span>);</span></div></div><div><div><div>3</div></div><div> </div></div><div><div><div>4</div></div><div><span>float</span><span> </span><span>D_GGX_Anisotropic</span><span>(</span><span>float</span><span><span> </span><span>NoH</span><span>,</span><span> </span></span><span>const</span><span><span> vec3 </span><span>h</span><span>,</span></span></div></div><div><div><div>5</div></div><div><span> </span><span>const</span><span><span> vec3 </span><span>t</span><span>,</span><span> </span></span><span>const</span><span><span> vec3 </span><span>b</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>at</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>ab</span><span>) {</span></span></div></div><div><div><div>6</div></div><div><span> </span><span>float</span><span><span> ToH </span><span>=</span><span> </span></span><span>dot</span><span>(t, h);</span></div></div><div><div><div>7</div></div><div><span> </span><span>float</span><span><span> BoH </span><span>=</span><span> </span></span><span>dot</span><span>(b, h);</span></div></div><div><div><div>8</div></div><div><span> </span><span>float</span><span><span> a2 </span><span>=</span><span> at </span><span>*</span><span> ab;</span></span></div></div><div><div><div>9</div></div><div><span><span> </span></span><span>highp vec3 v </span><span>=</span><span> </span><span>vec3</span><span><span>(ab </span><span>*</span><span> ToH, at </span><span>*</span><span> BoH, a2 </span><span>*</span><span> NoH);</span></span></div></div><div><div><div>10</div></div><div><span><span> </span></span><span>highp </span><span>float</span><span><span> v2 </span><span>=</span><span> </span></span><span>dot</span><span>(v, v);</span></div></div><div><div><div>11</div></div><div><span> </span><span>float</span><span><span> w2 </span><span>=</span><span> a2 </span><span>/</span><span> v2;</span></span></div></div><div><div><div>12</div></div><div><span> </span><span>return</span><span><span> a2 </span><span>*</span><span> w2 </span><span>*</span><span> w2 </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>/</span><span> PI);</span></span></div></div><div><div><div>13</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div></section><section><h3>参数<a href="#参数-1"><span>#</span></a></h3><p>各向异性材质模型在标准材质的基础上多出了一项各向异性系数 <strong>Anisotropy</strong>，该参数为 -1 到 1 的标量，负值表示向副切线方向对齐，正值表示向切线方向对齐。</p><p><img src="/assets/images/Filament/anisotropy.png" alt="" /> <em>Anisotropy 从零到一的变化</em></p></section></section> <section><h2>布料模型<a href="#布料模型"><span>#</span></a></h2><p>衣服、织物的布料通常由松散连接的丝线构成，这类材质会吸收和散射入射光。而微表面的 BRDF 模型假定材质表面有随机凹槽构成，在宏观上近似光滑平面，这导致微表面 BRDF 模型不太适合重建布料这种存在散射的材质。</p><p><img src="/assets/images/Filament/screenshot_cloth.png" alt="" /> <em>传统微表面 BRDF 模型下的布料（左）与 Filament 中的布料（右）</em></p><p>像天鹅绒这类材质，由于布料表面直立的纤维会产生前向和后向的散射。前向散射指的是，入射光来自于观察向量的反方向的散射。而后向散射指的就是，入射光来自于观察向量相同方向的散射。</p><p>其他类型的布料，如皮革、丝绸等，更适合使用硬表面的材质模型，如标准模型或是各向异性模型。</p><section><h3>布料的 Specular BRDF<a href="#布料的-specular-brdf"><span>#</span></a></h3><p>Filament 使用的布料 BRDF 是经过修改的微表面 BRDF。而在 BRDF 的各项中，分布函数项（NDF）对 BRDF 的贡献最大^{<a href="#user-content-fn-ashikhmin07">6</a>}。该分布项是逆高斯分布，有助于实现前向/后向散射的模糊照明，并在此基础上添加模拟镜面反射的偏移。描述天鹅绒材质的 NDF 描述如下：</p><span><span><span>Dvelvet(v,h,α)=cnorm(1+4exp(−cot2θhα2))D_{velvet}(v,h,\alpha) = c_{norm}(1 + 4 exp\left(\frac{-{cot}^2\theta_{h}}{\alpha ^2}\right))</span><span><span><span></span><span><span>D</span><span><span><span><span><span><span></span><span><span><span>v</span><span>e</span><span>l</span><span>v</span><span>e</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span>c</span><span><span><span><span><span><span></span><span><span><span>n</span><span>or</span><span>m</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>1</span><span></span><span>+</span><span></span></span><span><span></span><span>4</span><span>e</span><span>x</span><span>p</span><span></span><span><span><span>(</span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>−</span><span><span><span>co</span><span>t</span></span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>h</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span>)</span></span></span><span>)</span></span></span></span></span><p>该 NDF 是 <a href="https://www.semanticscholar.org/paper/Distribution-based-BRDFs-Ashikhmin-Premoze/c54e98f379334f881389962c8598148389db5c40" target="_blank">Ashikhmin00</a>^{<a href="#user-content-fn-ashikhmin00">7</a>} 中的变体，该 NDF 也有标准化的版本^{<a href="#user-content-fn-neubelt13">4</a>}：</p><span><span><span>Dvelvet(v,h,α)=1π(1+4α2)(1+4exp(−cot2θhα2)sin⁡4θh)D_{velvet}(v,h,\alpha) = \frac{1}{\pi(1 + 4\alpha^2)}(1 + 4 \frac{exp\left(\frac{-{cot}^2\theta_{h}}{\alpha^2}\right)}{\sin^4{\theta_{h}}})</span><span><span><span></span><span><span>D</span><span><span><span><span><span><span></span><span><span><span>v</span><span>e</span><span>l</span><span>v</span><span>e</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span><span>(</span><span>1</span><span></span><span>+</span><span></span><span>4</span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>(</span><span>1</span><span></span><span>+</span><span></span></span><span><span></span><span>4</span><span><span></span><span><span><span><span><span><span></span><span><span><span>sin</span><span><span><span><span><span><span></span><span><span>4</span></span></span></span></span></span></span></span><span></span><span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>h</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>e</span><span>x</span><span>p</span><span></span><span><span><span>(</span></span><span><span></span><span><span><span><span><span><span></span><span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>−</span><span><span><span>co</span><span>t</span></span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span><span>θ</span><span><span><span><span><span><span></span><span><span><span>h</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span>)</span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>)</span></span></span></span></span><p>标准化的等式中，分母可以进一步平滑为：</p><span><span><span>fr(v,h,α)=Dvelvet(v,h,α)4(n⋅l+n⋅l−(n⋅l)(n⋅v))f_{r}(v,h,\alpha) = \frac{D_{velvet}(v,h,\alpha)}{4(n \cdot l + n \cdot l - (n \cdot l)(n \cdot v))}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>r</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span></span><span>+</span><span></span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span></span><span>−</span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>))</span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>D</span><span><span><span><span><span><span></span><span><span><span>v</span><span>e</span><span>l</span><span>v</span><span>e</span><span>t</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span>α</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>GLSL 的实现如下，适配了半浮点数并避免了余切的计算，用三角函数恒等式替换，且在该 BRDF 中删去了 Fresnel 项：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> </span><span>D_Ashikhmin</span><span>(</span><span>float</span><span><span> </span><span>roughness</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>NoH</span><span>)</span></span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>// Ashikhmin 2007, "Distribution-based BRDFs"</span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span><span> a2 </span><span>=</span><span> roughness </span><span>*</span><span> roughness;</span></span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span><span> cos2h </span><span>=</span><span> NoH </span><span>*</span><span> NoH;</span></span></div></div><div><div><div>6</div></div><div><span> </span><span>float</span><span><span> sin2h </span><span>=</span><span> </span></span><span>max</span><span>(</span><span>1.0</span><span><span> </span><span>-</span><span> cos2h, </span></span><span>0.0078125</span><span>);</span><span> // 2^(-14/2), so sin2h^2 &gt; 0 in fp16</span></div></div><div><div><div>7</div></div><div><span> </span><span>float</span><span><span> sin4h </span><span>=</span><span> sin2h </span><span>*</span><span> sin2h;</span></span></div></div><div><div><div>8</div></div><div><span> </span><span>float</span><span><span> cot2 </span><span>=</span><span> </span><span>-</span><span>cos2h </span><span>/</span><span> (a2 </span><span>*</span><span> sin2h);</span></span></div></div><div><div><div>9</div></div><div><span> </span><span>return</span><span> </span><span>1.0</span><span><span> </span><span>/</span><span> (PI </span><span>*</span><span> (</span></span><span>4.0</span><span><span> </span><span>*</span><span> a2 </span><span>+</span><span> </span></span><span>1.0</span><span><span>) </span><span>*</span><span> sin4h) </span><span>*</span><span> (</span></span><span>4.0</span><span><span> </span><span>*</span><span> </span></span><span>exp</span><span><span>(cot2) </span><span>+</span><span> sin4h);</span></span></div></div><div><div><div>10</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>此外还有一种 NDF 的实现^{<a href="#user-content-fn-estevez17">8</a>}，不同于前者使用了逆高斯分布的 NDF，该分布以正弦函数的指数为基础，它的参数表达更为自然直观，效果更加柔和，被称为<em>Charlie Sheen</em>。</p><span><span><span>D(m)=(2+1α)sin⁡(θ)1α2πD(m) = \frac{(2+\frac{1}{\alpha})\sin(\theta)^\frac{1}{\alpha}}{2\pi}</span><span><span><span></span><span>D</span><span>(</span><span>m</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>2</span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span>(</span><span>2</span><span></span><span>+</span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>α</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>)</span><span></span><span>sin</span><span>(</span><span>θ</span><span><span>)</span><span><span><span><span><span><span></span><span><span><span></span><span><span><span><span><span><span></span><span><span><span>α</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>Filament 中的优化实现如下：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> </span><span>D_Charlie</span><span>(</span><span>float</span><span><span> </span><span>roughness</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>NoH</span><span>)</span></span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>// Estevez and Kulla 2017, "Production Friendly Microfacet Sheen BRDF"</span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span><span> invAlpha </span><span>=</span><span> </span></span><span>1.0</span><span><span> </span><span>/</span><span> roughness;</span></span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span><span> cos2h </span><span>=</span><span> NoH </span><span>*</span><span> NoH;</span></span></div></div><div><div><div>6</div></div><div><span> </span><span>float</span><span><span> sin2h </span><span>=</span><span> </span></span><span>max</span><span>(</span><span>1.0</span><span><span> </span><span>-</span><span> cos2h, </span></span><span>0.0078125</span><span>);</span><span> // 2^(-14/2), so sin2h^2 &gt; 0 in fp16</span></div></div><div><div><div>7</div></div><div><span> </span><span>return</span><span> (</span><span>2.0</span><span><span> </span><span>+</span><span> invAlpha) </span><span>*</span><span> </span></span><span>pow</span><span><span>(sin2h, invAlpha </span><span>*</span><span> </span></span><span>0.5</span><span><span>) </span><span>/</span><span> (</span></span><span>2.0</span><span><span> </span><span>*</span><span> PI);</span></span></div></div><div><div><div>8</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div></section><section><h3>光泽颜色<a href="#光泽颜色"><span>#</span></a></h3><p>为使美术更好控制布料材质的外观，引入了直接修改镜面反射率的<strong>光泽颜色（Sheen Color）</strong>：</p><p><img src="/assets/images/Filament/screenshot_cloth_sheen.png" alt="" /> <em>无光泽（左） vs 有光泽（右）</em></p></section><section><h3>布料的 Diffuse BRDF<a href="#布料的-diffuse-brdf"><span>#</span></a></h3><p>Filament 中的布料材质模型的漫反射项依然依赖于 Lambertian Diffuse BRDF，在布料的 diffuse BRDF 中，它被修改为能量守恒且可以提供此表面散射的，虽然这些修改并不是基于严谨的物理真实的效果修改的，但是可以在一定程度上模拟光线在布料表面的散射和吸收。</p><p>没有散射项的 Diffuse BRDF 项如下：</p><span><span><span>fd(v,h)=cdiffπ(1−F(v,h))f_{d}(v,h) = \frac{c_{diff}}{\pi}(1 - F(v,h))</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>d</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>c</span><span><span><span><span><span><span></span><span><span><span>d</span><span>i</span><span>f</span><span>f</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>(</span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span>F</span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>))</span></span></span></span></span><p>这里的 <span><span>F(v,h)F(v,h)</span><span><span><span></span><span>F</span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>)</span></span></span></span> 是布料的 Specular BRDF 的 Fresnel 项，在实践中可以选择忽略 <span><span>1−F(v,h)1 - F(v,h)</span><span><span><span></span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span>F</span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>)</span></span></span></span> 这一项，Filament 文档中认为不值得为该项徒增成本。</p><p>次表面散射的效果以 Wrapped lighting^{<a href="#user-content-fn-gpugemssss">9</a>} 的方式实现，这种方法会修改漫反射函数以使表面法线和光照方向垂直的点不全为黑，以此来提高漫反射的对比度，从而模拟光线的散射行为。Filament 中以能量守恒的形式实现：</p><span><span><span>fd(v,h)=cdiffπ(1−F(v,h))&lt;n⋅l+w(1+w)2&gt;&lt;csubsurface+n⋅l&gt;f_{d}(v,h) = \frac{c_{diff}}{\pi}(1 - F(v,h)) \left&lt; \frac{n \cdot l + w}{(1 + w)^2} \right&gt; \left&lt; c_{subsurface} + n \cdot l \right&gt;</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>d</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>c</span><span><span><span><span><span><span></span><span><span><span>d</span><span>i</span><span>f</span><span>f</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span>(</span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span>F</span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>))</span><span></span><span><span><span>⟨</span></span><span><span></span><span><span><span><span><span><span></span><span><span>(</span><span>1</span><span></span><span>+</span><span></span><span>w</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span></span><span>+</span><span></span><span>w</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span>⟩</span></span></span><span></span><span><span>⟨</span><span><span>c</span><span><span><span><span><span><span></span><span><span><span>s</span><span>u</span><span>b</span><span>s</span><span>u</span><span>r</span><span>f</span><span>a</span><span>ce</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>⟩</span></span></span></span></span></span><p>这里的 <span><span>ww</span><span><span><span></span><span>w</span></span></span></span> 即是描述漫反射光包裹几何体程度的值，介于 0 到 1 之间。为避免引入额外参数，Filament 将其固定为 <span><span>ww</span><span><span><span></span><span>w</span></span></span></span> = 0.5。Filament 里还提到一点，<strong>漫反射项不能乘以 <span><span>n⋅ln \cdot l</span><span><span><span></span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>l</span></span></span></span></strong>，我的猜测是因为 Wrapped lighting 模拟了表面的漫反射后，光线的强度<strong>不再</strong>与表面法线和光照方向的夹角直接关联，因此不能直接乘。</p><p><img src="/assets/images/Filament/screenshot_cloth_subsurface.png" alt="" /> <em>白色的布料（左）vs 具有棕色次表面散射的白色布料（右）</em></p><p>完整的布料 BRDF 实现如下：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>// specular BRDF</span></div></div><div><div><div>2</div></div><div><span>float</span><span><span> D </span><span>=</span><span> </span></span><span>distributionCloth</span><span><span>(</span><span>roughness</span><span>,</span><span> NoH</span><span>);</span></span></div></div><div><div><div>3</div></div><div><span>float</span><span><span> V </span><span>=</span><span> </span></span><span>visibilityCloth</span><span><span>(</span><span>NoV</span><span>,</span><span> NoL</span><span>);</span></span></div></div><div><div><div>4</div></div><div><span>vec3</span><span><span> F </span><span>=</span><span> sheenColor;</span></span></div></div><div><div><div>5</div></div><div><span>vec3</span><span><span> Fr </span><span>=</span><span> (D </span><span>*</span><span> V) </span><span>*</span><span> F;</span></span></div></div><div><div><div>6</div></div><div> </div></div><div><div><div>7</div></div><div><span>// diffuse BRDF</span></div></div><div><div><div>8</div></div><div><span>float</span><span><span> diffuse </span><span>=</span><span> </span></span><span>diffuse</span><span><span>(</span><span>roughness</span><span>,</span><span> NoV</span><span>,</span><span> NoL</span><span>,</span><span> LoH</span><span>);</span></span></div></div><div><div><div>9</div></div><div><span>#if</span><span> </span><span>defined</span><span>(</span><span>MATERIAL_HAS_SUBSURFACE_COLOR</span><span>)</span></div></div><div><div><div>10</div></div><div><span>// energy conservative wrap diffuse</span></div></div><div><div><div>11</div></div><div><span><span>diffuse </span><span>*=</span><span> </span></span><span>saturate</span><span>((</span><span>dot</span><span><span>(</span><span>n</span><span>,</span><span> light.l</span><span>)</span><span> </span><span>+</span><span> </span></span><span>0.5</span><span><span>)</span><span> </span><span>/</span><span> </span></span><span>2.25</span><span>);</span></div></div><div><div><div>12</div></div><div><span>#endif</span></div></div><div><div><div>13</div></div><div><span>vec3</span><span><span> Fd </span><span>=</span><span> diffuse </span><span>*</span><span> pixel.diffuseColor;</span></span></div></div><div><div><div>14</div></div><div> </div></div><div><div><div>15</div></div><div><span>#if</span><span> </span><span>defined</span><span>(</span><span>MATERIAL_HAS_SUBSURFACE_COLOR</span><span>)</span></div></div><div><div><div>16</div></div><div><span>// cheap subsurface scatter</span></div></div><div><div><div>17</div></div><div><span><span>Fd </span><span>*=</span><span> </span></span><span>saturate</span><span><span>(</span><span>subsurfaceColor </span><span>+</span><span> NoL</span><span>);</span></span></div></div><div><div><div>18</div></div><div><span><span>vec3 color </span><span>=</span><span> Fd </span><span>+</span><span> Fr </span><span>*</span><span> NoL;</span></span></div></div><div><div><div>19</div></div><div><span><span>color </span><span>*=</span><span> (lightIntensity </span><span>*</span><span> lightAttenuation) </span><span>*</span><span> lightColor;</span></span></div></div><div><div><div>20</div></div><div><span>#else</span></div></div><div><div><div>21</div></div><div><span><span>vec3 color </span><span>=</span><span> Fd </span><span>+</span><span> Fr;</span></span></div></div><div><div><div>22</div></div><div><span><span>color </span><span>*=</span><span> (lightIntensity </span><span>*</span><span> lightAttenuation </span><span>*</span><span> NoL) </span><span>*</span><span> lightColor;</span></span></div></div><div><div><div>23</div></div><div><span>#endif</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div></section><section><h3>参数<a href="#参数-2"><span>#</span></a></h3><p>布料模型不具有 <em>Metallic</em> 和 <em>Reflectance</em> 两个参数，额外添加了 <em>Sheen</em> 和 <em>SubsurfaceColor</em> 两个参数：</p> <table><thead><tr><th>参数</th><th>定义</th></tr></thead><tbody><tr><td>SheenColor</td><td>用于创建双色调镜面布料的镜面高光的颜色，默认为 0.04 以匹配标准反射率</td></tr><tr><td>SubsurfaceColor</td><td>通过材质散射和吸收后的漫反射颜色的色调</td></tr></tbody></table><p>创建类似天鹅绒的材质时，<em>BaseColor</em> 可以设为纯黑或其他较暗的颜色，色度应该反映在 <em>SheenColor</em> 上。而像牛仔布、棉布这类材质，<em>SheenColor</em> 就可以使用基于 <em>BaseColor</em> 的亮度。</p></section></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="https://www.unrealengine.com/en/blog/physically-based-shading-on-mobile" target="_blank">physically-based-shading-on-mobile</a></li> </ul></section> <section><h2>Footnotes<a href="#footnote-label"><span>#</span></a></h2> <ol> <li> <p><a href="https://media.disneyanimation.com/uploads/production/publication_asset/48/asset/s2012_pbs_disney_brdf_notes_v3.pdf" target="_blank">Physically Based Shading at Disney</a> <a href="#user-content-fnref-disney">↩</a> <a href="#user-content-fnref-disney-2">↩²</a></p> </li> <li> <p><a href="http://graphicrants.blogspot.com/2013/08/specular-brdf-reference.html" target="_blank">Specular BRDF Reference</a> <a href="#user-content-fnref-karis13a">↩</a></p> </li> <li> <p><a href="https://www.researchgate.net/publication/2378872_A_Microfacet_Based_Coupled_Specular-Matte_BRDF_Model_with_Importance_Sampling" target="_blank">A Microfacet Based Coupled Specular-Matte BRDF Model with Importance Sampling</a> <a href="#user-content-fnref-kelemen01">↩</a></p> </li> <li> <p><a href="https://blog.selfshadow.com/publications/s2013-shading-course/rad/s2013_pbs_rad_notes.pdf" target="_blank">Crafting a Next-Gen Material Pipeline for The Order: 1886</a> <a href="#user-content-fnref-neubelt13">↩</a> <a href="#user-content-fnref-neubelt13-2">↩²</a></p> </li> <li> <p><a href="https://blog.selfshadow.com/publications/s2017-shading-course/imageworks/s2017_pbs_imageworks_slides_v2.pdf" target="_blank">Revisiting Physically Based Shading at Imageworks</a> <a href="#user-content-fnref-kulla17">↩</a></p> </li> <li> <p><a href="https://www.semanticscholar.org/paper/Distribution-based-BRDFs-Ashikhmin-Premoze/c54e98f379334f881389962c8598148389db5c40" target="_blank">Distribution-based BRDFs</a> <a href="#user-content-fnref-ashikhmin07">↩</a></p> </li> <li> <p><a href="https://dl.acm.org/doi/pdf/10.1145/344779.344814" target="_blank">A microfacet-based BRDF generator</a> <a href="#user-content-fnref-ashikhmin00">↩</a></p> </li> <li> <p><a href="https://blog.selfshadow.com/publications/s2017-shading-course/imageworks/s2017_pbs_imageworks_sheen.pdf" target="_blank">Production Friendly Microfacet Sheen BRDF</a> <a href="#user-content-fnref-estevez17">↩</a></p> </li> <li> <p><a href="https://developer.nvidia.com/gpugems/gpugems/part-iii-materials/chapter-16-real-time-approximations-subsurface-scattering" target="_blank">Chapter 16. Real-Time Approximations to Subsurface Scattering</a> <a href="#user-content-fnref-gpugemssss">↩</a></p> </li> </ol> </section>https://yohiro.games/posts/filament-materials-parti/https://yohiro.games/posts/filament-materials-parti/Sat, 29 Jun 2019 00:00:00 GMT<p>本篇是 <a href="https://google.github.io/filament/Filament.html" target="_blank"><strong>Filament</strong></a> 的笔记，以及部分自己的理解。</p> <p>可以结合 Desktop 的渲染方式一起，看 Filament 的渲染为了更好地支持移动端，舍弃了哪些。</p> <p>也可以搭配 <a href="https://www.bilibili.com/video/BV1X7411F744/?share_source=copy_web&amp;vd_source=7a4dacf2c6974d860175d6d297f6d566" target="_blank">【GAMES101-现代计算机图形学入门-闫令琪】</a> 食用，风味更佳。</p> <section><h2>原则<a href="#原则"><span>#</span></a></h2><p>Filament 是用于 Android 的渲染引擎，设计原则包含以下几个方面：</p><ul> <li>性能，关注实时渲染中移动设备的性能表现，主要目标为 OpenGL ES3.x 版本的 GPU</li> <li>质量，同时兼顾中低性能的 GPU</li> <li>易用，方便美术同学直观且快速地迭代资产，因此提供易理解地参数以及物理上合理的视觉效果</li> <li>熟悉，该系统应尽可能使用物理单位，如以开尔文为单位地色温、以流明为单位的光照等</li> <li>灵活，支持非真实感渲染</li> </ul></section> <section><h2>PBR<a href="#pbr"><span>#</span></a></h2><p>选择采用 PBR 是因为它从艺术和生产效率的角度来看有好处，而且它能很好的兼容设计目标。</p><p>与传统模型相比，PBR 是一种可以更准确地表示材质及其与光的交互方式的方法。PBR 方法的核心是<code>材质和光照的分离</code>，可以创建在统一光照条件下看起来可信的资产。</p></section> <section><h2>概念<a href="#概念"><span>#</span></a></h2> <table><thead><tr><th>符号</th><th>定义</th></tr></thead><tbody><tr><td><span><span>vv</span><span><span><span></span><span>v</span></span></span></span></td><td>观察视角的单位向量</td></tr><tr><td><span><span>ll</span><span><span><span></span><span>l</span></span></span></span></td><td>入射光线的单位向量</td></tr><tr><td><span><span>nn</span><span><span><span></span><span>n</span></span></span></span></td><td>表面法线的单位向量</td></tr><tr><td><span><span>hh</span><span><span><span></span><span>h</span></span></span></span></td><td>单位半角向量</td></tr><tr><td><span><span>ff</span><span><span><span></span><span>f</span></span></span></span></td><td>BRDF</td></tr><tr><td><span><span>fdf_d</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>BRDF 的漫反射项</td></tr><tr><td><span><span>frf_r</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>BRDF 的镜面反射项</td></tr><tr><td><span><span>α\alpha</span><span><span><span></span><span>α</span></span></span></span></td><td>粗糙度</td></tr><tr><td><span><span>σ\sigma</span><span><span><span></span><span>σ</span></span></span></span></td><td>漫反射率</td></tr><tr><td><span><span>Ω\Omega</span><span><span><span></span><span>Ω</span></span></span></span></td><td>球体区域</td></tr><tr><td><span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>入射法向的反射率</td></tr><tr><td><span><span>f90f_{90}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>掠射角的反射率</td></tr><tr><td><span><span>χ+(a)\chi^+(a)</span><span><span><span></span><span><span>χ</span><span><span><span><span><span><span></span><span><span>+</span></span></span></span></span></span></span></span><span>(</span><span>a</span><span>)</span></span></span></span></td><td>阶跃函数（a&gt;0 则为 1，否则为 0）</td></tr><tr><td><span><span>niorn_{ior}</span><span><span><span></span><span><span>n</span><span><span><span><span><span><span></span><span><span><span>i</span><span>or</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></td><td>界面折射率（IOR，Index of refraction）</td></tr><tr><td><span><span>&lt;n⋅l&gt;\left&lt; n \cdot l \right&gt;</span><span><span><span></span><span><span>⟨</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>⟩</span></span></span></span></span></td><td>[0, 1] 的点积</td></tr><tr><td><span><span>&lt;a&gt;\left&lt; a \right&gt;</span><span><span><span></span><span><span>⟨</span><span>a</span><span>⟩</span></span></span></span></span></td><td>[0, 1] 的值</td></tr></tbody></table></section> <section><h2>材质系统<a href="#材质系统"><span>#</span></a></h2><blockquote><p>详见👉<a href="https://google.github.io/filament/Materials.html" target="_blank"><strong>Filament 材质指南</strong></a> 以及 <a href="https://google.github.io/filament/Material%20Properties.pdf" target="_blank"><strong>材质属性</strong></a> {: .prompt-info }</p></blockquote><section><h3>标准模型<a href="#标准模型"><span>#</span></a></h3><p>标准的材质模型通过 BSDF（双向散射分布函数）来表达，BSDF 有两个组成部分 BRDF（双向反射分布函数）以及 BTDF（双向透射函数）。 由于绝大多数材质对表面材质进行模拟，因而具有各项同性的标准材质模型会专注于 BRDF，从而忽略或近似 BTDF。</p><p>BRDF 将标准材质的表面分为：</p><ul> <li>漫反射项 <span><span>fdf_d</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></li> <li>镜面反射项 <span><span>frf_r</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></li> </ul><p></p><figure><img src="/assets/images/Filament/diagram_fr_fd.png" alt="fr_fd" /><figcaption>fr_fd</figcaption></figure> <em>BRDF 模型中的<span><span>fdf_d</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>和<span><span>frf_r</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span></em><p></p><p>完整的表达为：</p><span><span><span>f(v,l)=fd(v,l)+fr(v,l)f(v,l)=f_d(v,l)+f_r(v,l)</span><span><span><span></span><span>f</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>+</span><span></span></span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span></span></span></span></span><p>上述方程描述的是单一入射光，完整的渲染方程中将会对整个半球面上的入射光线 <span><span>ll</span><span><span><span></span><span>l</span></span></span></span> 进行积分。</p><p>通常，材质表面并非是完全光滑的，因此引入了微表面模型/微表面 BRDF </p><figure><img src="/assets/images/Filament/diagram_microfacet.png" alt="microfacet" /><figcaption>microfacet</figcaption></figure> <em>微表面模型的粗糙表面和光滑表面</em><p></p><p>在微表面，法线 N 位于入射光和观察方向之间的半角方向时会反射可见光。 </p><figure><img src="/assets/images/Filament/diagram_macrosurface.png" alt="microsurface" /><figcaption>microsurface</figcaption></figure><p></p><p>但是也并非所有符合上面条件的法线会贡献反射，因为微表面 BRDF 会考虑材质表面的遮蔽而产生的自阴影。 </p><figure><img src="/assets/images/Filament/diagram_shadowing_masking.png" alt="shadow masking" /><figcaption>shadow masking</figcaption></figure><p></p><p>粗糙度高的材质，表面朝向相机的面越少，表现为越模糊，因为入射光的能量被分散了。 </p><figure><img src="/assets/images/Filament/diagram_roughness.png" alt="roughness" /><figcaption>roughness</figcaption></figure> <em>光照对不同粗糙度的影响，从左到右表面逐渐光滑</em><p></p><p>下面的方程描述了微表面模型：</p><span><span><span>fx(v,l)=1∣n⋅v∣∣n⋅l∣∫ΩD(m,α)G(v,l,m)fm(v,l,m)(v⋅m)(l⋅m)dmf_x(v,l) = \frac{1}{|n \cdot v| |n \cdot l|} \int_\Omega D(m,\alpha) G(v,l,m) f_m(v,l,m) (v \cdot m) (l \cdot m) dm</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>x</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>∣</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>∣∣</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>∣</span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span>Ω</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>D</span><span>(</span><span>m</span><span>,</span><span></span><span>α</span><span>)</span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>m</span><span>)</span><span><span>f</span><span><span><span><span><span><span></span><span><span>m</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>m</span><span>)</span><span>(</span><span>v</span><span></span><span>⋅</span><span></span></span><span><span></span><span>m</span><span>)</span><span>(</span><span>l</span><span></span><span>⋅</span><span></span></span><span><span></span><span>m</span><span>)</span><span>d</span><span>m</span></span></span></span></span><p>其中 D 项描述微表面的法线分布，G 项对微表面的几何性质（主要是阴影和遮蔽）进行描述。主要的不同来自于对半球微表面的积分<span><span>fmf_m</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>m</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>： <img src="/assets/images/Filament/diagram_micro_vs_macro.png" alt="" /> <em>宏观层面的平面（左）和微观层面的微表面（右）</em> 在微观层面上，材质的表面并非完全平坦，就<code>无法再假设所有的入射光是平行的</code>，因此需要对半球进行积分，但对半球的完整的积分在实时渲染中不切实际，因此需要采用近似值。</p></section><section><h3>电介质和导体<a href="#电介质和导体"><span>#</span></a></h3><p>Filament 里对材质属性引入了两个概念：<em>电介质</em>和<em>导体</em>。</p><p>入射光照射到 BRDF 模拟的材质表面后，光被分解为漫反射和镜面反射两个分量，这是一种简化的模型。</p><p>实际上，会有入射光穿透表面，在材质内部进行散射，最后再以漫反射的形式离开表面： <img src="/assets/images/Filament/diagram_scattering.png" alt="" /> <em>漫反射的散射</em></p><p>这就是电介质和导体的区别。导体不会产生次表面散射，散射发生在电介质当中。</p><p><img src="/assets/images/Filament/diagram_brdf_dielectric_conductor.png" alt="" /> <em>电介质和导体表面的 BRDF 模型</em></p></section><section><h3>Specular BRDF<a href="#specular-brdf"><span>#</span></a></h3><p>在 Cook-Torrance 的微表面模型中，Specular BRDF 可描述为，</p><span><span><span>fr(v,l)=D(h,α)G(v,l,α)F(v,h,f0)4(n⋅v)(n⋅l)f_r(v,l) = \frac{D(h, \alpha) G(v, l, \alpha) F(v, h, f0)}{4 (n \cdot v)(n \cdot l)}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>D</span><span>(</span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span>F</span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span>f</span><span>0</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>在实时渲染领域常采用对 D、G、F 项的近似，<a href="http://graphicrants.blogspot.com/2013/08/specular-brdf-reference.html" target="_blank"><strong>这里</strong></a> 提供了更多关于 Specular BRDF 的参考。</p><section><h4>D 正态分布函数 (Normal Distribution Function)<a href="#d-正态分布函数-normal-distribution-function"><span>#</span></a></h4><p>正态分布函数（NDF）是描述现实世界物体表面分布的一种方式，但在实时渲染领域常用的是 Walter 描述的 GGX 分布，GGX 具有长衰减和短峰值的特点，GGX 的分布函数如下：</p><span><span><span>DGGX(h,α)=α2π((n⋅h)2(α2−1)+1)2D_{GGX}(h,\alpha) = \frac{\alpha^2}{\pi ( (n \cdot h)^2 (\alpha^2 - 1) + 1)^2}</span><span><span><span></span><span><span>D</span><span><span><span><span><span><span></span><span><span><span>GGX</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span><span>((</span><span>n</span><span></span><span>⋅</span><span></span><span>h</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>(</span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>−</span><span></span><span>1</span><span>)</span><span></span><span>+</span><span></span><span>1</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>下面是来自 UnrealEngine 中的实现，其中 a2 是<span><span>α2\alpha^2</span><span><span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>// GGX / Trowbridge-Reitz</span></div></div><div><div><div>2</div></div><div><span>// [Walter et al. 2007, "Microfacet models for refraction through rough surfaces"]</span></div></div><div><div><div>3</div></div><div><span>float</span><span> </span><span>D_GGX</span><span>( </span><span>float</span><span> a2, </span><span>float</span><span> NoH )</span></div></div><div><div><div>4</div></div><div><span>{</span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span> d = ( NoH * a2 - NoH ) * NoH + </span><span>1</span><span>; </span><span>// 2 mad</span></div></div><div><div><div>6</div></div><div><span> </span><span>return</span><span> a2 / ( PI*d*d ); </span><span>// 4 mul, 1 rcp</span></div></div><div><div><div>7</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>一个常见的优化手段是使用半精度的浮点数，即<code>half</code>类型进行计算。因为公式展开中的 <span><span>1−(n⋅h)21-(n \cdot h)^2</span><span><span><span></span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>h</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span> 项存在<code>精度问题</code>：</p><ul> <li>高光情况下，即当 <span><span>(n⋅h)2(n \cdot h)^2</span><span><span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>h</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span> 接近 1 时，该项会因为浮点数的差值计算问题被截断，导致结果为零。</li> <li><span><span>n⋅hn \cdot h</span><span><span><span></span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>h</span></span></span></span>本身在接近 1 时缺少足够的精度。</li> </ul><p>为避免精度造成的问题，可以用叉积的展开式代换，</p><span><span><span>∣a×b∣2=∣a∣2∣b∣2−(a⋅b)2| a \times b |^2 = |a|^2 |b|^2 - (a \cdot b)^2</span><span><span><span></span><span>∣</span><span>a</span><span></span><span>×</span><span></span></span><span><span></span><span>b</span><span><span>∣</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>∣</span><span>a</span><span><span>∣</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>∣</span><span>b</span><span><span>∣</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>−</span><span></span></span><span><span></span><span>(</span><span>a</span><span></span><span>⋅</span><span></span></span><span><span></span><span>b</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span><p>由于 <span><span>nn</span><span><span><span></span><span>n</span></span></span></span> 和 <span><span>ll</span><span><span><span></span><span>l</span></span></span></span> 是单位向量，便有</p><span><span><span>∣n×h∣2=1−(n⋅h)2|n \times h|^2 = 1 - (n \cdot h)^2</span><span><span><span></span><span>∣</span><span>n</span><span></span><span>×</span><span></span></span><span><span></span><span>h</span><span><span>∣</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>h</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span><p>这样一来，我们便可以直接使用叉积来直接计算 <span><span>1−(n⋅h)21-(n \cdot h)^2</span><span><span><span></span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>h</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></p><p>Filament 中的实现如下</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>#define</span><span> </span><span>MEDIUMP_FLT_MAX</span><span> </span><span>65504.0</span></div></div><div><div><div>2</div></div><div><span>#define</span><span> </span><span>saturateMediump</span><span><span>(</span><span>x</span><span>) </span></span><span>min</span><span><span>(x</span><span>,</span><span> MEDIUMP_FLT_MAX)</span></span></div></div><div><div><div>3</div></div><div> </div></div><div><div><div>4</div></div><div><span>float</span><span> </span><span>D_GGX</span><span>(</span><span>float</span><span><span> </span><span>roughness</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>NoH</span><span>,</span><span> </span></span><span>const</span><span><span> vec3 </span><span>n</span><span>,</span><span> </span></span><span>const</span><span><span> vec3 </span><span>h</span><span>) {</span></span></div></div><div><div><div>5</div></div><div><span><span> </span></span><span>vec3 NxH </span><span>=</span><span> </span><span>cross</span><span>(n, h);</span></div></div><div><div><div>6</div></div><div><span> </span><span>float</span><span><span> a </span><span>=</span><span> NoH </span><span>*</span><span> roughness;</span></span></div></div><div><div><div>7</div></div><div><span> </span><span>float</span><span><span> k </span><span>=</span><span> roughness </span><span>/</span><span> (</span></span><span>dot</span><span><span>(NxH, NxH) </span><span>+</span><span> a </span><span>*</span><span> a);</span></span></div></div><div><div><div>8</div></div><div><span> </span><span>float</span><span><span> d </span><span>=</span><span> k </span><span>*</span><span> k </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>/</span><span> PI);</span></span></div></div><div><div><div>9</div></div><div><span> </span><span>return</span><span> </span><span>saturateMediump</span><span>(d);</span></div></div><div><div><div>10</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div></section><section><h4>G 几何阴影（Geometric Shadowing）<a href="#g-几何阴影geometric-shadowing"><span>#</span></a></h4><p>根据 <em>Heitz 2014, “Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs”</em>^{<a href="#user-content-fn-heitz14">1</a>}，Filament 使用的 Smith 几何阴影公式如下：</p><span><span><span>G(v,l,α)=G1(l,α)G1(v,α)G(v,l,\alpha) = G_1(l,\alpha) G_1(v,\alpha)</span><span><span><span></span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span>G</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span><span>G</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>α</span><span>)</span></span></span></span></span><p>其中 <span><span>G1G_1</span><span><span><span></span><span><span>G</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 可使用多种模型，实时渲染中常使用 GGX 公式，</p><span><span><span>G1(v,α)=GGGX(v,α)=2(n⋅v)n⋅v+α2+(1−α2)(n⋅v)2G_1(v,\alpha) = G_{GGX}(v,\alpha) = \frac{2 (n \cdot v)}{n \cdot v + \sqrt{\alpha^2 + (1 - \alpha^2) (n \cdot v)^2}}</span><span><span><span></span><span><span>G</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span>G</span><span><span><span><span><span><span></span><span><span><span>GGX</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span></span><span>+</span><span></span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>2</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>完整版即为，</p><span><span><span>G(v,l,α)=2(n⋅l)n⋅l+α2+(1−α2)(n⋅l)22(n⋅v)n⋅v+α2+(1−α2)(n⋅v)2G(v,l,\alpha) = \frac{2 (n \cdot l)}{n \cdot l + \sqrt{\alpha^2 + (1 - \alpha^2) (n \cdot l)^2}} \frac{2 (n \cdot v)}{n \cdot v + \sqrt{\alpha^2 + (1 - \alpha^2) (n \cdot v)^2}}</span><span><span><span></span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span></span><span>+</span><span></span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>2</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span></span><span><span><span><span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span></span><span>+</span><span></span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>2</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>注意到 <span><span>G(v,l,α)G(v,l,\alpha)</span><span><span><span></span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span></span></span></span> 的分子为 <span><span>4(n⋅l)(n⋅v)4(n \cdot l) (n \cdot v)</span><span><span><span></span><span>4</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>l</span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>v</span><span>)</span></span></span></span> 这里再贴一次我们所使用的 specular BRDF，</p><span><span><span>fr(v,l)=D(h,α)G(v,l,α)F(v,h,f0)4(n⋅v)(n⋅l)f_r(v,l) = \frac{D(h, \alpha) G(v, l, \alpha) F(v, h, f0)}{4 (n \cdot v)(n \cdot l)}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>D</span><span>(</span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span>F</span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span>f</span><span>0</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>通过引入可见性函数 Visibility 项 <span><span>V(v,l,α)V(v,l,\alpha)</span><span><span><span></span><span>V</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span></span></span></span>，将 <span><span>frf_r</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 变为：</p><span><span><span>fr(v,l)=D(h,α)V(v,l,α)F(v,h,f0)f_r(v,l) = D(h, \alpha) V(v, l, \alpha) F(v, h, f_0)</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>r</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span>D</span><span>(</span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span>V</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span>F</span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span></span></span></span></span><p>其中</p><span><span><span>V(v,l,α)=G(v,l,α)4(n⋅v)(n⋅l)=V1(l,α)V1(v,α)V(v,l,\alpha) = \frac{G(v, l, \alpha)}{4 (n \cdot v) (n \cdot l)} = V_1(l,\alpha) V_1(v,\alpha)</span><span><span><span></span><span>V</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>4</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span>V</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span><span>V</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>α</span><span>)</span></span></span></span></span><p>便可消去分子，得到</p><span><span><span>V1(v,α)=1n⋅v+α2+(1−α2)(n⋅v)2V_1(v,\alpha) = \frac{1}{n \cdot v + \sqrt{\alpha^2 + (1 - \alpha^2) (n \cdot v)^2}}</span><span><span><span></span><span><span>V</span><span><span><span><span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span></span><span>+</span><span></span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>论文指出，通过引入微表面的高度来建模可以得到更好的结果。引入了高度<span><span>hh</span><span><span><span></span><span>h</span></span></span></span>的 Smith 函数：</p><span><span><span>G(v,l,h,α)=χ+(v⋅h)χ+(l⋅h)1+Λ(v)+Λ(l)G(v,l,h,\alpha) = \frac{\chi^+(v \cdot h) \chi^+(l \cdot h)}{1 + \Lambda(v) + \Lambda(l)}</span><span><span><span></span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>1</span><span></span><span>+</span><span></span><span>Λ</span><span>(</span><span>v</span><span>)</span><span></span><span>+</span><span></span><span>Λ</span><span>(</span><span>l</span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>χ</span><span><span><span><span><span><span></span><span><span>+</span></span></span></span></span></span></span></span><span>(</span><span>v</span><span></span><span>⋅</span><span></span><span>h</span><span>)</span><span><span>χ</span><span><span><span><span><span><span></span><span><span>+</span></span></span></span></span></span></span></span><span>(</span><span>l</span><span></span><span>⋅</span><span></span><span>h</span><span>)</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><span><span><span>Λ(m)=−1+1+α2tan2(θm)2=−1+1+α2(1−cos2(θm))cos2(θm)2\Lambda(m) = \frac{-1 + \sqrt{1 + \alpha^2 tan^2(\theta_m)}}{2} = \frac{-1 + \sqrt{1 + \alpha^2 \frac{(1 - cos^2(\theta_m))}{cos^2(\theta_m)}}}{2}</span><span><span><span></span><span>Λ</span><span>(</span><span>m</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>2</span></span></span><span><span></span><span></span></span><span><span></span><span><span>−</span><span>1</span><span></span><span>+</span><span></span><span><span><span><span><span><span></span><span><span>1</span><span></span><span>+</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>t</span><span>a</span><span><span>n</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>(</span><span><span>θ</span><span><span><span><span><span><span></span><span><span>m</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>2</span></span></span><span><span></span><span></span></span><span><span></span><span><span>−</span><span>1</span><span></span><span>+</span><span></span><span><span><span><span><span><span></span><span><span>1</span><span></span><span>+</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>co</span><span><span>s</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>(</span><span><span>θ</span><span><span><span><span><span><span></span><span><span>m</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>(</span><span>1</span><span>−</span><span>co</span><span><span>s</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>(</span><span><span>θ</span><span><span><span><span><span><span></span><span><span>m</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>))</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>其中<span><span>θm\theta_m</span><span><span><span></span><span><span>θ</span><span><span><span><span><span><span></span><span><span>m</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>是镜面法线<span><span>nn</span><span><span><span></span><span>n</span></span></span></span>与观察方向<span><span>vv</span><span><span><span></span><span>v</span></span></span></span>的夹角，因此有<span><span>cos(θm)=n⋅vcos(\theta_m) = n \cdot v</span><span><span><span></span><span>cos</span><span>(</span><span><span>θ</span><span><span><span><span><span><span></span><span><span>m</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span>n</span><span></span><span>⋅</span><span></span></span><span><span></span><span>v</span></span></span></span>，代换后得到</p><span><span><span>Λ(v)=12(α2+(1−α2)(n⋅v)2n⋅v−1)\Lambda(v) = \frac{1}{2} \left( \frac{\sqrt{\alpha^2 + (1 - \alpha^2)(n \cdot v)^2}}{n \cdot v} - 1 \right)</span><span><span><span></span><span>Λ</span><span>(</span><span>v</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>2</span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span><span><span>(</span></span><span><span></span><span><span><span><span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>v</span></span></span><span><span></span><span></span></span><span><span></span><span><span><span><span><span><span><span></span><span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span>−</span><span></span><span>1</span><span><span>)</span></span></span></span></span></span></span><p>由此得出可见性函数，</p><span><span><span>V(v,l,α)=0.5n⋅l(n⋅v)2(1−α2)+α2+n⋅v(n⋅l)2(1−α2)+α2V(v,l,\alpha) = \frac{0.5}{n \cdot l \sqrt{(n \cdot v)^2 (1 - \alpha^2) + \alpha^2} + n \cdot v \sqrt{(n \cdot l)^2 (1 - \alpha^2) + \alpha^2}}</span><span><span><span></span><span>V</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span><span><span><span><span><span></span><span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span></span><span>+</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span><span>+</span><span></span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span><span><span><span><span><span></span><span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>(</span><span>1</span><span></span><span>−</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span></span><span>+</span><span></span><span><span>α</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>0.5</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>Unreal 中的实现如下：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>// [Heitz 2014, "Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs"]</span></div></div><div><div><div>2</div></div><div><span>float</span><span> </span><span>Vis_SmithJoint</span><span>(</span><span>float</span><span> a2, </span><span>float</span><span> NoV, </span><span>float</span><span> NoL)</span></div></div><div><div><div>3</div></div><div><span>{</span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span> Vis_SmithV = NoL * </span><span>sqrt</span><span>(NoV * (NoV - NoV * a2) + a2);</span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span> Vis_SmithL = NoV * </span><span>sqrt</span><span>(NoL * (NoL - NoL * a2) + a2);</span></div></div><div><div><div>6</div></div><div><span> </span><span>return</span><span> </span><span>0.5</span><span> * </span><span>rcp</span><span>(Vis_SmithV + Vis_SmithL);</span></div></div><div><div><div>7</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>考虑到根号下都是平方项，且每项∈[0,1]，于是可优化为：</p><span><span><span>V(v,l,α)=0.5n⋅l(n⋅v(1−α)+α)+n⋅v(n⋅l(1−α)+α)V(v,l,\alpha) = \frac{0.5}{n \cdot l (n \cdot v (1 - \alpha) + \alpha) + n \cdot v (n \cdot l (1 - \alpha) + \alpha)}</span><span><span><span></span><span>V</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>(</span><span>1</span><span></span><span>−</span><span></span><span>α</span><span>)</span><span></span><span>+</span><span></span><span>α</span><span>)</span><span></span><span>+</span><span></span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>(</span><span>1</span><span></span><span>−</span><span></span><span>α</span><span>)</span><span></span><span>+</span><span></span><span>α</span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>0.5</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>虽然在数学上是错的，但对于移动设备的实时渲染是足够的。Filament 中的实现如下：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> </span><span>V_SmithGGXCorrelatedFast</span><span>(</span><span>float</span><span><span> </span><span>NoV</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>NoL</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>roughness</span><span>) {</span></span></div></div><div><div><div>2</div></div><div><span> </span><span>float</span><span><span> a </span><span>=</span><span> roughness;</span></span></div></div><div><div><div>3</div></div><div><span> </span><span>float</span><span><span> GGXV </span><span>=</span><span> NoL </span><span>*</span><span> (NoV </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>-</span><span> a) </span><span>+</span><span> a);</span></span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span><span> GGXL </span><span>=</span><span> NoV </span><span>*</span><span> (NoL </span><span>*</span><span> (</span></span><span>1.0</span><span><span> </span><span>-</span><span> a) </span><span>+</span><span> a);</span></span></div></div><div><div><div>5</div></div><div><span> </span><span>return</span><span> </span><span>0.5</span><span><span> </span><span>/</span><span> (GGXV </span><span>+</span><span> GGXL);</span></span></div></div><div><div><div>6</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>[Hammon17] 提出了相似的优化思路，通过插值来实现：</p><span><span><span>V(v,l,α)=0.5lerp(2(n⋅l)(n⋅v),(n⋅l)+(n⋅v),α)V(v,l,\alpha) = \frac{0.5}{lerp(2 (n \cdot l) (n \cdot v), (n \cdot l) + (n \cdot v), \alpha)}</span><span><span><span></span><span>V</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>α</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>l</span><span>er</span><span>p</span><span>(</span><span>2</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>)</span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>)</span><span>,</span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>)</span><span></span><span>+</span><span></span><span>(</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>)</span><span>,</span><span></span><span>α</span><span>)</span></span></span><span><span></span><span></span></span><span><span></span><span><span>0.5</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span></section><section><h4>F 菲涅尔（Fresnel）<a href="#f-菲涅尔fresnel"><span>#</span></a></h4><p>菲涅尔项定义了<code>光在两种不同介质的交界处如何处理反射和折射</code>，或者说<code>反射的能量与透射的能量的比率</code>。</p><p>反射光的强度不仅取决于视角，还取决于材质的折射率 IOR。将入射光线垂直于表面时（Normal）反射率记为<span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>，掠射角（Grazing）反射率记为<span><span>f90f_{90}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>。根据 [Schlick94] 描述，在 Cook-Torrance 的微表面模型中，Specular BRDF 的菲涅尔项的一种近似可写为：</p><span><span><span>FSchlick(v,h,f0,f90)=f0+(f90−f0)(1−v⋅h)5F_{Schlick}(v,h,f_0,f_{90}) = f_0 + (f_{90} - f_0)(1 - v \cdot h)^5</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span><span>S</span><span>c</span><span>h</span><span>l</span><span>i</span><span>c</span><span>k</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>h</span><span>,</span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>,</span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span></span><span><span></span><span>(</span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>−</span><span></span></span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span><span>(</span><span>1</span><span></span><span>−</span><span></span></span><span><span></span><span>v</span><span></span><span>⋅</span><span></span></span><span><span></span><span>h</span><span><span>)</span><span><span><span><span><span><span></span><span><span>5</span></span></span></span></span></span></span></span></span></span></span></span><p>Unreal 的实现如下：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float3</span><span> </span><span>F_Schlick</span><span>(</span><span>float3</span><span> F0, </span><span>float3</span><span> F90, </span><span>float</span><span> VoH)</span></div></div><div><div><div>2</div></div><div><span>{</span></div></div><div><div><div>3</div></div><div><span> </span><span>float</span><span> Fc = </span><span>Pow5</span><span>(</span><span>1</span><span> - VoH);</span></div></div><div><div><div>4</div></div><div><span> </span><span>return</span><span> F90 * Fc + (</span><span>1</span><span> - Fc) * F0;</span></div></div><div><div><div>5</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>该菲涅尔函数可当作入射反射率和掠射角反射率间的插值，可以取<span><span>f90f_{90}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>为 1.0 来达到近似。</p></section></section><section><h3>Diffuse BRDF<a href="#diffuse-brdf"><span>#</span></a></h3><p>漫反射中常用 Lambertian 函数，漫反射的 BRDF：</p><span><span><span>fd(v,l)=σπ1∣n⋅v∣∣n⋅l∣∫ΩD(m,α)G(v,l,m)(v⋅m)(l⋅m)dmf_d(v,l) = \frac{\sigma}{\pi} \frac{1}{| n \cdot v | | n \cdot l |} \int_\Omega D(m,\alpha) G(v,l,m) (v \cdot m) (l \cdot m) dm</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span>σ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span></span><span><span><span><span><span><span></span><span><span>∣</span><span>n</span><span></span><span>⋅</span><span></span><span>v</span><span>∣∣</span><span>n</span><span></span><span>⋅</span><span></span><span>l</span><span>∣</span></span></span><span><span></span><span></span></span><span><span></span><span><span>1</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span><span>∫</span><span><span><span><span><span><span></span><span><span>Ω</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>D</span><span>(</span><span>m</span><span>,</span><span></span><span>α</span><span>)</span><span>G</span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>m</span><span>)</span><span>(</span><span>v</span><span></span><span>⋅</span><span></span></span><span><span></span><span>m</span><span>)</span><span>(</span><span>l</span><span></span><span>⋅</span><span></span></span><span><span></span><span>m</span><span>)</span><span>d</span><span>m</span></span></span></span></span><p>Filament 中的实现，假定微表面半球面产生均一的漫反射，因此一个简单的 Lambertian BRDF 为</p><span><span><span>fd(v,l)=σπf_d(v,l) = \frac{\sigma}{\pi}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span>σ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>实现也非常简单，</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> </span><span>Fd_Lambert</span><span>() {</span></div></div><div><div><div>2</div></div><div><span> </span><span>return</span><span> </span><span>1.0</span><span><span> </span><span>/</span><span> PI;</span></span></div></div><div><div><div>3</div></div><div><span>}</span></div></div><div><div><div>4</div></div><div> </div></div><div><div><div>5</div></div><div><span>vec3</span><span><span> Fd </span><span>=</span><span> diffuseColor </span><span>*</span><span> </span></span><span>Fd_Lambert</span><span>();</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>迪士尼的 BRDF 和 Oren-Nayar 模型都考虑到了粗糙度的影响，并会在掠射角出产生细微的逆反射。迪士尼的 Diffuse BRDF 如下：</p><span><span><span>fd(v,l)=σπFSchlick(n,l,1,f90)FSchlick(n,v,1,f90)f_d(v,l) = \frac{\sigma}{\pi} F_{Schlick}(n,l,1, f_{90}) F_{Schlick}(n,v,1,f_{90})</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>v</span><span>,</span><span></span><span>l</span><span>)</span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>π</span></span></span><span><span></span><span></span></span><span><span></span><span><span>σ</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span><span>F</span><span><span><span><span><span><span></span><span><span><span>S</span><span>c</span><span>h</span><span>l</span><span>i</span><span>c</span><span>k</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>n</span><span>,</span><span></span><span>l</span><span>,</span><span></span><span>1</span><span>,</span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span><span><span>F</span><span><span><span><span><span><span></span><span><span><span>S</span><span>c</span><span>h</span><span>l</span><span>i</span><span>c</span><span>k</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>(</span><span>n</span><span>,</span><span></span><span>v</span><span>,</span><span></span><span>1</span><span>,</span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span></span></span></span></span><p>其中</p><span><span><span>f90=0.5+2⋅αcos2(θd)f_{90}=0.5 + 2 \cdot \alpha cos^2(\theta_d)</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>0.5</span><span></span><span>+</span><span></span></span><span><span></span><span>2</span><span></span><span>⋅</span><span></span></span><span><span></span><span>α</span><span>co</span><span><span>s</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span><span>(</span><span><span>θ</span><span><span><span><span><span><span></span><span><span>d</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span>)</span></span></span></span></span><p>Unreal 中对这两种模型的 Diffuse BRDF 的实现：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>// [Burley 2012, "Physically-Based Shading at Disney"]</span></div></div><div><div><div>2</div></div><div><span>float3</span><span> </span><span>Diffuse_Burley</span><span>( </span><span>float3</span><span> DiffuseColor, </span><span>float</span><span> Roughness, </span><span>float</span><span> NoV, </span><span>float</span><span> NoL, </span><span>float</span><span> VoH )</span></div></div><div><div><div>3</div></div><div><span>{</span></div></div><div><div><div>4</div></div><div><span> </span><span>float</span><span> FD90 = </span><span>0.5</span><span> + </span><span>2</span><span> * VoH * VoH * Roughness;</span></div></div><div><div><div>5</div></div><div><span> </span><span>float</span><span> FdV = </span><span>1</span><span> + (FD90 - </span><span>1</span><span>) * </span><span>Pow5</span><span>( </span><span>1</span><span> - NoV );</span></div></div><div><div><div>6</div></div><div><span> </span><span>float</span><span> FdL = </span><span>1</span><span> + (FD90 - </span><span>1</span><span>) * </span><span>Pow5</span><span>( </span><span>1</span><span> - NoL );</span></div></div><div><div><div>7</div></div><div><span> </span><span>return</span><span> DiffuseColor * ( (</span><span>1</span><span> / PI) * FdV * FdL );</span></div></div><div><div><div>8</div></div><div><span>}</span></div></div><div><div><div>9</div></div><div> </div></div><div><div><div>10</div></div><div><span>// [Gotanda 2012, "Beyond a Simple Physically Based Blinn-Phong Model in Real-Time"]</span></div></div><div><div><div>11</div></div><div><span>float3</span><span> </span><span>Diffuse_OrenNayar</span><span>( </span><span>float3</span><span> DiffuseColor, </span><span>float</span><span> Roughness, </span><span>float</span><span> NoV, </span><span>float</span><span> NoL, </span><span>float</span><span> VoH )</span></div></div><div><div><div>12</div></div><div><span>{</span></div></div><div><div><div>13</div></div><div><span> </span><span>float</span><span> a = Roughness * Roughness;</span></div></div><div><div><div>14</div></div><div><span> </span><span>float</span><span> s = a;</span><span>// / ( 1.29 + 0.5 * a );</span></div></div><div><div><div>15</div></div><div><span> </span><span>float</span><span> s2 = s * s;</span></div></div><div><div><div>16</div></div><div><span> </span><span>float</span><span> VoL = </span><span>2</span><span> * VoH * VoH - </span><span>1</span><span>; </span><span>// double angle identity</span></div></div><div><div><div>17</div></div><div><span> </span><span>float</span><span> Cosri = VoL - NoV * NoL;</span></div></div><div><div><div>18</div></div><div><span> </span><span>float</span><span> C1 = </span><span>1</span><span> - </span><span>0.5</span><span> * s2 / (s2 + </span><span>0.33</span><span>);</span></div></div><div><div><div>19</div></div><div><span> </span><span>float</span><span> C2 = </span><span>0.45</span><span> * s2 / (s2 + </span><span>0.09</span><span>) * Cosri * ( Cosri &gt;= </span><span>0</span><span> ? </span><span>rcp</span><span>( </span><span>max</span><span>( NoL, NoV ) ) : </span><span>1</span><span> );</span></div></div><div><div><div>20</div></div><div><span> </span><span>return</span><span> DiffuseColor / PI * ( C1 + C2 ) * ( </span><span>1</span><span> + Roughness * </span><span>0.5</span><span> );</span></div></div><div><div><div>21</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div><p>Lambertian diffuse BRDF 和 Disney diffuse BRDF 的效果对比。从最左侧边缘可以看出，Disney 的模型在掠射角有细微的不同。</p><p><img src="/assets/images/Filament/diagram_lambert_vs_disney.png" alt="" /> <em>Lambertian diffuse BRDF（左）和 Disney diffuse BRDF（右）</em></p></section><section><h3>标准模型总结<a href="#标准模型总结"><span>#</span></a></h3><p>镜面反射项 : Cook-Torrance 镜面反射微表面模型/GGX 正态分布函数/Smith-GGX 高度相关可见性函数/Schlick Fresnel 函数</p><p>漫反射项 : Lambert 漫反射模型</p><p>标准模型的 GLSL 实现：</p><div><div><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> </span><span>D_GGX</span><span>(</span><span>float</span><span><span> </span><span>NoH</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>a</span><span>) {</span></span></div></div><div><div><div>2</div></div><div><span> </span><span>float</span><span><span> a2 </span><span>=</span><span> a </span><span>*</span><span> a;</span></span></div></div><div><div><div>3</div></div><div><span> </span><span>float</span><span><span> f </span><span>=</span><span> (NoH </span><span>*</span><span> a2 </span><span>-</span><span> NoH) </span><span>*</span><span> NoH </span><span>+</span><span> </span></span><span>1.0</span><span>;</span></div></div><div><div><div>4</div></div><div><span> </span><span>return</span><span><span> a2 </span><span>/</span><span> (PI </span><span>*</span><span> f </span><span>*</span><span> f);</span></span></div></div><div><div><div>5</div></div><div><span>}</span></div></div><div><div><div>6</div></div><div> </div></div><div><div><div>7</div></div><div><span>vec3</span><span> </span><span>F_Schlick</span><span>(</span><span>float</span><span><span> </span><span>u</span><span>,</span><span> vec3 </span><span>f0</span><span>) {</span></span></div></div><div><div><div>8</div></div><div><span> </span><span>return</span><span><span> f0 </span><span>+</span><span> (</span></span><span>vec3</span><span>(</span><span>1.0</span><span><span>) </span><span>-</span><span> f0) </span><span>*</span><span> </span></span><span>pow</span><span>(</span><span>1.0</span><span><span> </span><span>-</span><span> u, </span></span><span>5.0</span><span>);</span></div></div><div><div><div>9</div></div><div><span>}</span></div></div><div><div><div>10</div></div><div> </div></div><div><div><div>11</div></div><div><span>float</span><span> </span><span>V_SmithGGXCorrelated</span><span>(</span><span>float</span><span><span> </span><span>NoV</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>NoL</span><span>,</span><span> </span></span><span>float</span><span><span> </span><span>a</span><span>) {</span></span></div></div><div><div><div>12</div></div><div><span> </span><span>float</span><span><span> a2 </span><span>=</span><span> a </span><span>*</span><span> a;</span></span></div></div><div><div><div>13</div></div><div><span> </span><span>float</span><span><span> GGXL </span><span>=</span><span> NoV </span><span>*</span><span> </span></span><span>sqrt</span><span><span>((</span><span>-</span><span>NoL </span><span>*</span><span> a2 </span><span>+</span><span> NoL) </span><span>*</span><span> NoL </span><span>+</span><span> a2);</span></span></div></div><div><div><div>14</div></div><div><span> </span><span>float</span><span><span> GGXV </span><span>=</span><span> NoL </span><span>*</span><span> </span></span><span>sqrt</span><span><span>((</span><span>-</span><span>NoV </span><span>*</span><span> a2 </span><span>+</span><span> NoV) </span><span>*</span><span> NoV </span><span>+</span><span> a2);</span></span></div></div><div><div><div>15</div></div><div><span> </span><span>return</span><span> </span><span>0.5</span><span><span> </span><span>/</span><span> (GGXV </span><span>+</span><span> GGXL);</span></span></div></div><div><div><div>16</div></div><div><span>}</span></div></div><div><div><div>17</div></div><div> </div></div><div><div><div>18</div></div><div><span>float</span><span> </span><span>Fd_Lambert</span><span>() {</span></div></div><div><div><div>19</div></div><div><span> </span><span>return</span><span> </span><span>1.0</span><span><span> </span><span>/</span><span> PI;</span></span></div></div><div><div><div>20</div></div><div><span>}</span></div></div><div><div><div>21</div></div><div> </div></div><div><div><div>22</div></div><div><span>void</span><span> </span><span>BRDF</span><span><span>(</span><span>...</span><span>) {</span></span></div></div><div><div><div>23</div></div><div><span><span> </span></span><span>vec3 h </span><span>=</span><span> </span><span>normalize</span><span><span>(v </span><span>+</span><span> l);</span></span></div></div><div><div><div>24</div></div><div> </div></div><div><div><div>25</div></div><div><span> </span><span>float</span><span><span> NoV </span><span>=</span><span> </span></span><span>abs</span><span>(</span><span>dot</span><span><span>(n, v)) </span><span>+</span><span> </span></span><span><span>1</span><span>e</span></span><span>-</span><span>5</span><span>;</span></div></div><div><div><div>26</div></div><div><span> </span><span>float</span><span><span> NoL </span><span>=</span><span> </span></span><span>clamp</span><span>(</span><span>dot</span><span>(n, l), </span><span>0.0</span><span>, </span><span>1.0</span><span>);</span></div></div><div><div><div>27</div></div><div><span> </span><span>float</span><span><span> NoH </span><span>=</span><span> </span></span><span>clamp</span><span>(</span><span>dot</span><span>(n, h), </span><span>0.0</span><span>, </span><span>1.0</span><span>);</span></div></div><div><div><div>28</div></div><div><span> </span><span>float</span><span><span> LoH </span><span>=</span><span> </span></span><span>clamp</span><span>(</span><span>dot</span><span>(l, h), </span><span>0.0</span><span>, </span><span>1.0</span><span>);</span></div></div><div><div><div>29</div></div><div> </div></div><div><div><div>30</div></div><div><span> </span><span>// perceptually linear roughness to roughness (see parameterization)</span></div></div><div><div><div>31</div></div><div><span> </span><span>float</span><span><span> roughness </span><span>=</span><span> perceptualRoughness </span><span>*</span><span> perceptualRoughness;</span></span></div></div><div><div><div>32</div></div><div> </div></div><div><div><div>33</div></div><div><span> </span><span>float</span><span><span> D </span><span>=</span><span> </span></span><span>D_GGX</span><span>(NoH, a);</span></div></div><div><div><div>34</div></div><div><span><span> </span></span><span>vec3 F </span><span>=</span><span> </span><span>F_Schlick</span><span>(LoH, f0);</span></div></div><div><div><div>35</div></div><div><span> </span><span>float</span><span><span> V </span><span>=</span><span> </span></span><span>V_SmithGGXCorrelated</span><span>(NoV, NoL, roughness);</span></div></div><div><div><div>36</div></div><div> </div></div><div><div><div>37</div></div><div><span> </span><span>// specular BRDF</span></div></div><div><div><div>38</div></div><div><span><span> </span></span><span>vec3 Fr </span><span>=</span><span> (D </span><span>*</span><span> V) </span><span>*</span><span> F;</span></div></div><div><div><div>39</div></div><div> </div></div><div><div><div>40</div></div><div><span> </span><span>// diffuse BRDF</span></div></div><div><div><div>41</div></div><div><span><span> </span></span><span>vec3 Fd </span><span>=</span><span> diffuseColor </span><span>*</span><span> </span><span>Fd_Lambert</span><span>();</span></div></div><div><div><div>42</div></div><div> </div></div><div><div><div>43</div></div><div><span> </span><span>// apply lighting...</span></div></div><div><div><div>44</div></div><div><span>}</span></div></div></code></pre><div><div></div><div></div></div></figure><div></div></div><span>展开</span><span>收起</span></div></div></section><section><h3>提升 BRDF<a href="#提升-brdf"><span>#</span></a></h3><p>一个好的 BRDF 函数是能量守恒的，上述探讨的 BRDF 存在两个问题。</p><p>漫反射获取的能量 : Lambert 模型的 Diffuse BRDF 没有考虑表面反射的光</p><p>镜面反射损失的能量 : Cook-Torrance BRDF 在微表面上建模，但考虑的是单次光的反射，这种近似使得高粗糙度下存在能量损失，导致其表面的能量不守恒。</p><p><img src="/assets/images/Filament/diagram_single_vs_multi_scatter.png" alt="" /> <em>单次反射光与多重散射</em></p><p>基于此，可以说，表面越粗糙，产生的多重散射越多，从而能量损失的越多。这种能量的损失带来的结果便是材质会变暗，金属表面更易受到这种影响，因为金属材质的反射都是镜面反射，参见下图的对比： <img src="/assets/images/Filament/material_metallic_energy_loss.png" alt="" /> <em>仅考虑了单次散射的金属材质</em></p><p><img src="/assets/images/Filament/material_metallic_energy_preservation.png" alt="" /> <em>考虑了多重散射的金属材质</em></p></section><section><h3>参数<a href="#参数"><span>#</span></a></h3><p><a href="https://media.disneyanimation.com/uploads/production/publication_asset/48/asset/s2012_pbs_disney_brdf_notes_v3.pdf" target="_blank"><strong>迪士尼的材质模型</strong></a>^{<a href="#user-content-fn-disney">2</a>} 包含* baseColor*、<em>subsurface</em>、<em>metallic</em>、<em>specular</em>、<em>specularTint</em>、<em>roughness</em>、<em>anisotropic</em>、<em>sheen</em>、<em>sheenTint</em>、<em>clearcoat</em>、*clearcoatGloss *共 11 项，考虑到实时渲染的性能要求以及方便美术同学和开发同学使用，因此，Filament 使用了简化模型。</p> <table><thead><tr><th>参数</th><th>定义</th></tr></thead><tbody><tr><td>BaseColor</td><td>非金属材质表面的漫反射<a href="https://zh.wikipedia.org/wiki/%E5%8F%8D%E7%85%A7%E7%8E%87" target="_blank">反照率</a>和金属材质表面的镜面颜色</td></tr><tr><td>Metallic</td><td>表面是电介质（0.0）或导体（1.0）</td></tr><tr><td>Roughness</td><td>表面的粗糙度</td></tr><tr><td>Reflectance</td><td>电介质表面法向入射<span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>时的菲涅耳反射率</td></tr><tr><td>Emissive</td><td>模拟自发光表面额外的漫反射反照率，常见于具有泛光效果的 HDR 管线中</td></tr><tr><td>Ambient Occlusion</td><td>定义材质表面某点半球面上接收的环境光量，是每像素阴影系数</td></tr></tbody></table><p><img src="/assets/images/Filament/material_parameters.png" alt="" /> <em>从上到下：不同的金属度、不同电介质粗糙度、不同的金属粗糙度、不同的反射率</em></p> <table><thead><tr><th>参数</th><th>类型和范围</th></tr></thead><tbody><tr><td>BaseColor</td><td>[0,1] 的 Linear RGB</td></tr><tr><td>Metallic</td><td>[0,1] 的标量</td></tr><tr><td>Roughness</td><td>[0,1] 的标量</td></tr><tr><td>Reflectance</td><td>[0,1] 的标量</td></tr><tr><td>Emissive</td><td>[0,1] 的 Linear RGB + 曝光补偿</td></tr><tr><td>Ambient Occlusion</td><td>[0,1] 的标量</td></tr></tbody></table><p>上述的类型以及范围是对 Shader 而言的，在参数到达 Shader 之前可以用 <em>sRGB</em> 表示，在传入 Shader 前转换到 <em>linear space</em> 即可。</p></section><section><h3>重映射<a href="#重映射"><span>#</span></a></h3><p>为了使美术同学更直观地使用标准材质模型，因此引入了对 <em>baseColor</em>, <em>roughness</em>, <em>reflectance</em> 的重映射。</p><section><h4>BaseColor<a href="#basecolor"><span>#</span></a></h4><p>材质的 baseColor 会受其<code>金属程度</code>影响。电介质材质具有单一颜色的镜面反射，但会保留 baseColor 作为漫反射颜色。而导体材质使用 baseColor 作为镜面反射的颜色，没有漫反射。</p><p>因此，对于漫反射的颜色，有以下转换：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>vec3</span><span><span> diffuseColor </span><span>=</span><span> (</span></span><span>1.0</span><span><span> </span><span>-</span><span> metallic) </span><span>*</span><span> baseColor.rgb;</span></span></div></div></code></pre><div><div></div><div></div></div></figure></div></section><section><h4>Roughness<a href="#roughness"><span>#</span></a></h4><p>在 Filament 中，使用者所指定的粗糙度叫做<code>perceptualRoughness</code> 感知粗糙度，是一种直观的、经验性的值，这种粗糙度会使用下面公式映射到线性空间，</p><span><span><span>α=perceptualRoughness2\alpha = perceptualRoughness^2</span><span><span><span></span><span>α</span><span></span><span>=</span><span></span></span><span><span></span><span>p</span><span>er</span><span>ce</span><span>pt</span><span>u</span><span>a</span><span>l</span><span>R</span><span>o</span><span>ug</span><span>hn</span><span>es</span><span><span>s</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span><p><img src="/assets/images/Filament/material_roughness_remap.png" alt="" /> <em>感知线性粗糙度 (PerceptualRoughness，上）和重映射的粗糙度（<span><span>α\alpha</span><span><span><span></span><span>α</span></span></span></span>，下）</em></p><p>可见，重映射的粗糙度更方便美术同学理解。若不经重映射，光滑金属表面的值必须限制在 0.0 到 0.05 之间的小范围内。</p><p>经过平方，重映射的粗糙度给出的结果在视觉上很直观，对于实时渲染来说也很友好。但是也要注意，由于计算中经常需要 Roughness 项，因此计算时浮点数的精度问题需要予以重视。比如 <em>mediump</em> 精度的 <em>float</em> 在移动 GPU 上一般会作为半精度也就是 <em>FP16</em> 来实现。</p><p>这样就会造成问题，比如计算 GGX 项中的 <span><span>1perceptualRoughness4\frac{1}{perceptualRoughness^4}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>p</span><span>er</span><span>ce</span><span>pt</span><span>u</span><span>a</span><span>l</span><span>R</span><span>o</span><span>ug</span><span>hn</span><span>es</span><span><span>s</span><span><span><span><span><span><span></span><span><span>4</span></span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 时，由于半精度浮点数可表示的最小值为 <span><span>2−142^{-14}</span><span><span><span></span><span><span>2</span><span><span><span><span><span><span></span><span><span><span>−</span><span>14</span></span></span></span></span></span></span></span></span></span></span></span> 或 <span><span>6.1×10−56.1 × 10^{-5}</span><span><span><span></span><span>6.1</span><span></span><span>×</span><span></span></span><span><span></span><span>1</span><span><span>0</span><span><span><span><span><span><span></span><span><span><span>−</span><span>5</span></span></span></span></span></span></span></span></span></span></span></span>，在不支持非规格化的设备上为了避免除以 0，这一项的结果必须不小于 <span><span>6.1×10−56.1 × 10^{-5}</span><span><span><span></span><span>6.1</span><span></span><span>×</span><span></span></span><span><span></span><span>1</span><span><span>0</span><span><span><span><span><span><span></span><span><span><span>−</span><span>5</span></span></span></span></span></span></span></span></span></span></span></span>， 为此 Roughness 必须被 clamp 到 0.089，也就是 <span><span>6.274×10−56.274 × 10^{-5}</span><span><span><span></span><span>6.274</span><span></span><span>×</span><span></span></span><span><span></span><span>1</span><span><span>0</span><span><span><span><span><span><span></span><span><span><span>−</span><span>5</span></span></span></span></span></span></span></span></span></span></span></span>。</p><p>很多时候为了控制镜面高光处于一个更小的范围，Roughness 也需要 clamp 到一个安全的范围，对于较低的 Roughness 值，这种 clamp 还可以避免高光出现的锯齿。</p><p>关于游戏中浮点数相关的内容可以看一下<a href="https://randomascii.wordpress.com/2012/09/09/game-developer-magazine-floating-point/" target="_blank"><strong>这篇文章</strong></a>。</p></section><section><h4>Reflectance<a href="#reflectance"><span>#</span></a></h4><p>电介质 : 菲涅尔项依赖于法向的镜面反射率 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> ，对于电介质材质是消色差的，可以用灰度来描述。Filament 中使用 <a href="https://media.contentapi.ea.com/content/dam/eacom/frostbite/files/s2014-pbs-frostbite-slides.pdf" target="_blank"><strong>Moving Frostbite to PBR</strong></a> 中所提到的电介质表面对反射率进行重映射：</p><span><span><span>f0=0.16∗reflectance2f_0 = 0.16 * {reflectance}^2</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>0.16</span><span></span><span>∗</span><span></span></span><span><span></span><span><span><span>r</span><span>e</span><span>f</span><span>l</span><span>ec</span><span>t</span><span>an</span><span>ce</span></span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span><p>这种做法的目标是将 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 映射到常见的电介质表面（约 4%）以及宝石（8% ~ 16%）的菲涅尔值的范围内。 </p><figure><img src="/assets/images/Filament/diagram_reflectance.png" alt="Diagram_Reflectance" /><figcaption>Diagram_Reflectance</figcaption></figure> <em>常见材质的反射率</em><p></p><p>假如折射率（IOR）已知，<span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span> 可以做如下计算：</p><span><span><span>f0=(nior−1)2(nior+1)2f_0 = \frac{(n_{ior} - 1)^2}{(n_{ior} + 1)^2}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>(</span><span><span>n</span><span><span><span><span><span><span></span><span><span><span>i</span><span>or</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>+</span><span></span><span>1</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>(</span><span><span>n</span><span><span><span><span><span><span></span><span><span><span>i</span><span>or</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>−</span><span></span><span>1</span><span><span>)</span><span><span><span><span><span><span></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span></span><p>而假如反射率已知，也可以反求出其折射率：</p><span><span><span>nior=21−f0−1n_{ior} = \frac{2}{1 - \sqrt{f_0}} - 1</span><span><span><span></span><span><span>n</span><span><span><span><span><span><span></span><span><span><span>i</span><span>or</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span>1</span><span></span><span>−</span><span></span><span><span><span><span><span><span></span><span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span><span><span></span><span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span><span><span></span><span></span></span><span><span></span><span><span>2</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span><span></span><span>−</span><span></span></span><span><span></span><span>1</span></span></span></span></span><p>下表中描述了自然界常见材质的菲涅尔反射率：</p> <table><thead><tr><th>材料</th><th>反射率</th><th>折射率</th><th>线性值</th></tr></thead><tbody><tr><td>水</td><td>2%</td><td>1.33</td><td>0.35</td></tr><tr><td>织物</td><td>4%~ 5.6%</td><td>1.5~ 1.62</td><td>0.5~ 0.59</td></tr><tr><td>常见液体</td><td>2%~ 4%</td><td>1.33~ 1.5</td><td>0.35~ 0.5</td></tr><tr><td>常见宝石</td><td>5%~ 16%</td><td>1.58~ 2.33</td><td>0.56~ 1.0</td></tr><tr><td>塑料/玻璃</td><td>4%~ 5%</td><td>1.5~ 1.58</td><td>0.5~ 0.56</td></tr><tr><td>其他介电材料</td><td>2%~ 5%</td><td>1.33~ 1.58</td><td>0.35~ 0.56</td></tr><tr><td>眼睛</td><td>2.5%</td><td>1.38</td><td>0.39</td></tr><tr><td>皮肤</td><td>2.8%</td><td>1.4</td><td>0.42</td></tr><tr><td>头发</td><td>4.6%</td><td>1.55</td><td>0.54</td></tr><tr><td>牙齿</td><td>5.8%</td><td>1.63</td><td>0.6</td></tr><tr><td>默认</td><td>4%</td><td>1.5</td><td>0.5</td></tr></tbody></table><p>在 Filament 中所有掠射角的反射率有 <span><span>F90=1.0F_{90} = 1.0</span><span><span><span></span><span><span>F</span><span><span><span><span><span><span></span><span><span><span>90</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span>1.0</span></span></span></span></p><p>导体 : 金属表面的镜面反射率不是消色的，是彩色的：</p><span><span><span>f0=baseColor∗metallicf_0 = {baseColor}* {metallic}</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span><span></span><span>=</span><span></span></span><span><span></span><span><span>ba</span><span>se</span><span>C</span><span>o</span><span>l</span><span>or</span></span><span></span><span>∗</span><span></span></span><span><span></span><span><span>m</span><span>e</span><span>t</span><span>a</span><span>l</span><span>l</span><span>i</span><span>c</span></span></span></span></span></span><p>对于电介质和金属材质而言，Filament 使用下面的方法计算 <span><span>f0f_0</span><span><span><span></span><span><span>f</span><span><span><span><span><span><span></span><span><span>0</span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span></span></span></span></span>:</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>vec3 f0 </span><span>=</span><span> </span><span>0.16</span><span> </span><span>*</span><span> reflectance </span><span>*</span><span> reflectance </span><span>*</span><span> (</span><span>1.0</span><span> </span><span>-</span><span> metallic) </span><span>+</span><span> baseColor </span><span>*</span><span> metallic;</span></div></div></code></pre><div><div></div><div></div></div></figure></div></section></section><section><h3>材质参考<a href="#材质参考"><span>#</span></a></h3><p>Filament 提供了一个<a href="https://google.github.io/filament/Material%20Properties.pdf" target="_blank"><strong>材质制作参考</strong></a>，帮助使用者制作自己的 PBR 材质。</p><p>对普通材质 : <em>BaseColor</em> 应该没有除微表面的遮挡外的一切光照信息。<em>金属度</em>为非 0 即 1 的值，纯导体为 1，纯电介质为 0，因此对于这两类材质的金属度应该为接近 0 或 1的值，中间值应用于表面类型的过渡，如金属到铁锈。</p><p>对非金属材质 : <em>BaseColor</em> 代表的是反射的颜色，应为 <em>sRGB 50~ 240</em> 或 <em>sRGB 30~ 240</em>。<em>金属度</em>应该为 0 或者接近 0 的值。反射率如果找不到合适的值，可以为 <em>sRGB 127 (Linear 0.5， Reflectance 4%)</em>。<em>反射率</em>不宜小于 <em>sRGB 90（Linear 0.33，Reflectance 2%）</em></p><p>对金属材质 : <em>BaseColor</em> 代表高光和反射的颜色，亮度应在 <em>67%~ 100% (sRGB 170~ 255)</em>，被氧化过或者更脏的金属可以考虑使用更低的值。<em>金属度</em>为 1 或者接近 1 的值。<em>反射率</em>可以被忽略，或由 BaseColor 计算而来。</p></section></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2><ul> <li><a href="http://graphicrants.blogspot.com/2013/08/specular-brdf-reference.html" target="_blank">Specular BRDF Reference</a></li> <li><a href="https://www.unrealengine.com/en/blog/physically-based-shading-on-mobile" target="_blank">physically-based-shading-on-mobile</a></li> <li><a href="https://randomascii.wordpress.com/2012/09/09/game-developer-magazine-floating-point/" target="_blank">Game Developer Magazine Floating Point</a></li> <li><a href="https://media.contentapi.ea.com/content/dam/eacom/frostbite/files/s2014-pbs-frostbite-slides.pdf" target="_blank">Moving Frostbite to PBR</a></li> </ul></section> <section><h2>Footnotes<a href="#footnote-label"><span>#</span></a></h2> <ol> <li> <p><a href="https://www.jcgt.org/published/0003/02/03/paper.pdf" target="_blank">Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs</a> <a href="#user-content-fnref-heitz14">↩</a></p> </li> <li> <p><a href="https://media.disneyanimation.com/uploads/production/publication_asset/48/asset/s2012_pbs_disney_brdf_notes_v3.pdf" target="_blank">Physically Based Shading at Disney</a> <a href="#user-content-fnref-disney">↩</a></p> </li> </ol> </section>https://yohiro.games/posts/zbuffer-redistribution/https://yohiro.games/posts/zbuffer-redistribution/Wed, 15 May 2019 00:00:00 GMT<section><h2>背景<a href="#背景"><span>#</span></a></h2><p>我们现在玩的游戏，通常会使用 D24S8 格式的 DepthStencil，即深度占 24 位，模板占 8 位，深度用来存储像素前后的遮挡关系，模板用于标记具体的像素，这种格式可以兼顾精度还有性能，因此适用于绝大多数情况。但是我们在游戏开发中时还是会遇到由于 Depth Buffer 的精度不足而造成的前后的面片闪烁，即 Z-fighting 问题。<br /></p><p>有很多种解决这个问题的方式，包括但不限于：</p><ul> <li>手动调整物件的纵深位置，通常是调整坐标 Z</li> <li>轻微调整相机近平面的位置</li> <li>…</li> </ul><p>我们知道，屏幕空间中的 Z 值将会映射到深度 d 中，深度 d 在 DirectX 环境下被映射到 [0.0, 1.0]，在 OpenGL 环境下被映射到 [-1.0, 1.0]，这就是我们所谓的 NDC。</p><p>深度 depth_buffer_value 的值可以这样描述：</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>depth_buffer_value </span><span>=</span><span> (</span><span>1</span><span>&lt;&lt;</span><span>N) </span><span>*</span><span> ( a </span><span>+</span><span> b </span><span>/</span><span> z );</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>其中，</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span><span> </span></span><span>N </span><span>=</span><span> number of bits of Z precision</span></div></div><div><div><div>2</div></div><div><span><span> </span></span><span>a </span><span>=</span><span> zFar </span><span>/</span><span> ( zFar </span><span>-</span><span> zNear );</span></div></div><div><div><div>3</div></div><div><span><span> </span></span><span>b </span><span>=</span><span> zFar </span><span>*</span><span> zNear </span><span>/</span><span> ( zNear </span><span>-</span><span> zFar );</span></div></div><div><div><div>4</div></div><div><span><span> </span></span><span>z </span><span>=</span><span> distance from the eye to the object</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>这就使得深度值与 <span><span>1z\frac{1}{z}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>z</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 成正比，即 depth_buffer_value 越小。具体表现为，越远的物体其精度越小，越近的物体精度越大。</p><p>因此，对于渲染 API 而言，它们期望投影矩阵中含有 <span><span>1z\frac{1}{z}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>z</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 项，因为当投影矩阵中含有 <span><span>1z\frac{1}{z}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>z</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 的前提下，透视才是正确的。</p></section> <section><h2>Reverse-Z<a href="#reverse-z"><span>#</span></a></h2><p></p><figure><img src="/assets/images/ZRedistribution/depth-perception-graph1-b.jpg" alt="depth perception" /><figcaption>depth perception</figcaption></figure><p></p><p>上图使用了整数的 z，可以看出 z 越远，d 的排布越稀疏，精度越低。</p><p></p><figure><img src="/assets/images/ZRedistribution/depth-precision-graph4-625x324.jpg" alt="float depth perception" /><figcaption>float depth perception</figcaption></figure><p></p><p>而上图使用浮点数 z 的情况则更为极端，可以看出精度绝大多数集中在了近平面一侧。</p><p>从上面的两张图可以看出，直接使用这种分布会使得精度的分布不均匀，导致了精度的浪费。既然精度集中在近平面一侧，那么将深度颠倒，令 1.0 是近平面，0.0 是远平面如何？</p><p>因此有所谓的 <code>reverse-z</code> 即反转 d 到 [1.0, 0.0] 的分布方式：</p><p></p><figure><img src="/assets/images/ZRedistribution/depth-precision-graph5-625x324.jpg" alt="reversed depth" /><figcaption>reversed depth</figcaption></figure><p></p><p>这种分布精度的分布比较均匀，越靠近近平面，depth 越接近 1.0。Unreal 中的 SceneDepthZ 就使用了这种分布方式：</p><p></p><figure><img src="/assets/images/ZRedistribution/UnrealZ.png" alt="unreal depth" /><figcaption>unreal depth</figcaption></figure><p></p></section> <section><h2>Logarithmic 分布^{<a href="#user-content-fn-logarithmic">1</a>}<a href="#logarithmic-分布1"><span>#</span></a></h2><p>在<a href="https://outerra.blogspot.com/2009/08/logarithmic-z-buffer.html" target="_blank">这篇文章</a>中提到了一种不同于 Reserve-Z 的，使用对数函数来映射深度的方式。</p><div><figure><figcaption></figcaption><pre><code><div><div><div>1</div></div><div><span>float</span><span> Fcoef </span><span>=</span><span> </span><span>2.0</span><span> </span><span>/</span><span> </span><span>log2</span><span><span>(</span><span>farplane </span></span><span>+</span><span> </span><span>1.0</span><span>);</span></div></div><div><div><div>2</div></div><div><span>gl_Position.z </span><span>=</span><span> </span><span>log2</span><span>(</span><span>max</span><span>(</span><span><span>1</span><span>e</span></span><span>-</span><span>6</span><span><span>,</span><span> </span></span><span>1.0</span><span> </span><span>+</span><span><span> gl_Position.w</span><span>)) </span></span><span>*</span><span> Fcoef </span><span>-</span><span> </span><span>1.0</span><span>;</span></div></div></code></pre><div><div></div><div></div></div></figure></div><p>但是这种方式会因为最终输出的深度与 <span><span>1z\frac{1}{z}</span><span><span><span></span><span><span></span><span><span><span><span><span><span></span><span><span><span>z</span></span></span></span><span><span></span><span></span></span><span><span></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span><span></span></span></span></span></span><span></span></span></span></span></span> 不成正比，从而造成透视不正确。</p><p>为修复这一问题需要在 Pixel Shader 中修改深度^{<a href="#user-content-fn-logarithmicfix">2</a>}，这会导致 EarlyZ Test 失效，所以我认为这种分布方式不适用于游戏引擎这种对实时性要求较高的系统。</p></section> <section><h2>扩展阅读<a href="#扩展阅读"><span>#</span></a></h2><ul> <li>图片来自 <a href="https://developer.nvidia.com/blog/visualizing-depth-precision/" target="_blank">Visualizing Depth Precision</a></li> <li><a href="https://www.sjbaker.org/steve/omniv/love_your_z_buffer.html" target="_blank">Learning to Love your Z-buffer</a></li> <li><a href="https://www.gamedev.net/blog/73/entry-2006307-tip-of-the-day-logarithmic-zbuffer-artifacts-fix/" target="_blank">Tip of the day: logarithmic zbuffer artifacts fix</a></li> <li><a href="https://outerra.blogspot.com/2012/11/maximizing-depth-buffer-range-and.html" target="_blank">Maximizing Depth Buffer Range and Precision</a></li> <li><a href="https://therealmjp.github.io/posts/attack-of-the-depth-buffer/" target="_blank">Attack of the depth buffer</a></li> <li><a href="https://stackoverflow.com/questions/7006510/density-of-floating-point-number-magnitude-of-the-number" target="_blank">density-of-floating-point-number-magnitude-of-the-number</a></li> </ul></section> <section><h2>参考<a href="#参考"><span>#</span></a></h2></section> <section><h2>Footnotes<a href="#footnote-label"><span>#</span></a></h2> <ol> <li> <p><a href="https://outerra.blogspot.com/2009/08/logarithmic-z-buffer.html" target="_blank">Logarithmic Depth Buffer</a> <a href="#user-content-fnref-logarithmic">↩</a></p> </li> <li> <p><a href="https://outerra.blogspot.com/2013/07/logarithmic-depth-buffer-optimizations.html" target="_blank">Logarithmic depth buffer optimizations &amp; fixes</a> <a href="#user-content-fnref-logarithmicfix">↩</a></p> </li> </ol> </section>https://yohiro.games/posts/dont-store-that-in-a-float/https://yohiro.games/posts/dont-store-that-in-a-float/Sat, 16 Mar 2019 00:00:00 GMT<p>之前做Time of Day系统的时候遇到过一个精度问题，然后看到了这样的一篇文章感觉挺有意思，所以拿来记录一下。</p> <p>原文传送门👉<a href="https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/" target="_blank"><strong>Don’t Store That in a Float | Random ASCII</strong></a></p> <hr /> <p>文章的核心是论证浮点数精度的重要性，并引入了浮点数的一些数学性质。</p> <p>有些类型的数据，如已流逝的游戏时间不应当使用float而应该用双精度的double。</p>https://yohiro.games/posts/tools/https://yohiro.games/posts/tools/Sat, 12 May 2018 00:00:00 GMT<section><h2>系统<a href="#系统"><span>#</span></a></h2><ul> <li> <p><a href="https://github.com/ventoy/Ventoy" target="_blank">Ventoy</a></p> <p>U 盘 PE 安装工具。</p> </li> </ul></section> <section><h2>文件管理<a href="#文件管理"><span>#</span></a></h2><ul> <li> <p><a href="https://github.com/stefankueng/grepWin" target="_blank">grepwin</a></p> <p>Windows 平台的搜索工具，可以在指定路径下快速搜索文本，支持正则表达式。</p> </li> <li> <p><a href="https://www.voidtools.com/zh-cn/downloads/" target="_blank">everything</a></p> <p>Windows 平台文件搜索工具，可以搭配 <a href="https://github.com/srwi/EverythingToolbar" target="_blank">EverythingToolbar</a> 一起使用。</p> </li> <li> <p><a href="https://github.com/windirstat/windirstat" target="_blank">windirstat</a></p> <p>用于查看 Windows 平台硬盘空间占用情况的工具。</p> </li> <li> <p><a href="https://github.com/henrypp/memreduct" target="_blank">memreduct</a></p> <p>Windows 轻量内存管理/监测工具。</p> </li> <li> <p><a href="https://github.com/chrdavis/SmartRename" target="_blank">SmartRename</a></p> <p>Windows 重命名工具。</p> </li> </ul></section> <section><h2>截屏/录屏<a href="#截屏录屏"><span>#</span></a></h2><ul> <li> <p><a href="https://github.com/NickeManarin/ScreenToGif" target="_blank">ScreenToGif</a></p> <p>可以在屏幕上截屏录制为 gif 的工具。</p> </li> <li> <p><a href="https://pixpinapp.com/" target="_blank">PixPin</a></p> <p>功能强大使用简单的截图/贴图工具。</p> </li> </ul></section> <section><h2>网络<a href="#网络"><span>#</span></a></h2><ul> <li> <p><a href="https://github.com/fatedier/frp" target="_blank">frp</a></p> <p>内网穿透工具。</p> </li> <li> <p><a href="https://github.com/kingToolbox/WindTerm" target="_blank">WindTerm</a></p> <p>免费 ssh 工具。</p> </li> </ul></section> <section><h2>阅读管理<a href="#阅读管理"><span>#</span></a></h2><ul> <li> <p><a href="https://github.com/DIYgod/RSSHub" target="_blank">RSSHub</a></p> <p>RSS 订阅管理。</p> </li> <li> <p><a href="https://github.com/logseq/logseq" target="_blank">logseq</a></p> <p>个人知识库管理。</p> </li> </ul></section> <section><h2>AI<a href="#ai"><span>#</span></a></h2><ul> <li> <p><a href="https://github.com/lobehub/lobe-chat" target="_blank">lobe-chat</a></p> <p>可自己部署的支持多种 AI 大模型的前端/客户端</p> </li> </ul></section>https://yohiro.games/posts/references/https://yohiro.games/posts/references/参考工具合集Fri, 11 May 2018 00:00:00 GMT<section><h2>工具<a href="#工具"><span>#</span></a></h2><ul> <li><a href="https://maicss.com/pyqt/" target="_blank">Python Qt</a></li> <li><a href="https://pthom.github.io/imgui_manual_online/manual/imgui_manual.html" target="_blank">ImGui Manual</a></li> </ul></section> <section><h2>渲染<a href="#渲染"><span>#</span></a></h2><section><h3>光照<a href="#光照"><span>#</span></a></h3><ul> <li><a href="https://developer.nvidia.com/gpugems/gpugems/part-iii-materials/chapter-19-image-based-lighting" target="_blank">Chapter 19. Image-Based Lighting</a></li> <li><a href="https://www.jianshu.com/p/86a65fed79aa" target="_blank">Unreal Engine中的间接光方案</a></li> <li><a href="https://www.unrealengine.com/en-US/blog/exploring-lumen-unreal-engine-5-s-dynamic-global-illumination-and-reflections-system" target="_blank">Exploring Lumen — Unreal Engine 5’s dynamic global illumination and reflections system</a></li> <li><a href="https://zhuanlan.zhihu.com/p/662681016" target="_blank">SIGGRAPH 2022-UE5 Lumen实时全局照明</a></li> </ul></section><section><h3>着色<a href="#着色"><span>#</span></a></h3><ul> <li><a href="https://pafuhana1213.hatenablog.com/entry/2019/04/09/104042" target="_blank">【UE4】Cel・Toon・Stylized・NPRシェーダ系アセット・資料 自分用まとめメモ （2020年8月更新）</a></li> <li><a href="https://qiita.com/com04/items/a7895160df8d854fe924" target="_blank">UE4のセルシェード実装方法を纏めたい</a></li> <li><a href="https://github.com/EpicGames/UnrealEngine/pull/1552" target="_blank">Toon shading model #1552</a></li> <li><a href="https://aqu.hatenablog.com/entry/2018/07/09/000805" target="_blank">UE4: 水彩 Post Process Material</a></li> <li><a href="https://forums.unrealengine.com/t/toon-shading-model/30226" target="_blank">Toon shading model</a></li> <li><a href="https://darkcatgame.tistory.com/28" target="_blank">UE4 Cel Shading : 카툰렌더링</a></li> <li><a href="https://www.4gamer.net/games/467/G046741/20210826001/" target="_blank">［CEDEC 2021］「BLUE PROTOCOL」のテクニカルディレクターが語る，UE4のアニメ表現。エンジン改造で実現したこだわりの実装とは</a></li> <li><a href="https://www.kodeco.com/100-unreal-engine-4-paint-filter-tutorial#toc-anchor-002" target="_blank">Unreal Engine 4 Paint Filter</a></li> <li><a href="http://walkingfat.com/%E3%80%8A%E5%A1%9E%E5%B0%94%E8%BE%BE-%E8%8D%92%E9%87%8E%E4%B9%8B%E6%81%AF%E3%80%8B%E4%B8%AD%E8%A7%92%E8%89%B2%E5%8F%97%E5%A4%9A%E4%B8%AA%E7%82%B9%E5%85%89%E6%BA%90%E5%BD%B1%E5%93%8D%E5%81%9A%E6%B3%95/" target="_blank">角色受多个点光源影响做法</a></li> <li><a href="https://dev.epicgames.com/community/learning/tutorials/J3ea/unreal-engine-deconstructing-a-stylized-comic-shader" target="_blank">Deconstructing A Stylized Comic Shader</a></li> <li><a href="https://medium.com/@lordned/unreal-engine-4-rendering-part-2-shaders-and-vertex-data-80317e1ae5f3" target="_blank">Unreal Engine 4 Rendering Part 2: Shaders and Vertex Data</a></li> <li><a href="https://www.tomlooman.com/unreal-engine-soft-outline/" target="_blank">Soft Outline</a></li> </ul></section><section><h3>贴图<a href="#贴图"><span>#</span></a></h3><ul> <li><a href="https://gameinstitute.qq.com/community/detail/101476" target="_blank">《枪神纪》贴图滤波方式实践</a></li> <li><a href="https://www.gamedevelopment.blog/texture-filter/" target="_blank">Texture Filter – Picking Pixels</a></li> <li><a href="https://developer.oculus.com/blog/common-rendering-mistakes-how-to-find-them-and-how-to-fix-them/" target="_blank">Common Rendering Mistakes: How to Find Them and How to Fix Them</a></li> </ul></section><section><h3>几何<a href="#几何"><span>#</span></a></h3><ul> <li><a href="https://cs.stackexchange.com/questions/12521/what-are-degenerate-polygons" target="_blank">What are degenerate polygons?</a></li> <li><a href="https://arm-software.github.io/opengl-es-sdk-for-android/ocean_f_f_t.html" target="_blank">Ocean Rendering with Fast Fourier Transform</a></li> <li><a href="http://www.betelge.com/blog/" target="_blank">Making an 8-bit NES game (Part 1): A minimal ROM</a></li> </ul></section><section><h3>其他<a href="#其他"><span>#</span></a></h3><ul> <li><a href="https://shaderbits.com/blog/custom-per-object-shadowmaps-using-blueprints" target="_blank">Custom Per Object Shadowmaps using Blueprints</a></li> <li><a href="https://ikrima.dev/ue4guide/graphics-development/render-architecture/detailed-render-flow/" target="_blank">Detailed render flow</a></li> <li><a href="https://gameinstitute.qq.com/community/detail/117893" target="_blank">【GAD翻译馆】小浮点格式——R11G11B10F精度</a></li> <li><a href="https://blog.csdn.net/sh15285118586/article/details/116723277" target="_blank">CustomStencil C++部分整理</a></li> <li><a href="http://www.cemyuksel.com/research/hairmodels/" target="_blank">HAIR Model Files</a></li> <li><a href="https://github.com/google/filament" target="_blank">filament</a></li> <li><a href="https://www.adriancourreges.com/blog/2020/12/29/graphics-studies-compilation/" target="_blank">Graphics Studies Compilation</a></li> </ul></section><section><h3>博客<a href="#博客"><span>#</span></a></h3><ul> <li><a href="https://www.jendrikillner.com/#about" target="_blank">Jendrik Illner</a></li> <li><a href="http://karim.naaji.fr/posts.html" target="_blank">Karim Naaji</a></li> <li><a href="https://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-graphics-pipeline-2011-index/" target="_blank">The ryg blog</a></li> <li><a href="https://mynameismjp.wordpress.com/2012/10/24/msaa-overview/" target="_blank">THE DANGER ZONE</a></li> <li><a href="http://aicdg.com/aa/" target="_blank">AICDG</a></li> <li><a href="https://therealmjp.github.io/posts/shader-permutations-part1/" target="_blank">therealmjp</a></li> <li><a href="https://zeux.io/" target="_blank">Zeux</a></li> </ul></section></section> <section><h2>引擎<a href="#引擎"><span>#</span></a></h2><section><h3>Framework<a href="#framework"><span>#</span></a></h3><ul> <li> <p><a href="https://jip.dev/notes/unreal-engine/#simulation-renderer-synchronization" target="_blank">Simulation Renderer Synchronization</a></p> </li> <li> <p><a href="https://unrealcpp.com/" target="_blank">Unreal Engine 4 C++ Tutorials</a></p> </li> <li> <p><a href="https://logins.github.io/graphics/2021/05/31/RenderGraphs.html#parallel-command-list-recording" target="_blank">Why Talking About Render Graphs</a></p> </li> </ul></section></section> <section><h2>Gameplay<a href="#gameplay"><span>#</span></a></h2><ul> <li><a href="https://jashking.github.io/2017/04/26/ue4-rts-selection-box/" target="_blank">UE4 RTS 拉选提示框</a></li> <li><a href="https://jinyuliao.github.io/blog/html/2017/12/15/ue4_dialogue_system_part1.html" target="_blank">UE4 Dialogue System Part1</a></li> <li><a href="https://github.com/BillEliot/GASDocumentation_Chinese" target="_blank">GASDocumentation_Chinese</a></li> <li><a href="https://qiita.com/YuukiOgino/items/07fec724d4a2bb2a9403/" target="_blank">UE4のBluePrintだけで壁蹴りを実装する</a></li> <li><a href="https://forums.unrealengine.com/t/blueprint-wall-jump-fail/3062/9" target="_blank">Wall Jump fail</a></li> </ul><section><h3>Old wiki<a href="#old-wiki"><span>#</span></a></h3><ul> <li><a href="https://nerivec.github.io/old-ue4-wiki/index.html" target="_blank">Old UE4 Wiki</a></li> </ul></section><section><h3>多线程<a href="#多线程"><span>#</span></a></h3><ul> <li><a href="https://www.docswell.com/s/EpicGamesJapan/5QMWWK-UE4_CEDECKYUSHU2021_MultiThread#p19" target="_blank">UE4_CEDECKYUSHU2021_MultiThread</a></li> <li><a href="https://unrealengine.hatenablog.com/entry/2022/07/31/224504" target="_blank">UE5 Task Systemでお手軽な非同期処理！ ※サンプルプロジェクト配布あり</a></li> </ul></section><section><h3>Streaming<a href="#streaming"><span>#</span></a></h3><ul> <li><a href="https://www.iteye.com/blog/avi-2524009" target="_blank">UE4 IO Store</a></li> <li><a href="https://imzlp.com/posts/9455/" target="_blank">PixelStreaming：基本概念与上手初探</a></li> <li><a href="http://kagring.blog.fc2.com/blog-entry-777.html" target="_blank">UE5/UE4 C++で複数のアセットファイルを非同期読み込み（ロード）する</a></li> </ul></section></section> <section><h2>效果<a href="#效果"><span>#</span></a></h2><ul> <li><a href="https://historia.co.jp/archives/21974/" target="_blank">簡単になったVAT（Vertex Animation Texture）で破壊してみよう</a></li> <li><a href="https://rwiiug.hatenablog.com/entry/UE4_OtegaruTransition" target="_blank">コピペでOK！シーンチェンジに使えるトランジション演出を実装しよう！</a></li> </ul></section> <section><h2>RHI<a href="#rhi"><span>#</span></a></h2><section><h3>D3D12<a href="#d3d12"><span>#</span></a></h3><ul> <li><a href="https://learn.microsoft.com/zh-cn/windows/win32/direct3d12/directx-12-programming-guide" target="_blank">DirectX12 Programming Guide</a></li> <li><a href="https://microsoft.github.io/DirectX-Specs/" target="_blank">DirectX Specs</a></li> <li><a href="https://devblogs.microsoft.com/directx/a-look-inside-d3d12-resource-state-barriers/" target="_blank">A Look Inside D3D12 Resource State Barriers</a></li> </ul></section><section><h3>Vulkan<a href="#vulkan"><span>#</span></a></h3><ul> <li> <p><a href="https://registry.khronos.org/vulkan/specs/1.3/html/" target="_blank">Vulkan Specs</a></p> </li> <li> <p><a href="https://gpuopen.com/learn/dcc-overview/" target="_blank">Getting the Most Out of Delta Color Compression</a></p> </li> <li> <p><a href="https://forums.unrealengine.com/t/writing-data-to-rdg-structured-buffer-general-rdg-questions/138823/4" target="_blank">Writing Data to RDG Structured Buffer / General RDG Questions</a></p> </li> <li> <p><a href="https://docs.unrealengine.com/5.0/en-US/mesh-drawing-pipeline-in-unreal-engine/" target="_blank">Mesh Drawing Pipeline</a></p> </li> </ul></section></section> <section><h2>物理<a href="#物理"><span>#</span></a></h2><ul> <li> <p><a href="https://glhub.blogspot.com/search/label/Chaos" target="_blank">[GDC 2019] UE4の新物理エンジンChaosの紹介と内製の理由</a></p> </li> <li> <p><a href="https://matthias-research.github.io/pages/publications/publications.html" target="_blank">Matthias Research</a></p> </li> <li> <p><a href="http://blog.mmacklin.com/" target="_blank">Miles Macklin</a></p> </li> <li> <p><a href="https://gdbooks.gitbooks.io/3dcollisions/content/Chapter2/static_aabb_plane.html" target="_blank">Intersections</a></p> </li> <li> <p><a href="https://github.com/SPARK-inc/SPCRJointDynamicsUE4" target="_blank">SPCRJointDynamics for UE4</a></p> </li> <li> <p><a href="https://ics.uci.edu/~shz/courses/cs114/docs/proj3/index.html" target="_blank">CS114 Project 3: Cloth Simulation using Mass-Spring System</a></p> </li> </ul></section> <section><h2>管线<a href="#管线"><span>#</span></a></h2><ul> <li><a href="https://www.jianshu.com/p/ecc9785f211d" target="_blank">【GDC2019】UE4 Mesh Drawing Pipeline Refactor</a></li> </ul><section><h3>工作流<a href="#工作流"><span>#</span></a></h3><ul> <li><a href="https://80.lv/articles/bird-house-working-on-a-stylized-landscape-in-ue4" target="_blank">Bird House: Working on a Stylized Landscape in UE4</a></li> <li><a href="https://www.docswell.com/s/EpicGamesJapan/K7JQ65-UE4-ACTF2022#p92" target="_blank">映像制作での Unreal Engine 活用のはじめ</a></li> <li><a href="https://ruyo.github.io/VRM4U/02_pbr/" target="_blank">PBR 背景と合わせる</a></li> </ul></section><section><h3>自动化<a href="#自动化"><span>#</span></a></h3><ul> <li><a href="https://jackknobel.com/How-To/BuildGraph" target="_blank">How To: BuildGraph</a></li> <li><a href="https://de45xmedrsdbp.cloudfront.net/Resources/files/BuildTools-2031748007.pdf" target="_blank">Build Automation</a></li> </ul></section><section><h3>其他<a href="#其他-1"><span>#</span></a></h3><ul> <li><a href="https://jashking.github.io/2018/04/20/ue4-filesystem/" target="_blank">UE4 文件系统</a></li> <li><a href="https://www.slideshare.net/EpicGamesJapan/ue4-137253042" target="_blank">UE4における自動プレイのポストモーテム</a></li> </ul></section></section> <section><h2>优化<a href="#优化"><span>#</span></a></h2><ul> <li> <p><a href="https://medium.com/@bryan.corell/using-async-collision-traces-in-unreal-engine-4-2cc312c825f5" target="_blank">Using Async Collision Traces in Unreal Engine 4</a></p> </li> <li> <p><a href="https://www.famitsu.com/news/201910/16184681.html" target="_blank">『ドラゴンクエストXI S』はいかにUnreal Engine 4で最適化されたか？ “UNREAL FEST EAST 2019”のセッションをお届け</a></p> </li> <li> <p><a href="https://stonelzp.github.io/ue4-performance/" target="_blank">UE4 Performance</a></p> </li> <li> <p><a href="https://www.docswell.com/s/EpicGamesJapan/ZEEL7Z-UE4_LargeScaleDevSQEX_Optimize#p53" target="_blank">実行速度の最適化のあれこれ　プラス　おまけ【UE4を用いた大規模開発事例紹介 スクウェア・エニックス様をお招きして 2019】</a></p> </li> <li> <p><a href="https://zeux.io/2009/01/31/view-frustum-culling-optimization-introduction/" target="_blank">View frustum culling optimization - Introduction</a></p> </li> <li> <p><a href="https://qiita.com/EGJ-Takashi_Suzuki/items/33449c17c61d1ed30079" target="_blank">CharacterMovementをマルチスレッド化してみよう</a></p> </li> <li> <p><a href="https://aras-p.info/texts/CompactNormalStorage.html" target="_blank">Compact Normal Storage for small G-Buffers</a></p> </li> </ul><section><h3>精度<a href="#精度"><span>#</span></a></h3><ul> <li><a href="https://bartwronski.com/2017/04/02/small-float-formats-r11g11b10f-precision/" target="_blank">Small float formats – R11G11B10F precision</a></li> <li><a href="https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/" target="_blank">Don’t Store That in a Float</a></li> </ul></section></section> <section><h2>资料库<a href="#资料库"><span>#</span></a></h2><ul> <li><a href="http://nikoladimitroff.github.io/Game-Engine-Architecture/" target="_blank">Game Engine Architecture Lectures</a></li> <li><a href="https://cedil.cesa.or.jp/" target="_blank">CEDiL</a></li> <li><a href="https://www.docswell.com/user/EpicGamesJapan?page=1" target="_blank">EpicGamesJapan</a></li> <li><a href="https://bartwronski.com/2016/10/30/dithering-part-three-real-world-2d-quantization-dithering/" target="_blank">Dithering part three – real world 2D quantization dithering</a></li> <li><a href="https://gregory-igehy.hatenadiary.com/entry/2018/02/24/023251" target="_blank">Gregory Igehy</a></li> <li><a href="https://unrealengine.hatenablog.com/" target="_blank">hatenablog</a></li> <li><a href="https://matlib.gpuopen.com/main/materials/all" target="_blank">MaterialX Library</a></li> <li><a href="https://ue5wiki.com/" target="_blank">虚幻社区知识库</a></li> <li><a href="http://www.teal-game.com/blog/customcrashreporter/" target="_blank">Custom Unreal Engine 4 Crash Reporter</a></li> <li><a href="https://ikrima.dev/" target="_blank">Game Development Notes#</a></li> <li><a href="https://www.slideshare.net/EpicGamesJapan/ue4input-latency" target="_blank">UE4のスレッドの流れとInput Latency改善の仕組み</a></li> <li><a href="https://ue5study.com/" target="_blank">UE5攻略リンク</a></li> <li><a href="http://kagring.blog.fc2.com/blog-category-13.html" target="_blank">kagring.blog</a></li> <li><a href="https://qiita.com/organizations/epicgamesjapan" target="_blank">qiita - epicgamesjapan</a></li> <li><a href="https://ikrima.dev/" target="_blank">Gamedev Guide</a></li> </ul><section><h3>编程<a href="#编程"><span>#</span></a></h3><ul> <li><a href="https://stackoverflow.com/questions/81656/where-do-i-find-the-current-c-or-c-standard-documents" target="_blank">Where do I find the current C or C++ standard documents?</a></li> <li><a href="https://isocpp.org/files/papers/N4860.pdf" target="_blank">Programming Languages — C++</a></li> </ul></section></section> <section><h2>动画<a href="#动画"><span>#</span></a></h2><ul> <li><a href="https://papalqi.cn/ue4%E5%8A%A8%E7%94%BB%E7%B3%BB%E7%BB%9F%E6%BA%90%E7%A0%81%E5%89%96%E6%9E%90/" target="_blank">UE4动画系统源码剖析</a></li> <li><a href="https://historia.co.jp/archives/21974" target="_blank">[UE5] 簡単になったVAT（Vertex Animation Texture）で破壊してみよう</a></li> </ul></section>