<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>道</title>
  
  <subtitle>常无欲以观其妙</subtitle>
  <link href="/atom.xml" rel="self"/>
  
  <link href="http://yoursite.com/"/>
  <updated>2019-03-26T00:19:13.644Z</updated>
  <id>http://yoursite.com/</id>
  
  <author>
    <name>Osiris</name>
    
  </author>
  
  <generator uri="http://hexo.io/">Hexo</generator>
  
  <entry>
    <title>Bash test&amp;compare</title>
    <link href="http://yoursite.com/2019/03/25/Base-test-compare/"/>
    <id>http://yoursite.com/2019/03/25/Base-test-compare/</id>
    <published>2019-03-25T15:14:47.000Z</published>
    <updated>2019-03-26T00:19:13.644Z</updated>
    
    <content type="html"><![CDATA[<h1 id="Bash-测试和比较函数"><a href="#Bash-测试和比较函数" class="headerlink" title="Bash 测试和比较函数"></a>Bash 测试和比较函数</h1><h2 id="test、-、-、-、和-if-then-else-解密"><a href="#test、-、-、-、和-if-then-else-解密" class="headerlink" title="test、[、[[、((、和 if-then-else 解密"></a>test、[、[[、((、和 if-then-else 解密</h2><p>from: <a href="https://www.ibm.com/developerworks/cn/linux/l-bash-test.html" target="_blank" rel="noopener">IBM Bash 测试和比较函数</a></p><p>Bash shell 在当今的许多 Linux® 和 UNIX® 系统上都可使用，是 Linux 上常见的默认 shell。Bash 包含强大的编程功能，其中包括丰富的可测试文件类型和属性的函数，以及在多数编程语言中可以使用的算术和字符串比较函数。理解不同的测试并认识到 shell 还能把一些操作符解释成 shell 元字符，是成为高级 shell 用户的重要一步。这篇文章摘自 developerWorks 教程 LPI 102 考试准备，主题 109: Shell、脚本、编程和编译，介绍了如何理解和使用 Bash shell 的测试和比较操作。</p><p>这个技巧解释了 shell 测试和比较函数，演示了如何向 shell 添加编程功能。您可能已经看到过使用 &amp;&amp; 和 || 操作符的简单 shell 逻辑，它允许您根据前一条命令的退出状态（正确退出或伴随错误退出）而执行后一条命令。在这个技巧中，将看到如何把这些基本的技术扩展成更复杂的 shell 编程。</p><h2 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h2><p>在任何一种编程语言中，学习了如何给变量分配值和传递参数之后，都需要测试这些值和参数。在 shell 中，测试会设置返回的状态，这与其他命令执行的功能相同。实际上，test 是个内置命令！</p><h3 id="test-和"><a href="#test-和" class="headerlink" title="test 和 ["></a>test 和 [</h3><p>内置命令 test 根据表达式expr 求值的结果返回 0（真）或 1（假）。也可以使用方括号：test expr 和 [ expr ] 是等价的。 可以用 $? 检查返回值；可以使用 &amp;&amp; 和 || 操作返回值；也可以用本技巧后面介绍的各种条件结构测试返回值。</p><p>清单 1. 一些简单测试<br><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]$ test <span class="number">3</span> -gt <span class="number">4</span> &amp;&amp; echo True <span class="params">||</span> echo <span class="literal">false</span></span><br><span class="line"><span class="literal">false</span></span><br><span class="line">[ian@pinguino ~]$ [ <span class="string">"abc"</span> != <span class="string">"def"</span> ];echo $?</span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian@pinguino ~]$ test -d <span class="string">"$HOME"</span> ;echo $?</span><br><span class="line"><span class="number">0</span></span><br></pre></td></tr></table></figure></p><p>在清单 1 的第一个示例中，-gt 操作符对两个字符值之间执行算术比较。在第二个示例中，用 [ ] 的形式比较两个字符串不相等。在最后一个示例中，测试 HOME 变量的值，用单目操作符 -d 检查它是不是目录。</p><p>可以用 -eq、 -ne、-lt、 -le、 -gt 或 -ge 比较算术值，它们分别表示等于、不等于、小于、小于等于、大于、大于等于。</p><p>可以分别用操作符 =、 !=、&lt; 和 &gt; 比较字符串是否相等、不相等或者第一个字符串的排序在第二个字符串的前面或后面。单目操作符 -z 测试 null 字符串，如果字符串非空 -n 返回 True（或者根本没有操作符）。</p><p>说明：shell 也用 &lt; 和 &gt; 操作符进行重定向，所以必须用 \&lt; 或 > 加以转义。清单 2 显示了字符串测试的更多示例。检查它们是否如您预期的一样。</p><p>清单 2. 一些字符串测试<br><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>test <span class="string">"abc"</span> = <span class="string">"def"</span> ;echo <span class="variable">$?</span></span><br><span class="line"><span class="number">1</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>[ <span class="string">"abc"</span> != <span class="string">"def"</span> ];echo <span class="variable">$?</span></span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>[ <span class="string">"abc"</span> \&lt; <span class="string">"def"</span> ];echo <span class="variable">$?</span></span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>[ <span class="string">"abc"</span> \&gt; <span class="string">"def"</span> ];echo <span class="variable">$?</span></span><br><span class="line"><span class="number">1</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>[ <span class="string">"abc"</span> \&lt;<span class="string">"abc"</span> ];echo <span class="variable">$?</span></span><br><span class="line"><span class="number">1</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>[ <span class="string">"abc"</span> \&gt; <span class="string">"abc"</span> ];echo <span class="variable">$?</span></span><br><span class="line"><span class="number">1</span></span><br></pre></td></tr></table></figure></p><p>表 1 显示了一些更常见的文件测试。如果被测试的文件存在，而且有指定的特征，则结果为 True。</p><p>表 1. 一些常见的文件测试</p><table><thead><tr><th>操作符</th><th>特征</th></tr></thead><tbody><tr><td>-d</td><td>目录</td></tr><tr><td>-e</td><td>存在（也可以用 -a</td></tr><tr><td>-f</td><td>普通文件｜</td></tr><tr><td>-h</td><td>符号连接</td></tr><tr><td>-p</td><td>命名管道</td></tr><tr><td>-r</td><td>可读</td></tr><tr><td>-s</td><td>非空</td></tr><tr><td>-S</td><td>套接字</td></tr><tr><td>-W</td><td>可写</td></tr><tr><td>-N</td><td>从上次读取之后已经做过修改</td></tr></tbody></table><p>除了上面的单目测试，还可以使用表 2 所示的双目操作符比较两个文件：</p><p>表 2. 测试一对文件</p><table><thead><tr><th>操作符</th><th>为 True 的情况</th></tr></thead><tbody><tr><td>-ot</td><td>测试file1 是否比 file2 更旧。</td></tr><tr><td>-ef</td><td>测试file1 是否比 file2的硬连接。</td></tr><tr><td>-nt</td><td>测试file1 是否比file2 更新。修改日期将用于这次和下次比较。</td></tr></tbody></table><p>其他一些测试可以用来测试文件许可之类的内容。请参阅 bash 手册获得更多细节或使用 help test 查看内置测试的简要信息。也可以用 help 命令了解其他内置命令。</p><p>-o 操作符允许测试利用 set -o 选项 设置的各种 shell 选项，如果设置了该选项，则返回 True (0)，否则返回 False (1)，如清单 3 所示。</p><p>清单 3. 测试 shell 选项<br><figure class="highlight gams"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]<span class="symbol">$</span> <span class="keyword">set</span> +o <span class="comment">nounset</span></span><br><span class="line">[ian@pinguino ~<span class="comment">]$ [ -o nounset ]</span>;echo <span class="symbol">$</span>?</span><br><span class="line"><span class="number">1</span></span><br><span class="line">[ian@pinguino ~]<span class="symbol">$</span> <span class="keyword">set</span> -u</span><br><span class="line">[ian@pinguino ~<span class="comment">]$ test  -o nounset</span>; echo <span class="symbol">$</span>?</span><br><span class="line"><span class="number">0</span></span><br></pre></td></tr></table></figure></p><p>最后，-a 和 -o 选项允许使用逻辑运算符 AND 和 OR 将表达式组合在一起。单目操作符 ! 可以使测试的意义相反。可以用括号把表达式分组，覆盖默认的优先级。请记住 shell 通常要在子 shell 中运行括号中的表达式，所以需要用 ( 和 ) 转义括号，或者把这些操作符括在单引号或双引号内。清单 4 演示了摩根法则在表达式上的应用。</p><p>清单 4. 组合和分组测试<br><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>test <span class="string">"a"</span> != <span class="string">"$HOME"</span> -a <span class="number">3</span> -ge <span class="number">4</span> ; echo <span class="variable">$?</span></span><br><span class="line"><span class="number">1</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>[ ! \( <span class="string">"a"</span> = <span class="string">"$HOME"</span> -o <span class="number">3</span> -lt <span class="number">4</span> \) ]; echo <span class="variable">$?</span></span><br><span class="line"><span class="number">1</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>[ ! \( <span class="string">"a"</span> = <span class="string">"$HOME"</span> -o <span class="string">'('</span> <span class="number">3</span> -lt <span class="number">4</span> <span class="string">')'</span> <span class="string">")"</span> ]; echo <span class="variable">$?</span></span><br><span class="line"><span class="number">1</span></span><br></pre></td></tr></table></figure></p><h3 id="和"><a href="#和" class="headerlink" title="(( 和 [["></a>(( 和 [[</h3><p>test 命令非常强大，但是很难满足其转义需求以及字符串和算术比较之间的区别。幸运的是，bash 提供了其他两种测试方式，这两种方式对熟悉 C、C++ 或 Java® 语法的人来说会更自然些。</p><p>(( ))复合命令 计算算术表达式，如果表达式求值为 0，则设置退出状态为 1；如果求值为非 0 值，则设置为 0。不需要对 (( 和 )) 之间的操作符转义。算术只对整数进行。除 0 会产生错误，但不会产生溢出。可以执行 C 语言中常见的算术、逻辑和位操作。 let 命令也能执行一个或多个算术表达式。它通常用来为算术变量分配值。</p><p>清单 5. 分配和测试算术表达式<br><figure class="highlight elixir"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>let x=<span class="number">2</span> y=<span class="number">2</span>**<span class="number">3</span> z=y*<span class="number">3</span>;echo <span class="variable">$?</span> <span class="variable">$x</span> <span class="variable">$y</span> <span class="variable">$z</span></span><br><span class="line"><span class="number">0</span> <span class="number">2</span> <span class="number">8</span> <span class="number">24</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>(( w=(y/x) + ( (~ ++x) &amp; <span class="number">0x0f</span> ) )); echo <span class="variable">$?</span> <span class="variable">$x</span> <span class="variable">$y</span> <span class="variable">$w</span></span><br><span class="line"><span class="number">0</span> <span class="number">3</span> <span class="number">8</span> <span class="number">16</span></span><br><span class="line">[ian<span class="variable">@pinguino</span> ~]<span class="variable">$ </span>(( w=(y/x) + ( (~ ++x) &amp; <span class="number">0x0f</span> ) )); echo <span class="variable">$?</span> <span class="variable">$x</span> <span class="variable">$y</span> <span class="variable">$w</span></span><br><span class="line"><span class="number">0</span> <span class="number">4</span> <span class="number">8</span> <span class="number">13</span></span><br></pre></td></tr></table></figure></p><p>同使用 (( )) 一样，利用复合命令 [[ ]] 可以对文件名和字符串使用更自然的语法。可以用括号和逻辑操作符把 test 命令支持的测试组合起来。</p><p>清单 6. 使用 [[ 复合命令<br><figure class="highlight kotlin"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[<span class="symbol">ian@</span>pinguino ~]$ [[ ( -d <span class="string">"<span class="variable">$HOME</span>"</span> ) &amp;&amp; ( -w <span class="string">"<span class="variable">$HOME</span>"</span> ) ]] &amp;&amp;  </span><br><span class="line">&gt;  echo <span class="string">"home is a writable directory"</span></span><br><span class="line">home <span class="keyword">is</span> a writable directory</span><br></pre></td></tr></table></figure></p><p>在使用 = 或 != 操作符时，复合命令 [[ 还能在字符串上进行模式匹配。匹配的方式就像清单 7 所示的通配符匹配。</p><p>清单 7. 用 [[ 进行通配符测试<br><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def .d,x--"</span> == a[abc]*\ <span class="string">?d</span>* ]]; echo $?</span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def c"</span> == a[abc]*\ <span class="string">?d</span>* ]]; echo $?</span><br><span class="line"><span class="number">1</span></span><br><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def d,x"</span> == a[abc]*\ <span class="string">?d</span>* ]]; echo $?</span><br><span class="line"><span class="number">1</span></span><br></pre></td></tr></table></figure></p><p>甚至还可以在 [[ 复合命令内执行算术测试，但是千万要小心。除非在 (( 复合命令内，否则 &lt; 和 &gt; 操作符会把操作数当成字符串比较并在当前排序序列中测试它们的顺序。清单 8 用一些示例演示了这一点。</p><p>清单 8. 用 [[ 包含算术测试<br><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def d,x"</span> == a[abc]*\ <span class="string">?d</span>* <span class="params">||</span> (( <span class="number">3</span> &gt; <span class="number">2</span> )) ]]; echo $?</span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def d,x"</span> == a[abc]*\ <span class="string">?d</span>* <span class="params">||</span> <span class="number">3</span> -gt <span class="number">2</span> ]]; echo $?</span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def d,x"</span> == a[abc]*\ <span class="string">?d</span>* <span class="params">||</span> <span class="number">3</span> &gt; <span class="number">2</span> ]]; echo $?</span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def d,x"</span> == a[abc]*\ <span class="string">?d</span>* <span class="params">||</span> a &gt; <span class="number">2</span> ]]; echo $?</span><br><span class="line"><span class="number">0</span></span><br><span class="line">[ian@pinguino ~]$ [[ <span class="string">"abc def d,x"</span> == a[abc]*\ <span class="string">?d</span>* <span class="params">||</span> a -gt <span class="number">2</span> ]]; echo $?</span><br><span class="line">-<span class="symbol">bash:</span> <span class="symbol">a:</span> unbound variable</span><br></pre></td></tr></table></figure></p><h2 id="条件测试"><a href="#条件测试" class="headerlink" title="条件测试"></a>条件测试</h2><p>虽然使用以上的测试和 &amp;&amp;、 || 控制操作符能实现许多编程，但 bash 还包含了更熟悉的 “if, then, else” 和 case 结构。学习完这些之后，将学习循环结构，这样您的工具箱将真正得到扩展。</p><h3 id="If、then、else-语句"><a href="#If、then、else-语句" class="headerlink" title="If、then、else 语句"></a>If、then、else 语句</h3><p>bash 的 if 命令是个复合命令，它测试一个测试或命令（$?）的返回值，并根据返回值为 True（0）或 False（不为 0）进行分支。虽然上面的测试只返回 0 或 1 值，但命令可能返回其他值。请参阅 LPI 102 考试准备，主题 109: Shell、脚本、编程和编译 教程学习这方面的更多内容。</p><p>Bash 中的 if 命令有一个 then 子句，子句中包含测试或命令返回 0 时要执行的命令列表，可以有一个或多个可选的 elif 子句，每个子句可执行附加的测试和一个 then 子句，子句中又带有相关的命令列表，最后是可选的 else 子句及命令列表，在前面的测试或 elif 子句中的所有测试都不为真的时候执行，最后使用 fi 标记表示该结构结束。</p><p>使用迄今为止学到的东西，现在能够构建简单的计算器来计算算术表达式，如清单 9 所示：</p><p>清单 9. 用 if、then、else 计算表达式<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]$ function mycalc ()</span><br><span class="line"><span class="meta">&gt;</span><span class="bash"> &#123;</span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">   <span class="built_in">local</span> x</span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">   <span class="keyword">if</span> [ <span class="variable">$#</span> -lt 1 ]; <span class="keyword">then</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">     <span class="built_in">echo</span> <span class="string">"This function evaluates arithmetic for you if you give it some"</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">   <span class="keyword">elif</span> (( $* )); <span class="keyword">then</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">     <span class="built_in">let</span> x=<span class="string">"$*"</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">     <span class="built_in">echo</span> <span class="string">"$* = <span class="variable">$x</span>"</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">   <span class="keyword">else</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">     <span class="built_in">echo</span> <span class="string">"$* = 0 or is not an arithmetic expression"</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash">   <span class="keyword">fi</span></span></span><br><span class="line"><span class="meta">&gt;</span><span class="bash"> &#125;</span></span><br><span class="line">[ian@pinguino ~]$ mycalc 3 + 4</span><br><span class="line">3 + 4 = 7</span><br><span class="line">[ian@pinguino ~]$ mycalc 3 + 4**3</span><br><span class="line">3 + 4**3 = 67</span><br><span class="line">[ian@pinguino ~]$ mycalc 3 + (4**3 /2)</span><br><span class="line">-bash: syntax error near unexpected token `('</span><br><span class="line">[ian@pinguino ~]$ mycalc 3 + "(4**3 /2)"</span><br><span class="line">3 + (4**3 /2) = 35</span><br><span class="line">[ian@pinguino ~]$ mycalc xyz</span><br><span class="line">xyz = 0 or is not an arithmetic expression</span><br><span class="line">[ian@pinguino ~]$ mycalc xyz + 3 + "(4**3 /2)" + abc</span><br><span class="line">xyz + 3 + (4**3 /2) + abc = 35</span><br></pre></td></tr></table></figure></p><p>这个计算器利用 local 语句将 x 声明为局部变量，只能在 mycalc 函数的范围内使用。let 函数具有几个可用的选项，可以执行与它密切关联的 declare 函数。请参考 bash 手册或使用 help let 获得更多信息。</p><p>如清单 9 所示，需要确保在表达式使用 shell 元字符 —— 例如(、)、*、&gt; 和 &lt; 时 —— 正确地对表达式转义。无论如何，现在有了一个非常方便的小计算器，可以像 shell 那样进行算术计算。</p><p>在清单 9 中可能注意到 else 子句和最后的两个示例。可以看到，把 xyz 传递给 mycalc 并没有错误，但计算结果为 0。这个函数还不够灵巧，不能区分最后使用的示例中的字符值，所以不能警告用户。可以使用字符串模式匹配测试（例如<br>[[ ! (“$<em>“ == </em>[a-zA-Z]* ]]<br>，或使用适合自己范围的形式）消除包含字母表字符的表达式，但是这会妨碍在输入中使用 16 进制标记，因为使用 16 进制标记时可能要用 0x0f 表示 15。实际上，shell 允许的基数最高为 64（使用 base#value 标记），所以可以在输入中加入 _ 和 @ 合法地使用任何字母表字符。8 进制和 16 进制使用常用的标记方式，开头为 0 表示八进制，开头为 0x 或 0X 表示 16 进制。清单 10 显示了一些示例。</p><p>清单 10. 用不同的基数进行计算<br><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]$ mycalc <span class="number">015</span></span><br><span class="line"><span class="number">015</span> = <span class="number">13</span></span><br><span class="line">[ian@pinguino ~]$ mycalc <span class="number">0xff</span></span><br><span class="line"><span class="number">0xff</span> = <span class="number">255</span></span><br><span class="line">[ian@pinguino ~]$ mycalc <span class="number">29</span>#<span class="number">37</span></span><br><span class="line"><span class="number">29</span>#<span class="number">37</span> = <span class="number">94</span></span><br><span class="line">[ian@pinguino ~]$ mycalc <span class="number">64</span>#<span class="number">1</span>az</span><br><span class="line"><span class="number">64</span>#<span class="number">1</span>az = <span class="number">4771</span></span><br><span class="line">[ian@pinguino ~]$ mycalc <span class="number">64</span>#<span class="number">1</span>azA</span><br><span class="line"><span class="number">64</span>#<span class="number">1</span>azA = <span class="number">305380</span></span><br><span class="line">[ian@pinguino ~]$ mycalc <span class="number">64</span>#<span class="number">1</span>azA_@</span><br><span class="line"><span class="number">64</span>#<span class="number">1</span>azA_@ = <span class="number">1250840574</span></span><br><span class="line">[ian@pinguino ~]$ mycalc <span class="number">64</span>#<span class="number">1</span>az*<span class="number">64</span>**<span class="number">3</span> + <span class="number">64</span>#A_@</span><br><span class="line"><span class="number">64</span>#<span class="number">1</span>az*<span class="number">64</span>**<span class="number">3</span> + <span class="number">64</span>#A_@ = <span class="number">1250840574</span></span><br></pre></td></tr></table></figure></p><p>对输入进行的额外处理超出了本技巧的范围，所以请小心使用这个计算器。</p><p>elif 语句非常方便。它允许简化缩进，从而有助于脚本编写。在清单 11 中可能会对 type 命令在 mycalc 函数中的输出感到惊讶。</p><p>清单 11. Type mycalc<br><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]$<span class="built_in"> type </span>mycalc</span><br><span class="line">mycalc is a function</span><br><span class="line">mycalc ()</span><br><span class="line">&#123;</span><br><span class="line">    local x;</span><br><span class="line">    <span class="keyword">if</span> [ <span class="variable">$#</span> -lt 1 ]; then</span><br><span class="line">        echo <span class="string">"This function evaluates arithmetic for you if you give it some"</span>;</span><br><span class="line">    <span class="keyword">else</span></span><br><span class="line">        <span class="keyword">if</span> (( $* )); then</span><br><span class="line">            let <span class="attribute">x</span>=<span class="string">"$*"</span>;</span><br><span class="line">            echo <span class="string">"$* = <span class="variable">$x</span>"</span>;</span><br><span class="line">        <span class="keyword">else</span></span><br><span class="line">            echo <span class="string">"$* = 0 or is not an arithmetic expression"</span>;</span><br><span class="line">        fi;</span><br><span class="line">    fi</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></p><p>当然，也可以只用 $(( 表达式 )) 和 echo 命令进行 shell 算术运算，如清单 12 所示。这样就不必学习关于函数或测试的任何内容，但是请注意 shell 不会解释元字符，例如 *，因此元字符不能在 (( 表达式 )) 或 [[ 表达式 ]] 中那样正常发挥作用。</p><p>清单 12. 在 shell 中用 echo 和 $(( )) 直接进行计算<br><figure class="highlight lsl"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[ian@pinguino ~]$  echo $((<span class="number">3</span> + (<span class="number">4</span>**<span class="number">3</span> /<span class="number">2</span>)))</span><br><span class="line"><span class="number">35</span></span><br></pre></td></tr></table></figure></p>]]></content>
    
    <summary type="html">
    
      linux shell test and compare syntex
    
    </summary>
    
      <category term="bash" scheme="http://yoursite.com/categories/bash/"/>
    
    
      <category term="linux" scheme="http://yoursite.com/tags/linux/"/>
    
      <category term="bash" scheme="http://yoursite.com/tags/bash/"/>
    
  </entry>
  
  <entry>
    <title>kubernetes 节点管理</title>
    <link href="http://yoursite.com/2019/01/17/kubernetes-%E8%8A%82%E7%82%B9%E7%AE%A1%E7%90%86/"/>
    <id>http://yoursite.com/2019/01/17/kubernetes-节点管理/</id>
    <published>2019-01-16T23:41:25.000Z</published>
    <updated>2019-01-17T13:39:10.554Z</updated>
    
    <content type="html"><![CDATA[<p>source: <a href="https://blog.csdn.net/dkfajsldfsdfsd/article/details/80985062" target="_blank" rel="noopener">https://blog.csdn.net/dkfajsldfsdfsd/article/details/80985062</a><br>翻译自: <a href="https://kubernetes.io/docs/concepts/architecture/nodes/" target="_blank" rel="noopener">https://kubernetes.io/docs/concepts/architecture/nodes/</a></p><p>在自己搭建的单点集群中，slaver虚机的IP发生变化时，在master上还总是能看到之前的结点信息。<br>下面的内容，解释了为什么失联的node不会被删掉。</p><p>－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－</p><p>node是kubernetes集群中的工作节点，可以是虚拟机也可以是物理机。node节点上运行一些服务用来运行pod以及与master通信等。一个node上的服务包括docker运行时环境、kubelet、kube-proxy以及其它一些可选的add-ons。</p><h1 id="节点状态"><a href="#节点状态" class="headerlink" title="节点状态"></a>节点状态</h1><p>node的状态包含如下几个方面的信息:<br><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">Addresses</span><br><span class="line">Condition</span><br><span class="line">Capacity</span><br><span class="line">Info</span><br><span class="line">Addresses</span><br></pre></td></tr></table></figure></p><p>节点的地址信息由node的配置决定，包含如下三项:<br><figure class="highlight crmsh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">HostName：同<span class="keyword">node</span><span class="title">内核决定的主机名，可以由kubelet</span>的--hostname-override选项覆盖。</span><br><span class="line">ExternalIP：外网访问IP。</span><br><span class="line">InternalIP：内网IP，集群内部的私有地址。</span><br></pre></td></tr></table></figure></p><h2 id="Conditions"><a href="#Conditions" class="headerlink" title="Conditions"></a>Conditions</h2><p>Conditions描述所有处于RUNNING状态的节点信息。</p><table><thead><tr><th style="text-align:center">Node Condition</th><th style="text-align:left">Description</th></tr></thead><tbody><tr><td style="text-align:center">OutOfDisk</td><td style="text-align:left"><em>True</em> if there is insufficient free space on the node for adding new pods, otherwise <em>False</em></td></tr><tr><td style="text-align:center">Ready</td><td style="text-align:left"><em>True</em> if the node is healthy and ready to accept pods,<br> <em>False</em> if the node is not healthy and is not accepting pods, and<br> <em>Unknown</em> if the node controller has not heard from the node in the last <em>node-monitor-grace-period</em> (default is 40 seconds)</td></tr><tr><td style="text-align:center">MemoryPressure</td><td style="text-align:left"><em>True</em> if pressure exists on the node memory – that is, if the node memory is low; otherwise <em>False</em></td></tr><tr><td style="text-align:center">PIDPressure</td><td style="text-align:left"><em>True</em> if pressure exists on the processes – that is, if there are too many processes on the node; otherwise <em>False</em></td></tr><tr><td style="text-align:center">DiskPressure</td><td style="text-align:left"><em>True</em> if pressure exists on the disk size – that is, if the disk capacity is low; otherwise <em>False</em></td></tr><tr><td style="text-align:center">NetworkUnavailable</td><td style="text-align:left"><em>True</em> if the network for the node is not correctly configured, otherwise <em>False</em></td></tr><tr><td style="text-align:center">ConfigOK</td><td style="text-align:left"><em>True</em> if the kubelet is correctly configured, otherwise <em>False</em></td></tr></tbody></table><p>关键问题是以上这些信息是如何计算判定的呢？</p><p>Node的conditions用json对象表示，如以下应答表示健康的node：<br><figure class="highlight prolog"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">"conditions"</span>: [</span><br><span class="line">  &#123;</span><br><span class="line">    <span class="string">"type"</span>: <span class="string">"Ready"</span>,</span><br><span class="line">    <span class="string">"status"</span>: <span class="string">"True"</span></span><br><span class="line">  &#125;</span><br><span class="line">]</span><br></pre></td></tr></table></figure></p><p>如果节点的Ready condition是“Unknown” 或者 “False”，kube-controller-manager则将这个节点标记为不可用状态。节点上所有的pod被Node Controller调度排除，默认的排除超时时间是５分钟。有时节点因为网络原因不可达，kubernetes的控制面无法与节点上的kubelet建立联系，删除pod的指令无法下发到kubelet上，则可能的情况是控制面一直试图联系失联节点直到通信恢复，在此期间在此失联节点上的pod一直持续运行。</p><p>在kubernetes１.５及以前版本中，node controller可能会强制从系统中删除不可达的pod。但是，在１.５及更高的版本中，node controller直到明确确认这些pod停止运行后才会将它们从系统记录中删除，也就是说node controller必需明确知道pod的状态如“Terminating” 或者 “Unknown”。对于永久失联的node，kubernetes无法确定这个node是暂时失联稍后恢复还是永久从集群中删除，在这种情况下需要管理员手动删除node对象，删除node会引发kubernetes删除节点上的所有pod，并且释放它们所占用的名称。</p><p>从1.8版本开始介绍了一种新的测试特性，就是自动创建taints表示conditions。要想打开这种特性，需要给API server，controller manager，scheduler传递一个开关： –feature-gates=…,TaintNodesByCondition=true。当TaintNodesByCondition被打开后，scheduler在调度时就会忽略掉节点的conditions，代之以查看节点的taints与pod的tolerations。</p><p>现在用户可以在旧的调度模式与新的调度模式，一种更灵活的调度模式之间作出选择。对于没有任何tolerations的pod使用旧的调度模式，但是，如果pod能够容忍在特定节点上被污染的话，就会被调度到那个特定节点上。</p><p>需要注意：node的condition被观测到taints被创建的时间跨度，通常小于１秒，因为这种时间延迟，有可能会轻微的增加pod被成功调度出去，但是kubelet却拒绝接收的情况。</p><h2 id="Capacity"><a href="#Capacity" class="headerlink" title="Capacity"></a>Capacity</h2><p>描述节点的可用资源：CPU、内在以及其上可以调度的最大的pod数等。</p><h2 id="Info"><a href="#Info" class="headerlink" title="Info"></a>Info</h2><p>节点常规信息，如内核版本、kubernetes版本，docker版本、操作系统名称等。这些信息由节点上的kubelet提供。</p><h1 id="节点管理"><a href="#节点管理" class="headerlink" title="节点管理"></a>节点管理</h1><p>node不同于pod、service等kubernetes内部创建的对象，它是由外部创建。当在kubernetes内部创建node时，只是在kubernetes内部创建一个表示已经存在节点的对象。创建完以后kubernetes会检查节点是否可用，比如创建了如下节点:<br><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  <span class="attr">"kind"</span>: <span class="string">"Node"</span>,</span><br><span class="line">  <span class="attr">"apiVersion"</span>: <span class="string">"v1"</span>,</span><br><span class="line">  <span class="attr">"metadata"</span>: &#123;</span><br><span class="line">    <span class="attr">"name"</span>: <span class="string">"10.240.79.157"</span>,</span><br><span class="line">    <span class="attr">"labels"</span>: &#123;</span><br><span class="line">      <span class="attr">"name"</span>: <span class="string">"my-first-k8s-node"</span></span><br><span class="line">    &#125;</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></p><p>kubernetes由提供的信息在内部创建一个代表节点的node对象，然后对节点执行可用性检查，如必需的服务是否在正常运行，是否具备运行pod的资格。如果不可用则不让这个节点参与任何集群活动。系统会保留不可用node并且持续监控它的状态，直到它可用为止，或者是用户明确删除创建的node对象。</p><p>目前，有三个组件会与kubernetes的node接口交互：node controller, kubelet, and kubectl。</p><h2 id="节点控制器"><a href="#节点控制器" class="headerlink" title="节点控制器"></a>节点控制器</h2><p>Node controller是kubernetes控制面中的一个组件，负责管理node的方方面面。</p><p>在node的整个生命周期中，node controller充当多种角色，首先在node注册入集群时为它分配一个CIDR地址块（如果此特性打开的话）。</p><p>第二个是维护node controller内部的可用节点列表，并与低层的云供商的可用节点的列表保持一致。当运行在云环境中时，如果node controller检测到某个节点不健康，它就会向供应商询问节点虚拟机是否可用，如果不可用就从列表删除。</p><p>第三个是监控节点的健康状态。Node controller检测到node不可达时，负责将node状态中的NodeReady condition更新到ConditionUnknown，然后将node上的所有pod排除（如果节点不可达的时候超过40s（默认值），会被标记成ConditionUnknown，此时并没有发生排除pod的动作，系统试图重新联系不可达的node，如果恢复的话则node重新被标记成NodeReady。如果不可达的时间超过５分钟（默认值）才会采取排除步骤）。系统每隔–node-monitor-period设置的秒数检测一次节点的状态。</p><p>在kubernetes１.４中，改善了node controller处理集群中node数过多的逻辑。从1.4开始，node controller在决定是否排除pod之前，先检测一次集群中所有node的状态。<br>在多数情况下，node controller通过–node-eviction-rate的设定值限制了排除pod的速率,默认是0.1次每秒，意思是对于单个node，排除的速率小于10秒一次。</p><p>当node在一个可用的zone变得不健康时，node的排除行为会发生改变，也就是说排除行为并不是一成不变。Node controller检测在zone内不健康node的百分数，如果不健康node的比例低于–unhealthy-zone-threshold (default 0.55)，那么排除速率将会降低：如果集群规模小（node数小于或者等于–large-cluster-size-threshold，默认50），则排除行为终止，则否，排除速率将会降低到–secondary-node-eviction-rate（默认0.01）每秒。如此策略的主要原因主要是为在将一个可用的zone分区时提供一种过渡缓冲。如果你的集群没有跨多个云供应商的话，那么你只有一个可用的zone，就是整个集群。</p><p>将node分布在不同可用zone的主要原因是当一个zone整体不能用时，可以将workload切换到另一个可用的zone。所以当一个zone不可用时，kubernetes以正常速率排除。极端情况下，如果所有zone都不可用，那么kubernetes假定master之间的联通性出现了问题，停止一切排除行为直到master之间的联通性恢复。</p><p>从1.6版本开始，当pod不容忍taints时，node controller也负责排除运行在node上的NoExecute taints，这是一个测试特性，默认禁止。Node controller负责添加与node问题对应的taint，比如不可达。</p><p>从1.8版本开始，node controller能够被用来创建表示node conditions的taint，这是1.8版本的一个测试特性。</p><h2 id="自注册节点"><a href="#自注册节点" class="headerlink" title="自注册节点"></a>自注册节点</h2><p>当node中kubelete的标志–register-node被设置成true时，kubelete尝试主动注册自已到系统中。这是一种被大多数distros使用的比较好的模式。本质是是由kubelete在系统中创建node对象而非用户手动。<br>自注册node的kubelet需要指定如下几个启动参数：</p><ul><li><strong>–kubeconfig</strong> - Path to credentials to authenticate itself to the apiserver.</li><li><strong>–cloud-provider</strong> - How to talk to a cloud provider to read metadata about itself.</li><li><strong>–register-node</strong> - Automatically register with the API server.</li><li><strong>–register-with-taints</strong> - Register the node with the given list of taints (comma separated <key>=<value>:<effect>). No-op if register-node is false.</effect></value></key></li><li><strong>–node-ip</strong> - IP address of the node.</li><li><strong>–node-labels</strong> - Labels to add when registering the node in the cluster.</li><li><strong>–node-status-update-frequency</strong> - Specifies how often kubelet posts node status to master.<br>目前kubelete被授权任意创建node对象，但在被际操作中它只创建、修改它自己。在将来的版本中计划将kubelete的授权收缩，使它只能创建、修改它自己。</li></ul><h1 id="手动管理node"><a href="#手动管理node" class="headerlink" title="手动管理node"></a>手动管理node</h1><p>集群管理员可以手动创建、修改node对象，当然先要把node的自注册功能关掉，设置node的kubelete标志–register-node=false。</p><p>用户可以手动设置node标签、或者使node不可以被安排调度。</p><p>标签可以被node selectors用来聚合node从而控制调度，比如约束pod只能调度到符合条件的node上。</p><p>使node不可调度，可以阻止node授受新的调度任务，对已经存在的pod没有影响。这个特性很有用。使node不可调度可运行如下命令：</p><pre>kubectl cordon <b>$NODENAME</b></pre><p>注意，使node不可调度，对DaemonSet类型的pod无效。</p><h2 id="Node-capacity"><a href="#Node-capacity" class="headerlink" title="Node capacity"></a>Node capacity</h2><p>Node的capacity也是node对象的一部分。自注册节点在创建node对象时自动注册此部分内容。但是，如果是手动管理node对象的话，则也需要手动设置此部分内部。</p><p>Kubernetes scheduler在调度pod之前，首先要确保node上有足够运行pod的资源，它要确保node上运行的所有container占用的资源不能超过节点的 capacity。但是，默认情况下在统计node的资源占用情况时，只计算由kubelete启动的容器，其它进程占用的资源不在统计之内。</p><p>如果想明确检视非pod进程占用的资源的话，按如下模板在每个node节点上创建manifest类型的pod：<br><figure class="highlight less"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="attribute">apiVersion</span>: v1</span><br><span class="line"><span class="attribute">kind</span>: Pod</span><br><span class="line"><span class="attribute">metadata</span>:</span><br><span class="line">  <span class="attribute">name</span>: resource-reserver</span><br><span class="line"><span class="attribute">spec</span>:</span><br><span class="line">  <span class="attribute">containers</span>:</span><br><span class="line">  - <span class="attribute">name</span>: sleep-forever</span><br><span class="line">    <span class="attribute">image</span>: k8s.gcr.io/<span class="attribute">pause</span>:<span class="number">0.8</span>.<span class="number">0</span></span><br><span class="line">    <span class="attribute">resources</span>:</span><br><span class="line">      <span class="attribute">requests</span>:</span><br><span class="line">        <span class="attribute">cpu</span>: <span class="number">100</span>m</span><br><span class="line">        <span class="attribute">memory</span>: <span class="number">100</span>Mi</span><br></pre></td></tr></table></figure></p>]]></content>
    
    <summary type="html">
    
      kubernetes节点管理
    
    </summary>
    
      <category term="kubernetes" scheme="http://yoursite.com/categories/kubernetes/"/>
    
    
      <category term="k8s" scheme="http://yoursite.com/tags/k8s/"/>
    
      <category term="nodes" scheme="http://yoursite.com/tags/nodes/"/>
    
  </entry>
  
  <entry>
    <title>kube-apiserver启动失败之etcd配置</title>
    <link href="http://yoursite.com/2019/01/06/kube-apiserver%E5%90%AF%E5%8A%A8%E5%A4%B1%E8%B4%A5%E4%B9%8Betcd%E9%85%8D%E7%BD%AE/"/>
    <id>http://yoursite.com/2019/01/06/kube-apiserver启动失败之etcd配置/</id>
    <published>2019-01-06T14:17:00.000Z</published>
    <updated>2019-01-06T14:25:17.701Z</updated>
    
    <content type="html"><![CDATA[<p>在配置了认证之后，apiserver服务启动时有时候会访问etcd集群失败。<br>原因和etcd服务的配置有关。在其配置文件中，有几个访问URL相关的参数，<br>下面是本地能正常启动服务的配置， 要注意其http  和  http_s_的区别。</p><figure class="highlight vala"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"># Human_readable name for this member.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "default"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_NAME="default"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the data directory.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "$&#123;name&#125;.etcd"</span></span><br><span class="line"><span class="meta"># distribution default: "/var/lib/etcd"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_DATA_DIR="/var/lib/etcd"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the dedicated wal directory.</span></span><br><span class="line"><span class="meta"># If this flag is set, etcd will write the WAL files</span></span><br><span class="line"><span class="meta"># to the walDir rather than the dataDir.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: ""</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_WAL_DIR=""</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Number of committed transactions to trigger a snapshot to disk.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 10000</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_SNAPSHOT_COUNT=10000</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Time (in milliseconds) of a heartbeat interval.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 100</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_HEARTBEAT_INTERVAL=100</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Time (in milliseconds) for an election to timeout.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 1000</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_ELECTION_TIMEOUT=1000</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># List of URLs to listen on for peer traffic.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "http://localhost:2380,http://localhost:7001"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_LISTEN_PEER_URLS="http://localhost:2380,http://localhost:7001"</span></span><br><span class="line">ETCD_LISTEN_PEER_URLS=<span class="string">"https://192.168.1.3:2380"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># List of URLs to listen on for client traffic.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "http://localhost:2379,http://localhost:4001"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_LISTEN_CLIENT_URLS="http://localhost:2379,http://localhost:4001"</span></span><br><span class="line">ETCD_LISTEN_CLIENT_URLS=<span class="string">"http://192.168.1.3:2379,http://127.0.0.1:2379"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Maximum number of snapshot files to retain (0 is unlimited)</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 5</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_MAX_SNAPSHOTS=5</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Maximum number of wal files to retain (0 is unlimited)</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 5</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_MAX_WALS=5</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Comma_separated white list of origins for CORS (cross_origin resource sharing).</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_CORS=</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># List of this member's peer URLs to advertise to the rest of the cluster.</span></span><br><span class="line"><span class="meta"># These addresses are used for communicating etcd data around the cluster.</span></span><br><span class="line"><span class="meta"># At least one must be routable to all cluster members.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "http://localhost:2380,http://localhost:7001"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380,http://localhost:7001"</span></span><br><span class="line">ETCD_INITIAL_ADVERTISE_PEER_URLS=<span class="string">"https://192.168.1.3:2380"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Initial cluster configuration for bootstrapping.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "default=http://localhost:2380,default=http://localhost:7001"</span></span><br><span class="line"><span class="meta"># distribution default: "default=http://localhost:2380,default=http://localhost:7001"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_INITIAL_CLUSTER="default=http://localhost:2380,default=http://localhost:7001"</span></span><br><span class="line"><span class="meta"># ETCD_INITIAL_CLUSTER="default=https://192.168.1.3:2380"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Initial cluster state ("new" or "existing").</span></span><br><span class="line"><span class="meta"># Set to new for all members present during initial static or DNS bootstrapping.</span></span><br><span class="line"><span class="meta"># If this option is set to existing, etcd will attempt to join the existing cluster.</span></span><br><span class="line"><span class="meta"># If the wrong value is set, etcd will attempt to start but fail safely.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "new"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_INITIAL_CLUSTER_STATE="new"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Initial cluster token for the etcd cluster during bootstrap.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "etcd_cluster"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_INITIAL_CLUSTER_TOKEN="etcd_cluster"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># List of this member's client URLs to advertise to the rest of the cluster.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "http://localhost:2379,http://localhost:4001"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379,http://localhost:4001"</span></span><br><span class="line">ETCD_ADVERTISE_CLIENT_URLS=<span class="string">"https://192.168.1.3:2379"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Discovery URL used to bootstrap the cluster.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_DISCOVERY=</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># DNS srv domain used to bootstrap the cluster.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_DISCOVERY_SRV=</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Expected behavior ("exit" or "proxy") when discovery services fails.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "proxy"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_DISCOVERY_FALLBACK="proxy"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># HTTP proxy to use for traffic to discovery service.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_DISCOVERY_PROXY=</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Proxy mode setting ("off", "readonly" or "on").</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: "off"</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PROXY="off"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Time (in milliseconds) an endpoint will be held</span></span><br><span class="line"><span class="meta"># in a failed state before being reconsidered for proxied requests.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 5000</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PROXY_FAILURE_WAIT=5000</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Time (in milliseconds) of the endpoints refresh interval.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 30000</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PROXY_REFRESH_INTERVAL=30000</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Time (in milliseconds) for a dial to timeout or 0 to disable the timeout.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 1000</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PROXY_DIAL_TIMEOUT=1000</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Time (in milliseconds) for a write to timeout or 0 to disable the timeout.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 5000</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PROXY_WRITE_TIMEOUT=5000</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Time (in milliseconds) for a read to timeout or 0 to disable the timeout.</span></span><br><span class="line"><span class="meta"># Don't change this value if you use watches because they are using long polling requests.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: 0</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PROXY_READ_TIMEOUT=0</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the client server TLS CA file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_CA_FILE=</span></span><br><span class="line">ETCD_CA_FILE=/etc/kubernetes/pki/apiserver/ca.pem</span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the client server TLS cert file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_CERT_FILE=</span></span><br><span class="line">ETCD_CERT_FILE=<span class="string">"/etc/kubernetes/pki/apiserver/etcd.pem"</span></span><br><span class="line"><span class="meta">#ETCD_CERT_FILE="--cert-file=/etc/kubernetes/pki/apiserver/etcd.pem"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the client server TLS key file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_KEY_FILE=</span></span><br><span class="line">ETCD_KEY_FILE=<span class="string">"/etc/kubernetes/pki/apiserver/etcd-key.pem"</span></span><br><span class="line"><span class="meta">#ETCD_KEY_FILE="--key-file=/etc/kubernetes/pki/apiserver/etcd-key.pem"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Enable client cert authentication.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: false</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_CLIENT_CERT_AUTH=false</span></span><br><span class="line">ETCD_CLIENT_CERT_AUTH=<span class="literal">true</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the client server TLS trusted CA key file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_TRUSTED_CA_FILE=</span></span><br><span class="line">ETCD_TRUSTED_CA_FILE=<span class="string">"/etc/kubernetes/pki/apiserver/ca.pem"</span></span><br><span class="line"><span class="meta">#ETCD_TRUSTED_CA_FILE="--trusted-ca-file=/etc/kubernetes/pki/apiserver/ca.pem"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># [DEPRECATED] Path to the peer server TLS CA file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PEER_CA_FILE=</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the peer server TLS cert file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PEER_CERT_FILE=</span></span><br><span class="line"><span class="meta">#ETCD_PEER_CERT_FILE="--peer-cert-file=/etc/kubernetes/pki/apiserver/etcd.pem"</span></span><br><span class="line">ETCD_PEER_CERT_FILE=<span class="string">"/etc/kubernetes/pki/apiserver/etcd.pem"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the peer server TLS key file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PEER_KEY_FILE=</span></span><br><span class="line">ETCD_PEER_KEY_FILE=<span class="string">"/etc/kubernetes/pki/apiserver/etcd-key.pem"</span></span><br><span class="line"><span class="meta">#ETCD_PEER_KEY_FILE="--peer-key-file=/etc/kubernetes/pki/apiserver/etcd-key.pem"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Enable peer client cert authentication.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: false</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PEER_CLIENT_CERT_AUTH=false</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Path to the peer server TLS trusted CA file.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_PEER_TRUSTED_CA_FILE=</span></span><br><span class="line">ETCD_PEER_TRUSTED_CA_FILE=<span class="string">"/etc/kubernetes/pki/apiserver/ca.pem"</span></span><br><span class="line"><span class="meta">#ETCD_PEER_TRUSTED_CA_FILE="--peer-trusted-ca-file=/etc/kubernetes/pki/apiserver/ca.pem"</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Drop the default log level to DEBUG for all subpackages.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: false (INFO for all packages)</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_DEBUG=false</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Set individual etcd subpackages to specific log levels.</span></span><br><span class="line"><span class="meta"># An example being etcdserver=WARNING,security=DEBUG</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: none (INFO for all packages)</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_LOG_PACKAGE_LEVELS=</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># Force to create a new one_member cluster.</span></span><br><span class="line"><span class="meta"># It commits configuration changes in force to remove all existing members in the cluster and add itself.</span></span><br><span class="line"><span class="meta"># It needs to be set to restore a backup.</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># default: false</span></span><br><span class="line"><span class="meta">#</span></span><br><span class="line"><span class="meta"># ETCD_FORCE_NEW_CLUSTER=false</span></span><br><span class="line"></span><br><span class="line"><span class="meta"># vim:ft=sh:</span></span><br></pre></td></tr></table></figure>]]></content>
    
    <summary type="html">
    
      kube-apiserver因为连接etcd失败导致服务启动失败，这和etcd的配置有关
    
    </summary>
    
      <category term="kubernetes" scheme="http://yoursite.com/categories/kubernetes/"/>
    
    
      <category term="etcd" scheme="http://yoursite.com/tags/etcd/"/>
    
      <category term="k8s" scheme="http://yoursite.com/tags/k8s/"/>
    
  </entry>
  
  <entry>
    <title>Kubernetes with CA</title>
    <link href="http://yoursite.com/2019/01/05/kubernetes-with-ca/"/>
    <id>http://yoursite.com/2019/01/05/kubernetes-with-ca/</id>
    <published>2019-01-05T02:35:48.000Z</published>
    <updated>2019-01-06T14:25:44.061Z</updated>
    
    <content type="html"><![CDATA[<p>文中涉及到的文件，可到<a href="https://github.com/liuyi01/kubernetes-starter.git" target="_blank" rel="noopener">github-kubernetes-starter</a>获取。</p><h1 id="完整集群部署-kubernetes-with-ca"><a href="#完整集群部署-kubernetes-with-ca" class="headerlink" title="完整集群部署 - kubernetes-with-ca"></a>完整集群部署 - kubernetes-with-ca</h1><h2 id="1-理解认证授权"><a href="#1-理解认证授权" class="headerlink" title="1. 理解认证授权"></a>1. 理解认证授权</h2><h4 id="1-1-为什么要认证"><a href="#1-1-为什么要认证" class="headerlink" title="1.1 为什么要认证"></a>1.1 为什么要认证</h4><p>想理解认证，我们得从认证解决什么问题、防止什么问题的发生入手。<br>防止什么问题呢？是防止有人入侵你的集群，root你的机器后让我们集群依然安全吗？不是吧，root都到手了，那就为所欲为，防不胜防了。<br>其实网络安全本身就是为了解决在某些假设成立的条件下如何防范的问题。比如一个非常重要的假设就是两个节点或者ip之间的通讯网络是不可信任的，可能会被第三方窃取，也可能会被第三方篡改。就像我们上学时候给心仪的女孩传纸条，传送的过程可能会被别的同学偷看，甚至内容可能会从我喜欢你修改成我不喜欢你了。当然这种假设不是随便想出来的，而是从网络技术现状和实际发生的问题中发现、总结出来的。kubernetes的认证也是从这个问题出发来实现的。</p><h4 id="1-2-概念梳理"><a href="#1-2-概念梳理" class="headerlink" title="1.2 概念梳理"></a>1.2 概念梳理</h4><p>为了解决上面说的问题，kubernetes并不需要自己想办法，毕竟是网络安全层面的问题，是每个服务都会遇到的问题，业内也有成熟的方案来解决。这里我们一起了解一下业内方案和相关的概念。</p><ul><li><strong>对称加密/非对称加密</strong><br>这两个概念属于密码学的东西，对于没接触过的同学不太容易理解。可以参考知乎大神的生动讲解：<a href="https://www.zhihu.com/question/33645891/answer/57721969" target="_blank" rel="noopener">《如何用通俗易懂的话来解释非对称加密》</a></li><li><strong>SSL/TLS</strong><br>了解了对称加密和非对称加密后，我们就可以了解一下SSL/TLS了。同样，已经有大神总结了非常好的入门文章：<a href="http://www.ruanyifeng.com/blog/2014/02/ssl_tls.html" target="_blank" rel="noopener">《SSL/TLS协议运行机制的概述》</a></li></ul><h4 id="1-3-什么是授权"><a href="#1-3-什么是授权" class="headerlink" title="1.3 什么是授权"></a>1.3 什么是授权</h4><p>授权的概念就简单多了，就是什么人具有什么样的权限，一般通过角色作为纽带把他们组合在一起。也就是一个角色一边拥有多种权限，一边拥有多个人。这样就把人和权限建立了一个关系。</p><h2 id="2-kubernetes的认证授权"><a href="#2-kubernetes的认证授权" class="headerlink" title="2. kubernetes的认证授权"></a>2. kubernetes的认证授权</h2><p>Kubernetes集群的所有操作基本上都是通过kube-apiserver这个组件进行的，它提供HTTP RESTful形式的API供集群内外客户端调用。需要注意的是：认证授权过程只存在HTTPS形式的API中。也就是说，如果客户端使用HTTP连接到kube-apiserver，那么是不会进行认证授权的。所以说，可以这么设置，在集群内部组件间通信使用HTTP，集群外部就使用HTTPS，这样既增加了安全性，也不至于太复杂。<br>对APIServer的访问要经过的三个步骤，前面两个是认证和授权，第三个是 Admission Control，它也能在一定程度上提高安全性，不过更多是资源管理方面的作用。</p><h4 id="2-1-kubernetes的认证"><a href="#2-1-kubernetes的认证" class="headerlink" title="2.1 kubernetes的认证"></a>2.1 kubernetes的认证</h4><p>kubernetes提供了多种认证方式，比如客户端证书、静态token、静态密码文件、ServiceAccountTokens等等。你可以同时使用一种或多种认证方式。只要通过任何一个都被认作是认证通过。下面我们就认识几个常见的认证方式。</p><ul><li><strong>客户端证书认证</strong><br>客户端证书认证叫作TLS双向认证，也就是服务器客户端互相验证证书的正确性，在都正确的情况下协调通信加密方案。<br>为了使用这个方案，api-server需要用–client-ca-file选项来开启。</li><li><strong>引导Token</strong><br>当我们有非常多的node节点时，手动为每个node节点配置TLS认证比较麻烦，这时就可以用到引导token的认证方式，前提是需要在api-server开启 experimental-bootstrap-token-auth 特性，客户端的token信息与预先定义的token匹配认证通过后，自动为node颁发证书。当然引导token是一种机制，可以用到各种场景中。</li><li><strong>Service Account Tokens 认证</strong><br>有些情况下，我们希望在pod内部访问api-server，获取集群的信息，甚至对集群进行改动。针对这种情况，kubernetes提供了一种特殊的认证方式：Service Account。 Service Account 和 pod、service、deployment 一样是 kubernetes 集群中的一种资源，用户也可以创建自己的 Service Account。<br>ServiceAccount 主要包含了三个内容：namespace、Token 和 CA。namespace 指定了 pod 所在的 namespace，CA 用于验证 apiserver 的证书，token 用作身份验证。它们都通过 mount 的方式保存在 pod 的文件系统中。</li></ul><h4 id="2-2-kubernetes的授权"><a href="#2-2-kubernetes的授权" class="headerlink" title="2.2 kubernetes的授权"></a>2.2 kubernetes的授权</h4><p>在Kubernetes1.6版本中新增角色访问控制机制（Role-Based Access，RBAC）让集群管理员可以针对特定使用者或服务账号的角色，进行更精确的资源访问控制。在RBAC中，权限与角色相关联，用户通过成为适当角色的成员而得到这些角色的权限。这就极大地简化了权限的管理。在一个组织中，角色是为了完成各种工作而创造，用户则依据它的责任和资格来被指派相应的角色，用户可以很容易地从一个角色被指派到另一个角色。<br>目前 Kubernetes 中有一系列的鉴权机制，因为Kubernetes社区的投入和偏好，相对于其它鉴权机制而言，RBAC是更好的选择。具体RBAC是如何体现在kubernetes系统中的我们会在后面的部署中逐步的深入了解。</p><h4 id="2-3-kubernetes的AdmissionControl"><a href="#2-3-kubernetes的AdmissionControl" class="headerlink" title="2.3 kubernetes的AdmissionControl"></a>2.3 kubernetes的AdmissionControl</h4><p>AdmissionControl - 准入控制本质上为一段准入代码，在对kubernetes api的请求过程中，顺序为：先经过认证 &amp; 授权，然后执行准入操作，最后对目标对象进行操作。这个准入代码在api-server中，而且必须被编译到二进制文件中才能被执行。<br>在对集群进行请求时，每个准入控制代码都按照一定顺序执行。如果有一个准入控制拒绝了此次请求，那么整个请求的结果将会立即返回，并提示用户相应的error信息。<br>常用组件（控制代码）如下：</p><ul><li>AlwaysAdmit：允许所有请求</li><li>AlwaysDeny：禁止所有请求，多用于测试环境</li><li>ServiceAccount：它将serviceAccounts实现了自动化，它会辅助serviceAccount做一些事情，比如如果pod没有serviceAccount属性，它会自动添加一个default，并确保pod的serviceAccount始终存在</li><li>LimitRanger：他会观察所有的请求，确保没有违反已经定义好的约束条件，这些条件定义在namespace中LimitRange对象中。如果在kubernetes中使用LimitRange对象，则必须使用这个插件。</li><li>NamespaceExists：它会观察所有的请求，如果请求尝试创建一个不存在的namespace，则这个请求被拒绝。</li></ul><h2 id="3-环境准备"><a href="#3-环境准备" class="headerlink" title="3. 环境准备"></a>3. 环境准备</h2><h4 id="3-1-停止原有kubernetes相关服务"><a href="#3-1-停止原有kubernetes相关服务" class="headerlink" title="3.1 停止原有kubernetes相关服务"></a>3.1 停止原有kubernetes相关服务</h4><p>开始之前我们要先把基础版本的集群停掉，包括service，deployments，pods以及运行的所有kubernetes组件<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#删除services</span></span><br><span class="line">$ kubectl delete services nginx-service</span><br><span class="line"></span><br><span class="line"><span class="comment">#删除deployments</span></span><br><span class="line">$ kubectl delete deploy kubernetes-bootcamp</span><br><span class="line">$ kubectl delete deploy nginx-deployment</span><br><span class="line"></span><br><span class="line"><span class="comment">#停掉worker节点的服务</span></span><br><span class="line">$ service kubelet stop &amp;&amp; rm -fr /var/lib/kubelet/*</span><br><span class="line">$ service kube-proxy stop &amp;&amp; rm -fr /var/lib/kube-proxy/*</span><br><span class="line">$ service kube-calico stop</span><br><span class="line"></span><br><span class="line"><span class="comment">#停掉master节点的服务</span></span><br><span class="line">$ service kube-calico stop</span><br><span class="line">$ service kube-scheduler stop</span><br><span class="line">$ service kube-controller-manager stop</span><br><span class="line">$ service kube-apiserver stop</span><br><span class="line">$ service etcd stop &amp;&amp; rm -fr /var/lib/etcd/*</span><br></pre></td></tr></table></figure></p><h4 id="3-2-生成配置（所有节点）"><a href="#3-2-生成配置（所有节点）" class="headerlink" title="3.2 生成配置（所有节点）"></a>3.2 生成配置（所有节点）</h4><p>跟基础环境搭建一样，我们需要生成kubernetes-with-ca的所有相关配置文件<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter</span><br><span class="line"><span class="comment">#按照配置文件的提示编辑好配置</span></span><br><span class="line">$ vi config.properties</span><br><span class="line"><span class="comment">#生成配置</span></span><br><span class="line">$ ./gen-config.sh with-ca</span><br></pre></td></tr></table></figure></p><h4 id="3-3-安装cfssl（所有节点）"><a href="#3-3-安装cfssl（所有节点）" class="headerlink" title="3.3 安装cfssl（所有节点）"></a>3.3 安装cfssl（所有节点）</h4><p>cfssl是非常好用的CA工具，我们用它来生成证书和秘钥文件<br>安装过程比较简单，如下：<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#下载</span></span><br><span class="line">$ wget -q --show-progress --https-only --timestamping \</span><br><span class="line">  https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 \</span><br><span class="line">  https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64</span><br><span class="line"><span class="comment">#修改为可执行权限</span></span><br><span class="line">$ chmod +x cfssl_linux-amd64 cfssljson_linux-amd64</span><br><span class="line"><span class="comment">#移动到bin目录</span></span><br><span class="line">$ mv cfssl_linux-amd64 /usr/<span class="built_in">local</span>/bin/cfssl</span><br><span class="line">$ mv cfssljson_linux-amd64 /usr/<span class="built_in">local</span>/bin/cfssljson</span><br><span class="line"><span class="comment">#验证</span></span><br><span class="line">$ cfssl version</span><br></pre></td></tr></table></figure></p><h4 id="3-4-生成根证书（主节点）"><a href="#3-4-生成根证书（主节点）" class="headerlink" title="3.4 生成根证书（主节点）"></a>3.4 生成根证书（主节点）</h4><p>根证书是证书信任链的根，各个组件通讯的前提是有一份大家都信任的证书（根证书），每个人使用的证书都是由这个根证书签发的。<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#所有证书相关的东西都放在这</span></span><br><span class="line">$ mkdir -p /etc/kubernetes/ca</span><br><span class="line"><span class="comment">#准备生成证书的配置文件</span></span><br><span class="line">$ cp ~/kubernetes-starter/target/ca/ca-config.json /etc/kubernetes/ca</span><br><span class="line">$ cp ~/kubernetes-starter/target/ca/ca-csr.json /etc/kubernetes/ca</span><br><span class="line"><span class="comment">#生成证书和秘钥</span></span><br><span class="line">$ <span class="built_in">cd</span> /etc/kubernetes/ca</span><br><span class="line">$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca</span><br><span class="line"><span class="comment">#生成完成后会有以下文件（我们最终想要的就是ca-key.pem和ca.pem，一个秘钥，一个证书）</span></span><br><span class="line">$ ls</span><br><span class="line">ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem</span><br></pre></td></tr></table></figure></p><h2 id="4-改造etcd"><a href="#4-改造etcd" class="headerlink" title="4. 改造etcd"></a>4. 改造etcd</h2><h4 id="4-1-准备证书"><a href="#4-1-准备证书" class="headerlink" title="4.1 准备证书"></a>4.1 准备证书</h4><p>etcd节点需要提供给其他服务访问，就要验证其他服务的身份，所以需要一个标识自己监听服务的server证书，当有多个etcd节点的时候也需要client证书与etcd集群其他节点交互，当然也可以client和server使用同一个证书因为它们本质上没有区别。<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#etcd证书放在这</span></span><br><span class="line">$ mkdir -p /etc/kubernetes/ca/etcd</span><br><span class="line"><span class="comment">#准备etcd证书配置</span></span><br><span class="line">$ cp ~/kubernetes-starter/target/ca/etcd/etcd-csr.json /etc/kubernetes/ca/etcd/</span><br><span class="line">$ <span class="built_in">cd</span> /etc/kubernetes/ca/etcd/</span><br><span class="line"><span class="comment">#使用根证书(ca.pem)签发etcd证书</span></span><br><span class="line">$ cfssl gencert \</span><br><span class="line">        -ca=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        -ca-key=/etc/kubernetes/ca/ca-key.pem \</span><br><span class="line">        -config=/etc/kubernetes/ca/ca-config.json \</span><br><span class="line">        -profile=kubernetes etcd-csr.json | cfssljson -bare etcd</span><br><span class="line"><span class="comment">#跟之前类似生成三个文件etcd.csr是个中间证书请求文件，我们最终要的是etcd-key.pem和etcd.pem</span></span><br><span class="line">$ ls</span><br><span class="line">etcd.csr  etcd-csr.json  etcd-key.pem  etcd.pem</span><br></pre></td></tr></table></figure></p><h4 id="4-2-改造etcd服务"><a href="#4-2-改造etcd服务" class="headerlink" title="4.2 改造etcd服务"></a>4.2 改造etcd服务</h4><p>建议大家先比较一下增加认证的etcd配置与原有配置的区别，做到心中有数。<br>可以使用命令比较：<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter/</span><br><span class="line">$ vimdiff kubernetes-simple/master-node/etcd.service kubernetes-with-ca/master-node/etcd.service</span><br></pre></td></tr></table></figure></p><p><strong>更新etcd服务：</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">$ cp ~/kubernetes-starter/target/master-node/etcd.service /lib/systemd/system/</span><br><span class="line">$ systemctl daemon-reload</span><br><span class="line">$ service etcd start</span><br><span class="line"><span class="comment">#验证etcd服务（endpoints自行替换）</span></span><br><span class="line">$ ETCDCTL_API=3 etcdctl \</span><br><span class="line">  --endpoints=https://192.168.1.102:2379  \</span><br><span class="line">  --cacert=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">  --cert=/etc/kubernetes/ca/etcd/etcd.pem \</span><br><span class="line">  --key=/etc/kubernetes/ca/etcd/etcd-key.pem \</span><br><span class="line">  endpoint health</span><br></pre></td></tr></table></figure></p><h2 id="5-改造api-server"><a href="#5-改造api-server" class="headerlink" title="5. 改造api-server"></a>5. 改造api-server</h2><h4 id="5-1-准备证书"><a href="#5-1-准备证书" class="headerlink" title="5.1 准备证书"></a>5.1 准备证书</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#api-server证书放在这，api-server是核心，文件夹叫kubernetes吧，如果想叫apiserver也可以，不过相关的地方都需要修改哦</span></span><br><span class="line">$ mkdir -p /etc/kubernetes/ca/kubernetes</span><br><span class="line"><span class="comment">#准备apiserver证书配置</span></span><br><span class="line">$ cp ~/kubernetes-starter/target/ca/kubernetes/kubernetes-csr.json /etc/kubernetes/ca/kubernetes/</span><br><span class="line">$ <span class="built_in">cd</span> /etc/kubernetes/ca/kubernetes/</span><br><span class="line"><span class="comment">#使用根证书(ca.pem)签发kubernetes证书</span></span><br><span class="line">$ cfssl gencert \</span><br><span class="line">        -ca=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        -ca-key=/etc/kubernetes/ca/ca-key.pem \</span><br><span class="line">        -config=/etc/kubernetes/ca/ca-config.json \</span><br><span class="line">        -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes</span><br><span class="line"><span class="comment">#跟之前类似生成三个文件kubernetes.csr是个中间证书请求文件，我们最终要的是kubernetes-key.pem和kubernetes.pem</span></span><br><span class="line">$ ls</span><br><span class="line">kubernetes.csr  kubernetes-csr.json  kubernetes-key.pem  kubernetes.pem</span><br></pre></td></tr></table></figure><h4 id="5-2-改造api-server服务"><a href="#5-2-改造api-server服务" class="headerlink" title="5.2 改造api-server服务"></a>5.2 改造api-server服务</h4><p><strong>查看diff</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter</span><br><span class="line">$ vimdiff kubernetes-simple/master-node/kube-apiserver.service kubernetes-with-ca/master-node/kube-apiserver.service</span><br></pre></td></tr></table></figure></p><p><strong>生成token认证文件</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#生成随机token</span></span><br><span class="line">$ head -c 16 /dev/urandom | od -An -t x | tr -d <span class="string">' '</span></span><br><span class="line">8afdf3c4eb7c74018452423c29433609</span><br><span class="line"></span><br><span class="line"><span class="comment">#按照固定格式写入token.csv，注意替换token内容</span></span><br><span class="line">$ <span class="built_in">echo</span> <span class="string">"8afdf3c4eb7c74018452423c29433609,kubelet-bootstrap,10001,\"system:kubelet-bootstrap\""</span> &gt; /etc/kubernetes/ca/kubernetes/token.csv</span><br></pre></td></tr></table></figure></p><p><strong>更新api-server服务</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ cp ~/kubernetes-starter/target/master-node/kube-apiserver.service /lib/systemd/system/</span><br><span class="line">$ systemctl daemon-reload</span><br><span class="line">$ service kube-apiserver start</span><br><span class="line"></span><br><span class="line"><span class="comment">#检查日志</span></span><br><span class="line">$ journalctl -f -u kube-apiserver</span><br></pre></td></tr></table></figure></p><h2 id="6-改造controller-manager"><a href="#6-改造controller-manager" class="headerlink" title="6. 改造controller-manager"></a>6. 改造controller-manager</h2><p>controller-manager一般与api-server在同一台机器上，所以可以使用非安全端口与api-server通讯，不需要生成证书和私钥。</p><h4 id="6-1-改造controller-manager服务"><a href="#6-1-改造controller-manager服务" class="headerlink" title="6.1 改造controller-manager服务"></a>6.1 改造controller-manager服务</h4><p><strong>查看diff</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter/</span><br><span class="line">$ vimdiff kubernetes-simple/master-node/kube-controller-manager.service kubernetes-with-ca/master-node/kube-controller-manager.service</span><br></pre></td></tr></table></figure></p><p><strong>更新controller-manager服务</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ cp ~/kubernetes-starter/target/master-node/kube-controller-manager.service /lib/systemd/system/</span><br><span class="line">$ systemctl daemon-reload</span><br><span class="line">$ service kube-controller-manager start</span><br><span class="line"></span><br><span class="line"><span class="comment">#检查日志</span></span><br><span class="line">$ journalctl -f -u kube-controller-manager</span><br></pre></td></tr></table></figure></p><h2 id="7-改造scheduler"><a href="#7-改造scheduler" class="headerlink" title="7. 改造scheduler"></a>7. 改造scheduler</h2><p>scheduler一般与apiserver在同一台机器上，所以可以使用非安全端口与apiserver通讯。不需要生成证书和私钥。</p><h4 id="7-1-改造scheduler服务"><a href="#7-1-改造scheduler服务" class="headerlink" title="7.1 改造scheduler服务"></a>7.1 改造scheduler服务</h4><p><strong>查看diff</strong><br>比较会发现两个文件并没有区别，不需要改造<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter/</span><br><span class="line">$ vimdiff kubernetes-simple/master-node/kube-scheduler.service kubernetes-with-ca/master-node/kube-scheduler.service</span><br></pre></td></tr></table></figure></p><p><strong>启动服务</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ service kube-scheduler start</span><br><span class="line"><span class="comment">#检查日志</span></span><br><span class="line">$ journalctl -f -u kube-scheduler</span><br></pre></td></tr></table></figure></p><h2 id="8-改造kubectl"><a href="#8-改造kubectl" class="headerlink" title="8. 改造kubectl"></a>8. 改造kubectl</h2><h4 id="8-1-准备证书"><a href="#8-1-准备证书" class="headerlink" title="8.1 准备证书"></a>8.1 准备证书</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#kubectl证书放在这，由于kubectl相当于系统管理员，我们使用admin命名</span></span><br><span class="line">$ mkdir -p /etc/kubernetes/ca/admin</span><br><span class="line"><span class="comment">#准备admin证书配置 - kubectl只需客户端证书，因此证书请求中 hosts 字段可以为空</span></span><br><span class="line">$ cp ~/kubernetes-starter/target/ca/admin/admin-csr.json /etc/kubernetes/ca/admin/</span><br><span class="line">$ <span class="built_in">cd</span> /etc/kubernetes/ca/admin/</span><br><span class="line"><span class="comment">#使用根证书(ca.pem)签发admin证书</span></span><br><span class="line">$ cfssl gencert \</span><br><span class="line">        -ca=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        -ca-key=/etc/kubernetes/ca/ca-key.pem \</span><br><span class="line">        -config=/etc/kubernetes/ca/ca-config.json \</span><br><span class="line">        -profile=kubernetes admin-csr.json | cfssljson -bare admin</span><br><span class="line"><span class="comment">#我们最终要的是admin-key.pem和admin.pem</span></span><br><span class="line">$ ls</span><br><span class="line">admin.csr  admin-csr.json  admin-key.pem  admin.pem</span><br></pre></td></tr></table></figure><h4 id="8-2-配置kubectl"><a href="#8-2-配置kubectl" class="headerlink" title="8.2 配置kubectl"></a>8.2 配置kubectl</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#指定apiserver的地址和证书位置（ip自行修改）</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-cluster kubernetes \</span><br><span class="line">        --certificate-authority=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        --embed-certs=<span class="literal">true</span> \</span><br><span class="line">        --server=https://192.168.1.102:6443</span><br><span class="line"><span class="comment">#设置客户端认证参数，指定admin证书和秘钥</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-credentials admin \</span><br><span class="line">        --client-certificate=/etc/kubernetes/ca/admin/admin.pem \</span><br><span class="line">        --embed-certs=<span class="literal">true</span> \</span><br><span class="line">        --client-key=/etc/kubernetes/ca/admin/admin-key.pem</span><br><span class="line"><span class="comment">#关联用户和集群</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-context kubernetes \</span><br><span class="line">        --cluster=kubernetes --user=admin</span><br><span class="line"><span class="comment">#设置当前上下文</span></span><br><span class="line">$ kubectl config use-context kubernetes</span><br><span class="line"></span><br><span class="line"><span class="comment">#设置结果就是一个配置文件，可以看看内容</span></span><br><span class="line">$ cat ~/.kube/config</span><br></pre></td></tr></table></figure><p><strong>验证master节点</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#可以使用刚配置好的kubectl查看一下组件状态</span></span><br><span class="line">$ kubectl get componentstatus</span><br><span class="line">NAME                 STATUS    MESSAGE              ERROR</span><br><span class="line">scheduler            Healthy   ok</span><br><span class="line">controller-manager   Healthy   ok</span><br><span class="line">etcd-0               Healthy   &#123;<span class="string">"health"</span>: <span class="string">"true"</span>&#125;</span><br></pre></td></tr></table></figure></p><h2 id="9-改造calico-node"><a href="#9-改造calico-node" class="headerlink" title="9. 改造calico-node"></a>9. 改造calico-node</h2><h4 id="9-1-准备证书"><a href="#9-1-准备证书" class="headerlink" title="9.1 准备证书"></a>9.1 准备证书</h4><p>后续可以看到calico证书用在四个地方：</p><ul><li>calico/node 这个docker 容器运行时访问 etcd 使用证书</li><li>cni 配置文件中，cni 插件需要访问 etcd 使用证书</li><li>calicoctl 操作集群网络时访问 etcd 使用证书</li><li>calico/kube-controllers 同步集群网络策略时访问 etcd 使用证书<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#calico证书放在这</span></span><br><span class="line">$ mkdir -p /etc/kubernetes/ca/calico</span><br><span class="line"><span class="comment">#准备calico证书配置 - calico只需客户端证书，因此证书请求中 hosts 字段可以为空</span></span><br><span class="line">$ cp ~/kubernetes-starter/target/ca/calico/calico-csr.json /etc/kubernetes/ca/calico/</span><br><span class="line">$ <span class="built_in">cd</span> /etc/kubernetes/ca/calico/</span><br><span class="line"><span class="comment">#使用根证书(ca.pem)签发calico证书</span></span><br><span class="line">$ cfssl gencert \</span><br><span class="line">        -ca=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        -ca-key=/etc/kubernetes/ca/ca-key.pem \</span><br><span class="line">        -config=/etc/kubernetes/ca/ca-config.json \</span><br><span class="line">        -profile=kubernetes calico-csr.json | cfssljson -bare calico</span><br><span class="line"><span class="comment">#我们最终要的是calico-key.pem和calico.pem</span></span><br><span class="line">$ ls</span><br><span class="line">calico.csr  calico-csr.json  calico-key.pem  calico.pem</span><br></pre></td></tr></table></figure></li></ul><h4 id="9-2-改造calico服务"><a href="#9-2-改造calico服务" class="headerlink" title="9.2 改造calico服务"></a>9.2 改造calico服务</h4><p><strong>查看diff</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter</span><br><span class="line">$ vimdiff kubernetes-simple/all-node/kube-calico.service kubernetes-with-ca/all-node/kube-calico.service</span><br></pre></td></tr></table></figure></p><blockquote><p>通过diff会发现，calico多了几个认证相关的文件：<br>/etc/kubernetes/ca/ca.pem<br>/etc/kubernetes/ca/calico/calico.pem<br>/etc/kubernetes/ca/calico/calico-key.pem<br>由于calico服务是所有节点都需要启动的，大家需要把这几个文件拷贝到每台服务器上</p></blockquote><p><strong>更新calico服务</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ cp ~/kubernetes-starter/target/all-node/kube-calico.service /lib/systemd/system/</span><br><span class="line">$ systemctl daemon-reload</span><br><span class="line">$ service kube-calico start</span><br><span class="line"></span><br><span class="line"><span class="comment">#验证calico（能看到其他节点的列表就对啦）</span></span><br><span class="line">$ calicoctl node status</span><br></pre></td></tr></table></figure></p><h2 id="10-改造kubelet"><a href="#10-改造kubelet" class="headerlink" title="10. 改造kubelet"></a>10. 改造kubelet</h2><p>我们这里让kubelet使用引导token的方式认证，所以认证方式跟之前的组件不同，它的证书不是手动生成，而是由工作节点TLS BootStrap 向api-server请求，由主节点的controller-manager 自动签发。</p><h4 id="10-1-创建角色绑定（主节点）"><a href="#10-1-创建角色绑定（主节点）" class="headerlink" title="10.1 创建角色绑定（主节点）"></a>10.1 创建角色绑定（主节点）</h4><p>引导token的方式要求客户端向api-server发起请求时告诉他你的用户名和token，并且这个用户是具有一个特定的角色：system:node-bootstrapper，所以需要先将 bootstrap token 文件中的 kubelet-bootstrap 用户赋予这个特定角色，然后 kubelet 才有权限发起创建认证请求。<br><strong>在主节点执行下面命令</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#可以通过下面命令查询clusterrole列表</span></span><br><span class="line">$ kubectl -n kube-system get clusterrole</span><br><span class="line"></span><br><span class="line"><span class="comment">#可以回顾一下token文件的内容</span></span><br><span class="line">$ cat /etc/kubernetes/ca/kubernetes/token.csv</span><br><span class="line">8afdf3c4eb7c74018452423c29433609,kubelet-bootstrap,10001,<span class="string">"system:kubelet-bootstrap"</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#创建角色绑定（将用户kubelet-bootstrap与角色system:node-bootstrapper绑定）</span></span><br><span class="line">$ kubectl create clusterrolebinding kubelet-bootstrap \</span><br><span class="line">         --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap</span><br></pre></td></tr></table></figure></p><h4 id="10-2-创建bootstrap-kubeconfig（工作节点）"><a href="#10-2-创建bootstrap-kubeconfig（工作节点）" class="headerlink" title="10.2 创建bootstrap.kubeconfig（工作节点）"></a>10.2 创建bootstrap.kubeconfig（工作节点）</h4><p>这个配置是用来完成bootstrap token认证的，保存了像用户，token等重要的认证信息，这个文件可以借助kubectl命令生成：（也可以自己写配置）<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#设置集群参数(注意替换ip)</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-cluster kubernetes \</span><br><span class="line">        --certificate-authority=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        --embed-certs=<span class="literal">true</span> \</span><br><span class="line">        --server=https://192.168.1.102:6443 \</span><br><span class="line">        --kubeconfig=bootstrap.kubeconfig</span><br><span class="line"><span class="comment">#设置客户端认证参数(注意替换token)</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-credentials kubelet-bootstrap \</span><br><span class="line">        --token=8afdf3c4eb7c74018452423c29433609 \</span><br><span class="line">        --kubeconfig=bootstrap.kubeconfig</span><br><span class="line"><span class="comment">#设置上下文</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-context default \</span><br><span class="line">        --cluster=kubernetes \</span><br><span class="line">        --user=kubelet-bootstrap \</span><br><span class="line">        --kubeconfig=bootstrap.kubeconfig</span><br><span class="line"><span class="comment">#选择上下文</span></span><br><span class="line">$ kubectl config use-context default --kubeconfig=bootstrap.kubeconfig</span><br><span class="line"><span class="comment">#将刚生成的文件移动到合适的位置</span></span><br><span class="line">$ mv bootstrap.kubeconfig /etc/kubernetes/</span><br></pre></td></tr></table></figure></p><h4 id="10-3-准备cni配置"><a href="#10-3-准备cni配置" class="headerlink" title="10.3 准备cni配置"></a>10.3 准备cni配置</h4><p><strong>查看diff</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter</span><br><span class="line">$ vimdiff kubernetes-simple/worker-node/10-calico.conf kubernetes-with-ca/worker-node/10-calico.conf</span><br></pre></td></tr></table></figure></p><p><strong>copy配置</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ cp ~/kubernetes-starter/target/worker-node/10-calico.conf /etc/cni/net.d/</span><br></pre></td></tr></table></figure></p><h4 id="10-4-改造kubelet服务"><a href="#10-4-改造kubelet服务" class="headerlink" title="10.4 改造kubelet服务"></a>10.4 改造kubelet服务</h4><p><strong>查看diff</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter</span><br><span class="line">$ vimdiff kubernetes-simple/worker-node/kubelet.service kubernetes-with-ca/worker-node/kubelet.service</span><br></pre></td></tr></table></figure></p><p><strong>更新服务</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">$ cp ~/kubernetes-starter/target/worker-node/kubelet.service /lib/systemd/system/</span><br><span class="line">$ systemctl daemon-reload</span><br><span class="line">$ service kubelet start</span><br><span class="line"></span><br><span class="line"><span class="comment">#启动kubelet之后到master节点允许worker加入(批准worker的tls证书请求)</span></span><br><span class="line"><span class="comment">#--------*在主节点执行*---------</span></span><br><span class="line">$ kubectl get csr|grep <span class="string">'Pending'</span> | awk <span class="string">'&#123;print $1&#125;'</span>| xargs kubectl certificate approve</span><br><span class="line"><span class="comment">#-----------------------------</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#检查日志</span></span><br><span class="line">$ journalctl -f -u kubelet</span><br></pre></td></tr></table></figure></p><h2 id="11-改造kube-proxy"><a href="#11-改造kube-proxy" class="headerlink" title="11. 改造kube-proxy"></a>11. 改造kube-proxy</h2><h4 id="11-1-准备证书"><a href="#11-1-准备证书" class="headerlink" title="11.1 准备证书"></a>11.1 准备证书</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#proxy证书放在这</span></span><br><span class="line">$ mkdir -p /etc/kubernetes/ca/kube-proxy</span><br><span class="line"></span><br><span class="line"><span class="comment">#准备proxy证书配置 - proxy只需客户端证书，因此证书请求中 hosts 字段可以为空。</span></span><br><span class="line"><span class="comment">#CN 指定该证书的 User 为 system:kube-proxy，预定义的 ClusterRoleBinding system:node-proxy 将User system:kube-proxy 与 Role system:node-proxier 绑定，授予了调用 kube-api-server proxy的相关 API 的权限</span></span><br><span class="line">$ cp ~/kubernetes-starter/target/ca/kube-proxy/kube-proxy-csr.json /etc/kubernetes/ca/kube-proxy/</span><br><span class="line">$ <span class="built_in">cd</span> /etc/kubernetes/ca/kube-proxy/</span><br><span class="line"></span><br><span class="line"><span class="comment">#使用根证书(ca.pem)签发calico证书</span></span><br><span class="line">$ cfssl gencert \</span><br><span class="line">        -ca=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        -ca-key=/etc/kubernetes/ca/ca-key.pem \</span><br><span class="line">        -config=/etc/kubernetes/ca/ca-config.json \</span><br><span class="line">        -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy</span><br><span class="line"><span class="comment">#我们最终要的是kube-proxy-key.pem和kube-proxy.pem</span></span><br><span class="line">$ ls</span><br><span class="line">kube-proxy.csr  kube-proxy-csr.json  kube-proxy-key.pem  kube-proxy.pem</span><br></pre></td></tr></table></figure><h4 id="11-2-生成kube-proxy-kubeconfig配置"><a href="#11-2-生成kube-proxy-kubeconfig配置" class="headerlink" title="11.2 生成kube-proxy.kubeconfig配置"></a>11.2 生成kube-proxy.kubeconfig配置</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#设置集群参数（注意替换ip）</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-cluster kubernetes \</span><br><span class="line">        --certificate-authority=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        --embed-certs=<span class="literal">true</span> \</span><br><span class="line">        --server=https://192.168.1.102:6443 \</span><br><span class="line">        --kubeconfig=kube-proxy.kubeconfig</span><br><span class="line"><span class="comment">#置客户端认证参数</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-credentials kube-proxy \</span><br><span class="line">        --client-certificate=/etc/kubernetes/ca/kube-proxy/kube-proxy.pem \</span><br><span class="line">        --client-key=/etc/kubernetes/ca/kube-proxy/kube-proxy-key.pem \</span><br><span class="line">        --embed-certs=<span class="literal">true</span> \</span><br><span class="line">        --kubeconfig=kube-proxy.kubeconfig</span><br><span class="line"><span class="comment">#设置上下文参数</span></span><br><span class="line">$ kubectl config <span class="built_in">set</span>-context default \</span><br><span class="line">        --cluster=kubernetes \</span><br><span class="line">        --user=kube-proxy \</span><br><span class="line">        --kubeconfig=kube-proxy.kubeconfig</span><br><span class="line"><span class="comment">#选择上下文</span></span><br><span class="line">$ kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig</span><br><span class="line"><span class="comment">#移动到合适位置</span></span><br><span class="line">$ mv kube-proxy.kubeconfig /etc/kubernetes/kube-proxy.kubeconfig</span><br></pre></td></tr></table></figure><h4 id="11-3-改造kube-proxy服务"><a href="#11-3-改造kube-proxy服务" class="headerlink" title="11.3 改造kube-proxy服务"></a>11.3 改造kube-proxy服务</h4><p><strong>查看diff</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter</span><br><span class="line">$ vimdiff kubernetes-simple/worker-node/kube-proxy.service kubernetes-with-ca/worker-node/kube-proxy.service</span><br></pre></td></tr></table></figure></p><blockquote><p>经过diff你应该发现kube-proxy.service没有变化</p></blockquote><p><strong>启动服务</strong><br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#如果之前的配置没有了，可以重新复制一份过去</span></span><br><span class="line">$ cp ~/kubernetes-starter/target/worker-node/kube-proxy.service /lib/systemd/system/</span><br><span class="line">$ systemctl daemon-reload</span><br><span class="line"></span><br><span class="line"><span class="comment">#安装依赖软件</span></span><br><span class="line">$ apt install conntrack</span><br><span class="line"></span><br><span class="line"><span class="comment">#启动服务</span></span><br><span class="line">$ service kube-proxy start</span><br><span class="line"><span class="comment">#查看日志</span></span><br><span class="line">$ journalctl -f -u kube-proxy</span><br></pre></td></tr></table></figure></p><h2 id="12-改造kube-dns"><a href="#12-改造kube-dns" class="headerlink" title="12. 改造kube-dns"></a>12. 改造kube-dns</h2><p>kube-dns有些特别，因为它本身是运行在kubernetes集群中，以kubernetes应用的形式运行。所以它的认证授权方式跟之前的组件都不一样。它需要用到service account认证和RBAC授权。<br><strong>service account认证：</strong><br>每个service account都会自动生成自己的secret，用于包含一个ca，token和secret，用于跟api-server认证<br><strong>RBAC授权：</strong><br>权限、角色和角色绑定都是kubernetes自动创建好的。我们只需要创建一个叫做kube-dns的 ServiceAccount即可，官方现有的配置已经把它包含进去了。</p><h4 id="12-1-准备配置文件"><a href="#12-1-准备配置文件" class="headerlink" title="12.1 准备配置文件"></a>12.1 准备配置文件</h4><p>我们在官方的基础上添加的变量，生成适合我们集群的配置。直接copy就可以啦<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/kubernetes-starter</span><br><span class="line">$ vimdiff kubernetes-simple/services/kube-dns.yaml kubernetes-with-ca/services/kube-dns.yaml</span><br></pre></td></tr></table></figure></p><blockquote><p>大家可以看到diff只有一处，新的配置没有设定api-server。不访问api-server，它是怎么知道每个服务的cluster ip和pod的endpoints的呢？这就是因为kubernetes在启动每个服务service的时候会以环境变量的方式把所有服务的ip，端口等信息注入进来。</p></blockquote><h4 id="12-2-创建kube-dns"><a href="#12-2-创建kube-dns" class="headerlink" title="12.2 创建kube-dns"></a>12.2 创建kube-dns</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ kubectl create -f ~/kubernetes-starter/target/services/kube-dns.yaml</span><br><span class="line"><span class="comment">#看看启动是否成功</span></span><br><span class="line">$ kubectl -n kube-system get pods</span><br></pre></td></tr></table></figure><h2 id="13-再试牛刀"><a href="#13-再试牛刀" class="headerlink" title="13. 再试牛刀"></a>13. 再试牛刀</h2><p>终于，安全版的kubernetes集群我们部署完成了。<br>下面我们使用新集群先温习一下之前学习过的命令，然后再认识一些新的命令，新的参数，新的功能。同样，具体内容请看<a href="https://coding.imooc.com/class/198.html" target="_blank" rel="noopener">视频教程</a>吧~</p>]]></content>
    
    <summary type="html">
    
      部署带CA的Kubernetes集群
    
    </summary>
    
    
      <category term="k8s" scheme="http://yoursite.com/tags/k8s/"/>
    
  </entry>
  
  <entry>
    <title>cfssl-config-sample</title>
    <link href="http://yoursite.com/2019/01/01/cfssl-config-sample/"/>
    <id>http://yoursite.com/2019/01/01/cfssl-config-sample/</id>
    <published>2019-01-01T14:07:12.000Z</published>
    <updated>2019-01-01T15:08:53.236Z</updated>
    
    <content type="html"><![CDATA[<h2 id="kubernetes认证文件生成工具-CFSSL"><a href="#kubernetes认证文件生成工具-CFSSL" class="headerlink" title="kubernetes认证文件生成工具 CFSSL"></a>kubernetes认证文件生成工具 CFSSL</h2><p>在生成认证文件的时候，往往会用到， 网上很多有示例文件，一直不明白其中的CN，L，OU，什么的是什么意思。<br>在<code>https://k8smeetup.github.io/docs/concepts/cluster-administration/certificates/</code>上有如下介绍：<br><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">C</span> = &lt;country&gt;</span><br><span class="line"><span class="attr">ST</span> = &lt;state&gt;</span><br><span class="line"><span class="attr">L</span> = &lt;city&gt;</span><br><span class="line"><span class="attr">O</span> = &lt;organization&gt;</span><br><span class="line"><span class="attr">OU</span> = &lt;organization unit&gt;</span><br><span class="line"><span class="attr">CN</span> = &lt;MASTER_IP&gt;</span><br></pre></td></tr></table></figure></p><h2 id="生成根证书，用以下命令："><a href="#生成根证书，用以下命令：" class="headerlink" title="生成根证书，用以下命令："></a>生成根证书，用以下命令：</h2><blockquote><p>$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca</p><p>#生成完成后会有以下文件（我们最终想要的就是ca-key.pem和ca.pem，一个秘钥，一个证书）<br>$ ls<br>ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem</p></blockquote><p>上述命令中用到的ca-csr.json是：<br><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  <span class="attr">"CN"</span>: <span class="string">"kubernetes"</span>,</span><br><span class="line">  <span class="attr">"key"</span>: &#123;</span><br><span class="line">    <span class="attr">"algo"</span>: <span class="string">"rsa"</span>,</span><br><span class="line">    <span class="attr">"size"</span>: <span class="number">2048</span></span><br><span class="line">  &#125;,</span><br><span class="line">  <span class="attr">"names"</span>: [</span><br><span class="line">    &#123;</span><br><span class="line">      <span class="attr">"C"</span>: <span class="string">"CN"</span>,</span><br><span class="line">      <span class="attr">"ST"</span>: <span class="string">"Beijing"</span>,</span><br><span class="line">      <span class="attr">"L"</span>: <span class="string">"XS"</span>,</span><br><span class="line">      <span class="attr">"O"</span>: <span class="string">"k8s"</span>,</span><br><span class="line">      <span class="attr">"OU"</span>: <span class="string">"System"</span></span><br><span class="line">    &#125;</span><br><span class="line">  ]</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></p><h2 id="CSFFL-工具在生成认证文件的时候，用到以下选项-cfssl-gencert-："><a href="#CSFFL-工具在生成认证文件的时候，用到以下选项-cfssl-gencert-：" class="headerlink" title="CSFFL 工具在生成认证文件的时候，用到以下选项(cfssl gencert)："></a>CSFFL 工具在生成认证文件的时候，用到以下选项(cfssl gencert)：</h2><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line">Usage of gencer<span class="variable">t:</span></span><br><span class="line">    Generate <span class="keyword">a</span> <span class="keyword">new</span> key <span class="built_in">and</span> cert from CSR:</span><br><span class="line">        cfssl gencert -initca CSRJSON</span><br><span class="line">        cfssl gencert -<span class="keyword">ca</span> cert -<span class="keyword">ca</span>-key key [-config config] [-<span class="keyword">profile</span> <span class="keyword">profile</span>] [-<span class="built_in">hostname</span> <span class="built_in">hostname</span>] CSRJSON</span><br><span class="line">        cfssl gencert -remote remote_host [-config config] [-<span class="keyword">profile</span> <span class="keyword">profile</span>] [-label label] [-<span class="built_in">hostname</span> <span class="built_in">hostname</span>] CSRJSON</span><br><span class="line"></span><br><span class="line">    Re-generate <span class="keyword">a</span> CA cert with the CA key <span class="built_in">and</span> CSR:</span><br><span class="line">        cfssl gencert -initca -<span class="keyword">ca</span>-key key CSRJSON</span><br><span class="line"></span><br><span class="line">    Re-generate <span class="keyword">a</span> CA cert with the CA key <span class="built_in">and</span> certificate:</span><br><span class="line">        cfssl gencert -renewca -<span class="keyword">ca</span> cert -<span class="keyword">ca</span>-key key</span><br><span class="line"></span><br><span class="line">Argument<span class="variable">s:</span></span><br><span class="line">        CSRJSON:    JSON <span class="keyword">file</span> containing the request, use <span class="string">'-'</span> <span class="keyword">for</span> reading JSON from stdin</span><br><span class="line"></span><br><span class="line">Flag<span class="variable">s:</span></span><br><span class="line">  -initca=false: initialise <span class="keyword">new</span> CA</span><br><span class="line">  -remote=<span class="string">""</span>: remote CFSSL server</span><br><span class="line">  -<span class="keyword">ca</span>=<span class="string">""</span>: CA used <span class="keyword">to</span> <span class="keyword">sign</span> the <span class="keyword">new</span> certificate -- accepts <span class="string">'[file:]fname'</span> <span class="built_in">or</span> <span class="string">'env:varname'</span></span><br><span class="line">  -<span class="keyword">ca</span>-key=<span class="string">""</span>: CA private key -- accepts <span class="string">'[file:]fname'</span> <span class="built_in">or</span> <span class="string">'env:varname'</span></span><br><span class="line">  -config=<span class="string">""</span>: path <span class="keyword">to</span> configuration <span class="keyword">file</span></span><br><span class="line">  -<span class="keyword">cn</span>=<span class="string">""</span>: certificate common name (CN)</span><br><span class="line">  -<span class="built_in">hostname</span>=<span class="string">""</span>: Hostname <span class="keyword">for</span> the cert, could <span class="keyword">be</span> <span class="keyword">a</span> comma-separated <span class="built_in">hostname</span> <span class="keyword">list</span></span><br><span class="line">  -<span class="keyword">profile</span>=<span class="string">""</span>: signing <span class="keyword">profile</span> <span class="keyword">to</span> use</span><br><span class="line">  -label=<span class="string">""</span>: key label <span class="keyword">to</span> use in remote CFSSL server</span><br><span class="line">  -loglevel=<span class="number">1</span>: Log level (<span class="number">0</span> = DEBUG, <span class="number">5</span> = FATAL)</span><br></pre></td></tr></table></figure><h3 id="CFSSL-config-文件"><a href="#CFSSL-config-文件" class="headerlink" title="CFSSL config 文件"></a>CFSSL config 文件</h3><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  <span class="attr">"signing"</span>: &#123;</span><br><span class="line">    <span class="attr">"default"</span>: &#123;</span><br><span class="line">      <span class="attr">"expiry"</span>: <span class="string">"87600h"</span></span><br><span class="line">    &#125;,</span><br><span class="line">    <span class="attr">"profiles"</span>: &#123;</span><br><span class="line">      <span class="attr">"kubernetes"</span>: &#123;</span><br><span class="line">        <span class="attr">"usages"</span>: [</span><br><span class="line">            <span class="string">"signing"</span>,</span><br><span class="line">            <span class="string">"key encipherment"</span>,</span><br><span class="line">            <span class="string">"server auth"</span>,</span><br><span class="line">            <span class="string">"client auth"</span></span><br><span class="line">        ],</span><br><span class="line">        <span class="attr">"expiry"</span>: <span class="string">"87600h"</span></span><br><span class="line">      &#125;</span><br><span class="line">    &#125;</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="生成证书"><a href="#生成证书" class="headerlink" title="生成证书"></a>生成证书</h2><figure class="highlight vim"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ cfssl gencert \</span><br><span class="line">        -<span class="keyword">ca</span>=/path/<span class="keyword">to</span>/<span class="keyword">ca</span>.pem \</span><br><span class="line">        -<span class="keyword">ca</span>-key=/path/<span class="keyword">to</span>/<span class="keyword">ca</span>-key.pem \</span><br><span class="line">        -config=/path/<span class="keyword">to</span>/<span class="keyword">ca</span>-config.json \</span><br><span class="line">        -<span class="keyword">profile</span>=kubernetes admin-csr.json | cfssljson -bare admin</span><br></pre></td></tr></table></figure><p> 我们最终要的是admin-key.pem和admin.pem</p>]]></content>
    
    <summary type="html">
    
      CFSSL CA和config文件 JSON格式示例和字段含意说明
    
    </summary>
    
    
  </entry>
  
  <entry>
    <title>k8s-cfssl-deployment</title>
    <link href="http://yoursite.com/2018/12/31/k8s-cfssl-deployment/"/>
    <id>http://yoursite.com/2018/12/31/k8s-cfssl-deployment/</id>
    <published>2018-12-31T02:46:05.000Z</published>
    <updated>2019-01-01T15:06:40.213Z</updated>
    
    <content type="html"><![CDATA[<p>Source ：<a href="http://blog.simlinux.com/archives/1953.html" target="_blank" rel="noopener">http://blog.simlinux.com/archives/1953.html</a></p><p>更详细的介绍，请参照 ：<br><code>https://coreos.com/os/docs/latest/generate-self-signed-certificates.html</code></p><h2 id="容器相关证书类型"><a href="#容器相关证书类型" class="headerlink" title="容器相关证书类型"></a>容器相关证书类型</h2><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">client certificate： 用于服务端认证客户端,例如etcdctl、etcd proxy、fleetctl、docker客户端</span><br><span class="line">server certificate: 服务端使用，客户端以此验证服务端身份,例如docker服务端、kube-apiserver</span><br><span class="line">peer certificate: 双向证书，用于etcd集群成员间通信</span><br></pre></td></tr></table></figure><h2 id="创建CA证书"><a href="#创建CA证书" class="headerlink" title="创建CA证书"></a>创建CA证书</h2><h3 id="生成默认CA配置"><a href="#生成默认CA配置" class="headerlink" title="生成默认CA配置"></a>生成默认CA配置</h3><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">mkdir /opt/ssl</span><br><span class="line">cd /opt/ssl</span><br><span class="line">cfssl print-defaults<span class="built_in"> config </span>&gt; ca-config.json</span><br><span class="line">cfssl print-defaults csr &gt; ca-csr.json</span><br></pre></td></tr></table></figure><h3 id="修改ca-config-json-分别配置针对三种不同证书类型的profile-其中有效期43800h为5年"><a href="#修改ca-config-json-分别配置针对三种不同证书类型的profile-其中有效期43800h为5年" class="headerlink" title="修改ca-config.json,分别配置针对三种不同证书类型的profile,其中有效期43800h为5年"></a>修改ca-config.json,分别配置针对三种不同证书类型的profile,其中有效期43800h为5年</h3><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">    <span class="attr">"signing"</span>: &#123;</span><br><span class="line">        <span class="attr">"default"</span>: &#123;</span><br><span class="line">            <span class="attr">"expiry"</span>: <span class="string">"43800h"</span></span><br><span class="line">        &#125;,</span><br><span class="line">        <span class="attr">"profiles"</span>: &#123;</span><br><span class="line">            <span class="attr">"server"</span>: &#123;</span><br><span class="line">                <span class="attr">"expiry"</span>: <span class="string">"43800h"</span>,</span><br><span class="line">                <span class="attr">"usages"</span>: [</span><br><span class="line">                    <span class="string">"signing"</span>,</span><br><span class="line">                    <span class="string">"key encipherment"</span>,</span><br><span class="line">                    <span class="string">"server auth"</span></span><br><span class="line">                ]</span><br><span class="line">            &#125;,</span><br><span class="line">            <span class="attr">"client"</span>: &#123;</span><br><span class="line">                <span class="attr">"expiry"</span>: <span class="string">"43800h"</span>,</span><br><span class="line">                <span class="attr">"usages"</span>: [</span><br><span class="line">                    <span class="string">"signing"</span>,</span><br><span class="line">                    <span class="string">"key encipherment"</span>,</span><br><span class="line">                    <span class="string">"client auth"</span></span><br><span class="line">                ]</span><br><span class="line">            &#125;,</span><br><span class="line">            <span class="attr">"peer"</span>: &#123;</span><br><span class="line">                <span class="attr">"expiry"</span>: <span class="string">"43800h"</span>,</span><br><span class="line">                <span class="attr">"usages"</span>: [</span><br><span class="line">                    <span class="string">"signing"</span>,</span><br><span class="line">                    <span class="string">"key encipherment"</span>,</span><br><span class="line">                    <span class="string">"server auth"</span>,</span><br><span class="line">                    <span class="string">"client auth"</span></span><br><span class="line">                ]</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="修改ca-csr-config"><a href="#修改ca-csr-config" class="headerlink" title="修改ca-csr.config"></a>修改ca-csr.config</h3><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">    <span class="attr">"CN"</span>: <span class="string">"My own CA"</span>,</span><br><span class="line">    <span class="attr">"key"</span>: &#123;</span><br><span class="line">        <span class="attr">"algo"</span>: <span class="string">"rsa"</span>,</span><br><span class="line">        <span class="attr">"size"</span>: <span class="number">2048</span></span><br><span class="line">    &#125;,</span><br><span class="line">    <span class="attr">"names"</span>: [</span><br><span class="line">        &#123;</span><br><span class="line">            <span class="attr">"C"</span>: <span class="string">"US"</span>,</span><br><span class="line">            <span class="attr">"L"</span>: <span class="string">"CA"</span>,</span><br><span class="line">            <span class="attr">"O"</span>: <span class="string">"My Company Name"</span>,</span><br><span class="line">            <span class="attr">"ST"</span>: <span class="string">"San Francisco"</span>,</span><br><span class="line">            <span class="attr">"OU"</span>: <span class="string">"Org Unit 1"</span>,</span><br><span class="line">            <span class="attr">"OU"</span>: <span class="string">"Org Unit 2"</span></span><br><span class="line">        &#125;</span><br><span class="line">    ]</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h3 id="生成CA证书和私钥"><a href="#生成CA证书和私钥" class="headerlink" title="生成CA证书和私钥"></a>生成CA证书和私钥</h3><figure class="highlight stata"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cfssl gencert -initca <span class="keyword">ca</span>-csr.json | cfssljson -bare <span class="keyword">ca</span> -</span><br></pre></td></tr></table></figure><p>生成ca.pem、ca.csr、ca-key.pem(CA私钥,需妥善保管)</p><h2 id="签发Server-Certificate"><a href="#签发Server-Certificate" class="headerlink" title="签发Server Certificate"></a>签发Server Certificate</h2><p>cfssl print-defaults csr &gt; server-csr.json<br><figure class="highlight prolog"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">...</span><br><span class="line">    <span class="string">"CN"</span>: <span class="string">"coreos1"</span>,</span><br><span class="line">    <span class="string">"hosts"</span>: [</span><br><span class="line">        <span class="string">"192.168.122.68"</span>,</span><br><span class="line">        <span class="string">"ext.example.com"</span>,</span><br><span class="line">        <span class="string">"coreos1.local"</span>,</span><br><span class="line">        <span class="string">"coreos1"</span></span><br><span class="line">    ],</span><br><span class="line">...</span><br></pre></td></tr></table></figure></p><h3 id="生成服务端证书和私钥"><a href="#生成服务端证书和私钥" class="headerlink" title="生成服务端证书和私钥"></a>生成服务端证书和私钥</h3><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cfssl gencert <span class="attribute">-ca</span>=ca.pem <span class="attribute">-ca-key</span>=ca-key.pem <span class="attribute">-config</span>=ca-config.json <span class="attribute">-profile</span>=server server-csr.json | cfssljson -bare server</span><br></pre></td></tr></table></figure><h2 id="签发Client-Certificate"><a href="#签发Client-Certificate" class="headerlink" title="签发Client Certificate"></a>签发Client Certificate</h2><figure class="highlight nsis"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cfssl <span class="literal">print</span>-defaults csr &gt; <span class="literal">admin</span>-csr.json</span><br></pre></td></tr></table></figure><figure class="highlight prolog"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">...</span><br><span class="line">    <span class="string">"CN"</span>: <span class="string">"client"</span>,</span><br><span class="line">    <span class="string">"hosts"</span>: [<span class="string">""</span>],</span><br><span class="line">...</span><br></pre></td></tr></table></figure><h3 id="生成客户端证书和私钥"><a href="#生成客户端证书和私钥" class="headerlink" title="生成客户端证书和私钥"></a>生成客户端证书和私钥</h3><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cfssl gencert <span class="attribute">-ca</span>=ca.pem <span class="attribute">-ca-key</span>=ca-key.pem <span class="attribute">-config</span>=ca-config.json <span class="attribute">-profile</span>=client admin-csr.json | cfssljson -bare admin</span><br></pre></td></tr></table></figure><h2 id="签发peer-certificate"><a href="#签发peer-certificate" class="headerlink" title="签发peer certificate"></a>签发peer certificate</h2><figure class="highlight arduino"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cfssl <span class="built_in">print</span>-defaults csr &gt; kube-proxy-csr.json</span><br></pre></td></tr></table></figure><figure class="highlight prolog"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">...</span><br><span class="line">    <span class="string">"CN"</span>: <span class="string">"member1"</span>,</span><br><span class="line">    <span class="string">"hosts"</span>: [</span><br><span class="line">        <span class="string">"192.168.122.101"</span>,</span><br><span class="line">        <span class="string">"ext.example.com"</span>,</span><br><span class="line">        <span class="string">"member1.local"</span>,</span><br><span class="line">        <span class="string">"member1"</span></span><br><span class="line">    ],</span><br><span class="line">...</span><br></pre></td></tr></table></figure><h3 id="为节点member1生成证书和私钥"><a href="#为节点member1生成证书和私钥" class="headerlink" title="为节点member1生成证书和私钥:"></a>为节点member1生成证书和私钥:</h3><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cfssl gencert <span class="attribute">-ca</span>=ca.pem <span class="attribute">-ca-key</span>=ca-key.pem <span class="attribute">-config</span>=ca-config.json <span class="attribute">-profile</span>=peer kube-proxy-csr.json | cfssljson -bare kube-proxy</span><br></pre></td></tr></table></figure><p>针对etcd服务,每个etcd节点上按照上述方法生成相应的证书和私钥</p><h2 id="最后校验证书"><a href="#最后校验证书" class="headerlink" title="最后校验证书"></a>最后校验证书</h2><p>校验生成的证书是否和配置相符<br><figure class="highlight stylus"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">openssl x509 -<span class="keyword">in</span> ca<span class="selector-class">.pem</span> -text -noout</span><br><span class="line">openssl x509 -<span class="keyword">in</span> server<span class="selector-class">.pem</span> -text -noout</span><br><span class="line">openssl x509 -<span class="keyword">in</span> client<span class="selector-class">.pem</span> -text -noout</span><br></pre></td></tr></table></figure></p><h2 id="KUBERNETES服务配置"><a href="#KUBERNETES服务配置" class="headerlink" title="KUBERNETES服务配置"></a>KUBERNETES服务配置</h2><h3 id="生成随机token文件"><a href="#生成随机token文件" class="headerlink" title="生成随机token文件"></a>生成随机token文件</h3><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#生成随机token</span></span><br><span class="line">$ head -c <span class="number">16</span> /dev/urandom <span class="params">| od -An -t x |</span> tr -d <span class="string">' '</span></span><br><span class="line"><span class="number">8</span>afdf3c4eb7c74018452423c29433609</span><br><span class="line"></span><br><span class="line"><span class="comment">#按照固定格式写入token.csv，注意替换token内容</span></span><br><span class="line">$ echo <span class="string">"8afdf3c4eb7c74018452423c29433609,kubelet-bootstrap,10001,\"system:kubelet-bootstrap\""</span> &gt; <span class="regexp">/etc/kubernetes</span><span class="regexp">/ca/kubernetes</span><span class="regexp">/token.csv</span></span><br></pre></td></tr></table></figure><p>在apiserver服务启动时添加配置项<code>KUBE_TOKEN_AUTH_FILE=&quot;--token-auth-file=/etc/kubernetes/pki/apiserver/token.csv&quot;</code></p><h3 id="master-节点配置-kubectl-下面-admin-即是生成的kubectl证书"><a href="#master-节点配置-kubectl-下面-admin-即是生成的kubectl证书" class="headerlink" title="master 节点配置 kubectl (下面 admin 即是生成的kubectl证书 )"></a>master 节点配置 kubectl (下面 admin 即是生成的kubectl证书 )</h3><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#指定apiserver的地址和证书位置（ip自行修改）</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-cluster kubernetes \</span><br><span class="line">        <span class="attribute">--certificate-authority</span>=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        <span class="attribute">--embed-certs</span>=<span class="literal">true</span> \</span><br><span class="line">        <span class="attribute">--server</span>=https://192.168.1.102:6443</span><br><span class="line"><span class="comment">#设置客户端认证参数，指定admin证书和秘钥</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-credentials admin \</span><br><span class="line">        <span class="attribute">--client-certificate</span>=/etc/kubernetes/ca/admin/admin.pem \</span><br><span class="line">        <span class="attribute">--embed-certs</span>=<span class="literal">true</span> \</span><br><span class="line">        <span class="attribute">--client-key</span>=/etc/kubernetes/ca/admin/admin-key.pem</span><br><span class="line"><span class="comment">#关联用户和集群</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-context kubernetes \</span><br><span class="line">        <span class="attribute">--cluster</span>=kubernetes <span class="attribute">--user</span>=admin</span><br><span class="line"><span class="comment">#设置当前上下文</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>use-context kubernetes</span><br><span class="line"></span><br><span class="line"><span class="comment">#设置结果就是一个配置文件，可以看看内容</span></span><br><span class="line">$ cat ~/.kube/config</span><br></pre></td></tr></table></figure><p>我们这里让kubelet使用引导token的方式认证，所以认证方式跟之前的组件不同，它的证书不是手动生成，而是由工作节点TLS BootStrap 向api-server请求，由主节点的controller-manager 自动签发。</p><h4 id="主节点创建角色绑定"><a href="#主节点创建角色绑定" class="headerlink" title="主节点创建角色绑定"></a>主节点创建角色绑定</h4><p>引导token的方式要求客户端向api-server发起请求时告诉他你的用户名和token，并且这个用户是具有一个特定的角色：system:node-bootstrapper，<br>所以需要先将 bootstrap token 文件中的 kubelet-bootstrap 用户赋予这个特定角色，然后 kubelet 才有权限发起创建认证请求。 在主节点执行下面命令<br><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#可以通过下面命令查询clusterrole列表</span></span><br><span class="line">$ kubectl -n kube-system <span class="builtin-name">get</span> clusterrole</span><br><span class="line"></span><br><span class="line"><span class="comment">#可以回顾一下token文件的内容</span></span><br><span class="line">$ cat /etc/kubernetes/ca/kubernetes/token.csv</span><br><span class="line">8afdf3c4eb7c74018452423c29433609,kubelet-bootstrap,10001,<span class="string">"system:kubelet-bootstrap"</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#创建角色绑定（将用户kubelet-bootstrap与角色system:node-bootstrapper绑定）</span></span><br><span class="line">$ kubectl create clusterrolebinding kubelet-bootstrap \</span><br><span class="line">         <span class="attribute">--clusterrole</span>=system:node-bootstrapper <span class="attribute">--user</span>=kubelet-bootstrap</span><br></pre></td></tr></table></figure></p><h4 id="工作节点创建bootstrap-kubeconfig"><a href="#工作节点创建bootstrap-kubeconfig" class="headerlink" title="工作节点创建bootstrap.kubeconfig"></a>工作节点创建bootstrap.kubeconfig</h4><p>这个配置是用来完成bootstrap token认证的，保存了像用户，token等重要的认证信息，这个文件可以借助kubectl命令生成：（也可以自己写配置）<br><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#设置集群参数(注意替换ip)</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-cluster kubernetes \</span><br><span class="line">        <span class="attribute">--certificate-authority</span>=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        <span class="attribute">--embed-certs</span>=<span class="literal">true</span> \</span><br><span class="line">        <span class="attribute">--server</span>=https://192.168.1.102:6443 \</span><br><span class="line">        <span class="attribute">--kubeconfig</span>=bootstrap.kubeconfig</span><br><span class="line"><span class="comment">#设置客户端认证参数(注意替换token)</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-credentials kubelet-bootstrap \</span><br><span class="line">        <span class="attribute">--token</span>=8afdf3c4eb7c74018452423c29433609 \</span><br><span class="line">        <span class="attribute">--kubeconfig</span>=bootstrap.kubeconfig</span><br><span class="line"><span class="comment">#设置上下文</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-context<span class="built_in"> default </span>\</span><br><span class="line">        <span class="attribute">--cluster</span>=kubernetes \</span><br><span class="line">        <span class="attribute">--user</span>=kubelet-bootstrap \</span><br><span class="line">        <span class="attribute">--kubeconfig</span>=bootstrap.kubeconfig</span><br><span class="line"><span class="comment">#选择上下文</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>use-context<span class="built_in"> default </span><span class="attribute">--kubeconfig</span>=bootstrap.kubeconfig</span><br></pre></td></tr></table></figure></p><p>kubelet.service服务添加下面的参数：<br><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">KUBELET_POD_INFRA_CONTAINER_IMAGE</span>=<span class="string">"--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/imooc/pause-amd64:3.0"</span></span><br><span class="line"><span class="attr">KUBELET_EXPERIMENTAL_BOOTSTRAP_KUBECONFIG</span>=<span class="string">"--experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig"</span></span><br><span class="line"><span class="attr">KUBELET_CERT_DIR</span>=<span class="string">"--cert-dir=/etc/kubernetes/pki/apiserver"</span></span><br><span class="line"><span class="attr">KUBELET_HAIRPIN_MODE</span>=<span class="string">"--hairpin-mode hairpin-veth"</span></span><br></pre></td></tr></table></figure></p><h3 id="kube-proxy-服务"><a href="#kube-proxy-服务" class="headerlink" title="kube-proxy 服务"></a>kube-proxy 服务</h3><pre><code>kube-proxy服务也需要生成证书。另外，需要生成kube-proxy.kubeconfig配置:</code></pre><figure class="highlight routeros"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#设置集群参数（注意替换ip）</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-cluster kubernetes \</span><br><span class="line">        <span class="attribute">--certificate-authority</span>=/etc/kubernetes/ca/ca.pem \</span><br><span class="line">        <span class="attribute">--embed-certs</span>=<span class="literal">true</span> \</span><br><span class="line">        <span class="attribute">--server</span>=https://192.168.1.102:6443 \</span><br><span class="line">        <span class="attribute">--kubeconfig</span>=kube-proxy.kubeconfig</span><br><span class="line"><span class="comment">#置客户端认证参数</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-credentials kube-proxy \</span><br><span class="line">        <span class="attribute">--client-certificate</span>=/etc/kubernetes/ca/kube-proxy/kube-proxy.pem \</span><br><span class="line">        <span class="attribute">--client-key</span>=/etc/kubernetes/ca/kube-proxy/kube-proxy-key.pem \</span><br><span class="line">        <span class="attribute">--embed-certs</span>=<span class="literal">true</span> \</span><br><span class="line">        <span class="attribute">--kubeconfig</span>=kube-proxy.kubeconfig</span><br><span class="line"><span class="comment">#设置上下文参数</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>set-context<span class="built_in"> default </span>\</span><br><span class="line">        <span class="attribute">--cluster</span>=kubernetes \</span><br><span class="line">        <span class="attribute">--user</span>=kube-proxy \</span><br><span class="line">        <span class="attribute">--kubeconfig</span>=kube-proxy.kubeconfig</span><br><span class="line"><span class="comment">#选择上下文</span></span><br><span class="line">$ kubectl<span class="built_in"> config </span>use-context<span class="built_in"> default </span><span class="attribute">--kubeconfig</span>=kube-proxy.kubeconfig</span><br></pre></td></tr></table></figure>]]></content>
    
    <summary type="html">
    
      K8S 认证工具CFSSL的使用及kube服务的配置
    
    </summary>
    
    
      <category term="CA" scheme="http://yoursite.com/tags/CA/"/>
    
      <category term="k8s" scheme="http://yoursite.com/tags/k8s/"/>
    
  </entry>
  
  <entry>
    <title>Concepts about Certification</title>
    <link href="http://yoursite.com/2018/12/30/Concepts-about-Certification/"/>
    <id>http://yoursite.com/2018/12/30/Concepts-about-Certification/</id>
    <published>2018-12-30T03:09:45.000Z</published>
    <updated>2019-01-05T02:37:02.921Z</updated>
    
    <content type="html"><![CDATA[<p>Source: <a href="https://www.cnblogs.com/guogangj/p/4118605.html" target="_blank" rel="noopener">https://www.cnblogs.com/guogangj/p/4118605.html</a></p><p>之前没接触过证书加密的话,对证书相关的这些概念真是感觉挺棘手的,因为一下子来了一大堆新名词,看起来像是另一个领域的东西,而不是我们所熟悉的编程领域的那些东西,起码我个人感觉如此,且很长时间都没怎么搞懂.写这篇文章的目的就是为了理理清这些概念,搞清楚它们的含义及关联,还有一些基本操作.</p><p>SSL<br>SSL - Secure Sockets Layer,现在应该叫”TLS”,但由于习惯问题,我们还是叫”SSL”比较多.http协议默认情况下是不加密内容的,这样就很可能在内容传播的时候被别人监听到,对于安全性要求较高的场合,必须要加密,https就是带加密的http协议,而https的加密是基于SSL的,它执行的是一个比较下层的加密,也就是说,在加密前,你的服务器程序在干嘛,加密后也一样在干嘛,不用动,这个加密对用户和开发者来说都是透明的.More:[维基百科]</p><p>OpenSSL - 简单地说,OpenSSL是SSL的一个实现,SSL只是一种规范.理论上来说,SSL这种规范是安全的,目前的技术水平很难破解,但SSL的实现就可能有些漏洞,如著名的”心脏出血”.OpenSSL还提供了一大堆强大的工具软件,强大到90%我们都用不到.</p><p>证书标准<br>X.509 - 这是一种证书标准,主要定义了证书中应该包含哪些内容.其详情可以参考RFC5280,SSL使用的就是这种证书标准.</p><p>编码格式<br>同样的X.509证书,可能有不同的编码格式,目前有以下两种编码格式.</p><p>PEM - Privacy Enhanced Mail,打开看文本格式,以”—–BEGIN…”开头, “—–END…”结尾,内容是BASE64编码.<br>查看PEM格式证书的信息:openssl x509 -in certificate.pem -text -noout<br>Apache和*NIX服务器偏向于使用这种编码格式.</p><p>DER - Distinguished Encoding Rules,打开看是二进制格式,不可读.<br>查看DER格式证书的信息:openssl x509 -in certificate.der -inform der -text -noout<br>Java和Windows服务器偏向于使用这种编码格式.</p><p>相关的文件扩展名<br>这是比较误导人的地方,虽然我们已经知道有PEM和DER这两种编码格式,但文件扩展名并不一定就叫”PEM”或者”DER”,常见的扩展名除了PEM和DER还有以下这些,它们除了编码格式可能不同之外,内容也有差别,但大多数都能相互转换编码格式.</p><p>CRT - CRT应该是certificate的三个字母,其实还是证书的意思,常见于*NIX系统,有可能是PEM编码,也有可能是DER编码,大多数应该是PEM编码,相信你已经知道怎么辨别.</p><p>CER - 还是certificate,还是证书,常见于Windows系统,同样的,可能是PEM编码,也可能是DER编码,大多数应该是DER编码.</p><p>KEY - 通常用来存放一个公钥或者私钥,并非X.509证书,编码同样的,可能是PEM,也可能是DER.<br>查看KEY的办法:openssl rsa -in mykey.key -text -noout<br>如果是DER格式的话,同理应该这样了:openssl rsa -in mykey.key -text -noout -inform der</p><p>CSR - Certificate Signing Request,即证书签名请求,这个并不是证书,而是向权威证书颁发机构获得签名证书的申请,其核心内容是一个公钥(当然还附带了一些别的信息),在生成这个申请的时候,同时也会生成一个私钥,私钥要自己保管好.做过iOS APP的朋友都应该知道是怎么向苹果申请开发者证书的吧.<br>查看的办法:openssl req -noout -text -in my.csr (如果是DER格式的话照旧加上-inform der,这里不写了)</p><p>PFX/P12 - predecessor of PKCS#12,对*nix服务器来说,一般CRT和KEY是分开存放在不同文件中的,但Windows的IIS则将它们存在一个PFX文件中,(因此这个文件包含了证书及私钥)这样会不会不安全？应该不会,PFX通常会有一个”提取密码”,你想把里面的东西读取出来的话,它就要求你提供提取密码,PFX使用的时DER编码,如何把PFX转换为PEM编码？<br>openssl pkcs12 -in for-iis.pfx -out for-iis.pem -nodes<br>这个时候会提示你输入提取代码. for-iis.pem就是可读的文本.<br>生成pfx的命令类似这样:openssl pkcs12 -export -in certificate.crt -inkey privateKey.key -out certificate.pfx -certfile CACert.crt</p><p>其中CACert.crt是CA(权威证书颁发机构)的根证书,有的话也通过-certfile参数一起带进去.这么看来,PFX其实是个证书密钥库.</p><p>JKS - 即Java Key Storage,这是Java的专利,跟OpenSSL关系不大,利用Java的一个叫”keytool”的工具,可以将PFX转为JKS,当然了,keytool也能直接生成JKS,不过在此就不多表了.</p><p>证书编码的转换<br>PEM转为DER openssl x509 -in cert.crt -outform der -out cert.der</p><p>DER转为PEM openssl x509 -in cert.crt -inform der -outform pem -out cert.pem</p><p>(提示:要转换KEY文件也类似,只不过把x509换成rsa,要转CSR的话,把x509换成req…)</p><p>获得证书<br>向权威证书颁发机构申请证书</p><p>用这命令生成一个csr: openssl req -newkey rsa:2048 -new -nodes -keyout my.key -out my.csr<br>把csr交给权威证书颁发机构,权威证书颁发机构对此进行签名,完成.保留好csr,当权威证书颁发机构颁发的证书过期的时候,你还可以用同样的csr来申请新的证书,key保持不变.</p><p>或者生成自签名的证书<br>openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -keyout key.pem -out cert.pem<br>在生成证书的过程中会要你填一堆的东西,其实真正要填的只有Common Name,通常填写你服务器的域名,如”yourcompany.com”,或者你服务器的IP地址,其它都可以留空的.<br>生产环境中还是不要使用自签的证书,否则浏览器会不认,或者如果你是企业应用的话能够强制让用户的浏览器接受你的自签证书也行.向权威机构要证书通常是要钱的,但现在也有免费的,仅仅需要一个简单的域名验证即可.有兴趣的话查查”沃通数字证书”.</p>]]></content>
    
    <summary type="html">
    
      认证的概念，常用证书的标准
    
    </summary>
    
    
      <category term="CA" scheme="http://yoursite.com/tags/CA/"/>
    
      <category term="SSL" scheme="http://yoursite.com/tags/SSL/"/>
    
  </entry>
  
</feed>
