Skip to content

Commit

Permalink
Deployed 1486395 with MkDocs version: 1.6.1
Browse files Browse the repository at this point in the history
  • Loading branch information
@github-actions
github-actions[bot] committed Jan 17, 2025
1 parent 2de8f3f commit 72a0ed3
Show file tree
Hide file tree
Showing 2 changed files with 33 additions and 33 deletions.
64 changes: 32 additions & 32 deletions api/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -992,15 +992,7 @@ <h3 id="structured_qa.model_loaders.load_llama_cpp_model" class="doc doc-heading

<details class="quote">
<summary>Source code in <code>src/structured_qa/model_loaders.py</code></summary>
<div class="language-python highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"><a href="#__codelineno-0-5"> 5</a></span>
<span class="normal"><a href="#__codelineno-0-6"> 6</a></span>
<span class="normal"><a href="#__codelineno-0-7"> 7</a></span>
<span class="normal"><a href="#__codelineno-0-8"> 8</a></span>
<span class="normal"><a href="#__codelineno-0-9"> 9</a></span>
<span class="normal"><a href="#__codelineno-0-10">10</a></span>
<span class="normal"><a href="#__codelineno-0-11">11</a></span>
<span class="normal"><a href="#__codelineno-0-12">12</a></span>
<span class="normal"><a href="#__codelineno-0-13">13</a></span>
<div class="language-python highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span></span><span class="normal"><a href="#__codelineno-0-13">13</a></span>
<span class="normal"><a href="#__codelineno-0-14">14</a></span>
<span class="normal"><a href="#__codelineno-0-15">15</a></span>
<span class="normal"><a href="#__codelineno-0-16">16</a></span>
Expand All @@ -1014,29 +1006,37 @@ <h3 id="structured_qa.model_loaders.load_llama_cpp_model" class="doc doc-heading
<span class="normal"><a href="#__codelineno-0-24">24</a></span>
<span class="normal"><a href="#__codelineno-0-25">25</a></span>
<span class="normal"><a href="#__codelineno-0-26">26</a></span>
<span class="normal"><a href="#__codelineno-0-27">27</a></span></pre></div></td><td class="code"><div><pre><span></span><code><span id="__span-0-5"><a id="__codelineno-0-5" name="__codelineno-0-5"></a><span class="k">def</span><span class="w"> </span><span class="nf">load_llama_cpp_model</span><span class="p">(</span><span class="n">model_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Llama</span><span class="p">:</span>
</span><span id="__span-0-6"><a id="__codelineno-0-6" name="__codelineno-0-6"></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="__span-0-7"><a id="__codelineno-0-7" name="__codelineno-0-7"></a><span class="sd"> Loads the given model_id using Llama.from_pretrained.</span>
</span><span id="__span-0-8"><a id="__codelineno-0-8" name="__codelineno-0-8"></a>
</span><span id="__span-0-9"><a id="__codelineno-0-9" name="__codelineno-0-9"></a><span class="sd"> Examples:</span>
</span><span id="__span-0-10"><a id="__codelineno-0-10" name="__codelineno-0-10"></a><span class="sd"> &gt;&gt;&gt; model = load_llama_cpp_model(&quot;allenai/OLMoE-1B-7B-0924-Instruct-GGUF/olmoe-1b-7b-0924-instruct-q8_0.gguf&quot;)</span>
</span><span id="__span-0-11"><a id="__codelineno-0-11" name="__codelineno-0-11"></a>
</span><span id="__span-0-12"><a id="__codelineno-0-12" name="__codelineno-0-12"></a><span class="sd"> Args:</span>
</span><span id="__span-0-13"><a id="__codelineno-0-13" name="__codelineno-0-13"></a><span class="sd"> model_id (str): The model id to load.</span>
</span><span id="__span-0-14"><a id="__codelineno-0-14" name="__codelineno-0-14"></a><span class="sd"> Format is expected to be `{org}/{repo}/{filename}`.</span>
</span><span id="__span-0-15"><a id="__codelineno-0-15" name="__codelineno-0-15"></a>
</span><span id="__span-0-16"><a id="__codelineno-0-16" name="__codelineno-0-16"></a><span class="sd"> Returns:</span>
</span><span id="__span-0-17"><a id="__codelineno-0-17" name="__codelineno-0-17"></a><span class="sd"> Llama: The loaded model.</span>
</span><span id="__span-0-18"><a id="__codelineno-0-18" name="__codelineno-0-18"></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="__span-0-19"><a id="__codelineno-0-19" name="__codelineno-0-19"></a> <span class="n">org</span><span class="p">,</span> <span class="n">repo</span><span class="p">,</span> <span class="n">filename</span> <span class="o">=</span> <span class="n">model_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">&quot;/&quot;</span><span class="p">)</span>
</span><span id="__span-0-20"><a id="__codelineno-0-20" name="__codelineno-0-20"></a> <span class="n">model</span> <span class="o">=</span> <span class="n">Llama</span><span class="o">.</span><span class="n">from_pretrained</span><span class="p">(</span>
</span><span id="__span-0-21"><a id="__codelineno-0-21" name="__codelineno-0-21"></a> <span class="n">repo_id</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">org</span><span class="si">}</span><span class="s2">/</span><span class="si">{</span><span class="n">repo</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">,</span>
</span><span id="__span-0-22"><a id="__codelineno-0-22" name="__codelineno-0-22"></a> <span class="n">filename</span><span class="o">=</span><span class="n">filename</span><span class="p">,</span>
</span><span id="__span-0-23"><a id="__codelineno-0-23" name="__codelineno-0-23"></a> <span class="n">n_ctx</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="c1"># 0 means that the model limit will be used, instead of the default (512) or other hardcoded value</span>
</span><span id="__span-0-24"><a id="__codelineno-0-24" name="__codelineno-0-24"></a> <span class="n">verbose</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="__span-0-25"><a id="__codelineno-0-25" name="__codelineno-0-25"></a> <span class="n">n_gpu_layers</span><span class="o">=-</span><span class="mi">1</span> <span class="k">if</span> <span class="n">torch</span><span class="o">.</span><span class="n">cuda</span><span class="o">.</span><span class="n">is_available</span><span class="p">()</span> <span class="k">else</span> <span class="mi">0</span><span class="p">,</span>
</span><span id="__span-0-26"><a id="__codelineno-0-26" name="__codelineno-0-26"></a> <span class="p">)</span>
</span><span id="__span-0-27"><a id="__codelineno-0-27" name="__codelineno-0-27"></a> <span class="k">return</span> <span class="n">model</span>
<span class="normal"><a href="#__codelineno-0-27">27</a></span>
<span class="normal"><a href="#__codelineno-0-28">28</a></span>
<span class="normal"><a href="#__codelineno-0-29">29</a></span>
<span class="normal"><a href="#__codelineno-0-30">30</a></span>
<span class="normal"><a href="#__codelineno-0-31">31</a></span>
<span class="normal"><a href="#__codelineno-0-32">32</a></span>
<span class="normal"><a href="#__codelineno-0-33">33</a></span>
<span class="normal"><a href="#__codelineno-0-34">34</a></span>
<span class="normal"><a href="#__codelineno-0-35">35</a></span></pre></div></td><td class="code"><div><pre><span></span><code><span id="__span-0-13"><a id="__codelineno-0-13" name="__codelineno-0-13"></a><span class="k">def</span><span class="w"> </span><span class="nf">load_llama_cpp_model</span><span class="p">(</span><span class="n">model_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Llama</span><span class="p">:</span>
</span><span id="__span-0-14"><a id="__codelineno-0-14" name="__codelineno-0-14"></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;</span>
</span><span id="__span-0-15"><a id="__codelineno-0-15" name="__codelineno-0-15"></a><span class="sd"> Loads the given model_id using Llama.from_pretrained.</span>
</span><span id="__span-0-16"><a id="__codelineno-0-16" name="__codelineno-0-16"></a>
</span><span id="__span-0-17"><a id="__codelineno-0-17" name="__codelineno-0-17"></a><span class="sd"> Examples:</span>
</span><span id="__span-0-18"><a id="__codelineno-0-18" name="__codelineno-0-18"></a><span class="sd"> &gt;&gt;&gt; model = load_llama_cpp_model(&quot;allenai/OLMoE-1B-7B-0924-Instruct-GGUF/olmoe-1b-7b-0924-instruct-q8_0.gguf&quot;)</span>
</span><span id="__span-0-19"><a id="__codelineno-0-19" name="__codelineno-0-19"></a>
</span><span id="__span-0-20"><a id="__codelineno-0-20" name="__codelineno-0-20"></a><span class="sd"> Args:</span>
</span><span id="__span-0-21"><a id="__codelineno-0-21" name="__codelineno-0-21"></a><span class="sd"> model_id (str): The model id to load.</span>
</span><span id="__span-0-22"><a id="__codelineno-0-22" name="__codelineno-0-22"></a><span class="sd"> Format is expected to be `{org}/{repo}/{filename}`.</span>
</span><span id="__span-0-23"><a id="__codelineno-0-23" name="__codelineno-0-23"></a>
</span><span id="__span-0-24"><a id="__codelineno-0-24" name="__codelineno-0-24"></a><span class="sd"> Returns:</span>
</span><span id="__span-0-25"><a id="__codelineno-0-25" name="__codelineno-0-25"></a><span class="sd"> Llama: The loaded model.</span>
</span><span id="__span-0-26"><a id="__codelineno-0-26" name="__codelineno-0-26"></a><span class="sd"> &quot;&quot;&quot;</span>
</span><span id="__span-0-27"><a id="__codelineno-0-27" name="__codelineno-0-27"></a> <span class="n">org</span><span class="p">,</span> <span class="n">repo</span><span class="p">,</span> <span class="n">filename</span> <span class="o">=</span> <span class="n">model_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">&quot;/&quot;</span><span class="p">)</span>
</span><span id="__span-0-28"><a id="__codelineno-0-28" name="__codelineno-0-28"></a> <span class="n">model</span> <span class="o">=</span> <span class="n">Llama</span><span class="o">.</span><span class="n">from_pretrained</span><span class="p">(</span>
</span><span id="__span-0-29"><a id="__codelineno-0-29" name="__codelineno-0-29"></a> <span class="n">repo_id</span><span class="o">=</span><span class="sa">f</span><span class="s2">&quot;</span><span class="si">{</span><span class="n">org</span><span class="si">}</span><span class="s2">/</span><span class="si">{</span><span class="n">repo</span><span class="si">}</span><span class="s2">&quot;</span><span class="p">,</span>
</span><span id="__span-0-30"><a id="__codelineno-0-30" name="__codelineno-0-30"></a> <span class="n">filename</span><span class="o">=</span><span class="n">filename</span><span class="p">,</span>
</span><span id="__span-0-31"><a id="__codelineno-0-31" name="__codelineno-0-31"></a> <span class="n">n_ctx</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="c1"># 0 means that the model limit will be used, instead of the default (512) or other hardcoded value</span>
</span><span id="__span-0-32"><a id="__codelineno-0-32" name="__codelineno-0-32"></a> <span class="n">verbose</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
</span><span id="__span-0-33"><a id="__codelineno-0-33" name="__codelineno-0-33"></a> <span class="n">n_gpu_layers</span><span class="o">=-</span><span class="mi">1</span> <span class="k">if</span> <span class="n">gpu_available</span><span class="p">()</span> <span class="k">else</span> <span class="mi">0</span><span class="p">,</span>
</span><span id="__span-0-34"><a id="__codelineno-0-34" name="__codelineno-0-34"></a> <span class="p">)</span>
</span><span id="__span-0-35"><a id="__codelineno-0-35" name="__codelineno-0-35"></a> <span class="k">return</span> <span class="n">model</span>
</span></code></pre></div></td></tr></table></div>
</details>
</div>
Expand Down
Loading

0 comments on commit 72a0ed3

Please sign in to comment.