Skip to content

Commit

Permalink
Deployed 5302433 to master with MkDocs 1.5.3 and mike 2.0.0
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions[bot] committed Apr 9, 2024
1 parent 10692ba commit 9e8b742
Show file tree
Hide file tree
Showing 3 changed files with 171 additions and 171 deletions.
2 changes: 1 addition & 1 deletion master/modelserving/v1beta1/custom/custom_model/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -1753,7 +1753,7 @@ <h2 id="parallel-model-inference">Parallel Model Inference<a class="headerlink"
<span class="n">ray</span><span class="o">.</span><span class="n">init</span><span class="p">(</span><span class="n">num_cpus</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">num_gpus</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">kserve</span><span class="o">.</span><span class="n">ModelServer</span><span class="p">()</span><span class="o">.</span><span class="n">start</span><span class="p">({</span><span class="s2">"custom-model"</span><span class="p">:</span> <span class="n">AlexNetModel</span><span class="p">})</span>
</code></pre></div>
The more details for ray fractional cpu and gpu can be found <a href="https://docs.ray.io/en/latest/serve/scaling-and-resource-allocation.html#fractional-cpus-and-fractional-gpus">here</a>.</p>
The more details for ray fractional cpu and gpu can be found <a href="https://docs.ray.io/en/latest/serve/resource-allocation.html#fractional-cpus-and-fractional-gpus">here</a>.</p>
<p>The full code example can be found <a href="https://github.com/kserve/kserve/blob/release-0.11/python/custom_model/model_remote.py">here</a>.</p>
<p>Modify the <code>Procfile</code> to <code>web: python -m model_remote</code> and then run the above <code>pack</code> command, it builds the serving image which launches
each model as separate python worker and web server routes to the model workers by name.</p>
Expand Down
Loading

0 comments on commit 9e8b742

Please sign in to comment.