Skip to content

PyTorch example #579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

PyTorch example #579

wants to merge 4 commits into from

Conversation

msaroufim
Copy link

@msaroufim msaroufim commented Apr 25, 2025

Description

This PR provides an end to end example of integrating PyTorch with cuda-python. In particular this PR

  1. Ensures cuda-python reuses the same CUDA stream as PyTorch similarly to what Triton does. This logic is likely useful enough to encapsulate into its own helper on the core Stream object in cuda-python
  2. Shows how to pass in pointers to torch tensors to a kernel

Limitation

  1. scalars needs to be encapsulated in a torch.Tensor otherwise I was running into correctness issues

Example of running this locally

(cuda-p) [marksaroufim@devvm3083]~/cuda-python/cuda_core/examples% python pytorch_example.py 
PyTorch stream: <torch.cuda.Stream device=cuda:0 cuda_stream=0x0>
Using PyTorch stream ID: 0
Successfully created CUDA Python stream from PyTorch stream
Single precision test passed!
Double precision test passed!
All tests passed successfully!

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Copy link
Contributor

copy-pr-bot bot commented Apr 25, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@leofang leofang added P1 Medium priority - Should do cuda.core Everything related to the cuda.core module example Improvements or additions to code examples labels Apr 26, 2025
@leofang leofang self-requested a review April 26, 2025 03:48
@leofang leofang assigned msaroufim and unassigned msaroufim Apr 26, 2025
@leofang leofang added this to the cuda.core beta 4 milestone Apr 26, 2025
@leofang
Copy link
Member

leofang commented Apr 26, 2025

Many thanks to your contributions, @msaroufim! 🙏 Please give us a bit of time to unblock you -- This repository, including cuda.core, originally was not allowed to accept external contributions. However, we've received legal approval to make changes for cuda.core, and #583 is the last step to open up the gate. Once we finalize the license change we can review/merge your PRs!

@leofang leofang added the blocked This task is currently blocked by other tasks label Apr 26, 2025
@msaroufim
Copy link
Author

No worries @leofang I understand, great repo! So looking forward to contributing more soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked This task is currently blocked by other tasks cuda.core Everything related to the cuda.core module example Improvements or additions to code examples P1 Medium priority - Should do
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants