Replies: 1 comment
-
|
Fellow CUDA-kindergartener here... First off, anyone who stumbles across this that hasn't seen what But it sounds like you're afraid of assembler? Perhaps I can motivate you to dig into it! ... It's just moving bits and bytes around with different sized bulldozers 😅 If you need inspiration for any of these... just look at what you can do in 4k: https://www.pouet.net/prodlist.php?type%5B0%5D=4k&order=thumbup ... don't get me started on the 128b category! If those don't impress you, maybe start with the 65535 byte category? 😄 Or try to generate just one of the effects yourself? (Flame effect?) When you watch these, keep in mind that all graphics and sound is algorithmically created, so the tracker, 3d engine, and content all fits in 4096 bytes... or 256!!! Hope you make it back to CUDA after that! Here are some of the adjacent foundations, which I think you should be familiar with, even play around with: (You don't have to start here, but it'll be fun, I promise! https://eater.net/8bit/ - or my personal favorite https://github.com/slu4coder/Minimal-UART-CPU-System - And of course you can emulate these, and pretty much anything, nowadays. There are incredible communities around these projects, and even fantasy consoles - which are consoles conceived of, and programmed, long before anyone even gets to making hardware!) Come on, considering everything that's been done for you to get to this point... don't you think the least you could do is understand some of the foundations... GPU architecture is probably the best thing to understand (https://en.wikipedia.org/wiki/Graphics_processing_unit), but before you can understand that, you need to understand CPU architecture (https://en.wikipedia.org/wiki/Processor_design). You have LLM's (GPT4, Claude3) that can answer your dumbest questions with a bit of intelligence, and help you level up faster than ever!) Please let us know what gave you the biggest breakthroughs! For me, the biggest leap in understanding and motivation so far, has been working my way through the code in this very project, https://github.com/karpathy/llm.c. Also... @geohot created a community around https://github.com/tinygrad/tinygrad that has worked its way all the way to writing custom firmware and assembler for AMD cards here: https://github.com/geohot/7900xtx/ https://www.youtube.com/@geohotarchive) Maybe it's just a matter of time before we get those architectures into LLVM... and on to https://godbolt.org/ ... anyone interested in low level code that hasn't come across that, really should take a look! Hope at least one of these links help! |
Beta Was this translation helpful? Give feedback.
-
The focus and pace of Cuda Kernel development in this project is pretty amazing - unlike anything I have seen before.
Especially the code contributions by @ngc92 @ademeure - and @karpathy's attempt to keep the project on the educational-rails is very thoughtful!
I want to level up to @ngc92 - the deep expertise is very inspiring - but I feel I'm in the Cuda-KinderGarden and contributions to the project are at such a high level that I'm going though a sense of resentment.
In the llama2.c and nanoGPT projects I knew the language, but some of the kernels in this project feels like Assembly code :O.
This is not a criticism because very visibly it's a lot of personal willingness and work - furthermore in the future I believe I'll be thankful to each of you efforts for the open development and contributions to this project.
As kindly as I can ask, how can I grow in this area? (specifically the Cuda Kernels :D - I want to join the party)
Beta Was this translation helpful? Give feedback.
All reactions