Learn to solve parallel problems with GPU-accelerated C++ code and create reusable libraries that can be accessed from other programming languages Key Features Harness the power of GPU parallelism to accelerate real-world tasks - Utilize CUDA streams and scale performance with custom C++ solutions - Create reusable GPU libraries and expose them to Python seamlessly Book Description Written by Paulo Motta, a senior researcher with decades of experience, this comprehensive GPU programming book is an essential guide for leveraging the power of parallelism to accelerate your computations. The first section introduces the concept of parallelism and provides practical advice on how to think about and utilize it effectively. Starting with a basic GPU program, you then gain hands-on experience in managing the device. This foundational knowledge is then expanded by parallelizing the program to illustrate how GPUs enhance performance. The second section explores GPU architecture and implementation strategies for parallel algorithms, and offers practical insights into optimizing resource usage for efficient execution. In the final section, you will explore advanced topics such as utilizing CUDA streams. You will also learn how to package and distribute GPU-accelerated libraries for the Python ecosystem, extending the reach and impact of your work. Combining expert insight with real-world problem solving, this book is a valuable resource for developers and researchers aiming to harness the full potential of GPU computing. The blend of theoretical foundations, practical programming techniques, and advanced optimization strategies it offers is sure to help you succeed in the fast-evolving field of GPU programming. What you will learn Manage GPU devices and accelerate your applications - Apply parallelism effectively using CUDA and C++ - Choose between existing libraries and custom GPU solutions - Package GPU code into libraries for use with Python - Explore advanced topics such as CUDA streams - Implement optimization strategies for resource-efficient execution Who this book is for C++ developers and programmers interested in accelerating applications using GPU programming will benefit from this book. It is suitable for those with solid C++ experience who want to explore high-performance computing techniques. Familiarity with operating system fundamentals will help when dealing with device memory and communication in advanced chapters. Table of Contents Introduction to Parallel Programming - Getting Started - Hello CUDA - Hello again, but in parallel - A closer look into the GPU world - Data Management and Persistence - Performance strategies - Using multiple GPUs - Exposing your code as a Python Library - Exploring the existing GPU models “This Packt release introduces concepts with practical examples and small code snippets you can actually execute on NVIDIA GPUs. The writing style is simple, sometimes humorous, and easy to follow without much background in this area. If you have existing knowledge of C++ and an interest in GPU programming, this one is worth a look.” Demitri Swan, Senior Software Engineer at Apple “What I love so far is how the book demystifies the GPU world: from the fundamentals of parallelism and SIMD execution, to incredibly clear explanations of streaming multiprocessors, memory hierarchies, and how kernels actually run across thousands of threads (Figures 1.4–1.7 break this down beautifully) The hands-on approach is gold, building your first CUDA kernel, understanding thread/block/grid indexing, exploring device properties, and even setting up a full dev workflow with Docker + VS Code. And then it escalates into the fun parts: memory transfers, shared memory, events, profiling with Nsight, and writing truly optimized kernels If you're diving into high-performance computing, AI systems engineering, or just want to understand the machinery behind modern GPU acceleration, this book is like having a guided tour of the GPU’s internals, practical, visual, and insanely well-structured. Highly recommended. ” Michael Erlihson, Head of AI Stealth Mode Paulo Motta completed the PhD in Computer Science with an emphasis in parallel systems at PUC-Rio in 2011. Currently, Paulo Motta is a Senior Research Software Development Engineer at Microsoft and a postdoctoral researcher on quantum walks simulations with Hiperwalk at the National Scientific Computing Laboratory in Brazil. Paulo is a senior member of IEEE Computer Society with over 25 years' experience in software development and 9 years experience as a university professor.