-
Notifications
You must be signed in to change notification settings - Fork 813
How can I confirm that my program is actually running AOT kernels #15087
-
I'm working with Intel GPUs and find it's difficult to debug AOT/JIT problems. As kernels will run with JIT if there is no AOT exist for current runtime hardware, a program which intent to be AOT will run successfully anyway even if AOT settings are misconfigured (e.g. set target to PVC while intent to work on a DG2 platform). It is very hard to expose the problem as the performance difference is not all ways that obvious. The problem will be even more hard to located if in a project with multiple groups of kernels each with their own AOT settings.
Therefore, I'm wondering if it's possible to confirm that a program is running AOT kernels. For example, it will be helpful if there is an option to disable the JIT capability so that the program can just failed given no AOT kernels available in a runtime environment.
Beta Was this translation helpful? Give feedback.
All reactions
Using SYCL_DUMP_IMAGES is not a guarantee here. The thing is that even if you don't see a SPIR-V module dumped this way but some device-specific binary, there can still be a fallback to JIT compilation for Intel GPUs.
The thing is that Intel GPU offline compiler (ocloc) silently embeds SPIR-V into binaries it produces so that when that binary is attempted to be used on a device it was not compiled for, a JIT compilation can be performed.
That can be mitigated by passing -exclude_ir flag to device compiler (through -Xsycl-target-backend flag if I'm not mistaken with its spelling). Note that SYCL RT itself does not have a capability to perform AOT -> JIT fallback and by applying this change...
Replies: 1 comment 1 reply
-
Consider me as beginner with this topic, but if I understood correctly, the difference between AOT/JIT is that kernels are compiled directly to target architecture with AOT and to SPIR-V with JIT. If this is the case, maybe checking the target triple of dumped kernels would give you this information:
clang -fsycl hello.cpp -o hello
SYCL_DUMP_IMAGES=1 ./hello
llvm-spirv -r sycl_spirv64.spv -o sycl.bc
llvm-dis sycl.bc -o sycl.ll
head sycl.ll
; ModuleID = 'ss.bc'
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v16:16:16-v24:32:32-v32:32:32-v48:64:64-v64:64:64-v96:128:128-v128:128:128-v192:256:256-v256:256:256-v512:512:512-v1024:1024:1024-G1"
target triple = "spir64-unknown-unknown"
I think if the target of dumped kernel is spir64, it implies JIT (But not sure if this is the case). If target is something else you can't even translate it with llvm-spirv.
Beta Was this translation helpful? Give feedback.
All reactions
-
Using SYCL_DUMP_IMAGES is not a guarantee here. The thing is that even if you don't see a SPIR-V module dumped this way but some device-specific binary, there can still be a fallback to JIT compilation for Intel GPUs.
The thing is that Intel GPU offline compiler (ocloc) silently embeds SPIR-V into binaries it produces so that when that binary is attempted to be used on a device it was not compiled for, a JIT compilation can be performed.
That can be mitigated by passing -exclude_ir flag to device compiler (through -Xsycl-target-backend flag if I'm not mistaken with its spelling). Note that SYCL RT itself does not have a capability to perform AOT -> JIT fallback and by applying this change you may encounter that your app doesn't work at all on HW it was not AOT-compiled for.
Beta Was this translation helpful? Give feedback.