JNA CoCreateInstance returns S_OK but ShellLink/IShellLinkW fails, COM pointer is non-null but object unusable

Question

I'm making a Java app that scans the Windows Recent Files folder and resolves .lnk shorcuts to their actual file paths. I'm doing so using JNA the JNA API and simulating IShellLinkW and IPersistFile.

Every time I run my code, CoCreateInstance returns S_OK and the COM pointer is non-null, but my ShellLink object fails immediately as well as my IPersistFile.load.

This is my debug log (I generated a lot of debug messages) after clearing the recent files folder and downloading a dummy pdf:

Resolving: pdf-sample_0.pdf.lnk
===== ShellLink.createShellLink() BEGIN =====
CLSID_SHELL_LINK = {00021401-0000-0000-C000-000000000046}
IID_SHELL_LINK_W = {000214F9-0000-0000-C000-000000000046}
[1] Calling CoInitialize...
[1] CoInitialize HRESULT = 1 (S_FALSE)
[2] Calling CoCreateInstance...
[2] CoCreateInstance HRESULT = 0 (S_OK)
[2] HRESULT (hex) = 0x00000000
[2] Returned COM pointer (ppv) = native@0x1eeef60df60
!!! ShellLink create FAILED !!!
!!! HRESULT = 0x00000000
!!! ppv = native@0x1eeef60df60
Error resolving: pdf-sample_0.pdf.lnk

My question: Why would JNA treat this COM pointer as invalid when CoCreateInstance returns S_OK? Am I using the incorrect COM IDs for the classes/interfaces? Wrong virtual table indices? Or something else?

Relevant Code

1. Relevant Config.java constants

public static final String WIN32_CLASS_ID_OBJECT_SHELL_LINK = "{00021401-0000-0000-C000-000000000046}";
public static final String WIN32_INTERFACE_ID_SHELL_LINK_WIDE = "{000214F9-0000-0000-C000-000000000046}";
public static final String WIN32_INTERFACE_ID_PERSIST_FILE = "{0000010B-0000-0000-C000-000000000046}";
public static final int WIN32_MAX_PATH = 260;
public static final int WIN32_VTABLE_GETPATH = 20;
public static final int WIN32_VTABLE_RESOLVE = 21;
public static final int WIN32_VTABLE_LOAD = 5;

2. ShellLink.createLink snippet (seems like the main issue is happening here)

public static IShellLinkW createShellLink() {
 // Initialize COM
 WinNT.HRESULT initRes = Ole32.INSTANCE.CoInitialize(null);
 // Create the ShellLink COM object
 PointerByReference ppv = new PointerByReference();
 WinNT.HRESULT hres = Ole32.INSTANCE.CoCreateInstance(
 CLSID_SHELL_LINK,
 null,
 WTypes.CLSCTX_INPROC_SERVER,
 IID_SHELL_LINK_W,
 ppv
 );
 Pointer rawPtr = ppv.getValue();
 // Failure
 if (hres == null || hres.intValue() != WinNT.S_OK || rawPtr == null) {
 Ole32.INSTANCE.CoUninitialize();
 return null;
 }
 // Success
 return new IShellLinkW(rawPtr);
}

Even though hres == S_OK and rawPtr != null, this if block still runs.

I am running a Windows 10 machine, and the code runs on Java 17 with JNA 5.13.0. The folder I'm scanning is %APPDATA%\Microsoft\Windows\Recent. If it helps, this app is a JavaFX app.

If it helps, here's a link to the GitHub repository. I would very much appreciate some guidance, thank you!

• Daniel Widdis: Your vtbl indices are wrong. I don't see in your question the actual line of code that is failing and giving "Error resolving: pdf-sample_0.pdf.lnk" but if it's using the vtbl index that's the likely cause. Where are you calling createShellLink()? Which method on it are you using? Also you say "this if block still runs" which would imply you're returning null rather than a new object, but your debug doesn't show you getting an NPE trying to call any methods on it. Please clarify your question with the actual code causing your error.
• DuncG: You can also use jextract and Foreign Function and Memory API to access lnk shortcuts - see Windows shortcut (.lnk) parser in Java

Answer 1

Am I using the incorrect COM IDs for the classes/interfaces?

Those look correct.

Wrong virtual table indices?

Yes, you're using the wrong virtual table indices.

The Unknown class consumes the first three indices (0, 1, 2) and after that you'd count the IShellLinkW functions in order:

public static final int WIN32_VTABLE_GETPATH = 3; // not 20
public static final int WIN32_VTABLE_RESOLVE = 19; // not 21

Your question doesn't include the actual method call producing the error, but given the message includes the word "resolve" the above changes will likely fix your problem.

Your IPersistFile functions start indexing at 4 (it extends IPersist with one function and Unknown with three) so this one is correct.

public static final int WIN32_VTABLE_LOAD = 5;

One other suggestion to improve your code: rather than manually checking all the HRESULT values, just use ComUtils methods like checkRC() or FAILED()/SUCCEEDED().