Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

How to implement Futures #586

Unanswered
0c3d7r4 asked this question in Q&A
Discussion options

For GraalJS, there's a section on Promises in the documentation:

Value jsPromise = context.eval(ID, "Promise.resolve(42);");
Consumer<Object> javaThen = (value)
 -> System.out.println("Resolved from JavaScript: " + value);
jsPromise.invokeMember("then", javaThen);

So I've tried similar with Python, given a script

from asyncio import run,sleep
from typing import TYPE_CHECKING
if TYPE_CHECKING:
 # only for static analysis / editor
 from promise_types import Promise # will not execute at runtime
async def read1():
 print('[🥇 py]', 'reading! test/fixture/index.js')
 r = fs.read('test/fixture/index.js')
 print("awaiting on",r)
 res = await r
 print('[🥇 py]', 'read test/fixture/index.js')
 return res
async def read2():
 print('[🥇 py]', 'reading test1/fixture/hello-world.js')
 r = fs.read('test1/fixture/hello-world.js')
 print("awaiting on",r)
 res = await r
 print('[🥇 py]', 'read test1/fixture/hello-world.js')
 return res
async def main():
 try:
 R1=await read1()
 await sleep(0.5)
 R2=await read2()
 except Exception as er:
 print('[🥇 py] ❌', str(er))
 raise
run(main())

and an adapter:

def new_future():
 loop,running=ensure_main_loop()
 fut=loop.create_future()
 return fut

In Java, we create a future:

public <T> Value newPromise(BiConsumer<Consumer<T>, Consumer<Throwable>> callback) {
 var pyBindings = context.getBindings("python");
 var new_future = pyBindings.getMember("new_future");
 var future = new_future.execute();
 // create pure Java callbacks that complete the Python future
 Consumer<T> resolve = result -> future.invokeMember("set_result", context.asValue(result));
 Consumer<Throwable> reject = error -> future.invokeMember("set_exception", error);
 // call user lambda with pure Java callbacks
 callback.accept(resolve, reject);
 return future;
 }

With the Java-based APIs:

 private Object read(Value... arguments) {
 if (arguments.length == 0) {
 throw new RuntimeException("Filename required");
 }
 var filename = arguments[0].asString();
 var encoding = arguments.length > 1 ? arguments[1].asString() : "utf8";
 if (program.DEBUG) {
 System.out.println("{🗿 graal} Starting read " + filename);
 }
 var p = program.Promise.<String>newPromise((BiConsumer<Consumer<String>, Consumer<Throwable>>) (resolve, reject) -> {
 CompletableFuture
 .supplyAsync(() -> {
 try {
 var path = Paths.get(filename);
 if (!Files.exists(path)) {
 var error = new RuntimeException("File not found: " + filename);
 throw error;
 }
 var bytes = Files.readAllBytes(path);
 var content = new String(bytes, getCharset(encoding));
 return content;
 } catch (Exception e) {
 throw new RuntimeException("Failed to read file: " + e.getMessage(), e);
 }
 }, program.virtualThreadExecutor)
 .whenComplete((content, error) -> {
 if (program.DEBUG) {
 if (error != null) {
 System.out.println("{🗿 graal} 📕 Failed read " + filename);
 } else {
 System.out.println("{🗿 graal} 📖 Completed read " + filename);
 }
 }
 final var c = content;
 final var e = error;
 program.loop(() -> {
 if (e != null) {
 var cause = (e.getCause() != null) ? e.getCause() : e;
 reject.accept(e);
 // program.error(cause.getMessage())
 } else {
 resolve.accept(c);
 }
 });
 });
 // return null;
 });
 if (program.DEBUG) {
 System.out.println("{🗿 graal} returning future");
 }
 return p;
 }
 private ProxyObject createFSObject() {
 return ProxyObject.fromMap(new java.util.HashMap<String, Object>() {
 {
 put("read", (ProxyExecutable) MyFileSystem.this::read);
 put("readBuffer", (ProxyExecutable) MyFileSystem.this::readBuffer);
 }
 });
 }

I've spent an entire weekend hacking and it just gives inconsistent results... Sometime it run, other times it doesn't... When the promise is resolved on the same tick (during debug, returning an already finished future), it's fine but it's pending first, it stops working

[🥇 py] reading! test/fixture/index.js
{🗿 graal} Starting read test/fixture/index.js
{🗿 graal} 📖 Completed read test/fixture/index.js
{🔁 loop} completed task
{🗿 graal} returning promise
awaiting on <Future finished result='// some js file'>
^ <<< fine on finished futures
[🥇 py] read test/fixture/index.js
[🥇 py] reading test1/fixture/hello-world.js
{🗿 graal} Starting read test1/fixture/hello-world.js
{🗿 graal} 📖 Completed read test1/fixture/hello-world.js
{🔁 loop} completed task
{🗿 graal} returning promise
awaiting on <Future finished result=''>
[🥇 py] read test1/fixture/hello-world.js

but when the promise is pending initially, it gets stuck

[🥇 py] reading! test/fixture/index.js
{🗿 graal} Starting read test/fixture/index.js
{🗿 graal} returning promise
awaiting on <Future pending>
{🗿 graal} 📖 Completed read test/fixture/index.js
... <<<<< does not return to where the promise was awaited

Is there more docs on futures and how i can return promised returns from my java infra?

You must be logged in to vote

Replies: 1 comment

Comment options

The thing I understood so far, is that "It is safe to use a context instance from a single thread. It is also safe to use it with multiple threads if they do not access the context at the same time.". So basically I'm trying to make access to context through the same 1 thread throughout the program. HOWEVER when this works in JS:

 JsResult run(Source source) throws InterruptedException, ExecutionException {
 CompletableFuture<JsResult> finalResult = new CompletableFuture<>();
 // Submit everything to the JS event loop
 loop(() -> {
 try {
 var result = context.eval(source); // Eval runs on gold-event-loop thread
 var state = "pending";
 if (result.hasMember("state")) {
 state = result.invokeMember("state").asString();
 }
 if ("fulfilled".equals(state)) {
 finalResult.complete(JsResult.ok(result.invokeMember("value")));
 } else if ("rejected".equals(state)) {
 var reason = result.invokeMember("reason");
 finalResult.completeExceptionally(
 new RuntimeException(reason.toString()));
 } else if (result != null && result.hasMember("then")) {
 // Promise: attach handlers
 result.invokeMember("then", (ProxyExecutable) args -> {
 finalResult.complete(JsResult.ok(args.length > 0 ? args[0] : null));
 return null;
 }, (ProxyExecutable) args -> {
 var jsError = args.length > 0 ? args[0] : null;
 finalResult.complete(JsResult.error(jsError));
 return null;
 });
 } else {
 // Not a Promise, resolve immediately
 finalResult.complete(JsResult.ok(result));
 }
 } catch (Throwable t) {
 // Any immediate JS error
 finalResult.complete(JsResult.error(t));
 }
 });
 // Pump JS event loop until the result completes (or fails)
 eventLoop.runUntil(finalResult::isDone);
 // Stop the event loop now to avoid dangling tasks
 close();
 // Propagate exceptions to caller
 return finalResult.get();
 }

With top-level await, the result is gotten immediately as pending promise (top-level) and the event loop is freed so that other promises can get resolved, but in Python, similar code

 PyResult run(Source source) throws InterruptedException, ExecutionException {
 CompletableFuture<PyResult> finalResult = new CompletableFuture<>();
 loop(()->{
 try {
 var result = context.eval(source);
 finalResult.complete(PyResult.ok(result));
 }catch(Throwable err) {
 finalResult.complete(PyResult.error(err));
 }
 });
 eventLoop.runUntil(finalResult::isDone);
 return finalResult.get();
 }

does not even work, because context.eval on top-level code that called asyncio.run, blocks the whole of the event loop, making it impossible to resolve inner futures which are prerequisite to resolving the top-level one... it's basically a dead-lock, which does not happen in JS because context.eval returns a promise.
and even I go fancy-pancy and run the top-level python module non in the event loop but in the main program loop, those futures still won't be resolved. i don't understand how is this not documented... what is even the point of graalpython if not java integration

// event-loop related
 void loop(Runnable callable) {
 eventLoop.submit(callable);
 }
 private void loop() {
 try {
 while (running.get()) {
 var task = queue.take();
 currentTask = task;
 context.enter();
 try {
 task.run(); // the task in python doesn't get complete because it's the main loop started by us or something.
 } finally {
 context.leave();
 }
 }
 } catch (InterruptedException ignored) {
 // System.out.println("error 100 in the event loop");
 }
 }
You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
1 participant

AltStyle によって変換されたページ (->オリジナル) /