OCaml Weekly News

opam install slug

If you use esy

esy add @opam/slug

• OCaml syntax:

  Slug.slugify "my string";;
  - : string = "my-string"
  (* Custom separator *)
  Slug.slugify ?sep: "_" "my string";;
  - : string = "my_string"
  (* Retain uppercase *)
  Slug.slugify ?lowercase: false "My String";;
  - : string = "My-String"
  (* Use locale *)
  let with_vi = Slug.(Charmap.mk_charmap [Slug_data.base; Slug_data.vi]);;
  - : Charmap.t = <abstr>
  Slug.slugify ?charmap: with_vi "Đ";;
  - : string = "d"
  Slug.slugify "Đ";;
  - : string = "dj"
  (* Custom characters map *)
  let custom_map = Slug.(Charmap.mk_charmap [Slug_data.base; [ ("M", "z"); ("m", "z") ]]);;
  val custom_map : Charmap.t = <abstr>
  Slug.slugify ?charmap: custom_map "Mm";;
  - : string = "zz"
  

• ReasonML syntax

  Slug.slugify("my string");
  - : string = "my-string"
  /* Custom separator */
  Slug.slugify(~sep = "_", "my string");
  - : string = "my_string"
  /* Retain uppercase */
  Slug.slugify(~lowercase = false, "My String");
  - : string = "My-String"
  /* Use locale */
  let with_vi = Slug.(Charmap.mk_charmap([Slug_data.base, Slug_data.vi]));
  let with_vi : Charmap.t = <abstr>
  Slug.slugify(~charmap = with_vi, "Đ");
  - : string = "d"
  Slug.slugify("Đ");
  - : string = "dj"
  /* Custom characters map */
  let custom_map = Slug.(Charmap.mk_charmap([Slug_data.base, [ ("M", "z"), ("m", "z") ]]));;
  let custom_map : Charmap.t = <abstr>
  Slug.slugify(~charmap = custom_map, "Mm");
  - : string = "zz"
  

Please don't send PR to update data/*. They are auto-generated from upstream library.

Please send PRs about new locales to node-slugify.

If you really need it, you can use a custom_map instead.

MIT. Data are downloaded from node-slugify

• ocaml-multicore/ocaml-multicore#573 Backport trunk safepoints PR to multicore

  A work-in-progress to backport the Safepoints PR from ocaml/ocaml to Multicore OCaml.

• ocaml-multicore/ocaml-multicore#584 Modernise signal handling

  A patch to bring the Multicore OCaml signals implementation closer to upstream OCaml.

• ocaml-multicore/ocaml-multicore#598 Do not deliver signals to threads that have blocked them

  A draft PR to not deliver signals to threads that are in a blocked state. The without-systhreads case needs to be handled.

• ocaml-multicore/ocaml-multicore#600 Expose a few more GC variables in headers

  The caml_young_start, caml_young_limit and caml_minor_heap_wsz variables have been defined in the runtime.

• ocaml-multicore/ocaml-multicore#601 Domain better participants

  The iterations 0(Max_domains) from STW signalling and 0(n_running_domains) from domain creation have now been removed.

• ocaml-multicore/ocaml-multicore#603 Systhreads tick thread

  An initial draft PR for porting the tick thread to Multicore OCaml.

Enhancements

Upstream

Fixes

Cleanups

Sundries

• ocaml-multicore/eventlog-tools#3 Use ocaml/setup-ocaml@v2

  An update to .github/workflows/main.yml to build for ocaml/setup-ocaml@v2.

• ocaml-multicore/parallel-programming-in-multicore-ocaml#7 Add a section on Domain-Local Storage

  The README.md file now includes a section on Domain-Local Storage.

• ocaml-multicore/eio#26 Grand Central Dispatch Backend

  The implemention of the Grand Central Dispatch (GCD) backend for Eio is a work-in-progress.

• ocaml-multicore/domainslib#34 Fix initial value accounting in parallel_for_reduce

  A patch to fix the initial value in parallel_for_reduce as it was being accounted for multiple times.

• ocaml-multicore/domainslib#36 Switch to default Random module

  The library has been updated to use the default Random module as it stores its state in Domain-Local Storage which can be called from multiple domains. The Sandmark results are given below:

  9a68704f282ac07167fb7d909334befa437593f0.png

• ocaml-multicore/multicore-opam#56 Base-effects depends strictly on 4.12

  A query on the use of strict 4.12.0 lower bound for OCaml in base-effects.base/opam.

• ocsigen/lwt#860 Lwt_domain: An interfacet to Multicore parallelism

  The Lwt_domain module has been ported to domainslib Task pool for performing computations to CPU cores using Multicore OCaml's Domains. A few benchmark results obtained on an Intel Xeon Gold 5120 processor with 24 isolated cores is shown below:

  1e2d018635857f3603a1b7f9573176840efabede.png

Ocaml-Uring

Parallel Programming in Multicore OCaml

Eio

• Additions
  

  • ocaml-multicore/eio#41 Add eio.mli file

    A lib_eio/eio.mli file containing modules for Generic, Flow, Network, and Stdenv have been added to the repository.

  • ocaml-multicore/eio#45 Add basic domain manager

    The PR allows you to run a CPU-intensive task on another domain, and adds a mutex to traceln to avoid overlapping output.

  • ocaml-multicore/eio#46 Add Eio.Time and allow cancelling sleeps

    Use psq instead of bheap library to allow cancellations. The Eio.Time module has been added to lib_eio/eio.ml.

  • ocaml-multicore/eio#53 Add Switch.sub_opt

    A new Switch.sub_opt implementation has been added to allow running a function with a new switch. Also, Switch.sub has been modified so that it is not a named argument.

  • ocaml-multicore/eio#54 Initial FS abstraction

    A module Dir has been added to allow file system abstraction along with the ability to create files and directories. On Linux, it uses openat2 and RESOLVE_BENEATH.

  • ocaml-multicore/eio#56 Add with_open_in, with_open_out and with_open_dir helpers

    The Eio.Dir module now contains a with_open_in, with_open_out and with_open_dir helper functions.

  • ocaml-multicore/eio#58 Add Eio_linux.{readv, writev}

    The Eio_linux.{readv, writev} functions have been added to lib_eio_linux/eio_linux.ml which uses the new OCaml-Uring API.

  • ocaml-multicore/eio#59 Add Eio_linux.noop and a simple benchmark

    A Eio_linux.noop implementation has been added for benchmarking Uring dispatch.

  • ocaml-multicore/eio#61 Add generic Enter effect to simplify scheduler

    A Enter effect has been introduced to simplify the scheduler operations, and this does not have much effect on the noop benchmark as illustrated below:

    3c1b2df002e1f57be2850d8b81d5eb08afb097dd.png

• Improvements
  

  • ocaml-multicore/eio#38 Rename Flow.write to Flow.copy

    The code and documentation have been updated to rename Flow.write to Flow.copy for better clarity.

  • ocaml-multicore/eio#36 Use uring for accept

    The enqueue_accept function now uses Uring.accept along with the effect Accept.

  • ocaml-multicore/eio#37 Performance improvements

    Optimisation for Eunix.free and process completed events with Uring.peek for better performance results.

  • ocaml-multicore/eio#48 Simplify Suspend operation

    The Suspend effect has been simplified by replacing the older Await and Yield effects with the code from Eio.

  • ocaml-multicore/eio#52 Split Linux support out to eio_linux library

    eunix now has common code that is shared by different backends, and eio_linux provides a Linux io-uring backend. The tests and the documentation have been updated to reflect the change.

  • ocaml-multicore/eio#57 Reraise exceptions with backtraces

    Added support to store a reference to a backtrace when a switch catches an exception. This is useful when you want to reraise the exception later.

  • ocaml-multicore/eio#60 Simplify handling of completions

    The PR adds Job and Job_no_cancel in type io_job along with additional Log.debug messages.

• Cleanups
  

  • ocaml-multicore/eio#42 Merge fibreslib into eio

    The Fibreslib code is now merged with eio. You will now need to open Eio.Std instead of opening Fibreslib.

  • ocaml-multicore/eio#47 Clean up the network API

    The network APIs have been updated with few changes such as renaming bind to listen, replacing Unix.shutdown_command with our own type in Eio API, and replacing Unix.sockaddr with a custom type.

  • ocaml-multicore/eio#49 Remove Eio.Private.Waiters and Eio.Private.Switch

    The Eio.Private.Waiters and Eio.Private.Switch modules have been removed, and waiting is now handled using the Eio library.

  • ocaml-multicore/eio#55 Some API and README cleanups

    The PR has multiple cleanups and documentation changes. The README.md has been modified to use Eio.Flow.shutdown instead of Eio.Flow.close, and a Time section has been added. The Eio.Network module has been changed to Eio.Net. The Time.now and Time.sleep_until methods have been added to lib_eio/eio.ml.

• Documentation
  

  • ocaml-multicore/eio#43 Add design note about determinism

    The README.md documentation has been updated with few design notes on Determinism.

  • ocaml-multicore/eio#50 README improvements

    Updated README.md and added doc/prelude.ml for use with MDX.

Handling Cancellation

Sundries

• ocaml-bench/sandmark-nightly#1 Cannot alter comparison input values

  The Timestamp and Variant fields in the dropdown option in the parallel_nightly.ipynb notebook get reset when recomputing the whole workbook.

  e029e7d420487a4f3f63893dd4412322c1933787_2_1380x278.png

• ocaml-bench/sandmark#230 Build for 4.13.0+trunk with dune.2.8.1

  The ocaml-migrate-parsetree.2.2.0 and ppxlib.0.22.2 packages are now available for 4.13.0+trunk, and we are currently porting the Irmin Layers benchmark in Sandmark from using Irmin 2.4 to 2.6.

• ocaml-bench/sandmark#231 View results for a set of benchmarks in the nightly notebooks

  A feature request to filter the list of benchmarks when using the Sandmark Jupyter notebooks.

• ocaml-bench/sandmark#233 Update pausetimes_multicore to fit with the latest Multicore changes

  The pausetimes are now updated for both the 4.12.0 upstream and 4.12.0 Multicore branches to use the new Common Trace Format (CTF). The generated graphs for both the sequential and parallel pausetime results are illustrated below:

  9aa420116b999df5b89cd106b211673b3d2afb1d_2_1380x458.png

  cc380d59494290056577f76f977b0ffcb4199e55_2_1380x710.png

• ocaml-bench/sandmark#235 Update selected benchmarks as a set for baseline benchmark

  The baseline benchmark for comparison should only be one from the user selected benchmarks in the Jupyter notebooks.

  fba96b8d43ee766bff262ec5ca0c48e356e8d9cb.png

• ocaml-bench/sandmark#236 Implement pausetimes support in sandmark_nightly

  The sequential and parallel pausetimes graph results need to be implemented in the Sandmark nightly Jupyter notebooks. The results are similar to the Figures 10 and 12 produced in the Retrofitting Parallelism ont OCaml, ICFP 2020 paper.

• ocaml-bench/sandmark#237 Run sandmark_nightly on a larger machine

  The testing of Sandmark nightly sequential and parallel benchmark runs have been done on a 24-core machine, and we would like to deploy the same on a 64+ core machine to benefit from the recent improvements to Domainslib.

• ocaml-bench/sandmark#241 Switch to default Random module

  An on-going discussion on whether to switch to using Random.State for the sequential Minilight, global roots micro-benchmarks and Evolutionary Algorithm.

• ocaml-bench/sandmark#232 num_domains -> num_additional_domains

  The benchmarks have been updated to now use num_additional_domains, to be consistent with the naming in Domainslib.

• ocaml-bench/sandmark#239 Port grammatrix to Task pool

  The Multicore Grammatrix benchmark has now been ported to use Domainslib Task pool. The time and speedup graphs are given below:

  d1e2d18707d543bc73cfa967c7dbfc2dd4783416.png

  4b920039c0ebf9b1fbac4597b08ecf40f6e0f14f.png

• ocaml/ocaml#10039 Safepoints

  The PR is currently being testing and evaluated for both ARM64 and PowerPC architectures, in particular, the branch relaxations applied to Ipoll instructions.

Our thanks to all the OCaml users, developers and contributors in the community for their continued support to the project. Stay safe!

(* worker *)
let f () =
 let rec alloc n x =
 if n = 0 then x else alloc (n-1) (()::x)
 in
 (* allocate 300'000 kw *)
 for i = 0 to 100_000 do ignore (alloc 1_000 []) done
(* monitor *)
let g () =
 match Memprof_limits.limit_allocations ~limit:330_000L f with
 | Ok ((), n) -> Printf.printf "success (est. alloc. %#Lu kw)\n" n
 | Error _ -> print_endline "out of fuel"
(* main *)
let () =
 Memprof_limits.start_memprof_limits () ;
 g ()