Filesystem#

Note

The libuv filesystem operations are different from socket operations. Socket operations use the non-blocking operations provided by the operating system. Filesystem operations use blocking functions internally, but invoke these functions in a thread pool and notify watchers registered with the event loop when they complete.

All filesystem operations have two forms — synchronous and asynchronous.

Synchronous operations are slightly easier to program with, and are faster, because they run in the current thread instead of the thread pool (note again: this is only an issue for filesystem operations, not sockets or pipes). However, they can block the thread. This can happen, for example, if you use a synchronous operation to access a network filesystem which is slow or unavailable. It can also happen locally. So, synchronous operations should not be used in robust applications or libraries. Synchronous filesystem operations are found in module Luv.File.Sync.

Asynchronous versions of all the same operations are found in the main module Luv.File. These each take a callback, to which they pass their result, instead of returning their result directly, as synchronous operations do.

As already mentioned, asynchronous filesystem operations are slower than synchronous, because running an asynchronous operation requires communicating twice with a thread in the libuv thread pool: once to start the operation, and once more when it is complete. In some cases, you can mitigate this cost by manually running multiple operations in one request in the thread pool. To do that, use Luv.Thread_pool.queue_work, and run multiple synchronous operations in the function you pass to it.

Note that this only offers a performance benefit if you run multiple operations. Running only one operation is equivalent to simply using the asynchronous API — that’s how the asynchronous API is implemented.

The rest of this chapter sticks to the asynchronous API.

Reading/writing files#

A file descriptor is obtained by calling Luv.File.open_:

Luv.File.open_ :
  ?mode:Luv.File.Mode.t list ->
  string ->
  Luv.File.Open_flag.t list ->
  ((Luv.File.t, Luv.Error.t) result -> unit) ->
    unit

Despite the verbose signature, usage is simple:

Luv.File.open_ "foo" [`RDONLY] (fun result -> (* ... *))

Luv.File.Open_flag.t and Luv.File.Mode.t expose the standard Unix open flags and file modes, respectively. libuv takes care of converting them to appropriate values on Windows. mode is optional, because it is only used if open_ creates the file.

File descriptors are closed using Luv.File.close:

Luv.File.close : Luv.File.t -> ((unit, Luv.Error.t) result -> unit) -> unit

Finally, reading and writing are done with Luv.File.read and Luv.File.write:

Luv.File.read :
  Luv.File.t ->
  Luv.Buffer.t list ->
  ((Unsigned.Size_t.t, Luv.Error.t) result -> unit) ->
    unit

Luv.File.write :
  Luv.File.t ->
  Luv.Buffer.t list ->
  ((Unsigned.Size_t.t, Luv.Error.t) result -> unit) ->
    unit

Let’s put all this together into a simple implementation of cat:

cat.ml

let () =
  Luv.File.open_ Sys.argv.(1) [`RDONLY] begin function
    | Error e ->
      Printf.eprintf "Error opening file: %s\n" (Luv.Error.strerror e)
    | Ok file ->
      let buffer = Luv.Buffer.create 1024 in

      let rec on_read = function
        | Error e ->
          Printf.eprintf "Read error: %s\n" (Luv.Error.strerror e)
        | Ok bytes_read ->
          let bytes_read = Unsigned.Size_t.to_int bytes_read in
          if bytes_read = 0 then
            Luv.File.close file ignore
          else
            Luv.File.write
              Luv.File.stdout
              [Luv.Buffer.sub buffer ~offset:0 ~length:bytes_read]
              on_write

      and on_write = function
        | Error e ->
          Printf.eprintf "Write error: %s\n" (Luv.Error.strerror e)
        | Ok _bytes_written ->
          Luv.File.read file [buffer] on_read

      in

      Luv.File.read file [buffer] on_read
  end;

  ignore (Luv.Loop.run () : bool)

Note

on_write is not robust, because we don’t check that the number of bytes written is actually the number of bytes we requested to write. Fewer bytes could have been written, in which case we would need to try again to write the remaining bytes.

Warning

Due to the way filesystems and disk drives are configured for performance, a write that succeeds may not be committed to disk yet.

Filesystem operations#

All the standard filesystem operations like unlink, rmdir, stat are supported both synchronously and asynchronously. The easiest way to see the full list, with simplified signatures, is to scroll down through Luv.File.Sync.

File change events#

All modern operating systems provide APIs to put watches on individual files or directories and be informed when the files are modified. libuv wraps common file change notification libraries [1], and the wrapper is exposed by Luv as module Luv.FS_event. To demonstrate, let’s build a simple utility which runs a command whenever any of the watched files change:

./onchange <command> <file1> [file2] ...

onchange.ml

let () =
  match Array.to_list Sys.argv with
  | [] | [_] | [_; _] ->
    Printf.eprintf "Usage: onchange <command> <file1> [file2]..."

  | _::command::files ->
    files |> List.iter begin fun target ->
      let watcher = Luv.FS_event.init () |> Result.get_ok in

      Luv.FS_event.start ~recursive:true watcher target begin function
        | Error e ->
          Printf.eprintf
            "Error watching %s: %s\n" target (Luv.Error.strerror e);
          ignore (Luv.FS_event.stop watcher);
          Luv.Handle.close watcher ignore

        | Ok (file, events) ->
          if List.mem `RENAME events then
            prerr_string "renamed ";
          if List.mem `CHANGE events then
            prerr_string "changed ";
          prerr_endline file;

          let exit_status = Sys.command command in
          if exit_status <> 0 then
            Stdlib.exit exit_status
      end
    end;

  ignore (Luv.Loop.run () : bool)