diff options
Diffstat (limited to 'guile-zig-local/README.md')
| -rw-r--r-- | guile-zig-local/README.md | 447 |
1 files changed, 447 insertions, 0 deletions
diff --git a/guile-zig-local/README.md b/guile-zig-local/README.md new file mode 100644 index 0000000..ec4a664 --- /dev/null +++ b/guile-zig-local/README.md @@ -0,0 +1,447 @@ +# Guile & Zig + +Zig is a blazingly fast new programing language that adhers to a stable ABI and that makes it suitable to link against other programming languages. Zig comes withouth a garbage collector, and that makes it even better. + +This repo contains code to show how we can link Zig with Guile - but, really, zig directly supports any language that allows for a C FFI. + +## Install + +To create an environment with GNU Guix you can run with [guix.scm](guix.scm). + +``` +guix shell -C -D -f guix.scm +zig version +guile -v +``` + +This way guile can be run in an emacs shell with M-x shell -> guile. + +## Compile zig + +Run zit source tests with + +``` +zig test my.zig +``` + +Create libmy.so and show exported symbols with: + +``` +zig build-lib -dynamic my.zig +nm -D --demangle libmy.so +``` + +## Print zig hello world from C + +The quick way is to create a C function that calls a zig function (no guile yet): + +@1 +``` +zig build-lib -dynamic my.zig +zig cc `pkg-config --cflags guile-3.0` test.c -fPIC libmy.so $GUIX_ENVIRONMENT/lib/libguile-3.0.so -o test +env LD_LIBRARY_PATH=.:$GUIX_ENVIRONMENT/lib ./test +hello world +``` + +The `LD_LIBRARY_PATH` tells the runtime to look in the current working directory for mylib.so. + +## Print zig hello from guile + +Now we have a shared lib it should be easy to load. + +A shared library can be loaded into a running Guile process with the function load-extension. See the [guile documentation](https://www.gnu.org/software/guile/manual/html_node/Combining-with-C.html). In addition to the name of the library to load, this function also expects the name of a function from that library that will be called to initialize it: + + +// gcc `pkg-config --cflags guile-3.0` \ +// -shared -o libmy.so -fPIC .c + +``` +zig cc `pkg-config --cflags guile-3.0` test.c -fPIC libmy.so -o test +env LD_LIBRARY_PATH=. guile +``` + +```guile +(load-extension "libmy" "init_module") +``` +Should loads without error. Next we can call a function after we add a mapping to the init_module. What we want to do is: + +```guile +(hello_zig) +possibly unbound variable hello_zig +``` + +Hello stranger. + +We need to do a bit more work to make guile accept zig calls. An quick example in C is [here](https://www.gnu.org/software/guile/manual/html_node/A-Sample-Guile-Extension.html#A-Sample-Guile-Extension). libguile.h contains a range of C macros to facilite bindings. C macros (shudder - we want to get rid of those). GNU Guile is a dynamic language - that means values carry information about their type at runtime. This requires mapping values back and forth. This is true for any dynamic language - including Python, Ruby, R, etc. + +One fun aspect of GNU Guile is that almost all its functions are implemented in C with a scm_ prefix. So, if you look up functions, you'll the the guile call and the C call next to each other. In the Guile manual you'll find more information on [translating types](https://www.gnu.org/software/guile/manual/html_node/Dynamic-Types.html). You'll find a C example: + +```C +SCM my_incrementing_function (SCM a, SCM flag) +{ + SCM result; + + if (scm_is_true (flag)) + result = scm_sum (a, scm_from_int (1)); + else + result = a; + + return result; +} +``` + +that shows that the SCM macro represents different types of parameters in Guile. SCM is defined in scm.h saying `SCM values can hold proper scheme objects only.' + +If we add this function to test.c we can see its expanded version as + +``` +zig cc -E -I$GUIX_ENVIRONMENT/include/guile/3.0 test.c libmy.so -L$GUIX_ENVIRONMENT -lguile -o test +``` + +note the `-E` switch. The function expands to + +```C +SCM my_incrementing_function (SCM a, SCM flag) +{ + SCM result; + + if ((!((((((scm_t_bits) (0? (*(volatile SCM *)0=((flag))): (flag))) & ~(((scm_t_bits) (0? (*(volatile SCM + *)0=(((SCM) ((((((1)) << 8) + scm_tc8_flag)))))): ((SCM) ((((((1)) << 8) + scm_tc8_flag)))))) ^ ((scm_t_bi +ts) (0? (*(volatile SCM *)0=(((SCM) ((((((0)) << 8) + scm_tc8_flag)))))): ((SCM) ((((((0)) << 8) + scm_tc8_ +flag)))))))) == (((scm_t_bits) (0? (*(volatile SCM *)0=(((SCM) ((((((1)) << 8) + scm_tc8_flag)))))): ((SCM) + ((((((1)) << 8) + scm_tc8_flag)))))) & ((scm_t_bits) (0? (*(volatile SCM *)0=(((SCM) ((((((0)) << 8) + scm +_tc8_flag)))))): ((SCM) ((((((0)) << 8) + scm_tc8_flag)))))))))))) + result = scm_sum (a, scm_from_int32 (1)); + else + result = a; + + return result; +} +``` + +There is a bit of work going on, but mostly it is simple packing and unpacking described [here](https://www.gnu.org/software/guile/manual/html_node/Relationship-Between-SCM-and-scm_005ft_005fbits.html). To use this in Zig we'll need to translate the C macros for some conversions. Note that `scm_sum` and `scm_from_int` are simple function calls. The real challenge is converting SCM to boolean with `scm_is_true`. + +Zig comes with a utility that does that: + +@2 +``` +zig translate-c -I$GUIX_ENVIRONMENT/include/guile/3.0/ $GUIX_ENVIRONMENT/include/guile/3.0/libguile.h > test-include.zig + +``` + +And, interestingly translates all of that including + +```zig +pub const scm_t_timespec = struct_timespec; +pub const scm_t_off = i64; +pub const scm_t_signed_bits = isize; +pub const scm_t_bits = usize; +pub const struct_scm_unused_struct = extern struct { + scm_unused_field: u8, +}; +pub const SCM = [*c]struct_scm_unused_struct; +pub const scm_tc8_flag: c_int = 4; +pub const scm_tc8_char: c_int = 12; +pub const scm_tc8_unused_0: c_int = 20; +pub const scm_tc8_unused_1: c_int = 28; +pub const enum_scm_tc8_tags = c_uint; +pub const scm_t_subr = ?*anyopaque; +pub const struct_scm_dynamic_state = opaque {}; +pub const scm_t_dynamic_state = struct_scm_dynamic_state; +pub const struct_scm_print_state = extern struct { + handle: SCM, + revealed: c_int, + writingp: c_ulong, + fancyp: c_ulong, + level: c_ulong, + length: c_ulong, + hot_ref: SCM, + list_offset: c_ulong, + top: c_ulong, + ceiling: c_ulong, + ref_vect: SCM, + highlight_objects: SCM, +}; +pub const scm_print_state = struct_scm_print_state; +pub const struct_scm_dynstack = extern struct { + base: [*c]scm_t_bits, + top: [*c]scm_t_bits, + limit: [*c]scm_t_bits, +}; +``` + +that may look familiar to Guile hackers and matches the C + +```C +typedef struct scm_unused_struct { char scm_unused_field; } *SCM; +``` + +typed struct * that is SCM. Now from Zig, unless we start using real Guile internals we may get away with simply using our own opaque pointer that is passed around. + +Let's try compiling this conversion and it gives one error + +```sh +zig test test-include.zig +test-include.zig:6522:30: error: use of undeclared identifier 'sched_priority' +pub const __sched_priority = sched_priority; +``` + +Commenting out that line and the compile passes. That must be too good to be true(!) +Let's try compiling test.c with + +``` +zig cc `pkg-config --cflags guile-3.0` test.c -fPIC libmy.so $GUIX_ENVIRONMENT/lib/libguile-3.0.so -o test +env LD_LIBRARY_PATH=$GUIX_ENVIRONMENT/lib:. ./test +Hello world from 3 to 4 +``` + +Now we add the zig call without `scm_is_false` + +``` +zig build-lib -dynamic my.z +env LD_LIBRARY_PATH=$GUIX_ENVIRONMENT/lib:. ./test +Hello world from 3 to 4 + from 3 to 4 +``` + +This worked! With reintroduced `scm_is_false` we get + +``` +./test-include.zig:6119:62: error: unable to evaluate constant expression +pub inline fn scm_is_true(x: anytype) @TypeOf(!(scm_is_false(x) != 0)) { +``` + +Remember that is a C macro converted automatically. If we use a C function it works: + +``` +zig build-lib -dynamic my.zig +zig cc `pkg-config --cflags guile-3.0` test.c -fPIC libmy.so $GUIX_ENVIRONMENT/lib/libguile-3.0.so -o test +env LD_LIBRARY_PATH=$GUIX_ENVIRONMENT/lib:. ./test +Hello world from 3 to 4 +Hello world from 4 to 5 +``` + +Now let's try loading guile again + +``` +env LD_LIBRARY_PATH=. guile +``` + +and + +``` +(load-extension "libmy" "init_module") +(hello_zig) +possibly unbound variable hello_zig +``` + +while + +``` +nm -D libmy.so +000000000000aa80 T hello_zig +000000000000ab40 T my_incrementing_zig_function +000000000003d6c0 W __zig_probe_stack +``` + +shows the symbols are exported. Hmm. We need to tell guile first there exists a function. Apparently I am not the [first](https://github.com/ziglang/zig/issues/1675). After a bit of try and error with zig 0.9 we get: + +```zig +pub export fn ping_zig(i: SCM) SCM { + return i; +} + +export fn init_module() void { + // var c_ping_zig = @intToPtr(*anyopaque, @ptrToInt(&ping_zig)); <- ampersand for zig 0.10 and after + var c_ping_zig = @intToPtr(*anyopaque, @ptrToInt(ping_zig)); + var res = guile.scm_c_define_gsubr("ping_zig", 1, 0, 0, c_ping_zig); + _ = res; +} +``` + +which runs in guile with + +```guile +(load-extension "libmy" "init_module") +(ping_zig 0) +$1 = 0 +(ping_zig "TEST") +$2 = "TEST" +``` + +Woot! That works great. Note we did not have to deal with a type system. + +With a recent version of zig the function pointer needs an ampersand. + +## Lists + +The work horse of data structures in Lisp is the list. Let us try to find an element that equals 2 and increment the value from Zig. First attempt + +```zig +export fn my_increment_in_list_zig(lst: SCM) SCM { + var lst2 = guile.scm_memv(guile.scm_from_int(2), lst); + return lst2; +} +``` + +```guile +scheme@(guile-user)> (load-extension "libmy" "init_module") +scheme@(guile-user)> (incr_list_zig '(3 4 2 "test")) +$1 = (2 "test") +``` + +Now let's increment with + +```zig +export fn my_increment_in_list_zig(lst: SCM) SCM { + var lst2 = guile.scm_memv(guile.scm_from_int(2), lst); + var lst3 = guile.scm_list_set_x(lst2, 1, guile.scm_from_int(3)); + return lst3; +} +``` + +Now this segfaults. Let's try the debugger with + +``` +env LD_LIBRARY_PATH=. gdb --args guile +``` + +and we get + +``` +Thread 1 "guile" received signal SIGSEGV, Segmentation fault. +0x00007ffff7f02dc5 in scm_to_uint64 () from /gnu/store/qlmpcy5zi84m6dikq3fnx5dz38qpczlc-guile-3.0.8/lib/libguile-3.0.so.1 +``` + +ah, I think the index 1 should be a SCM value. Indeed does the job + +```zig +export fn my_increment_in_list_zig(lst: SCM) SCM { + var lst2 = guile.scm_memv(guile.scm_from_int(2), lst); + lst2 = guile.scm_list_set_x(lst2, guile.scm_from_int(0), guile.scm_from_int(3)); + return lst; // note original is updated! +} +``` + +```guile +(load-extension "libmy" "init_module") +(incr_list_zig (list 3 4 2 "test")) +$1 = (3 4 3 "test") +``` + +## Arrays + +Arrays are continuous in memory. Guile provides a special page on [accessing arrays from C](https://www.gnu.org/software/guile/manual/html_node/Accessing-Arrays-from-C.html). + +It says: Accessing Arrays from C. +For interworking with external C code, Guile provides an API to allow C code to access the elements of a Scheme array. +In particular, for uniform numeric arrays, the API exposes the underlying uniform data as a C array of numbers of the relevant type. +This can be very useful. Note that the C and Zig code need to protect and array when it starts modifying it. And for multithreaded access locking is also required. See above documentation for more. + +We'll want to create an [unboxed vector](https://www.gnu.org/software/guile/docs/docs-2.2/guile-ref/Uniform-Numeric-Vectors.html#Uniform-Numeric-Vectors) of double. This is the scheme version: + +```guile +(use-modules (srfi srfi-4)) +(define v #f64(3.1415 2.71 1.0)) +(f64vector-length v) +$28 = 3 +(f64vector-set! v 1 3.71) +v +$29 = #f64(3.1415 3.71 1.0) +``` + +The zig version: + +```zig +// Increment the second value in the unboxed vector +export fn my_increment_in_f64vector_zig(vec: SCM) SCM { + _ = guile.scm_f64vector_set_x(vec, guile.scm_from_int(1), guile.scm_from_double(3.7)); + return vec; // note original is updated! +} +``` + +does it + +```guile +(define v #f64(3.1415 2.71 1.0)) +(incr_f64vector_zig v) +$1 = #f64(3.1415 3.7 1.0) +``` + +Now in the final step we want to update the value as a raw Zig array. The +C Function: + +```C +const double * scm_f64vector_elements(SCM vec, scm_t_array_handle *handle, size_t *lenp, ssize_t *incp) +``` + +should return a pointer. The Zig translated version is + +```zig +pub extern fn scm_f64vector_elements(uvec: SCM, h: [*c]scm_t_array_handle, lenp: [*c]usize, incp: [*c]isize) [*c]const f64; +``` + +This code unboxes the double vector: + +```zig +export fn my_increment_in_f64vector_zig(vec: SCM) SCM { + // _ = guile.scm_f64vector_set_x(vec, guile.scm_from_int(1), guile.scm_from_double(3.7)); + var handle: guile.scm_t_array_handle = undefined; + var lenp: usize = undefined; + var incp: isize = undefined; + var data = guile.scm_f64vector_elements(vec, &handle, &lenp, &incp); + p("\nZig says {any},{any},{any}\n",.{data[0],data[1],data[2]}); + return vec; // note original is updated! +} +``` + +When you try to mutate it will complain! We need to set the vector to mutable. Replacing +the line with: + +``` +var data = guile.scm_f64vector_writable_elements(vec, &handle, &lenp, &incp); +``` + +gets a complaint: + +``` +Wrong type (expecting mutable f64vector): #f64(3.1415 2.71 1.0) +``` + +Try again by createing a mutable vector. This is the full code + +```zig +export fn my_increment_in_f64vector_zig(vec: SCM) SCM { + // _ = guile.scm_f64vector_set_x(vec, guile.scm_from_int(1), guile.scm_from_double(3.7)); + var handle: guile.scm_t_array_handle = undefined; + var lenp: usize = undefined; + var incp: isize = undefined; + var data = guile.scm_f64vector_writable_elements(vec, &handle, &lenp, &incp); + p("\nZig says {any},{any},{any}\n",.{data[0],data[1],data[2]}); + data[1] += 1.0; + guile.scm_array_handle_release(&handle); + return vec; // note original is updated! +} +``` + +```guile +(load-extension "libmy" "init_module") +(define v (list->f64vector '(3.1415 2.71 1.0))) +(display "Guile says ") +(display v) +(incr_f64vector_zig v) +(display "Guile now says ") +(display v) +``` + +displays + +``` +Guile says #f64(3.1415 2.71 1.0) +Zig says 3.1415e+00,2.71e+00,1.0e+00 +Guile now says #f64(3.1415 3.71 1.0) +``` + +## Sharing a pointer + +If you want to move pointers around, both guile and zig can handle 'foreign pointers'. Zig has *anyopaque and guile has [foreign pointers]( https://www.gnu.org/software/guile/manual/html_node/Foreign-Pointers.html#Foreign-Pointers). |