Adjective Noun

Exportation Exploration

2018-10-26 09:30, Tags: perl

Well it's been another long while since I updated. You know how it is, things in life exist in a hierarchy, and playing with code comes behind family and work. In any case, I'm still working on my combinatorics modules... and as I move closer to an actual release, I've been ruminating on different ways to export functions from my module.

Maybe this is an unpopular opinion, but I have a slight distaste for modules that import functions by default. It's unfortunate that the most Huffmanised (quickest, easiest) way of exporting functions...

unit module Foo

sub bar is export { ... }

... results in polluting the users namespace. I would have hoped that we as a community would have moved to strongly preferring selective imports... but TIMTOWTDI?

A running theme of mine seems to be "let's look at what Python does", so why stop now? Typically, if I import a namespace in Python, I can only use symbols from that namespace by prefixing them with the namespace.

>>> import string
>>> list(string.ascii_lowercase)[:10]
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
>>> ascii_lowercase
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    NameError: name 'ascii_lowercase' is not defined

Alternatively, I can selectively import individual functions, and only those functions are exported, not the namespace they came from...

>>> from string import ascii_lowercase
>>> ascii_lowercase
>>> list(ascii_lowercase)[:10]
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
>>> string.ascii_lowercase
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'string' is not defined

As far as I know, symbols are never imported implicitly. I think this export/import system is one of the things Python got mostly right.

So what can Perl 6'ers start doing right now to do better. If your module is OO-based (ie. it's functions are exported via method calls on an instantiated class) then this doesn't apply... but if your module exports individual functions, I would urge you not to use is export trait, at least not without tags.

There's a couple of options when it comes to enforcing selective imports. The easiest is by tagging all your imports. I don't think that's necessarily what export tags were created for, but as a module writer, it's the certainly the simplest way to avoid polluting the users namespace.

unit module Foo

sub bar is export(:bar) { ... }
sub baz is export(:baz) { ... }

The :ALL tag is implied by the is export trait

The downside is I have to type the function name twice, and make sure I spell it correctly the second time, and remember to rename the tag if I rename the function.

Then, users can selectively import functions like this:

use Foo :bar;

say bar();
# baz() or Foo::baz() not available here

Again, I'm not sure this is the intended use for tags, but as you know, TIMTOWTDI.

In Perl 5, the generally standard way to export functions is to have your package inherit from the core Exporter module, then define a list of functions that are exported by default in the package scoped array @EXPORT, and selective imports in @EXPORT_OK.

package Foo;

use base 'Exporter';
our @EXPORT_OK = qw( bar baz )

sub bar { ... }
sub baz { ... }

This means that enforcing selective imports in Perl 5 requires a minimum of only 2 lines of code, but this also suffers from having to type the function name a second time.

Similar to how @EXPORT_OK was defined, the importing code declares a list of words to the module to selectively import those names...

use Foo qw( bar );

say bar()
# baz() not available, though Foo::baz() is

To get similar import syntax in Perl 6 - in contrast to Perl 5's 2 lines - one has to jump through a few hoops.

my %exports;
module Foo {
    sub bar { ... }
    sub baz { ... }

    %exports = MY::.grep(*.key.starts-with: '&');
}

multi sub EXPORT(*@names) {
    my %imports;
    for @names -> $name {
        unless %exports{"&$name"}:exists {
            die("Unknown name for export: '$name'");
        }
        %imports{"&$name"} := %exports{"&$name"}
    }
    return %imports
}

It's a little esoteric, but I'm creating an %exports hash that contains all symbol names in the Foo namespace that start with & (ie. the names of Callable symbols) and creating a key/value Pair from the name/symbol. Later on in the EXPORT sub, I iterate through any provided names, and if they exist in %exports, then they are added to the %imports hash. It also allows me to fail early if the user tries to import something I don't export.

Despite the verbosity, at least I didn't have to type my function names twice... but this exports all my functions regardless if I want to export them or not! This could be controlled via a trait... but I'll come back to that.

The result is that I can then import individual symbols with a list of names, like so.

use Foo <bar>

say bar();
# baz() or Foo::baz() not available here

As per the comment above, Foo::baz() is also unavailable. In most cases that doesn't matter, and I could just import the symbols I want. However, there were situations in Perl 5 where it was handy to be able to use the fully qualified name, typically when there was a name collision.

In those cases, it would be nice to be able to use the full name, or perhaps import a symbol using a different name. Python users are very used being able to alias symbols on import

>>> from itertools import combinations_with_replacement as cwr
>>> list(cwr('ABC', 2))
[('A', 'A'), ('A', 'B'), ('A', 'C'), ('B', 'B'), ('B', 'C'), ('C', 'C')]

... and given a little modification to my EXPORT function, I can kinda do something similar with Perl 6.

multi sub EXPORT(*@args) {
    my %imports;
    for @args -> $arg {
        my ($name, $symbol) =  $arg ~~ Pair ?? |$arg.kv !! $arg, $arg;
        unless %exports{"&$name"}:exists {
            die("Unknown name for export: '$name'");
        }
        %imports{"&$symbol"} := %exports{"&$name"}
    }
    return %imports
}

Which allows me to do this

use Foo \[ 'bar', baz => 'frob' ];

say bar();
say frob(); # calls the Foo::baz() function

You might have seen functions that allow you to export a symbol of your choice. Here I'm just (ab)using that syntax... but it should be possible to add some sugar around this syntax.

Ultimately, as it stands, I can't say I'm entirely happy with the current state of exporting and selective importing. I think making the is export export names by default was the wrong choice, particularly considering the :DEFAULT tag exists. Ideally, is export would have made the function available for selective import by name. Module writers wishing to pollute their users namespace would then have to write is export(:DEFAULT). To put it another way, Perl 6 should make it easier to do the right thing.

The upshot is that Perl 6 is a young language, and there's still the possibility for course correction. I'd like to see some simpler syntax for making functions exportable (and eventually, alias-able). For starters, a new is exportable that allows symbols to be exported by name (and not default) would go some ways to improving the situation.

I've made some attempts... I can improve my example above (which made all Callable's exportable) by using a trait to mark exportable functions.

multi sub trait_mod:<is>(Routine:D \r, :$exportable!) {
    r.^mixin( role { method is_exportable { True } } )
}

my %exports;
module Foo {
    sub bar is exportable { ... }
    sub baz is exportable { ... }

    %exports = MY::.grep(*.value.?is_exportable);
}

This still requires me to paste in my funky EXPORT sub. I know I can write a module that exports an EXPORT multi, but I don't know how to access the importers PseudoStash to handle the exporting. The closest I got was reaching into CALLER::CALLER::() from the EXPORT sub, which feels wrong.

I truly don't understand the intricacies of exporting enough to know how to do it. As always, it's highly probable that I'm doing things completely wrong, or missing something obvious. Ultimately what I'd like is something like this...

# Exportable by 'foo'
sub foo is exportable { ... }

# Exportable by 'bar' and the :b tag
sub bar is exportable(:b) { ... }

# Exportable by 'baz' and the :b tag
sub baz is exportable(:b) { ... }

# Is not exportable
sub qux { ... }

The beauty is that is would practically be as easy as is export, doesn't require typing the function name twice, and would arguable provide a better experience for both module writers, and end users. I suspect this could possibly be done in the module space, but I think eventually having something like this in core would encourage it's usage.

If you know how to write a trait like this, I'd be interested on your input. I'm also interested in other users thoughts in general on Perl 6's current export functionality. Comment on Reddit.

Update!