Tag python

Well it's been another long while since I updated. You know how it is, things in life exist in a hierarchy, and playing with code comes behind family and work. In any case, I'm still working on my combinatorics modules... and as I move closer to an actual release, I've been ruminating on different ways to export functions from my module.

Maybe this is an unpopular opinion, but I have a slight distaste for modules that import functions by default. It's unfortunate that the most Huffmanised (quickest, easiest) way of exporting functions...

unit module Foo

sub bar is export { ... }

... results in polluting the users namespace. I would have hoped that we as a community would have moved to strongly preferring selective imports... but TIMTOWTDI?

A running theme of mine seems to be "let's look at what Python does", so why stop now? Typically, if I import a namespace in Python, I can only use symbols from that namespace by prefixing them with the namespace.

>>> import string
>>> list(string.ascii_lowercase)[:10]
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
>>> ascii_lowercase
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ascii_lowercase' is not defined

Alternatively, I can selectively import individual functions, and only those functions are exported, not the namespace they came from...

>>> from string import ascii_lowercase
>>> ascii_lowercase
>>> list(ascii_lowercase)[:10]
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j']
>>> string.ascii_lowercase
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'string' is not defined

As far as I know, symbols are never imported implicitly. I think this export/import system is one of the things Python got mostly right.

So what can Rakoons start doing right now to do better. If your module is OO-based (ie. it's functions are exported via method calls on an instantiated class) then this doesn't apply... but if your module exports individual functions, I would urge you not to use is export trait, at least not without tags.

There's a couple of options when it comes to enforcing selective imports. The easiest is by tagging all your imports. I don't think that's necessarily what export tags were created for, but as a module writer, it's the certainly the simplest way to avoid polluting the users namespace.

unit module Foo

sub bar is export(:bar) { ... }
sub baz is export(:baz) { ... }

The :ALL tag is implied by the is export trait

The downside is I have to type the function name twice, and make sure I spell it correctly the second time, and remember to rename the tag if I rename the function.

Then, users can selectively import functions like this:

use Foo :bar;

say bar();
# baz() or Foo::baz() not available here

Again, I'm not sure this is the intended use for tags, but as you know, TIMTOWTDI.

In Perl, the generally standard way to export functions is to have your package inherit from the core Exporter module, then define a list of functions that are exported by default in the package scoped array @EXPORT, and selective imports in @EXPORT_OK.

package Foo;

use parent 'Exporter';
our @EXPORT_OK = qw( bar baz )

sub bar { ... }
sub baz { ... }

This means that enforcing selective imports in Perl requires a minimum of only 2 lines of code, but this also suffers from having to type the function name a second time.

Similar to how @EXPORT_OK was defined, the importing code declares a list of words to the module to selectively import those names...

use Foo qw( bar );

say bar()
# baz() not available, though Foo::baz() is

To get similar import syntax in Raku - in contrast to Perl's 2 lines - one has to jump through a few hoops.

my %exports;
module Foo {
    sub bar { ... }
    sub baz { ... }

    %exports = MY::.grep(*.key.starts-with: '&');
}

multi sub EXPORT(*@names) {
    my %imports;
    for @names -> $name {
        unless %exports{"&$name"}:exists {
            die("Unknown name for export: '$name'");
        }
        %imports{"&$name"} := %exports{"&$name"}
    }
    return %imports
}

It's a little esoteric, but I'm creating an %exports hash that contains all symbol names in the Foo namespace that start with & (ie. the names of Callable symbols) and creating a key/value Pair from the name/symbol. Later on in the EXPORT sub, I iterate through any provided names, and if they exist in %exports, then they are added to the %imports hash. It also allows me to fail early if the user tries to import something I don't export.

Despite the verbosity, at least I didn't have to type my function names twice... but this exports all my functions regardless if I want to export them or not! This could be controlled via a trait... but I'll come back to that.

The result is that I can then import individual symbols with a list of names, like so.

use Foo <bar>

say bar();
# baz() or Foo::baz() not available here

As per the comment above, Foo::baz() is also unavailable. In most cases that doesn't matter, and I could just import the symbols I want. However, there were situations in Perl where it was handy to be able to use the fully qualified name, typically when there was a name collision.

In those cases, it would be nice to be able to use the full name, or perhaps import a symbol using a different name. Python users are very used being able to alias symbols on import

>>> from itertools import combinations_with_replacement as cwr
>>> list(cwr('ABC', 2))
[('A', 'A'), ('A', 'B'), ('A', 'C'), ('B', 'B'), ('B', 'C'), ('C', 'C')]

... and given a little modification to my EXPORT function, I can kinda do something similar with Raku.

multi sub EXPORT(*@args) {
    my %imports;
    for @args -> $arg {
        my ($name, $symbol) =  $arg ~~ Pair ?? |$arg.kv !! $arg, $arg;
        unless %exports{"&$name"}:exists {
            die("Unknown name for export: '$name'");
        }
        %imports{"&$symbol"} := %exports{"&$name"}
    }
    return %imports
}

Which allows me to do this

use Foo \[ 'bar', baz => 'frob' ];

say bar();
say frob(); # calls the Foo::baz() function

You might have seen functions that allow you to export a symbol of your choice. Here I'm just (ab)using that syntax... but it should be possible to add some sugar around this syntax.

Ultimately, as it stands, I can't say I'm entirely happy with the current state of exporting and selective importing. I think making the is export export names by default was the wrong choice, particularly considering the :DEFAULT tag exists. Ideally, is export would have made the function available for selective import by name. Module writers wishing to pollute their users namespace would then have to write is export(:DEFAULT). To put it another way, Raku should make it easier to do the right thing.

The upshot is that Raku is a young language, and there's still the possibility for course correction. I'd like to see some simpler syntax for making functions exportable (and eventually, alias-able). For starters, a new is exportable that allows symbols to be exported by name (and not default) would go some ways to improving the situation.

I've made some attempts... I can improve my example above (which made all Callable's exportable) by using a trait to mark exportable functions.

multi sub trait_mod:<is>(Routine:D \r, :$exportable!) {
    r.^mixin( role { method is_exportable { True } } )
}

my %exports;
module Foo {
    sub bar is exportable { ... }
    sub baz is exportable { ... }

    %exports = MY::.grep(*.value.?is_exportable);
}

This still requires me to paste in my funky EXPORT sub. I know I can write a module that exports an EXPORT multi, but I don't know how to access the importers PseudoStash to handle the exporting. The closest I got was reaching into CALLER::CALLER::() from the EXPORT sub, which feels wrong.

I truly don't understand the intricacies of exporting enough to know how to do it. As always, it's highly probable that I'm doing things completely wrong, or missing something obvious. Ultimately what I'd like is something like this...

# Exportable by 'foo'
sub foo is exportable { ... }

# Exportable by 'bar' and the :b tag
sub bar is exportable(:b) { ... }

# Exportable by 'baz' and the :b tag
sub baz is exportable(:b) { ... }

# Is not exportable
sub qux { ... }

The beauty is that is would practically be as easy as is export, doesn't require typing the function name twice, and would arguable provide a better experience for both module writers, and end users. I suspect this could possibly be done in the module space, but I think eventually having something like this in core would encourage it's usage.

If you know how to write a trait like this, I'd be interested on your input. I'm also interested in other users thoughts in general on Raku's current export functionality. Comment on Reddit.

Update!

In my last post, I briefly touched on the concept of wrapping functions. I also learned that they are similar to decorators in Python. Apart from one time I used the @property decorator in a Python class to make some attributes read-only, I didn't really know what they were. I just figured it was some weird Python syntax. I've since learned a little be more and played around with them in Python, Perl, and Raku.

A decorator is a function that takes another function as it's argument, and typically does something "around" that function, which is why it's also referred to "wrapping" a function. A decorator can't change what the wrapped function does internally, but it can can run code before or after calling that function, or not call it at all.

I may use the words 'wrapper' and 'decorator' interchangeably, by which I mean 'a function that wraps another function'

There are some quintessential applications for decorators; the main ones being caching, logging, and timing of functions. As a reference point, here is a timing decorator in Python 3.6.

import time

def timed(func):
    name = func.__name__
    def wrapped(*args):
        start = time.time()
        res = func(*args)
        print(f"Run time for function '{name}' was {time.time() - start:f}")
        return res
    return wrapped

@timed
def costly(n):
    time.sleep(n);
    return 'Have a string'

x = costly(3)
# OUTPUT: Run time for function 'costly' was 3.02231

print(x)
# OUTPUT: Have a string

In the above example, I grab the name of the function, then create the wrapper function. My wrapper kicks off a timer, then runs the original (decorated) function and assigns the result to a variable res. I then stop the time, print out the stats then return the result.

So without further ado, or much explanation, here's a Raku sub trait that achieves the same result.

multi sub trait_mod:<is>(Routine $func, :$timed) {
    $func.wrap({
        my $start = now;
        my $res = callsame;
        note "Run time for function '{$func.name}' was {now - $start}";
        $res;
    })
}

sub costly($n) is timed {
    sleep($n);
    return 'Have a string';
}

my $x = costly(3);
# OUTPUT: Run time for function 'costly' was 3.0030732

say $x;
# OUTPUT: Have a string

Most of this should be fairly obvious, except maybe callsame, which I covered in my last post... but if you need a refresher, it tells the dispatcher to call the same function that was just called. Also, note the note function, which is exactly like say except that it outputs to STDERR.

Traits wrap a function at (some time around) compile time, but sometimes you might want to wrap a function at runtime, or rather... You might want to decide whether you want to wrap a function at runtime; which functions you want wrapped with what; and when.

Take debugging for example. It would be trivial to create a trait that reports to STDERR when a function has been called, and with what arguments... but adding and removing a trait everytime you want to debug - especially on multiple functions - can get a little unwieldy.

Typically when you debug with print statements (we all do it!) you might manage your programs DEBUG mode via a global variable. At runtime you can inspect the variable and wrap your desired functions accordingly.

constant DEBUG = True;

sub foo($n) {
    return $n × $n;
}

&foo.wrap(&debug) if DEBUG;

my $x = foo(42);

sub debug(|c) {
    my &func = nextcallee;
    my $res = func(|c);
    note "Calling '{&func.name}' with args {c.perl} returned: {$res.perl}";
    $res;
}

# STDERR: Calling 'foo' with args \(42) returned: 1764

The .wrap() method actually returns something called a WrapHandle, which is handy if you want to be able to unwrap your function at any point. It also means you can decide which wrappers get removed.

Perhaps you have a logging wrapper, something that performs a similar role as the debug wrapper, but instead punts the information to your logger of choice, or maybe just a text file. You want to disable the debugger at some point, but keep logging.

my $wh-logger = &foo.wrap(&logger);

my $wh-debug = &foo.wrap(&debug) if DEBUG;

my $x = foo(42);

# Success threshold, debugging is no longer required
&foo.unwrap($wh-debug) if DEBUG;

# Calls to 'foo' still hit the logger
my $y = foo(19);

The beauty of wrappers is your wrapped functions don't have to know they are being wrapped. They can concern themselves with their core purpose. Additionally they only need to be wrapped once, instead of, for example, manually calling your logger function all over the place.

So these decorator things are nice, but I still use Perl quite a lot, and I wanted to know if there was a way to wrap functions in Perl with the same syntactic niceness that trait's provide. What I eventually landed on was attributes, and Attribute::Handlers.

Like trait mods (and Python decorators), attributes are added at the point of your function declarations. Attribute::Handles just makes working with them a little easier. Here's the example from up top, implemented with Perl.

use v5.26;
use warnings; no warnings 'redefine';
use experimental 'signatures';
use Time::HR 'gethrtime';
use Attribute::Handlers;

sub Timed( $pkg, $sym, $code, @ ) :ATTR {
    my $func = substr( ${$sym}, length($pkg) + 3 );
    *$sym = sub (@args) {
        my $start = gethrtime();
        my $res   = $code->(@args);
        my $time  = ( gethrtime() - $start ) / 1_000_000_000;
        say {*STDERR} "Run time for function '$func' was $time";
        return $res;
    }
}

sub costly($n) :Timed {
    sleep($n);
    return 'Have a string';
}

my $x = costly(3);
# STDERR: Run time for function 'costly' was 3.001124

say $x;
# OUTPUT: Have a string

A few caveats to note about Perl... This is classed as redefining a symbol that already exists, and Perl will give a warning that the wrapped function has been redefined, so I disabled that warning. It will also give a warning if you give your attribute an all-lowercase name, as lowercase attributes are reserved for possible future use. Also, as far as I found, the only way to import attributes from a module is to declare them into the UNIVERSAL namespace, for example: UNIVERSAL::Timed, which technically means you don't even need to export them from your module, so... Yay, I guess.

One final note. It's curious to me that I'm talking about "wrapping" and "decorating" this close to December, when those words typically mean something else entirely. Happy holidays!

Reddit comments

This is Part 7 in a series. You can start at the Intro here

This will be the last entry in this series on using Matplotlib in Perl 6. As I said in the Intro, I was just playing around, found it interesting, and decided to write about it. It also provided a handy excuse to talk about some of the nice features in Perl 6. I never had any intention of creating a full-featured wrapper for Matplotlib, but my hope is that - through this series - I have shown how entirely possibly it is to use Python modules in a transparent manner in Perl 6 via a wrapper.

Last time I took way too long to get to one graph, so I wanna show a few this time. I've covered enough ground that my Matplotlib wrapper can handle basic graphs nicely. Let me just bask in the warmth of how I can churn out a few simple plots without having to fix anything. This time I'm eyeing a few scatter and polar plots. First up is is scatter_demo.

use Matplotlib;
my $plt = Matplotlib::Plot.new;

constant N = 50;
my $x = rand xx N;
my $y = rand xx N;
my $c = rand xx N;
my $s = ( rand xx N ).map( π × ( 15 × * )² );

$plt.scatter( $x, $y, :$c, :$s, :alpha(0.5) );
$plt.show();

Which produces a random variation on this

Jumping straight into another scatter plot, this one is polar_scatter_demo. As you might glean from the name, this one uses a polar projection, as well as the always pleasant-looking HSV color map.

use Matplotlib;

my $plt = Matplotlib::Plot.new;

constant N = 150;
my $r      = ( rand xx N ).map( * × 2 ).List;
my $theta  = ( rand xx N ).map( * × τ );

my $s = $r.map( 200 × *² );
my $c = $theta;

my $ax = $plt.subplot( 111, :projection<polar> );
$ax.scatter( $theta, $r, :$c, :$s, :cmap<hsv>, :alpha(0.75) );

$plt.show()

Which gives me a random variation on this

Nice stuff here.
Using the great list repetition operator covered in Part 2, simple passing of named arguments, mapping an operation over a list of items using real unicode math symbols... and they're nice looking plots too!

But this wouldn't a complete post if I didn't get a curve ball, which I will encounter in the next example, polar_bar_demo. Here's the Python.

import numpy as np
import matplotlib.pyplot as plt

# Compute pie slices
N = 20
theta = np.linspace(0.0, 2 * np.pi, N, endpoint=False)
radii = 10 * np.random.rand(N)
width = np.pi / 4 * np.random.rand(N)

ax = plt.subplot(111, projection='polar')
bars = ax.bar(theta, radii, width=width, bottom=0.0)

# Use custom colors and opacity
for r, bar in zip(radii, bars):
    bar.set_facecolor(plt.cm.viridis(r / 10.))
    bar.set_alpha(0.5)

plt.show()

There's nothing here that looks like it shouldn't work. Last time I compared this series to a predictable movie franchise, but this time there's a plot twist. It's not a method call that threw me this time. Don't get me wrong, I still need to handle that .cm method, so I'll cover that first. Taking a scene from the Part 6 script, I implemented another anonymous class to handle the call to plt.cm

class Matplotlib::Plot {
    method cm {
        class {
            method FALLBACK($name, $idx) {
                $py.run("matplotlib.pyplot.cm.{$name}($idx)", :eval);
            }
        }.new();
    }
    # stuff ...
}

This run looks a little different due to the use of the curlies around the $name variable. This is because had I tried to eval "matplotlib.pyplot.cm.$name(...)" Perl will interpret the $name() as a Perl function

my $f = *.uc;
say $f('hello');    # Result: 'HELLO'

So I need a way to prevent it from being interpreted as a function call. I initially used a a sprintf style statement, but on freenode #perl6, timotimo suggested using curlies instead. This is one of my favourite syntax features of Perl 6 so I can't believe I didn't think of it, and haven't talked about it. Essentially, in Perl 6, any time you see a pair of curlies, it's a code block... even inside a string!

my ( $x, $y ) = ( 100, 3 );
say "$x ÷ $y is { floor( $x ÷ $y ) }, with a remainder of { $x % $y }";
# OUTPUT: 100 ÷ 3 is 33, with a remainder of 1

You can even nest these contructs without Perl getting confused. I should also mention that anytime you see curlies, you should also think: new lexical scope. Any lexical variables declared inside a code block are not visible outside of that code block.

So now the .cm method is working as expected, but that's not the curve ball. No, the real problem occurs later on with this line

my $bars = $ax.bar( $theta, $@radii, :$width, :bottom(0.0) );

for $bars Z @radii -> ( $bar, $r ) { ... }

Perl has a zip() function, but I prefer the versatility of the Z infix operator, because it can also be paired with any other infix operator for easy map-like operations

> (1, 2, 3) Z+ (4, 5, 6)
(5 7 9)
> ( 'A'..'Z' Z~ 1..26 )[^6]
(A1 B2 C3 D4 E5 F6)

However - as far as Perl is concerned - the $bars variable I get back from $ax.bar() is not iterable ie... It is another Inline::Python::PythonObject. I tried to convert it to something iterable via Inline::Python but nothing was working. In desperation, I checked to see if there were any native Python methods on the object that could use to get at the list of things inside.

I modified the python version to run print(dir(bars)) after I got the bars variable and saw a few of the usual suspects: __str__, __repr__, __len__, and others.
Again, maybe there's a better way I could have got at the elements in this thing, but ultimately what worked for me was __getslice__(0, N). This gives me a list slice from the first (0-th) element, up to the N-th element; I could then zip those with the elements in radii. Here's my working code

use Numpl;
use Matplotlib;

my $np   = Numpl.new;
my $plt  = Matplotlib::Plot.new;

# Compute pie slices
my constant N = 20;

my $theta = $np.linspace( 0.0, τ, N, :endpoint(False) );

my @radii = ( rand xx N ).map( * × 10 ).map( *.Num );
my $width = ( rand xx N ).map( π ÷ 4 × * );

my $ax = $plt.subplot( 111, :projection<polar> );
my $bars = $ax.bar( $theta, $@radii, :$width, :bottom(0.0) );

for $bars.__getslice__(0, N) Z @radii -> ( $bar , $r ) {
    my $rgb = $plt.cm.viridis($r ÷ 10);
    $bar.set_facecolor($rgb);
    $bar.set_alpha(0.5);
}

$plt.show()

The only other change I made here was using τ instead of π × 2 because why not. Here's the final plot I will do from the gallery.

But it doesn't feel right to end on an example plot like this. I haven't yet done any plots outside of what exists in the Matplotlib gallery.

In keeping with a Perl 6 kind of flavour, there is curve known as the butterfly curve. The maths behind it is a little beyond me... I tried and failed to get the polar equation in that wikipidia article working, but eventually gave up and got the parametric equation working this way.

use Numpl;
use Matplotlib;

my $np  = Numpl.new;
my $plt = Matplotlib::Plot.new;

my @v = (0, 0.02 ... 72);

sub comp($n) { e ** cos($n) - 2 × cos(4 × $n) - sin($n ÷ 12)⁵ }

my $x = @v.map(-> $t { sin($t) × comp($t) });
my $y = @v.map(-> $t { cos($t) × comp($t) });

$plt.plot( $x, $y );

$plt.show();

Nothing special code-wise here. Just creating a range of values, mapping those values over a given computation, then plotting them.

This looks nice enough, but I couldn't leave it without experimenting with colour. I imagined I could change the colour of the curve based on location, but unless I'm mistaken, it seems Matplotlib doesn't have that functionality. You can, however, mimic that behaviour with a scatter plot and lotsa points... So, I changed a few lines, played with the numbers a bit, and ended up with this

my @v = (0, 0.005 ... 72); # more points!

# stuff...

# $plt.plot($x, $y); # don't need this anymore
my $m = 2;
my $c = ([Z] $x, $y).map(-> ($a, $b) { cos($a × $m) + sin($b × $m) });
$plt.scatter( $x, $y, :$c );

$plt.show();

And now we have this beautifully colourful butterfly curve.

So this is the end of my exploratory series on using Matplotlib in Perl 6. If you're hankering for more, moritz has been covering similar ground over on his blog recently. He's also more knowledgeable than me on all things Perl 6.

I feel confident - and I hope you do too - that given the need to do some plotting, I would reach for Matplotlib without hesitation... even if only for the fact that I'm now more familiar with it than I am with any other plotting library.

If you'd like to play around with my Matplotlib or Numpl modules as they existed after this series, I've put them up in a gist. You're welcome to fork them and do whatever you will.

Thanks for coming on this journey with me.

This is Part 6 in a series. You can start at the Intro here.

By this stage this series is beginning to resemble a bad movie franchise. We're back again for another round, featuring the same central characters and themes. The story is essentially the same each time, just the obstacles to overcome change a little. Which is to say, I'm about ready to wrap this series up, but I ran into a few more interesting things so may as well talk about them.

The next plot I tried was another style sheet example, this time it's plot_dark_background. Here's the Python

import numpy as np
import matplotlib.pyplot as plt

plt.style.use('dark_background')

fig, ax = plt.subplots()

L = 6
x = np.linspace(0, L)
ncolors = len(plt.rcParams['axes.prop_cycle'])
shift = np.linspace(0, L, ncolors, endpoint=False)
for s in shift:
    ax.plot(x, np.sin(x + s), 'o-')
ax.set_xlabel('x-axis')
ax.set_ylabel('y-axis')
ax.set_title("'dark_background' style sheet")

plt.show()

Again, most of this stuff is (or looks) straight-forward, but there's a couple new curve balls. First up I want to tackle the easier problem, which is that linspace is being used here in ways my function doesn't handle. At one point, linspace is called with only 2 arguments, and the other time it is called with an endpoint keyword.

Before I even look at that, I want to quickly fix another issue I ran into when playing with my linspace function. I found that when using an irrational number (such as π) as the $end number, something unexpected happened

> Numpl.linspace(0, 3.14159265358979, 7)[5..7]
(2.6179938779914917 3.14159265358979 Nil)
> Numpl.linspace(0, pi, 7)[5..7]
(2.61799387799149 3.14159265358979 3.66519142918809)

I've asked for a Sequence from 0 to π, divided into 7 linear steps. When given a number literal, it works as expected, where the 7th item (index 6) is "pi" and the 8th item is non-existent, ie. Nil. However, when using the built-in π constant, it doesn't consider the 7th item exactly equal to π and the sequence keeps generating infinitely. To fix this, rather than check for equality I will check for "approximate equality" using the =~= operator... but what does "approximate" mean, anyway?

Perl has a global TOLERANCE variable, which by default is 1e-15, or 0.000000000000001. This global can be modified lexically if I want to compare values with a lower tolerance, but the default value works fine for me.

> pi
3.14159265358979
> pi == 3.14159265358979
False
> pi =~= 3.14159265358979
True

So now that's sorted, I moved on to updating the function to act more like the numpy version. I tested in Python and found that when called with 2 arguments, numpy.linspace will default to 50 steps. I modified the function prototype so that $steps is an optional positional just by appending a ? to the parameter. In the multi-sub, I declared $steps to have a default value of 50, and added a named $endpoint parameter... I just need to do something if $endpoint is False. Maybe there's a fancier way to do this, but I just did a simple if-condition. The end result is this

proto method linspace(Numeric $start, Numeric $end, Int $steps?) { * }
multi method linspace($start, $end, 0 ) { Empty  }
multi method linspace($start, $end, 1 ) { $start }
multi method linspace($start, $end, $steps = 50, :$endpoint = True) {
    if $endpoint {
        my $step = ( $end - $start ) ÷ ( $steps - 1 );
        return $start, * + $step ... * =~= $end
    }
    else {
        my $step = ( $end - $start ) ÷ $steps;
        return $start, * + $step ...^ * =~= $end
    }
}

I've spent a lot of time just talking about linspace and I've still got a lot of ground to cover, so I'll just move right into this:

ncolors = plt.rcParams['axes.prop_cycle']

I'm familiar enough with Python to recognise dictionary syntax when I see it.

A dictionary is what Python calls it's associative array type, which in Perl is called a hash. I choose to use all three terms interchangeably.

It seems that rcParams is a class attribute that returns an associative array which can then be subscripted by key. This is actually a unique problem which can't be fully solved using all my old "tricks", and there are a few reasons for that. Sure, I could use Inline::Python::run to return the dictionary... I found that by wrapping the call to matplotlib.pyplot.rcParams with dict in the Python, Inline::Python dutifully returned a Perl hash.

class Matplotlib::Plot {
    # stuff
    method rcParams {
        $py.run("dict(matplotlib.pyplot.rcParams)", :eval)
    }
}

Then I could call that method from my main program, subscript a random key and all seemed ok. (I'm going to use another key axes.xmargin for demonstrative purposes)

say $plt.rcParams<axes.xmargin>    # Result: 0.05 

But! I imagine some other plots might involve changing some of those "params", so how to overcome that? I need my wrapper to be aware if I'm trying to assign a value to that key and run the appropriate Python code. Perl 6 does indeed provide various special methods that allow you to implement an object that behaves like a hash or a list, for example

$obj.foo('bar'); # Calls 'foo' method with argument 'bar'
$obj[2];         # Calls AT-POS method with argument 2 
$obj{'baz'};     # Calls AT-KEY method with argument 'baz'

So all I need to do is implement the relevant methods in my class, which in this case is AT-KEY and ASSIGN-KEY. You can probably guess the latter is called when calling the object like a hash and assigning something to it. Rather than define a "fully-grown" class just to get this functionality, I implemented an anonymous class.

method rcParams {
    class {
        method AT-KEY($key) {
            $py.run("matplotlib.pyplot.rcParams['$key']", :eval);
        }
        method ASSIGN-KEY($key, $value) {
            $py.run("matplotlib.pyplot.rcParams['$key'] = $value");
        }
    }.new();
}

This is just like my style method on Matplotlib::Plot that instantiates a Matplotlib::Plot::Style class, except I've defined the class inside the method anonymously.
A class by any other name would smell as sweet. Now I can do this

$plt.rcParams<axes.xmargin> = 0.08;
say $plt.rcParams<axes.xmargin>    # Result: 0.08

The code inside $py.run() is essentially a string eval, which means, if I pass it a string it will fail because Python will evaluate the $value variable as a bare word. I need to wrap it in quotes, but only if it's a string. This is easily worked around with multiple dispatch. You also might have also noticed I'm not using the :eval option inside ASSIGN-KEY. When I tried to use :eval I got errors; I don't think Python assignments return anything. Typically in Perl, they return the assignment, so I'm going to manually return the provided value to make it more Perlish.

method rcParams {
    class {
        method AT-KEY($key) { ... } # Same as above
        multi method ASSIGN-KEY($key, Str $value) {
            $py.run("matplotlib.pyplot.rcParams['$key'] = '$value'");
            return $value;
        }
        multi method ASSIGN-KEY($key, $value) {
            $py.run("matplotlib.pyplot.rcParams['$key'] = $value");
            return $value;
        }
    }.new();
}

I'd be done here if it weren't for one minor issue. The actual key want from the dictionary - axes.prop_cycle - is itself a dictionary. It seems Inline::Python does not convert this to a Perl hash, and what I get back is an Inline::Python::PythonObject. I hacked away a bit and settled on the first thing that worked.

method rcParams {
    class {
        multi method AT-KEY($key) {
            $py.run("matplotlib.pyplot.rcParams['$key']", :eval);
        }
        multi method AT-KEY('axes.prop_cycle') {
            $py.run(
                "list(matplotlib.pyplot.rcParams['$key'])", :eval
            ).map(|*.values);
        }
        # stuff...
    }
}

Essentially, when I want that particular key, I run exactly the same Python code except that I wrap it in list(). Oddly, wrapping it in dict() didn't help. What I got back was a list of key/value pairs, except, they were all the same key...

[{color => #1f77b4} {color => #ff7f0e} {color => #2ca02c} {color => #d62728}
 {color => #9467bd} {color => #8c564b} {color => #e377c2} {color => #7f7f7f}
 {color => #bcbd22} {color => #17becf}]

So I just run the list through a Perl map and just extract the values. The | there is the short hand syntax for a Slip, a type that "slips" an item into the surrounding list, kind of like the * in Python 3. This is so I get a single list, rather than a list-of-lists.

Boy would you look at the time! I try to keep these posts relatively short but there was so much to talk about. We can finally get to the crescendo of this entry in the franchise... Here's the Perl

use Matplotlib;
use Numpl;

my $plt = Matplotlib::Plot.new;
my $np  = Numpl.new;

$plt.style.use('dark_background');

my ( $fig, $ax ) = $plt.subplots();

constant L = 6;
my @x = $np.linspace(0, L);

my $ncolors = $plt.rcParams<axes.prop_cycle>;

my @shift = $np.linspace(0, L, +$ncolors, :endpoint(False) );

for @shift -> $s {
    my $wave = @x.map(-> $x { sin($x + $s) });
    $ax.plot($@x, $wave, 'o-' );
}

$ax.set_xlabel('x-axis');
$ax.set_ylabel('y-axis');
$ax.set_title("'dark_background' style sheet");

$plt.show();

The only thing here that I don't think I've covered before is that +$ncolors in my second call to linspace. Here, $ncolors is a list, but I can't just pass it to linspace, which expects an Int as it's 3rd positional argument... so I manually coerce it to an Int by prefixing it with +.

Hey, wait a minute! I didn't even need the values in rcParams<axes.prop_cycle> at all! I just need it's number of elements. I could just as easily replace +$ncolors with a literal 10 and it would do the same thing. Still, I don't consider all that work a waste. It may well come in handy for me later... or maybe you. Anyways, here's the resulting graph

Well that's nice, isn't it... Even if it took a while to get here. As stated at the top of this post, I am almost ready to wrap up the series, but want to cover a few more things.

To be continued...

This is Part 5 in a series. You can start at the Intro here.

All the graphs I've done so far are using the standard Matplotlib style. In some of them, I might have specified some colour here or there, but Matplotlib also provides "style sheets" for changing the overall look and tone of a graph. That's what's happening here.

import matplotlib.pyplot as plt
plt.style.use('bmh')

It looks like style is another sub-package to matplotlib.pyplot, which then calls a method called use. In a similar vein to how I solved my issues with mlab, I'll need to create a new Matplotlib::Plot::Style class and also make it possible to instantiate that class from a pyplot object. I figured I could sort that out later... First I need to actually get the graph working.

First things first, I need to define a plot_beta_hist function that's called several times later.

from numpy.random import beta
# stuff...
def plot_beta_hist(ax, a, b):
    ax.hist(beta(a, b, size=10000), histtype="stepfilled",
            bins=25, alpha=0.8, normed=True)

We've got another numpy.random function here. This time it's beta which is... err, something to do with... um, probability distributions... parametrized by, uh... shape parameters... *blink*
Look, Wikipedia's over there, you can read all about it. I checked RosettaCode and found nothing so I tried to look up an algorithm, but the sites I found all used a lot of fancy symbols. This stuff was way over my head.

I really wanted to avoid making another wrapper for numpy, so I turned to my faithful stead, Perl 5. I found Math::Random on MetaCPAN which has a random_beta function!

For those unaware, Perl 6 is a different language to Perl 5 with a completely different runtime, so you can't natively use Perl 5 modules. However - just like Inline::Python - there is also an Inline::Perl5 module in the Perl 6 ecosystem. It's also made by Stefan Seifert (thanks Stefan!), and it's also much more developed and stable... to the point where you can import pretty much import any Perl 5 module and just use it as if it were a native module. No need for creating wrappers. It's as easy as this.

use Math::Random:from<Perl5> <random_beta>;

I don't even need to say use Inline::Perl5, I just use Math::Random and tell Perl 6 that this it's a Perl 5 module. Inline::Perl5 is automatically imported and does the rest. In this particular case, I'm also importing the random_beta function I need... And that's it!
Oh, and I'm also making use of that < word quoting > syntax covered way back in Part 2.

Perl has always been known as a great glue language, and here we see Perl 6 not simply spawning processes and parsing the output, but actually communicating directly with Python and Perl 5 and combining them to form one big delicious langwich! It really is a thing of beauty.

I converted the rest of the code (sans style sheet) and ran it. A wild graph appears! What the...

This is why you should read documentation. In the original Python, numpy.random.beta takes it's α and β values first followed by the size. However Math::Random::random_beta takes a size value first, followed by α and β. Serves me right. I swapped the arguments the right way around. Here's the final code (again, sans style sheet).

use Matplotlib;
use Math::Random:from<Perl5> <random_beta>;

my $plt = Matplotlib::Plot.new;

#$plt.style.use('bmh');

sub plot_beta_hist($ax, $a, $b) {
    my $beta = random_beta( 10000, $a, $b );
    $ax.hist(
        $beta,
        :histtype('stepfilled'),
        :bins(25), :alpha(0.8),
        :normed(True)
    );
}

my ( $fig, $ax ) = $plt.subplots();
plot_beta_hist( $ax, 10, 10 );
plot_beta_hist( $ax,  4, 12 );
plot_beta_hist( $ax, 50, 12 );
plot_beta_hist( $ax,  6, 55 );
$ax.set_title("'bmh' style sheet");

$plt.show()

The only real difference in my plot_beta_hist function is I've assigned the result (a list) to a scalar variable $beta before passing it to the hist method. On Reddit, raiph and I discussed other ways to pass scalar variables to Python.

You could wrap the call to random_beta in a scalar container

$ax.hist(
    $( random_beta( $a, $b, 10000 ) ),
    # other args...
);

or you could assign it to the anonymous scalar so it gets evaluated to a scalar before being passed to the function

$ax.hist(
    $ = random_beta( $a, $b, 10000 ),
    # other args...
);

There's more ways, and if you're playing along at home the choice is yours, but I like the readability of mine. Whichever one is used, the result is finally something that looks close to the example in the gallery.

Alright, I got a graph, but here's a quick life lesson. Just because someone presents data in a pretty graph doesn't mean it's true. This is a prime example of a misleading graph. It clearly says atop the graph it's using a 'bmh' style sheet. Lies! So, lets get back to that.

Similar to what I had done in Part 4, I created another class wrapping matplotlib.pyplot.style and also made it possible to instantiate this class when calling the style method on a Matplotlib::Plot object.

class Matplotlib::Plot::Style {
    method FALLBACK($name, |c) {
        $py.call('matplotlib.pyplot.style', $name, |c)
    }
}

class Matplotlib::Plot {
    method style {
        Matplotlib::Plot::Style.new
    }
    # stuff ...
}

Problem solved, right? Nope... I got an error back from Python saying name 'use' does not exist which is typically something Python says when you haven't imported something before trying to use it, but that didn't look like the case here. Additionally, while Inline::Python is kind of amazing for what it does, it's not really documented as it's still under development. I was really only aware of 2 methods: call and run.

The call method is literally for calling class methods. It takes 2 mandatory arguments: a package, and a method name; optional arguments can also be called. The run method is more for running arbitrary bits (ie, not class methods). It just takes one mandatory argument: some python code as a string. I managed to get it working using run instead. Again, my Python ignorance is showing here, but it seems style.use is not a method call, but probably an attribute or something.

I added a use method to my Matplotlib::Plot::Style class and called run with the relevant python code. While researching pyplot.style I found it also has an available attribute that returns a list of all the style sheets! That could come in handy, so I added it as well. For both these uses of run, I'm passing the eval option, which tells Inline:Python to return values from the evaluated python code.

class Matplotlib::Plot::Style {
    method use($name) {
        $py.run("matplotlib.pyplot.style.use('$name')", :eval)
    }
    method available {
        $py.run("matplotlib.pyplot.style.available", :eval)
    }
    method FALLBACK($name, |c) {
        say $name;
        $py.call('matplotlib.pyplot.style', $name, |c)
    }
}

The python code in is in double-quotes so that variables get interpolated, but I also had to put single-quotes around the argument for it to work... but it did work, and that's all I really care about. Look at this thing

So there you have it... One more thing! Remember I can get a list available styles? I can use this to pick a random style to display each time.

my $name = $plt.style.available.pick;
$plt.style.use($name);
# stuff...
$ax.set_title("'$name' style sheet");

So that's part 5. I didn't think I would write this much, but I'm still having fun here.

To be continued...

This is Part 4 in a series. You can start at the Intro here.

The last post of this series was kind of more about replicating numpy's linspace function in Perl 6 than it was about testing the limits of this matplotlib wrapper. In this part I am going to hit one of those limits and encounter a minor short-coming of the wrapper as it currently exists.

If you're following along, I had decided to take a shot at one of the histograms and jumped into the first one named histogram_demo_features. I took a glance at the Python code and froze...

import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt

Straight off the bat I'm in trouble: Queue dramatic music. The Matplotlib wrapper I'd been using was really a wrapper for matplotlib's sub-module pyplot. My wrapper gave me no access to other sub-modules like mlab. I was going to have to modify the wrapper.

In Python, importing a package doesn't necessarily give me access to the sub-packages, eg. If I import the matplotlib base package, I can't use matplotlib.pyplot or matplotlib.mlab... but! If I import only matplotlib.pyplot, I can now also use matplotlib.mlab. Maybe (probably) I just don't understand Python packaging.

Stefan Seifert, the guy who created the Inline::Python module is a clever guy. Me, I'm not so clever, so I just kinda hacked away until things worked, and this is what I landed on. I imported matplotlib at the package level (which will run when the wrapped modules is used) and defined pyplot and mlab as their own class.

use Inline::Python;
my $py = Inline::Python.new();
$py.run('import matplotlib.pyplot');

class Matplotlib::Mlab {
    method FALLBACK($name, |c) {
        $py.call('matplotlib.mlab', $name, |c);
    }
}

class Matplotlib::Plot {
    method FALLBACK($name, |c) {
        $py.call('matplotlib.pyplot', $name, |c);
    }
}

class Matplotlib {
    method FALLBACK($name, |c) {
        $py.call('matplotlib', $name, |c);
    }
}

Oh, I also dropped the py from pyplot in my module because... reasons. I've also got a class for the top-level matplotlib package. I'm not sure if there's methods in there I will need to call, but it doesn't hurt to be prepared.

I don't even know if jumped this hurdle, or just kinda kicked it over and stumbled ahead. Let me know if you have a more sane way to I could have done this. In any case, what mattered to me most at the time was that it worked and I could move on to playing with plots. To use this fancy new wrapper in all my previous examples, all I need to do is change the class instantiation from this

my $plt = Matplotlib.new;

... to this

my $plt = Matplotlib::Plot.new;

Which actually maps closer the Python code, anyways. With that out of the way I can move on the next few lines of code.

np.random.seed(0)

# example data
mu = 100  # mean of distribution
sigma = 15  # standard deviation of distribution
x = mu + sigma * np.random.randn(437)

I can guess what random.seed does. Pseudo-random number generators (or PRNG's) use an algorithm to compute a random number; this is the "pseudo" part of pseudo-random. Provided you start the algorithm at the same number (the seed) each time, the result is always the same. How the seed is obtained normally (and how the random numbers are generated) differs between operating systems and programming languages. The seed function in Perl is called srand, so that part's easy.

Then we come to randn. A quick search led me to this StackOverflow post where I learned that it creates a "normal distribution." That link jumps to one of the replies, which is from an actual statistician! This helpful human explains that a normal distribution is "a distribution where the values are more likely to occur near the mean value". So, think bell curve.

I'm not a stats guy. Heck, I'm not even a maths guy... So I headed to RosettaCode to grab a normal distribution function in Perl 6. I modified it slightly (hopefully without breaking it) so that behaves like a very simple clone of numpy.random.randn, and like numpy, stuck it in it's own sub-module to the Numpl module I created in Part 3.

class Numpl::Random {
    method randn($n) {
        sqrt( -2 × log(rand) ) × cos( τ × rand ) xx $n;
    }
}
class Numpl {
    # linspace stuff ...

    method random {
        Numpl::Random.new();
    }
}

Which means I could now do this

my $np = Numpl.new;
my $x = $np.random.randn(437)

The next few lines are pretty straight-forward, so moving now to mlab.normpdf. Ok, so the comment there tells me that this thing adds a line of "best fit", but what the heck does it have to do with PDF? Being curious, I did a search and found out it stands for 'Probability Density Function'. With my curiosity quenched, I converted the rest of the code to Perl without much fanfare.

use Numpl;
use Matplotlib;

my $np   = Numpl.new;
my $plt  = Matplotlib::Plot.new;
my $mlab = Matplotlib::Mlab.new;

srand(0);

# example data
my $mu = 100;    # mean of distribution
my $sigma = 15;  # standard deviation of distribution
my $x = $np.random.randn(437).map( * × $sigma + $mu );

my $num_bins = 50;

my ( $fig, $ax ) = $plt.subplots();

# the histogram of the data
my ( $n, $bins, $patches ) = $ax.hist( $x, $num_bins, :normed(1) );

# add a 'best fit' line
my $y = $bins.map(-> $value {
    $mlab.normpdf( $value, $mu, $sigma )
});
$ax.plot($bins, $y, '--');
$ax.set_xlabel('Smarts');
$ax.set_ylabel('Probability density');
$ax.set_title('Histogram of IQ: $\mu=100$, $\sigma=15$');

# Tweak spacing to prevent clipping of ylabel
$fig.tight_layout();
$plt.show();

So, um, yeah... Not much to say here that hasn't been covered. I'm using a map again on the results of randn and norpdf. The rest is pretty standard translation stuff, and here's the result.

Even though I am seeding the PRNG, Perl will generate random numbers differently than Python, so this doesn't look exactly like the one in the gallery. You can remove the srand to get a different graph each time. The colours, however, are a little... academic. I think next I'll try applying one of the style sheets to a graph.

To be continued...

This is Part 3 in a series. You can start at the Intro here.

So far this Perl 6 wrapper class for matplotlib is going well. With the first graph in the gallery under my belt, I moved on to second example, fill_demo.py. Here's the Python code

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 1, 500)
y = np.sin(4 * np.pi * x) * np.exp(-5 * x)

fig, ax = plt.subplots()

ax.fill(x, y, zorder=10)
ax.grid(True, zorder=5)
plt.show()

Being a rather simple graph, you might think that the Perl code would be fairly straight forward, and you'd be right. The only minor curveball is another numpy function linspace, and that rather hairy looking operation for the y value.

To reiterate what I've said in previous parts, I'm not familiar with numpy at all. Or at least, I wasn't before I started working on these graphs. I inserted a print(x) into the Python code to see what linspace did. In hindsight, the name is obvious; It simply creates a linearly spaced sequence of numbers. Here's a quick demonstration

>>> numpy.linspace(0, 1, 3)
array([ 0. ,  0.5,  1. ])
>>> numpy.linspace(0, 1, 5)
array([ 0.  ,  0.25,  0.5 ,  0.75,  1.  ])
>>> numpy.linspace(0, 1, 6)
array([ 0. ,  0.2,  0.4,  0.6,  0.8,  1. ])

In the case of x in the graph, it's a sequence of 500 numbers evenly spaced from 0 to 1. So how would I do this in Perl. It is a monotonic sequence, so maybe the Sequence operator can help us again. So far we've seen a couple simple Sequences. The most simple sequence defines the start of a number series, and Perl lazily generates the rest.

> 0, 2, 4, 8 ... 1024
(0 2 4 8 16 32 64 128 256 512 1024)

This is nice and all, but I had to define 4 numbers for Perl to understand that sequence. If I dropped the 8, then it would have generated a sequence of even numbers. Instead, I can define a function which calculates how the next value should be generated.

> 0, 2, * × 2 ... 1024
(0 2 4 8 16 32 64 128 256 512 1024)

Here we see our old friend, the Whatever *. This time I'm asking for the Sequence starting 0, 2, then I define a mini-function that takes Whatever the last number in the Sequence was and multiplies it by 2. Sequences are a fascinating part of Perl 6 that could occupy a blog post all their own, but for now, that's enough of a foundation to create a simplified linspace function.

sub linspace( $start, $end, $steps ) {
    my $step = ( $end - $start ) ÷ ( $steps - 1 );
    return $start, * + $step ... $end;
}

linspace( 0, 1, 3 ); # Result: (0, 0.5, 1)
linspace( 0, 1, 5 ); # Result: (0, 0.25, 0.5, 0.75, 1)
linspace( 0, 1, 6 ); # Result: (0, 0.2, 0.4, 0.6, 0.8, 1)

This function does what I need, but it could stand to be a little more robust. I'll make a module to house this function, just in case I need it for future plots. I'll also add type constraints to the parameters, and multiple dispatch functions to handle steps of 0 or 1. I'm calling my library Numpl. Any resemblance to actual libraries is purely coincidental.

class Numpl {

    proto method linspace( Real $start, Real $end, Int $steps ) { * }

    multi method linspace( $start, $end, 0 ) { Empty }

    multi method linspace( $start, $end, 1 ) { $start }

    multi method linspace( $start, $end, Int $steps ) {
        my $step = ( $end - $start ) ÷ ( $steps - 1 );
        return $start, * + $step ... $end
    }
}

So that was a very scenic tour around the Sequence operator, but now that we have a linspace function, the rest is smooth sailing. Getting back to the plot, the final code now looks like this.

use Matplotlib;
use Numpl;

my $plt = Matplotlib.new;

my @x = Numpl.linspace( 0, 1, 500 );
my @y = @x.map(-> $x {
    sin( 4 × π × $x ) × exp( -5 × $x )
});

my ( $fig, $ax ) = $plt.subplots();

$ax.fill( $@x, $@y, :zorder(10) );
$ax.grid( True, :zorder(5) );
$plt.show();

Oh yeah, there was that hairy looking operation for the y values. To refresh your memory, the python looked like this

y = np.sin(4 * np.pi * x) * np.exp(-5 * x)

I can pretty much copy this operation exactly as it appears and put it inside the map. I've put the map operation on it's own line as I think this aids readability.

You might have also noticed I'm using proper @arrays in this one, and there is a valid reason for this. Sequences are lazy, and typically can only be iterated over once. x is iterated over in the map and that would prevent any later iterations. Here I've assigned the Sequence to a fully reified Array, which also means the Sequence is evaluated eagerly at assignment (ie. not lazily). As covered in part 2, Python doesn't like Perl arrays as positional arguments, so I coerce them to scalar values when I pass them.

The other way I could have handled this was to coerce my Sequence to a List on assignment like so.

my $x = Numpl.linspace( 0, 1, 500 ).List;

Then I could have assigned it to a scalar variable and still be free to iterate over it as many times as I like. If you have no need for laziness from your linspace function, you could modify it to return a reified List either coercing the return value as above, or by instructing the Sequence to be eagerly evaluated.

return ( $start, * + $step ... $end ).List
# or
return eager $start, * + $step ... $end

But I like to be able to control the laziness of my linspace result from outside the function.

Whichever way you do it, the end result should look like this

The wrapper module is working out well... a little too well. I wanted to tackle something a little more complex, and scrolling through the matplotlib gallery I saw it. It's high time for a histogram.

To be continued...

This is Part 2 in a series. You can start at the Intro here.

In Part 1 of this series, I managed to use matplotlib in Perl 6 to output a simple graph from the matplotlib tutorial. In this post, I will tackle the first graph in the matplotlib gallery, barh_demo.py.

import matplotlib.pyplot as plt
import numpy as np

plt.rcdefaults()
fig, ax = plt.subplots()

# Example data
people = ('Tom', 'Dick', 'Harry', 'Slim', 'Jim')
y_pos = np.arange(len(people))
performance = 3 + 10 * np.random.rand(len(people))
error = np.random.rand(len(people))

ax.barh(y_pos, performance, xerr=error, align='center',
        color='green', ecolor='black')
ax.set_yticks(y_pos)
ax.set_yticklabels(people)
ax.invert_yaxis()  # labels read top-to-bottom
ax.set_xlabel('Performance')
ax.set_title('How fast do you want to go today?')

plt.show()

Again, numpy is being used, but I figured Perl 6 can probably do whatever is needed natively, so I proceeded to convert the imports to Perl parlance (perlance?) and got started.

use Matplotlib;
my $plt = Matplotlib.new;

$plt.rcdefaults();
my ( $fig, $ax ) = $plt.subplots();

As mentioned in the tail end of Part 1, Python likes scalar values, so from here on in I'm going to stick to using mostly scalar variables in my Perl code. I may also provide alternative methods of doing things if I think they are noteworthy or interesting. Now on with the show!

I need to create the example data... people is just your average list of strings. Perl has always had nice quoting construct to create a list of words, so that ones is as simple as

my $people = < Tom Dick Harry Slim Jim >;

Now what's numpy.arange doing? I knew that len(people) was 5, but I didn't know what arange was doing with that 5. It turns out it simply creates an array of values starting at 0, incrementing by one and up to 5, ie. [0, 1, 2, 3, 4]. As seen in Part 1, I can use the handy Sequence operator for that, but creating lists like this is fairly common which is why Perl 6 also provides a convenient shortcut for it:

my $list = (0 ...^ 5); # Result: (0, 1, 2, 3, 4)
my $list = ^5;         # same thing

Of course, I can use a variable instead of a literal integer. As covered in Part 1, lists used in a numerical context are coerced to their number of elements. These features combined mean that my y_pos assignment looks like this.

my $y_pos = ^$people;

For the next two lists of values, performance and error, numpy is being used to generate some random values. Both values reference len(people), and this equates to the number of random values returned: 5. Perl has a fairly standard rand function that - without arguments - returns a random value between 0 and 1. However, I can use the "list repetition" operator (xx) to create a list of random values. Again referring to Part 1, I know that mathematical operations on a numpy array are equivalent to mapping that operation across each element, so I ended up with this.

my $performance = ( rand xx $people ).map( 10 × * + 3 );
my $xerr = rand xx $people;

In those 3 lines of code, The $people list variable is used in a numerical context so it is evaluated to it's number of elements: 5. You could always be more explicit in your code and say $people.elems if you prefer.

You may have noticed I called my list of error values xerr instead of error and there's a good reason. Perl 6 provides a convenient syntax for passing named arguments (kwargs in Python) when a variable has the same name as the named argument.
Oh, and one more thing. Remember that < word quoting > syntax I used before? I can use that when passing named arguments too. Here's a quick illustration using an example function foo() that takes 1 positional parameter, and 1 named parameter.

sub foo( $positional, :$named ) { ... }

# Calling the function
foo( 'bar', :named('baz') );

# Using the word-quoting construct
foo( 'bar', :named<baz> );

# or if I had a $named variable
my $named = 'baz';
foo( 'bar', :$named );

To some, it may seem odd to have so many ways to pass a named argument (and truthfully there's more) but I hope what comes through here is that the syntax tries to consistent in how it's used... But I digress... Less plodding and more plotting!

With the above out of the way, the rest of the code is pretty straight forward. Here's the final code.

use Matplotlib;
my $plt = Matplotlib.new;

$plt.rcdefaults();
my ( $fig, $ax ) = $plt.subplots();

# Example data
my $people = < Tom Dick Harry Slim Jim >;
my $y_pos = ^$people;
my $performance = ( rand xx $people ).map( 10 × * + 3 );
my $xerr = rand xx $people;

$ax.barh(
    $y_pos, $performance,
    :$xerr, :align<center>, :color<green>, :ecolor<black>
);
$ax.set_yticks($y_pos);
$ax.set_yticklabels($people);
$ax.invert_yaxis(); # labels read top-to-bottom
$ax.set_xlabel('Performance');
$ax.set_title('How fast do you want to go today?');

$plt.show();

This graph is based on random data, so the output will vary slightly with each run, but ultimately I get something along the lines of this when I run it.

Well that was mostly painless. The wrapper module is working as expected, and I was able to emulate a basic function of numpy with relative ease. I glanced back to the the Matplotlib gallery... Next in line was the fill_demo. Let's go!

This is Part of a series. You can start at the Intro here

Looking at the graphs in the Matplotlib tutorial, the first one is a straight line (boring). The second is a few red dots plotting a curve (getting better). The 3rd graph looked a more interesting: Three curved plots consisting of different patterns and colors. Here's the Python code.

#!/usr/bin/env python

import numpy as np
import matplotlib.pyplot as plt

# evenly sampled time at 200ms intervals
t = np.arange(0., 5., 0.2)

# red dashes, blue squares and green triangles
plt.plot(t, t, 'r--', t, t**2, 'bs', t, t**3, 'g^')
plt.show()

I ran it in python to confirm it worked as expected (it did). I then proceeded to copy and paste the code into a new Perl 6 file and changed the import statements to use statements.

use Matplotlib;
my $plt = Matplotlib.new;

Now for the data... The t values were generated by a numpy function called arange. I didn't know what that did, and rather than wrap numpy as well, I figured I could probably do whatever it was doing with Perl 6. As the comment suggested, it was a simple number sequence. Perl 6 has a handy Sequence operator for just this sort of thing.

my @t  = 0, 0.2 ...^ 5;

This defines a Sequence starting at 0, followed by 0.2, then increasing at the same interval until is gets up to (^) 5. If I wanted to include 5 in the Sequence, I would just drop the ^. Using some well placed print statements, I confirmed by @t values were the same as the python version.

I then pasted the rest of the Python code into my Perl script, inserted a @ in front of each t and ran it. Things did not go as planned. Well, I got a graph, but it wasn't quite right

Some of you might have already guessed where I went wrong. Spoiler, I don't have any experience with numpy arrays (or numpy for that matter). A few more print statements let me know where I was going wrong. You see, when referring to a list (or array) in a numerical context, Perl will coerce the array to it's number of elements

my @t = (2, 3, 4);
@t ** 2; # Result: 9

However numpy arrays will apply that operation to each value in the array. I could create an object in Perl 6 that behaves the same way, but for now I'll just use Perl's map function which is perfectly apt for this sort of thing. Perl 6 provides a number of ways to map values in an array: I can be very explicit, or very succinct.

@t.map(-> $x { $x ** 2 }) # Result: (4, 9, 16)
@t.map(*²)                # same thing

Note the lack of curlies {} in the second example. This is a special syntax where * takes on the value of whatever value it was passed. In fact, Perl 6 actually refers to that * as Whatever. This WhateverCode is best suited for simple operations, which perfectly describes a binary mathematical operation such as exponentiation.

Additionally, Perl contains quite a few unicode operators; My favourites are × for multiplication and ÷ for division, instead of the historical compromise of * and /. In the second example above, I am using the superscript of 2 (²) to raise Whatever (*) to the power of 2.

I modified my arguments accordingly and ran it again. Another graph! This one also had 2 dots... But they were in a different place to last time, so that's progress, right? I looked at the gist again and noticed the arguments to plot were inside square brackets. Accepting it was probably due a difference in how each language handled positional arguments, I diligently wrapped my arguments to plot in square brackets.

$plt.plot([
    @t, @t,         'r--',    # red dashes
    @t, @t.map(*²), 'bs',     # blue squares
    @t, @t.map(*³), 'g^'      # green triangles
]);

$plt.show();

This time when I ran it and was delighted when a graph identical to the one on the tutorial appeared before my eyes.

Wondering about those square brackets, I played around a little with the call to plot. I quickly learned that Python prefers to be given what Perl calls "scalar" variables. In simple terms, a scalar is a variable that Perl will treats as a single "thing". Passing a Perl array of 3 @things to a Python function would cause those 3 values to get expanded out to 3 positional arguments. Creating a scalar list $things or coercing $@things to be passed in a scalar context works much better.

With regard to that map, it seems that by giving the map directly to plot, it was more "listy" than scalar. By assigning the result to a scalar first, I could then pass those values to plot without having to wrap all the arguments in square brackets, like so..

my $t  = [ 0, 0.2 ...^ 5 ];
my $t2 = $t.map(*²);
my $t3 = $t.map(*³);

$plt.plot(
    $t, $t,  'r--',    # red dashes
    $t, $t2, 'bs',     # blue squares
    $t, $t3, 'g^'      # green triangles
);

$plt.show();

So that about wraps up Part 1. This graph was kind of interesting, but I knew matplotlib was capable of more. I felt no great excitement from the other images in the tutorial, so headed off to the gallery to see what else I could sink my teeth into.

To be continued...

I'm a network engineer by trade, but I write a lot of scripts at work to automate or simplify aspects of my job for myself and my colleagues. Most of the tools I write are predominantly doing text parsing, and I use Perl 5 for that. I use Python on occasion where I feel like it's a better fit, but mostly I prefer Perl. I'm also a fan of Perl 6 and have been playing with it since the official release in December '15.

I've been aware of matplotlib (and numpy) for a while, but in my line of work I've never really had a need to use either. That said, I have the awareness that matplotlib is a great plotting library, something that Perl 6 is lacking, but not all is lost.

While perusing the comments in this HackerNews thread, someone linked to a gist that used Inline::Python (a module for executing Python code and accessing Python libraries from Perl 6) to wrap matplotlib in a Perl 6 class. Was this really all that I needed to do to use matplotlib in Perl 6? I headed to matplotlib tutorials page to try it out.

---

So that's what this series will be about. I found the process of figuring out how to use matplotlib in Perl 6 interesting, so decided to write about it. Now, I don't want to lead anyone down the garden path, so I'm putting this up front. After the first plot, I discovered that wrapper in the gist will fail when passing named arguments. I've added a comment to that gist about this. There are also more shortcomings to this module, but I'll cover those in later parts to this series.

The wrapper module I'll be working with for the first few diagrams looked like this, though it will change as I discover limitations.

class Matplotlib {
    use Inline::Python;
    has $!py;

    submethod BUILD() {
        $!py = Inline::Python.new();
        $!py.run('import matplotlib.pyplot')
    }

    method FALLBACK($name, |c) {
        $!py.call('matplotlib.pyplot', $name, |c);
    }
}

Essentially this class creates a new Inline::Python object and runs import matplotlib.pyplot when it get instantiated. There are no normal methods defined, so any class methods route to the FALLBACK method, which takes the name of that method and calls the matplotlib.pyplot method of the same name.

But what's that |c? That's the syntax for a special type in Perl 6, a Capture. Any additional argument are "captured" in the Capture object. A capture doesn't do any unpacking. It encases them in carbonite and ships them off to wherever to be unpacked later. Capture objects are very useful when you want to punt arguments to another function.

There's also an Inline::Python module for Perl 5 if you want to try to use that with pyplot. I took a stab at it but ran into difficulty with named arguments, but I gave up pretty quickly since Perl 6 was nicer for the number-y stuff.

Anyway, this wrapper module will cover most generic uses of pyplot, and I get a few plots in before I hit my first hurdle, so head on over to Part 1 to start the journey.

If you'd like to play around with my modules as they existed after this series, I've put them up in a gist. You're welcome to fork them and do whatever you will.