Firstly I’d like to point out that, funnily enough, I’m not very familiar with Java or Perl6. It’s entirely possible that I’ll make all sorts of errors in the following details. If you spot any do please let me know.

Background

The Java language ecosystem is big and mature after years of heavy investment of time and money.

It doesn’t have a central repository of Open Source modules like CPAN (though Maven repositories like these are similar I guess). It does, however, have a number of mature high quality class libraries, and a very large number of developers familiar with those libraries (more on that below).

Goals

The primary goal of the java2perl6api project is to make it easy to create Perl 6 class libraries that mirror Java equivalents. By mirror I mean share the same method names and semantics at a high level (though not at a low-level, more on that below).

Secondary goals are to do that well enough that:

• the documentation for Java classes can serve as primary the documentation for the corresponding Perl 6 classes. The Perl 6 classes need only document the differences in behavior, which these should be minimal and ‘natural’. The same applies to books describing the Java classes.
• Java developers familiar with the Java classes should feel comfortable working with the corresponding Perl 6 classes.
• and, hopefully, some way can be found to convert test suites for the Java classes into Perl 6 code that’ll test the corresponding Perl 6 classes. (I appreciate that this is a non-trivial proposition, but there are viable approaches available, like xmlvm.) Even if that can’t be done, extracting and translating tests manually is less work, and more effective, than creating them from scratch for a new API.

Why?

Firstly, creating good APIs is hard. Java APIs like JDBC 3.0 and NIO.2 are the result of years of professional effort and demanding commercial experience. Why not build on that experience?

I appreciate that Java APIs are often limited by the constraints of the language, such as the lack of closures, and that Perl 6 can probably express any given set of semantics more effectively than Java. My point here is that some Java APIs embody, however inelegantly, years of hard won experience that we can benefit from. I’d rather make new mistakes than repeat old ones.

Secondly, there are many more Java developers than Perl developers. Many many more if job vacancies are any indication:

I think we’d be foolish not to try to smooth the path for any Java developers who might be interested in Perl 6. The java2perl6api project is just one small aspect of that.

I really hope someone starts writing a “Perl 6 for Java Developers” tutorial. Perl 6 has the potential to become a very popular language¹. Getting just a tiny percentage of Java developers (and Computer Science majors and their teachers) interested in it could be a big help.

Thirdly, any future DBI for Perl 6 and Parrot needs a much better foundation than the very limited and poorly defined one that underlies the Perl 5 DBI. I plan to adopt the JDBC 3.0 API and test suite for that internal role. (You could call this a “Test Suite Driven Strategy”.) I’ll talk more about that in a future blog post.

The History java2perl6api

I’ve been kicking around various ideas for integrating Java and Perl6/Parrot for years. I think I first decided to use JDBC as the inspiration for the DBI-to-driver API in 2006.

You may remember back in 2004, around the 10th anniversary of the DBI, the Perl Foundation setup a “DBI Development Fund” that people could donate to. I’ve never drawn any money from that fund. I want to use it to oil other peoples wheels.

In 2007 Best Practical sponsored Perl 6 Microgrants through the Perl Foundation. I asked if I could piggyback my idea for a Java to Perl 6 API translator onto their microgrant management process but using money from the DBI Development Fund. TPF and Best Practical kindly agreed. I posted a description of the task and Phil Crow volunteered and was awarded the microgrant in April 2007.

At OSCON in July 2007 I gave lightning talk called “Database interfaces for open source languages suck” which explained the rationale for using JDBC as a foundation for the DBI-to-driver API and mentioned Phil’s java2perl6 project.

Development ground to a halt around the end of 2007 for various reasons. It picked up again for a few months after OSCON 2009 (where I gave a short lightning talk asking for help) then stalled again in October. Partly because we seemed to have hit a limitation with Rakudo and partly because I was focussed on Devel::NYTProf version 3 and then version 4, which took way more time than I expected.

There’s life in the project again now. We’ve dodged the earlier problem, put the code on github, brought it into sync with current Rakudo Perl 6 syntax, and generally instilled some momentum.

The Current java2perl6api

Let’s take a look at a simple example.

To generate a perl6 file that mirrors the API of the java.sql.Savepoint class you’d just execute java2perl6api like this:

$ java2perl6api java.sql.Savepoint
loading java.sql.Savepoint
wrote java/sql/Savepoint.pm6 - interface java.sql.Savepoint
checking java/sql/Savepoint.pm6 - interface java.sql.Savepoint

That’s loaded and parsed the description of the java.sql.Savepoint class (from the javap command), generated a corresponding perl6 module, and run perl6 to validate it.

The generated module (with some whitespace and cruft removed) looks like this:

use v6;
role java::sql::Savepoint {
    method getSavepointId (
    --> Int   #  int
    ) { ... }
    method getSavepointName (
    --> Str   #  java.lang.String
    ) { ... }
};
=begin pod
=head1 Java
  Compiled from "Savepoint.java"
  public interface java.sql.Savepoint{
      public abstract int getSavepointId() throws java.sql.SQLException;
      public abstract java.lang.String getSavepointName() throws java.sql.SQLException;
  }
=end pod

The pod section shows the description of the class that javap returned. The java2perl6api utility parsed that Java interface and generated the corresponding Perl6 role. The ‘java.sql.Savepoint’ has been mapped to ‘java::sql::Savepoint’. The generated methods are stubs using ... (the “yada, yada, yada” operator). The types int and java.lang.String have been mapped to Int and Str. Because the only types used were built-ins, no type declarations were added.

Currently java2perl6api handles the above plus overloaded methods (which generate multi methods), multiple implements clauses (which generate multiple does clauses). There’s also partial support for class/interface constants (which currently generate exported methods).

The default behavior is to recursively process any Java types referenced by the class which aren’t mapped to Perl 6 types. So executing java2perl6api java.sql.Connection, for example, will generate 48 Perl 6 modules! (Because java.sql.Connection refers to many types, including java.sql.Array which refers to many types including java.sql.ResultSet which refers to java.net.URL which refers to java.net.Proxy etc. etc.) The --norecurse options disables this behavior.

Normally you’ll want to use the recursion but instead of letting it drill all the way into the Java types, you would supply your own ‘typemap’ specification via an option. That tells java2perl6api which Java types you want to map to which Perl 6 types. So instead of recursing into the java.net.URL type to generate a java/net/URL.pm6 file, for example, you can tell java2perl6api to use a specific Perl 6 type. Perhaps just Str for now.

How this relates to JDBC / DBDI / DBI v2

I want to start applying java2perl6api to the JDBC classes now to create a “Database Driver Interface” or “DBDI” for Perl 6.

Starting with the DriverManager class and the Connection interface I’ll use java2perl6api to generate corresponding Perl 6 roles with heavy stubbing out of types. Basically anything I don’t need to think about right now will be mapped to the Any type.

I’ll start fleshing out some basic implementation logic for each in a Perl 6 class that does the corresponding role. I’ll probably use PostgreSQL as the first driver and the guts of MiniDBD::Pg as inspiration.

The first minor milestones will be creating connections, then execute non-selects, then selects then prepared statements. Somewhere along the way I expect they’ll be a Perl 6 DBDI driver implemented for the Perl 6 MiniDBI project. The next key step would be to start refactoring the code heavily so anyone wanting to implement a new driver should only have to implement the driver specific parts. (There are some JDBC driver toolkits that can provide useful ideas for that.)

What needs doing

There’s a TODO file in the repository that lists the current items that need working on.

One fairly simple item is to add a --prefix option to specify an extra leading name for the generated role. So java.sql.Savepoint with a prefix of DBDI would generate a DBDI::java::sql::Savepoint role.

Another item, less simple but more important, is to automatically discover the values of constants and embed them into the generated file. Probably the best way to do that is to extend the parser (which uses Parse::RecDescent) to parse the verbose-mode output of javap, which includes those details.

There are plenty of others.

How you can get involved

Firstly, come and say “Hi!” in the #dbdi IRC channel on irc.freenode.net.

The code is on github. You can get commit access by asking on the #perl6 channel.

There’s also a mailing list at dbdi-dev@perl.org which you can subscribe to.

I look forward to hearing from you!

1. 
   When I say “Perl 6 has the potential to become a very popular language” I do so with typical British Understatement.

Dublin.pm

Aidan Kehoe and I had a very small but interesting Dublin.pm meeting that night. Their first since 2004! Our wide-ranging discussions that night included me trying to understand what led Dublin.pm to flounder instead of flourish. I think a key factor was the (implicit?) expectation that members should make technical presentations.

Living in the west of Ireland there aren’t enough local Perl users (that I’ve found so far) to have a viable Perl Mongers group. So I setup the Limerick Open Source meetup instead.

Here’s what worked for us: We sit around in a quiet comfy hotel bar and chat. Naturally the chat tends towards the technical, and laptops are produced and turned around to illustrate a point or show results of a search, a chunk of video etc. There’s no set agenda, no declared topics, and no presentations. And yet, I think it’s fair to say, that everyone who’s come along has learnt interesting (albeit random) stuff.

I’d like to hear from perl mongers, in groups of all sizes, what kinds of balance between the social and technical aspects of Perl Mongers meetings works (or doesn’t work) for you.

OSSBarcamp

At OSSBarcamp I gave a ~15 minute ‘lightning talk’ on Devel::NYTProf in the morning, and a ~50 minute talk on Perl Myths in the afternoon.

The Perl Myths talk was a major update to my previous version, now over 18 months old, incorporating lots of updated graphs and other fresh information.

There is so much happy vibrant productive life in the Perl community that updating the presentation has been lovely experience. I keep having to revise the numbers on the slides upwards. There are lots of great graphs and they’re all going upwards too! (Many thanks to Barbie for the great new graphs of CPAN stats.)

I’ve put a PDF of the slides, with notes, on slideshare. Best viewed full-screen or downloaded.

I made a screencast but I think I’ll hang on to that until after I give the same talk, updated again, at the Italian Perl Workshop (IPW09) in Pisa in October — I’m really looking forward to that! I’ll make another screencast there and decide then which to upload.

After OSSBarcamp last week, and before IPW09 in late October, I’ll be flying to Moscow, visa permitting, to give a talk at the HighLoad++ (translated) conference. I’ve never been to Russia before so that’s going to be an amazing experience!

It needs updating though, and that’s where you can help…

It’s a strongly fact base talk with lots of graphs, numbers, and references. A lot has happened in the Perl community since I wrote the talk over 18 months ago!

Lots of the information needs to be checked and updated. The jobs graphs are easy, as are the basic CPAN numbers.

I’d like to get updated versions of: the graph of CPAN uploads (slide 23), test counts for perl 5.10.1 (slide 44), the graph of cpan-testers reports (slide 47).

One item that’s not in the slides, which were written in Feb 2008, but is in the screencast made later at OSCON (at 7:00) is the statement that “One quarter of all CPAN distributions have been updated in the last 4 months! Half of all in the last 17 months!” That’s an impressive and important statistic but it need updating. I think I got that from Schwern’s excellent “Perl is unDead” talk, but I don’t know how he got the figures or, more importantly, how to update them. (And schwern.org seems down so I can’t get the slides.)

At OSCON I also talked about myths relating to Perl 6 (again, not in the old slides, but starting at 19:20 in the screencast). I’d like to cover Perl 6 when I talk in Dublin in September, but the status of Parrot and Perl 6 has changed in the last 18 months even more than Perl 5. So I need help with good sources of hard data for Parrot and Perl 6, similar to what I have in the screencast but updated.

More generally, I’m also looking for new sources of information — hard data about the vigourous health of perl and its community. Have you seen any relevant talks/slides or blog posts you can recommend?

Can you help with any of the above?

Thank you in advance!

Two versions of the slides are available: one with just the slides on a landscape page, and another with slides and notes on a portrait page.

I also have a screencast of the presentation which I hope to edit and upload before long. (I’ll update this page and post a new note when I do.)

I’ve uploaded my Perl Myths presentation to slideshare.net and google video:

“Perl has it’s share of myths. This presentation debunks a few popular ones with hard facts. Surprise yourself with the realities.”

While I agree with Andy Lester that Good Perl code is the best form of evangelism, I wanted to put together a presentation that others could refer to when they encounter misinformation about Perl. I cover these myths that I’ve heard recently:

• Perl is dead
• Perl is hard to read / test / maintain
• Perl 6 is killing Perl 5

and pull in a wealth of upto date information, some of it quite surprising even to those familiar with Perl and its community. There are two versions, plus a video. I recommend the one with notes (which have useful extra detail and context for the slides) which is best viewed as a PDF. There’s also one without notes which I’ve embedded here:

I videoed an extended version of this presentation at IWTC in Dublin in February. The first 40 minutes or so correspond with the slides above. In the remaining 30 minutes or so I talk about Parrot and Perl 6. I’ve embedded the video below, but wordpress forces me to use a small size so you’ll probably prefer to view it at video.google.com: