Comments on: Wishlist of PL/Perl Enhancements for PostgreSQL 8.5

By: Shahaf Abileah

Shahaf Abileah — Tue, 04 Oct 2011 21:29:03 +0000

Thanks Tim!

I dug in deeper and discovered that the majority of the overhead was attributed to a single line of Perl code used to deserialize the json string into an object. I then discovered the JSON::XS library which reduced the cost dramatically. It appears that in my case the overhead of calling plperl is insignificant compared to the cost of the actual work I’m doing in the plperl function.

By: TimBunce

TimBunce — Tue, 04 Oct 2011 21:23:17 +0000

Yes, there are some plperl performance improvements in Postgres 9.x. You’ll also get more performance by using a Perl that was configured without threads (you may have to build and install a separate perl for that and then configure a build PostgreSQL to use that). Neither will “solve” the overhead, but they’ll reduce it.

You should also ask for help on the pgsql-performance mailing list (after searching the archives http://archives.postgresql.org/pgsql-performance/)

Good luck.

By: Shahaf Abileah

Shahaf Abileah — Tue, 04 Oct 2011 17:28:34 +0000

Do you know whether there are any plperl performance improvements in Postgres 9.x?

I’m running an experiment where I store json data in a big (~5KB) text column in Postgres. I wrote functions using plperlu to extract values out of the json using jsonpath expressions (making use of existing json and jsonpath libraries for Perl). I then created functional indexes (using these functions) to avoid having to actually run the functions at query-time (as long as the query has a matching expression). It all works, but performance is not where I’d like it to be. In my test setup it takes ~2 ms to insert the large record. If I add one functional index, it takes ~11 ms. If I add two indexes, it takes ~19 ms. In other words, it takes about 8 ms for each execution of the plperl function. If migrating to 9.x would solve this overhead, awesome. If not, I wonder: are any standard ways to tune or analyze plperl functions?

thanks!

–S

By: NYTProf v4 – Now with string-eval x-ray vision! « Not this…

NYTProf v4 – Now with string-eval x-ray vision! « Not this… — Wed, 09 Jun 2010 19:49:58 +0000

[...] those months I’d also started working on enhancements for PostgreSQL PL/Perl. That project turned into something of an epic adventure with more than its fair share of highs and [...]

By: TimBunce

TimBunce — Fri, 26 Mar 2010 11:37:40 +0000

It would be interesting and useful to add, but it’s probably quite a lot of work.

By: Oleg Bartunov

Oleg Bartunov — Fri, 26 Mar 2010 10:39:48 +0000

Did you consider support of pg’s internal type in perl ? I imagine a lot of people will benefit writing support functions for new operators, instead of using C.

By: TimBunce

TimBunce — Wed, 07 Oct 2009 22:13:49 +0000

I see your point. What’s needed is some equivalent of a “use” statement that can be put at the start of plperl functions. I have a few ideas – I’ll give it some thought. Thanks.

By: Peter Eisentraut

Peter Eisentraut — Wed, 07 Oct 2009 12:24:21 +0000

Item 1 could be controversial. It’s undoubtedly useful, but it creates an action at a distance that effectively invites users to create mutually incompatible PL/Perl installation. Ideally, you would want users to create libraries of reusable PL/Perl functions, but when those only work with certain at_init_do settings, then you create a big mess. Look at PHP; they have done something quite similar with their php.ini.

By: TimBunce

TimBunce — Tue, 06 Oct 2009 20:40:49 +0000

Suggested change: rename plperl.at_init_do to plperl.at_perl_init_do and add plperl.at_safe_init_do which would be similar but specify code to run when the Safe compartment is initialized. It could thus be set by per-user/per-role GUC.

By: TimBunce

TimBunce — Tue, 06 Oct 2009 20:03:15 +0000

It’s not fixed in 5.10.1. I’ve asked for an update on the status. Meanwhile I’ve added a comment on the ticket with a simple workaround that would be effective for plperl and that I’ll include in my changes.

By: Andrew G.

Andrew G. — Tue, 06 Oct 2009 12:31:29 +0000

Points 1-3 are primarily attempts to work around the lack of features that should have been in Safe.pm in the first place.

For comparison, in TCL if you create a safe interpreter using ::safe::interpCreate, then the safe interpreter can load modules etc. in the normal way, subject to (a) the specified mapping between host filenames and the virtual filenames seen inside the safe interpreter, and (b) the fact that the loaded modules are still running inside the safe interpreter and have no more access than it does.

A proper system of external interfaces for Safe would obviate the need for many of the features you are requesting.

Incidentally, is the sort{} + threads + Safe bug (#60374) fixed yet? (and if it is, why is it still open?)

By: TimBunce

TimBunce — Tue, 06 Oct 2009 08:40:27 +0000

Thank you, yes, I mean at the end of the process. I’m not sure what you mean by “Most of the rest of your points would be better addressed at the perl level”.

By: Zbigniew Lukasiak

Zbigniew Lukasiak — Tue, 06 Oct 2009 08:01:25 +0000

I don’t have anything smart to write here – but I wanted to express my support, especially for the point 1 above. It was a long time ago that I learned about PL/Perl and I was initially thrilled about the possibilities – but after I learned about the constraints for loading modules I realized that it’s application is very limited.

By: Andrew G.

Andrew G. — Tue, 06 Oct 2009 04:08:48 +0000

Currently PostgreSQL doesn’t execute END blocks at server shutdown

You do realize that this would be impossible, right? That interpreters are created per-connection and no interpreter state can be preserved past the end of the session (each session is a separate process)? Most of the rest of your points would be better addresses at the perl level (Safe is a joke compared to, for example, TCL's safe interpreters).