http://search.cpan.org/perldoc?dist_surveyor

Re MyCPAN, I had honestly completely forgotten about the data license fees. I had flagged it in my mind as “not an option for us” and when I wrote this post I only remembered your previous blog post where it seemed clear there was still work to be done before it was complete.

I’m sorry for misrepresenting the state of MyCPAN. It certainly wasn’t intentional. I’ve updated the post. Let me know if it’s not clear.

The DPAN you mention in your slides isn’t the part of this process that does the part you are trying to do. You’ve never seen the stuff I told you that I had because I haven’t made it public. You declined to license it.

I’m perfectly fine with you not using it, but don’t misrepresent things.

I dug in deeper and discovered that the majority of the overhead was attributed to a single line of Perl code used to deserialize the json string into an object. I then discovered the JSON::XS library which reduced the cost dramatically. It appears that in my case the overhead of calling plperl is insignificant compared to the cost of the actual work I’m doing in the plperl function.

You should also ask for help on the pgsql-performance mailing list (after searching the archives http://archives.postgresql.org/pgsql-performance/)

Good luck.

I’m running an experiment where I store json data in a big (~5KB) text column in Postgres. I wrote functions using plperlu to extract values out of the json using jsonpath expressions (making use of existing json and jsonpath libraries for Perl). I then created functional indexes (using these functions) to avoid having to actually run the functions at query-time (as long as the query has a matching expression). It all works, but performance is not where I’d like it to be. In my test setup it takes ~2 ms to insert the large record. If I add one functional index, it takes ~11 ms. If I add two indexes, it takes ~19 ms. In other words, it takes about 8 ms for each execution of the plperl function. If migrating to 9.x would solve this overhead, awesome. If not, I wonder: are any standard ways to tune or analyze plperl functions?

thanks!

–S

(On my initial run it had 39 distributions in dists_not_matched, 106 modules in no_dist_found, and seemed to be confused by the archlib, so there were 600+ modules in skipped_modules->bad_mod_info like “x86_64-linux-thread-multi::Moose”.)