Introducing Data::Tumbler and Test::WriteVariants

For some time now Jens Rehsack (‎Sno‎), H.Merijn Brand (‎Tux‎) and I have been working on bootstrapping a large project to provide a common test suite for the DBI that can be reused by drivers to test their conformance to the DBI specification.

This post isn’t about that. This post is about two spin-off modules that might seem unrelated: Data::Tumbler and Test::WriteVariants, and the Perl QA Hackathon that saw them released.

Continue reading

Migrating a complex search query from DBIx::Class to Elasticsearch

At the heart of one of our major web applications at TigerLead is a property listing search. The search supports all the obvious criteria, like price range and bedrooms, more complex ones like school districts, plus a “full-text” search field.

This is the story of moving the property listing search logic from querying a PostgreSQL instance to querying an ElasticSearch cluster. Continue reading

NYTProf v5 – Flaming Precision

As soon as I saw a Flame Graph visualization I knew it would make a great addition to NYTProf. So I’m delighted that the new Devel::NYTProf version 5.00, just released, has a Flame Graph as the main feature of the index page.

nytprof-v5-flamegraph.png

In this post I’ll explain the Flame Graph visualization, the new ‘subroutine calls event stream’ that makes the Flame Graph possible, and other recent changes, including improved precision in the subroutine profiler. Continue reading

Introducing Devel::SizeMe – Visualizing Perl Memory Use

For a long time I’ve wanted to create a module that would shed light on how perl uses memory. This year I decided to do something about it.

My research and development didn’t yield much fruit in time for OSCON in July, where my talk ended up being about my research and plans. (I also tried to explain that RSS isn’t a useful measurement for this, and that malloc buffering means even total process size isn’t a very useful measurement.) I was invited to speak at YAPC::Asia in Tokyo in September and really wanted to have something worthwhile to demonstrate there.

I’m delighted to say that some frantic hacking (aka Conference Driven Development) yielded a working demo just in time and, after a little more polish, I’ve now uploaded Devel::SizeMe to CPAN.

In this post I want to introduce you to Devel::SizeMe, show some screenshots, a screencast of the talk and demo, and outline current issues and plans for future development. Continue reading

What’s actually installed in that perl library?

A key part of my plan for Upgrading from Perl 5.8 is the ability to take a perl library installed for one version of perl, and reinstall it for a different version of perl.

To do that you have to know exactly what distributions were installed in the original library. And not just which distributions, but which versions of those distributions.

I’ve a solution for that now. It turned out to be rather harder to solve than I’d thought… Continue reading

Upgrading from Perl 5.8

Imagine…

  1. You have a production system, with many different kinds of application services running on many servers, all using the perl 5.8.8 supplied by the system.
  2. You want to upgrade to use perl 5.14.1
  3. You don’t want to change the system perl.
  4. You’re using CPAN modules that are slightly out of date but you can’t upgrade them because newer versions have dependencies that require perl 5.10.
  5. The perl application codebase is large and has poor test coverage.
  6. You want developers to be able to easily test their code with different versions of perl.
  7. You don’t want a risky all-at-once “big bang” upgrade. Individual production installations should be able to use different perl versions, even if only for a few days, and to switch back and forth easily.
  8. You want to simplify future perl upgrades.

I imagine there are lots of people in similar situations.

In this post I want to explore how I’m tackling a similar problem, both for my own benefit and in the hope it’ll be useful to others. Continue reading