My talk on Devel::NYTProf seemed to go well. This year I covered not just NYTProf and the new features in v3 (not yet released) but also added a section on how to use NYTProf to optimize your perl code.

Here’s a quick summary, with links to the slides and screen-cast, and outline what’s still to be done before v3 gets released (getting closer by the day).The first two-thirds of the talk covered general profiling topics and an overview of NYTProf v3. This was an updated and compressed version of the talk I gave last year.

The final third covered optimization techniques.

I began with stern warning against optimization unless there’s a clear need, followed by a selection of quotes, among which is the well known classic from Donald Knuth: “premature optimization is the root of all evil”. What few people remember is that that’s just a fragment of a more balanced statement:

“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.”

After getting an implicit nod from Donald I quoted Rob Pike on the importance of measurement:

“Bottlenecks occur in surprising places, so don’t try to second guess and put in a speed hack until you have proven that’s where the bottleneck is.”

Then I outlined the sequence of steps I usually take when performance profiling and optimizing code. These were grouped into three phases: “Low Hanging Fruit”, “Deeper Changes”, and “Structural Changes”.

I’ve generated a PDF of the slides and made it available on slideshare.net. I’ve also uploaded the 45 minute screencast to my blip.tv channel.

Let me know if you find them useful.

This year I’d like to spend some time talking about how interpret the raw information and using it to guide code changes. Approaches like common sub-expression elimination and moving invariant code out of loops are straight-forward. They’re ‘low hanging fruit’ with no API changes involved. Good for a first-pass through the code.

Moving loops down into lower-level code is an example of a deeper change I’ve found useful. There are many more. I’d like to collect them to add to the talk and the NYTProf documentation.

So here’s a question for you: after looking at the NYTProf report, how did you identify what you needed to do to fix the problems?

I’m interested in your experiences. How you used NYTProf, how you interpreted the raw information NYTProf presented, and then, critically, how you decided what code changes to make to improve performance. What worked, what didn’t. The practice, not the theory.

Could you to take a moment to think back over the times you’ve used NYTProf, the testing strategy you’ve used, and the code changes you’ve made as a result? Ideally go back and review the diffs and commit comments.

Then send me an email — tell me your story!

The more detail the better! Ideally with actual code (or pseudo-code) snippets².

1. OSCON is in San Jose this year, July 20-24th. You can use the code ‘os09fos’ to get a 20% discount.
2. Annotated diff’s would be greatly appreciated. I’ll give credit for any examples used, naturally, and I’ll happily anonymize any code snippets that aren’t open source.