It needs updating though, and that’s where you can help… Continue reading
I’ve had a great week at OSCON. The talks are excellent but the real value is in the relationships formed and renewed in the “hallway track”. I’m honoured and humbled to be able to call many great people my friends.
My talk on Devel::NYTProf seemed to go well. This year I covered not just NYTProf and the new features in v3 (not yet released) but also added a section on how to use NYTProf to optimize your perl code.
Here’s a quick summary, with links to the slides and screen-cast, and outline what’s still to be done before v3 gets released (getting closer by the day). Continue reading
I’m looking for a CRM system implemented in Perl. As it turns out, so are the Perl Foundation.
So I thought I’d summarize my interpretation of the comments on that thread, as much for my own benefit as yours, and see if this post flushes out any further information.
We’ll start with the smaller/personal projects and work up from there…
I recall other bloggers complaining of unattributed redistribution of their work. Now a site called rapid-dev.net has started redistributing Plant Perl posts, including mine, with an advert at the top.
I wouldn’t mind if the page had clear attribution, but it doesn’t. In fact, at the bottom it says “Author: hoanatwho”.
That doesn’t feel right. Especially as many of my posts, and probably many others from Planet Perl, use the first-person pronoun “I”.
Why does this matter? A couple of months ago Merlin Mann wrote a long but excellent piece that explains why far better than I could.
Nobody but me is allowed to decide why I make things. And — if and when I choose to give away the things that I make — nobody but me is allowed to define how or where I’ll do it. I am independent.
Merlin discusses, with his typical style, the motivations of those who make their work available for free, and the perils of presuming to understand their motives. Although written mostly about bloggers it seems very applicable to authors of Open Source software. For me it echoes how I feel about coding and, to an extent, the freedom that Perl give me to express my thoughts.
Looking at the Planet Perl page I see it has no licence. Perhaps that should be fixed — even if only to say that the license of the feeds being aggregated must be respected.
At OSCON this year1 I’m giving a “State-of-the-art Profiling with Devel::NYTProf” talk. It’ll be an update of the one I gave last year, including coverage of new features added since then (including, hopefully, two significant new features that are in development).
This year I’d like to spend some time talking about how interpret the raw information and using it to guide code changes. Approaches like common sub-expression elimination and moving invariant code out of loops are straight-forward. They’re ‘low hanging fruit’ with no API changes involved. Good for a first-pass through the code.
Moving loops down into lower-level code is an example of a deeper change I’ve found useful. There are many more. I’d like to collect them to add to the talk and the NYTProf documentation.
So here’s a question for you: after looking at the NYTProf report, how did you identify what you needed to do to fix the problems?
I’m interested in your experiences. How you used NYTProf, how you interpreted the raw information NYTProf presented, and then, critically, how you decided what code changes to make to improve performance. What worked, what didn’t. The practice, not the theory.
Could you to take a moment to think back over the times you’ve used NYTProf, the testing strategy you’ve used, and the code changes you’ve made as a result? Ideally go back and review the diffs and commit comments.
Then send me an email — tell me your story!
The more detail the better! Ideally with actual code (or pseudo-code) snippets2.
Ever copied and pasted a chunk from perldoc output and found you were getting mysterious errors from perl? I have.
I’ve learnt to rewrite the ‘-’ characters because although they look like ‘-’ characters they’re really a unicode HYPHEN: U+2010. Some other chars get mangled too, but that’s the most frequent problem for me.
So I was delighted to see a blog post by marcus ramberg called Fixing the POD synopsis in OSX wherein he fingers nroff as being the problem and gives a simple solution:
alias perldoc='perldoc -t'
Trouble is using perldoc -t means you loose the nice bold text that nroff gives you. So I went digging…
It seems the problem only affects people using UTF-8 and that nroff has a -T option that lets you specify an output encoding to use. So if perldoc ran ‘nroff -Tascii’ instead of plain ‘nroff’ that would avoid the hypen problem and let us keep the bold text.
It turns out that perldoc has an option to specify the nroff command to use, so the solution is simple:
alias perldoc="perldoc -n 'nroff -Tascii'"
This is, of course, still a hack. My main worry is that pod docs using non-ascii characters may get mangled. A much better fix would be to arrange for the ascii characters to not get mapped to unicode at all. So I went digging again…
groff -m tty-char ... and tty-char refers to a tty-char.tmac file that defines the character mappings. The groff man pages point me to groff_tmac man pages which tell me I can get groff to look for .tmac files elsewhere by passing a -Mdir option or setting the GROFF_TMAC_PATH environment variable.
I looked at the default file, /usr/share/groff/1.19.2/tmac/tty-char.tmac on my Mac, and… decided it was time to go to sleep! The formatting is probably simple enough but I’m out of tuits.
So, what’s needed is for someone to determine what change is needed to the tty-char.tmac file, or the files it refers to, to avoid unwanted conversions to unicode. Then put a modified file into a directory and either add a -Mdir option to the nroff alias above, or set the GROFF_TMAC_PATH environment variable. Setting the env var has the benefit of ‘fixing’ all the man pages.
So, anyone want to dig deeper? (For all I know the solution can already be found on google…)
It is sad, but inevitable, that the TIOBE index of programming language “popularity” (sic) would be gamed.
Once you start measuring something, and advertising the results, people with an interest in particular outcomes naturally start to look for ways to influence those results. (It’s the Observer Effect writ large.)
The fact that TIOBE’s methodology, which I’ve discussed previously here and here, is simplistic makes it particularly open to gaming. Anyone, or any community, with access to many web pages can simply add the magic phrase “foo programming”, where foo is their language of choice, to get counted.
And it seems that’s exactly what the Delphi community did at the end of 20081. They made a concerted effort and it seems to have paid off. (I’d be very interested in hearing about similar behaviour in other language communities.)
Is that behaviour gaming? The author of the post who exhorted is readers to “Update your Delphi related blog or site to say Delphi programming on every page in visible text (update the template). Stand up and be counted. You can make a difference!” doesn’t seem to think so, as he also said “I am not suggesting we game the system, just that we help TCPI get an accurate count.“
An accurate count of what, exactly? That’s always been the fundamental question with TIOBE. It should be obvious that most web pages that talk about “delphi programming” wouldn’t actually contain the phrase “delphi programming”. The same applies to every other language. That’s the paradox at the heart of the TIOBE Index. And yet, somehow, TIOBE seem to think that counting pages containing the phrase “delphi programming” lets them claim that:
The ratings are based on the number of skilled engineers world-wide, courses and third party vendors.
Eh? How can they possibly defend that claim? Certainly their documented definition doesn’t support it, or even mention it.
I presume they’re thinking that CV’s, job postings, and adverts are most likely to contain the magic phrase. It should be obvious, again, that the number of CV’s, job postings, and adverts referring to a given programming language would naturally only be a small fraction of the total web pages referring to the language. (And only distantly related to the “popularity” of a language.) Yet that “small fraction” is what TIOBE measure and make bold claims about.
The fact that TIOBE is making a comparison based on a small fraction makes it even more troubling that TIOBE CEO Paul Jansen appears to support language communities changing their pages to include the magic “foo programming” phrase. In an email quoted on delphi.org he says:
For your information, I think your action has already some effect. Tonight’s run shows that Delphi is #8 at this moment. There is a realistic chance that Delphi will become “TIOBE’s Language of the Year 2008″
He’s endorsing the artificial insertion of the magic phrase. Clearly this distorts the TIOBE index in favour of language communities that infect as many pages as possible with the magic phrase.
That sure seems like an invitation to game the system! It’s likely to lead to other language communities doing the same, and so to further devaluation of the TIOBE Index.
I have, on a couple of occasions, used the phrase “perl programming” in blog posts for my own amusement, and linked it to my original TIOBE or not TIOBE – “Lies, damned lies, and statistics” post. I haven’t suggested that others do the same. TIOBE’s endorsement of artificial insertion changes that. Now it seems like we’re going to get a dumb “race to the bottom” to see which language community controls the most web pages.
If, as a result, the TIOBE Index is affected significantly, then I simply hope they’ll drop their pretentious claims and state clearly exactly what they’re counting, how they’re doing it, and what it means: not much.