Venkman 0.9.88.1 (or, why your scripts might seem to be gone)

A few days ago, the kind people at AMO approved version 0.9.88 of the JavaScript Debugger Venkman. The last release was… 2 years ago! Funnily enough, some users immediately found a show-stopping issue on Firefox 4, which is fixed in Venkman 0.9.88.1, which is currently waiting for review now also available on AMO (thanks to Nils Maier for reviewing!).

So what changed, you wonder? Well, obviously we fixed some bugs. I’d like to briefly discuss just one of them, because it has some repercussions for how you use Venkman:

We no longer start the debugger service when the browser starts (by default)

There were several bugs on file about how Venkman and Firebug made Firefox slower. On top of that, there were recent changes in how components were handled, which meant we could no longer start Venkman (and ask for the debugger service to start) as early as we used to. In fact, the native support from the debugger service to start “really early” was removed by some of the Firefox/Gecko developers as part of this effort. Venkman relied on this support (Firebug doesn’t, they do something similar from their own code).

Why was it important to start the debugger service early?

Because the debugger service only “knew” about functions, variables and script that was loaded when it was started. In other words, if you loaded a script, and then loaded the debugger service after doing so, you wouldn’t be able to debug those files.

So if you no longer start the debugger service early, how does this work?

Thanks to some great work by Wladimir Palant (of AdblockPlus fame), as of Firefox 3.5 we can also get the debug information about functions when they are called. So, when you start Venkman, we will start the debugger service, and for every JS function call, it will tell Venkman about the new code.

I don’t see my scripts or functions when I open Venkman. How do I debug my code?

You have two options:

  • You can load the file (if it’s not shown in the scripts view, try File > Open File (local files) or Open Web Location (urls)) and then set a “future” breakpoint. Venkman will stop there when the code is executed. Instead of loading the file manually, keep in mind any call into a function in the file will make it appear in the scripts list. It shouldn’t be hard to load the file.
  • You can reload the page, and Venkman will see all the scripts appear just like it used to.

Can I get the old behaviour back?

We would prefer it if you tried to work with Venkman the way it is now and let us know about things you think we should do differently or improve. If you’re unable to debug code now that you could debug fine before, please let us know!

If you really need to start the debugger service early, just enter “/startup-init true” (without quotes) at the Venkman console, and we will start the debugger service early (specifically, at the ‘profile-after-change’ notification – slightly later than before, but for most usecases this should not matter).

What about Firebug?

There’s a Firebug 1.6a20 alpha release that is compatible with Firefox 4. I don’t know if they still start the debugger service early if that pane is enabled.

Moving from CVS to Mercurial

Yesterday, Venkman moved from CVS to Hg. For the benefit of others who may want to move other extensions, here is a rough attempt to outline the steps I took.

First, some notes of caution:

  • HG uses a character encoding specific to your machine. For my mac, that turned out to be mac-roman. You usually don’t want that, and need to tell it you want it to use utf8 instead. I could actually not convince it to do this for the author map – it stored the authors in mac-roman anyway. Check your imports carefully.
  • HG’s default ConvertExtension is your friend, as long as you tell it what to do the right way.
  • You probably want to create a test repo (I ended up using several, in trial and error!) at hg.mozilla.org/users. Instructions for doing that may be found here. Keep in mind that once you stick something in a Mercurial repo, it will be there forever. This means anyone pulling from your repo will have to keep downloading all those files. Be careful about what you push to the repo, because once you’ve pushed something there’s no going back (short of asking IT to delete the repo and trying again).

Alright. Down to the actual import. Here is the actual steps I ended up following. When I say “ended up” I mean “after trying N different things which didn’t work”, where N was large enough to keep me busy for a fair number of hours.

  1. Check out the relevant code from CVS. In my case, this meant:
    cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/extensions/venkman
  2. Enable the convert and mq extensions. Edit your ~/.hgrc file and include:
    [extensions]
    hgext.convert=
    mq =
  3. Decide whether you want all the branches in CVS. If, like me, you’re importing an extension, you probably don’t care about the Firefox/Mozilla release branches. You just want trunk history. In order to do this, we ask the convert extension to split up branches. Edit your ~/.hgrc file and include:
    [convert]
    hg.usebranchnames=0
    hg.clonebranches=1
  4. Now you can run convert, more or less like this:
    hg --encoding utf8 convert vnkCVS/mozilla/extensions/venkman venkman-initial
  5. Inspect your handiwork: you will now end up with several directories, one for each branch. You presumably want “default”, which should correspond to trunk. We will convert from hg to hg in a bit, to obtain just that.  Go back to your ~/.hgrc file and remove the [convert] section you added in step 3. This way, the new repo we’ll convert to won’t still have the “default” directory.
  6. Additionally, we still need to map authors. CVS used something like: “foo%somecompany.com”, and in Mercurial, we expect something of the form “John Doe <johndoe@mozilla.com>”. So we need to define an author map file, in which each line simply maps one set of authors to the next. For example:
    johndoe%mozilla.com = John Doe <johndoe@mozilla.com> . I uploaded the Author map for Venkman CVS to hg import that I used. It probably does not cover everyone who committed to your repo. If there are aliases that you’re not familiar with (Mozilla CVS is pretty old!) then Google and asking on IRC are easy ways of figuring out who’s who. To create a complete author map, I used some ad-hoc sed magic on:

    hg log | grep "user:" > foo.txt
    
  7. You may want to unify the entries of committers who have committed using different committer IDs. Ohloh can help there.
  8. Run hg convert again:
    hg --encoding utf8 convert --authors myAuthorMap.txt venkman-initial/default/ venkman-final/
  9. Check hg log. If there are loads of tags in which you’re not interested, you can use the hg strip command to strip the last hg revision, which added all the tags.
  10. Verify everything worked. Then add the correct “default-push” line to the .hg/hgrc file for the new repo you created, and push to your user repo. Check that everything is correct by reviewing the hgweb overview of things. If so, change the default-push line to point to the “real” repo, and push all the changesets there. You’re done!

This may not be the easiest or best way to do things, but that’s the way I managed – suggestions/improvements appreciated!

SightCity Frankfurt, ChatZilla release, Privacy, Venkman issues, misc.

It’s been way too long since I posted anything here, for which I apologize. There are a couple of things that deserve mention here at the present time.

SightCity

I recently got back from visiting SightCity in Frankfurt am Main, Germany. The conference experience was excellent, and I had a great time with Steve, Ben and Marco, all of whom have written more pleasantly and/or extravagantly about our experiences there than I ever could. So this is all you’re getting from me, here. 🙂

ChatZilla Releases

ChatZilla 0.9.82 was released, quickly followed by ChatZilla 0.9.82.1 after a couple of nasty regressions. Almost all of the releases’ features and fixes fall squarely into the “polish” bin (eg. dragging tabs for channels named only “#” now works, and doubleclicking a user in the userlist opens a query tab, some minor accessibility and localizability fixes, etc.), except for one: CEIP, short for Customer Experience Improvement Program. Customer not being very apt, I suppose, but it’s a standard name for what is essentially a data collecting tool.

Data collecting?! Yes, data collecting. I wrote a Privacy Policy about what we do. Please read it first, before flaming me/us. Really, read it – it’s quite short! As you will have read by now, I hope, we ask first if it would be OK to collect anonymous statistics. Without consent, ChatZilla doesn’t save anything you do, nor (consequently) does anything get sent. We never collect anything personal, and both in the policy and in the UI, which is accessible from the Help menu in ChatZilla, we indicate a bunch of things we specifically won’t collect. So what can we collect, then, and how is it useful? Well, examples include the length of sessions (do people run ChatZilla for days at a time without interruption, or only five minutes?) and how tabs are handled. We can already, after just a few days, see that many people seem to be closing lots of network tabs, which may lead us to prioritize bug 249188, for instance.

So I hope that this post helps clarify that we’re not turning evil. If you disagree, and had already turned it on, you’re free to turn it off again, at your leasure. Do let us know what you think we’re doing wrong, though.

Venkman trunk issue

If you’re using Venkman with a Gecko trunk product (eg. Firefox 3 RC1, Thunderbird Shredder 3.0a1, …) you may have found that viewing source code stopped working a while ago. This happened because of a change in the way unprivileged content, like the source view itself (which is plain old HTML) is allowed to access chrome content (like the stylesheet for the source view, unfortunately). Fixing it properly is not trivial. Right now, I have suggested a more or less wallpaper fix, because I am too busy to do something nicer, and it doesn’t seem like anyone else is willing to go and fix it instead. This basically allows unprivileged content to access the chrome content again. A better solution would be to channel the stylesheet through the jsd protocol. If anyone wants to step up to the plate and fix that, that’d be awesome. In the meantime, the wallpaper patch is waiting for review. If you’re in need of a working Venkman, I uploaded an XPI to bug 428848.

Misc

I’m nearing the end of my BSc degree. I’m working on my thesis at the moment, and finishing off the two courses that remain. Perhaps I’ll write more about the thesis once I have something I can demo or screenshot in-action. For now, I’d just like to happily announce that I was conditionally accepted into the 1 year MSc Advanced Computing course at Imperial College, London. So, if you know of a good place for a grad student to live in central London (South Kensington), let me know!

Venkman back amongst the living

Venkman 0.9.87.2 was made public by an addons.mozilla.org editor a few hours ago. It was released a few days after 0.9.87.1, which turned out to have one major regression: adding more than one watch expression through the “Watches” UI no longer worked. This regression had been present on trunk versions of Venkman since June this year. That is 5 solid months of nobody noticing major pieces of functionality being completely broken. As far as I’m concerned, there are two important lessons to be learned:

  • The lack of a testing framework means we don’t catch regressions immediately. I don’t have time to fix this, as I’m a full time student whose major Mozilla commitments are elsewhere anyway. If anyone is interested in making this happen, let me know and I’ll try to get you started.
  • Hardly anybody uses trunk versions of Venkman. I guess the set of users includes trunk SeaMonkey users, but that’s about it. This, too, is really really bad for quality. I’m guessing some sort of nightly update system would help alleviate this, but to my knowledge it is impossible to have more than one “update channel” for an add-on, short of changing the add-on’s ID, of course…

Finally, something I’ve said before and am saying again now: Please report problems. There’s a bug database, it’s there for a reason. If you have clear steps to reproduce, I promise to try to give the issue attention, but I need to hear about it! Too often I hear people complain that “I could never get it to work” or “It randomly breaks”. That sucks, but if things don’t get more specific than that I’m not going to be able to do anything about it. So, give the latest version a whirl, and let me know about any issues you have with it. Thanks!

Venkman rescue meeting

So yesterday, there was a small gettogether on IRC in #venkman to discuss the future of Venkman. Right now the trunk builds are not fully functional anymore, and the last checkins were several months ago. So in fact, it’s in a pretty miserable state.

On to more positive things:

  1. There is a raw log of the conversation on IRC for those interested.
  2. WeirdAl and I will be going through the current list of open Venkman bugs today (2007-09-02) starting 12pm PST (noon) – and we need your help! Please drop by in #venkman on IRC if you want to help out doing triage and all that.
  3. We will try to prioritize the open bugs so we get a reasonably usable trunk build back, and get a new release out ASAP. We can’t currently make promises about how long this will take, as it will depend on how easy it’ll turn out to be to fix some of the current bugs – but we are aiming for a new Venkman release on AMO within about 2 weeks.
  4. We will try to make a todo list on a wiki somewhere (probably wiki.mozilla.org) to keep track of what will need to be done.
  5. We will try to get the MozPad folks involved to support Venkman as it is currently one of the only reasonable debugging tools for chrome-level JavaScript (as Firebug doesn’t work there).

All in all, we hope to get Venkman back to usability and happiness very soon, and to keep it there. If you can help out, please do! Drop by in #venkman, leave a comment here, or on WeirdAl’s weblog. Any help is much appreciated!