Debugging chrome JS and mochitests

Update: if you’ve enabled the correct preferences, you can now use the --jsdebugger commandline flag to open the debugger immediately! You can also use this flag with mochitests. For more info, see my new post.

Earlier today, when I was still sound asleep, some of the students from the MSU capstone project that Jared and I are mentoring asked about debugging their add-on’s JS. Rather than just answering them, I figured I’d write a blogpost as perhaps the information is useful to others as well.

While there is ongoing work to make debugging add-ons built with the SDK a more natural experience, you can already debug browser and add-on JS alike using the Browser Debugger. In fact, while it’s not very intuitive (and again, there’s a bug for making that better) you can also use the Browser Debugger to debug mochitests. I have already been using this method to figure out what broke in recent test failures on the UX branch, and am pleased to have something more robust than dump statements to figure those out.

Generally, here is how you start using the Browser Debugger:

  1. Open the Firefox Developer Tools Toolbox (Tools > Web Developer > Toggle Tools in the menus)
  2. Click the little ‘gear’ icon in the toolbox to open the settings page.
  3. Toggle the “Enable chrome debugging” and “Enable remote debugging” checkboxes. (you may need to scroll down in the toolbox pane to see these)
  4. You can now close the toolbox, and either restart Firefox as the checkboxes suggest… or just open a new window.
  5. There will now be a “Browser Debugger” item in the Web Developer submenu under Tools (sadly, there is no keyboard shortcut). Use it to open the debugger.
  6. The Browser Debugger starts another Firefox process to remotely debug your existing Firefox process, and your existing one will therefore prompt you whether you want to allow access to the debugger (as doing so essentially gives full control of the debuggee to the debugger). Click “OK” in this dialog.

Now the browser debugger’s list of source files should start filling up. You can use the debugger much like you’d use the web debugger that is in the Toolbox: filter for sources, or even search through fulltext of sources to quickly find what you need, set (conditional) breakpoints, add watches, etc.

For mochitests, the situation is much the same except that, if you’re like me, you’ll quickly get annoyed because you get to go through the steps of enabling and starting the debugger manually every time. So I filed a bug and attached a patch to at least eliminate the opening of new windows, but got stuck trying to get it to pass all the tests.

In any case, you can start your test(s) with the --no-autorun option, and then go through the steps outlined above (opening a new window instead of restarting). Make sure you close the toolbox and the extra window, as these might otherwise interfere with running your tests. When you’re done setting your breakpoints and such, click the button in the mochitest window to start running your test(s). Happy debugging!

You don’t know what you’ve got ’till it’s gone – or, tab load detection gone wrong

TL;DR: if you own/wrote code that adds an event listener for the "load" event on a tab, doublecheck that you’re listening for that event on a tab’s <browser> element, and not on the <tab> (eg. the value returned by gBrowser‘s selectedTab property).

In the past two weeks some of us on the Firefox front-end team have been working to get the UX tinderbox builds green again. When we initially started work on Australis, we broke a couple of tests and in the interests of moving forward those were not immediately fixed – we were on a branch, out of the way of everyone else, and it wasn’t our number one priority. Over time this resulted in pretty orange trees, and so we recently moved to fix all the tests we’d broken and return the tree to a functioning, green state.

Although some tests did catch bugs on our side, in general we found subtle bugs in existing tests, which we have (helped) fix both on mozilla-central and the UX branch. One of those that I figured deserved a blogpost of its own was a pattern that tended to look roughly like this:

let tab = targetBrowser.selectedTab;
let win = tab.linkedBrowser.contentWindow;
let expectedReadyState = aURL == "about:blank" ? ["interactive", "complete"] : ["complete"];

if (aOnload) {
  let handler = function() {
    if (tab.linkedBrowser.currentURI.spec != aURL ||
        expectedReadyState.indexOf((win.document || {}).readyState) == -1) {
      return;
    }
    tab.removeEventListener("load", handler, false);
    executeSoon(aOnload);
  }
  tab.addEventListener("load", handler, false);
}

Unfortunately this code broke with the Australis changes, which include removing the empty favicon for pages which don’t have one. The bug in the code may not be immediately obvious, but what the code was doing was adding an event listener on a tabbed browser’s <tab> element (the leaf in the tabstrip at the top), rather than the corresponding <browser> (containing the actual page). Why did that work at all? Well, the tab element contains an image: the page’s favicon. When that loaded, a load event bubbled up. As we’re removing the favicon if the page doesn’t have one, rather than replacing it with an empty placeholder as we do on current versions of Firefox, no such event fires for pages without a favicon, and the relevant tests/code broke completely in Australis (for some of these tests, it is quite possible they caused occasional random orange before).

The manual checking of the page’s readyState is a clue that this isn’t the best way to detect when the page has loaded. Instead of using tab.addEventListener, use tab.linkedBrowser.addEventListener (and the same for removeEventListener, of course).

I’ve attempted to audit the mozilla-central tree for this kind of code, but I may not have been thorough enough by just grepping for things like tab.addEventListener. If you have written code depending on tab loads, be it in tests, or add-ons, or elsewhere, doublecheck how you detect these events!

15 years of Mozilla – and my first day

Today, Mozilla celebrates its 15th birthday. By a cunning plan sheer coincidence, today is also my first day as an employee, after being involved with the project for a little over 8 years. I’m extremely happy with everything Mozilla achieved in the years behind us, and after a period of being a little less active, I’m pleased to be able to help fulltime in the time to come.

I’m in the Firefox Desktop team, and I’m excited! I’ll (continue to) see all of you around, I’m sure (and will be in Mountain View between April 8th and 20th, if that helps).

Venkman 0.9.88.1 (or, why your scripts might seem to be gone)

A few days ago, the kind people at AMO approved version 0.9.88 of the JavaScript Debugger Venkman. The last release was… 2 years ago! Funnily enough, some users immediately found a show-stopping issue on Firefox 4, which is fixed in Venkman 0.9.88.1, which is currently waiting for review now also available on AMO (thanks to Nils Maier for reviewing!).

So what changed, you wonder? Well, obviously we fixed some bugs. I’d like to briefly discuss just one of them, because it has some repercussions for how you use Venkman:

We no longer start the debugger service when the browser starts (by default)

There were several bugs on file about how Venkman and Firebug made Firefox slower. On top of that, there were recent changes in how components were handled, which meant we could no longer start Venkman (and ask for the debugger service to start) as early as we used to. In fact, the native support from the debugger service to start “really early” was removed by some of the Firefox/Gecko developers as part of this effort. Venkman relied on this support (Firebug doesn’t, they do something similar from their own code).

Why was it important to start the debugger service early?

Because the debugger service only “knew” about functions, variables and script that was loaded when it was started. In other words, if you loaded a script, and then loaded the debugger service after doing so, you wouldn’t be able to debug those files.

So if you no longer start the debugger service early, how does this work?

Thanks to some great work by Wladimir Palant (of AdblockPlus fame), as of Firefox 3.5 we can also get the debug information about functions when they are called. So, when you start Venkman, we will start the debugger service, and for every JS function call, it will tell Venkman about the new code.

I don’t see my scripts or functions when I open Venkman. How do I debug my code?

You have two options:

  • You can load the file (if it’s not shown in the scripts view, try File > Open File (local files) or Open Web Location (urls)) and then set a “future” breakpoint. Venkman will stop there when the code is executed. Instead of loading the file manually, keep in mind any call into a function in the file will make it appear in the scripts list. It shouldn’t be hard to load the file.
  • You can reload the page, and Venkman will see all the scripts appear just like it used to.

Can I get the old behaviour back?

We would prefer it if you tried to work with Venkman the way it is now and let us know about things you think we should do differently or improve. If you’re unable to debug code now that you could debug fine before, please let us know!

If you really need to start the debugger service early, just enter “/startup-init true” (without quotes) at the Venkman console, and we will start the debugger service early (specifically, at the ‘profile-after-change’ notification – slightly later than before, but for most usecases this should not matter).

What about Firebug?

There’s a Firebug 1.6a20 alpha release that is compatible with Firefox 4. I don’t know if they still start the debugger service early if that pane is enabled.

ChatZilla and Firefox 3.5

Just a short note, quite some time ago some trunk changes broke ChatZilla. Accessibility also required some changes. All of these fixes have happened (thanks to dbaron for helping out with the layout fix) – but there has not yet been a release. Seeing as we’re getting close to 3.5, I thought I should note that those affected by these issues (say, people on 3.5b4) can download a nightly ChatZilla build here. New builds appear there after every (set of) change(s) in CVS. This should fix any compatibility issues. If it does not, please file a bug.

Edit: ChatZilla 0.9.85 (released some weeks ago…) has all these fixes, and will work fine with 3.5. Oops! 🙂

Moving from CVS to Mercurial

Yesterday, Venkman moved from CVS to Hg. For the benefit of others who may want to move other extensions, here is a rough attempt to outline the steps I took.

First, some notes of caution:

  • HG uses a character encoding specific to your machine. For my mac, that turned out to be mac-roman. You usually don’t want that, and need to tell it you want it to use utf8 instead. I could actually not convince it to do this for the author map – it stored the authors in mac-roman anyway. Check your imports carefully.
  • HG’s default ConvertExtension is your friend, as long as you tell it what to do the right way.
  • You probably want to create a test repo (I ended up using several, in trial and error!) at hg.mozilla.org/users. Instructions for doing that may be found here. Keep in mind that once you stick something in a Mercurial repo, it will be there forever. This means anyone pulling from your repo will have to keep downloading all those files. Be careful about what you push to the repo, because once you’ve pushed something there’s no going back (short of asking IT to delete the repo and trying again).

Alright. Down to the actual import. Here is the actual steps I ended up following. When I say “ended up” I mean “after trying N different things which didn’t work”, where N was large enough to keep me busy for a fair number of hours.

  1. Check out the relevant code from CVS. In my case, this meant:
    cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co mozilla/extensions/venkman
  2. Enable the convert and mq extensions. Edit your ~/.hgrc file and include:
    [extensions]
    hgext.convert=
    mq =
  3. Decide whether you want all the branches in CVS. If, like me, you’re importing an extension, you probably don’t care about the Firefox/Mozilla release branches. You just want trunk history. In order to do this, we ask the convert extension to split up branches. Edit your ~/.hgrc file and include:
    [convert]
    hg.usebranchnames=0
    hg.clonebranches=1
  4. Now you can run convert, more or less like this:
    hg --encoding utf8 convert vnkCVS/mozilla/extensions/venkman venkman-initial
  5. Inspect your handiwork: you will now end up with several directories, one for each branch. You presumably want “default”, which should correspond to trunk. We will convert from hg to hg in a bit, to obtain just that.  Go back to your ~/.hgrc file and remove the [convert] section you added in step 3. This way, the new repo we’ll convert to won’t still have the “default” directory.
  6. Additionally, we still need to map authors. CVS used something like: “foo%somecompany.com”, and in Mercurial, we expect something of the form “John Doe <johndoe@mozilla.com>”. So we need to define an author map file, in which each line simply maps one set of authors to the next. For example:
    johndoe%mozilla.com = John Doe <johndoe@mozilla.com> . I uploaded the Author map for Venkman CVS to hg import that I used. It probably does not cover everyone who committed to your repo. If there are aliases that you’re not familiar with (Mozilla CVS is pretty old!) then Google and asking on IRC are easy ways of figuring out who’s who. To create a complete author map, I used some ad-hoc sed magic on:

    hg log | grep "user:" > foo.txt
    
  7. You may want to unify the entries of committers who have committed using different committer IDs. Ohloh can help there.
  8. Run hg convert again:
    hg --encoding utf8 convert --authors myAuthorMap.txt venkman-initial/default/ venkman-final/
  9. Check hg log. If there are loads of tags in which you’re not interested, you can use the hg strip command to strip the last hg revision, which added all the tags.
  10. Verify everything worked. Then add the correct “default-push” line to the .hg/hgrc file for the new repo you created, and push to your user repo. Check that everything is correct by reviewing the hgweb overview of things. If so, change the default-push line to point to the “real” repo, and push all the changesets there. You’re done!

This may not be the easiest or best way to do things, but that’s the way I managed – suggestions/improvements appreciated!

Notifications for new issues in Google Code

Dear Lazyweb,

For Chrome List, every now and again people report issues they are having on its Google Code website, using the “Issues List”. This is very useful, and I would like this to continue. However, for some reason, I don’t get email for these issues that people report. I need to manually check every now and then, then star new issues so I get email for them.

Can I change this? I’ve googled around, I’ve gone through all the options in “Administer” three times, but I can’t find it. I would really like to have those emails, though, because right now sometimes I don’t check that list for a few days/weeks/months, and people’s problems don’t get solved…

Thanks!

Back/Forward and JS + XHR + URL Manipulation

Steps to reproduce:

  1. Go to your Facebook inbox page, pick a random message, and start typing a reply
  2. Halfway through, realize you want to read a different message before finishing this one.
  3. ‘Accidentally’ left-click instead of middle-click the link at the top for the inbox.
  4. Hit escape to stop the load so as to be able to still have the text you type

Expected result:

Stop loading the inbox page, get to keep your written stuff

Actual result:

This is JavaScript executing. There’s no stopping it from throwing away your text anymore by returning to the inbox – and what’s worse, the back/forward history is so confused by what’s going on (as you hit escape) that it won’t be able to take you back to the reply page and restore your message to the textbox (as back/forward generally does in Firefox). Congrats, you just lost N minutes worth of typing, and who knows what kind of brilliant ideas!

Question to the Lazyweb: whose fault is this? Can it be fixed? How? (Yes, I’d prefer to just use email all the time, too, but unfortunately that’s not always an option, and I’m sure there are webmail implementations which have similar problems, especially once you turn off automatic draft saving)

Absence & Citation Needed

First things first, as of July 2nd, I will be off to Taizé again. I won’t be back until the start of September, and I’m not sure about the exact date. Because I will have to spend a lot of time taking care of my stay in London at Imperial College after that time, I suppose one could say I will be gone in July, August and September, for all intents and purposes related to Mozilla. And actually, I’m writing my thesis now, so if you have things that you need me to do you can try to still get to me, but it might be too late already. I’ll try, though. Emails and comments here are free, so give it a shot if you need me.

Second, in connection to this thesis thing:

Dear Lazyweb,

I would like a way to make LaTeX do a [citation needed] in true Wikipedia style. I often find myself thinking “if I want to write something like this, I need a reference to back it up”, but searching for a good one for minutes/hours on end breaks up my writing process, so I’d much rather make a note to self and keep writing, especially for things which I am very sure of there must be a reference for. Example: “the POSIT algorithm was conceived in the 1990s by John Doe[citation needed]”.

Yes, I know that by default, LaTeX will do a [?] if I try to reference something that doesn’t exist, but that’s (a) not obvious enough for my taste, and (b) it might be the result of a misspelled BibTeX id or such, and I’d rather not mix the usecases of “deliberately left blank” and “oops, misspelled the citation identifier and/or forgot to re-run BibTeX after adding citations”. I haven’t found an easy way to do this.

Thanks!
Gijs

SightCity Frankfurt, ChatZilla release, Privacy, Venkman issues, misc.

It’s been way too long since I posted anything here, for which I apologize. There are a couple of things that deserve mention here at the present time.

SightCity

I recently got back from visiting SightCity in Frankfurt am Main, Germany. The conference experience was excellent, and I had a great time with Steve, Ben and Marco, all of whom have written more pleasantly and/or extravagantly about our experiences there than I ever could. So this is all you’re getting from me, here. 🙂

ChatZilla Releases

ChatZilla 0.9.82 was released, quickly followed by ChatZilla 0.9.82.1 after a couple of nasty regressions. Almost all of the releases’ features and fixes fall squarely into the “polish” bin (eg. dragging tabs for channels named only “#” now works, and doubleclicking a user in the userlist opens a query tab, some minor accessibility and localizability fixes, etc.), except for one: CEIP, short for Customer Experience Improvement Program. Customer not being very apt, I suppose, but it’s a standard name for what is essentially a data collecting tool.

Data collecting?! Yes, data collecting. I wrote a Privacy Policy about what we do. Please read it first, before flaming me/us. Really, read it – it’s quite short! As you will have read by now, I hope, we ask first if it would be OK to collect anonymous statistics. Without consent, ChatZilla doesn’t save anything you do, nor (consequently) does anything get sent. We never collect anything personal, and both in the policy and in the UI, which is accessible from the Help menu in ChatZilla, we indicate a bunch of things we specifically won’t collect. So what can we collect, then, and how is it useful? Well, examples include the length of sessions (do people run ChatZilla for days at a time without interruption, or only five minutes?) and how tabs are handled. We can already, after just a few days, see that many people seem to be closing lots of network tabs, which may lead us to prioritize bug 249188, for instance.

So I hope that this post helps clarify that we’re not turning evil. If you disagree, and had already turned it on, you’re free to turn it off again, at your leasure. Do let us know what you think we’re doing wrong, though.

Venkman trunk issue

If you’re using Venkman with a Gecko trunk product (eg. Firefox 3 RC1, Thunderbird Shredder 3.0a1, …) you may have found that viewing source code stopped working a while ago. This happened because of a change in the way unprivileged content, like the source view itself (which is plain old HTML) is allowed to access chrome content (like the stylesheet for the source view, unfortunately). Fixing it properly is not trivial. Right now, I have suggested a more or less wallpaper fix, because I am too busy to do something nicer, and it doesn’t seem like anyone else is willing to go and fix it instead. This basically allows unprivileged content to access the chrome content again. A better solution would be to channel the stylesheet through the jsd protocol. If anyone wants to step up to the plate and fix that, that’d be awesome. In the meantime, the wallpaper patch is waiting for review. If you’re in need of a working Venkman, I uploaded an XPI to bug 428848.

Misc

I’m nearing the end of my BSc degree. I’m working on my thesis at the moment, and finishing off the two courses that remain. Perhaps I’ll write more about the thesis once I have something I can demo or screenshot in-action. For now, I’d just like to happily announce that I was conditionally accepted into the 1 year MSc Advanced Computing course at Imperial College, London. So, if you know of a good place for a grad student to live in central London (South Kensington), let me know!