I have been experimenting to see how much of the Firefox GUI is actually required vs that which is just fluff. As requested by @tsuki_chama, here is a list of plugins and config I am using.
1. Start with Firefox 4. On Ubuntu Lucid / Maverick you can use “sudo apt-add-repository ppa:mozillateam/firefox-stable” and then do an upgrade.
2. Add Pentadactyl and set the GUI options to disable nearly everything. (for reference, here is my config).
3. Grab the Firefox extension called “Hide Caption Titlebar Plus” and configure as desired. To be even more skimpy you can leave off the min/max/close buttons and just use alt+space.
4. Compiz is needed to remove the window border (idea stolen from here). Edit the settings for the “Window Decoration” pluging and set “Decoration windows” field to “any & !(role=browser & class=Firefox)”.
bonus: Add the Firefox theme Grey and Black.