I don't think it's possible to disable it after it's loaded. You'd have to reload the page or close it in the top right corner. You can either re-open the preview window or re-enter the url to reload the page without loading Firebug.
Well, couldn't the button do this? "Disabling Firebug" just reloads the page?
Other way it's confusing to have this option "enable firebug", when it's already enabled.
Other way it's confusing to have this option "enable firebug", when it's already enabled.