Here's a good article that will allow you to reduce and cut-down on memory usage in Firefox. Maybe you're looking for a solution that will affect the memory of Firefox as downloads are in progress, but why not reduce the overall memory usage while at it?

I guess it's just the way Firefox works. It seems to download files into memory, then flush the data to disk. As soon as I start a download, I can watch the memory usage slowly go upwards until the download completes, then the memory is released. Other browsers, like Internet Explorer, do not do this. Just curious if there was a tweakable about:config option.

The main issue with this, is that if Firefox uses up all available memory and would normally go to swapping into paged memory, the browser will lock up and I have to force kill the application. This is a problem when downloading things like ISO images.