Ok, I found it, and will write it here, if anybody needs this later.
Create a template for the downloads (ce_downloads) and in the foreach loop, the anchor should be:
PHP Code:
<a href="<?php echo preg_replace("/.*file\=(.*)/", "$1", $file['href']); ?>" title="<?php echo $file['title']; ?>" onclick="return !window.open(this.href)">
This will remove the the 'file' get parameter from the link, and will open in a new window.
If you need to support .html files, too (how was in my case) you need to edit your .htaccess.
Bookmarks