I kind of hate to recommend something that costs a lot... but, well, that's what I use. In other words, Adobe Creative Suite. It costs a bunch, but given the work you seem to be doing, it might well be worth it.
The Bridge app is great for previewing things. I don't do stills much, so I'm not an expert, but it seems to have what you want. You can rate items 1-5 stars, or leave them unrated, or flag them as rejected (in the Label menu bar). Then, you can filter to show only items with a minimum number of stars; or only unrated items; or only rejected items. Having filtered, you have all the usual file management tools available, like delete. Of course you can jump into Photoshop / Camera Raw. For stills, I use Bridge instead of folders, not as well as. There are also labels that you can add. It seems good for stills, but not so great for video -- it can preview MXF files, but not CRM or MP4??? Maybe I don't have all the right codecs installed.
For video, I just load all my rushes into Premiere Pro. I then have a Metalogging workspace set up where I can preview the videos, type in comments, set the "Good" flag, etc. Works pretty well. I then organise the clips into bins, and then go into the Editing workspace.
For video, seeing as it's generally so precious, the default rule is that you never delete rushes -- you never know what might turn out to be useful, even if only for a blooper reel. However, I've been doing a huge landscape video project (just for myself) for the last year, and yes, I have deleted a lot of rushes -- just manually from the folders, once I'm sure I don't need them. This is tedious, but we're talking handfuls of files, not thousands.
