Today I decided to bite the bullet and use Amarok to do the heavy lifting. I modified the RockBox WPS cover creator script to use album names first, and then fall back to artist title since shell scripting is asinine and one has to jump through way too many hoops to handle special characters common in album titles. It still takes some effort because you can't run it on all the albums at once, so I've only gotten through a fraction of my music.
Amarok is a really useful tool for doing stuff like that, though funny enough, I actually don't like using it to play music. But hey, now I've got some album art, and it looks good!