7.15.2004

Yet another edit to the mp3 blog grabbing script

So I was sitting around thinking, "How could I possibly beat this dead horse and make it even uglier in the process" when it dawned on me. I could not only do it, but I could do it and it would actually be a Good Thing.

This one should drastically cut down the time required to search through the mp3s for new ones. Instead of having one huge list of all the songs you've ever downloaded, it now gets a list of all songs available on all the blogs when it's ran and uses that list the next time it's ran. Sort of a rotating log kinda thing based on the assumption that once the songs scroll off the blogs' front page, there's no reason to keep a record of them anymore.

Note: If you wanted to keep a record of every song you've downloaded and what site it came from, just delete the "#" on the eighth line from the bottom (where it says "#| tee -a masterlist;"). It won't be a pretty list, but it will be searchable at least.

#! /bin/sh
echo 'Visiting blogs.....'
function fetch {
echo ------------------- >> .songtemp
echo Site visited: $1 >> .songtemp
echo Music downloaded: >> .songtemp
lynx -dump $1 | grep "http" | awk '{$1=""; print }' | grep "mp3$\|zip$" | sed 's/^[ \t]*//' | grep http >> .songtemp;
}

##### ---------------bloglist--------------- #####
# add blogs here, one to a line, preceded by "fetch"
fetch http://villa-straylight.blogspot.com/
fetch http://newflux.blogspot.com/
fetch http://tofuhut.blogspot.com/
fetch http://www.mysticalbeast.blogspot.com/
fetch http://www.tangmonkey.com/blogs/music/
fetch http://thenewpink.net/womenfolk/
fetch http://amillionlovesongs.blogspot.com/
fetch http://www.londonlee.com/blog.html/
fetch http://www.sixeyes.blogspot.com/
fetch http://losingmyedge.com/
##### ---------------bloglist--------------- #####

cat .songtemp | grep http | grep "mp3$\|zip$" >> .tmp
echo 'Checking for new music.....'
cat ./.tmp | sed 's/[ ^t]*$//' | grep -v -f ./.downloaded > .tmp2;
wget -N -T 30 -A.mp3,.zip -i ./.tmp2;
echo
cat .songtemp | grep -v -f ./.downloaded | perl -pi -e s/\%20/\ /g #| tee -a masterlist;
rm ./.downloaded
cat ./.tmp >> ./.downloaded;
rm ./.tmp;
rm ./.tmp2;
rm .songtemp;
echo -------------------
echo Done



<< Home

This page is powered by Blogger. Isn't yours?