Mercurial > rinkhals

annotate data/sounds/get-sources @ 93:1fd56b625b24

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result Added some nicely licensed chicken and rooster samples from OLPC
author David Fraser <davidf@sjsoft.com>
date Wed, 02 Sep 2009 11:19:18 +0000
parents a5ce010f9fb4
children 55105d3bdab1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
1 #!/usr/bin/env python
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
2
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
3 """Script to fetch sources defined in sources.txt"""
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
4
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
5 import ConfigParser
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
6 import logging
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
7 import os
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
8 import subprocess
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
9 import urllib
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
10 import urllib2
93
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
11 import zipfile
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
12 try:
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
13 import pymedia.audio.acodec as acodec
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
14 import pymedia.muxer as muxer
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
15 except ImportError, e:
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
16 logging.info("pymedia not installed, will use transcode to convert files: %s", e)
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
17 acodec = None
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
18 muxer = None
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
19
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
20 def iter_sources(filename=None):
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
21 """fetches a bunch of sound sources from the descriptions in the given filename"""
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
22 if filename is None:
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
23 filename = os.path.join(os.path.dirname(os.path.abspath(__file__)), "sources.txt")
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
24 source_config = ConfigParser.RawConfigParser()
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
25 source_config.read(filename)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
26 for section in source_config.sections():
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
27 yield section, dict(source_config.items(section))
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
28
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
29 def convert_audio(source_filename, target_filename, source_format, target_format):
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
30 """converts audio between files"""
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
31 logging.info("Converting %s (format %s) to %s (format %s)", source_filename, source_format, target_filename, target_format)
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
32 if not acodec or not muxer:
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
33 logging.debug("pymedia not present: will try use transcode")
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
34 if source_format == "aiff":
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
35 source_format = "mplayer"
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
36 options = ["-y", "null,%s" % target_format, "-i", source_filename, "-o", target_filename]
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
37 if source_format not in ["wav", "mp3", "ogg"]:
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
38 options += ["-x", "null,%s" % source_format]
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
39 subprocess.call(["transcode"] + options)
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
40 return
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
41 source_file = open(source_filename, 'rb')
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
42 s = source_file.read(8192)
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
43 dm = muxer.Demuxer(source_format)
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
44 frames = dm.parse(s)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
45 print dm.hasHeader(), dm.getInfo()
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
46 dec = acodec.Decoder(dm.streams[0])
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
47 frame = r[0]
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
48 r = dec.decode(frame[1])
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
49 print r.sample_rate, r.channels, r.bitrate, r.sample_length
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
50
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
51 params = {
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
52 'id': acodec.getCodecId(target_format),
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
53 'bitrate': r.bitrate,
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
54 'sample_rate': r.sample_rate,
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
55 'ext': target_format,
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
56 'channels': r.channels }
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
57 enc = acodec.Encoder(params)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
58 enc.setInfo(dec.getInfo())
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
59
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
60 target_file = open(target_filename, 'wb')
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
61 while len(s):
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
62 frames = enc.encode(r.data)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
63 target_file.write(enc.mux(frames))
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
64 s = source_file.read(1024)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
65
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
66 r = dec.decode(s)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
67 target_file.write(enc.flush())
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
68 target_file.close()
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
69
88
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
70 def lazy_login(options):
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
71 """performs a lazy login for the given options"""
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
72 if not options.get("lazy", False):
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
73 # this login has already happened
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
74 return
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
75 options["lazy"] = False
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
76 options = options.copy()
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
77 url = options.pop("url")
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
78 params = urllib.urlencode(options)
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
79 logging.info("Logging in to %s", url)
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
80 f = opener.open(url, params)
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
81 contents = f.read()
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
82 f.close()
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
83
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
84 def handle_logins(config):
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
85 """logs in to necessary sites and returns urllib2 opener with cookies set up"""
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
86 opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
88
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
87 opener.weblogin = {}
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
88 urllib2.install_opener(opener)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
89 for section in config.sections():
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
90 options = dict(config.items(section))
88
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
91 opener.weblogin[section] = options
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
92 opener.weblogin[section]["lazy"] = True
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
93 return opener
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
94
93
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
95 def extract_archive(archive_filename, archive_member, target_filename):
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
96 """extracts file from an archive"""
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
97 archive = zipfile.ZipFile(archive_filename)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
98 contents = archive.read(archive_member, "rb")
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
99 target_file = open(target_filename, "wb")
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
100 target_file.write(contents)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
101 target_file.close()
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
102
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
103 if __name__ == "__main__":
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
104 logging.getLogger().setLevel(logging.INFO)
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
105 target_dir = os.path.dirname(os.path.abspath(__file__))
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
106 web_config = ConfigParser.RawConfigParser()
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
107 web_config.read(os.path.join(target_dir, "web.ini"))
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
108 opener = handle_logins(web_config)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
109 for filename, source_options in iter_sources(os.path.join(target_dir, "sources.txt")):
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
110 download_filename = filename
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
111 orig_ext = source_options.get("originalextension", None)
93
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
112 archive_ext = source_options.get("archiveextension", None)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
113 is_archive = archive_ext
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
114 if archive_ext:
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
115 download_ext = archive_ext
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
116 else:
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
117 download_ext = orig_ext
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
118 url = source_options["url"]
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
119 target_name, target_ext = os.path.splitext(filename)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
120 target_ext = target_ext.lstrip(".").lower()
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
121 if is_archive:
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
122 download_filename = url[url.rfind("/")+1:]
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
123 elif download_ext:
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
124 download_filename = target_name + "." + download_ext
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
125 download_filename = os.path.join(target_dir, download_filename)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
126 if not os.path.exists(download_filename):
88
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
127 if "weblogin" in source_options:
a5ce010f9fb4 Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents: 87
diff changeset
128 lazy_login(opener.weblogin[source_options["weblogin"]])
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
129 logging.info("Downloading %s to %s", url, download_filename)
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
130 contents = opener.open(url).read()
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
131 if "<html" in contents[:1024].lower():
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
132 logging.error("%s returned HTML rather than file contents", url)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
133 continue
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
134 f = open(download_filename, "wb")
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
135 f.write(contents)
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
136 f.close()
93
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
137 target_filename = os.path.join(target_dir, filename)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
138 if not os.path.exists(target_filename):
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
139 if is_archive:
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
140 if not orig_ext:
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
141 archive_filename = source_options.get("archivemember", filename)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
142 orig_ext = os.path.splitext(archive_filename)[1].lstrip(".")
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
143 archive_filename = target_name + "." + orig_ext
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
144 source_filename = os.path.join(target_dir, archive_filename)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
145 if not os.path.exists(source_filename):
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
146 extract_archive(download_filename, source_options.get("archivemember", filename), source_filename)
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
147 else:
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
148 source_filename = download_filename
87
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
149 orig_format = source_options.get("originalformat", orig_ext)
615396b21744 Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents: 85
diff changeset
150 target_format = source_options.get("targetformat", target_ext)
93
1fd56b625b24 Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents: 88
diff changeset
151 convert_audio(source_filename, target_filename, orig_format, target_format)
85
782e45d70ea8 Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff changeset
152