Mercurial > rinkhals
annotate data/sounds/get-sources @ 510:3e4bb2c9556c
Lazy loading of save game snapshots.
author | Simon Cross <hodgestar@gmail.com> |
---|---|
date | Thu, 26 Nov 2009 23:15:04 +0000 |
parents | 55105d3bdab1 |
children |
rev | line source |
---|---|
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
1 #!/usr/bin/env python |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
2 |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
3 """Script to fetch sources defined in sources.txt""" |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
4 |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
5 import ConfigParser |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
6 import logging |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
7 import os |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
8 import subprocess |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
9 import urllib |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
10 import urllib2 |
93
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
11 import zipfile |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
12 try: |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
13 import pymedia.audio.acodec as acodec |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
14 import pymedia.muxer as muxer |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
15 except ImportError, e: |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
16 logging.info("pymedia not installed, will use transcode to convert files: %s", e) |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
17 acodec = None |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
18 muxer = None |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
19 |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
20 def iter_sources(filename=None): |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
21 """fetches a bunch of sound sources from the descriptions in the given filename""" |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
22 if filename is None: |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
23 filename = os.path.join(os.path.dirname(os.path.abspath(__file__)), "sources.txt") |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
24 source_config = ConfigParser.RawConfigParser() |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
25 source_config.read(filename) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
26 for section in source_config.sections(): |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
27 yield section, dict(source_config.items(section)) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
28 |
113
55105d3bdab1
Renamed fire-rifle to fire-machinegun, and added a single-shot rifle
David Fraser <davidf@sjsoft.com>
parents:
93
diff
changeset
|
29 def convert_audio(source_filename, target_filename, source_format, target_format, converter=None): |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
30 """converts audio between files""" |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
31 logging.info("Converting %s (format %s) to %s (format %s)", source_filename, source_format, target_filename, target_format) |
113
55105d3bdab1
Renamed fire-rifle to fire-machinegun, and added a single-shot rifle
David Fraser <davidf@sjsoft.com>
parents:
93
diff
changeset
|
32 if converter == "oggenc": |
55105d3bdab1
Renamed fire-rifle to fire-machinegun, and added a single-shot rifle
David Fraser <davidf@sjsoft.com>
parents:
93
diff
changeset
|
33 subprocess.call(["oggenc", source_filename, "-o", target_filename]) |
55105d3bdab1
Renamed fire-rifle to fire-machinegun, and added a single-shot rifle
David Fraser <davidf@sjsoft.com>
parents:
93
diff
changeset
|
34 return |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
35 if not acodec or not muxer: |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
36 logging.debug("pymedia not present: will try use transcode") |
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
37 if source_format == "aiff": |
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
38 source_format = "mplayer" |
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
39 options = ["-y", "null,%s" % target_format, "-i", source_filename, "-o", target_filename] |
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
40 if source_format not in ["wav", "mp3", "ogg"]: |
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
41 options += ["-x", "null,%s" % source_format] |
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
42 subprocess.call(["transcode"] + options) |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
43 return |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
44 source_file = open(source_filename, 'rb') |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
45 s = source_file.read(8192) |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
46 dm = muxer.Demuxer(source_format) |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
47 frames = dm.parse(s) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
48 print dm.hasHeader(), dm.getInfo() |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
49 dec = acodec.Decoder(dm.streams[0]) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
50 frame = r[0] |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
51 r = dec.decode(frame[1]) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
52 print r.sample_rate, r.channels, r.bitrate, r.sample_length |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
53 |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
54 params = { |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
55 'id': acodec.getCodecId(target_format), |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
56 'bitrate': r.bitrate, |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
57 'sample_rate': r.sample_rate, |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
58 'ext': target_format, |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
59 'channels': r.channels } |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
60 enc = acodec.Encoder(params) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
61 enc.setInfo(dec.getInfo()) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
62 |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
63 target_file = open(target_filename, 'wb') |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
64 while len(s): |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
65 frames = enc.encode(r.data) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
66 target_file.write(enc.mux(frames)) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
67 s = source_file.read(1024) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
68 |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
69 r = dec.decode(s) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
70 target_file.write(enc.flush()) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
71 target_file.close() |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
72 |
88
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
73 def lazy_login(options): |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
74 """performs a lazy login for the given options""" |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
75 if not options.get("lazy", False): |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
76 # this login has already happened |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
77 return |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
78 options["lazy"] = False |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
79 options = options.copy() |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
80 url = options.pop("url") |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
81 params = urllib.urlencode(options) |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
82 logging.info("Logging in to %s", url) |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
83 f = opener.open(url, params) |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
84 contents = f.read() |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
85 f.close() |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
86 |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
87 def handle_logins(config): |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
88 """logs in to necessary sites and returns urllib2 opener with cookies set up""" |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
89 opener = urllib2.build_opener(urllib2.HTTPCookieProcessor()) |
88
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
90 opener.weblogin = {} |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
91 urllib2.install_opener(opener) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
92 for section in config.sections(): |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
93 options = dict(config.items(section)) |
88
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
94 opener.weblogin[section] = options |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
95 opener.weblogin[section]["lazy"] = True |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
96 return opener |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
97 |
93
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
98 def extract_archive(archive_filename, archive_member, target_filename): |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
99 """extracts file from an archive""" |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
100 archive = zipfile.ZipFile(archive_filename) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
101 contents = archive.read(archive_member, "rb") |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
102 target_file = open(target_filename, "wb") |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
103 target_file.write(contents) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
104 target_file.close() |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
105 |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
106 if __name__ == "__main__": |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
107 logging.getLogger().setLevel(logging.INFO) |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
108 target_dir = os.path.dirname(os.path.abspath(__file__)) |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
109 web_config = ConfigParser.RawConfigParser() |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
110 web_config.read(os.path.join(target_dir, "web.ini")) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
111 opener = handle_logins(web_config) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
112 for filename, source_options in iter_sources(os.path.join(target_dir, "sources.txt")): |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
113 download_filename = filename |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
114 orig_ext = source_options.get("originalextension", None) |
93
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
115 archive_ext = source_options.get("archiveextension", None) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
116 is_archive = archive_ext |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
117 if archive_ext: |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
118 download_ext = archive_ext |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
119 else: |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
120 download_ext = orig_ext |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
121 url = source_options["url"] |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
122 target_name, target_ext = os.path.splitext(filename) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
123 target_ext = target_ext.lstrip(".").lower() |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
124 if is_archive: |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
125 download_filename = url[url.rfind("/")+1:] |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
126 elif download_ext: |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
127 download_filename = target_name + "." + download_ext |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
128 download_filename = os.path.join(target_dir, download_filename) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
129 if not os.path.exists(download_filename): |
88
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
130 if "weblogin" in source_options: |
a5ce010f9fb4
Added lazy login capability so we don't login unless downloading
David Fraser <davidf@sjsoft.com>
parents:
87
diff
changeset
|
131 lazy_login(opener.weblogin[source_options["weblogin"]]) |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
132 logging.info("Downloading %s to %s", url, download_filename) |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
133 contents = opener.open(url).read() |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
134 if "<html" in contents[:1024].lower(): |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
135 logging.error("%s returned HTML rather than file contents", url) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
136 continue |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
137 f = open(download_filename, "wb") |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
138 f.write(contents) |
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
139 f.close() |
93
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
140 target_filename = os.path.join(target_dir, filename) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
141 if not os.path.exists(target_filename): |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
142 if is_archive: |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
143 if not orig_ext: |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
144 archive_filename = source_options.get("archivemember", filename) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
145 orig_ext = os.path.splitext(archive_filename)[1].lstrip(".") |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
146 archive_filename = target_name + "." + orig_ext |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
147 source_filename = os.path.join(target_dir, archive_filename) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
148 if not os.path.exists(source_filename): |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
149 extract_archive(download_filename, source_options.get("archivemember", filename), source_filename) |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
150 else: |
1fd56b625b24
Added ability to handle extracting from zip archives after download, although the code is becoming slightly hairy as a result
David Fraser <davidf@sjsoft.com>
parents:
88
diff
changeset
|
151 source_filename = download_filename |
87
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
152 orig_format = source_options.get("originalformat", orig_ext) |
615396b21744
Improved login, added sampleswap-sourced public domain sound, handle aiff conversion, don't parse parameters in config files
David Fraser <davidf@sjsoft.com>
parents:
85
diff
changeset
|
153 target_format = source_options.get("targetformat", target_ext) |
113
55105d3bdab1
Renamed fire-rifle to fire-machinegun, and added a single-shot rifle
David Fraser <davidf@sjsoft.com>
parents:
93
diff
changeset
|
154 converter = source_options.get("converter", None) |
55105d3bdab1
Renamed fire-rifle to fire-machinegun, and added a single-shot rifle
David Fraser <davidf@sjsoft.com>
parents:
93
diff
changeset
|
155 convert_audio(source_filename, target_filename, orig_format, target_format, converter) |
85
782e45d70ea8
Adds directory for sounds, and script to download them based on sources.txt (and web.ini, which contains password info for freesound.org) despite licensing disputes
David Fraser <davidf@sjsoft.com>
parents:
diff
changeset
|
156 |