blob: 2de537d423e4ab5c77061c591937fd67dae1a7c1 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 1"
date="2018-03-06T20:33:44Z"
content="""
only marginally related since may be the abstraction level is at different level? but thought to add thoughts on \"bundling\" also calls to get stuff from web remote:
although both curl and wget do allow for multiple URLs to be specified within a single invocation (which manual also recommends for download of multiple files since avoids reestablishing the session) I could not find any way to provide those URLs with target filenames as it would be needed for git-annex to store into corresponding target key files. The only way would be to parse wget output (which IIRC git annex already does for progress reporting), get the target filename and move into a proper location
[[!format sh \"\"\"
$> wget -iwget-links -nv --show-progress --clobber
highres001.nii.gz?ve 100%[===================>] 5.40M 2.30MB/s in 2.3s
2018-03-06 15:31:49 URL:http://openneuro.s3.amazonaws.com/ds000001/ds000001_R1.1.0/uncompressed/sub001/anatomy/highres001.nii.gz?versionId=8TJ17W9WInNkQPdiQ9vS7wo8ZJ9llF80 [5663237/5663237] -> \"highres001.nii.gz?versionId=8TJ17W9WInNkQPdiQ9vS7wo8ZJ9llF80.2\" [1]
inplane001.nii.gz?ve 100%[===================>] 653.88K 1.10MB/s in 0.6s
2018-03-06 15:31:50 URL:http://openneuro.s3.amazonaws.com/ds000001/ds000001_R1.1.0/uncompressed/sub001/anatomy/inplane001.nii.gz?versionId=ystKDnaPkdzSwzdRPZH0PtMMknZJCQV4 [669578/669578] -> \"inplane001.nii.gz?versionId=ystKDnaPkdzSwzdRPZH0PtMMknZJCQV4.1\" [1]
FINISHED --2018-03-06 15:31:50--
Total wall clock time: 3.2s
Downloaded: 2 files, 6.0M in 2.9s (2.06 MB/s)
\"\"\"]]
"""]]
|
