summaryrefslogtreecommitdiff
path: root/doc/bugs/importfeed_can__39__t_deal_with_pycon_rss_feeds.mdwn
blob: 1cfe85c0f856c0d8ebaa4deb407faba312449562 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
### Please describe the problem.

[[git-annex-importfeed]] cannot reliably import feeds from <pyvideo.org/category/65/pycon-us-2015/rss>

### What steps will reproduce the problem?

    git annex importfeed pyvideo.org/category/65/pycon-us-2015/rss

### What version of git-annex are you using? On what operating system?

5.20141125

### Please provide any additional information below.

It seems related to the fact that the `enclosure` XML item links to the youtube page instead of the actual video, which sounds a little crazy... so maybe the feed is just wrong and this is invalid.

In [this pull request to the conference proceedings repo](https://github.com/RichiH/conference_proceedings/pull/42), i hacked around that problem with a [simple loop](https://github.com/anarcat/conference_proceedings/blob/pycon/PyCon/README.md):

    curl -s http://pyvideo.org/category/65/pycon-us-2015/rss | xmllint --xpath '//enclosure/@url' - | sed 's/url=/\n/g' | xargs git annex addurl --fast

xargs doesn't actually work so well because every once in a while, youtube will return a 403 and the whole pipeline crumbles to the ground. I found this to work better:

    curl -s http://pyvideo.org/category/65/pycon-us-2015/rss | xmllint --xpath '//enclosure/@url' - | sed 's/url="/\n/g;s/"//' | while read url ; do git annex addurl --fast $url ; done

but it's pretty ugly! --[[anarcat]]