mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

weird rss feeds with youtube enclosures and unicorns

Browse source
This commit is contained in:
https://id.koumbit.net/anarcat 2015-05-30 06:15:25 +00:00 • committed by admin
parent fac6a81afb
commit 17c013aa92
1 changed files with 25 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

25
doc/bugs/importfeed_can__39__t_deal_with_pycon_rss_feeds.mdwn Normal file
View file

@ -0,0 +1,25 @@
### Please describe the problem.
[[git-annex-importfeed]] cannot reliably import feeds from <pyvideo.org/category/65/pycon-us-2015/rss>
### What steps will reproduce the problem?
git annex importfeed pyvideo.org/category/65/pycon-us-2015/rss
### What version of git-annex are you using? On what operating system?
5.20141125
### Please provide any additional information below.
It seems related to the fact that the `enclosure` XML item links to the youtube page instead of the actual video, which sounds a little crazy... so maybe the feed is just wrong and this is invalid.
In [this pull request to the conference proceedings repo](https://github.com/RichiH/conference_proceedings/pull/42), i hacked around that problem with a [simple loop](https://github.com/anarcat/conference_proceedings/blob/pycon/PyCon/README.md):
curl -s http://pyvideo.org/category/65/pycon-us-2015/rss | xmllint --xpath '//enclosure/@url' - | sed 's/url=/\n/g' | xargs git annex addurl --fast
xargs doesn't actually work so well because every once in a while, youtube will return a 403 and the whole pipeline crumbles to the ground. I found this to work better:
curl -s http://pyvideo.org/category/65/pycon-us-2015/rss | xmllint --xpath '//enclosure/@url' - | sed 's/url="/\n/g;s/"//' | while read url ; do git annex addurl --fast $url ; done
but it's pretty ugly! --[[anarcat]]