mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/doc/git-annex-importfeed.mdwn

118 lines
3.6 KiB
Text
Raw Normal View History

splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
# NAME
git-annex importfeed - import files from podcast feeds
# SYNOPSIS
git annex importfeed `[url ...]`
# DESCRIPTION
convert importfeed to youtube-dl Fully working, including --fast/--relaxed. Note that, while git-annex addurl --relaxed is not going to check youtube-dl, I kept git annex importfeed --relaxed checking it. Thinking is that, let's not break people's importfeed cron jobs, and importfeed does not typically have to check a large number of new items, so it's ok if it's a little bit slower when used with youtube playlist feeds. importfeed's behavior is also improved (?) when a feed has links in it to non-media files. Before, those were skipped. Now, the content of the link is downloaded. This had to be done, because trying to use youtube-dl is slow, and if those were skipped, it would have to check every time importfeed was run. While this behavior change may not be desirable for some feeds, that intersperse links to web pages with enclosures, it will be desirable for other feeds, that have non-enclosure directy links to media files. Remove old quvi modules. This commit was sponsored by Øyvind Andersen Holm.
2017-11-29 21:05:27 +00:00
Imports the contents of podcasts and other feeds. Only downloads files whose
update importfeed man page, including a mention of annex.genmetadata
2015-03-31 17:48:13 +00:00
content has not already been added to the repository before, so you can
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
delete, rename, etc the resulting files and repeated runs won't duplicate
them.
default to not using youtube-dl, for security Pity, but same reasoning as curl applies to it. This commit was sponsored by Peter on Patreon.
2018-06-17 18:46:22 +00:00
When `youtube-dl` is installed, it can be used to download links in the feed.
documentation update for youtube-dl Code not updated yet. This commit was sponsored by Thomas Hochstein on Patreon.
2017-11-28 18:05:58 +00:00
This allows importing e.g., YouTube playlists.
default to not using youtube-dl, for security Pity, but same reasoning as curl applies to it. This commit was sponsored by Peter on Patreon.
2018-06-17 18:46:22 +00:00
(However, this is disabled by default as it can be a security risk.
rename annex.security.allowed-http-addresses Renamed annex.security.allowed-http-addresses to annex.security.allowed-ip-addresses because it is not really specific to the http protocol, also limiting eg, git-annex's use of ftp and via youtube-dl, several other protocols. The old name for the config will still work. If both old and new name are set, the new name will win.
2019-05-30 16:43:40 +00:00
See the documentation of annex.security.allowed-ip-addresses
default to not using youtube-dl, for security Pity, but same reasoning as curl applies to it. This commit was sponsored by Peter on Patreon.
2018-06-17 18:46:22 +00:00
in [[git-annex]](1) for details.)
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
update importfeed man page, including a mention of annex.genmetadata
2015-03-31 17:48:13 +00:00
To make the import process add metadata to the imported files from the feed,
`git config annex.genmetadata true`
document the 2_ files
2020-06-24 18:31:46 +00:00
By default, the downloaded files are put in a directory with the title
of the feed, and files are named based on the title of the item in the
feed. This can be changed using the --template option.
Existing files are not overwritten by this command. If "some feed/foo.mp3"
already exists, it will instead write to "some feed/2\_foo.mp3"
(or 3, 4, etc). Sometimes a feed will change an item's url,
resulting in the new url being downloaded to such a filename.
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
# OPTIONS
* `--force`
doc/*.mdwn: Various typo fixes
2015-05-30 14:54:14 +00:00
Force downloading items it's seen before.
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
document the 2_ files
2020-06-24 18:31:46 +00:00
* `--relaxed`, `--fast`, `--raw`
These options behave the same as when using [[git-annex-addurl]](1).
* `--fast`
Avoid immediately downloading urls. The url is still checked
(via HEAD) to verify that it exists, and to get its size if possible.
* `--relaxed`
Don't immediately download urls, and avoid storing the size of the
url's content. This makes git-annex accept whatever content is there
at a future point.
* `--raw`
Prevent special handling of urls by youtube-dl, bittorrent, and other
special remotes. This will for example, make importfeed
download a .torrent file and not the contents it points to.
addurl, importfeed: Added --no-raw option Forces eg, download with youtube-dl without falling back to raw download. Since youtube-dl failing due to an url not being supported is difficult to distinguish from it failing due to being blocked in some way, this can be useful to avoid the fallback of git-annex downloading the raw web page and adding that. Since --raw also prevents using special remotes, --no-raw also allows special remote downloads. Although it's always possible that some special remote may claim an url and fall back to raw download of the content, which --no-raw cannot prevent. Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-06-27 15:13:38 +00:00
* `--no-raw`
Require content pointed to by the url to be downloaded using youtube-dl
or a special remote, rather than the raw content of the url. if that
cannot be done, the import will fail, and the next import of the feed
will retry.
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
* `--template`
Controls where the files are stored.
The default template is '${feedtitle}/${itemtitle}${extension}'
improve docs of template variables
2020-06-24 17:23:16 +00:00
The available variables in the template include these that
are information about the feed: feedtitle, feedauthor
And these that are information about individual items in the feed:
itemtitle, itemauthor, itemsummary, itemdescription, itemrights,
itemid.
Also, title is itemtitle but falls back to feedtitle if the item has no
title, and author is itemauthor but falls back to feedauthor.
(All of the above are also added as metadata when annex.genmetadata is
set.)
importfeed: Added some additional --template variables for date and time This commit was sponsored by Ethan Aubin.
2020-06-24 18:24:50 +00:00
The extension variable is the extension of the file in the feed,
or sometimes ".m" if no extension can be determined.
improve docs of template variables
2020-06-24 17:23:16 +00:00
The template also has some variables for when an item was published.
importfeed: Added some additional --template variables for date and time This commit was sponsored by Ethan Aubin.
2020-06-24 18:24:50 +00:00
itempubyear (YYYY), itempubmonth (MM), itempubday (DD), itempubhour (HH),
itempubminute (MM), itempubsecond (SS),
itempubdate (YYYY-MM-DD or if the feed's date cannot be parsed, the raw
improve docs of template variables
2020-06-24 17:23:16 +00:00
value from the feed).
(These use the UTC time zone, not the local time zone.)
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
Added --no-check-gitignore option for finer grained control than using --force. add, addurl, importfeed, import: Added --no-check-gitignore option for finer grained control than using --force. (--force is used for too many different things, and at least one of these also uses it for something else. I would like to reduce --force's footprint until it only forces drops or a few other data losses. For now, --force still disables checking ignores too.) addunused: Don't check .gitignores when adding files. This is a behavior change, but I justify it by analogy with git add of a gitignored file adding it, asking to add all unused files back should add them all back, not skip some. The old behavior was surprising. In Command.Lock and Command.ReKey, CheckGitIgnore False does not change behavior, it only makes explicit what is done. Since these commands are run on annexed files, the file is already checked into git, so git add won't check ignores.
2020-09-18 17:12:04 +00:00
* `--no-check-gitignore`
By default, gitignores are honored and it will refuse to download an
url to a file that would be ignored. This makes such files be added
despite any ignores.
split out common options to its own page and mention it on each subcommand page Sometimes users would get confused because an option they were looking for was not mentioned on a subcommand's man page, and they had not noticed that the main git-annex man page had a list of common options. This change lets each subcommand mention the common options, similarly to how the matching options are handled. This commit was sponsored by Svenne Krap on Patreon.
2021-05-10 19:00:13 +00:00
* Also the [[git-annex-common-options]](1) can be used.
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
# SEE ALSO
[[git-annex]](1)
expand manpages cross-references significantly i found that most man pages only had references to the main git-annex manpage, which i stillfind pretty huge and hard to navigate through. i tried to sift through all the man pages and add cross-references between relevant pages. my general rule of thumb is that links should be both ways unless one of the pages is a more general page that would become ridiculously huge if all backlinks would be added (git-annex-preferred-content comes to mind). i have also make the links one per line as this is how it was done in the metadata pages so far. i did everything but the plumbing, utility and test commands, although some of those are linked from the other commands so cross-links were added there as well.
2015-05-29 16:12:55 +00:00
[[git-annex-addurl]](1)
splitting up the man page Common command man pages all split out and often expanded. A few sections split out into their own pages. Still need to do all the other commands..
2015-03-23 19:36:10 +00:00
# AUTHOR
Joey Hess <id@joeyh.name>
Warning: Automatically converted into a man page by mdwn2man. Edit with care.
Reference in a new issue Copy permalink