aboutsummaryrefslogtreecommitdiff
path: root/Command/ImportFeed.hs
Commit message (Collapse)AuthorAge
* fix build with old feedGravatar Joey Hess2014-07-19
|
* include feed url in repeated problems messageGravatar Joey Hess2014-07-14
|
* really add non-date metadata tooGravatar Joey Hess2014-07-03
|
* import metadata from feedsGravatar Joey Hess2014-07-03
| | | | | | | | | | | | | | | | | | | | | | When annex.genmetadata is set, metadata from the feed is added to files that are imported from it. Reused the same feedtitle and itemtitle, feedauthor, itemauthor, etc names that are used in --template. Also added title and author, which are the item title/author if available, falling back to the feed title/author. These are more likely to be common metadata fields. (There is a small bit of dupication here, but once git gets around to packing the object, it will compress it away.) The itempubdate field is not included in the metadata as a string; instead it is used to generate year and month fields, same as is done when adding files with annex.genmetadata set. This commit was sponsored by Amitai Schlair, who cooincidentially is responsible for ikiwiki generating nice feed metadata!
* allow building with old feed lib, w/o pubdateGravatar Joey Hess2014-04-21
|
* replace (Key, Backend) with KeyGravatar Joey Hess2014-04-17
| | | | | | | | | | Only fsck and reinject and the test suite used the Backend, and they can look it up as needed from the Key. This simplifies the code and also speeds it up. There is a small behavior change here. Before, all commands would warn when acting on an annexed file with an unknown backend. Now, only fsck and reinject show that warning.
* importfeed: Filename template can now contain an itempubdate variable. Needs ↵Gravatar Joey Hess2014-04-07
| | | | feed 0.3.9.2.
* Probe for quvi version at run time.Gravatar Joey Hess2014-02-28
| | | | | Overhead: git annex addurl runs quvi --version once. And more bloat to Annex state..
* add UrlOptions sum typeGravatar Joey Hess2014-02-24
|
* reorganize some files and importsGravatar Joey Hess2014-01-26
|
* fix inversion of control in CommandSeek (no behavior changes)Gravatar Joey Hess2014-01-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've been disliking how the command seek actions were written for some time, with their inversion of control and ugly workarounds. The last straw to fix it was sync --content, which didn't fit the Annex [CommandStart] interface well at all. I have not yet made it take advantage of the changed interface though. The crucial change, and probably why I didn't do it this way from the beginning, is to make each CommandStart action be run with exceptions caught, and if it fails, increment a failure counter in annex state. So I finally remove the very first code I wrote for git-annex, which was before I had exception handling in the Annex monad, and so ran outside that monad, passing state explicitly as it ran each CommandStart action. This was a real slog from 1 to 5 am. Test suite passes. Memory usage is lower than before, sometimes by a couple of megabytes, and remains constant, even when running in a large repo, and even when repeatedly failing and incrementing the error counter. So no accidental laziness space leaks. Wall clock speed is identical, even in large repos. This commit was sponsored by an anonymous bitcoiner.
* fix form of quvi urlGravatar Joey Hess2014-01-05
|
* check if quvi page url has already been downloaded beforeGravatar Joey Hess2014-01-05
| | | | | | That is the url that is typically stored in the git-annex branch. Kept the check for the video url too, just in case.
* importfeed: Support youtube playlists.Gravatar Joey Hess2013-12-29
|
* add readFileStrictAnyEncodingGravatar Joey Hess2013-11-20
|
* Send a git-annex user-agent when downloading urls.Gravatar Joey Hess2013-09-28
| | | | | | | | | Overridable with --user-agent option. Not yet done for S3 or WebDAV due to limitations of libraries used -- nether allows a user-agent header to be specified. This commit sponsored by Michael Zehrer.
* hlintGravatar Joey Hess2013-09-25
| | | | test suite still passes
* fix error propigating when unable to download feed itemGravatar Joey Hess2013-09-03
|
* importfeed: Also ignore transient problems with downloading content from feeds.Gravatar Joey Hess2013-09-03
|
* Youtube support! (And 53 other video hosts)Gravatar Joey Hess2013-08-22
| | | | | | | | | | When quvi is installed, git-annex addurl automatically uses it to detect when an page is a video, and downloads the video file. web special remote: Also support using quvi, for getting files, or checking if files exist in the web. This commit was sponsored by Mark Hepburn. Thanks!
* importfeed: Fix handling of dots in extensions.Gravatar Joey Hess2013-08-03
|
* importfeed: Ignores transient problems with feeds. Only exits nonzero when a ↵Gravatar Joey Hess2013-08-03
| | | | feed has repeatedly had a problems for at least 1 day.
* improve importfeed --force; try to match existing files to avoid unncessary ↵Gravatar Joey Hess2013-08-01
| | | | duplication
* importfeed --force: re-download urls that have been seen beforeGravatar Joey Hess2013-07-31
|
* improve commentGravatar Joey Hess2013-07-28
|
* fix bug in makeUniqueGravatar Joey Hess2013-07-28
| | | | Returned the possibly non-unique file
* better extension handlingGravatar Joey Hess2013-07-28
| | | | | | | When there's no extension, don't use "none", but "". When there is an extension, it starts with a dot, so don't put a redundant dot in the default format.
* if a feed cannot be downloaded or has no enclosures, failGravatar Joey Hess2013-07-28
|
* don't crash on encoding issues in feedsGravatar Joey Hess2013-07-28
| | | | | | filesystem encoding to the rescue once more! IIRC this was the main bug in hpodder.
* show a side action when finding known urlsGravatar Joey Hess2013-07-28
|
* importfeed: git-annex becomes a podcatcher in 150 LOCGravatar Joey Hess2013-07-28