diff options
Diffstat (limited to 'doc/todo')
102 files changed, 1387 insertions, 26 deletions
diff --git a/doc/todo/A_really_simple_way_to_pair_devices_like_bittorent_sync/comment_5_677e958c3f2effec7528b484aeb6478d._comment b/doc/todo/A_really_simple_way_to_pair_devices_like_bittorent_sync/comment_5_677e958c3f2effec7528b484aeb6478d._comment new file mode 100644 index 000000000..07b001c2e --- /dev/null +++ b/doc/todo/A_really_simple_way_to_pair_devices_like_bittorent_sync/comment_5_677e958c3f2effec7528b484aeb6478d._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA" + nickname="Laszlo" + subject="comment 5" + date="2013-08-25T07:48:18Z" + content=""" +What is the problem with bittorrent protocol in general? +It is some technicality or purely philosophical? + +Best, + Laszlo + +"""]] diff --git a/doc/todo/A_really_simple_way_to_pair_devices_like_bittorent_sync/comment_6_56e53803fdede895cba717e6b6e9a1bb._comment b/doc/todo/A_really_simple_way_to_pair_devices_like_bittorent_sync/comment_6_56e53803fdede895cba717e6b6e9a1bb._comment new file mode 100644 index 000000000..41e1bda78 --- /dev/null +++ b/doc/todo/A_really_simple_way_to_pair_devices_like_bittorent_sync/comment_6_56e53803fdede895cba717e6b6e9a1bb._comment @@ -0,0 +1,23 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" + nickname="develop" + subject="comment 6" + date="2013-08-25T08:39:15Z" + content=""" +I just did a cursory search on haskell torrent support. And the required pieces do seem to be be there. +https://github.com/jlouis/combinatorrent or https://github.com/astro/haskell-torrent for downloading. i'm not sure if either supports DHT, but that exists here https://github.com/aninhumer/haskell-dht + +That said, i think implementing this would require some quite major overhauls in the system. It probably won't be trivial to implement. + +Note: This is for straight \"bittorrent\", not for \"bittorrent sync\". Bittorrent sync is closed source, and while an API might come at some point, it doesn't currently exist. + +I do seem to recall joeyh talking about supporting further transport protocols(perhaps through hooks). So I'm adding the above links for future reference if this does get implemented. + +But IMHO, this doesn't seem like a trivial feature to add. It might have to take some refactoring of some core git-annex parts. Certain things have to be changed quite a bit. + +Currently a git-annex client doesn't really require anything(except rsync) to sync from a remote. With bittorrent with DHT support to share between clients, suddenly git-annex will have to maintain a constant bittorrent thread(maybe multiple) that constantly seeds all the files in the git-annex repository, while waiting for a potential remote to request data. + +So even if this happens, it is probably gonna take some time. + +Just my 2cents. +"""]] diff --git a/doc/todo/Bittorrent-like_features.mdwn b/doc/todo/Bittorrent-like_features.mdwn index 4ec991a65..41988a422 100644 --- a/doc/todo/Bittorrent-like_features.mdwn +++ b/doc/todo/Bittorrent-like_features.mdwn @@ -1,31 +1,45 @@ -I made an oops and created a wishlist thread in the forum regarding bittorrent-like behaviour. Sorry, my bad. +There are two different possible ways git-annex could use bittorrent: -Here's the original thread: -http://git-annex.branchable.com/forum/Wishlist:_Bittorrent-like_transfers/ +Let's describe those one by one. -I think I summed up pretty well what bittorrent-like features could be added to git-annex in one of the posts, so I'll copy and paste some of it here (with slight clarifications added in). +[[!toc]] ->Disclaimer: I'm thinking out loud of what could make git-annex even more awesome. I don't expect this to be implemented any time soon. Please pardon any dumbassery. +Downloading files from multiple git-annex sources simultaneously +================================================================ ->Having your remotes (optionally!) act like a swarm would be an awesome feature to have because you bring in a lot of new features that optimize storage, bandwidth, and overall traffic usage. This would be made a lot easier if parts of it were implemented in small steps that added a nifty feature. The best part is, each of these could be implemented by themselves, and they're all features that would be really useful. -> ->Step 1. Concurrent downloads of a file from remotes. -> ->This would make sense to have, it saves upload traffic on your remotes, and you also get faster DL speeds on the receiving end. -> ->Step 2. Implementing part of the super-seeding capabilities. -> ->You upload pieces of a file to different remotes from your laptop, and on your desktop you can download all those pieces and put them together again to get a complete file. If you really wanted to get fancy, you could build in redundancy (ala RAID) so if a remote or two gets lost, you don't lose the entire file. This would be a very efficient use of storage if you have a bunch of free cloud storage accounts (~1GB each) and some big files you want to back up. -> ->Step 3. Setting it up so that those remotes could talk to one another and share those pieces. -> ->This is where it gets more like bittorrent. Useful because you upload 1 copy and in a few hours, have say, 5 complete copies on 5 different remotes. You could add or remove remotes from a swarm locally, and push those changes to those remotes, which then adapt themselves to suit the new rules and share those with other remotes in the swarm (rules should be GPG-signed as a safety precaution). Also, if/when deltas get implemented, you could push that delta to the swarm and have all the remotes adopt it. This is cooler than regular bittorrent because the shared file can be updated. As a safety precaution, the delta could be GPG signed so a corrupt file doesn't contaminate the entire swarm. Each remote could have bandwidth/storage limits set in a dotfile. -> ->This is a high-level idea of how it might work, and it's also a HUGE set of features to add, but if implemented, you'd be saving a ton of resources, adding new use cases, and making git-annex more flexible. +Having your remotes (optionally!) act like a swarm would be an awesome feature to have because you bring in a lot of new features that optimize storage, bandwidth, and overall traffic usage. This would be made a lot easier if parts of it were implemented in small steps that added a nifty feature. The best part is, each of these could be implemented by themselves, and they're all features that would be really useful. -And this: + 1. Concurrent downloads of a file from remotes. ->Obviously, Step 3 would only work on remotes that you have control of processes on, but if given login credentials to cloud storage remotes (potentially dangerous!) they could read/write to something like dropbox or rsync. -> ->Another thing, this would be completely trackerless. You just use remote groups (or create swarm definitions) and share those with your remotes. **It's completely decentralized!** + This would make sense to have, it saves upload traffic on your remotes, and you also get faster DL speeds on the receiving end. + 2. Implementing part of the super-seeding capabilities. + + You upload pieces of a file to different remotes from your laptop, and on your desktop you can download all those pieces and put them together again to get a complete file. If you really wanted to get fancy, you could build in redundancy (ala RAID) so if a remote or two gets lost, you don't lose the entire file. This would be a very efficient use of storage if you have a bunch of free cloud storage accounts (~1GB each) and some big files you want to back up. + + 3. Setting it up so that those remotes could talk to one another and share those pieces. + + This is where it gets more like bittorrent. Useful because you upload 1 copy and in a few hours, have say, 5 complete copies on 5 different remotes. You could add or remove remotes from a swarm locally, and push those changes to those remotes, which then adapt themselves to suit the new rules and share those with other remotes in the swarm (rules should be GPG-signed as a safety precaution). Also, if/when deltas get implemented, you could push that delta to the swarm and have all the remotes adopt it. This is cooler than regular bittorrent because the shared file can be updated. As a safety precaution, the delta could be GPG signed so a corrupt file doesn't contaminate the entire swarm. Each remote could have bandwidth/storage limits set in a dotfile. + +This is a high-level idea of how it might work, and it's also a HUGE set of features to add, but if implemented, you'd be saving a ton of resources, adding new use cases, and making git-annex more flexible. + +Obviously, Step 3 would only work on remotes that you have control of processes on, but if given login credentials to cloud storage remotes (potentially dangerous!) they could read/write to something like dropbox or rsync. + +Another thing, this would be completely trackerless. You just use remote groups (or create swarm definitions) and share those with your remotes. **It's completely decentralized!** + +This was originally posted [[as a forum post|forum/Wishlist:_Bittorrent-like_transfers]] by [[users/GLITTAH]]. + +Using an external client (addurl torrent support) +================================================= + +The alternative to this would be to add `addurl` support for bittorrent files. The same way we can now add Youtube videos to a git-annex repository thanks to [[quvi]], we could also simply do: + + git annex addtorrent debian-live-7.0.0-amd64-standard.iso.torrent + +or even better: + + git annex addurl http://cdimage.debian.org/debian-cd/current-live/amd64/bt-hybrid/debian-live-7.0.0-amd64-standard.iso.torrent + +This way, a torrent would just become another source for a specific file. When we `get` the file, it fires up `$YOUR_FAVORITE_TORRENT_CLIENT` to download the file. + +That way we avoid the implementation complexity of shoving a complete bittorrent client within the assistant. The `get` operation would block until the torrent is downloaded, i guess... --[[anarcat]] diff --git a/doc/todo/Bittorrent-like_features/comment_1_f4c110ef35ebf4fd89f06edf2c4f0c48._comment b/doc/todo/Bittorrent-like_features/comment_1_f4c110ef35ebf4fd89f06edf2c4f0c48._comment new file mode 100644 index 000000000..eba291af9 --- /dev/null +++ b/doc/todo/Bittorrent-like_features/comment_1_f4c110ef35ebf4fd89f06edf2c4f0c48._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="Gastlag" + ip="109.190.97.30" + subject="Gittorrent" + date="2013-08-28T21:49:56Z" + content=""" +May this could interest you : few years ago somes tried to mix Git and Bittorrent. + +http://www.advogato.org/article/994.html +http://utsl.gen.nz/gittorrent/rfc.html +http://code.google.com/p/gittorrent/ +https://git.wiki.kernel.org/index.php/SoC2010Application#Did_your_organization_participate_in_past_GSoCs.3F_If_so.2C_please_summarize_your_involvement_and_the_successes_and_challenges_of_your_participation +"""]] diff --git a/doc/todo/Build_for_Synology_DSM/comment_13_314255fd503d125b5aeae2f62acfd592._comment b/doc/todo/Build_for_Synology_DSM/comment_13_314255fd503d125b5aeae2f62acfd592._comment new file mode 100644 index 000000000..3c54a9271 --- /dev/null +++ b/doc/todo/Build_for_Synology_DSM/comment_13_314255fd503d125b5aeae2f62acfd592._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnrP-0DGtHDJbWSXeiyk0swNkK1aejoN3c" + nickname="sebastien" + subject="comment 13" + date="2013-08-06T12:18:35Z" + content=""" +I post an issue to github synocommunity for that, i hope somenone have some time to package this great features. +"""]] diff --git a/doc/todo/Build_for_Synology_DSM/comment_15_9525cd0d75ff4c15182d10a855774b69._comment b/doc/todo/Build_for_Synology_DSM/comment_15_9525cd0d75ff4c15182d10a855774b69._comment new file mode 100644 index 000000000..c3edf99e2 --- /dev/null +++ b/doc/todo/Build_for_Synology_DSM/comment_15_9525cd0d75ff4c15182d10a855774b69._comment @@ -0,0 +1,30 @@ +[[!comment format=mdwn + username="lorenzo" + ip="84.75.27.69" + subject="Running Debian squeeze binaries on libc 2.5 based NAS" + date="2013-10-27T23:56:26Z" + content=""" +Following the suggestions in this page I tried to run the binaries that debian provides on my Lacie NetworkSpace which is another one of these NAS devices with old libc. After uploading the binaries and required libraries and using `LD_LIBRARY_PATH` to force the loader to use the version I uploaded of the libraries I was still having a segfault (similar to what Franck was experiencing) while running git-annex in a chroot was working. + +It turns out that it is possible to solve the problem without having to use chroot by not loading the binary directly but by substituting it with a script that calls the correct `ld-linux.so.3`. Assume you have uncompressed the files from the deb packages in `/opt/git-annex`. + +First create a directory `/opt/git-annex/usr/bin/git-annex.exec` and copy the executable `/opt/git-annex/usr/bin/git-annex` there. + +Then create script `/opt/git-annex/usr/bin/git-annex` with the following contents: + + #!/bin/bash + + PREFIX=/opt/git-annex + + export GCONV_PATH=$PREFIX/usr/lib/gconv + + exec $PREFIX/lib/ld-linux.so.3 --library-path $PREFIX/lib/:$PREFIX/usr/lib/ $PREFIX/usr/bin/git-annex.exec/git-annex \"$@\" + +The `GCONV_PATH` setting is important to prevent the app from failing with the message: + + git-annex.exec: mkTextEncoding: invalid argument (Invalid argument) + +The original executable is moved to a different directory instead of being simply renamed to make sure that `$0` is correct when the executable starts. The parameter for the linker `--library-path` is used instead of the environment variable `LD_LIBRARY_PATH` to make sure that the programs exec'ed by git-annex do not have the variable set. + +Some more info about the approach: [[http://www.novell.com/coolsolutions/feature/11775.html]] +"""]] diff --git a/doc/todo/Deleting_Unused_Files_by_Age.mdwn b/doc/todo/Deleting_Unused_Files_by_Age.mdwn new file mode 100644 index 000000000..b72768bca --- /dev/null +++ b/doc/todo/Deleting_Unused_Files_by_Age.mdwn @@ -0,0 +1,13 @@ +I periodically move unused files to one of my servers. What I would like to +do is drop any unused file that has been unused for say more than 6 months? +I would like to not drop all unused files. + +> It strikes me that this is quite similar to how git handles deleting +> stale refs with the reflog. So, if `git annex unused` were changed to +> also look at the reflog, it would keep all files referred to by all refs +> in the reflog, until the reflog expires. You could then set reflog expiry +> to 6 months, and be done. +> +> However, I think that many users expect git annex unused to be able to +> immediately find and remove a file after it's been deleted. So this +> probably needs to be a configurable behavior. --[[Joey]] diff --git a/doc/todo/Option_for_browser_to_launch_webapp_with.mdwn b/doc/todo/Option_for_browser_to_launch_webapp_with.mdwn new file mode 100644 index 000000000..dae601169 --- /dev/null +++ b/doc/todo/Option_for_browser_to_launch_webapp_with.mdwn @@ -0,0 +1,7 @@ +Firefox is my default browser, but as we all know, it doesn't load quickly. If I don't have Firefox running but I want to access the git-annex webapp, I'd rather launch the webapp in some small, quick browser like QupZilla than wait for Firefox to load. + +Could git-annex have a setting, maybe a "webapp --browser" option and/or a setting in the config file, to specify the browser to launch? + +> git-annex uses the standard `git config web.browser` if you set it. +> [[done]] +> --[[Joey]] diff --git a/doc/todo/Show_repo_type_in_repo_list.mdwn b/doc/todo/Show_repo_type_in_repo_list.mdwn new file mode 100644 index 000000000..40fbe6537 --- /dev/null +++ b/doc/todo/Show_repo_type_in_repo_list.mdwn @@ -0,0 +1 @@ +It would be helpful to show each repo's type in the list. diff --git a/doc/todo/Show_repo_type_in_repo_list/comment_1_ac6eb1072ef902a094b79dd8e0917c4d._comment b/doc/todo/Show_repo_type_in_repo_list/comment_1_ac6eb1072ef902a094b79dd8e0917c4d._comment new file mode 100644 index 000000000..10c0c17df --- /dev/null +++ b/doc/todo/Show_repo_type_in_repo_list/comment_1_ac6eb1072ef902a094b79dd8e0917c4d._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.47" + subject="comment 1" + date="2013-11-02T23:45:39Z" + content=""" +Currently if you go to the edit page for the repository, it shows some information about it, including its type and often its location, at the bottom of the page. + +I tend to feel that putting anything else in the repo list would result in it being too cluttered. +"""]] diff --git a/doc/todo/Show_repo_type_in_repo_list/comment_2_6979c487f707a724a048d20e2e5744e6._comment b/doc/todo/Show_repo_type_in_repo_list/comment_2_6979c487f707a724a048d20e2e5744e6._comment new file mode 100644 index 000000000..7d4a36324 --- /dev/null +++ b/doc/todo/Show_repo_type_in_repo_list/comment_2_6979c487f707a724a048d20e2e5744e6._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmNu4V5fvpLlBhaCUfXXOB0MI5NXwh8SkU" + nickname="Adam" + subject="comment 2" + date="2013-11-02T23:59:16Z" + content=""" +I understand. Well, here's what I'm looking at right now. :) + +[[http://alphapapa.net/outbox/view.png]] + +I just think it would be very useful to say \"Client\" or \"Transfer\" or \"Full Backup\" next to each one, and there's often plenty of room, depending on the size of the window. Especially when one's trying to wrap one's head around git-annex, being reminded what each repo is for would help a lot. Otherwise you basically have to put it in the name or description yourself...so why not just go ahead and show it? :) + +Another option might be to have an icon for each type. +"""]] diff --git a/doc/todo/Show_repo_type_in_repo_list/comment_3_529254a6cc20de7259d60a3cbc5ccaf7._comment b/doc/todo/Show_repo_type_in_repo_list/comment_3_529254a6cc20de7259d60a3cbc5ccaf7._comment new file mode 100644 index 000000000..3f8b82695 --- /dev/null +++ b/doc/todo/Show_repo_type_in_repo_list/comment_3_529254a6cc20de7259d60a3cbc5ccaf7._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.47" + subject="comment 3" + date="2013-11-03T00:19:34Z" + content=""" +I'd be happy to put in some icons if someone finds some good (and suitably licensed) ones that cover the different types of repositories. +"""]] diff --git a/doc/todo/Sync_repo_names__63__.mdwn b/doc/todo/Sync_repo_names__63__.mdwn new file mode 100644 index 000000000..d3bb59f04 --- /dev/null +++ b/doc/todo/Sync_repo_names__63__.mdwn @@ -0,0 +1,10 @@ +It's very confusing to me that the same repo viewed from different client systems can have different names and descriptions. This implies that making changes to a remote repo from one system only affects how that system sees the repo, but it seems to affect how the entire git-annex "pair" or "network of repos" sees it. + +I think it would be good if the names and descriptions of repos were synced across clients. + +> The descriptions of repositories are synced. (They're stored in git-annex:uuid.log) +> +> git allows for the same repository to be referred to using as many different remote names as you want to set up. git-annex inherits this, +> and I can't see this changing; there are very good reasons for remotes to +> have this flexability. [[done]] +> --[[Joey]] diff --git a/doc/todo/Wishlist:_additional_environment_variables_for_hooks.mdwn b/doc/todo/Wishlist:_additional_environment_variables_for_hooks.mdwn new file mode 100644 index 000000000..6e852b9f2 --- /dev/null +++ b/doc/todo/Wishlist:_additional_environment_variables_for_hooks.mdwn @@ -0,0 +1,14 @@ +It would be nice if a couple of additional environment variables to be set for hook uses. + +In particular: + + GIT_ANNEX_DIRECT=`git config annex.direct` + +and + + GIT_TOP_LEVEL=`git rev-parse --show-toplevel` + + +I've made some changes to flickrannex to allow the sub-directories above the uploaded image to be added as tags. This change has been merged into trunk: [[https://github.com/TobiasTheViking/flickrannex]] + +What I needed was both the environment variables mentioned above. One is set as part of the annex-hook and the other I guestimate from the file path. If it was set in git-annex it would be much cleaner (and accurate). So...I think this info would be useful for other hook. diff --git a/doc/todo/Wishlist:_sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn b/doc/todo/Wishlist:_sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn new file mode 100644 index 000000000..ff7773d3e --- /dev/null +++ b/doc/todo/Wishlist:_sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn @@ -0,0 +1,7 @@ +In certain situations different client annexes might get the same remote repository added, but before being synced. + +Once the two clients sync they will both have two remotes with the same name. But only one UUID will have any content(Assuming only one client pushed). + +It would be nice to have some (automatic?) way to resolve this conflict. + +Not sure if anything sane can be done if both clients have pushed? diff --git a/doc/todo/__96__git_annex_import_--lazy__96___--_Delete_everything_that__39__s_in_the_source_directory_and_also_in_the_target_annex.mdwn b/doc/todo/__96__git_annex_import_--lazy__96___--_Delete_everything_that__39__s_in_the_source_directory_and_also_in_the_target_annex.mdwn index c3f681685..996c03461 100644 --- a/doc/todo/__96__git_annex_import_--lazy__96___--_Delete_everything_that__39__s_in_the_source_directory_and_also_in_the_target_annex.mdwn +++ b/doc/todo/__96__git_annex_import_--lazy__96___--_Delete_everything_that__39__s_in_the_source_directory_and_also_in_the_target_annex.mdwn @@ -24,3 +24,6 @@ As per IRC 01:04:31 < RichiH> thus i would rather see this upstream and not hacked locally The only failure mode I see in the above is "file has been dropped elsewhere, numcopies not fulfilled, but that info is not synched to the local repo, yet" -- This could be worked around by always importing the data. + +> [[done]] as `git annex import --deduplicate`. +> --[[Joey]] diff --git a/doc/todo/__96__git_annex_import_--lazy__96___--_Delete_everything_that__39__s_in_the_source_directory_and_also_in_the_target_annex/comment_1_0cc16eb17151309113cec6d1cccf203d._comment b/doc/todo/__96__git_annex_import_--lazy__96___--_Delete_everything_that__39__s_in_the_source_directory_and_also_in_the_target_annex/comment_1_0cc16eb17151309113cec6d1cccf203d._comment new file mode 100644 index 000000000..0b4e22e7c --- /dev/null +++ b/doc/todo/__96__git_annex_import_--lazy__96___--_Delete_everything_that__39__s_in_the_source_directory_and_also_in_the_target_annex/comment_1_0cc16eb17151309113cec6d1cccf203d._comment @@ -0,0 +1,20 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U" + nickname="Richard" + subject="comment 1" + date="2013-08-06T14:22:03Z" + content=""" +To expand a bit on the use case: + +I have several migration directories which I simply moved to new systems or disks with the help of `cp -ax` or `rsync`. +As I don't _need_ the data per se and merely want to hold on to it in case I ever happen to need it again and as disk space is laughably cheap, I have a lot of duplicates. +While I can at least detect bit flips with the help of checksum lists, cleaning those duplicates of duplicated duplicates is quite some effort. +To make things worse, photos, music, videos, letter and whatnot are thrown into the same container directories. + +All in all, getting data out of those data dumps and into a clean structure is quite an effort. +`git annex import --lazy` would help with this effort as I could start with the first directory, sort stuff by hand, and annex it. +As soon as data lives in any of my annexes, I could simply run `git annex import --lazy` to get rid of all duplicates while retaining the unannexed files. +Iterating through this process a few times, I will be left with clean annexes on the one hand and stuff I can simply delete on the other hand. + +I could script all this by hand on my own machine, but I am _certain_ that others would find easy, integrated, and unit tested support for whittling down data dumps over time useful. +"""]] diff --git a/doc/todo/__96__git_annex_status__47__version__96___should_print_the_local_OS.mdwn b/doc/todo/__96__git_annex_status__47__version__96___should_print_the_local_OS.mdwn new file mode 100644 index 000000000..9c2afecb4 --- /dev/null +++ b/doc/todo/__96__git_annex_status__47__version__96___should_print_the_local_OS.mdwn @@ -0,0 +1,6 @@ +That would make assessing weird reports like [[bugs/Should_UUID__39__s_for_Remotes_be_case_sensitive__63__/]] easier and quicker. + +> No, if people want to file a bug report, it's up to them to tell me +> relevant details about their OS. I'm not going down the rathole +> of making git-annex muck about trying to gather such information. +> [[done]] --[[Joey]] diff --git a/doc/todo/__96__git_annex_sync_--auto__96___or___96__git_annex_auto__96___--_synchronize_symlinks__44___tracking_info__44___and_actual_data.mdwn b/doc/todo/__96__git_annex_sync_--auto__96___or___96__git_annex_auto__96___--_synchronize_symlinks__44___tracking_info__44___and_actual_data.mdwn new file mode 100644 index 000000000..d48b4426f --- /dev/null +++ b/doc/todo/__96__git_annex_sync_--auto__96___or___96__git_annex_auto__96___--_synchronize_symlinks__44___tracking_info__44___and_actual_data.mdwn @@ -0,0 +1,3 @@ +As per DebConf13: Introduce a one-shot command to synchronize everything, including data, with the other remotes. + +Especially useful for the debconf annex. diff --git a/doc/todo/add_metadata_to_annexed_files.mdwn b/doc/todo/add_metadata_to_annexed_files.mdwn new file mode 100644 index 000000000..0fc3e8953 --- /dev/null +++ b/doc/todo/add_metadata_to_annexed_files.mdwn @@ -0,0 +1,5 @@ +I would like to attach metadata to annexed files (objects) without cluttering the workdir with files containing this metadata. A common use case would be to add titles to my photo collection that could than end up in a generated photo album. + +Depending on the implementation it might also be possible to use the metadata facility for a threaded commenting system. + +The first question is whether the metadata is attached to the objects and thus shared by all paths pointing to the same data object or to paths in the worktree. I've no preference here at this point. diff --git a/doc/todo/add_metadata_to_annexed_files/comment_1_38af9b352020194e9ace34d7dde188cf._comment b/doc/todo/add_metadata_to_annexed_files/comment_1_38af9b352020194e9ace34d7dde188cf._comment new file mode 100644 index 000000000..8460300a7 --- /dev/null +++ b/doc/todo/add_metadata_to_annexed_files/comment_1_38af9b352020194e9ace34d7dde188cf._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 1" + date="2013-08-24T19:58:54Z" + content=""" +I don't know if git-annex is the right vehicle to fix this. It seems that a more generic fix that would work in non-git-annex repos would be better. + +I can answer your question though: The metadata such as urls and locations that git-annex stores in the git-annex branch is attached to objects, and not to work tree paths. +"""]] diff --git a/doc/todo/checksum_verification_on_transfer.mdwn b/doc/todo/checksum_verification_on_transfer.mdwn new file mode 100644 index 000000000..e87907d56 --- /dev/null +++ b/doc/todo/checksum_verification_on_transfer.mdwn @@ -0,0 +1,7 @@ +Since most file transfers, particularly to/from encrypted special remotes involve git-annex streaming through the contents of the file anyway, it should be possible to add a verification of the checksum nearly for free. The main thing needed is probably a faster haskell checksum library than Data.Digest.Pure.Sha, which is probably slow enough to be annoying. + +I have not verified if an upload could be aborted before sending the data to the remote if a checksum failure is detected. It may be dependent on the individual special remote implementations. Some probably stream the encrypted data directly out the wire, while others need to set up a temp file to run a command on. It would certianly be possible to at least make the upload abort and fail if a bad checksum was detected. + +Doing the same for downloads is less useful, because the data is there locally to be fscked. The real advantage would be doing the check for uploads, to ensure that hard-to-detect corrupted files don't reach special remotes. + +--[[Joey]] diff --git a/doc/todo/checksum_verification_on_transfer/comment_1_30f77e631608b9751f9032f97d58cc30._comment b/doc/todo/checksum_verification_on_transfer/comment_1_30f77e631608b9751f9032f97d58cc30._comment new file mode 100644 index 000000000..5de1251da --- /dev/null +++ b/doc/todo/checksum_verification_on_transfer/comment_1_30f77e631608b9751f9032f97d58cc30._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U" + nickname="Richard" + subject="comment 1" + date="2013-09-09T11:50:05Z" + content=""" +Doing this during downloads would still be nice. + +While the files are easier to fsck, users will need to actually do this. If it happenend automatically, it would increase safety and reduce disk i/o. + +Of course, this will not detect degradation during/after writing. + +If you don't make it the default, please at least make it optional for us bordering on OCD when it comes to data storage. + + +Richard +"""]] diff --git a/doc/todo/direct_mode_guard.mdwn b/doc/todo/direct_mode_guard.mdwn index 7322cec63..bb7f90897 100644 --- a/doc/todo/direct_mode_guard.mdwn +++ b/doc/todo/direct_mode_guard.mdwn @@ -20,3 +20,86 @@ add`. The git wrapper could instead be another program, or it could be something like `git annex git add` --[[Joey]] + +---- + +Or, no git wrapper could be provided. Limit the commands to only git-annex +commands. This should be all that is needed to manage a direct mode +repository simply, and if the user is doing something complicated that +needs git access, they can set `GIT_DIR=.git-annex` and be careful not to +shoot off their foot. (Or can just switch to indirect mode!) + +This wins on simplicity, and if it's the wrong choice a git wrapper +can be added later. --[[Joey]] + +--- + +Implementation: Pretty simple really. Already did the hard lifting to +support `GIT_DIR`, so only need to override the default git directory +in direct mode when that's not set to `.git-annex`. + +A few things hardcode ".git", including Assistant.Threads.Watcher.ignored +and `Seek.withPathContents`, and parts of `Git.Construct`. + +--- + +Transition: git-annex should detect when it's in a direct mode repository +with a .git directory and no .git-annex directory, and transparently +do the move to transition to the new scheme. (And remember that `git annex +indirect` needs to move it back.) + +# alternative approach: move index + +Rather than moving .git, maybe move .git/index? + +This would cause git to think that all files in the tree were deleted. +So git commit -a would make a commit that removes them from git history. +But, the files in the work tree are not touched by this. + +Also, git checkout, git merge, and other things that manipulate the work +tree refuse to do anything if they'd change a file that they think is +untracked. + +Hmm, this does't solve the user accidentially running git add on an annexed +file; the whole file still gets added. + +# alternative approach: fake bare repo + +Set core.bare to true. This prevents all work tree operations, +so prevents any foot shooting. It still lets the user run commands like +git log, even on files in the tree, and git fetch, and push, and git +config, etc. + +Even better, it integrates with other tools, like `mr`, so they know +it's a git repo. + +This seems really promising. But of course, git-annex has its own set of +behaviors in a bare repo, so will need to recognise that this repo is not +really bare, and avoid them. + +> [[done]]!! --[[Joey]] + +(Git may also have some bare repo behaviors that are unwanted. One example +is that git allows pushes to the current branch in a bare repo, +even when `receive.denyCurrentBranch` is set.) + +> This is indeed a problem. Indeed, `git annex sync` successfully +> pushes changes to the master branch of a fake bare direct mode repo. +> +> And then, syncing in the repo that was pushed to causes the changes +> that were pushed to the master branch to get reverted! This happens +> because sync commits; commit sees that files are staged in index +> differing from the (pushed) master, and commits the "changes" +> which revert it. +> +> Could fix this using an update hook, to reject the updated of the master +> branch. However, won't work on crippled filesystems! (No +x bit) +> +> Could make git annex sync detect this. It could reset the master +> branch to the last one committed, before committing. Seems very racy +> and hard to get right! +> +> Could make direct mode operate on a different branch, like +> `annex/direct/master` rather than `master`. Avoid pushing to that +> branch (`git annex sync` can map back from it to `master` and push there +> instead). A bit clumsy, but works. diff --git a/doc/todo/direct_mode_guard/comment_2_85bdb9dc601b87bd7c77150d7b0a5cde._comment b/doc/todo/direct_mode_guard/comment_2_85bdb9dc601b87bd7c77150d7b0a5cde._comment new file mode 100644 index 000000000..7cf37a917 --- /dev/null +++ b/doc/todo/direct_mode_guard/comment_2_85bdb9dc601b87bd7c77150d7b0a5cde._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawm7AuSfii_tCkLyspL6Mr0ATlO6OxLNYOo" + nickname="Georg" + subject="comment 2" + date="2013-09-20T11:29:04Z" + content=""" +Maybe make a git sub-namespace of commands. Yeah, I know, something like git annex git-add sounds a bit on the verbose side, but it would allow access to possibly all git commands regardless of name clashes. +"""]] diff --git a/doc/todo/faster_gnupg_cipher.mdwn b/doc/todo/faster_gnupg_cipher.mdwn index a1cfd428d..f5ff062d2 100644 --- a/doc/todo/faster_gnupg_cipher.mdwn +++ b/doc/todo/faster_gnupg_cipher.mdwn @@ -1 +1,9 @@ -Apparently newer gnupg has support for hardware-accelerated AES-NI. It would be good to have an option to use that. I also wonder if using the same symmetric key for many files presents a security issues (and whether using GPG keys directly would be more secure). +Apparently newer gnupg has support for hardware-accelerated AES-NI. It +would be good to have an option to use that. I also wonder if using the +same symmetric key for many files presents a security issues (and whether +using GPG keys directly would be more secure). + +> [[done]]; you can now use encryption=pubkey when setting up a special +> remote to use pure public keys without the hybrid symmetric key scheme. +> Which you choose is up to you. Also, annex.gnupg-options can configure +> the ciphers used. --[[Joey]] diff --git a/doc/todo/faster_gnupg_cipher/comment_3_bd0c975494333dfe558de048d888ace8._comment b/doc/todo/faster_gnupg_cipher/comment_3_bd0c975494333dfe558de048d888ace8._comment new file mode 100644 index 000000000..d0b98b7f6 --- /dev/null +++ b/doc/todo/faster_gnupg_cipher/comment_3_bd0c975494333dfe558de048d888ace8._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="guilhem" + ip="129.16.20.209" + subject="comment 3" + date="2013-08-19T13:44:35Z" + content=""" +AES-NI acceleration will be used by default providing you're using +the new modularized GnuPG (v2.x) and libgcrypt ≥ 1.5.0. Of course it +only speeds up AES encryption, while GnuPG uses CAST by default; you can +either set `personal-cipher-preferences` to AES or AES256 in your +`gpg.conf` or, as joeyh hinted at, set `remote.<name>.annex-gnupg-options` +as described in the manpage. + +By the way, I observed a significant speed up when using `--compress-algo none`. +Image, music and video files are typically hard to compress further, and it seems +that's where gpg spent most of its time, at least on the few files I benchmarked. +"""]] diff --git a/doc/todo/faster_rsync_remotes.mdwn b/doc/todo/faster_rsync_remotes.mdwn index 5ece25008..8c40b2816 100644 --- a/doc/todo/faster_rsync_remotes.mdwn +++ b/doc/todo/faster_rsync_remotes.mdwn @@ -1 +1,4 @@ Using an rsync remote is currently very slow when there are a lot of files, since rsync appears to be called for each file copied. It would be awesome if each call to rsync was amortized to copy many files; rsync is very good at copying many small files quickly. + +> [[done]]; bug submitter was apparently not using a version +> with rsync connection caching. --[[Joey]] diff --git a/doc/todo/http_git_annex_404_retry.mdwn b/doc/todo/http_git_annex_404_retry.mdwn new file mode 100644 index 000000000..38ab860bb --- /dev/null +++ b/doc/todo/http_git_annex_404_retry.mdwn @@ -0,0 +1,16 @@ +A repository like http://annex.debconf.org/debconf-share/ has a git repo +published via http. When getting files from such a repo, git-annex tries +two urls. One url would be used by a bare repo, and the other by a non-bare +repo. (This is due to the directory hashing change.) Result is every file +download from a non-bare http repo starts with a 404 and then it retries +with the right url. + +Since git-annex already downloads the .git/config to find the uuid of the +http repo, it could also look at it to see if the repo is bare. If not, +set a flag, and try the two urls in reverse order, which would almost +always avoid this 404 problem. + +(The real solution is probably to flag day and get rid of the old-style +directory hashing, but that's been discussed elsewhere.) + +--[[Joey]] diff --git a/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template.mdwn b/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template.mdwn new file mode 100644 index 000000000..46d9de34f --- /dev/null +++ b/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template.mdwn @@ -0,0 +1,5 @@ +It would be great to be able to use the pubDate of the entries with the --template option of importfeed. + +Text.Feed.Query has a getItemPublishDate (and a getFeedPubDate, if we want some kind of ${feeddate}). + +The best would be to allow a reformating of the date(s) with (for example) %Y-%m-%D diff --git a/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template/comment_1_62752c760fc12eca0c34d67d58753d00._comment b/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template/comment_1_62752c760fc12eca0c34d67d58753d00._comment new file mode 100644 index 000000000..cc3d85faf --- /dev/null +++ b/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template/comment_1_62752c760fc12eca0c34d67d58753d00._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="gueux" + ip="2a01:240:fe6d:0:7986:3659:a8bd:64f1" + subject="syntax" + date="2013-09-12T14:05:16Z" + content=""" +use \"itemdate\" and \"feeddate\" as names? + +use ${itemdate=%Y-%m-%D} syntax option? +"""]] diff --git a/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template/comment_2_21672360060f48bc2eacfa535ff4c94d._comment b/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template/comment_2_21672360060f48bc2eacfa535ff4c94d._comment new file mode 100644 index 000000000..c8770ec6e --- /dev/null +++ b/doc/todo/importfeed:_allow___36____123__itemdate__125___with_--template/comment_2_21672360060f48bc2eacfa535ff4c94d._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.2.134" + subject="comment 2" + date="2013-09-13T19:53:52Z" + content=""" +getItemPublishDate returns a String, which can contain any of several date formats. Deferred until the feed library has something more sane. +Upstream bug: <https://github.com/sof/feed/issues/6> + +As for how to format the date in the feed, I would be ok with having itemdate (YYYYMMDD), itemyear (YYYY), itemmonth (MM) and itemday (DD). Full date formatting seems like overkill here. +"""]] diff --git a/doc/todo/keep_annexed_files_for_a_while.mdwn b/doc/todo/keep_annexed_files_for_a_while.mdwn new file mode 100644 index 000000000..cf85b11f3 --- /dev/null +++ b/doc/todo/keep_annexed_files_for_a_while.mdwn @@ -0,0 +1,8 @@ +I don't want files that I dropped to immediately disappear from my local or all of my remotes repos on the next sync. Especially in situations where changes to the git-annex repo get automatically and immediately replicated to remote repos, I want a configurable "grace" period before files in .git/annex/objects get really deleted. + +This has similarities to the "trash" on a desktop. It might also be nice to + +* configure a maximum amount of space of the "trash" +* have a way to see the contents of the trash to easily recover deleted files + +Maybe it would make sense to just move dropped files to the desktops trash? "git annex trash" as an alternative to drop? diff --git a/doc/todo/makefile:_respect___36__PREFIX.mdwn b/doc/todo/makefile:_respect___36__PREFIX.mdwn new file mode 100644 index 000000000..12d7274b9 --- /dev/null +++ b/doc/todo/makefile:_respect___36__PREFIX.mdwn @@ -0,0 +1,25 @@ +The `Makefile` should respect a `PREFIX` passed on the commandline so git-annex can be installed in (say) `$HOME`. + +Simple patch: + +[[!format diff """ +diff --git a/Makefile b/Makefile +index b8995b2..5b1a6d4 100644 +--- a/Makefile ++++ b/Makefile +@@ -3,7 +3,7 @@ all=git-annex $(mans) docs + + GHC?=ghc + GHCMAKE=$(GHC) $(GHCFLAGS) --make +-PREFIX=/usr ++PREFIX?=/usr + CABAL?=cabal # set to "./Setup" if you lack a cabal program + + # Am I typing :make in vim? Do a fast build. +"""]] + +--[[anarcat]] + +> [[done]] --[[Joey]] + +> > [[thanks]]! ;) --[[anarcat]] diff --git a/doc/todo/mdwn2man:_make_backticks_bold.mdwn b/doc/todo/mdwn2man:_make_backticks_bold.mdwn new file mode 100644 index 000000000..21707a309 --- /dev/null +++ b/doc/todo/mdwn2man:_make_backticks_bold.mdwn @@ -0,0 +1,22 @@ +The traditionnal way of marking commandline flags in a manpage is with a `.B` (for Bold, I guess). It doesn't seem to be used by mdwn2man, which makes the manpage look a little more dull than it could. + +The following patch makes those options come out more obviously: + +[[!format diff """ +diff --git a/Build/mdwn2man b/Build/mdwn2man +index ba5919b..7f819ad 100755 +--- a/Build/mdwn2man ++++ b/Build/mdwn2man +@@ -8,6 +8,7 @@ print ".TH $prog $section\n"; + + while (<>) { + s{(\\?)\[\[([^\s\|\]]+)(\|[^\s\]]+)?\]\]}{$1 ? "[[$2]]" : $2}eg; ++ s/\`([^\`]*)\`/\\fB$1\\fP/g; + s/\`//g; + s/^\s*\./\\&./g; + if (/^#\s/) { +"""]] + +I tested it against the git-annex manpage and it seems to work well. --[[anarcat]] + +> [[done]], thanks --[[Joey]] diff --git a/doc/todo/nicer_whereis_output.mdwn b/doc/todo/nicer_whereis_output.mdwn new file mode 100644 index 000000000..871eee01a --- /dev/null +++ b/doc/todo/nicer_whereis_output.mdwn @@ -0,0 +1,100 @@ +We had some informal discussions on IRC about improving the output of the `whereis` command. + +[[!toc levels=2]] + +First version: columns +====================== + +[[mastensg]] started by implementing a [simple formatter](https://gist.github.com/mastensg/6500982) that would display things in columns [screenshot](http://www.ping.uio.no/~mastensg/whereis.png) + +Second version: Xs +================== + +After some suggestions from [[joey]], [[mastensg]] changed the format slightly ([screenshot](http://www.ping.uio.no/~mastensg/whereis2.png)): + +[[!format txt """ +17:01:34 <joeyh> foo +17:01:34 <joeyh> |bar +17:01:34 <joeyh> ||baz (untrusted) +17:01:34 <joeyh> ||| +17:01:34 <joeyh> XXx 3? img.png +17:01:36 <joeyh> _X_ 1! bigfile +17:01:37 <joeyh> XX_ 2 zort +17:01:39 <joeyh> __x 1?! maybemissing +17:02:09 * joeyh does a s/\?/+/ in the above +17:02:24 <joeyh> and decrements the counters for untrusted +17:03:37 <joeyh> __x 0+! maybemissing +"""]] + +Third version: incremental +========================== + +Finally, [[anarcat]] worked on making it run faster on large repositories, in a [fork](https://gist.github.com/anarcat/6502988) of that first gist. Then paging was added (so headers are repeated). + +Fourth version: tuning and blocked +================================== + +[[TobiasTheViking]] provided some bugfixes, and the next step was to implement the trusted/untrusted detection, and have a counter. + +This required more advanced parsing of the remotes, and instead of starting to do some JSON parsing, [[anarcat]] figured it was time to learn some Haskell instead. + +Current status: needs merge +=========================== + +So right now, the most recent version of the python script is in [anarcat's gist](https://gist.github.com/anarcat/6502988) and works reasonably well. However, it doesn't distinguish between trusted and untrusted repos and so on. + +Furthermore, we'd like to see this factored into the `whereis` command directly. A [raw.hs](http://codepad.org/miVJb5oK) file has been programmed by `mastensg`, and is now available in the above gist. It fits the desired output and prototypes, and has been `haskellized` thanks to [[guilhem]]. + +Now we just need to merge those marvelous functions in `Whereis.hs` - but I can't quite figure out where to throw that code, so I'll leave it to someone more familiar with the internals of git-annex. The most recent version is still in [anarcat's gist](https://gist.github.com/anarcat/6502988). --[[anarcat]] + +Desired output +-------------- + +The output we're aiming for is: + + foo + |bar + ||baz (untrusted) + ||| + XXx 2+ img.png + _X_ 1! bigfile + XX_ 2 zort + __x 0+! maybemissing + +Legend: + + * `_` - file missing from repo + * `x` - file may be present in untrusted repo + * `X` - file is present in trusted repo + * `[0-9]` - number of copies present in trusted repos + * `+` - indicates there may be more copies present + * `!` - indicates only one copy is left + +Implementation notes +-------------------- + +[[!format txt """ +20:48:18 <joeyh> if someone writes me a headerWhereis :: [(RemoteName, TrustLevel)] -> String and a formatWhereis :: [(RemoteName, TrustLevel, UUID)] -> [UUD] -> FileName -> String , I can do the rest ;) +20:49:22 <joeyh> make that second one formatWhereis :: [(RemoteName, TrueLevel, Bool)] -> FileName -> String +20:49:37 <joeyh> gah, typos +20:49:45 <joeyh> suppose you don't need the RemoteName either +"""]] + +> So, I incorporated this, in a new remotes command. +> Showing all known repositories seemed a bit much +> (I have 30-some known repositories in some cases), +> so just showing configured remotes seems a good simplification. +> [[done]] +> --[[Joey]] + +> > I would have prefered this to be optional since I don't explicitely configure all remotes in git, especially if I can't reach them all the time (e.g. my laptop). It seems to me this should at least be an option, but I am confused as to why `Remote.List.remoteList` doesn't list all remotes the same way `Remote.remote_list` does... Also, it's unfortunate that the +/!/count flags have been dropped, it would have been useful... Thanks for the merge anyways! --[[done]] +> > +> > The more I look at this, the more i think there are a few things wrong with the new `remotes` command. +> > +> > 1. the name is confusing: being a git addict, I would expect the `git annex remote` command to behave like the `git remote` command: list remotes, add remotes, remove remotes and so on. it would actually be useful to have such a command (which would replace `initremote`, I guess). i recommend replacing the current `whereis` command, even if enabled through a special flag +> > +> > 2. its behavior is inconsistent with other git annex commands: `git annex status`, for example, lists information about all remotes, regardless of whether they are configured in git. `remotes` (whatever it's called), should do the same, or at least provide an option to allow the user to list files on all remotes. The way things stand, there is no way to list files on non-git remotes, even if they are added explicitely as a remote, if the remote is not actually reachable: the files are just marked as absent (even thought `whereis` actually finds them). i recommend showing all remotes regardless, either opt-in or opt-out using a flag. +> > +> > 3. having the `!` flag, at least, would be useful because it would allow users to intuitively grep for problematic files without having to learn extra syntax. same with + and having an explicit count. +> > +> > thanks. --[[anarcat]] diff --git a/doc/todo/redundancy_stats_in_status/comment_2_686ced0684d10511caf07953c64cd5b6._comment b/doc/todo/redundancy_stats_in_status/comment_2_686ced0684d10511caf07953c64cd5b6._comment new file mode 100644 index 000000000..a4711b2a3 --- /dev/null +++ b/doc/todo/redundancy_stats_in_status/comment_2_686ced0684d10511caf07953c64cd5b6._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.152.108.220" + subject="comment 2" + date="2013-09-25T18:03:55Z" + content=""" +`git annex status .` or otherwise running it with a directory has recently started walking all the location logs for all files in the directory in order to display variance from configured numcopies. It would be easy to add a redundancy counter to that. + +It would slow down the global status when not passed a directory to add redundancy info there. Maybe local is enough? +"""]] diff --git a/doc/todo/smudge/comment_3_4e7c25fe24a1e71f58a8354fa64f41c2._comment b/doc/todo/smudge/comment_3_4e7c25fe24a1e71f58a8354fa64f41c2._comment new file mode 100644 index 000000000..cd64b7001 --- /dev/null +++ b/doc/todo/smudge/comment_3_4e7c25fe24a1e71f58a8354fa64f41c2._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawn1QhtPvsRBV7pfaDW_ZTPFv_ZIxSzQ8Rg" + nickname="Paul Léo" + subject="comment 3" + date="2013-11-13T20:41:52Z" + content=""" +> SHA1 has a harder job. Would not want to re-sha1 the file every time, probably. So it'd need a local cache of file stat info, mapped to known objects. + +I think that is not true? If gits wants the file to be cleaned, it thinks that the file was changed. So you would have to SHA1 it anyway if you don't want rely on WORM (which git already does in the first step anyway). +"""]] diff --git a/doc/todo/support_for_lossy_remotes.mdwn b/doc/todo/support_for_lossy_remotes.mdwn index e757343f4..23083b2d7 100644 --- a/doc/todo/support_for_lossy_remotes.mdwn +++ b/doc/todo/support_for_lossy_remotes.mdwn @@ -3,3 +3,9 @@ I'm curious if there's a possibility to support lossy remotes. It may be handy t 1. an online place that their videos are available from 2. a worst-case scenario "backup" 3. a remote that they could download smaller video files + +> [[done]]; lossy remotes are supported as seen with `git annex addurl +> --fast` and also with the new addurl support for using quvi to get +> videos fro youtube. Just make a key with a URL or something in it, and +> no size or checksum, and any content will be assumed to be the right +> content. --[[Joey]] diff --git a/doc/todo/sync_my_local_git-annex_from_a_dump_remote.mdwn b/doc/todo/sync_my_local_git-annex_from_a_dump_remote.mdwn new file mode 100644 index 000000000..524782bc7 --- /dev/null +++ b/doc/todo/sync_my_local_git-annex_from_a_dump_remote.mdwn @@ -0,0 +1,6 @@ +As discussed on debconf, I have the following use case: + +* I have a dump remote, a folder on my webserver where files are uploaded through the web app. I don't have git on the webserver, just a plain folder. +* I have git-annex repo on a development server. The development server polls the webserver (ssh/ftp) once in an hour and synchronizes the state of the local git-annex repo with the state found on the webserver and commits that. +* This is not meant to be backup facility. I just want to be able to have a state on my development machine that is very likely to the state on the webserver. + diff --git a/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_1_81d63854f89f00855cda5ace0fc8262a._comment b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_1_81d63854f89f00855cda5ace0fc8262a._comment new file mode 100644 index 000000000..d9abb3a3c --- /dev/null +++ b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_1_81d63854f89f00855cda5ace0fc8262a._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="2001:4978:f:21a::2" + subject="comment 1" + date="2013-08-13T21:44:17Z" + content=""" +We had a conversation about this IRL. At the time, I thought I understood what you wanted. Reading the above, I am not so sure. + +What I thought you wanted was something like `git annex mirror --from remote`, which would, for each object known to git-annex that the location log said was present on the remote, make sure that the local repo had the object too, and for each object that the location log said was not present on the remote, drop it from the local repo (if numcopies etc allowed). + +`git annex mirror --to remote` could also be used as the complement of the above. + +If that's not the sort of thing you meant, let me know. +"""]] diff --git a/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_2_66822b72b1450e79e8edd0c6c21d5aa6._comment b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_2_66822b72b1450e79e8edd0c6c21d5aa6._comment new file mode 100644 index 000000000..3d459371f --- /dev/null +++ b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_2_66822b72b1450e79e8edd0c6c21d5aa6._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="http://thkoch2001.myopenid.com/" + nickname="thkoch" + subject="pseudocode" + date="2013-08-14T04:58:22Z" + content=""" +lets say my local annex is in direct mode, then the following might already do what I want: + +cd $LOCAL_ANNEX +rsync --recursive --delete $REMOTE . +git annex add && git commit + +It would however be nice if I could do the same with an annex in indirect mode or even a bare annex. +"""]] diff --git a/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_3_b9f73375e2c732e798495f8ee6970c7c._comment b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_3_b9f73375e2c732e798495f8ee6970c7c._comment new file mode 100644 index 000000000..df4be033b --- /dev/null +++ b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_3_b9f73375e2c732e798495f8ee6970c7c._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 3" + date="2013-08-24T16:35:33Z" + content=""" +Seems to me that this could easily be dealt with by installing git-annex on the webserver, making the directory there a git repository, and using either a cron job or `git annex watch` to commit files as they were changed there. + +Then you can make a direct mode, indirect mode, or even a bare clone on your local machine and use git-annex to get the files. + +Maybe you have good reasons for not wanting to go that route. And rsync on a direct mode repository should work, provided to tell it to not delete `.git`. :P I don't see any way to make rsync work in an indirect mode repository. As for trying to make git-annex handle this import over rsync itself in a way that would work in an indirect mode repository, let alone a bare repository -- I don't see a good way to do it and it seems quite special case and likely to get quite complicated to implement. + +In the meantime, I did implement `git annex mirror`, which I think is a much more interesting and generally useful tool to have. And could even be used in my recommended solution above. +"""]] diff --git a/doc/todo/untracked_remotes.mdwn b/doc/todo/untracked_remotes.mdwn new file mode 100644 index 000000000..883b5acff --- /dev/null +++ b/doc/todo/untracked_remotes.mdwn @@ -0,0 +1,9 @@ +Seems that a fairly common desire in some use cases is to be able to make a +clone of a repository and be able to get files, without updating the +location tracking information. (And without even recording a uuid in the +remote.log.) Use cases include wanting to have temporary +clones without cluttering history, and centralized development where the +developers don't care to know about one-another's systems. + +It seems that such an untracked repository would need to automatically +consider itself untrusted. Is that enough to avoid losing data? diff --git a/doc/todo/whishlist:_git_annex_drop_--dry-run.mdwn b/doc/todo/whishlist:_git_annex_drop_--dry-run.mdwn new file mode 100644 index 000000000..6bbfd7a4d --- /dev/null +++ b/doc/todo/whishlist:_git_annex_drop_--dry-run.mdwn @@ -0,0 +1,5 @@ +It'd be useful to be able to see what `git annex drop` would do *before* asking it to drop any files. + +For example, I just set up my preferred contents expressions, and I don't know if I got them right. Before dropping anything from this repo, it'd be nice to check what would happen. I know git annex drop will only drop files that are above their minimum numcopies, but I'd still like to avoid heavyweight copying in case I got my preferred contents expressions wrong. + +> [[done]]; added --want-get and --want-drop. --[[Joey]] diff --git a/doc/todo/whishlist:_git_annex_drop_--dry-run/comment_1_20ecfa8ffa52c238d21b904076ac69a2._comment b/doc/todo/whishlist:_git_annex_drop_--dry-run/comment_1_20ecfa8ffa52c238d21b904076ac69a2._comment new file mode 100644 index 000000000..098d399e3 --- /dev/null +++ b/doc/todo/whishlist:_git_annex_drop_--dry-run/comment_1_20ecfa8ffa52c238d21b904076ac69a2._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.47" + subject="comment 1" + date="2013-10-28T17:11:04Z" + content=""" +It would be nice, but it adds quite a lot of complexity to have a --dry-run, and if I add it to just drop, the next bug is going to ask for get to have it.. + +I feel that the right approach is to add a --wanted, which could then be used with find to find files that are and are not wanted, according to the preferred content settings. To see what it would want to get: `git annex find --wanted --not --in .` To see what it would want to drop: `git annex find --not --wanted --in .` +"""]] diff --git a/doc/todo/whishlist:_git_annex_drop_--dry-run/comment_2_d19bc268c9467d24baa8d8f77a6e5e09._comment b/doc/todo/whishlist:_git_annex_drop_--dry-run/comment_2_d19bc268c9467d24baa8d8f77a6e5e09._comment new file mode 100644 index 000000000..3f1102985 --- /dev/null +++ b/doc/todo/whishlist:_git_annex_drop_--dry-run/comment_2_d19bc268c9467d24baa8d8f77a6e5e09._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnWwEEA3CurHkBjIYaJsJzFc4jtY2SCkrQ" + nickname="Diego" + subject="comment 2" + date="2013-10-29T00:28:51Z" + content=""" +That makes sense, and thanks for adding this feature so quickly! +"""]] diff --git a/doc/todo/windows_support.mdwn b/doc/todo/windows_support.mdwn index 08327dba8..a8cbd6db8 100644 --- a/doc/todo/windows_support.mdwn +++ b/doc/todo/windows_support.mdwn @@ -9,7 +9,15 @@ now! --[[Joey]] * rsync special remotes are known buggy. * Bad file locking, it's probably not safe to run more than one git-annex process at the same time on Windows. -* No support for the assistant or webapp. * Ssh connection caching does not work on Windows, so `git annex get` has to connect twice to the remote system over ssh per file, which is much slower than on systems supporting connection caching. +* Webapp doesn't build yet. +* `git annex watch` builds, but does not quite work. +* `git annex assistant` builds, but has not been tested, and is known + to not download any files. (transferrer doesn't built yet) +* watch and assistant cannot be built with cabal. Possibly due to too many + files overflowing command line length limit at link stage. + To build a binary with them: + `ghc --make git-annex.hs -threaded -XPackageImports -DWITH_ASSISTANT=1 -DWITH_WIN32NOTIFY=1` + (should add all the other flags cabal uses) diff --git a/doc/todo/wishlist:_Freeing_X_space_on_remote_Y.mdwn b/doc/todo/wishlist:_Freeing_X_space_on_remote_Y.mdwn new file mode 100644 index 000000000..5fec39d98 --- /dev/null +++ b/doc/todo/wishlist:_Freeing_X_space_on_remote_Y.mdwn @@ -0,0 +1 @@ +As suggested during the first Gitify BoF during DebConf13: Adding a way to have on-demand dropping of content in a given remote would allow a user to quickly free up disk space on demand while still heeding numcopies etc. diff --git a/doc/todo/wishlist:___34__quiet__34___annex_get_for_centralized_use_case.mdwn b/doc/todo/wishlist:___34__quiet__34___annex_get_for_centralized_use_case.mdwn index 27f4744c8..d53fa56ab 100644 --- a/doc/todo/wishlist:___34__quiet__34___annex_get_for_centralized_use_case.mdwn +++ b/doc/todo/wishlist:___34__quiet__34___annex_get_for_centralized_use_case.mdwn @@ -7,3 +7,8 @@ We would like to be able to get the files without updating the `git-annex` branc I think the easiest way would be to just add an option to not update the `git-annex` branch when `annex get` is executed. Thoughts? + +> See [[untracked_remotes]] for a todo item that will probably +> be useful in this sitation. Since that describes better what +> this bug report seems to be asking for, I am closing this one. +> [[closed|done]] --[[Joey]] diff --git a/doc/todo/wishlist:___39__get__39___queue_and_schedule..mdwn b/doc/todo/wishlist:___39__get__39___queue_and_schedule..mdwn new file mode 100644 index 000000000..8919bae3f --- /dev/null +++ b/doc/todo/wishlist:___39__get__39___queue_and_schedule..mdwn @@ -0,0 +1,30 @@ +During the campaign adding a chunking feature to obscure filesize for encrypted files was added to the roadmap. But there is still one potentially valuable set* of data that git-annex can help obscure: when you access your remotes. + +This data can be used to determine when a user is actively using a remote, but if a remote is always accessed at the same time that data becomes less useful. Somebody could still monitor total traffic over a long period and figure out that a remote was more active in a given week or month, but scheduling reduces the resolution of your access times and their data. Maybe this isn't the most important feature to add, but it would be nice to have, and could possibly be built on top of the existing git-annex scheduler. It could work by queuing inter-remote transactions ('get', 'copy', 'sync', etc.), so that jobs start at the same time every day, or even the same time and day every week. + +Possible syntax examples: +###Setting up the schedule: +git annex queue schedule Monday:1730 (starts every monday at 5:30PM) + +git annex queue schedule 1400 (starts every day at 2PM) + +###Queuing git-annex commands: +git annex queue prepend sync (pretends 'sync' to the very front of the queue) + +git annex queue append get file.ISO (appends to queue file.ISO for retrieval from a repo) + +###Viewing/editing queue: +git annex queue view (view the current queue, jobs displayed with corresponding numbers) + +git annex queue rm 20 (removes job 20 from queue) + + +\*The four I can think of are: + +* File contents (solved by crypto) + +* File size (solved on the remote by chunking, but total traffic usage can't be helped) + +* User IP/Remote IP (solved by VPN - outside scope of git-annex, unless someone writes a plugin) + +* Access times (obscured by a queue and scheduling) diff --git a/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__.mdwn b/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__.mdwn new file mode 100644 index 000000000..626f5a03f --- /dev/null +++ b/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__.mdwn @@ -0,0 +1,5 @@ +Also suggested during the first Gitify BoF during DebConf13: + +`git annex drop` deletes immediately. In some situations a mechanism to tell git-annex "I would like to hold onto this data if possible, but if you need the space, please delete it" could be nice. + +An obvious question would be how to do cleanups. With the assistant, that's easy. On CLI, at the very least `git annex fsck` should list, and optionally delete, that data. diff --git a/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__/comment_1_c83a6cddd0ce222205a149cfa41ca395._comment b/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__/comment_1_c83a6cddd0ce222205a149cfa41ca395._comment new file mode 100644 index 000000000..32d0c0112 --- /dev/null +++ b/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__/comment_1_c83a6cddd0ce222205a149cfa41ca395._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://cstork.org/" + nickname="Chris Stork" + subject="How should this interact with the trust model and location tracking?" + date="2013-10-04T11:13:11Z" + content=""" +This could become complicated. AFAIU, right now git-annex keeps track of files as either present or absent. With this feature it's tempting to introduce a third state 'potentially dropped' (or 'dropped in a relaxed fashion') but do you then treat them as if they were dropped depending in wether they are on a trusted or untrusted repo? Or maybe a potentially dropped file in a trusted repo is treated as a file in a semitrusted repo? This becomes convoluted. You also need a command to undrop a file in case you decide that you really want to keep it and in order to do this you need a command to see which files are up for relaxed dropping.... + +As an alternative approach maybe it makes sense to extend [[preferred content]] expressions to take file sizes and disk usage into account. +"""]] diff --git a/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__/comment_2_353fbc2bcc40cb8c9af42907a34c6e5a._comment b/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__/comment_2_353fbc2bcc40cb8c9af42907a34c6e5a._comment new file mode 100644 index 000000000..40f299fa0 --- /dev/null +++ b/doc/todo/wishlist:___96__git_annex_drop_--relaxed__96__/comment_2_353fbc2bcc40cb8c9af42907a34c6e5a._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.152.108.243" + subject="comment 2" + date="2013-10-04T20:17:07Z" + content=""" +I don't think that a third state would be necessary. Actually dropping the file when it happens would need to do the same numcopies verification that `git annex drop` does now. + +I agree it might be simpler to first improve the power of preferred content expressions. Unfortunately one thing that cannot be put in them is anything that probes the current state of the system. This is because repo A on machine X needs to be able to calculate the preferred content of repo B on machine Y. +But I could certainly add file size as a preferred content term, since that info is known throughout the network. +"""]] diff --git a/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp.mdwn b/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp.mdwn new file mode 100644 index 000000000..85c8874fc --- /dev/null +++ b/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp.mdwn @@ -0,0 +1,6 @@ +It could be useful for Linux users and package maintainers to have systemd .service file samples to launch the assistant and the webapp. + +For multi-user systems, we could have a git-annex-webapp@username.service . + +See [systemd documentation](http://www.freedesktop.org/software/systemd/man/systemd.service.html) for informations about those files. +See [archlinux wiki](https://wiki.archlinux.org/index.php/Systemd/Services) for .service examples. diff --git a/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp/comment_1_b89e90f9a70748c95aaf81740a40b76e._comment b/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp/comment_1_b89e90f9a70748c95aaf81740a40b76e._comment new file mode 100644 index 000000000..5e8c8a374 --- /dev/null +++ b/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp/comment_1_b89e90f9a70748c95aaf81740a40b76e._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="64.134.31.139" + subject="comment 1" + date="2013-10-19T15:33:18Z" + content=""" +Wouldn't this involve running it as root or a dedicated user? Neither is ideal. git-annex already uses .desktop files to auto-start when the user who is using it logs in. +"""]] diff --git a/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp/comment_2_d64361380cb18b98ddb43ada1c9f540a._comment b/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp/comment_2_d64361380cb18b98ddb43ada1c9f540a._comment new file mode 100644 index 000000000..5dfeedf37 --- /dev/null +++ b/doc/todo/wishlist:_add_systemd_services_file_samples_for_assistant_and_webapp/comment_2_d64361380cb18b98ddb43ada1c9f540a._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmVE2R20dyfPQIzdi6urVUUAXtD6eeBsr0" + nickname="Benoît" + subject="comment 2" + date="2013-10-19T15:43:36Z" + content=""" + Not necessarily. I have a systemd instance handling my userland services. See [archlinux wiki systemd/User](https://wiki.archlinux.org/index.php/Systemd/User). +"""]] diff --git a/doc/todo/wishlist:_allow_the_same_remote_to_be_accissable_via_different_methods.mdwn b/doc/todo/wishlist:_allow_the_same_remote_to_be_accissable_via_different_methods.mdwn new file mode 100644 index 000000000..849e73cc3 --- /dev/null +++ b/doc/todo/wishlist:_allow_the_same_remote_to_be_accissable_via_different_methods.mdwn @@ -0,0 +1,5 @@ +This is a wishlist item: + +Please allow the same remote to be available via different remotes. So in my LAN my remote is available using a ssh-connection, and when I travel with my laptop, the git-annex can also reach this remote using the Jabber transport. + +> [[done]]; this has always been fully supported. --[[Joey]] diff --git a/doc/todo/wishlist:_allow_the_same_remote_to_be_accissable_via_different_methods/comment_1_abb6263f3807160222bba1122475c89c._comment b/doc/todo/wishlist:_allow_the_same_remote_to_be_accissable_via_different_methods/comment_1_abb6263f3807160222bba1122475c89c._comment new file mode 100644 index 000000000..a95ba56f9 --- /dev/null +++ b/doc/todo/wishlist:_allow_the_same_remote_to_be_accissable_via_different_methods/comment_1_abb6263f3807160222bba1122475c89c._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="2001:4978:f:21a::2" + subject="comment 1" + date="2013-08-07T16:09:26Z" + content=""" +You can have as many git remotes as you like all pointing at the same repository via different paths. git-annex fully supports this AFAIK. Are you having some problem with it? +"""]] diff --git a/doc/todo/wishlist:_allow_users_to_provide_UUID_when_running___96__git_annex_init__96__.mdwn b/doc/todo/wishlist:_allow_users_to_provide_UUID_when_running___96__git_annex_init__96__.mdwn new file mode 100644 index 000000000..0dc9ec08a --- /dev/null +++ b/doc/todo/wishlist:_allow_users_to_provide_UUID_when_running___96__git_annex_init__96__.mdwn @@ -0,0 +1,5 @@ +As there's no way to permanently hide remotes and I have to recreate two repos now, I would love to be able to re-use the old UUIDs to remove clutter. + +> git-annex already provides a way to do this: Copy `.git/config` from the +> original repo (or use `git-config` to set `annex.uuid`) *before* running +> `git annex init`. [[done]] --[[Joey]] diff --git a/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync.mdwn b/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync.mdwn new file mode 100644 index 000000000..873988eea --- /dev/null +++ b/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync.mdwn @@ -0,0 +1 @@ +The `annex.largefiles` feature is very nice to mix annexed files with normal git managed files. I'd like to be able to configure this setting on the webapp and that the configuration directive would be synchronized accross all remotes. diff --git a/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync/comment_1_db632de391ce9fce42af51a770ed3aeb._comment b/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync/comment_1_db632de391ce9fce42af51a770ed3aeb._comment new file mode 100644 index 000000000..e402d26c3 --- /dev/null +++ b/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync/comment_1_db632de391ce9fce42af51a770ed3aeb._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.47" + subject="comment 1" + date="2013-11-05T16:46:28Z" + content=""" +It might make sense to sync this across remotes and have it edited with `git annex vicfg` + +Putting it in the webapp would need some nice interface for constructing the underlying expression. Might be doable for at least simple filtering (ie, files larger than XX or with extensions .A, .B, .C). I tend to think of this as a setting for people comfortable with the command line though. +"""]] diff --git a/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync/comment_2_4a0931d9884054d764fde315d4fe4851._comment b/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync/comment_2_4a0931d9884054d764fde315d4fe4851._comment new file mode 100644 index 000000000..0e24014d2 --- /dev/null +++ b/doc/todo/wishlist:_annex.largefiles_configuration_in_webapp_and_sync/comment_2_4a0931d9884054d764fde315d4fe4851._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawne_amN4fko4p5cRY_9EYwaYuJKNn7LRio" + nickname="Tobias" + subject="feedback" + date="2013-11-05T21:23:11Z" + content=""" +> It might make sense to sync this across remotes and have it edited with git annex vicfg + +That would be great! + +> Putting it in the webapp would need some nice interface for constructing the underlying expression. Might be doable for at least simple filtering (ie, files larger than XX or with extensions .A, .B, .C). I tend to think of this as a setting for people comfortable with the command line though. + +I could live with a simple filtering interface without too many fancy stuff. The fancy stuff could be done on the command line if needed... +"""]] diff --git a/doc/todo/wishlist:_annex.largefiles_support_for_mimetypes/comment_1_304431bb62b5b8a64edc37a11bbaff59._comment b/doc/todo/wishlist:_annex.largefiles_support_for_mimetypes/comment_1_304431bb62b5b8a64edc37a11bbaff59._comment new file mode 100644 index 000000000..b06008475 --- /dev/null +++ b/doc/todo/wishlist:_annex.largefiles_support_for_mimetypes/comment_1_304431bb62b5b8a64edc37a11bbaff59._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlhE8Xar6m4x3JSILXdm-Y5KhwHTH5qzKQ" + nickname="bruno" + subject="+1" + date="2013-10-14T14:43:11Z" + content=""" +Big +1 for this feature: this would really help configuring annex.largefiles +"""]] diff --git a/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space.mdwn b/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space.mdwn new file mode 100644 index 000000000..acc8b363e --- /dev/null +++ b/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space.mdwn @@ -0,0 +1 @@ +An interesting feature, when an archived file cannot be removed from all clients because of the minimum number of copies required, would be to remove it from the repositories with the smallest amount of free space available. diff --git a/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space/comment_1_6813fdc7ecc98765a5d35d34163a1712._comment b/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space/comment_1_6813fdc7ecc98765a5d35d34163a1712._comment new file mode 100644 index 000000000..89fe4c069 --- /dev/null +++ b/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space/comment_1_6813fdc7ecc98765a5d35d34163a1712._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.153.14.105" + subject="comment 1" + date="2013-09-19T21:01:30Z" + content=""" +That might be useful in some cases. git-annex can only tell how much free space is available on remotes that are mounted to the local filesystem. Did you use case involve removable drives? +"""]] diff --git a/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space/comment_2_21a249cedca1ceb80d10784004735524._comment b/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space/comment_2_21a249cedca1ceb80d10784004735524._comment new file mode 100644 index 000000000..d2b29c239 --- /dev/null +++ b/doc/todo/wishlist:_archive_from_remote_with_the_least_free_space/comment_2_21a249cedca1ceb80d10784004735524._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmOsy6nbvPyXLd--qqjPMLnVIzxgZwtKlQ" + nickname="Nicolas" + subject="comment 2" + date="2013-09-20T19:03:13Z" + content=""" +I was not thinking of removable drives, but only of \"client\" repositories. Ideally, git-annex would query the remaining space of all connected client repositories to choose on which repositories to drop a copy. +"""]] diff --git a/doc/todo/wishlist:_detection_of_merge_conflicts.mdwn b/doc/todo/wishlist:_detection_of_merge_conflicts.mdwn new file mode 100644 index 000000000..1b4caeff0 --- /dev/null +++ b/doc/todo/wishlist:_detection_of_merge_conflicts.mdwn @@ -0,0 +1,13 @@ +A conflict during sync or merge is something that requires user intervention, or at least notification. For that reason it would be nice if git annex returned a nonzero exit status when such a conflict happened during a sync or a merge. This is what git does after a conflicting pull, and would make it easier to spot a conflict in automated syncs without having to parse annex output or the logs. + +> Good idea, [[done]]. --[[Joey]] + +Also, it would be nice if your new `git annex status` were able to inform about remaining conflicts in the repo, for instance by reporting files with variant-XXX suffix. + +> Hmm, that would need a separate pass through the whole tree, since +> currently it can use `git ls-files` to find only modified/deleted/new +> files. I would rather not make the new `git annex status` slower for +> this. +> +> It would be possible to add it to `git annex info` (old `status`) +> which already has to look through the entire work tree. diff --git a/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied.mdwn b/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied.mdwn new file mode 100644 index 000000000..837f0a587 --- /dev/null +++ b/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied.mdwn @@ -0,0 +1,6 @@ +When addWatcher gets a permission denied, it would be helpful to display the name of the object on which the permission was denied, in the error message which shows in the webapp. + +> I have made the inotify code more robust; now it doesn't crash if it +> cannot read a directory or a file, and only logs a warning, which includes +> the directory name. +> [[done]] --[[Joey]] diff --git a/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied/comment_1_d2665e7347689b520d37561cfddf0aa8._comment b/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied/comment_1_d2665e7347689b520d37561cfddf0aa8._comment new file mode 100644 index 000000000..de0528855 --- /dev/null +++ b/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied/comment_1_d2665e7347689b520d37561cfddf0aa8._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.152.108.220" + subject="comment 1" + date="2013-09-25T18:47:13Z" + content=""" +This is an exception from the inotify library, which is what contains the `addWatch` function. I catch and display the exception. Since `addWatch` is only passed a directory to watch, the most I could do is tack on the name of the directory when displaying the exception. That does not seem likely to be much help? +"""]] diff --git a/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied/comment_2_db153571a32fb072453ed583e3e9ccf4._comment b/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied/comment_2_db153571a32fb072453ed583e3e9ccf4._comment new file mode 100644 index 000000000..e0199a42d --- /dev/null +++ b/doc/todo/wishlist:_display_name_of_object_when_addWatcher_gets_a_permission_denied/comment_2_db153571a32fb072453ed583e3e9ccf4._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawl-g0hYpGY11pBP_42lHh5GWTyFuB4UwH8" + nickname="Nicolas" + subject="comment 2" + date="2013-09-25T23:08:56Z" + content=""" +Well, of course it would not be as helpful as if the inotify exception would contain the name of the exact object on which it got a permission denied (would this be a valid wishlist request for inotify?), but I think that displaying the name of the directory would already be better than nothing. +"""]] diff --git a/doc/todo/wishlist:_display_status_of_remotes_in_the_webapp.mdwn b/doc/todo/wishlist:_display_status_of_remotes_in_the_webapp.mdwn new file mode 100644 index 000000000..741466994 --- /dev/null +++ b/doc/todo/wishlist:_display_status_of_remotes_in_the_webapp.mdwn @@ -0,0 +1 @@ +It would be nice to have an indication of the status of the remotes in the webapp, for example with a field showing "In Sync", "Syncing", or the date of the last successful synchronization for unreachable remotes. diff --git a/doc/todo/wishlist:_dropping_git-annex_history.mdwn b/doc/todo/wishlist:_dropping_git-annex_history.mdwn new file mode 100644 index 000000000..8286699c7 --- /dev/null +++ b/doc/todo/wishlist:_dropping_git-annex_history.mdwn @@ -0,0 +1,28 @@ +In real life discussions with git-annex users at DebConf, the idea was proposed to have a way to drop the history of the git-annex branch, and replace it with a new branch with just the current state of the repository. + +The only thing that breaks when this is done, in theory, is `git annex log`, which can't show the location history +of files. + +The crucial thing is that this operation would only need to be done in one repository, and it would then record some information in its (new) git-annex branch, so when it was pushed to other repositories, git-annex there could notice that history had been dropped, and do the same. So, even if you have rarely used offline archive repositories, the history dropping would eventually reach them, without needing to remember to do it. + +There was speculation that it would be enough to record eg, the SHA of the top commit on the old branch. That may not be good enough, because another remote may have not gotten that SHA into its branch yet, or may have commits on top of that SHA. + +Maybe instead we want to record the SHA of the *first* commit to the old git-annex branch. This way, we can tell if the branch that got deleted is the one we're currently using. And if it is, we create a new branch with the current state of *our* branch, and then union merge the other branch into it. + +Hmm, another wrinkle is that, when this indication propigates from remote A to remote B, remote B may also have some git-annex branches available for remotes C and D, which have not transitioned, and E, which has transitioned already. It seems B should first union merge C and D into B, and then flatten B to B', and then union merge A and E into B'. + +I think that'd work! + +--[[Joey]] + +Will also allow dropping dead remotes from history. Just remove all +references to them when rewriting the branch. May or may not be desirable; +I sometimes care about dead remotes that I hope to one day recuscitate. +(OTOH, I can always run git annex fsck in them to get their location +tracking back, if I do manage to get them back.) + +--[[Joey]] + +See also : [[forum/safely_dropping_git-annex_history]] + +> [[done]] --[[Joey]] diff --git a/doc/todo/wishlist:_dropping_git-annex_history/comment_1_a4bee2e26b22a9bdaadc05b7227769ef._comment b/doc/todo/wishlist:_dropping_git-annex_history/comment_1_a4bee2e26b22a9bdaadc05b7227769ef._comment new file mode 100644 index 000000000..043e674ed --- /dev/null +++ b/doc/todo/wishlist:_dropping_git-annex_history/comment_1_a4bee2e26b22a9bdaadc05b7227769ef._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 1" + date="2013-08-24T19:39:45Z" + content=""" +BTW, a motivation for this is that some of us have old repositories that have been upgrades all the way from annex.version 1 and have a lot of cruft in them because of it. (I have repos that have been upgraded from annex.version 0, but this would not help with that cruft which is on the master branch!) + +Also, people worry that eg, a large copy back and forth bloats history, and having a way to unbloat it if it ever gets actually annoyingly bloated would stop them pestering me. ;) +"""]] diff --git a/doc/todo/wishlist:_dropping_git-annex_history/comment_2_f6d750bfe0c9d8a2aa6bc218ca5c49cc._comment b/doc/todo/wishlist:_dropping_git-annex_history/comment_2_f6d750bfe0c9d8a2aa6bc218ca5c49cc._comment new file mode 100644 index 000000000..a60973b82 --- /dev/null +++ b/doc/todo/wishlist:_dropping_git-annex_history/comment_2_f6d750bfe0c9d8a2aa6bc218ca5c49cc._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U" + nickname="Richard" + subject="comment 2" + date="2013-08-27T20:02:23Z" + content=""" +If starting commit id _and_ commit id from when history is being dropped are documented, you could potentially drop more data. + +* Don't have any commits in common? Full merge. +* Only share the starting ids? Reduce local history as much as possible, and then merge. +* Share both starting id and have the last id somewhere in history? Take history from last id up to current, reduce that, and merge. + +-- RichiH +"""]] diff --git a/doc/todo/wishlist:_encrypted_git_remote_on_hosting_site_from_webapp.mdwn b/doc/todo/wishlist:_encrypted_git_remote_on_hosting_site_from_webapp.mdwn new file mode 100644 index 000000000..b7b1bad0c --- /dev/null +++ b/doc/todo/wishlist:_encrypted_git_remote_on_hosting_site_from_webapp.mdwn @@ -0,0 +1 @@ +It would be great to be able to do **private encrypted git remote on hosting site** and **multiuser encrypted git remote on hosting site** as explained in [[tips/fully encrypted git repositories with gcrypt]] through the webapp. I think it's a pretty common usecase that can be very useful for people not owning a proper server. diff --git a/doc/todo/wishlist:_generic_annex.cost-command.mdwn b/doc/todo/wishlist:_generic_annex.cost-command.mdwn new file mode 100644 index 000000000..6adf1460e --- /dev/null +++ b/doc/todo/wishlist:_generic_annex.cost-command.mdwn @@ -0,0 +1,17 @@ +### Current setup + +ATM git-annex has + +remote.<name>.annex-cost +remote.<name>.annex-cost-command # command is not provided cmdline options by annex + +to set the cost for a given remote. That requires setting up one of those variables per each host, and possibly hardcoding options for the annex-cost-command providing e.g. the remote name. + +### Suggestion + +wouldn't it be more general and thus more flexible to have a repository-wide + +annex.cost-command + +which could take options %remote, %file and assessed accordingly per each file upon '--get' request to allow maximal flexibility: e.g. some files might better be fetched from remotes supporting transfer compression, some from the web, etc. Also it might be worth providing %remote_kind ("special" vs "git") to disambiguate %remote's? + diff --git a/doc/todo/wishlist:_make_git_annex_reinject_work_in_direct_mode.mdwn b/doc/todo/wishlist:_make_git_annex_reinject_work_in_direct_mode.mdwn new file mode 100644 index 000000000..41c8e574b --- /dev/null +++ b/doc/todo/wishlist:_make_git_annex_reinject_work_in_direct_mode.mdwn @@ -0,0 +1,18 @@ +### Please describe the problem. + +`git annex reinject` refuses to work while in direct mode. + +When in direct mode git annex reinject could simply perform `rm $symlink; mv $file_copy .; git annex add $file`. I prefer having git annex doing that so I am sure I am not messing up (mistakenly adding new files for instance) and everything is properly managed. + +### What version of git-annex are you using? On what operating system? + +git-annex 4.20130516.1 + +~~~~ +$ lsb_release -a +No LSB modules are available. +Distributor ID: Ubuntu +Description: Ubuntu 12.04.2 LTS +Release: 12.04 +Codename: precise +~~~~ diff --git a/doc/todo/wishlist:_more_descriptive_commit_messages_in_git-annex_branch.mdwn b/doc/todo/wishlist:_more_descriptive_commit_messages_in_git-annex_branch.mdwn new file mode 100644 index 000000000..3a891fc9b --- /dev/null +++ b/doc/todo/wishlist:_more_descriptive_commit_messages_in_git-annex_branch.mdwn @@ -0,0 +1,39 @@ +as of git-annex version 3.20110719, all git-annex commits only contain the word "update" as a commit message. given that the contents of the commit are pretty non-descriptive (SHA1 hashes for file names, uuids for repository names), i suggest to have more descriptive commit messages, as shown here: + + /mnt/usb_disk/photos/2011$ git annex get + /mnt/usb_disk/photos/2011$ git show git-annex + [...] + usb-disk-photos: get 2011 + + * 10 files retrieved from 2 sources (9 from local-harddisk, 1 from my-server) + * 120 files were already present + * 2 files could not be retrieved + /mnt/usb_disk/photos/2011$ cd ~/photos/2011/07 + ~/photos/2011/07$ git copy --to my-server + ~/photos/2011/07$ git show git-annex + [...] + local-harddisk: copy 2011/07 to my-server + + * 20 files pushed + ~/photos/2011/07$ + +in my opinion, the messages should at least contain + +* what command was used +* in which repository they were executed +* which files or directories they affected (not necessarily all files, but what was given on command line or implicitly from the working directory) + +--[[chrysn]] + +> The implementation of the git-annex branch precludes more descriptive +> commit messages, since a single commit can include changes that were +> previously staged to the branch's index file, or spooled to its journal +> by other git-annex commands (either concurrently running or +> interrupted commands, or even changes needed to automatically merge +> other git-annex branches). +> +> It would be possible to make it *less* verbose, with an empty commit +> message. :) --[[Joey]] + +>> Closing as this is literally impossible to do without making +>> git-annex worse. [[done]] --[[Joey]] diff --git a/doc/todo/wishlist:_option_to_print_more_info_with___39__unused__39__.mdwn b/doc/todo/wishlist:_option_to_print_more_info_with___39__unused__39__.mdwn new file mode 100644 index 000000000..7a9b81f72 --- /dev/null +++ b/doc/todo/wishlist:_option_to_print_more_info_with___39__unused__39__.mdwn @@ -0,0 +1,37 @@ +It would be nice if the 'unused' command could optionally display info about the actual files behind its cryptic keys. + +I created a (very rough) bash script that simply splices in some info from git log -S'KEY' --numstat into the unused list, like so: + + arand@mas:~/annex(master)$ bash ~/utv/scripts/annex-vunused + unused . (checking for unused data...) (checking master...) (checking synced/master...) (checking origin/HEAD...) (checking seagate/master...) + Some annexed data is no longer used by any files: + NUMBER KEY + 1 SHA256E-s1073741824--49bc20df15e412a64472421e13fe86ff1c5165e18b2afccf160d4dc19fe68a14.img + 8f479a4 Sat Feb 23 16:14:12 2013 +0100 remove bigfile + 0 1 dummy_bigfile.img + 2988d18 Sat Feb 23 16:13:48 2013 +0100 dummy file + 1 0 dummy_bigfile.img + (To see where data was previously used, try: git log --stat -S'KEY')To remove unwanted data: git-annex dropunused NUMBER + ok +The script: + + #!/bin/bash + + pipe="$(mktemp -u)" + mkfifo "$pipe" + + git annex unused >"$pipe" || exit 1 & + + while read -r line + do + key="$(echo "$line" | sed 's/^[^-]*-\([^-]*\)-.*/\1/')" + echo -n "$line" + test -n "$key" && \ + echo && \ + git log --format="%h %cd %s" --numstat -S"$key" | \ + sed '/^$/d;/git-annex automatic sync/,/^ /d;s/^/\t\t/' + + done < "$pipe" + rm "$pipe" + +It would be nice if something like this was available as an option, since it's good way to get a quick overview of what the content is, and if it's safe to drop it. diff --git a/doc/todo/wishlist:_perform_fsck_remotely.mdwn b/doc/todo/wishlist:_perform_fsck_remotely.mdwn new file mode 100644 index 000000000..f2187d912 --- /dev/null +++ b/doc/todo/wishlist:_perform_fsck_remotely.mdwn @@ -0,0 +1,39 @@ +Currently, when `fsck`'ing a remote, files are first downloaded to a temporary +file locally, decrypted if needed, and finally digested; the temporary file is +then either thrown away, or quarantined, depending on the value of that digest. + +Whereas this approach works with any kind of remote, in the particular case +where the user is granted execution rights on the digest command, one could +avoid cluttering the network and digest the file remotely. I propose the +addition of a per-remote git option `annex-remote-fsck` to switch between the +two behaviors. + + +There is an issue with encrypted specialremotes, though. As hinted at +[[here|tips/beware_of_SSD_wear_when_doing_fsck_on_large_special_remotes/#comment-70055f166f7eeca976021d24a736b471]], +since the digest of a ciphertext can't be deduced from that of a plaintext in +general one would needs, before sending an encrypted file to such a remote, to +digest it and store that digest somewhere (together with the cipher's size and +perhaps other meta-information). + +The usual directory structure (`.../.../{backend}-s{size}--{digest}.log`) seems +perfectly suitable to store these informations. Lines there would look like +`{timestamp}s {numcopy} {UUID} {remote digest}`. Of course, it implies that +remote digest commands are trustworthy (are doing the right thing), and that +the digest output are not tampered by others who have access to the git repo. +But that's outside the current threat model, I guess. + +Actually, since git-annex always includes a MDC in the ciphertexts, we could do +something clever and even avoid running a digest algorithm. According to the +[[OpenPGP standard|https://tools.ietf.org/html/rfc4880#section-5.14]] the MDC +is essentially a SHA-1 hash of the plaintext. I'm still investigating if it's +even possible, but in theory it would be enough (with non-chained ciphers at +least) to download a few bytes from the encrypted remote, decrypt those bytes +to retrieve the hash, and compare that hash with the known value. Of course +there is a downside here, namely that files tampered anywhere but on the MDC +packets would not be detected by `fsck` (but gpg will warn when decrypting the +file). + + +My 2 cents :-) Is there something I missed? I suppose there was a reason to +perform `fsck` locally at the first place... diff --git a/doc/todo/wishlist:_perform_fsck_remotely/comment_1_db92311dcdb1ef0ab0413f83e191c70c._comment b/doc/todo/wishlist:_perform_fsck_remotely/comment_1_db92311dcdb1ef0ab0413f83e191c70c._comment new file mode 100644 index 000000000..6bf6af23a --- /dev/null +++ b/doc/todo/wishlist:_perform_fsck_remotely/comment_1_db92311dcdb1ef0ab0413f83e191c70c._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 1" + date="2013-08-22T15:18:35Z" + content=""" +The only reason fsck is done locally for remotes is ease of implementation and it being a generic operation that supports any kind of special remote. + +Seems that the the only types of remotes where a remote fsck is a possibility are some rsync remotes and git remotes. +git remotes already have git-annex installed, so the fsck could be run locally on the remote system using it. + +I don't know if I see a benefit with the MDC check. Any non-malicious data corruption on the remote is likely to affect the body of the file and not the small portion that holds the MDC. So checking the MDC does not seem much better than the current existence check done by `git annex fsck --fast --from remote`. + +As for storing the remote digest on the git-annex branch, my initial reaction was just that it's potentially a lot of bloat. Thinking about it some more, when using non-shared encryption, there is currently no way, given just a clone of a git repository, to match up files in git with encrypted objects stored on a special remote. So storing the remote digest might be considered to weaken the security. +"""]] diff --git a/doc/todo/wishlist:_perform_fsck_remotely/comment_2_2f0dbaf143d94290bfbebb6869eb7241._comment b/doc/todo/wishlist:_perform_fsck_remotely/comment_2_2f0dbaf143d94290bfbebb6869eb7241._comment new file mode 100644 index 000000000..5418ff991 --- /dev/null +++ b/doc/todo/wishlist:_perform_fsck_remotely/comment_2_2f0dbaf143d94290bfbebb6869eb7241._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="guilhem" + ip="129.16.20.209" + subject="comment 2" + date="2013-08-22T16:56:55Z" + content=""" +Oh yeah, the MDC paragraph was pretty much pointless indeed. Oops :-P + +I agree that this would potentially add some noise to the index, and weaken the +security, but depending on the threat model and people's preferences that's an +option that's worth considering IMHO. +"""]] diff --git a/doc/todo/wishlist:_query_things_like_description__44___trust_level.mdwn b/doc/todo/wishlist:_query_things_like_description__44___trust_level.mdwn new file mode 100644 index 000000000..d158850cd --- /dev/null +++ b/doc/todo/wishlist:_query_things_like_description__44___trust_level.mdwn @@ -0,0 +1,4 @@ +It would be helpful to have a way to query things like a repository's description and trust level, without having to poke in the git-annex branch. For example, "git annex describe ." currently clears the description but could print the current one instead. + +> `git annex status` now breaks down the repository list by type. [[done]] +> --[[Joey]] diff --git a/doc/todo/wishlist:_query_things_like_description__44___trust_level/comment_1_14311384788312b96e550749ab7de9ea._comment b/doc/todo/wishlist:_query_things_like_description__44___trust_level/comment_1_14311384788312b96e550749ab7de9ea._comment new file mode 100644 index 000000000..3ac4ba267 --- /dev/null +++ b/doc/todo/wishlist:_query_things_like_description__44___trust_level/comment_1_14311384788312b96e550749ab7de9ea._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joey.kitenet.net/" + nickname="joey" + subject="comment 1" + date="2011-10-27T17:09:33Z" + content=""" +`git annex describe` only sets the description to avoid complication. Imagine using it in a script for example. + +`git annex status` shows the description. It does not show the trust level because I have not thought of a visually pleasing and compact way to show it in the repository list there.. suggestions appreciated, since the same list is used by `whereis`, and showing trust levels there would be particularly useful. +"""]] diff --git a/doc/todo/wishlist:_query_things_like_description__44___trust_level/comment_2_342d1ac07573c7ef4e27f003a692e261._comment b/doc/todo/wishlist:_query_things_like_description__44___trust_level/comment_2_342d1ac07573c7ef4e27f003a692e261._comment new file mode 100644 index 000000000..3bb92919f --- /dev/null +++ b/doc/todo/wishlist:_query_things_like_description__44___trust_level/comment_2_342d1ac07573c7ef4e27f003a692e261._comment @@ -0,0 +1,32 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U" + nickname="Richard" + subject="comment 2" + date="2011-10-29T18:28:13Z" + content=""" +Possible solutions: + +This: + + trusted repositories: + UUID -- foo + semi-trusted repositories: + UUID -- bar + untrusted repositories: + UUID -- baz + +or this: + + UUID -- trusted -- foo + UUID -- semi-trusted -- bar + UUID -- untrusted -- baz + +or this: + + known repositories (!/*/X): + UUID -- ! foo + UUID -- * bar + UUID -- X baz + +If you want to reformat this output, putting 'here', 'origin', etc into fixed formatting might make sense, as well. -- Richard +"""]] diff --git a/doc/todo/wishlist:_simple_url_for_webapp.mdwn b/doc/todo/wishlist:_simple_url_for_webapp.mdwn new file mode 100644 index 000000000..4549f2e74 --- /dev/null +++ b/doc/todo/wishlist:_simple_url_for_webapp.mdwn @@ -0,0 +1,36 @@ +### Please describe the problem. + +The environment is os/x with chrome as the browser. + +Let's say I close the tab with the webapp running in it. The 'git-annex webapp' process is still running, according to 'ps'. + +So I open a new tab, but then what do I type into the browser url bar to get the app back? What is usually there is a loopback address and an authorisation hash. + +* Should I double-click on the git-annex icon in the dock (or Applications directory)? +* I figured out from observing the startup that if I give the url ://localhost/Users/me/annex/.git/annex/webapp.html I will get redirected to the right place. +Should I set up a bookmark for that? + +### What steps will reproduce the problem? + +see above. + +### What version of git-annex are you using? On what operating system? + +Version: 4.20130723-ge023649 +Build flags: Assistant Webapp Pairing Testsuite S3 WebDAV FsEvents XMPP DNS +os: os/x 10.8.4 + +### Please provide any additional information below. + +I notice that in the webapp ui, all the items at the top of the page highlight when one hovers over them and have useful URLs attached, +with the exception of the 'git-annex' item at the far left.What if that had the entry point url attached to it (so one could bookmark that)? + +> The git-annex assistant is designed to stay running in the background whether you have the web browser open or not. You can open the web display at any time by +> using the git-annex menu item (on linux) or running the git-annex-webapp +> program (which is in the DMG on OSX). +> +> If the file:// url were exposed to users, it would not work if +> the assistant had not already been started. This is why there is a program +> to open the webapp, not an url. +> +> Not a bug; [[done]] --[[Joey]] diff --git a/doc/todo/wishlist:_simple_url_for_webapp/comment_1_552aad504fbb68d1f85abfde8c535e69._comment b/doc/todo/wishlist:_simple_url_for_webapp/comment_1_552aad504fbb68d1f85abfde8c535e69._comment new file mode 100644 index 000000000..1211be9b5 --- /dev/null +++ b/doc/todo/wishlist:_simple_url_for_webapp/comment_1_552aad504fbb68d1f85abfde8c535e69._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkfHTPsiAcHEEN7Xl7WxiZmYq-vX7azxFY" + nickname="Vincent" + subject="comment 1" + date="2013-07-24T14:46:22Z" + content=""" +typo + +url should be - file://localhost/Users/me/annex/.git/annex/webapp.html +"""]] diff --git a/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote.mdwn b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote.mdwn index 4227678ba..229dc258b 100644 --- a/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote.mdwn +++ b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote.mdwn @@ -10,3 +10,13 @@ The [[Web special remote|special remotes/web]] could possibly be improved by det > this. > > --[[Joey]] + +> > There's a library for this called [quvi](http://quvi.sourceforge.net/) which supports many +> > different sites and also allows fetching the URL (as opposed to just +> > downloading the file). It seems to me this would be the best tool +> > for this task. One problem to consider here is that a single youtube +> > URL may yield different file contents depending on the quality +> > chosen. Also, it seems that the URLs guessed by quvi may be +> > ephemeral. --[[anarcat]] + +> [[done]]!!! --[[Joey]] diff --git a/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_2_81f7f893ac36c145b31f02db6a682a17._comment b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_2_81f7f893ac36c145b31f02db6a682a17._comment new file mode 100644 index 000000000..a25b3c89d --- /dev/null +++ b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_2_81f7f893ac36c145b31f02db6a682a17._comment @@ -0,0 +1,20 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="quvi thoughts. excited!" + date="2013-08-22T18:22:51Z" + content=""" +Anarcat's quvi suggestion is interesting, because it seems to simplify the whole thing down to just `addurl`, which git-annex is already good at. + +If quvi manages to find the url that can be used to download the actual video file, without needing to run a special downloader, then all you really need, it seems, is `git annex addurl --relaxed $(quvi youtube-url)` The --relaxed will make git-annex not care if the content or size of the url's content varies in the future. Since --relaxed skips the actual download, you'd want to follow that with `git annex get`, since we don't know how long these urls will last.. + +I suppose git-annex could, if quvi is available, make any attempt to download a web special remote url that +matches the `quvi --support` output run the url through quvi to get the real url and download that instead. The difficulties with this approach: + +* would need to read and parse `quvi --support` every time it gets an url from the web special remote? (I don't think I'd want to link with libquvi, although that would be possible, just because this is an edge thing.) +* what it an url that had been supported stopped being supported -- we'd not want to download the raw url in that case +* putting the quvi support here doesn't allow relaxed mode to be set when `addurl` adds the url. + +Maybe it would be better to keep the support in `addurl`, and record the url generated by quvi. That url would probably be more likely to break in the future, but that kind of breakage is why `git annex untrust web` is wise.. +Keeping the support in `addurl` would also let it use the title that quvi also returns to determine the filename it creates. +"""]] diff --git a/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_3_a7e3cd68c5e5f05139151a58f358df95._comment b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_3_a7e3cd68c5e5f05139151a58f358df95._comment new file mode 100644 index 000000000..c4d8cf754 --- /dev/null +++ b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_3_a7e3cd68c5e5f05139151a58f358df95._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 3" + date="2013-08-22T18:44:15Z" + content=""" +Since the quvi urls are quite likely to break as the CDNs etc change things around, maybe it would be better to somehow have addurl tag an url as needing to be downloaded with quvi. Then `git annex get` could re-run quvi to get an url to download. + +We could expand url syntax for this. `quvi:http://youtube.com/foo` +This also allows for future expansion for other similar things. + +I'd be inclined to still make `addurl` automatically try quvi to see if an url is supported, rather than requiring the user fix up the url themselves. But if trying quvi turns out to be too expensive, manually specifying it in the url would also work. +"""]] diff --git a/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment new file mode 100644 index 000000000..ee0ab45e7 --- /dev/null +++ b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://rmunn.myopenid.com/" + nickname="rmunn" + subject="comment 4" + date="2013-08-24T15:31:36Z" + content=""" +Either quvi or youtube-dl might be a good possibility: youtube-dl has the --get-url option (or -g for short) that outputs just the download URL (and nothing else) to stdout. So if for some reason quvi turned out not to be suitable, it wouldn't necessarily mean the idea would have to be abandoned. +"""]] diff --git a/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_5_a0612ae05dbda7f7935be648b42b30fc._comment b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_5_a0612ae05dbda7f7935be648b42b30fc._comment new file mode 100644 index 000000000..38ac09511 --- /dev/null +++ b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_5_a0612ae05dbda7f7935be648b42b30fc._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://id.koumbit.net/anarcat" + ip="72.0.72.144" + subject="aaaaawesome!" + date="2013-08-26T04:43:27Z" + content=""" +wow, thanks! i am happy my little suggestion led to an actual implentation, great! +"""]] diff --git a/doc/todo/wishlist:_support_drop__44___find_on_special_remotes.mdwn b/doc/todo/wishlist:_support_drop__44___find_on_special_remotes.mdwn new file mode 100644 index 000000000..24cacbf71 --- /dev/null +++ b/doc/todo/wishlist:_support_drop__44___find_on_special_remotes.mdwn @@ -0,0 +1,18 @@ +Currently there is no way to drop files, or list what files are available, on a special remote. +It would be good if "git annex drop" and "git annex find" supported the --from argument. + +> I agree, drop should support --from. +>> [[done]] --[[Joey]] +> +> To find files *believed* to be present in a given remote, use +> `git annex find --in remote` +> Note that it might show out of date info, since it does not actually go +> check the current contents of the remote. The only reason to support +> `find --from` would be to always check, but I don't think that's needed. +> --[[Joey]] + +For commands that don't support the --from argument, it would also be nice to print an error. +Currently running "git annex drop --from usbdrive" doesn't behave as hoped and instead drops +all content from the local annex. + +> This is done now. --[[Joey]] diff --git a/doc/todo/wishlist:_support_drop__44___find_on_special_remotes/comment_1_f11ed642a83d965076778a162f701e84._comment b/doc/todo/wishlist:_support_drop__44___find_on_special_remotes/comment_1_f11ed642a83d965076778a162f701e84._comment new file mode 100644 index 000000000..6028933b4 --- /dev/null +++ b/doc/todo/wishlist:_support_drop__44___find_on_special_remotes/comment_1_f11ed642a83d965076778a162f701e84._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joey.kitenet.net/" + nickname="joey" + subject="comment 1" + date="2011-10-27T17:13:43Z" + content=""" +Well, I don't think you mean \"special remotes\", but just any old remote (special or not). +"""]] diff --git a/doc/todo/wishlist:_unify_directory_scheme_for_the_store.mdwn b/doc/todo/wishlist:_unify_directory_scheme_for_the_store.mdwn new file mode 100644 index 000000000..83ce53127 --- /dev/null +++ b/doc/todo/wishlist:_unify_directory_scheme_for_the_store.mdwn @@ -0,0 +1,20 @@ +In regular repos, objects are stored in files of the form: .git/annex/objects/xY/z1/SHA1-.../SHA1-.... (scheme 1) + +On (some) special remotes, the corresponding file is stored at: .../abc/def/SHA1-... (scheme 2) + +I'm not sure why the same scheme as in .git/objects isn't used, but it would be useful that the two-directory prefix were the same for all objects stores. + +My use case is: I synchronize a git repo, say containing photos, to a server on which I can't install git-annex. I want the server to store all annexed files. For the photos to be viewed online, the annex store must use the scheme 1 (because the symlinks point to files with scheme 1). So I need to rsync .git/annex/objects manually from my desktop, because a git-annex rsync remote uses scheme 2. On the other hand, the repo on this server is not known by git-annex (like it would if I used a rsync remote). + +At least it would be valuable (to get around above problem) to have a plumbing command giving the 2-directory prefix from a given key, for example: + +$ git annex prefix-dir SHA1-s2--3f786850e387550fdab836ed7e6dc881 + +7w/88 + +f18/122 + + +Even if the 2 schemes were unified, this prefix-dir command would still be useful when hacking around git-annex (for now I need to maintain a dictionary structure). + +Thanks a lot. diff --git a/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_1_44da58beaaab359ecaba7fb905ca4ae1._comment b/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_1_44da58beaaab359ecaba7fb905ca4ae1._comment new file mode 100644 index 000000000..86c33d434 --- /dev/null +++ b/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_1_44da58beaaab359ecaba7fb905ca4ae1._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.152.108.243" + subject="comment 1" + date="2013-10-04T20:50:33Z" + content=""" +Using the mixed case hash directory names is not desirable because people want to use git-annex on a variety of filesystems and operating systems that treat them in a variety of broken ways. However, migrating to the all lower case hash directory names would require changing every git-annex symlink in every git repository, and I do not want to inflict that on my users. + +It seems to me that the best solution to your problem is to install git-annex on your server, which should not be very hard. +"""]] diff --git a/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_2_bc698c501ecdb56df57171f4f3bb831a._comment b/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_2_bc698c501ecdb56df57171f4f3bb831a._comment new file mode 100644 index 000000000..5c5269f8c --- /dev/null +++ b/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_2_bc698c501ecdb56df57171f4f3bb831a._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnpdM9F8VbtQ_H5PaPMpGSxPe_d5L1eJ6w" + nickname="Rafaël" + subject="comment 2" + date="2013-10-05T10:45:16Z" + content=""" +Thank you for prompt answer. I didn't know there were tarballs, +and indeed I managed to install them easily (although I had to +manually install glibc version 2.9, only 2.7 was installed). + +I found that bare git repos also use lower case hash directory +names... I still would be happy with an optional migration to all +lower case, with a new key-value backend, to avoid this little +complication which happens sometimes (say when converting a repo +from non-bare to bare). +"""]] diff --git a/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_3_e555d0dbbaa05528806905c6a940724b._comment b/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_3_e555d0dbbaa05528806905c6a940724b._comment new file mode 100644 index 000000000..e6acad3b9 --- /dev/null +++ b/doc/todo/wishlist:_unify_directory_scheme_for_the_store/comment_3_e555d0dbbaa05528806905c6a940724b._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnpdM9F8VbtQ_H5PaPMpGSxPe_d5L1eJ6w" + nickname="Rafaël" + subject="comment 3" + date="2013-10-07T13:33:40Z" + content=""" +By the way, I just had the case above, i.e. convert a bare repo to non-bare. In order to keep the annex files, I cloned the bare one, and git-annex move'ed all annex content to the new repo, and to my surprise it was slow, as if all files where copied (or maybe they were only checksummed?), instead of being only renamed (old and new repos were on the same partition). So I restate that at least a command line tool giving the prefix dirs would be useful, to allow scripting for this kind of situation. +"""]] diff --git a/doc/todo/wishlist:_use_hardlinks_for_local_clones.mdwn b/doc/todo/wishlist:_use_hardlinks_for_local_clones.mdwn new file mode 100644 index 000000000..4b0694422 --- /dev/null +++ b/doc/todo/wishlist:_use_hardlinks_for_local_clones.mdwn @@ -0,0 +1,9 @@ +as far as I know, if you `git clone` locally a git-annex enabled repository, it will not have all the files available. you would need to use `git annex get` and all files would be copied over, wasting a significant amount of space. + +`git-clone` has this `--local` flags which hardlinks objects in `.git/objects`, but also, maybe more interestingly, has a `--shared` option to simply tell git to look in another repo for objects. it seems to me git-annex could leverage those functionalities to avoid file duplication when using local repositories. + +this would be especially useful for [ikiwiki](http://ikiwiki.info/forum/ikiwiki_and_big_files). + +This is a [[wishlist]], but I would also welcome implementation pointers to do this myself, thanks! --[[anarcat]] + +> [[dup|done]] diff --git a/doc/todo/wishlist:_use_hardlinks_for_local_clones/comment_1_85064fafe472a5bd395d60ce8f7acb56._comment b/doc/todo/wishlist:_use_hardlinks_for_local_clones/comment_1_85064fafe472a5bd395d60ce8f7acb56._comment new file mode 100644 index 000000000..4ef5f8414 --- /dev/null +++ b/doc/todo/wishlist:_use_hardlinks_for_local_clones/comment_1_85064fafe472a5bd395d60ce8f7acb56._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.152.108.220" + subject="comment 1" + date="2013-09-25T17:14:28Z" + content=""" +git-annex uses cp --reflink=auto. So on a filesystem supporting COW file copies, like btrfs, `git annex get` will not use any disk space when getting from the same filesystem. + +I do not like the idea of using hardlinks, because changing the file in one repository would change it in the other, which may not be desired. + +[[union_mounting]] seems to cover this item pretty well, so I will close this as a duplicate. +"""]] diff --git a/doc/todo/wishlist_degraded_files.mdwn b/doc/todo/wishlist_degraded_files.mdwn new file mode 100644 index 000000000..0b265c5eb --- /dev/null +++ b/doc/todo/wishlist_degraded_files.mdwn @@ -0,0 +1,5 @@ +This is an idea to have a small placeholder file that is put into +place when the file's actual content is not available in the local +annex. + +Details being discussed here: <http://bugs.debian.org/728552> |