roadmap for next year's work

7 files changed, 120 insertions, 23 deletions

diff --git a/doc/design/assistant/android.mdwn b/doc/design/assistant/android.mdwn
index 3325d4397..8beaffabc 100644
--- a/doc/design/assistant/android.mdwn
+++ b/doc/design/assistant/android.mdwn
@@ -6,7 +6,7 @@ to improve it.
 * [[bugs/Android_app_permission_denial_on_startup]]
 * S3 doesn't work (at least to Internet Archive: 
   "connect: does not exist (connection refused)")
-* Get app into Goole Play and/or FDroid
+* Get app into Google Play and/or FDroid
 
 ## TODO
 
diff --git a/doc/design/assistant/chunks.mdwn b/doc/design/assistant/chunks.mdwn
new file mode 100644
index 000000000..267e202e2
--- /dev/null
+++ b/doc/design/assistant/chunks.mdwn
@@ -0,0 +1,4 @@
+To avoid leaking even the size of your encrypted files to cloud storage
+providers, add a mode that stores fixed size chunks.
+
+May be a useful starting point for [[deltas]].
diff --git a/doc/design/assistant/disaster_recovery.mdwn b/doc/design/assistant/disaster_recovery.mdwn
new file mode 100644
index 000000000..28dd41c5a
--- /dev/null
+++ b/doc/design/assistant/disaster_recovery.mdwn
@@ -0,0 +1,53 @@
+The assistant should help the user recover their repository when things go
+wrong.
+
+## dangling lock files
+
+There are a few ways a git repository can get broken that are easily fixed.
+One is left over index.lck files. When a commit to a repository fails,
+check that nothing else is using it, fix the problem, and redo the commit.
+
+This should be done on both the current repository and any local
+repositories. Maybe also make git-annex-shell be able to do it remotely?
+
+## incremental fsck
+
+Add webapp UI to enable incremental fsck, and choose when to start and how
+long to run each day.
+
+When fsck finds a damanged file, queue a download of the file from a
+remote. If no accessible remote has the file, prompt the user to eg, connect
+a drive containing it.
+
+## git-annex-shell remote fsck
+
+Would be nice; otherwise remote fsck is too expensive (downloads
+everything) to have the assistant do. (remote fsck --fast might be worth
+having the assistant do)
+
+## git fsck
+
+Have the sanity checker run git fsck periodically (it's fairly inexpensive,
+but still not too often, and should be ioniced and niced). 
+
+If committing to the repository fails, after resolving any dangling lock
+files (see above), it should git fsck.
+
+If git fsck finds problems, launch git repository repair.
+
+## git repository repair
+
+There are several ways git repositories can get damanged. 
+
+The most common is empty files in .git/annex/objects and commits that refer
+to those objects. When the objects have not yet been pushed anywhere.
+I've several times recovered from this manually by
+removing the bad files and resetting to before the commits that referred to
+them. Then re-staging any divergence in the working tree. This could
+perhaps be automated.
+
+As long as the git repository has at least one remote, another method is to
+clone the remote, sync from all other remotes, move over .git/config and
+.git/annex/objects, and tar up the old broken git repo and `git annex add`
+it. This should be automatable and get the user back on their feet. User
+could just click a button and have this be done.
diff --git a/doc/design/assistant/gpgkeys.mdwn b/doc/design/assistant/gpgkeys.mdwn
new file mode 100644
index 000000000..582082f2e
--- /dev/null
+++ b/doc/design/assistant/gpgkeys.mdwn
@@ -0,0 +1,24 @@
+Currently the assistant sets up a shared encryption key, which is checked
+into git, so anyone who gets the repository can decrypt files that are
+stored encrypted on special remotes.
+
+To support using gpg keys in the assistant, we need two things:
+
+1. Help user set up a gpg key if they don't have one. This could be a
+   special-purpose key dedicated to being used by git-annex. It might be
+   nice to leave the user with a securely set up general purpose key,
+   but that would certianly preclude prompting for its password in the
+   webapp. Indeed, the password prompt is the main problem here.
+   Best solution would be to get gpg agent working on all supported 
+   platforms.
+2. Help user learn the gpg keys of people they want to share their repo
+   with, and give them access. If the public key was recorded in the git-annex
+   branch, this could be easily determined when sharing repositories with
+   friends. Or, use MonkeySphere..
+
+-----
+
+Another gpg key security thing is that currently git-annex stores
+crypto creds in memory while it's running. Should use locked memory. See
+<https://github.com/vincenthz/hs-securemem> and
+<https://github.com/vincenthz/hs-securemem/issues/1>
diff --git a/doc/design/assistant/sshpassword.mdwn b/doc/design/assistant/sshpassword.mdwn
new file mode 100644
index 000000000..e38769867
--- /dev/null
+++ b/doc/design/assistant/sshpassword.mdwn
@@ -0,0 +1,12 @@
+Currently the assistant sets up dedicated ssh keys, that can just use
+git-annex. This is ok. The problem is that the initial 2 connections to the
+ssh server when setting up these keys involve a password prompt, which is
+done at the console unless the system happens to have a working ssh agent
+that can pop up a dialog. That can be confusing.
+
+It would be nice to have the webapp prompt for the password. Can it be done
+securely?
+
+This might come down to a simple change to the webapp to prompt for the
+password, and then rather a lot of pain to make the webapp use HTTPS so we
+can be pretty sure noone is sniffing the (localhost) connection.
diff --git a/doc/design/assistant/xmpp.mdwn b/doc/design/assistant/xmpp.mdwn
index 5e5efbaa5..4bdc58874 100644
--- a/doc/design/assistant/xmpp.mdwn
+++ b/doc/design/assistant/xmpp.mdwn
@@ -13,28 +13,6 @@ who share a repository, that is stored in the [[cloud]].
   I have seen a log where a push's packet 1 and 3 arrived, but 2 did not.
   To deal with this, need at least a 1 packet buffer and ACK or resend
   request implemented over top of XMPP. Essentially, TCP over XMPP. :(
-* Add an encryption layer that does not rely on trusting the XMPP server's
-  security (currently it's encrypted only to the server, not end-to-end).
-  There are a few options for how to generate the key for eg,
-  AES encryption:
-  * Do a simple Diffie-Hellman shared key generation when starting each XMPP
-    push session. Would not protect the users from active MITM by the
-    XMPP server, but would prevent passive data gathering attacks from
-    getting useful data. Since the key is ephemeral, would provide
-    Forward Security.
-  * Do D-H key generation, but at pairing, not push time. Allows for an
-    optional confirmation step, using eg, BubbleBabble to compare the
-    keys out of band. ("I see xebeb-dibyb-gycub-kacyb-modib-pudub-sefab-vifuc-bygoc-daguc-gohec-kuxax .. do you too?")
-  * Prompt both users for a passphrase when XMPP pairing, and 
-    use SPEKE (or similar methods like J-PAKE) to generate a shared key.
-    Avoids active MITM attacks. Makes pairing harder, especially pairing
-    between one's own devices, since the passphrase has to be entered on
-    all devices. Also problimatic when pairing more than 2 devices,
-    especially when adding a device to the set later, since there
-    would then be multiple different keys in use.
-  * Rely on the user's gpg key, and do gpg key verification during XMPP
-    pairing. Problimatic because who wants to put their gpg key on their
-    phone? Also, require the users be in the WOT and be gpg literate.
 
 ## design goals
 
diff --git a/doc/design/assistant/xmpp_security.mdwn b/doc/design/assistant/xmpp_security.mdwn
new file mode 100644
index 000000000..6b2b728f2
--- /dev/null
+++ b/doc/design/assistant/xmpp_security.mdwn
@@ -0,0 +1,26 @@
+Currently [[xmpp]] relies on the SSL connection to the server for security.
+The server can see git repository data pushed through it. (Also, the SSL
+connection is not pinned or really checked well at all.)
+
+Add an encryption layer that does not rely on trusting the XMPP server's
+security. There are a few options for how to generate the key for eg,
+AES encryption:
+
+  * Do a simple Diffie-Hellman shared key generation when starting each XMPP
+    push session. Would not protect the users from active MITM by the
+    XMPP server, but would prevent passive data gathering attacks from
+    getting useful data. Since the key is ephemeral, would provide
+    Forward Security.
+  * Do D-H key generation, but at pairing, not push time. Allows for an
+    optional confirmation step, using eg, BubbleBabble to compare the
+    keys out of band. ("I see xebeb-dibyb-gycub-kacyb-modib-pudub-sefab-vifuc-bygoc-daguc-gohec-kuxax .. do you too?")
+  * Prompt both users for a passphrase when XMPP pairing, and 
+    use SPEKE (or similar methods like J-PAKE) to generate a shared key.
+    Avoids active MITM attacks. Makes pairing harder, especially pairing
+    between one's own devices, since the passphrase has to be entered on
+    all devices. Also problimatic when pairing more than 2 devices,
+    especially when adding a device to the set later, since there
+    would then be multiple different keys in use.
+  * Rely on the user's gpg key, and do gpg key verification during XMPP
+    pairing. Problimatic because who wants to put their gpg key on their
+    phone? Also, require the users be in the WOT and be gpg literate.