summaryrefslogtreecommitdiff
path: root/P2P
Commit message (Collapse)AuthorAge
* Always use filesystem encoding for all file and handle reads and writes.Gravatar Joey Hess2016-12-24
| | | | | This is a big scary change. I have convinced myself it should be safe. I hope!
* enable-tor: When run as a regular user, test a connection back to the hidden ↵Gravatar Joey Hess2016-12-24
| | | | | | | | | | | | | | | | | | | service over tor. This way we know that after enable-tor, the tor hidden service is fully published and working, and so there should be no problems with it at pairing time. It has to start up its own temporary listener on the hidden service. It would be nice to have it start the remotedaemon running, so that extra step is not needed afterwards. But, there may already be a remotedaemon running, in communication with the assistant and we don't want to start another one. I thought about trying to HUP any running remotedaemon, but Windows does not make it easy to do that. In any case, having the user start the remotedaemon themselves lets them know it needs to be running to serve the hidden service. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
* refactorGravatar Joey Hess2016-12-24
|
* Revert "close"Gravatar Joey Hess2016-12-24
| | | | | | This reverts commit 6c089cff615bc44ca94cc4ca77dc69f281dd2ab8. Commit contained incomplete work.
* closeGravatar Joey Hess2016-12-22
|
* include tor-annex in hidden service directory namesGravatar Joey Hess2016-12-21
| | | | | | To make it easier to manage/delete them etc. Backwards compatablity is preserved for existing tor configs.
* Revert "p2p --link now defaults to setting up a bi-directional link"Gravatar Joey Hess2016-12-16
| | | | | | | | This reverts commit 6aa7e136b5d246228723f4c9996bda11f66c4445. On second thought, this was an overcomplication of what should be the lowest-level primitive. Let's build bi-directional links at the pairing level with eg magic wormhole.
* p2p --link now defaults to setting up a bi-directional linkGravatar Joey Hess2016-12-16
| | | | | | | | | | | | | | | | | | | | | | | | | Both the local and remote git repositories get remotes added pointing at one-another. Makes pairing twice as easy! Security: The new LINK command in the protocol can be sent repeatedly, but only by a peer who has authenticated with us. So, it's entirely safe to add a link back to that peer, or to some other peer it knows about. Anything we receive over such a link, the peer could send us over the current connection. There is some risk of being flooded with LINKs, and adding too many remotes. To guard against that, there's a hard cap on the number of remotes that can be set up this way. This will only be a problem if setting up large p2p networks that have exceptional interconnectedness. A new, dedicated authtoken is created when sending LINK. This also allows, in theory, using a p2p network like tor, to learn about links on other networks, like telehash. This commit was sponsored by Bruno BEAUFILS on Patreon.
* fix build with old ghcGravatar Joey Hess2016-12-10
|
* hang up connection after relayingGravatar Joey Hess2016-12-09
| | | | | | | Seems that git upload-pack outputs a "ONCDN " that is not read by the remote git receive-pack. This fixes: [2016-12-09 17:08:32.77159731] P2P > ERROR protocol parse error: "ONCDN "
* avoid exposing auth tokens in debugGravatar Joey Hess2016-12-09
|
* debug dump P2P messagesGravatar Joey Hess2016-12-09
|
* make tor hidden service work when directory watching is not availableGravatar Joey Hess2016-12-09
| | | | Avoid crashing when built w/o inotify..
* remotedaemon: git change detection over tor hidden serviceGravatar Joey Hess2016-12-09
|
* only start ref change watcher thread once per P2P connectionGravatar Joey Hess2016-12-09
| | | | | | This is more efficient. Note that the peer will get CHANGED messages for all refs changed since the connection opened, even if those changes happened before it sent NOTIFYCHANGE.
* refactor ref change watchingGravatar Joey Hess2016-12-09
| | | | | | | | | | | | | | | | | | Added to change notification to P2P protocol. Switched to a TBChan so that a single long-running thread can be started, and serve perhaps intermittent requests for change notifications, without buffering all changes in memory. The P2P runner currently starts up a new thread each times it waits for a change, but that should allow later reusing a thread. Although each connection from a peer will still need a new watcher thread to run. The dependency on stm-chans is more or less free; some stuff in yesod uses it, so it was already indirectly pulled in when building with the webapp. This commit was sponsored by Francois Marier on Patreon.
* git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks.Gravatar Joey Hess2016-12-09
| | | | | | | | | | | | | | | | | | | | | The attacker could just send a very lot of data, with no \n and it would all be buffered in memory until the kernel killed git-annex or perhaps OOM killed some other more valuable process. This is a low impact security hole, only affecting communication between local git-annex and git-annex-shell on the remote system. (With either able to be the attacker). Only those with the right ssh key can do it. And, there are probably lots of ways to construct git repositories that make git use a lot of memory in various ways, which would have similar impact as this attack. The fix in P2P/IO.hs would have been higher impact, if it had made it to a released version, since it would have allowed DOSing the tor hidden service without needing to authenticate. (The LockContent and NotifyChanges instances may not be really exploitable; since the line is read and ignored, it probably gets read lazily and does not end up staying buffered in memory.)
* typoGravatar Joey Hess2016-12-09
|
* content removal is supposed to succed if the content was already not presentGravatar Joey Hess2016-12-09
|
* update progress logs in remotedaemon send/receiveGravatar Joey Hess2016-12-08
|
* fix memory leakGravatar Joey Hess2016-12-08
| | | | | | | I'm unsure why this fixed it, but it did. Seems to suggest that the memory leak is not due to a bug in my code, but that ghc didn't manage to take full advantage of laziness, or was failing to gc something it could have.
* convert P2P runners from Maybe to Either StringGravatar Joey Hess2016-12-08
| | | | | | So we get some useful error messages when things fail. This commit was sponsored by Peter Hogg on Patreon.
* fix laziness problem in git relayingGravatar Joey Hess2016-12-08
| | | | | | | | The switch to hGetMetered subtly changed the laziness of how DATA was read, and broke git protocol relaying. Fix by sending received data to the git process's stdin immediately, which ensures that the lazy bytestring is all read from the peer before going on to process the next message from the peer.
* fix build with old ghcGravatar Joey Hess2016-12-08
|
* fix math error that caused resumes to always failGravatar Joey Hess2016-12-07
|
* ReadWriteMode not AppendModeGravatar Joey Hess2016-12-07
| | | | AppendMode does not allow seeking..
* open file for append, not write, so resuming worksGravatar Joey Hess2016-12-07
| | | | | WriteMode zeros any existing content, so the seek filled with zeros, and verification failed after download.
* offset meters when resumingGravatar Joey Hess2016-12-07
|
* more p2p progress metersGravatar Joey Hess2016-12-07
| | | | | | | | | Display progress meter on send and receive from remote. Added a new hGetMetered that can read an exact number of bytes (or less), updating a meter as it goes. This commit was sponsored by Andreas on Patreon.
* update progress meter when sending to p2p remoteGravatar Joey Hess2016-12-07
| | | | This commit was sponsored by Thom May on Patreon.
* need to auth with the peerGravatar Joey Hess2016-12-06
|
* refactorGravatar Joey Hess2016-12-06
|
* added StoreContentToGravatar Joey Hess2016-12-06
| | | | | | | | This is needed in addition to StoreContent, because retrieveKeyFile can be used to retrieve to different destination files, not only the tmp file for a key. This commit was sponsored by Ole-Morten Duesund on Patreon.
* stub Remote.P2PGravatar Joey Hess2016-12-06
| | | | | | | Similar to GCrypt remotes, P2P remotes have an url, so Remote.Git has to separate them out and handle them, passing off to Remote.P2P. This commit was sponsored by Ignacio on Patreon.
* plumb assicated files through P2P protocol for updating transfer logsGravatar Joey Hess2016-12-02
| | | | | | | | | | ReadContent can't update the log, since it reads lazily. This part of the P2P monad will need to be rethought. Associated files are heavily sanitized when received from a peer; they could be an exploit vector. This commit was sponsored by Jochen Bartl on Patreon.
* plumb peer uuid through to runLocalGravatar Joey Hess2016-12-02
| | | | This will allow updating transfer logs with the uuid.
* initial implementation of P2P.Annex runnerGravatar Joey Hess2016-12-02
| | | | | | | | | | | | | | | Untested, and it does not yet update transfer logs. Verifying transferred content is modeled on git-annex-shell recvkey. In a direct mode or annex.thin repository, content can change while it's being transferred. So, verification is always done, even if annex.verify would normally prevent it. Note that a WORM or URL key could change in a way the verification doesn't catch. That can happen in git-annex-shell recvkey too. We don't worry about it, because those key backends don't guarantee preservation of data. (Which is to say, I worried about it, and then convinced myself again it was ok.)
* catch non-IO exceptions tooGravatar Joey Hess2016-12-02
|
* make remote-daemon able to send and receive objects over torGravatar Joey Hess2016-12-02
| | | | | | | | Each worker thread needs to run in the Annex monad, but the remote-daemon's liftAnnex can only run 1 action at a time. Used Annex.Concurrent to deal with that. P2P.Annex is incomplete as of yet.
* improve Local monadGravatar Joey Hess2016-12-02
|
* make sure that the specified number of bytes of DATA are always sentGravatar Joey Hess2016-12-02
| | | | | | | It's possible, in direct or thin mode, that an object file gets truncated or appended to as it's being sent. This would break the protocol badly, so make sure never to send too many bytes, and to close the protocol connection if too few bytes are available.
* cleanupsGravatar Joey Hess2016-12-01
|
* more flexible types for Proto runnersGravatar Joey Hess2016-12-01
| | | | This will allow a runner in the Annex monad.
* better commentsGravatar Joey Hess2016-11-30
|
* actually check p2p authtokens for tor connectionsGravatar Joey Hess2016-11-30
| | | | This commit was sponsored by Ethan Aubin.
* use P2P auth for git-remote-tor-annexGravatar Joey Hess2016-11-30
| | | | | | | This changes the environment variable name to the more generic GIT_ANNEX_P2P_AUTHTOKEN. This commit was sponsored by andrea rota.
* implement p2p --linkGravatar Joey Hess2016-11-30
| | | | This commit was sponsored by Riku Voipio.
* implement p2p commandGravatar Joey Hess2016-11-30
|
* commentsGravatar Joey Hess2016-11-29
|
* finish git-annex enable-torGravatar Joey Hess2016-11-29
| | | | | Make it stash the address away for git-annex p2p to use later, rather than outputting it. And, look up the UUID itself.