diff options
author | 2016-09-08 12:04:09 -0400 | |
---|---|---|
committer | 2016-09-08 12:04:09 -0400 | |
commit | a79530f79a0d418a4ef9efd129b64fd261c49f1b (patch) | |
tree | 9cf01d27ba6767cc5e14a0d7103e4b6bb2d2c9cf | |
parent | ef820c4dd3aac720f29a5c4a8f7e60f63b4879dd (diff) |
designs
-rw-r--r-- | doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_1_b71937cb772b3400839755bb11a1ffe4._comment | 65 |
1 files changed, 65 insertions, 0 deletions
diff --git a/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_1_b71937cb772b3400839755bb11a1ffe4._comment b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_1_b71937cb772b3400839755bb11a1ffe4._comment new file mode 100644 index 000000000..df74f19cb --- /dev/null +++ b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_1_b71937cb772b3400839755bb11a1ffe4._comment @@ -0,0 +1,65 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2016-09-08T15:32:00Z" + content=""" +yoh pointed out in email that reusing the special remote interface would +not work with -J because it doesn't tell what key progress is being shown +for. + +Also, note that --json -J does not currently output json; -J overrides the +--json to instead use concurrent-output for progress display. The current +way that json is emitted incrementally would need to be changed to be usable +with -J. + +Is there any need for a program that is driving git-annex with --json +to use -J, instead of just starting several git-annex processes? The only +major benefit I can think of is the recently added better selection between +multiple remotes by parallel `git annex get`. Other performance benefits +between -J and multiple processes should mostly be small. + +To suppose --json -J, instead of a partial json object that later gets added +to as the command progresses: + + {"command":"get","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar","note":"from d1..." + +We'd need something like this, with each json object buffered +and only output when it was complete (and output serialized so +objects are written one at a time): + + {"command":"get","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar","note":"from d1...", "inprogress":true} + {"command":"get","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","success":true} + +If we're doing that, we might as well use json for the progress info too, +in between the two json objects in the example above: + + {"percent-progress":"25%","byte-progress":500,"key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3"} + {"percent-progress":"75%","byte-progress":1500","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3"} + +But this is a large change to the json format, and would probably break +all consumers of it. I don't like that, but also don't like the complexity +of having two different varieties of json output for parallel and +non-parallel modes. + +Hmm, if we're buffering the "command" json objects, we could keep them +formatted the same as they are now, only displaying them when done. And +progress objects could be inserted before: + + {"percent-progress":"25%","byte-progress":500,"key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar"} + {"percent-progress":"75%","byte-progress":1500","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar"} + {"command":"get","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar","note":"from d1...","success":true} + +That seems a bit verbose, but I think all that info could be needed by +consumers of the progress objects. Hmm, since the "command" json objects +are being buffered, we could even include the currently buffered object +nested inside the progress object: + + {"percent-progress":"25%","byte-progress":500,"action":{"command":"get","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar","note":"from d1..."}} + {"percent-progress":"75%","byte-progress":1500,"action":{"command":"get","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar","note":"from d1..."}} + {"command":"get","key":"SHA256E-s5242880--82cb4363f596cb66e7bc6e4cbfd2bfe8a8b6ac7e6d02557cc0e3944ec8faafc3","file":"bar","note":"from d1...","success":true} + +That's not a lot more verbose than the earlier version, and it ensures that +consumers of the progress objects have all possible information available +(including the name of the remote being downloaded from in the above +example). +"""]] |