summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGravatar CandyAngel <CandyAngel@web>2015-06-17 07:51:53 +0000
committerGravatar admin <admin@branchable.com>2015-06-17 07:51:53 +0000
commit623991ff30b640155bf4c291131d30c652a58732 (patch)
tree8c75dd46d330033379cbefd233cfc3c9c48f6f77
parent1be90b59322a99d56663bf16251ebf67a83ea5ca (diff)
Added a comment
-rw-r--r--doc/forum/Handling_a_large_number_of_files/comment_1_33d2e19fbfdf5a0bfb7b4fbc840824a2._comment13
1 files changed, 13 insertions, 0 deletions
diff --git a/doc/forum/Handling_a_large_number_of_files/comment_1_33d2e19fbfdf5a0bfb7b4fbc840824a2._comment b/doc/forum/Handling_a_large_number_of_files/comment_1_33d2e19fbfdf5a0bfb7b4fbc840824a2._comment
new file mode 100644
index 000000000..055f69f26
--- /dev/null
+++ b/doc/forum/Handling_a_large_number_of_files/comment_1_33d2e19fbfdf5a0bfb7b4fbc840824a2._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="CandyAngel"
+ subject="comment 1"
+ date="2015-06-17T07:51:53Z"
+ content="""
+I've posted a few tips on this site about dealing with large (file count) repositories.. I should probably collate them into a single post in tips (and I shall go so!).
+
+In terms of directories containing hundreds of thousands of files, my research indicates that the slowness stems from how the filesystem (at least, for ext3) fetches the listing, as it requires multiple \"trips\" to query it (it only returns 32KB of metadata at a time or something like that).
+
+So you can really speed it up by partitioning your files in such a way that a directory listing returns quickly. Personally, I use fpart and 5000 files per directory, but I think you can go up to about 20000.
+
+If you do 'find', and it takes a really long time to *start* printing paths, you have too many files :)
+"""]]