summaryrefslogtreecommitdiff
path: root/doc/bugs/Unable_to_parallel_fsck.mdwn
blob: 2d419915dc6eec910ae2a4e3dc0d7bdbf49562cb (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
### Please describe the problem.

If I run git annex fsck with the parallel jobs switch (e.g. -J2) it fails when it encounters files with non-ASCII filenames. It works fine without
this switch.


### What steps will reproduce the problem?

Make a git annex repository with non-ascii filenames, add the files, and then run fsck with the parallel switch. A script to do this
is included below.


### What version of git-annex are you using? On what operating system?

Arch Linux (64bit)

git-annex version: 6.20160114-g297a744
build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3(multipartupload) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput DNS Feeds Quvi TDFA TorrentParser
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
local repository version: 5
supported repository versions: 5 6
upgrade supported from repository versions: 0 1 2 4 5

### Please provide any additional information below.

[[!format sh """
#!/bin/bash
set -o errexit -o nounset

mkdir testing-repo
cd testing-repo
git init
git annex init testing-repo

make_fake_file() {
  local filename="$1"
  mkdir -p "$(dirname "$filename")"
  echo "hello world" > "$filename"
  git annex add "$filename"
}

export -f make_fake_file
parallel -j1 'make_fake_file {}' <<EOF
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/folder.jpg
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/01 - 遥か彼方 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/02 - 未来の破片 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/03 - アンダースタンド - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/04 - 君という花 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/05 - リライト - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/06 - 君の街まで - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/07 - ループ & ループ - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/08 - ブラックアウト - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/09 - ブルートレイン - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/10 - 或る街の群青 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/11 - アフターダーク - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/12 - 転がる岩、君に朝が降る - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/13 - ムスタング - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/14 - 藤沢ルーザー - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/15 - 新世紀のラブソング - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/16 - ソラニン - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/17 - マーチングバンド - ASIAN KUNG-FU GENERATION.flac
EOF

git commit -m 'Add testing files.'
git annex fsck -J2
"""]]

On my system this produces:

[[!format sh """
fsck ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/01 - git-annex: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character)
FAIL 1
"""]]

(The FAIL 1 output is just my terminal printing that the exit code was 1)

### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Plenty. In fact I've been using it for a long time - I just only recently tried to use -J2 to speed up the fsck'ing :)


[[!meta title="-J can crash on displaying filenames not supported by current locale"]]