mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

add getFileSize, which can get the real size of a large file on Windows

Browse source 
Avoid using fileSize which maxes out at just 2 gb on Windows.
Instead, use hFileSize, which doesn't have a bounded size.
Fixes support for files > 2 gb on Windows.

Note that the InodeCache code only needs to compare a file size,
so it doesn't matter it the file size wraps. So it has been
left as-is. This was necessary both to avoid invalidating existing inode
caches, and because the code passed FileStatus around and would have become
more expensive if it called getFileSize.

This commit was sponsored by Christian Dietrich.
This commit is contained in:
Joey Hess 2015-01-20 16:58:48 -04:00
parent 87c4f0e320
commit 4f657aa14e
22 changed files with 94 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

2
doc/bugs/Direct_mode_sync_fails_to_transfer_a_10GB_file.mdwn
View file

@ -56,3 +56,5 @@ git-annex: sync: 1 failed
# End of transcript or log.
"""]]
> [[fixed|done]] --[[Joey]]

25
doc/bugs/Direct_mode_sync_fails_to_transfer_a_10GB_file/comment_1_cb9e9dada7baf4e48725a9483b3b448a._comment Normal file
View file

@ -0,0 +1,25 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2015-01-20T19:41:44Z"
content="""
In my own test, I made a 10 gb file, and the key git-annex came up with had
a size of -2147483648 which is clearly screwed up.. But that's what
getFileStatus reports the size as. This was in an 32 bit XP VM.
Hmm, unix-compat's getFileStatus calls getFileSize, which yields a
FileOffset. The maxBound of that on linux is a nice large
9223372036854775807, but on Windows, it appears to be 2147483647.
Compare with using hFileSize, which yields an Integer. So,
getFileSize and fileSize are unsafe on Windows due to FileOffset being so
small on Windows.
I have now corrected all places in git-annex that used the unsafe fileSize.
It will behave correctly on Windows now.
However, if you still have the repo with the big file, it's key still has
the wrong size. To fix, you can "git annex unannex" the file, and then "git
annex add" it back, after upgrading to the current daily build, or the next
release of git-annex.
"""]]