mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/CHANGELOG

7551 lines
377 KiB
Text
Raw Normal View History

avoid exporting non-annexed symlinks So that importing does not replace them with plain files. This works similarly to how the previous handling of submodules and matchers did, except that annexed symlinks still get exported as plain files of course, it's only non-annexed symlinks that it does not make sense to export. When symlinks have previously been exported, updating the export will unexport them after upgrading to this commit. Sponsored-by: Kevin Mueller on Patreon
2022-01-03 18:21:50 +00:00
git-annex (8.20211232) UNRELEASED; urgency=medium
* export: When a non-annexed symlink is in the tree to be exported, skip it.
* import: When the previously exported tree contained a non-annexed symlink,
preserve it in the imported tree so it does not get deleted.
renameremote: Better handling of case where there are multiple special remotes with a name Instead of renaming one at random, error out and ask that a uuid be specified. Sponsored-by: Brett Eisenberg on Patreon
2022-01-05 19:24:02 +00:00
* enableremote, renameremote: Better handling of the unusual case where
enableremote: Better handling of the unusual case where multiple special remotes have been initialized with the same name Before it would pick one at random, though preferring ones that were not dead over dead ones. Now, if one is dead and the other not, it will use the non-dead one. But if both are not dead, or both dead, it will error out, suggesting the user clarify what they want to enable. Sponsored-by: Luke Shumaker on Patreon
2022-01-05 19:12:01 +00:00
multiple special remotes have been initialized with the same name.
fix handling of corrupted data received from git remote Recover from corrupted content being received from a git remote due eg to a wire error, by deleting the temporary file when it fails to verify. This prevents a retry from failing again. Reversion introduced in version 8.20210903, when incremental verification was added. Only the git remote seems to be affected, although it is certianly possible that other remotes could later have the same issue. This only affects things passed to getViaTmp that return (False, UnVerified) due to verification failing. As far as getViaTmp can tell, that could just as well mean that the transfer failed in a way that would resume, so it cannot delete the temp file itself. Remote.Git and P2P.Annex use getViaTmp internally, while other remotes do not, which is why only it seems affected. A better fix perhaps would be to improve the types of the callback passed to getViaTmp, so that some other value could be used to indicate the state where the transfer succeeded but verification failed. Sponsored-by: Boyd Stephen Smith Jr.
2022-01-07 17:17:43 +00:00
* Recover from corrupted content being received from a git remote,
by deleting the temporary file when it fails to verify. This prevents
a retry from failing again.
(reversion introduced in version 8.20210903)
avoid exporting non-annexed symlinks So that importing does not replace them with plain files. This works similarly to how the previous handling of submodules and matchers did, except that annexed symlinks still get exported as plain files of course, it's only non-annexed symlinks that it does not make sense to export. When symlinks have previously been exported, updating the export will unexport them after upgrading to this commit. Sponsored-by: Kevin Mueller on Patreon
2022-01-03 18:21:50 +00:00
-- Joey Hess <id@joeyh.name> Mon, 03 Jan 2022 14:01:14 -0400
releasing package git-annex version 8.20211231
2021-12-31 19:11:50 +00:00
git-annex (8.20211231) upstream; urgency=medium
Fix build with old versions of feed library
2021-11-23 20:06:51 +00:00
releasing package git-annex version 8.20211231
2021-12-31 19:11:50 +00:00
* Improved support for using git-annex in a read-only repository,
git-annex branch information from remotes that cannot be merged into
the git-annex branch will now not crash it, but will be merged in
memory.
addurl, youtube-dl: When --check-raw prevents downloading an url, still continue with any downloads that come after it, rather than erroring out Sponsored-By: Mark Reidenbach on Patreon
2021-11-28 23:40:06 +00:00
* addurl, youtube-dl: When --check-raw prevents downloading an url,
still continue with any downloads that come after it, rather than
erroring out.
avoid concurrent threads trying to take pid lock at same time Seem there are several races that happen when 2 threads run PidLock.tryLock at the same time. One involves checkSaneLock of the side lock file, which may be deleted by another process that is dropping the lock, causing checkSaneLock to fail. And even with the deletion disabled, it can still fail, Probably due to linkToLock failing when a second thread overwrites the lock file. The same can happen when 2 processes do, but then one process just fails to take the lock, which is fine. But with 2 threads, some actions where failing even though the process as a whole had the pid lock held. Utility.LockPool.PidLock already maintains a STM lock, and since it uses LockShared, 2 threads can hold the pidlock at the same time, and when the first thread drops the lock, it will remain held by the second thread, and so the pid lock file should not get deleted until the last thread to hold it drops the lock. Which is the right behavior, and why a LockShared STM lock is used in the first place. The problem is that each time it takes the STM lock, it then also calls PidLock.tryLock. So that was getting called repeatedly and concurrently. Fixed by noticing when the shared lock is already held, and stop calling PidLock.tryLock again, just use the pid lock that already exists then. Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed to take the lock, which was entirely wrong. It should only drop the side lock. Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
* Fix locking problems when annex.pidlock is set and concurrency is
enabled eg with -J.
Improve error message display when autoinit fails Due to eg, a permissions problem.
2021-12-09 18:38:12 +00:00
* Improve error message display when autoinit fails due to eg, a
permissions problem.
releasing package git-annex version 8.20211231
2021-12-31 19:11:50 +00:00
* export: Avoid unncessarily re-exporting non-annexed files that were
already exported.
improve git command queue flushing with time limit So that eg, addurl of several large files that take time to download will update the index for each file, rather than deferring the index updates to the end. In cases like an add of many smallish files, where a new file is being added every few seconds. In that case, the queue will still build up a lot of changes which are flushed at once, for best performance. Since the default queue size is 10240, often it only gets flushed once at the end, same as before. (Notice that updateQueue updated _lastchanged when adding a new item to the queue without flushing it; that is necessary to avoid it flushing the queue every 5 minutes in this case.) But, when it takes more than a 5 minutes to add a file, the overhead of updating the index immediately is probably small, so do it after each file. This avoids git-annex potentially taking a very very long time indeed to stage newly added files, which can be annoying to the user who would like to get on with doing something with the files it's already added, eg using git mv to rename them to a better name. This is only likely to cause a problem if it takes say, 30 seconds to update the index; doing an extra 30 seconds of work after every 5 minute file add would be less optimal. Normally, updating the index takes significantly less time than that. On a SSD with 100k files it takes less than 1 second, and the index write time is bound by disk read and write so is not too much worse on a hard drive. So I hope this will not impact users, although if it does turn out to, the time limit could be made configurable. A perhaps better way to do it would be to have a background worker thread that wakes up every 60 seconds or so and flushes the queue. That is made somewhat difficult because the queue can contain Annex actions and so this would add a new source of concurrency issues. So I'm trying to avoid that approach if possible. Sponsored-by: Erik Bjäreholt on Patreon
2021-12-14 15:48:07 +00:00
* Improve git command queue flushing so that eg, addurl of several
large files that take time to download will update the index for each
file, rather than deferring the index updates to the end.
improve sync message in export edge case sync: Better error message when unable to export to a remote because remote.name.annex-tracking-branch is configured to a ref that does not exist. It does not suggest how to fix the problem because there are several possible solutions: Change the git config to point to something that does exist, git add some files, or put files on the special remote that will be imported and so populate the ref. I considered just silently not doing anything, which is what it does when annex-tracking-branch = master and nothing has been committed to master yet. But it seems better to be explicit about it, since this is a fairly confusing situation to find yourself in. Sponsored-By: Max Thoursie on Patreon
2021-12-23 18:44:03 +00:00
* sync: Better error message when unable to export to a remote because
remote.name.annex-tracking-branch is configured to a ref that does not
exist.
releasing package git-annex version 8.20211231
2021-12-31 19:11:50 +00:00
* Fix build with ghc 9.0.1
* Fix build with old versions of feed library.
Fix build with old versions of feed library
2021-11-23 20:06:51 +00:00
releasing package git-annex version 8.20211231
2021-12-31 19:11:50 +00:00
-- Joey Hess <id@joeyh.name> Fri, 31 Dec 2021 15:03:36 -0400
Fix build with old versions of feed library
2021-11-23 20:06:51 +00:00
releasing package git-annex version 8.20211123
2021-11-23 19:20:24 +00:00
git-annex (8.20211123) upstream; urgency=medium
importfeed: Display url before starting youtube-dl download It was displaying a blank line before.
2021-11-17 17:23:55 +00:00
soften language in changelog This bug mostly would happen when the downloads ran very fast or were all failing (how I reproduced it), because there have to be two downloads that finish very close to the same time to trigger the race. So most users of -J probably would not see much impact from the bug.
2021-11-19 16:52:22 +00:00
* Bugfix: When -J was enabled, getting files could leak an
ever-growing number of git cat-file processes.
support git 2.34.0's handling of merge conflict between annexed and non-annexed file This version of git -- or its new default "ort" resolver -- handles such a conflict by staging two files, one with the original name and the other named file~ref. Use unmergedSiblingFile when the latter is detected. (It doesn't do that when the conflict is between a directory and a file or symlink though, so see previous commit for how that case is handled.) The sibling file has to be deleted separately, because cleanConflictCruft may not delete it -- that only handles files that are annex links, but the sibling file may be the non-annexed file side of the conflict. The graftin code had assumed that, when the other side of a conclict is a symlink, the file in the work tree will contain the non-annexed content that we want it to contain. But that is not the case with the new git; the file may be the annex link and needs to be replaced with the content, while the annex link will be written as a -variant file. (The weird doesDirectoryExist check in graftin turns out to still be needed, test suite failed when I tried to remove it.) Test suite passes with new git with ort resolver default. Have not tried it with old git or other defaults. Sponsored-by: Noam Kremen on Patreon
2021-11-22 19:40:03 +00:00
* Support git's new "ort" resolver, which became the default in git 2.34.0,
and broke the test suite and automatic merge resolution of a conflict
between an annexed file and a non-annexed file.
releasing package git-annex version 8.20211123
2021-11-23 19:20:24 +00:00
* importfeed: Display url before starting youtube-dl download.
importfeed: Display url before starting youtube-dl download It was displaying a blank line before.
2021-11-17 17:23:55 +00:00
releasing package git-annex version 8.20211123
2021-11-23 19:20:24 +00:00
-- Joey Hess <id@joeyh.name> Tue, 23 Nov 2021 15:19:16 -0400
importfeed: Display url before starting youtube-dl download It was displaying a blank line before.
2021-11-17 17:23:55 +00:00
releasing package git-annex version 8.20211117
2021-11-17 16:20:29 +00:00
git-annex (8.20211117) upstream; urgency=medium
metadata --batch: Avoid crashing when a non-annexed file is input Turns out that CommandStart actions do not have their exceptions caught, which is why the giveup was causing a crash. Mostly these actions do not do very much work on their own, but it does seem possible there are other commands whose CommandStart also throws an exception. So, my first attempt at a fix was to catch those exceptions. But, --json-error-messages then causes a difficulty, because in order to output a json error message, an action needs to have been started; that sets up the json object that the error message will be included in a field of. While it would be possible to output an object with just an error field, this would be json output of a format that the user has no reason to expect, that happens only in an exceptional circumstance. That is something I have always wanted to avoid with the json output; while git-annex man pages don't document what the json looks like, the output has always been made to be self-describing. Eg, it includes "error-messages":[] even when there's no errors. With that ruled out, it doesn't seem a good idea to catch CommandStart exceptions and display the error to stderr when --json-error-messages is set. And so I don't know if it makes sense to catch exceptions from that at all. Maybe I'd have a different opinion if --json-error-messages did not exist though. So instead, output a blank line like other batch commands do. This also leaves open the possibility of implementing support for matching object with metadata --json, which would also want to output a blank line when the input didn't match. Sponsored-by: Dartmouth College's DANDI project
2021-11-01 17:38:14 +00:00
add git-annex filter-process filter-process: New command that can make git add/checkout faster when there are a lot of unlocked annexed files or non-annexed files, but that also makes git add of large annexed files slower. Use it by running: git config filter.annex.process 'git-annex filter-process' Fully tested and working, but I have not benchmarked it at all. And, incremental hashing is not done when git add uses it, so extra work is done in that case. Sponsored-by: Mark Reidenbach on Patreon
2021-11-04 19:02:36 +00:00
* filter-process: New command that can make git add/checkout faster when
there are a lot of unlocked annexed files or non-annexed files, but that
also makes git add of large annexed files slower. Use it by running:
git config filter.annex.process 'git-annex filter-process'
Fix a typo in the name of youtube-dl (reversion introduced in version 8.20210903)
2021-11-13 12:58:36 +00:00
* Fix a typo in the name of youtube-dl
(reversion introduced in version 8.20210903)
releasing package git-annex version 8.20211117
2021-11-17 16:20:29 +00:00
* git-lfs: Fix interoperability with gitlab's implementation of the
git-lfs protocol, which requests Content-Encoding chunked.
importfeed: Fix a crash when used in a non-unicode locale See comment for analysis. At first I thought I'd need to convert all T.unpack in git-annex, but luckily not -- so long as the Text is read from a file, the filesystem encoding is applied and T.unpack is fine. It's only when using Feed that the filesystem encoding is not applied. While this fixes the crash, it does result in some mojibake, eg: itemid=http://www.manager-tools.com/2014/01/choosing-a-company-work-chapter-7-���-questions/ Have not tracked that down, but it must be unrelated, because I've verified that it roundtrips when using encodeUf8: joey@darkstar:~/src/git-annex>LANG=C ghci Utility/FileSystemEncoding.hs ghci> useFileSystemEncoding ghci> Just f <- Text.Feed.Import.parseFeedFromFile "/home/joey/tmp/career_tools_podcasts.xml" ghci> Just (_, x) = Text.Feed.Query.getItemId (Text.Feed.Query.feedItems f !! 0) ghci> decodeBS (Data.Text.Encoding.encodeUtf8 x) "http://www.manager-tools.com/2014/01/choosing-a-company-work-chapter-7-\56546\56448\56467-questions/" ghci> writeFile "foo" $ decodeBS (Data.Text.Encoding.encodeUtf8 x) Writes a file containing the ENDASH character. Sponsored-by: Jochen Bartl on Patreon
2021-11-15 17:32:31 +00:00
* importfeed: Fix a crash when used in a non-unicode locale.
releasing package git-annex version 8.20211117
2021-11-17 16:20:29 +00:00
* migrate: New --remove-size option.
* uninit: Avoid error message when no commits have been made to the
repository yet.
* uninit: Avoid error message when there is no git-annex branch.
* metadata --batch: Avoid crashing when a non-annexed file is input,
instead output a blank line like other batch commands do.
* metadata --batch --json: Reject input whose "fields" does not consist
of arrays of strings. Such invalid input used to be silently ignored.
metadata --batch: Avoid crashing when a non-annexed file is input Turns out that CommandStart actions do not have their exceptions caught, which is why the giveup was causing a crash. Mostly these actions do not do very much work on their own, but it does seem possible there are other commands whose CommandStart also throws an exception. So, my first attempt at a fix was to catch those exceptions. But, --json-error-messages then causes a difficulty, because in order to output a json error message, an action needs to have been started; that sets up the json object that the error message will be included in a field of. While it would be possible to output an object with just an error field, this would be json output of a format that the user has no reason to expect, that happens only in an exceptional circumstance. That is something I have always wanted to avoid with the json output; while git-annex man pages don't document what the json looks like, the output has always been made to be self-describing. Eg, it includes "error-messages":[] even when there's no errors. With that ruled out, it doesn't seem a good idea to catch CommandStart exceptions and display the error to stderr when --json-error-messages is set. And so I don't know if it makes sense to catch exceptions from that at all. Maybe I'd have a different opinion if --json-error-messages did not exist though. So instead, output a blank line like other batch commands do. This also leaves open the possibility of implementing support for matching object with metadata --json, which would also want to output a blank line when the input didn't match. Sponsored-by: Dartmouth College's DANDI project
2021-11-01 17:38:14 +00:00
releasing package git-annex version 8.20211117
2021-11-17 16:20:29 +00:00
-- Joey Hess <id@joeyh.name> Wed, 17 Nov 2021 12:18:49 -0400
metadata --batch: Avoid crashing when a non-annexed file is input Turns out that CommandStart actions do not have their exceptions caught, which is why the giveup was causing a crash. Mostly these actions do not do very much work on their own, but it does seem possible there are other commands whose CommandStart also throws an exception. So, my first attempt at a fix was to catch those exceptions. But, --json-error-messages then causes a difficulty, because in order to output a json error message, an action needs to have been started; that sets up the json object that the error message will be included in a field of. While it would be possible to output an object with just an error field, this would be json output of a format that the user has no reason to expect, that happens only in an exceptional circumstance. That is something I have always wanted to avoid with the json output; while git-annex man pages don't document what the json looks like, the output has always been made to be self-describing. Eg, it includes "error-messages":[] even when there's no errors. With that ruled out, it doesn't seem a good idea to catch CommandStart exceptions and display the error to stderr when --json-error-messages is set. And so I don't know if it makes sense to catch exceptions from that at all. Maybe I'd have a different opinion if --json-error-messages did not exist though. So instead, output a blank line like other batch commands do. This also leaves open the possibility of implementing support for matching object with metadata --json, which would also want to output a blank line when the input didn't match. Sponsored-by: Dartmouth College's DANDI project
2021-11-01 17:38:14 +00:00
releasing package git-annex version 8.20211028
2021-10-28 16:00:56 +00:00
git-annex (8.20211028) upstream; urgency=medium
remove git-annex-shell compat code * Removed support for accessing git remotes that use versions of git-annex older than 6.20180312. * git-annex-shell: Removed several commands that were only needed to support git-annex versions older than 6.20180312. (lockcontent, recvkey, sendkey, transferinfo, commit) The P2P protocol was added in that version, and used ever since, so this code was only needed for interop with older versions. "git-annex-shell commit" is used by newer git-annex versions, though unnecessarily so, because the p2pstdio command makes a single commit at shutdown. Luckily, it was run with stderr and stdout sent to /dev/null, and non-zero exit status or other exceptions are caught and ignored. So, that was able to be removed from git-annex-shell too. git-annex-shell inannex, recvkey, sendkey, and dropkey are still used by gcrypt special remotes accessed over ssh, so those had to be kept. It would probably be possible to convert that to using the P2P protocol, but it would be another multi-year transition. Some git-annex-shell fields were able to be removed. I hoped to remove all of them, and the very concept of them, but unfortunately autoinit is used by git-annex sync, and gcrypt uses remoteuuid. The main win here is really in Remote.Git, removing piles of hairy fallback code. Sponsored-by: Luke Shumaker
2021-10-11 19:35:54 +00:00
* Removed support for accessing git remotes that use versions of
git-annex older than 6.20180312.
* git-annex-shell: Removed several commands that were only needed to
support git-annex versions older than 6.20180312.
negotiate P2P protocol version for tor remotes This negotiation is not supported by versions of git-annex older than 6.20180312. Well, maybe really 6.20180227 or so, but using that in the changelog simplifies things since it was the version for the other changes as well. See commit c81768d425725f868ddf23333d49eed0f3fda011 for the back story. As well as allowing for future protocol improvements, this will result in negoatiating protocol version 1, which is an improvement over default version 0. In fact, it looks like no supported version of git-annex will use protocol version 0, since version 1 was introduced in 6.20180227. Still, removing the code for version 0 seems unncessary. See commit 31e1adc005e8181c5190a9f39a3f148e10f4d364. Sponsored-by: Brett Eisenberg on Patreon.
2021-10-11 19:53:49 +00:00
* Negotiate P2P protocol version with tor remotes, allowing
use of protocol version 1. This negotiation is not supported
by versions of git-annex older than 6.20180312.
releasing package git-annex version 8.20211028
2021-10-28 16:00:56 +00:00
* Fix bug that caused stale git-annex branch information to read
when annex.private or remote.name.annex-private is set.
improve sqlite MultiWriter handling of read after write This removes a messy caveat that was easy to forget and caused at least one bug. The price paid is that, after a write to a MultiWriter db, it has to close the db connection that it had been using to read, and open a new connection. So it might be a little bit slower. But, writes are usually batched together, so there's often only a single write, and so there should not be much of a slowdown. Notice that SingleWriter already closed the db connection after a write, so paid the same overhead. This is the second try at fixing a bug: git-annex get when run as the first git-annex command in a new repo did not populate all unlocked files. (Reversion in version 8.20210621) Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-10-19 19:13:29 +00:00
* git-annex get when run as the first git-annex command in a new repo
did not populate all unlocked files.
(Reversion in version 8.20210621)
releasing package git-annex version 8.20211028
2021-10-28 16:00:56 +00:00
* Fix using lookupkey and info inside a subdirectory of the git repository.
Fix using lookupkey inside a subdirectory Caused by dirContains ".." "foo" being incorrectly False. Also added a test of dirContains, which includes all the previous bug fixes I could find and some obvious cases. Reversion in version 8.20211011 Sponsored-by: Brett Eisenberg on Patreon
2021-10-26 18:58:44 +00:00
(Reversion in version 8.20211011)
releasing package git-annex version 8.20211028
2021-10-28 16:00:56 +00:00
* Avoid some sqlite crashes on Windows SubSystem for Linux (WSL).
* Call annex.freezecontent-command on the annex object file
avoid redundant freezeContent call This opens the potential for the object file to be in place but git-annex is interrupted before it can freeze it. git-annex fsck already fixes that situation, which can also occur when lockContentForRemoval thaws content. Also improve comment to not be Windows-specific.
2021-10-27 18:11:34 +00:00
after it has been moved into place in annex/objects/. This allows
the hook to freeze the file in ways that prevent moving it, such as
removing the Windows delete permission.
Thanks, Reiko Asakura.
fix addurl concurrency issue addurl: Support adding the same url to multiple files at the same time when using -J with --batch --with-files. Implementation was easier than expected, was able to reuse OnlyActionOn. While it will download the url's content multiple times, that seems like the best thing to do; see my comment for why. Sponsored-by: Dartmouth College's DANDI project
2021-10-27 20:15:41 +00:00
* addurl: Support adding the same url to multiple files at the same
time when using -J with --batch --with-files.
releasing package git-annex version 8.20211028
2021-10-28 16:00:56 +00:00
* When retrieval from a chunked remote fails, display the error that
occurred when downloading the chunk, rather than the error that
occurred when trying to download the unchunked content, which is less
likely to actually be stored in the remote.
* Avoid crashing tilde expansion on user who does not exist.
* test: Put gpg temp home directory in system temp directory,
not filesystem being tested.
remove git-annex-shell compat code * Removed support for accessing git remotes that use versions of git-annex older than 6.20180312. * git-annex-shell: Removed several commands that were only needed to support git-annex versions older than 6.20180312. (lockcontent, recvkey, sendkey, transferinfo, commit) The P2P protocol was added in that version, and used ever since, so this code was only needed for interop with older versions. "git-annex-shell commit" is used by newer git-annex versions, though unnecessarily so, because the p2pstdio command makes a single commit at shutdown. Luckily, it was run with stderr and stdout sent to /dev/null, and non-zero exit status or other exceptions are caught and ignored. So, that was able to be removed from git-annex-shell too. git-annex-shell inannex, recvkey, sendkey, and dropkey are still used by gcrypt special remotes accessed over ssh, so those had to be kept. It would probably be possible to convert that to using the P2P protocol, but it would be another multi-year transition. Some git-annex-shell fields were able to be removed. I hoped to remove all of them, and the very concept of them, but unfortunately autoinit is used by git-annex sync, and gcrypt uses remoteuuid. The main win here is really in Remote.Git, removing piles of hairy fallback code. Sponsored-by: Luke Shumaker
2021-10-11 19:35:54 +00:00
releasing package git-annex version 8.20211028
2021-10-28 16:00:56 +00:00
-- Joey Hess <id@joeyh.name> Thu, 28 Oct 2021 11:58:29 -0400
remove git-annex-shell compat code * Removed support for accessing git remotes that use versions of git-annex older than 6.20180312. * git-annex-shell: Removed several commands that were only needed to support git-annex versions older than 6.20180312. (lockcontent, recvkey, sendkey, transferinfo, commit) The P2P protocol was added in that version, and used ever since, so this code was only needed for interop with older versions. "git-annex-shell commit" is used by newer git-annex versions, though unnecessarily so, because the p2pstdio command makes a single commit at shutdown. Luckily, it was run with stderr and stdout sent to /dev/null, and non-zero exit status or other exceptions are caught and ignored. So, that was able to be removed from git-annex-shell too. git-annex-shell inannex, recvkey, sendkey, and dropkey are still used by gcrypt special remotes accessed over ssh, so those had to be kept. It would probably be possible to convert that to using the P2P protocol, but it would be another multi-year transition. Some git-annex-shell fields were able to be removed. I hoped to remove all of them, and the very concept of them, but unfortunately autoinit is used by git-annex sync, and gcrypt uses remoteuuid. The main win here is really in Remote.Git, removing piles of hairy fallback code. Sponsored-by: Luke Shumaker
2021-10-11 19:35:54 +00:00
releasing package git-annex version 8.20211011
2021-10-11 16:53:17 +00:00
git-annex (8.20211011) upstream; urgency=medium
borg: Avoid trying to extract xattrs, ACLS, and bsdflags when retrieving from a borg repository That broke restoring on linux from a borg backup made on OSX. Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-09-03 16:10:14 +00:00
improved bwrate limiting implementation New method is much better. Avoids unrestrained transfer at the beginning (except for the first block. Keeps right at or a few kb/s below the configured limit, with very little varation in the actual reported bandwidth. Removed the /s part of the config as it's not needed. Ready to merge. Sponsored-by: Luke Shumaker on Patreon
2021-09-22 19:14:28 +00:00
* Added annex.bwlimit and remote.name.annex-bwlimit config to limit
the bandwidth of transfers. It works for git remotes and many
but not all special remotes.
fix remote git config vs global git config order Bug fix: Git configs such as annex.verify were incorrectly overriding per-remote git configs such as remote.name.annex-verify. This dates all the way back to 2013, commit 8a5b397ac47b21906f1fbb6254f5b70b3beb2a52, where hlint apparently somehow confused me into parsing in the wrong order. Before that it was correct. Amazing noone has noticed until now. Sponsored-by: Kevin Mueller on Patreon
2021-09-22 14:41:39 +00:00
* Bug fix: Git configs such as annex.verify were incorrectly overriding
per-remote git configs such as remote.name.annex-verify.
(Reversion in version 4.20130323)
ImportableContentsChunkable This improves the borg special remote memory usage, by letting it only load one archive's worth of filenames into memory at a time, and building up a larger tree out of the chunks. When a borg repository has many archives, git-annex could easily OOM before. Now, it will use only memory proportional to the number of annexed keys in an archive. Minor implementation wart: Each new chunk re-opens the content identifier database, and also a new vector clock is used for each chunk. This is a minor innefficiency only; the use of continuations makes it hard to avoid, although putting the database handle into a Reader monad would be one way to fix it. It may later be possible to extend the ImportableContentsChunkable interface to remotes that are not third-party populated. However, that would perhaps need an interface that does not use continuations. The ImportableContentsChunkable interface currently does not allow populating the top of the tree with anything other than subtrees. It would be easy to extend it to allow putting files in that tree, but borg doesn't need that so I left it out for now. Sponsored-by: Noam Kremen on Patreon
2021-10-06 21:05:32 +00:00
* borg: Significantly improved memory use when a borg repository
contains many archives.
borg: Avoid trying to extract xattrs, ACLS, and bsdflags when retrieving from a borg repository That broke restoring on linux from a borg backup made on OSX. Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-09-03 16:10:14 +00:00
* borg: Avoid trying to extract xattrs, ACLS, and bsdflags when
retrieving from a borg repository.
sped up git-annex smudge --clean by 25% Disabling git-annex branch update for this command is ok, because it does not use any information from the branch, but only logs the location when it adds a key. Sponsored-by: Dartmouth College's Datalad project
2021-09-24 18:15:20 +00:00
* Sped up git-annex smudge --clean by 25%.
resume properly when copying a file to/from a local git remote is interrupted (take 2) This method avoids breaking test_readonly. Just check if the dest file exists, and avoid CoW probing when it does, so when CoW probing fails, it can resume where the previous non-CoW copy left off. If CoW has been probed already to work, delete the dest file since a CoW copy will presumably work. It seems like it would be almost as good to just skip CoW copying in this case too, but consider that the dest file might have started to be copied from some other remote, not using CoW, but CoW has been probed to work to copy from the current place. Sponsored-by: Dartmouth College's Datalad project
2021-09-27 20:03:01 +00:00
* Resume where it left off when copying a file to/from a local git remote
was interrupted.
avoid uncessary call to inAnnex sync --content: Avoid a redundant checksum of a file that was incrementally verified, when used on NTFS and perhaps other filesystems. When sync has just gotten the content, it does not need to check inAnnex a second time. On NTFS, for some reason the write of the inode cache after it gets the content is not immediately able to be read, and with an empty/non-matching inode cache due to that stale data, inAnnex falls back to hashing the whole object to determine if it's present. Sponsored-by: Brock Spratlen on Patreon
2021-10-01 16:02:35 +00:00
* sync --content: Avoid a redundant checksum of a file that was
incrementally verified, when used on NTFS and perhaps other filesystems.
reinject: Fix crash when reinjecting a file from outside the repository Commit 4bf7940d6b912fbf692b268f621ebd41ed871125 introduced this problem, but was otherwise doing a good thing. Problem being that fileRef "/foo" used to return ":./foo", which was actually wrong, but as long as there was no foo in the local repository, catKey could operate on it without crashing. After that fix though, fileRef would return eg "../../foo", resulting in fileRef returning ":./../../foo", which will make git cat-file crash since that's not a valid path in the repo. Fix is simply to make fileRef detect paths outside the repo and return Nothing. Then catKey can be skipped. This needed several bugfixes to dirContains as well, in previous commits. In Command.Smudge, this led to needing to check for Nothing. That case should actually never happen, because the fileoutsiderepo check will detect it earlier. Sponsored-by: Brock Spratlen on Patreon
2021-10-01 18:04:18 +00:00
* reinject: Fix crash when reinjecting a file from outside the repository.
(Reversion in version 8.20210621)
avoid cursor jitter when updating progress display When the progress display gets longer, and then shorter again, it causes the cursor to jitter back and forth. Somehow I never noticed this until this morning, but then it became intolerable to watch. To fix it, pad the progress display to the maximum length it's occupied. Sponsored-by: Svenne Krap on Patreon
2021-10-07 14:58:49 +00:00
* Avoid cursor jitter when updating progress display.
borg: Avoid trying to extract xattrs, ACLS, and bsdflags when retrieving from a borg repository That broke restoring on linux from a borg backup made on OSX. Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-09-03 16:10:14 +00:00
releasing package git-annex version 8.20211011
2021-10-11 16:53:17 +00:00
-- Joey Hess <id@joeyh.name> Mon, 11 Oct 2021 12:52:14 -0400
borg: Avoid trying to extract xattrs, ACLS, and bsdflags when retrieving from a borg repository That broke restoring on linux from a borg backup made on OSX. Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-09-03 16:10:14 +00:00
releasing package git-annex version 8.20210903
2021-09-03 16:01:12 +00:00
git-annex (8.20210903) upstream; urgency=medium
add: When adding a dotfile, avoid treating its name as an extension.
2021-08-03 16:22:58 +00:00
deal better with clock skew situations, using vector clocks * Deal with clock skew, both forwards and backwards, when logging information to the git-annex branch. * GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1) rather than needing to be advanced each time a new change is made. * Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex. When changing a file in the git-annex branch, the vector clock to use is now determined by first looking at the current time (or GIT_ANNEX_VECTOR_CLOCK when set), and comparing it to the newest vector clock already in use in that file. If a newer time stamp was already in use, advance it forward by a second instead. When the clock is set to a time in the past, this avoids logging with an old timestamp, which would risk that log line later being ignored in favor of "newer" line that is really not newer. When a log entry has been made with a clock that was set far ahead in the future, this avoids newer information being logged with an older timestamp and so being ignored in favor of that future-timestamped information. Once all clocks get fixed, this will result in the vector clocks being incremented, until finally enough time has passed that time gets back ahead of the vector clock value, and then it will return to usual operation. (This latter situation is not ideal, but it seems the best that can be done. The issue with it is, since all writers will be incrementing the last vector clock they saw, there's no way to tell when one writer made a write significantly later in time than another, so the earlier write might arbitrarily be picked when merging. This problem is why git-annex uses timestamps in the first place, rather than pure vector clocks.) Advancing forward by 1 second is somewhat arbitrary. setDead advances a timestamp by just 1 picosecond, and the vector clock could too. But then it would interfere with setDead, which wants to be overrulled by any change. So it could use 2 picoseconds or something, but that seems weird. It could just as well advance it forward by a minute or whatever, but then it would be harder for real time to catch up with the vector clock when forward clock slew had happened. A complication is that many log files contain several different peices of information, and it may be best to only use vector clocks for the same peice of information. For example, a key's location log file contains InfoPresent/InfoMissing for each UUID, and it only looks at the vector clocks for the UUID that is being changed, and not other UUIDs. Although exactly where the dividing line is can be hard to determine. Consider metadata logs, where a field "tag" can have multiple values set at different times. Should it advance forward past the last tag? Probably. What about when a different field is set, should it look at the clocks of other fields? Perhaps not, but currently it does, and this does not seems like it will cause any problems. Another one I'm not entirely sure about is the export log, which is keyed by (fromuuid, touuid). So if multiple repos are exporting to the same remote, different vector clocks can be used for that remote. It looks like that's probably ok, because it does not try to determine what order things occurred when there was an export conflict. Sponsored-by: Jochen Bartl on Patreon
2021-08-03 20:45:20 +00:00
* Deal with clock skew, both forwards and backwards, when logging
information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.
add: When adding a dotfile, avoid treating its name as an extension.
2021-08-03 16:22:58 +00:00
* add: When adding a dotfile, avoid treating its name as an extension.
rsync special remote: Stop displaying rsync progress, and use git-annex's own progress display Reasons are same as in commit cee14f147ac229c99f001b3a0aaaefb4cb6f7785. (It was already done when using -J.) Sponsored-by: Mark Reidenbach on Patreon
2021-08-09 16:06:10 +00:00
* rsync special remote: Stop displaying rsync progress, and use
git-annex's own progress display.
incremental verify for webdav special remote Sponsored-by: Dartmouth College's DANDI project
2021-08-16 20:51:58 +00:00
* Many special remotes now checksum content while it is being retrieved,
incremental verification for gitlfs and httpalso And that should be all the special remotes supporting it on linux now, except for in the odd edge case here and there. Sponsored-by: Dartmouth College's DANDI project
2021-08-18 19:13:14 +00:00
instead of in a separate pass at the end. This is supported for all
special remotes on Linux (except for bittorrent), and for many
on other OS's (except for adb, external, gcrypt, hook, and rsync).
unused: Skip the refs/annex/last-index ref that git-annex recently started creating This was unlikely to cause any problem, but it is unsightly to mention normally hidden refs, and it might have done a bit of unnecessary work to check that ref. Sponsored-by: Noam Kremen on Patreon
2021-08-24 16:58:14 +00:00
* unused: Skip the refs/annex/last-index ref that git-annex recently
started creating.
Fix test suite failure on Windows It would be better if the Arbitrary instance avoided generating impossible filenames like "foo/c:bar", but proably this is the only place that splits the file from the directory and then uses the file without the directory.. At least on the quickcheck properties. Sponsored-by: Svenne Krap on Patreon
2021-08-24 18:03:29 +00:00
* Fix test suite failure on Windows.
--batch-keys New --batch-keys option added to these commands: get, drop, move, copy, whereis git-annex-matching-options had to be reworded since some of its options can be used to match on keys, not only files. Sponsored-by: Luke Shumaker on Patreon
2021-08-25 18:20:33 +00:00
* New --batch-keys option added to these commands:
get, drop, move, copy, whereis
Added annex.youtube-dl-command config This can be used to run some forks of youtube-dl. Sponsored-by: Brett Eisenberg on Patreon
2021-08-27 13:44:23 +00:00
* Added annex.youtube-dl-command config. This can be used to run some
forks of youtube-dl.
Run cp -a with --no-preserve=xattr, to avoid problems with copied xattrs Including them breaking permissions setting on some NFS servers. Sponsored-by: Dartmouth College's Datalad project
2021-08-27 17:09:34 +00:00
* Run cp -a with --no-preserve=xattr, to avoid problems with copied
xattrs, including them breaking permissions setting on some NFS
servers.
defer write permissions checking in import until after copy to repo This should complete the fix started in 6329997ac44691937f1d7fe6a71da3184237b13b, fixing the actual cause of the test suite failure this time. Sponsored-by: Dartmouth College's Datalad project
2021-09-02 17:45:21 +00:00
* add, import: Detect when xattrs or perhaps ACLs prevent removing
write permissions from an annexed file, and fail with an informative
message.
Fix support for readonly git remotes Boolean blindness oops. (Reversion in version 8.20210621) Sponsored-by: Dartmouth College's Datalad project
2021-08-30 16:34:19 +00:00
* Fix support for readonly git remotes.
(Reversion in version 8.20210621)
improve url download failure display * When downloading urls fail, explain which urls failed for which reasons. * web: Avoid displaying a warning when downloading one url failed but another url later succeeded. Some other uses of downloadUrl use urls that are effectively internal use, and should not all be displayed to the user on failure. Eg, Remote.Git tries different urls where content could be located depending on how the remote repo is set up. Exposing those urls to the user would lead to wild goose chases. So had to parameterize it to control whether it displays urls or not. A side effect of this change is that when there are some youtube urls and some regular urls, it will try regular urls first, even if the youtube urls are listed first. This seems like an improvement if anything, but in any case there's no defined order of urls that it's supposed to use. Sponsored-by: Dartmouth College's Datalad project
2021-09-01 19:28:22 +00:00
* When downloading urls fail, explain which urls failed for which
reasons.
* web: Avoid displaying a warning when downloading one url failed
but another url later succeeded.
add: When adding a dotfile, avoid treating its name as an extension.
2021-08-03 16:22:58 +00:00
releasing package git-annex version 8.20210903
2021-09-03 16:01:12 +00:00
-- Joey Hess <id@joeyh.name> Fri, 03 Sep 2021 12:00:46 -0400
add: When adding a dotfile, avoid treating its name as an extension.
2021-08-03 16:22:58 +00:00
releasing package git-annex version 8.20210803
2021-08-03 16:20:45 +00:00
git-annex (8.20210803) upstream; urgency=medium
whereused implemented except --historical Sponsored-by: Jack Hill on Patreon
2021-07-14 18:25:52 +00:00
* whereused: New command, finds what files use a key, or where a key
was used historically.
fall back to checking lower case hash directories in normal repo Fix a bug that prevented getting content from a repository that started out as a bare repository, or had annex.crippledfilesystem set, and was converted to a non-bare repository. This unfortunately means that inAnnex check gets slowed down by a stat call in normal repos when the content is not present. Oh well, such is the cost of backwards compatability with old mistakes. Sponsored-by: Mark Reidenbach on Patreon
2021-07-15 16:16:31 +00:00
* Fix a bug that prevented getting content from a repository that
started out as a bare repository, or had annex.crippledfilesystem
set, and was converted to a non-bare repository.
Fix retrieval of content from borg repos accessed over ssh It was making the borgrepo path absolute.. even when it was a ssh repository. Made BorgRepo a newtype, to guard against accidentially treating it like a FilePath. Sponsored-by: Graham Spencer on Patreon
2021-07-15 16:38:55 +00:00
* Fix retrieval of content from borg repos accessed over ssh.
sync --quiet * sync: When --quiet is used, run git commit, push, and pull without their ususual output. * merge: When --quiet is used, run git merge without its usual output. This might also make --quiet work better for some other commands that make commits, like git-annex adjust. Sponsored-by: Kevin Mueller on Patreon
2021-07-19 15:28:31 +00:00
* sync: When --quiet is used, run git commit, push, and pull without
their ususual output.
* merge: When --quiet is used, run git merge without its usual output.
remove direct mode remnant of merging unrelated histories sync, merge, post-receive: Avoid merging unrelated histories, which used to be allowed only to support direct mode repositories. (However, sync does still merge unrelated histories when importing trees from special remotes, and the assistant still merges unrelated histories always.) See 556b2ded2ba8270846fa207255b4c2def6ef5d8a for why this was added back in 2016, for direct mode. This is a behavior change, which might break something that was relying on sync merging unrelated histories, but git had a good reason to prevent it, since it's easy to foot shoot with it, and git-annex should follow suit. Sponsored-by: Noam Kremen on Patreon
2021-07-19 15:40:48 +00:00
* sync, merge, post-receive: Avoid merging unrelated histories,
which used to be allowed only to support direct mode repositories.
(However, sync does still merge unrelated histories when importing
trees from special remotes, and the assistant still merges unrelated
histories.)
sync, merge: Added --allow-unrelated-histories option Which is the same as the git merge option. After last commit, this turns out to be needed in the test suite, and when doing git-annex import from special remote, followed by a git-annex merge. Sponsored-by: Svenne Krap on Patreon
2021-07-19 16:08:24 +00:00
* sync, merge: Added --allow-unrelated-histories option, which
is the same as the git merge option.
check if object is modified before starting to send it Fix bug that caused some transfers to incorrectly fail with "content changed while it was being sent", when the content was not changed. While I don't know how to reproduce the problem that several people reported, it is presumably due to the inode cache somehow being stale. So check isUnmodified', and if it's not modified, include the file's current inode cache in the set to accept, when checking for modification after the transfer. That seems like the right thing to do for another reason: The failure says the file changed while it was being sent, but if the object file was changed before the transfer started, that's wrong. So it needs to check before allowing the transfer at all if the file is modified. (Other calls to sameInodeCache or elemInodeCaches, when operating on inode caches from the database, could also be problimatic if the inode cache is somehow getting stale. This does not address such problems.) Sponsored-by: Dartmouth College's Datalad project
2021-07-26 21:33:49 +00:00
* Fix bug that caused some transfers to incorrectly fail with
"content changed while it was being sent", when the content was not
changed.
fix problem populating pointer files This is a result of an audit of every use of getInodeCaches, to find places that misbehave when the annex object is not in the inode cache, despite pointer files for the same key being in the inode cache. Unfortunately, that is the case for objects that were in v7 repos that upgraded to v8. Added a note about this gotcha to getInodeCaches. Database.Keys.reconcileStaged, then annex.thin is set, would fail to populate pointer files in this situation. Changed it to check if the annex object is unmodified the same way inAnnex does, falling back to a checksum if the inode cache is not recorded. Sponsored-by: Dartmouth College's Datalad project
2021-07-27 18:21:09 +00:00
* Fix bug that could prevent pointer files from being populated,
in a repository that was upgraded from v7.
fsck: Detect and correct stale or missing inode caches for object files An easy way to see this in action is to have an unlocked file, and touch the object file. While all code that compares inode caches for object files needs to be prepared for this kind of problem and fall back to verification, having fsck notice it and correct it is cheap (as long as fsck is being run anyway) and ensures that if it happens for some unusual reason, there's a way for the user to notice that it's happening. Not that, when annex.thin is in use, the earlier call to isUnmodified (and also potentially earlier calls to inAnnex in eg, verifyLocationLog) will fix up the same problem silently. That might prevent the warning being displayed, although probably it still will be, because the Database.Keys write of the InodeCache will be queued but will not have happened yet. I can't see a way to improve this, but it's not great. Sponsored-by: Dartmouth College's Datalad project
2021-07-29 18:06:13 +00:00
* fsck: Detect and correct stale or missing inode caches.
Fix a rounding bug in display of data sizes Eg, showImprecise 1 1.99 returned "1.1" rather than "2". The 9 rounded upward to 10, and that was wrongly used as the decimal, rather than carrying the 1. Sponsored-by: Jack Hill on Patreon
2021-07-30 13:56:04 +00:00
* Fix a rounding bug in display of data sizes.
work around strange auto-init bug git-annex get when run as the first git-annex command in a new repo did not populate unlocked files. (Reversion in version 8.20210621) I am not entirely happy with this, because I don't understand how 428c91606b434512d1986622e751c795edf4df44 caused the problem in the first place, and I don't fully understand how skipping calling scanAnnexedFiles during autoinit avoids the problem. Kept the explicit call to scanAnnexedFiles during git-annex init, so that when reconcileStaged is expensive, it can be made to run then, rather than at some later point when the information is needed. Sponsored-by: Brock Spratlen on Patreon
2021-07-30 22:36:03 +00:00
* git-annex get when run as the first git-annex command in a new repo
did not populate unlocked files.
(Reversion in version 8.20210621)
whereused implemented except --historical Sponsored-by: Jack Hill on Patreon
2021-07-14 18:25:52 +00:00
releasing package git-annex version 8.20210803
2021-08-03 16:20:45 +00:00
-- Joey Hess <id@joeyh.name> Tue, 03 Aug 2021 12:20:09 -0400
whereused implemented except --historical Sponsored-by: Jack Hill on Patreon
2021-07-14 18:25:52 +00:00
releasing package git-annex version 8.20210714
2021-07-14 16:23:24 +00:00
git-annex (8.20210714) upstream; urgency=medium
assistant: Avoid unncessary git repository repair In a situation where git fsck gets confused about a commit that is made while it's running. Sponsored-by: Graham Spencer on Patreon
2021-06-30 21:57:49 +00:00
* assistant: Avoid unncessary git repository repair in a situation where
git fsck gets confused about a commit that is made while it's running.
addurl: Avoid crashing when used on beegfs. Sponsored-by: Dartmouth College's DANDI project
2021-07-05 17:02:40 +00:00
* addurl: Avoid crashing when used on beegfs.
--debug output goes to stderr again, not stdout Reversion in version 8.20210428 Sponsored-by: Dartmouth College's Datalad project
2021-07-12 13:40:38 +00:00
* --debug output goes to stderr again, not stdout.
(Reversion in version 8.20210428)
fix init reversion when core.sharedRepository = group init: Fix misbehavior when core.sharedRepository = group that caused it to enter an adjusted branch. (Reversion in version 8.20210630) Commit 4b1b9d7a83f0da71665658e6234e699e182cbddc made init call freezeContent in case there was a hook that could prevent writing in situations where perms don't. But with the above git config, freezeContent does not prevent write at all. So init needs to do what freezeContent does with a non-shared git config. Or init could check for that config, and skip the probing, since it won't actually be preventing write to any files. But that would make init too aware if details of Annex.Perms, and also would break if the git config were changed after init. Sponsored-by: Dartmouth College's Datalad project
2021-07-12 14:15:49 +00:00
* init: Fix misbehavior when core.sharedRepository = group that
close bug and mention another aspect of the reversion in changelog
2021-07-12 14:45:57 +00:00
caused it to enter an adjusted branch and set annex.crippledfilesystem
fix init reversion when core.sharedRepository = group init: Fix misbehavior when core.sharedRepository = group that caused it to enter an adjusted branch. (Reversion in version 8.20210630) Commit 4b1b9d7a83f0da71665658e6234e699e182cbddc made init call freezeContent in case there was a hook that could prevent writing in situations where perms don't. But with the above git config, freezeContent does not prevent write at all. So init needs to do what freezeContent does with a non-shared git config. Or init could check for that config, and skip the probing, since it won't actually be preventing write to any files. But that would make init too aware if details of Annex.Perms, and also would break if the git config were changed after init. Sponsored-by: Dartmouth College's Datalad project
2021-07-12 14:15:49 +00:00
(Reversion in version 8.20210630)
addistant: honor annex.delayadd for non-large files assistant: When adding non-large files to git, honor annex.delayadd configuration. Also, don't add non-large files to git when they are still being written to. This came for free, since the changes to non-large files get queued up with the ones to large files, and run through the lsof check. Sponsored-by: Luke Shumaker on Patreon
2021-07-13 16:15:40 +00:00
* assistant: When adding non-large files to git, honor annex.delayadd
configuration. Also, don't add non-large files to git when they
are still being written to.
assistant: Avoid unncessary git repository repair In a situation where git fsck gets confused about a commit that is made while it's running. Sponsored-by: Graham Spencer on Patreon
2021-06-30 21:57:49 +00:00
releasing package git-annex version 8.20210714
2021-07-14 16:23:24 +00:00
-- Joey Hess <id@joeyh.name> Wed, 14 Jul 2021 12:22:55 -0400
assistant: Avoid unncessary git repository repair In a situation where git fsck gets confused about a commit that is made while it's running. Sponsored-by: Graham Spencer on Patreon
2021-06-30 21:57:49 +00:00
releasing package git-annex version 8.20210630
2021-06-30 15:48:33 +00:00
git-annex (8.20210630) upstream; urgency=medium
push synced/master before synced/git-annex sync: Partly work around github behavior that first branch to be pushed to a new repository is assumed to be the head branch, by not pushing synced/git-annex first. github expects master (or whatever the name is) to be pushed first, but git-annex sync can't, because it's got to also support pushes to non-bare repos where pushing master fails, as explained in the big comment. So pushing synced/master is not entirely a fix, but at least it makes github default to a branch with the stuff the user expects in it, not a bunch of annex log files. Aside from fixing github to not make this assumption, or improving the git push protocol to include what the current HEAD is, the only other approach I can think of is to identify git push's progress messages and display those when pushing master, while filtering out error messages about non-fast-forward etc. But git doesn't provide a way to separate out or identify its progress messages. Sponsored-by: Luke Shumaker on Patreon
2021-06-21 16:32:21 +00:00
releasing package git-annex version 8.20210630
2021-06-30 15:48:33 +00:00
* Fixed bug that interrupting git-annex repair (or assistant) while
it was fixing repository corruption would lose objects that were
contained in pack files. Also the assistant might sometimes have
crashed with the same result.
* repair: Fix reversion in version 8.20200522 that prevented fetching
missing objects from remotes.
push synced/master before synced/git-annex sync: Partly work around github behavior that first branch to be pushed to a new repository is assumed to be the head branch, by not pushing synced/git-annex first. github expects master (or whatever the name is) to be pushed first, but git-annex sync can't, because it's got to also support pushes to non-bare repos where pushing master fails, as explained in the big comment. So pushing synced/master is not entirely a fix, but at least it makes github default to a branch with the stuff the user expects in it, not a bunch of annex log files. Aside from fixing github to not make this assumption, or improving the git push protocol to include what the current HEAD is, the only other approach I can think of is to identify git push's progress messages and display those when pushing master, while filtering out error messages about non-fast-forward etc. But git doesn't provide a way to separate out or identify its progress messages. Sponsored-by: Luke Shumaker on Patreon
2021-06-21 16:32:21 +00:00
* sync: Partly work around github behavior that first branch to be pushed
to a new repository is assumed to be the head branch, by not pushing
synced/git-annex first.
Added annex.freezecontent-command and annex.thawcontent-command configs Freeze first sets the file perms, and then runs freezecontent-command. Thaw runs thawcontent-command before restoring file permissions. This is in case the freeze command prevents changing file perms, as eg setting a file immutable does. Also, changing file perms tends to mess up previously set ACLs. git-annex init's probe for crippled filesystem uses them, so if file perms don't work, but freezecontent-command manages to prevent write to a file, it won't treat the filesystem as crippled. When the the filesystem has been probed as crippled, the hooks are not used, because there seems to be no point then; git-annex won't be relying on locking annex objects down. Also, this avoids them being run when the file perms have not been changed, in case they somehow rely on git-annex's setting of the file perms in order to work. Sponsored-by: Dartmouth College's Datalad project
2021-06-21 18:40:20 +00:00
* Added annex.freezecontent-command and annex.thawcontent-command
configs.
Improve display of errors when transfers fail Transfers from or to a local git repo could fail without a reason being given, if the content failed to verify, or if the object file's stat changed while it was being copied. Now display messages in these cases. Sponsored-by: Jack Hill on Patreon
2021-06-25 17:04:17 +00:00
* Improve display of errors when transfers fail.
dropping unused marks as dead Dropping an object with drop --unused or dropunused will mark it as dead, preventing fsck --all from complaining about it after it's been dropped from all repositories. If another repository still has a copy, it won't be treated as dead until it's also dropped from there. The drop has to use --unused, can't be --key or something else, because this indicates that the user has recently ran git-annex unused. If it checked the unused log on every drop, bad things would happen when the unused log was out of date, eg a file used to be unused but then got re-added. Marking such a file as dead could be confusing. When the user uses --unused/dropunused, they must consider the unused information to be up-to-date. The particular workflow this enables is: git annex add foo git annex unannex foo git annex unused git annex drop --unused / dropunused git annex fsck --all # no warnings The docs for git-annex unannex say to use git-annex unused and dropunused, so the user should be pointed in this direction when they want to undo an accidental add. Sponsored-by: Brock Spratlen on Patreon
2021-06-25 19:22:05 +00:00
* Dropping an unused object with drop --unused or dropunused will
mark it as dead, preventing fsck --all from complaining about it
after it's been dropped from all repositories.
addurl, importfeed: Added --no-raw option Forces eg, download with youtube-dl without falling back to raw download. Since youtube-dl failing due to an url not being supported is difficult to distinguish from it failing due to being blocked in some way, this can be useful to avoid the fallback of git-annex downloading the raw web page and adding that. Since --raw also prevents using special remotes, --no-raw also allows special remote downloads. Although it's always possible that some special remote may claim an url and fall back to raw download of the content, which --no-raw cannot prevent. Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-06-27 15:13:38 +00:00
* addurl, importfeed: Added --no-raw option that forces download
with youtube-dl or a special remote. In particular this can avoid
falling back to raw download when youtube-dl is blocked for some
reason.
push synced/master before synced/git-annex sync: Partly work around github behavior that first branch to be pushed to a new repository is assumed to be the head branch, by not pushing synced/git-annex first. github expects master (or whatever the name is) to be pushed first, but git-annex sync can't, because it's got to also support pushes to non-bare repos where pushing master fails, as explained in the big comment. So pushing synced/master is not entirely a fix, but at least it makes github default to a branch with the stuff the user expects in it, not a bunch of annex log files. Aside from fixing github to not make this assumption, or improving the git push protocol to include what the current HEAD is, the only other approach I can think of is to identify git push's progress messages and display those when pushing master, while filtering out error messages about non-fast-forward etc. But git doesn't provide a way to separate out or identify its progress messages. Sponsored-by: Luke Shumaker on Patreon
2021-06-21 16:32:21 +00:00
releasing package git-annex version 8.20210630
2021-06-30 15:48:33 +00:00
-- Joey Hess <id@joeyh.name> Wed, 30 Jun 2021 11:48:16 -0400
push synced/master before synced/git-annex sync: Partly work around github behavior that first branch to be pushed to a new repository is assumed to be the head branch, by not pushing synced/git-annex first. github expects master (or whatever the name is) to be pushed first, but git-annex sync can't, because it's got to also support pushes to non-bare repos where pushing master fails, as explained in the big comment. So pushing synced/master is not entirely a fix, but at least it makes github default to a branch with the stuff the user expects in it, not a bunch of annex log files. Aside from fixing github to not make this assumption, or improving the git push protocol to include what the current HEAD is, the only other approach I can think of is to identify git push's progress messages and display those when pushing master, while filtering out error messages about non-fast-forward etc. But git doesn't provide a way to separate out or identify its progress messages. Sponsored-by: Luke Shumaker on Patreon
2021-06-21 16:32:21 +00:00
releasing package git-annex version 8.20210621
2021-06-21 16:17:46 +00:00
git-annex (8.20210621) upstream; urgency=medium
fromkey unlocked files support fromkey: Create an unlocked file when used in an adjusted branch where the file should be unlocked, or when configured by annex.addunlocked. There is some overlap with code in Annex.Ingest, however it's not quite the same because ingesting has a temp file with the content, where here the content, if any, is in the annex object file. So it eg, makes sense for Annex.Ingest to copy the execute mode of the content file, but it does not make sense for fromkey to do that. Also changed in passing to stage the file in git directly, rather than using git add. One consequence of that is that if the file is gitignored, it will still get added, rather than the old behavior: The following paths are ignored by one of your .gitignore files: ignored hint: Use -f if you really want to add them. hint: Turn this message off by running hint: "git config advice.addIgnoredFile false" git-annex: user error (xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--"] exited 123) That old behavior was a surprise to me, and so I consider it a bug, and doubt anyone would have relied on it. Note that, when on an --hide-missing branch, it is possible to fromkey a key that is not present (needs --force). The annex link or pointer file still gets written in this case. It doesn't seem to make any sense not to write it, because then fromkey would not do anything useful in this case, and this way the file can be committed and synced to master, and the branch re-adjusted to hide the new missing file. This commit was sponsored by Noam Kremen on Patreon.
2021-05-03 15:26:18 +00:00
New matching options --excludesamecontent and --includesamecontent The normalisation of filenames turns out to be the tricky part here, because the associated files coming out of the keys db may look like "./foo/bar" or "../bar". For the former to match a glob like "foo/*", it needs to be normalised. Note that, on windows, normalise "./foo/bar" = "foo\\bar" which a glob like "foo/*" won't match. So the glob is matched a second time, on the toInternalGitPath, so allowing the user to provide a glob with the slashes in either direction. However, this still won't support some wacky edge cases like the user providing a glob of "foo/bar\\*" Sponsored-by: Dartmouth College's Datalad project
2021-05-25 17:05:42 +00:00
* New matching options --excludesamecontent and --includesamecontent
prevent dropping required content of other file using same content When two files have the same content, and a required content expression matches one but not the other, dropping the latter file will fail as it would also remove the content of the required file. This will slow down drop (w/o --auto), dropunused, mirror, and move, by one keys db lookup per file. But I did include an optimisation to avoid a double db lookup in the drop --auto / sync --content case. I suspect that dropunused could also use PreferredContentChecked True, but haven't entirely thought it through and it's rarely used with enough files for the optimisation to matter. Sponsored-by: Dartmouth College's Datalad project
2021-05-25 14:57:06 +00:00
* When two files have the same content, and a required content expression
matches one but not the other, dropping the latter file will fail as it
would also remove the content of the required file.
drop, move, mirror: when two files have the same content, honor the max numcopies and requiredcopies Eg, before with a .gitattributes like: *.2 annex.numcopies=2 *.1 annex.numcopies=1 And foo.1 and foo.2 having the same content and key, git-annex drop foo.1 foo.2 would succeed, leaving just 1 copy, despite foo.2 needing 2 copies. It dropped foo.1 first and then skipped foo.2 since its content was gone. Now that the keys database includes locked files, this longstanding wart can be fixed. Sponsored-by: Noam Kremen on Patreon
2021-06-15 15:38:44 +00:00
* drop, move, mirror: When two files have the same content, and
verify associated files when checking numcopies Most of this is just refactoring. But, handleDropsFrom did not verify that associated files from the keys db were still accurate, and has now been fixed to. A minor improvement to this would be to avoid calling catKeyFile twice on the same file, when getting the numcopies and mincopies value, in the common case where the same file has the highest value for both. But, it avoids checking every associated file, so it will scale well to lots of dups already. Sponsored-by: Kevin Mueller on Patreon
2021-06-15 15:12:27 +00:00
different numcopies or requiredcopies values, use the higher value.
clearer language
2021-05-24 18:54:51 +00:00
* drop --auto: When two files have the same content, and a preferred content
expression matches one but not the other, do not drop the content.
improve changelog
2021-05-25 14:08:29 +00:00
* sync --content, assistant: When two unlocked files have the same
content, and a preferred content expression matches one but not the
other, do not drop the content. (This was already the case for locked
files.)
fix longstanding indeterminite preferred content for duplicated file problem * drop: When two files have the same content, and a preferred content expression matches one but not the other, do not drop the file. * sync --content, assistant: Fix an edge case where a file that is not preferred content did not get dropped. The sync --content edge case is that handleDropsFrom loaded associated files and used them without verifying that the information from the database was not stale. It seemed best to avoid changing --want-drop's behavior, this way when debugging a preferred content expression with it, the files matched will still reflect the expression. So added a note to the --want-drop documentation, to make clear it may not behave identically to git-annex drop --auto. While it would be possible to introspect the preferred content expression to see if it matches on filenames, and only look up the associated files when it does, it's generally fairly rare for 2 files to have the same content, and the database lookup is already avoided when there's only 1 file, so I did not implement that further optimisation. Note that there are still some situations where the associated files database does not get locked files recorded in it, which will prevent this fix from working. Sponsored-by: Dartmouth College's Datalad project
2021-05-24 18:02:50 +00:00
* sync --content, assistant: Fix an edge case where a file that is not
preferred content did not get dropped.
filter-branch: New command, useful to produce a filtered version of the git-annex branch, eg when splitting a repository
2021-05-17 18:16:46 +00:00
* filter-branch: New command, useful to produce a filtered version of the
git-annex branch, eg when splitting a repository.
fromkey unlocked files support fromkey: Create an unlocked file when used in an adjusted branch where the file should be unlocked, or when configured by annex.addunlocked. There is some overlap with code in Annex.Ingest, however it's not quite the same because ingesting has a temp file with the content, where here the content, if any, is in the annex object file. So it eg, makes sense for Annex.Ingest to copy the execute mode of the content file, but it does not make sense for fromkey to do that. Also changed in passing to stage the file in git directly, rather than using git add. One consequence of that is that if the file is gitignored, it will still get added, rather than the old behavior: The following paths are ignored by one of your .gitignore files: ignored hint: Use -f if you really want to add them. hint: Turn this message off by running hint: "git config advice.addIgnoredFile false" git-annex: user error (xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--"] exited 123) That old behavior was a surprise to me, and so I consider it a bug, and doubt anyone would have relied on it. Note that, when on an --hide-missing branch, it is possible to fromkey a key that is not present (needs --force). The annex link or pointer file still gets written in this case. It doesn't seem to make any sense not to write it, because then fromkey would not do anything useful in this case, and this way the file can be committed and synced to master, and the branch re-adjusted to hide the new missing file. This commit was sponsored by Noam Kremen on Patreon.
2021-05-03 15:26:18 +00:00
* fromkey: Create an unlocked file when used in an adjusted branch
where the file should be unlocked, or when configured by annex.addunlocked.
fileRef: make paths relative and simplified Fix behavior of several commands, including reinject, addurl, and rmurl when given an absolute path to an unlocked file, or a relative path that leaves and re-enters the repository. To avoid slowing down all the cases where the paths are already ok with an unncessary call to getCurrentDirectory, put in an optimisation in relPathCwdToFile. That will probably also speed up other parts of git-annex by some small amount, but I have not benchmarked. Note that I did not convert branchFileRef, because it seems likely that it will be used with a file that is not provided by the user, so is already in a sane format. This is certainly true for the way git-annex uses it, though maybe arguable to the extent Git.Ref is a reusable library.
2021-05-07 17:25:59 +00:00
* Fix behavior of several commands, including reinject, addurl, and rmurl
when given an absolute path to an unlocked file, or a relative path
that leaves and re-enters the repository.
smudge: check for known annexed inodes before checking annex.largefiles smudge: Fix a case where an unlocked annexed file that annex.largefiles does not match could get its unchanged content checked into git, due to git running the smudge filter unecessarily. When the file has the same inodecache as an already annexed file, we can assume that the user is not intending to change how it's stored in git. Note that checkunchangedgitfile already handled the inverse case, where the file was added to git previously. That goes further and actually sha1 hashes the new file and checks if it's the same hash in the index. It would be possible to generate a key for the file and see if it's the same as the old key, however that could be considerably more expensive than sha1 of a small file is, and it is not necessary for the case I have, at least, where the file is not modified or touched, and so its inode will match the cache. git-annex add was changed, when adding a small file, to remove the inode cache for it. This is necessary to keep the recipe in doc/tips/largefiles.mdwn for converting from annex to git working. It also avoids bugs/case_where_using_pathspec_with_git-commit_leaves_s.mdwn which the earlier try at this change introduced.
2021-05-10 17:05:08 +00:00
* smudge: Fix a case where an unlocked annexed file that annex.largefiles
does not match could get its unchanged content checked into git,
due to git running the smudge filter unecessarily.
reinject: Error out when run on a file that is not annexed rather than silently skipping it
2021-05-07 17:31:03 +00:00
* reinject: Error out when run on a file that is not annexed, rather
than silently skipping it.
assistant: Fix a crash on startup by avoiding using forkProcess ghc 8.8.4 seems to have changed something that broke code that has been successfully using forkProcess since 2012. Likely a change to GC internals. Since forkProcess has never had clear documentation about how to use it safely, avoid using it at all. Instead, when git-annex needs to daemonize itself, re-run the git-annex command, in a new process group and session. This commit was sponsored by Luke Shumaker on Patreon.
2021-05-12 19:08:03 +00:00
* assistant: Fix a crash on startup by avoiding using forkProcess.
add createMessage init: When annex.commitmessage is set, use that message for the commit that creates the git-annex branch. This will be used by filter-branch too, and it seems to make sense to let annex.commitmessage affect it.
2021-05-17 17:07:47 +00:00
* init: When annex.commitmessage is set, use that message for the commit
that creates the git-annex branch.
Added annex.adviceNoSshCaching config. Sponsored-by: Brock Spratlen on Patreon
2021-05-27 16:37:39 +00:00
* Added annex.adviceNoSshCaching config.
add --size-limit option When this option is not used, there should be effectively no added overhead, thanks to the optimisation in b3cd0cc6ba4e5b9e2ae0abd9c8b2ec32475e09d2. When an action fails on a file, the size of the file still counts toward the size limit. This was necessary to support concurrency, but also generally seems like the right choice. Most commands that operate on annexed files support the option. export and import do not, and I don't know if it would make sense for export to.. Why would you want an incomplete export? sync doesn't, and while it would be easy to make it support it for transferring files, it's not clear if dropping files should also take the size limit into account. Commands like add that don't operate on annexed files don't support the option either. Exiting 101 not yet implemented. Sponsored-by: Denis Dzyubenko on Patreon
2021-06-04 20:08:42 +00:00
* Added --size-limit option.
Future proof activity log parsing When the log has an activity that is not known, eg added by a future version of git-annex, it used to be treated as no activity at all, which would make git-annex expire think it should expire the repository, despite it having some kind of recent activity. Hopefully there will be no reason to add a new activity until enough time has passed that this commit is in use everywhere. Sponsored-by: Jake Vosloo on Patreon
2021-06-14 18:18:06 +00:00
* Future proof activity log parsing.
fix exponential blowup when adding lots of identical files This was an old problem when the files were being added unlocked, so the changelog mentions that being fixed. However, recently it's also affected locked files. The fix for locked files is kind of stupidly simple. moveAnnex already handles populating unlocked files, and only does it when the object file was not already present. So remove the redundant populateUnlockedFiles call. (That call was added all the way back in cfaac52b88e157dd4e71626fe68af37015b9c9bd, and has always been unncessary.) Sponsored-by: Dartmouth College's Datalad project
2021-06-15 13:32:12 +00:00
* Fix an exponential slowdown when large numbers of duplicate files are
being added in unlocked form.
fromkey unlocked files support fromkey: Create an unlocked file when used in an adjusted branch where the file should be unlocked, or when configured by annex.addunlocked. There is some overlap with code in Annex.Ingest, however it's not quite the same because ingesting has a temp file with the content, where here the content, if any, is in the annex object file. So it eg, makes sense for Annex.Ingest to copy the execute mode of the content file, but it does not make sense for fromkey to do that. Also changed in passing to stage the file in git directly, rather than using git add. One consequence of that is that if the file is gitignored, it will still get added, rather than the old behavior: The following paths are ignored by one of your .gitignore files: ignored hint: Use -f if you really want to add them. hint: Turn this message off by running hint: "git config advice.addIgnoredFile false" git-annex: user error (xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--"] exited 123) That old behavior was a surprise to me, and so I consider it a bug, and doubt anyone would have relied on it. Note that, when on an --hide-missing branch, it is possible to fromkey a key that is not present (needs --force). The annex link or pointer file still gets written in this case. It doesn't seem to make any sense not to write it, because then fromkey would not do anything useful in this case, and this way the file can be committed and synced to master, and the branch re-adjusted to hide the new missing file. This commit was sponsored by Noam Kremen on Patreon.
2021-05-03 15:26:18 +00:00
releasing package git-annex version 8.20210621
2021-06-21 16:17:46 +00:00
-- Joey Hess <id@joeyh.name> Mon, 21 Jun 2021 12:17:24 -0400
fromkey unlocked files support fromkey: Create an unlocked file when used in an adjusted branch where the file should be unlocked, or when configured by annex.addunlocked. There is some overlap with code in Annex.Ingest, however it's not quite the same because ingesting has a temp file with the content, where here the content, if any, is in the annex object file. So it eg, makes sense for Annex.Ingest to copy the execute mode of the content file, but it does not make sense for fromkey to do that. Also changed in passing to stage the file in git directly, rather than using git add. One consequence of that is that if the file is gitignored, it will still get added, rather than the old behavior: The following paths are ignored by one of your .gitignore files: ignored hint: Use -f if you really want to add them. hint: Turn this message off by running hint: "git config advice.addIgnoredFile false" git-annex: user error (xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--"] exited 123) That old behavior was a surprise to me, and so I consider it a bug, and doubt anyone would have relied on it. Note that, when on an --hide-missing branch, it is possible to fromkey a key that is not present (needs --force). The annex link or pointer file still gets written in this case. It doesn't seem to make any sense not to write it, because then fromkey would not do anything useful in this case, and this way the file can be committed and synced to master, and the branch re-adjusted to hide the new missing file. This commit was sponsored by Noam Kremen on Patreon.
2021-05-03 15:26:18 +00:00
releasing package git-annex version 8.20210428
2021-04-28 16:16:45 +00:00
git-annex (8.20210428) upstream; urgency=medium
Fix build with persistent-2.12.0.1 persistent stopped using askLogFunc, and the thing to use is askLoggerIO from monad-logger. Bumped the dep to the first version that contained that. Note that the i386ancient build uses a newer monad-logger than 0.3.10, so the new versioned dep should not break it, and presumably nothing else either. This commit was sponsored by Noam Kremen on Patreon.
2021-04-01 16:19:47 +00:00
initremote --private
2021-04-23 18:47:46 +00:00
* New annex.private and remote.name.annex-private configs that can
prevent storing information about a repository and remotes in the
git-annex branch.
* initremote: Added --private option to set up a private special remote.
update
2021-04-26 17:12:51 +00:00
* importfeed: Made "checking known urls" phase run 12 times faster.
added --debugfilter (and annex.debugfilter)
2021-04-05 19:21:20 +00:00
* Added --debugfilter (and annex.debugfilter)
diffdriver: Support unlocked files
2021-04-08 18:32:09 +00:00
* diffdriver: Support unlocked files.
forget: Preserve currently exported trees Avoiding problems with exporttree remotes in some unusual circumstances. This commit was sponsored by Brett Eisenberg on Patreon.
2021-04-13 19:00:23 +00:00
* forget: Preserve currently exported trees, avoiding problems with
exporttree remotes in some unusual circumstances.
fsck: avoid redundant checksum when transfer is Verified When downloading content from a remote, if the content is able to be verified during the transfer, skip checksumming it a second time. Note that in this case, the fsck output does not include "(checksum)" which it does when the checksumming is done separately from the download. This commit was sponsored by Brock Spratlen on Patreon.
2021-04-14 17:22:54 +00:00
* fsck: When downloading content from a remote, if the content is able
to be verified during the transfer, skip checksumming it a second time.
directory CoW on store Not for exports to directory yet though.
2021-04-14 19:11:00 +00:00
* directory: When cp supports reflinks, use it.
update
2021-04-26 17:12:51 +00:00
* Avoid excess commits to the git-annex branch when stall detection is
enabled.
git-annex-config: Allow setting annex.securehashesonly Which has otherwise been supported since 2019, but was missing from the list of allowed repo-global configs. Reordered the list to match the order in the git-annex-config man page, to make them easy to cross-compare. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2021-04-26 17:50:37 +00:00
* git-annex-config: Allow setting annex.securehashesonly, which has
otherwise been supported since 2019, but was missing from the list of
allowed repo-global configs.
fix hardcoded origin name in checkAdjustedClone init: Fix a crash when the repo's was cloned from a repo that had an adjusted branch checked out, and the origin remote is not named "origin". The only other hardcoding of the name of origin is in: - Upgrade.V2, which can be ignored probably - Annex.Branch, which doesn't fail if it has some other name, but just doesn't set up the git-annex branch with quite as linear a history in that case.
2021-04-14 22:53:27 +00:00
* init: Fix a crash when the repo's was cloned from a repo that had an
adjusted branch checked out, and the origin remote is not named "origin".
update
2021-04-26 17:12:51 +00:00
* Fix some bugs that made git-annex not see recently recorded status
information when configured with annex.alwayscommit=false.
check mincopies is satisfied even when numcopies is known to be satisfied I had been assuming that numcopies would be a larger or at most equal to mincopies, so no need to check both. But users get confused and use configs that don't really make sense, so make sure to handle mincopies being larger than numcopies. Also add something to the mincopies man page to discourage this misconfiguration. This commit was sponsored by Denis Dzyubenko on Patreon.
2021-04-27 17:37:03 +00:00
* When mincopies is set to a larger value than numcopies, make sure that
mincopies is satisfied. Before, it assumed a sane configuration would
have numcopies larger or equal to mincopies. It's still a good idea
not to configure git-annex this way.
Avoid more than 1 gpg password prompt at the same time Which could happen occasionally before when concurrency is enabled. While not much of a problem when it did happen, better to avoid it. Also, since it seems likely the gpg-agent sometimes fails in such a situation, this makes it not happen when running a single git-annex command with concurrency enabled. This commit was sponsored by Jake Vosloo on Patreon.
2021-04-27 20:36:33 +00:00
* Avoid more than 1 gpg password prompt at the same time, which
could happen occasionally before when concurrency is enabled.
update
2021-04-26 17:12:51 +00:00
* Fix build with persistent-2.12.0.1
Fix build with persistent-2.12.0.1 persistent stopped using askLogFunc, and the thing to use is askLoggerIO from monad-logger. Bumped the dep to the first version that contained that. Note that the i386ancient build uses a newer monad-logger than 0.3.10, so the new versioned dep should not break it, and presumably nothing else either. This commit was sponsored by Noam Kremen on Patreon.
2021-04-01 16:19:47 +00:00
releasing package git-annex version 8.20210428
2021-04-28 16:16:45 +00:00
-- Joey Hess <id@joeyh.name> Wed, 28 Apr 2021 12:13:24 -0400
Fix build with persistent-2.12.0.1 persistent stopped using askLogFunc, and the thing to use is askLoggerIO from monad-logger. Bumped the dep to the first version that contained that. Note that the i386ancient build uses a newer monad-logger than 0.3.10, so the new versioned dep should not break it, and presumably nothing else either. This commit was sponsored by Noam Kremen on Patreon.
2021-04-01 16:19:47 +00:00
releasing package git-annex version 8.20210330
2021-03-30 18:33:28 +00:00
git-annex (8.20210330) upstream; urgency=medium
export: When a submodule is in the tree to be exported, skip it.
2021-03-12 16:19:31 +00:00
call out behavior change
2021-03-22 15:34:23 +00:00
* Behavior change: When autoenabling special remotes of type S3, weddav,
or glacier, do not take login credentials from environment variables,
as the user may not be expecting the autoenable to happen, or may
have those set for other purposes and not intend git-annex to use them.
releasing package git-annex version 8.20210330
2021-03-30 18:33:28 +00:00
* New annex.supportunlocked config that can be set to false to avoid
some expensive things needed to support unlocked files, if you do not
use them.
* Fix bug importing from a special remote into a subdirectory more than
one level deep, which generated unusual git trees that could confuse
git merge.
* borg: Fix a bug that prevented importing keys of type URL and WORM.
* borg: Support importing files that are hard linked in the borg backup.
* export: When a submodule is in the tree to be exported, skip it.
* import: When the previously exported tree contained a submodule,
preserve it in the imported tree so it does not get deleted.
* export --json: Fill in the file field.
rmurl: remove all forms of an url, no matter what the downloader is set to * rmurl: When youtube-dl was used for an url, it no longer needs to be prefixed with "yt:" in order to be removed. * rmurl: If an url is both used by the web and also claimed by another special remote, fix a bug that caused the url to to not be removed. The youtube-dl change is a consequence of how the bug fix is implemented. But I also think it's the right thing to do. Consider that, before, git-annex addurl $url followed by git-annex rmurl $url would not remove the url in the case where youtube-dl was used. That was surprising behavior. In the unlikely case where a special remote claims an url, and it's been added using OtherDownloader, but it was also added already as a web url, it seems better for rmurl to remove both than to arbitrarily remove only one. And in the case the bug report was filed for, when an url was added as a web url, but a special remote now claims it, that should not prevent rmurl removing the web url. Calling setUrlMissing lets other callers of it behave differently. Probably the calls to it in eg, Remote.External and Remote.BitTorrent are fine, since they don't mangle the url and just remove what was provided, and the OtherDownloader form of a bittorrent url, respectively. I suspect unregisterurl needs to have a similar change made to rmurl, for similar reasons.
2021-03-22 16:09:15 +00:00
* rmurl: When youtube-dl was used for an url, it no longer needs to be
prefixed with "yt:" in order to be removed.
* rmurl: If an url is both used by the web and also claimed by another
special remote, fix a bug that caused the url to to not be removed.
unregisterurl: remove all forms of an url, no matter what the downloader is set to unregisterurl: Fix a bug that caused an url to not be unregistered when it is claimed by a special remote other than the web. See commit f175d4cc908a3c4042ded4072906bf21a2468b89 for rationalle.
2021-03-22 16:17:17 +00:00
* unregisterurl: Fix a bug that caused an url to not be unregistered
when it is claimed by a special remote other than the web.
whereis: Don't include yt: prefix when showing url to content retrieved with youtube-dl I don't think this was really intentional behavior. It may be that it was useful to include it so it could be passed to rmurl, since without it rmurl would not actually remove the url. Since that was changed earlier today, now seems like a good time to clean up the display of these urls. This commit was sponsored by Jochen Bartl on Patreon.
2021-03-22 23:56:02 +00:00
* whereis: Don't include yt: prefix when showing url to content
retrieved with youtube-dl.
releasing package git-annex version 8.20210330
2021-03-30 18:33:28 +00:00
* webdav: Work around some buggy webdav server behavior involving
renaming files.
Make --debug also enable debugging in child git-annex processes Especially necessary with stalldetection using child processes for transfers. This commit was sponsored by Jack Hill on Patreon.
2021-03-22 18:25:28 +00:00
* Make --debug also enable debugging in child git-annex processes.
fix fsck --from --all to not fall over trying to check required content fsck: When --from is used in combination with --all or similar options, do not verify required content, which can't be checked properly when operating on keys. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2021-03-22 19:00:53 +00:00
* fsck: When --from is used in combination with --all or similar options,
do not verify required content, which can't be checked properly when
operating on keys.
Sped up git-annex init in a clone of an existing repository Seems that hasOrigin was never finding origin's git-annex branch, so a new one got created each time. And so then it later needed to merge the two branches, which is expensive. Added --no-track to git branch to avoid it displaying a message about setting up tracking branches. Of course there's no reason to make the git-annex branch a tracking branch since git-annex auto-merges it.
2021-03-23 19:22:51 +00:00
* Sped up git-annex init in a clone of an existing repository.
Improved display of errors when accessing a git http remote fails. New error message: Remote foo not usable by git-annex; setting annex-ignore http://localhost/foo/config download failed: Configuration of annex.security.allowed-ip-addresses does not allow accessing address ::1 If git config parse fails, or the git config file is not available at the url, a better error message for that is also shown. This commit was sponsored by Mark Reidenbach on Patreon.
2021-03-24 18:19:32 +00:00
* Improved display of errors when accessing a git http remote fails.
releasing package git-annex version 8.20210330
2021-03-30 18:33:28 +00:00
* Fix build with attoparsec-0.14.
export: When a submodule is in the tree to be exported, skip it.
2021-03-12 16:19:31 +00:00
releasing package git-annex version 8.20210330
2021-03-30 18:33:28 +00:00
-- Joey Hess <id@joeyh.name> Tue, 30 Mar 2021 13:01:17 -0400
export: When a submodule is in the tree to be exported, skip it.
2021-03-12 16:19:31 +00:00
releasing package git-annex version 8.20210310
2021-03-10 17:59:00 +00:00
git-annex (8.20210310) upstream; urgency=medium
Windows: Correct the path to the html help file for 64 bit build.
2021-02-24 17:19:42 +00:00
close todo
2021-03-05 18:46:09 +00:00
* When non-annexed files in a tree are exported to a special remote,
importing from the special remote keeps the files non-annexed,
as long as their content has not changed, rather than converting
them to annexed files.
(Such a conversion will still happen when importing from a remote
an old git-annex exported such a tree to before; export the tree
with the new git-annex before importing to avoid that.)
Added support for git-remote-gcrypt's rsync URIs Which access a remote using rsync over ssh, and which git pushes to much more efficiently than ssh urls. There was some old partial support for rsync URIs from 2013, but it seemed incomplete, and did not use rsync over ssh. Weird. I'm not sure if there's any remaining benefit to using the non-rsync url forms with gcrypt, now that this is implemented? Updated docs to encourage using the rsync urls. This commit was sponsored by Svenne Krap on Patreon.
2021-03-09 19:58:09 +00:00
* Added support for git-remote-gcrypt's rsync URIs, which access a remote
using rsync over ssh, and which git pushes to much more efficiently
than ssh urls.
reorder
2021-03-10 14:15:45 +00:00
* unregisterurl: New command.
* registerurl: Allow it to be used in a bare repository.
* Prevent combinations of options such as --all with --include.
* Fixed handling of --mimetype or --mimeencoding combined with
options like --all or --unused.
* Fix handling of --branch combined with --unlocked or --locked.
Fix support for local gcrypt repositories with a space in their URI Git.Remote.parseRemoteLocation had a hack to handle URIs that contained characters like spaces, which is something git unfortunately allows despite not being a valid URI. However, that hack looked for "//" to guess something was an URI, and these gcrypt URIs, being to a local path, don't contain that. So instead escape all illegal characters and check if the resulting thing is an URI. And that was already done by Git.Construct.fromUrl, so internally the gcrypt URI with a space looks like "gcrypt::foo%20bar" and that needs to be de-escaped when converting back from URI to local repo path. This change might also allow a few other almost-valid URIs to be handled as URIs by git-annex. None that contain "//" will change, and any behavior change should result in git-annex doing closer to a right thing than it did before, probably. This commit was sponsored by Noam Kremen on Patreon.
2021-03-09 16:30:13 +00:00
* Fix support for local gcrypt repositories with a space in their URI.
reorder
2021-03-10 14:15:45 +00:00
* uninit: Fix a small bug that left a lock file in .git/annex
* Windows: Correct the path to the html help file for 64 bit build.
git on OSX dmg updated to fix CVE This mostly affects OSX and (possibly) Windows, but the Windows installer does not bundle git. The linux standalone builds are not updated yet pending debian stable getting a backport of the security fix, but the security hole is unlikely to affect linux as case-insensitive filesystems that support symlinks are a rarity on it. Using the linux standalone build on windows via WSL is another way it could be affected. This commit was sponsored by Brett Eisenberg on Patreon.
2021-03-10 17:53:11 +00:00
* OSX dmg: Updated bundled git to 2.30.2 which fixes CVE-2021-21300.
Windows: Correct the path to the html help file for 64 bit build.
2021-02-24 17:19:42 +00:00
releasing package git-annex version 8.20210310
2021-03-10 17:59:00 +00:00
-- Joey Hess <id@joeyh.name> Wed, 10 Mar 2021 13:58:15 -0400
Windows: Correct the path to the html help file for 64 bit build.
2021-02-24 17:19:42 +00:00
releasing package git-annex version 8.20210223
2021-02-23 18:40:45 +00:00
git-annex (8.20210223) upstream; urgency=medium
formatLsTree did not use a tab where git does Fixed that, and made parserLsTree accept the space as well as tab. Fixes a reversion that made import of a tree from a special remote result in a merge that deleted files that were not preferred content of that special remote.
2021-01-28 16:36:37 +00:00
reorder
2021-02-08 21:54:29 +00:00
* annex.stalldetection can now be set to "true" to make git-annex
do automatic stall detection when it detects a remote is updating its
wording
2021-02-22 16:51:52 +00:00
transfer progress consistently enough for stall detection to work.
reorder
2021-02-08 21:54:29 +00:00
* When annex.stalldetection is not enabled and a likely stall is
detected, display a suggestion to enable it.
addon commands Seems only fair, that, like git runs git-annex, git-annex runs git-annex-foo. Implementation relies on O.forwardOptions, so that any options are passed through to the addon program. Note that this includes options before the subcommand, eg: git-annex -cx=y foo Unfortunately, git-annex eats the --help/-h options. This is because it uses O.hsubparser, which injects that option into each subcommand. Seems like this should be possible to avoid somehow, to let commands display their own --help, instead of the dummy one git-annex displays. The two step searching mirrors how git works, it makes finding git-annex-foo fast when "git annex foo" is run, but will also support fuzzy matching, once findAllAddonCommands gets implemented. This commit was sponsored by Dr. Land Raider on Patreon.
2021-02-02 20:32:25 +00:00
* Commands can be added to git-annex, by installing a program in PATH
with a name starting with "git-annex-"
formatLsTree did not use a tab where git does Fixed that, and made parserLsTree accept the space as well as tab. Fixes a reversion that made import of a tree from a special remote result in a merge that deleted files that were not preferred content of that special remote.
2021-01-28 16:36:37 +00:00
* Fix a reversion that made import of a tree from a special remote
result in a merge that deleted files that were not preferred content
of that special remote.
releasing package git-annex version 8.20210223
2021-02-23 18:40:45 +00:00
* Bugfix: fsck --from a ssh remote did not actually check that the
content on the remote is not corrupted.
* unannex, uninit: When an annexed file is modified, don't overwrite
the modified version with an older version from the annex.
When adding files to an adjusted branch set up by --unlock-present, add them unlocked, not locked Missed this when implementing it because of the default case catching the new constructor. So, removed that default case to make sure future types of adjusted branches don't make the same mistake. Complicated by git-annex addurl --fast which adds the file whose content is not present, so it needs to stay unlocked when on such a branch. This commit was sponsored by Brock Spratlen on Patreon.
2021-01-25 17:55:01 +00:00
* When adding files to an adjusted branch set up by --unlock-present,
add them unlocked, not locked.
fix handling of implicit and before parens Fix an oddity in matching options and preferred content expressions such as "foo (bar or baz)", which was incorrectly handled as if it were "(foo or bar) and baz)" rather than the intended "foo and (bar or baz)" Seemed like a change to consume should be able to handle this case better, but I was having trouble writing it that way, so instead added a separate pass that inserts the implicit ands explicitly. Also added several test cases to make sure versions with and without explicit ands generate the same.
2021-01-28 17:50:38 +00:00
* Fix an oddity in matching options and preferred content expressions
such as "foo (bar or baz)", which was incorrectly handled as if
it were "(foo or bar) and baz)" rather than the intended
"foo and (bar or baz)"
releasing package git-annex version 8.20210223
2021-02-23 18:40:45 +00:00
* Checksum as content is received from a remote git-annex repository,
rather than doing it in a second pass.
* Tahoe: Avoid verifying hash after download, since tahoe does sufficient
verification itself.
* unannex, uninit: Don't run git rm once per annexed file,
for a large speedup.
reorder
2021-02-08 21:54:29 +00:00
* When a git remote is configured with an absolute path, use that
path, rather than making it relative.
get: Improve output when failing to get a file fails showTriedRemotes lists the remotes it tried to access. So there's no need to list those again in "Try making some of these remotes available".
2021-01-29 19:11:19 +00:00
* get: Improve output when failing to get a file fails.
Fix build on openbsd. Thanks, James Cook for the patch.
2021-02-01 15:56:17 +00:00
* Fix build on openbsd.
Thanks, James Cook for the patch.
Include libkqueue.h file needed to build the assistant on BSDs I suspect this is a bug in cabal sdist, because with Includes: Utility/libkqueue.h the file is not included, but putting it in extra-files does get it into the tarball.
2021-02-01 16:00:56 +00:00
* Include libkqueue.h file needed to build the assistant on BSDs.
formatLsTree did not use a tab where git does Fixed that, and made parserLsTree accept the space as well as tab. Fixes a reversion that made import of a tree from a special remote result in a merge that deleted files that were not preferred content of that special remote.
2021-01-28 16:36:37 +00:00
releasing package git-annex version 8.20210223
2021-02-23 18:40:45 +00:00
-- Joey Hess <id@joeyh.name> Tue, 23 Feb 2021 14:40:14 -0400
formatLsTree did not use a tab where git does Fixed that, and made parserLsTree accept the space as well as tab. Fixes a reversion that made import of a tree from a special remote result in a merge that deleted files that were not preferred content of that special remote.
2021-01-28 16:36:37 +00:00
releasing package git-annex version 8.20210127
2021-01-27 15:13:25 +00:00
git-annex (8.20210127) upstream; urgency=medium
add: Significantly speed up adding lots of non-large files to git * add: Significantly speed up adding lots of non-large files to git, by disabling the annex smudge filter when running git add. * add --force-small: Run git add rather than updating the index itself, so any other smudge filters than the annex one that may be enabled will be used.
2021-01-04 17:12:28 +00:00
mincopies This is conceptually very simple, just making a 1 that was hard coded be exposed as a config option. The hard part was plumbing all that, and dealing with complexities like reading it from git attributes at the same time that numcopies is read. Behavior change: When numcopies is set to 0, git-annex used to drop content without requiring any copies. Now to get that (highly unsafe) behavior, mincopies also needs to be set to 0. It seemed better to remove that edge case, than complicate mincopies by ignoring it when numcopies is 0. This commit was sponsored by Denis Dzyubenko on Patreon.
2021-01-06 18:11:08 +00:00
* Added mincopies configuration. This is like numcopies, but is
docs for requirednumcopies Not implemented yet.
2021-01-05 18:22:27 +00:00
enforced even more strictly. While numcopies can be violated in
concurrent drop situations involving special remotes that do not
mincopies This is conceptually very simple, just making a 1 that was hard coded be exposed as a config option. The hard part was plumbing all that, and dealing with complexities like reading it from git attributes at the same time that numcopies is read. Behavior change: When numcopies is set to 0, git-annex used to drop content without requiring any copies. Now to get that (highly unsafe) behavior, mincopies also needs to be set to 0. It seemed better to remove that edge case, than complicate mincopies by ignoring it when numcopies is 0. This commit was sponsored by Denis Dzyubenko on Patreon.
2021-01-06 18:11:08 +00:00
support locking, mincopies cannot be. The default value has always
wording
2021-01-06 18:26:39 +00:00
been 1, but now it can be set to higher values if desired.
mincopies This is conceptually very simple, just making a 1 that was hard coded be exposed as a config option. The hard part was plumbing all that, and dealing with complexities like reading it from git attributes at the same time that numcopies is read. Behavior change: When numcopies is set to 0, git-annex used to drop content without requiring any copies. Now to get that (highly unsafe) behavior, mincopies also needs to be set to 0. It seemed better to remove that edge case, than complicate mincopies by ignoring it when numcopies is 0. This commit was sponsored by Denis Dzyubenko on Patreon.
2021-01-06 18:11:08 +00:00
* Behavior change: When numcopies is set to 0, git-annex used to drop
content without requiring any copies. Now to get that (highly unsafe)
behavior, mincopies also needs to be set to 0.
Behavior change: git-annex trust now needs --force Since unconsidered use of trusted repositories can lead to data loss. Trusted has always been this way, but it used to be acceptable for git-annex to be set up so that data could be lost without using --force, and most or all other ways that can happen have already been eliminated. This commit was sponsored by Mark Reidenbach on Patreon.
2021-01-07 13:59:52 +00:00
* Behavior change: git-annex trust now needs --force, since unconsidered
use of trusted repositories can lead to data loss.
Behavior change: --trust-glacier option no longer overrides trust Since that can lead to data loss, which should never be enabled by an option other than --force. This commit was sponsored by Jake Vosloo on Patreon.
2021-01-07 14:37:43 +00:00
* Behavior change: --trust and --trust-glacier options no longer overrides
trust, since that can lead to data loss, which should never be enabled
by an option other than --force.
add: Significantly speed up adding lots of non-large files to git * add: Significantly speed up adding lots of non-large files to git, by disabling the annex smudge filter when running git add. * add --force-small: Run git add rather than updating the index itself, so any other smudge filters than the annex one that may be enabled will be used.
2021-01-04 17:12:28 +00:00
* add: Significantly speed up adding lots of non-large files to git,
by disabling the annex smudge filter when running git add.
* add --force-small: Run git add rather than updating the index itself,
so any other smudge filters than the annex one that may be enabled will
be used.
fix --time-limit It got broken in several ways by the streaming seeking optimisations around version 8.20201007. Moved time limit checking out of the matcher, which was a hack in the first place. So everywhere that uses Limit.getMatcher needs to check time limit. Well, almost everywhere. Command.Info uses it, but it does not make sense to time limit getting info. And Command.MultiCast uses it just to build up a list of files that then get passed to a command, so it would never have hit the timeout in a useful way. This implementation is a little more expensive when at time limit than necessary, since it continues seeking only to discard everything after the time limit. I did try making it close the file handles to force a faster shutdown, but that didn't work and hung. Could certianly be improved somehow, but seeking is probably not the expensive bit when a time limit is hit, so this seems acceptable for now.
2021-01-04 19:25:28 +00:00
* Fix --time-limit, which got broken in several ways by some optimisations
in version 8.20201007.
merged fix from kyle
2021-01-07 17:47:29 +00:00
* When syncing changes back from an adjusted branch to the basis branch,
include deletions of submodules.
Thanks, Kyle Meyer for the patch.
Bug fix: export with -J could fail when two files had the same content. Exporting is done inside a call to writeLockDbWhile which guarantees there is only one process uploading to a given ExportLocation.
2021-01-13 18:50:48 +00:00
* Bug fix: export with -J could fail when two files had the same content.
Bug fix: Fix tilde expansion in ssh urls when the tilde is the last character in the url. Thanks, Grond for the patch.
2021-01-18 16:22:48 +00:00
* Bug fix: Fix tilde expansion in ssh urls when the tilde is the last
character in the url.
Thanks, Grond for the patch.
avoid crashing when there are remotes using unparseable urls Including the non-standard URI form that git-remote-gcrypt uses for rsync. Eg, "ook://foo:bar" cannot be parsed because "bar" is not a valid port number. But git could have a remote with that, it would try to run git-remote-ook to handle it. So, git-annex has to allow for such things, rather than crashing. This commit was sponsored by Luke Shumaker on Patreon.
2021-01-18 18:52:56 +00:00
* Avoid crashing when there are remotes using unparseable urls.
Including the non-standard URI form that git-remote-gcrypt uses for rsync.
omit inode from ContentIdentifier for directory special remote Directory special remotes with importtree=yes now avoid unncessary overhead when inodes of files have changed, as happens whenever a FAT filesystem gets remounted. A few unusual edge cases of modifications won't be detected and imported. I think they're unusual enough not to be a concern. It would be possible to add a config setting that controls whether to compare inodes too, but does not seem worth bothering the user about currently. I chose to continue to use the InodeCache serialization, just with the inode zeroed. This way, if I later change my mind or make it configurable, can parse it back to an InodeCache and operate on it. The overhead of storing a 0 in the content identifier log seems worth it. There is a one-time cost to this change; all directory special remotes with importtree=yes will re-hash all files once, and will update the content identifier logs with zeroed inodes. This commit was sponsored by Brett Eisenberg on Patreon.
2021-01-19 16:57:15 +00:00
* Directory special remotes with importtree=yes now avoid unncessary
wording
2021-01-25 21:40:08 +00:00
hashing when inodes of files have changed, as happens whenever a FAT
omit inode from ContentIdentifier for directory special remote Directory special remotes with importtree=yes now avoid unncessary overhead when inodes of files have changed, as happens whenever a FAT filesystem gets remounted. A few unusual edge cases of modifications won't be detected and imported. I think they're unusual enough not to be a concern. It would be possible to add a config setting that controls whether to compare inodes too, but does not seem worth bothering the user about currently. I chose to continue to use the InodeCache serialization, just with the inode zeroed. This way, if I later change my mind or make it configurable, can parse it back to an InodeCache and operate on it. The overhead of storing a 0 in the content identifier log seems worth it. There is a one-time cost to this change; all directory special remotes with importtree=yes will re-hash all files once, and will update the content identifier logs with zeroed inodes. This commit was sponsored by Brett Eisenberg on Patreon.
2021-01-19 16:57:15 +00:00
filesystem gets remounted.
fix a bug that prevented git-annex init from working in a submodule This is probably a reversion, but not sure what caused it. By the time Annex.Init runs fixupUnusualReposAfterInit, another git-annex process has at least sometimes already done the necessary fixups. (Eg, one run indirectly by a git command.) But since the Repo is cached, it doesn't realize and does them again. So, avoid crashing when git config --unset fails. This commit was sponsored by Jack Hill on Patreon.
2021-01-21 19:33:15 +00:00
* Fix a bug that prevented git-annex init from working in a submodule.
Fix a bug in view filename generation when a metadata value ended with "/" Or ":" or "\" on Windows, eg "c:" again.
2021-01-22 18:05:14 +00:00
* Fix a bug in view filename generation when a metadata value ended with
"/" (or ":" or "\" on Windows)
adjust: Fix some bad behavior when unlocked files use URL keys. This avoids the smudge --clean filter failing on the URL keys. git checkout runs the post-checkout hook, which runs smudge --update. That populates all the pointer files, but it neglected to store their inode caches in the keys db. With that done, and the keys db flushed before smudge --clean gets run (by restagePointerFile), the isUnmodifiedCheap check can tell the file is not modified, so will not try to re-ingest it, which does not work with URL keys because they do not support genKey. It also seems possible that the isUnmodifiedCheap was also failing for non-URL keys, which would cause them to be re-ingested, leading to a lot of extra work. I have not verified that, but don't see why it wouldn't have happened. So this probably also speeds up checking out adjusted branches. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2021-01-25 21:25:42 +00:00
* adjust: Fix some bad behavior when unlocked files use URL keys.
support modifying and running git add on an unlocked file that used an URL key Avoids the smudge --clean filter failing because URL keys do not support genKey. Instead the modified content will be added using the default backend. This commit was sponsored by Jochen Bartl on Patreon.
2021-01-25 21:34:58 +00:00
* smudge: Fix some bad behavior when git add is run on an unlocked
releasing package git-annex version 8.20210127
2021-01-27 15:13:25 +00:00
file that used an URL key.
* Added GETGITREMOTENAME to external special remote protocol.
* Windows: Work around win32 length limits when dealing with lock files.
add: Significantly speed up adding lots of non-large files to git * add: Significantly speed up adding lots of non-large files to git, by disabling the annex smudge filter when running git add. * add --force-small: Run git add rather than updating the index itself, so any other smudge filters than the annex one that may be enabled will be used.
2021-01-04 17:12:28 +00:00
releasing package git-annex version 8.20210127
2021-01-27 15:13:25 +00:00
-- Joey Hess <id@joeyh.name> Wed, 27 Jan 2021 11:09:25 -0400
add: Significantly speed up adding lots of non-large files to git * add: Significantly speed up adding lots of non-large files to git, by disabling the annex smudge filter when running git add. * add --force-small: Run git add rather than updating the index itself, so any other smudge filters than the annex one that may be enabled will be used.
2021-01-04 17:12:28 +00:00
releasing package git-annex version 8.20201129
2020-12-29 17:51:40 +00:00
git-annex (8.20201129) upstream; urgency=medium
Fix hang on shutdown of external special remote using ASYNC protocol extension. Reversion introduced in version 8.20201007, one release after the 1st release with the extension. Surprisingly, hClose can hang if another thread is reading from the handle. This is because it uses takeMVar. The use of cancel here does mean that, if receiveMessageAddonProcess or Remote.External.AsyncExtension.receiveloop allocated some resource in a non-async-exception safe way, they might not get a chance to clean it up. They do not appear to, and anyway, this only happens when git-annex is shutting down, so any recource that did leak would not be a problem. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2020-11-30 17:03:47 +00:00
merged borg Still a couple related todos, but it's basically usable now.
2020-12-22 20:22:44 +00:00
* New borg special remote. This is a new kind of remote, that examines
borg backups of git-annex repositories, learns what files have been
backed up, and can restore files from the backup and so on. As well
as backups, it can be useful for archival storage, since borg can
efficiently store many related versions of files.
stall detection is working New config annex.stalldetection, remote.name.annex-stalldetection, which can be used to deal with remotes that stall during transfers, or are sometimes too slow to want to use. This commit was sponsored by Luke Shumaker on Patreon.
2020-12-08 19:22:18 +00:00
* New config annex.stalldetection, remote.name.annex-stalldetection,
which can be used to deal with remotes that stall during transfers,
or are sometimes too slow to want to use.
reorg
2020-12-15 16:12:03 +00:00
* Support special remotes that are configured with importtree=yes but
without exporttree=yes.
avoid hGetMetered 0 closing the handle This is an edge case, which happened to be triggered by the P2P protocol seeing DATA 0. When reading 0 bytes, getting an empty string does not mean the handle has reached EOF. I verified there was in fact a bug, where get of an empty file followed by another file would get the empty file and then fail with "handle is closed". This fixes it. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2020-12-01 19:39:22 +00:00
* Fix bug that made the next download after an empty file from a ssh
or tor remote fail.
improve p2p protocol handling of requested object not available Avoid spurious "verification of content failed" message when downloading content from a ssh or tor remote fails due to the remote no longer having a copy of the content. The P2P protocol already handled this case by sending DATA 0, followed by VALID. But VALID was not really right, because the data is not the requested data. So, send DATA 0, followed by INVALID. Old versions of git-annex handle INVALID the same as VALID in this case. Now new versions avoid displaying an incorrect message. It would be better for the P2P protocol to have a different way to indicate this, like perhaps sending INVALID without DATA. But that would be a breaking change and need a new protocol verison. Since INVALID already is part of the protocol and already needs to be handled, using it for this special case too seems ok, and avoids the complication of another protocol version. This commit was sponsored by Jochen Bartl on Patreon.
2020-12-01 20:05:55 +00:00
* Avoid spurious "verification of content failed" message when downloading
content from a ssh or tor remote fails due to the remote no longer
having a copy of the content.
reorg
2020-12-15 16:12:03 +00:00
* Fix bug that matched include= and exclude= in preferred/required content
expressions relative to the current directory, rather than the path
from the top of the repository.
(Reversion introduced in version 8.20201116.)
* Fix hang on shutdown of external special remote using ASYNC protocol
extension.
(Reversion introduced in version 8.20201007.)
improve some edge cases around partial initialization * Guard against running in a repo where annex.uuid is set but annex.version is set, or vice-versa. * Avoid autoinit when a repo does not have annex.version or annex.uuid set, but has a git-annex objects directory, suggesting it was used by git-annex before.
2020-12-14 17:17:43 +00:00
* Guard against running in a repo where annex.uuid is set but
typo
2020-12-15 01:35:31 +00:00
annex.version is not set, or vice-versa.
improve some edge cases around partial initialization * Guard against running in a repo where annex.uuid is set but annex.version is set, or vice-versa. * Avoid autoinit when a repo does not have annex.version or annex.uuid set, but has a git-annex objects directory, suggesting it was used by git-annex before.
2020-12-14 17:17:43 +00:00
* Avoid autoinit when a repo does not have annex.version or annex.uuid
set, but has a git-annex objects directory, suggesting it was used
clarify
2020-12-15 16:09:27 +00:00
by git-annex before, and the git config may have been lost.
importfeed: Avoid using youtube-dl when a feed does not contain an enclosure, but only a link to an url which youtube-dl does not support This is common in some feeds, which might mix some items with enclosures, with others that link to posts or whatever. Before this, it would try to use youtube-dl and fail, or if youtube-dl was not allowed, it would incorrectly complain that an url was supported by youtube-dl.
2020-12-15 05:13:21 +00:00
* importfeed: Avoid using youtube-dl when a feed does not contain an
enclosure, but only a link to an url which youtube-dl does not support.
reorg
2020-12-15 16:12:03 +00:00
* initremote: Prevent enabling encryption with exporttree=yes or
importtree=yes.
Windows: include= and exclude= containing '/' will also match filenames that are written using '\' And vice-versa, but it's better to use '/' for portability. Notably, standardPreferredContent contains "archive/*" and that might not match if the filename ends up coming in with the slashes the other way around.
2020-12-15 16:39:34 +00:00
* Windows: include= and exclude= containing '/' will also match filenames
that are written using '\'. (And vice-versa, but it's better to use '/'
for portability.)
graft in imported tree to avoid gc Fix a bug that could prevent getting files from an importtree=yes remote, because the imported tree was allowed to be garbage collected.
2020-12-23 18:27:38 +00:00
* Fix a bug that could prevent getting files from an importtree=yes
remote, because the imported tree was allowed to be garbage collected.
releasing package git-annex version 8.20201129
2020-12-29 17:51:40 +00:00
* stack.yaml: Updated to lts-16.27.
Fix hang on shutdown of external special remote using ASYNC protocol extension. Reversion introduced in version 8.20201007, one release after the 1st release with the extension. Surprisingly, hClose can hang if another thread is reading from the handle. This is because it uses takeMVar. The use of cancel here does mean that, if receiveMessageAddonProcess or Remote.External.AsyncExtension.receiveloop allocated some resource in a non-async-exception safe way, they might not get a chance to clean it up. They do not appear to, and anyway, this only happens when git-annex is shutting down, so any recource that did leak would not be a problem. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2020-11-30 17:03:47 +00:00
releasing package git-annex version 8.20201129
2020-12-29 17:51:40 +00:00
-- Joey Hess <id@joeyh.name> Tue, 29 Dec 2020 12:52:58 -0400
Fix hang on shutdown of external special remote using ASYNC protocol extension. Reversion introduced in version 8.20201007, one release after the 1st release with the extension. Surprisingly, hClose can hang if another thread is reading from the handle. This is because it uses takeMVar. The use of cancel here does mean that, if receiveMessageAddonProcess or Remote.External.AsyncExtension.receiveloop allocated some resource in a non-async-exception safe way, they might not get a chance to clean it up. They do not appear to, and anyway, this only happens when git-annex is shutting down, so any recource that did leak would not be a problem. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2020-11-30 17:03:47 +00:00
releasing package git-annex version 8.20201127
2020-11-27 16:57:02 +00:00
git-annex (8.20201127) upstream; urgency=medium
examinekey: Added a "file" format variable For consistency with find, and for easier scripting.
2020-11-16 13:56:32 +00:00
Merge branch 'master' into symlink-missing
2020-11-16 14:03:12 +00:00
* adjust: New --unlock-present mode which locks files whose content is not
present (so the broken symlink is visible), while unlocking files whose
content is present.
annex.adjustedbranchrefresh Added annex.adjustedbranchrefresh git config to update adjusted branches set up by git-annex adjust --unlock-present/--hide-missing. Note, in a few cases, I was not able to make the adjusted branch be updated in calls to moveAnnex, because information about what file corresponds to a key is not available. They are: * If two files point to one file, then eg, `git annex get foo` will update the branch to unlock foo, but will not unlock bar, because it does not know about it. Might be fixable by making `git annex get bar` do something besides skipping bar? * git-annex-shell recvkey likewise (so sends over ssh from old versions of git-annex) * git-annex setkey * git-annex transferkey if the user does not use --file * git-annex multicast sends keys with no associated file info Doing a single full refresh at the end, after any incremental refresh, will deal with those edge cases.
2020-11-16 18:09:55 +00:00
* Added annex.adjustedbranchrefresh git config to update adjusted
branches set up by git-annex adjust --unlock-present/--hide-missing.
update
2020-11-18 19:16:49 +00:00
* Fix hang when an external special remote program exited but
the stderr pipe to it was left open, due to a daemon having inherited
the file descriptor.
don't rely on exception for http 416 Fix a bug that could make resuming a download from the web fail when the entire content of the file is actually already present locally. What a mess that Request can throw exceptions or not, depending on how it's configured. Makes it very hard if you need to handle some specific http status codes in a function like this! Implementing everything two ways did not seem appealing, if possible at all, so I decided to override the Request if it did come configured to throw exception on non-2xx http status. Other exceptions, like from http-client-restricted, or due to a redirect to a non-http url, still get thrown. This commit was sponsored by Luke Shumaker on Patreon.
2020-11-19 18:03:00 +00:00
* Fix a bug that could make resuming a download from the web fail
when the entire content of the file is actually already present
locally.
releasing package git-annex version 8.20201127
2020-11-27 16:57:02 +00:00
* examinekey: Added a "file" format variable for consistency with find,
and for easier scripting.
* init: When writing hook scripts, set all execute bits, not only
the user execute bit.
upgrade: Support an edge case upgrading a v5 direct mode repo where nothing had ever been committed to the head branch This commit was sponsored by Jack Hill on Patreon.
2020-11-24 16:31:17 +00:00
* upgrade: Support an edge case upgrading a v5 direct mode repo
where nothing had ever been committed to the head branch.
Made the test suite significantly less noisy Only displaying git-annex and git command output when something went wrong. A few could still leak stderr. These include the couple of calls to readProcess, which reads stdin but lets stderr through. But they don't leak any usually, so probably only would when failing anyway. Currently, there is no excess output at all! This commit was sponsored by Brock Spratlen on Patreon.
2020-11-24 18:07:46 +00:00
* Made the test suite significantly less noisy, only displaying command
output when something failed.
releasing package git-annex version 8.20201127
2020-11-27 16:57:02 +00:00
* Fix building without the torrent library.
Thanks, Kyle Meyer.
* Fix build on Windows.
* Prevent windows assistant from trying (and failing) to upgrade
itself, which has never been supported on windows.
examinekey: Added a "file" format variable For consistency with find, and for easier scripting.
2020-11-16 13:56:32 +00:00
releasing package git-annex version 8.20201127
2020-11-27 16:57:02 +00:00
-- Joey Hess <id@joeyh.name> Fri, 27 Nov 2020 12:54:11 -0400
examinekey: Added a "file" format variable For consistency with find, and for easier scripting.
2020-11-16 13:56:32 +00:00
releasing package git-annex version 8.20201116
2020-11-16 13:38:29 +00:00
git-annex (8.20201116) upstream; urgency=medium
use MatchingKey when a Key is known This fixes a bug where a file that was not preferred content could be transferred to a remote. This happened when the file got deleted after the sync started running. The only time checkMatcher is run without a Key is in calls to checkFileMatcher, which are only done by add, addurl, import, and smudge --clean. Those won't be affected by this kind of race. Anything else that might be precaching and have a similar race as sync will also be fixed, but I don't know if it actually affected anything other than sync. As well as fixing a bug, this also probably makes sync and --auto faster by avoiding the redundant key lookup. This commit was sponsored by Graham Spencer on Patreon.
2020-11-09 19:12:08 +00:00
move: Fix a regression in the last release that made move --to not honor numcopies settings This commit was sponsored by Svenne Krap on Patreon.
2020-11-13 18:19:32 +00:00
* move: Fix a regression in the last release that made move --to not
honor numcopies settings.
use MatchingKey when a Key is known This fixes a bug where a file that was not preferred content could be transferred to a remote. This happened when the file got deleted after the sync started running. The only time checkMatcher is run without a Key is in calls to checkFileMatcher, which are only done by add, addurl, import, and smudge --clean. Those won't be affected by this kind of race. Anything else that might be precaching and have a similar race as sync will also be fixed, but I don't know if it actually affected anything other than sync. As well as fixing a bug, this also probably makes sync and --auto faster by avoiding the redundant key lookup. This commit was sponsored by Graham Spencer on Patreon.
2020-11-09 19:12:08 +00:00
* sync --content: Fix a bug where a file that was not preferred content
could be transferred to a remote. This happened when the file got deleted
after the sync started running.
add warning on add of annex link Warn when adding a annex symlink or pointer file that uses a key that is not known to the repository, to prevent confusion if the user has copied it from some other repository. This commit was sponsored by Jake Vosloo on Patreon.
2020-11-10 16:10:51 +00:00
* Warn when adding a annex symlink or pointer file that uses a key that
is not known to the repository, to prevent confusion if the user has
copied it from some other repository.
examinekey: Added two new format variables: objectpath and objectpointer
2020-11-12 17:02:31 +00:00
* examinekey: Added two new format variables: objectpath and objectpointer
examinekey: Added --migrate-to-backend Note that, the way the SeekInput parser is written to support batch mode, it's actually possible to do git-annex examinekey "SHA1--foo foo.tar.gz" --migrate-to-backend=SHA1E While that might be kind of useful to support multiple migrations not using batch mode, I have not documented it. It would be better to take pairs of key and file in that case.
2020-11-12 18:08:13 +00:00
* examinekey: Added --migrate-to-backend
use MatchingKey when a Key is known This fixes a bug where a file that was not preferred content could be transferred to a remote. This happened when the file got deleted after the sync started running. The only time checkMatcher is run without a Key is in calls to checkFileMatcher, which are only done by add, addurl, import, and smudge --clean. Those won't be affected by this kind of race. Anything else that might be precaching and have a similar race as sync will also be fixed, but I don't know if it actually affected anything other than sync. As well as fixing a bug, this also probably makes sync and --auto faster by avoiding the redundant key lookup. This commit was sponsored by Graham Spencer on Patreon.
2020-11-09 19:12:08 +00:00
releasing package git-annex version 8.20201116
2020-11-16 13:38:29 +00:00
-- Joey Hess <id@joeyh.name> Mon, 16 Nov 2020 09:36:35 -0400
use MatchingKey when a Key is known This fixes a bug where a file that was not preferred content could be transferred to a remote. This happened when the file got deleted after the sync started running. The only time checkMatcher is run without a Key is in calls to checkFileMatcher, which are only done by add, addurl, import, and smudge --clean. Those won't be affected by this kind of race. Anything else that might be precaching and have a similar race as sync will also be fixed, but I don't know if it actually affected anything other than sync. As well as fixing a bug, this also probably makes sync and --auto faster by avoiding the redundant key lookup. This commit was sponsored by Graham Spencer on Patreon.
2020-11-09 19:12:08 +00:00
releasing package git-annex version 8.20201103
2020-11-03 15:53:11 +00:00
git-annex (8.20201103) upstream; urgency=medium
Fix build on Windows with network-3 inet_addr was removed, but all this needs is localhost, so hardcoding it should work fine. It may be that this windows ifdef is no longer needed. It was added in 2013 with a note that getAddrInfo didn't work on windows, but it seems likely such a problem would have been fixed since.
2020-10-08 14:50:39 +00:00
move: Improve resuming a move that was interrupted after the object was transferred In cases where numcopies checks prevented the resumed move from dropping the object from the source repository, it now relies on a log of recent moves to replicate the behavior of the interrupted command. Performance: Probably noticable impact, since it has to add to the log, check the log, and remove from the log. Seems worth it to avoid this annoying edge case. The log functions are pretty well optimised to avoid unncessary work. An performance improvement to make later would be to avoid cleanup doing anything if it's not written to the log file, and has confirmed that the log file does not contain the log line. This commit was sponsored by Jake Vosloo on Patreon.
2020-10-21 14:31:56 +00:00
* move: Improve resuming a move that was interrupted after the object
was transferred, in cases where numcopies checks prevented the resumed
move from dropping the object from the source repository.
speculatively use remote's configured chunk size as a fallback When a special remote has chunking enabled, but no chunk sizes are recorded (or the recorded ones are not found), speculatively try chunks using the configured chunk size. This makes eg, git-annex fsck --from remote be able to fix up the location log of a file that the git-annex branch does not indicate is stored on the remote. Note that fsck does *not* fix up the chunk log to indicate the chunk size. So, changing the chunk config of the remote after that will still prevent accessing the chunks stored on it. Maybe fsck should, but I wanted to start with this and see if it's needed.
2020-10-22 16:57:58 +00:00
* When a special remote has chunking enabled, but no chunk sizes are
recorded (or the recorded ones are not found), speculatively try
chunks using the configured chunk size.
deal with .git pointer file in Git.CurrentRepo This fixes the bug. Note, it's only done when GIT_DIR is set. When it's not set, Git.Construct already handled it. This is why it was only noticed with this git submodule command. This commit was sponsored by Brett Eisenberg on Patreon.
2020-10-23 18:56:12 +00:00
* Fixed some problems that prevented this command from working:
git submodule foreach git annex init
releasing package git-annex version 8.20201103
2020-11-03 15:53:11 +00:00
* Improve shutdown process for external special remotes and external
backends. Make sure to relay any remaining stderr from the process,
and potentially avoid the process getting a SIGPIPE if it writes to
stderr too late.
* Fix a bug that prevented linux standalone bundle from working on a fresh
install.
Windows build changed to one done by the datalad-extensions project using Github actions This is a cleaner build than on Jenkins because the whole environment setup is handled by the CI config, at least up to the point of "get a random bag of Windows bytes". Also, the Jenkins autobuilder has been intermittently failing for a long time, not due to any problem with git-annex but just a failure to clean up directories. Also, this build runs the test suite, and it is (mostly) passing. Test suite always failed in the jenkins environment. Also, this build includes libmagic. Here is the build workflow used by github actions: https://github.com/datalad/datalad-extensions/blob/master/.github/workflows/build-git-annex-windows.yaml The libmagic build has its own workflow: https://github.com/datalad/file-windows/blob/master/.github/workflows/build.yml (Also cleaned up some windows build cruft I don't use anymore.) There is no build-version file to link to. I've opened a todo requesting one: https://github.com/datalad/datalad-extensions/issues/55
2020-10-26 17:06:50 +00:00
* Windows build changed to one done by the datalad-extensions project
using Github actions.
changelog
2020-10-26 17:43:40 +00:00
* Windows build now includes libmagic, so mimetype and mimeencoding
will work.
Thanks to John Thorvald Wodder II and Yaroslav Halchenko for their work
on this.
view: Avoid using ':' from metadata when generating a view Because it's a special character on Windows ("c:"). Use same technique already used for '/' and '\'. I didn't record how I generated their encoded forms before, so am sure there was a better way, but the way I did it now is to look at ghci> encodeFilePath "∕" "\226\136\149" And then the difference from that to "\56546\56456\56469" is adding 56320 to each, to get up to the escaped code plane. See comment for why I think handling ':' is ok, but that other illegal windows filenames won't. Note that, this should be enough to make the test suite always work. Other windows illegal filenames will fail at checkout time when it tries to put the illegal filename on the filesystem.
2020-10-26 19:37:55 +00:00
* view: Avoid using ':' from metadata when generating a view, because
it's a special character on Windows ("c:")
releasing package git-annex version 8.20201103
2020-11-03 15:53:11 +00:00
* Fix a memory leak introduced in the last release.
* add, import: Fix a reversion in 7.20191009 that broke handling
of --largerthan and --smallerthan.
view: Fix a reversion in 8.20200522 that broke entering or changing views. Commit 2dc7b5186a586acb768744242083ad48d658a538 messed up indentation. This commit was sponsored by Noam Kremen on Patreon.
2020-11-02 18:47:08 +00:00
* view: Fix a reversion in 8.20200522 that broke entering or changing views.
releasing package git-annex version 8.20201103
2020-11-03 15:53:11 +00:00
* Fix build on Windows with network-3.
* testremote: Display exceptions when tests fail, to aid debugging.
Fix build on Windows with network-3 inet_addr was removed, but all this needs is localhost, so hardcoding it should work fine. It may be that this windows ifdef is no longer needed. It was added in 2013 with a note that getAddrInfo didn't work on windows, but it seems likely such a problem would have been fixed since.
2020-10-08 14:50:39 +00:00
releasing package git-annex version 8.20201103
2020-11-03 15:53:11 +00:00
-- Joey Hess <id@joeyh.name> Tue, 03 Nov 2020 11:40:56 -0400
Fix build on Windows with network-3 inet_addr was removed, but all this needs is localhost, so hardcoding it should work fine. It may be that this windows ifdef is no longer needed. It was added in 2013 with a note that getAddrInfo didn't work on windows, but it seems likely such a problem would have been fixed since.
2020-10-08 14:50:39 +00:00
releasing package git-annex version 8.20201007
2020-10-07 18:10:56 +00:00
git-annex (8.20201007) upstream; urgency=medium
aws-0.22 improved its support for setting etags, which improves support for versioned S3 buckets. Remove placeholder version number I used when implementing the feature in aws. This commit was sponsored by Ethan Aubin.
2020-09-14 22:37:49 +00:00
add SeekInput (not yet used) No behavior changes (hopefully), just adding SeekInput and plumbing it through to the JSON display code for later use. Over the course of 2 grueling days. withFilesNotInGit reimplemented in terms of seekHelper should be the only possible behavior change. It seems to test as behaving the same. Note that seekHelper dummies up the SeekInput in the case where segmentPaths' gives up on sorting the expanded paths because there are too many input paths. When SeekInput later gets exposed as a json field, that will result in it being a little bit wrong in the case where 100 or more paths are passed to a git-annex command. I think this is a subtle enough problem to not matter. If it does turn out to be a problem, fixing it would require splitting up the input parameters into groups of < 100, which would make git ls-files run perhaps more than is necessary. May want to revisit this, because that fix seems fairly low-impact.
2020-09-14 20:49:33 +00:00
* --json output now includes a new field "input" which is the input
add "input" field to json output The use case of this field is mostly to support -J combined with --json. When that is implemented, a user will be able to look at the field to determine which of the requests they have sent it corresponds to. The field typically has a single value in its list, but in some cases mutliple values (eg 2 command-line params) are combined together and the list will have more. Note that json parsing was already non-strict, so old git-annex metadata --json --batch can be fed json produced by the new git-annex and will not stumble over the new field.
2020-09-15 20:22:44 +00:00
value (filename, url, etc) that caused a json object to be output.
support --batch -J --batch combined with -J now runs batch requests concurrently for many commands. Before, the combination was accepted, but did not enable concurrency. Since the output of batch requests can be in any order, --json with the new "input" field is recommended to be used, to determine which batch request each response corresponds to. If --json is not used, batch mode still runs concurrently, using the usual concurrent-output. That will not be very useful for most batch mode users, probably, but who knows. If a program was using --batch -J before, and was parsing non-json output, this could break it. But, it was relying on git-annex not supporting concurrency despite it being enabled, so it should have expected concurrent output. So, I think that's ok. annex.jobs does not enable concurrency in --batch mode, because that would confuse programs that use --batch but don't expect concurrency.
2020-09-16 15:58:19 +00:00
* --batch combined with -J now runs batch requests concurrently for many
commands. Before, the combination was accepted, but did not enable
concurrency. Since the output of batch requests can be in any order,
--json with the new "input" field is recommended to be used,
to determine which batch request each response corresponds to.
aws-0.22 improved its support for setting etags, which improves support for versioned S3 buckets. Remove placeholder version number I used when implementing the feature in aws. This commit was sponsored by Ethan Aubin.
2020-09-14 22:37:49 +00:00
* aws-0.22 improved its support for setting etags, which improves
support for versioned S3 buckets.
Serialize use of C magic library, which is not thread safe. This fixes failures uploading to S3 when using -J. This commit was sponsored by Denis Dzyubenko on Patreon.
2020-09-17 21:27:42 +00:00
* Serialize use of C magic library, which is not thread safe.
This fixes failures uploading to S3 when using -J.
Added --no-check-gitignore option for finer grained control than using --force. add, addurl, importfeed, import: Added --no-check-gitignore option for finer grained control than using --force. (--force is used for too many different things, and at least one of these also uses it for something else. I would like to reduce --force's footprint until it only forces drops or a few other data losses. For now, --force still disables checking ignores too.) addunused: Don't check .gitignores when adding files. This is a behavior change, but I justify it by analogy with git add of a gitignored file adding it, asking to add all unused files back should add them all back, not skip some. The old behavior was surprising. In Command.Lock and Command.ReKey, CheckGitIgnore False does not change behavior, it only makes explicit what is done. Since these commands are run on annexed files, the file is already checked into git, so git add won't check ignores.
2020-09-18 17:12:04 +00:00
* add, addurl, importfeed, import: Added --no-check-gitignore option
for finer grained control than using --force.
import: Check gitignores when importing trees from special remotes It seemed best to do this, for consistency with every other way files can get into a git-annex repo. Although it's just a bit strange that a local .gitignore file affects the pseudo-commits made for the remote that's imported from. This commit was sponsored by Brett Eisenberg on Patreon.
2020-09-30 14:41:59 +00:00
* import: Check gitignores when importing trees from special remotes.
Added --no-check-gitignore option for finer grained control than using --force. add, addurl, importfeed, import: Added --no-check-gitignore option for finer grained control than using --force. (--force is used for too many different things, and at least one of these also uses it for something else. I would like to reduce --force's footprint until it only forces drops or a few other data losses. For now, --force still disables checking ignores too.) addunused: Don't check .gitignores when adding files. This is a behavior change, but I justify it by analogy with git add of a gitignored file adding it, asking to add all unused files back should add them all back, not skip some. The old behavior was surprising. In Command.Lock and Command.ReKey, CheckGitIgnore False does not change behavior, it only makes explicit what is done. Since these commands are run on annexed files, the file is already checked into git, so git add won't check ignores.
2020-09-18 17:12:04 +00:00
* addunused: Don't check .gitignores when adding files.
Improve the "Try making some of these repositories available" message With some hints for the user for what to do. Took care to avoid changing the json output. It would have been ok to add the new separated lists to it, in addition to the old list, but I didn't do that because I didn't see much point.
2020-09-22 18:10:30 +00:00
* Improve the "Try making some of these repositories available"
message, with some hints for the user for what to do.
Improve --debug output to show pid of processes that are started and stopped getPid returns Nothing if the process has already been stopped, and in that case, the pid will not be displayed. I think that would only happen if waitForProcess or similar gets called more than once on the same process handle though. getPid on unix has an overhead of only a MVar read. On Windows it needs to make a syscall, so will be probably more expensive. While the added expense happens even when debug logging is disabled, it should be small enough compared with the overhead of starting a process that it's not a problem. (It does occur to me that a debugM that took an IO String could only run it when debugging is really enabled, which would improve performance. It does not seem possible to use the current hslogger interface to do that though; it does not expose the information that would be needed.)
2020-09-24 16:39:57 +00:00
* Improve --debug output to show pid of processes that are started and
stopped.
sync --all avoid unncessary first pass Sped up seeking to around twice as fast, by avoiding a pass over the worktree files when preferred content expressions of the local repo and remotes don't use include=/exclude=. Thanks to Lukey for identifying the optimisation. This commit was sponsored by Brock Spratlen on Patreon.
2020-09-24 19:12:09 +00:00
* sync --all: Sped up seeking to around twice as fast, by avoiding a
pass over the worktree files when preferred content expressions of the
local repo and remotes don't use include=/exclude=.
seek: defer matcher check until more info is known Sped up seeking for files to operate on, when using options like --copies or --in, by around 20%. Benchmark showed an increase for --copies from 155 seconds to 121 seconds, and --in remote will be similar to that. For --in here, the speedup was less, 5-10% or so. (both warm cache) This commit was sponsored by Jack Hill on Patreon.
2020-09-24 21:59:05 +00:00
* Sped up seeking for files to operate on, when using options like
--copies or --in, by around 20%
import --no-content: Check annex.largefiles Import small files into git, the same as is done when importing with content. Which means, for small files, --no-content does download them. If the largefiles expression needs the file content available (due to mimetype or mimeencoding being used), the import will fail. This commit was sponsored by Jake Vosloo on Patreon.
2020-09-28 17:22:16 +00:00
* import --no-content: Check annex.largefiles, and import small
files into git, the same as is done when importing with content.
If the largefiles expression needs the file content available
(due to mimetype or mimeencoding being used), the import will fail.
sync --no-content import from directory special remote sync: When run without --content, import without copying from importtree=yes directory special remotes. (Other special remotes may support this later as well.) This commit was sponsored by Svenne Krap on Patreon.
2020-09-28 19:29:08 +00:00
* sync: When run without --content, import without copying from
importtree=yes directory special remotes.
(Other special remotes may support this later as well.)
addurl: Avoid a redundant git ignores check for speed Ensure that checkCanAdd is used everywhere a file is added to git, so git add is run with -f, presumably avoiding the work it would usually do to check ignores.
2020-09-29 17:00:41 +00:00
* addurl: Avoid a redundant git ignores check for speed.
upgrade: Avoid an upgrade failure of a bare repo in unusual circumstances
2020-09-29 17:45:14 +00:00
* upgrade: Avoid an upgrade failure of a bare repo in unusual circumstances.
httpalso: Support being used with special remotes that do not have encryption= in their config.
2020-09-29 17:56:27 +00:00
* httpalso: Support being used with special remotes that do not have
encryption= in their config.
Parse youtube-dl progress output Which lets progress be displayed when doing concurrent downloads. Amoung other things, like --json-progress etc. The youtube-dl output is no longer displayed, except for any errors. This commit was sponsored by Denis Dzyubenko on Patreon.
2020-09-29 21:53:48 +00:00
* Parse youtube-dl progress output, which lets progress be displayed
when doing concurrent downloads.
Fix build with Benchmark build flag Broke a while ago during optimisation work, and not noticed since the flag is disabled by default. This commit was sponsored by Brock Spratlen on Patreon.
2020-10-02 17:30:24 +00:00
* Fix build with Benchmark build flag.
Enable building with git-annex benchmark by default Only turning it off when the criterion library is not installed. Not enabled for osx or i386ancient yet since that will need some invesitgation to update their respective stack.yaml files.
2020-10-02 17:57:10 +00:00
* Enable building with git-annex benchmark by default, only turning it
off when the criterion library is not installed.
runshell: Fix a edge case where rm errors were sent to stdout, which could confuse things parsing git-annex output.
2020-10-05 16:44:40 +00:00
* runshell: Fix a edge case where rm errors were sent to stdout, which
could confuse things parsing git-annex output.
runshell: Update files atomically when preparing to run git-annex This does not make it entirely idempotent, but it's a start.
2020-10-05 17:38:34 +00:00
* runshell: Update files atomically when preparing to run git-annex.
Fix a build failure on Windows.
2020-10-07 16:04:54 +00:00
* Fix a build failure on Windows.
aws-0.22 improved its support for setting etags, which improves support for versioned S3 buckets. Remove placeholder version number I used when implementing the feature in aws. This commit was sponsored by Ethan Aubin.
2020-09-14 22:37:49 +00:00
releasing package git-annex version 8.20201007
2020-10-07 18:10:56 +00:00
-- Joey Hess <id@joeyh.name> Wed, 07 Oct 2020 12:19:05 -0400
aws-0.22 improved its support for setting etags, which improves support for versioned S3 buckets. Remove placeholder version number I used when implementing the feature in aws. This commit was sponsored by Ethan Aubin.
2020-09-14 22:37:49 +00:00
releasing package git-annex version 8.20200908
2020-09-08 18:20:47 +00:00
git-annex (8.20200908) upstream; urgency=medium
async extension done
2020-08-14 19:24:34 +00:00
rename http special remote to httpalso "http" was too generic and easy to confuse with web. The new name makes clear it's used in addition to some other remote. And other protocols can use the same naming scheme.
2020-09-02 14:41:27 +00:00
* Added httpalso special remote, which is useful for accessing
content stored on other remotes that is published by http.
async extension done
2020-08-14 19:24:34 +00:00
* The external special remote protocol got an ASYNC extension.
This can be used by an external special remote to let a single process
perform concurrent actions, rather than multiple processes being
started, when that is more efficient.
reorder
2020-09-08 16:54:17 +00:00
* Retry transfers to exporttree=yes remotes same as for other remotes.
* import: Retry downloads that fail, same as is done for downloads generally.
* Limit retrying of failed transfers when forward progress is being made
to 5, to avoid some unusual edge cases where too much retrying could
result in far more data transfer than makes sense.
* Exposed annex.forward-retry git config, to configure the forward retry
behavior that git-annex has had for a long time.
* sync, assistant, merge: When merge.directoryRenames is not set,
default it it to "false", which works better with automatic merge
conflict resolution than git's ususual default of "conflict".
(This is not done when automatic merge conflict resolution is disabled.)
* resolvemerge: Improve cleanup of cruft left in the working tree
by a conflicted merge.
* Support git remotes where .git is a file, not a directory,
eg when --separate-git-dir was used.
make viaTmp honor umask Fixed several cases where files were created without file mode bits that the umask would usually set. This included exports to the directory special remote, torrent files used by the bittorrent special remote, hooks written by git-annex init, and some log files in .git/annex/ Audited all calls, looking for ones that didn't want the umask bits to be set. All such turned out to already set the specific restrictive file mode they wanted.
2020-09-02 18:54:07 +00:00
* Fixed several cases where files were created without file mode bits
that the umask would usually set. This included exports to the
directory special remote, torrent files used by the bittorrent special
remote, hooks written by git-annex init, and some log files in .git/annex/
wording
2020-08-25 18:47:17 +00:00
* Fix reversion in 7.20190322 that made addurl --file not be honored
addurl --file youtube-dl reversion fix addurl: Fix reversion in 7.20190322 that made --file not be honored when youtube-dl was used to download media. 8758f9c561 was on the right track, but missed that | otherwise prevented the code it added from being used. Also, refactored out a common function. This commit was sponsored by Graham Spencer on Patreon.
2020-08-25 16:56:38 +00:00
when youtube-dl was used to download media.
add gitAnnexChildProcess and use instead of incorrect use of runsGitAnnexChildProcess Fixes reversion in 8.20200617 that made annex.pidlock being enabled result in some commands stalling, particularly those needing to autoinit. Renamed runsGitAnnexChildProcess to make clearer where it should be used. Arguably, it would be better to have a way to make any process git-annex runs have the env var set. But then it would need to take the pid lock when running any and all processes, and that would be a problem when git-annex runs two processes concurrently. So, I'm left doing it ad-hoc in places where git-annex really does run a child process, directly or indirectly via a particular git command.
2020-08-25 18:57:25 +00:00
* Fix reversion in 8.20200617 that made annex.pidlock being enabled
result in some commands stalling, particularly those needing to
autoinit.
reorder
2020-09-08 16:54:17 +00:00
* Display warning when external special remote does not start up
properly, or is not usable.
Display a message when git-annex has to wait for a pid lock file held by another process
2020-08-26 17:05:34 +00:00
* Display a message when git-annex has to wait for a pid lock file
held by another process.
test: Stop gpg-agent daemons that are started for the test framework's gpg key They normally shutdown when the GNUPGHOME directory is deleted, but on NFS they keep the directory from being deleted. And also, this avoids a number of them piling up while the test suite is running.
2020-08-28 18:28:42 +00:00
* test: Stop gpg-agent daemons that are started for the test framework's
gpg key.
Removed the S3 and WebDAV build flags So these special remotes are always supported. IIRC these build flags were added because the dep chains were a bit too long, or perhaps because the libraries were not available in Debian stable, or something like that. That was long ago, those reasons no longer apply, and users get confused when builtin special remotes are not available, so it seems best to remove the build flags now. If this does cause a problem it can be reverted of course.. This commit was sponsored by Jochen Bartl on Patreon.
2020-09-08 16:42:59 +00:00
* Removed the S3 and WebDAV build flags so these special remotes are
always supported.
reorder
2020-09-08 16:54:17 +00:00
* stack.yaml: Updated to lts-16.10.
async extension done
2020-08-14 19:24:34 +00:00
releasing package git-annex version 8.20200908
2020-09-08 18:20:47 +00:00
-- Joey Hess <id@joeyh.name> Tue, 08 Sep 2020 14:20:11 -0400
async extension done
2020-08-14 19:24:34 +00:00
git-annex (8.20200814) upstream; urgency=medium
fix build with dlist-1.0 That removed the list function. This new implementation appears to actually be more efficient anyway, since it avoids toList.
2020-07-21 16:58:51 +00:00
external backends implemented
2020-07-29 21:24:21 +00:00
* Added support for external backend programs. So if you want a hash
that git-annex doesn't support, or something stranger, you can write a
small program to implement it.
reorder
2020-08-10 20:18:35 +00:00
* Fix a bug in find --branch in the previous version.
* importfeed: Fix reversion that caused some '.' in filenames to be
replaced with '_'
Fix a lock file descriptor leak that could occur when running commands like git-annex add with -J Bug was introduced as part of a different FD leak fix in version 6.20160318.
2020-07-21 19:30:47 +00:00
* Fix a lock file descriptor leak that could occur when running commands
like git-annex add with -J. Bug was introduced as part of a different FD
leak fix in version 6.20160318.
Fix a hang when using git-annex with an old openssh 7.2p2 Which had some weird inheriting of ssh FDs by sshd. Bug was introduced in git-annex version 7.20200202.7.
2020-07-21 20:14:25 +00:00
* Fix a hang when using git-annex with an old openssh 7.2p2, which had
some weird inheriting of ssh FDs by sshd. Bug was introduced in
git-annex version 7.20200202.7.
better benchmark for move/copy speedup
2020-07-24 17:34:12 +00:00
* move, copy --to: Sped up seeking files by 2x.
drop performance improvements Sped up seeking files to drop by 2x, and also some performance improvements to checking numcopies. Interestingly, the seek speedup is not due to precaching, but I think is due to calling getParsed earlier. Annex.Drop had to be changed to check inAnnex there, since it was removed from Command.Drop. All other users of Command.Drop already checked inAnnex themselves. This commit was sponsored by Ryan Newton on Patreon.
2020-07-24 17:27:30 +00:00
* drop: Sped up seeking files to drop by 2x, and also some performance
improvements to checking numcopies.
Deal with unusual IFS settings in the shell scripts for linux standalone and OSX app. Thanks, Yaroslav Halchenko
2020-07-24 18:46:50 +00:00
* Deal with unusual IFS settings in the shell scripts for linux
standalone and OSX app.
Thanks, Yaroslav Halchenko
stop symlink check once the top of the working tree is reached Avoid complaining that a file with "is beyond a symbolic link" when the filepath is absolute and the symlink in question is not actually inside the git repository. This assumes that inodes remain stable while the command is running. I think they always will, the filesystems where they are unstable change them across mounts. (If inodes were not stable, it would just complain about symlinks in the path that are not inside the working tree.) (On windows, I don't want to assume anything about inodes, they could be random numbers for all I know. But if they were, this would still be ok, as long as windows doesn't have symlinks that are detected by isSymbolicLink. Which seems a fair bet.)
2020-08-07 00:14:30 +00:00
* Avoid complaining that a file with "is beyond a symbolic link"
when the filepath is absolute and the symlink in question is not
actually inside the git repository.
reorder
2020-08-10 20:18:35 +00:00
* Slightly sped up the linux standalone bundle.
* Support building with dlist-1.0
fix build with dlist-1.0 That removed the list function. This new implementation appears to actually be more efficient anyway, since it avoids toList.
2020-07-21 16:58:51 +00:00
fixed
2020-08-10 17:12:42 +00:00
-- Joey Hess <id@joeyh.name> Mon, 10 Aug 2020 13:04:04 -0400
fix build with dlist-1.0 That removed the list function. This new implementation appears to actually be more efficient anyway, since it avoids toList.
2020-07-21 16:58:51 +00:00
guidance on size and mtime fields
2020-07-20 23:56:47 +00:00
git-annex (8.20200720.1) upstream; urgency=medium
Fix a bug in find --batch in the previous version.
2020-07-20 23:39:05 +00:00
* Fix a bug in find --batch in the previous version.
guidance on size and mtime fields
2020-07-20 23:56:47 +00:00
-- Joey Hess <id@joeyh.name> Mon, 20 Jul 2020 19:39:11 -0400
Fix a bug in find --batch in the previous version.
2020-07-20 23:39:05 +00:00
releasing package git-annex version 8.20200720
2020-07-20 18:41:12 +00:00
git-annex (8.20200720) upstream; urgency=medium
fix deadlock Fix a deadlock that could occur after git-annex got an unlocked file, causing the command to hang indefinitely. Known to happen on vfat filesystems, possibly others. Note that a deadlock is still theoretically possible, if anything smudge --clean does causes it to run the git queue for some other reason. Apparently that doesn't happen, but will need to keep an eye on it.
2020-06-18 16:56:29 +00:00
import: Added --no-content option, which avoids downloading files from a special remote Only supported by some special remotes: directory I need to check the rest and they're currently missing methods until I do. git-annex sync --no-content does not yet use this to do imports
2020-07-03 17:41:57 +00:00
* import: Added --no-content option, which avoids downloading files
no importKey for android yet adb shell has sha256sum sha1sum and some others, so they could be used. They're provided by toybox, so seem about as likely to keep working as find and stat, which it already depends on. Or to not add a dep, could use stat the same as getExportContentIdentifier to get a mtime, and make a WORM key. But do I really want this to default to WORM? Unsure what's the best path, so punting for now.
2020-07-03 18:02:50 +00:00
from a special remote. Currently only supported by the directory
special remote.
Honor annex.largefiles when importing a tree from a special remote. This commit was sponsored by Martin D on Patreon.
2020-06-23 20:07:18 +00:00
* Honor annex.largefiles when importing a tree from a special remote.
import: Added --no-content option, which avoids downloading files from a special remote Only supported by some special remotes: directory I need to check the rest and they're currently missing methods until I do. git-annex sync --no-content does not yet use this to do imports
2020-07-03 17:41:57 +00:00
(Except for when --no-content is used.)
fix deadlock Fix a deadlock that could occur after git-annex got an unlocked file, causing the command to hang indefinitely. Known to happen on vfat filesystems, possibly others. Note that a deadlock is still theoretically possible, if anything smudge --clean does causes it to run the git queue for some other reason. Apparently that doesn't happen, but will need to keep an eye on it.
2020-06-18 16:56:29 +00:00
* Fix a deadlock that could occur after git-annex got an unlocked
file, causing the command to hang indefinitely. Known to happen on
vfat filesystems, possibly others.
Build with the http-client-restricted library when available Otherwise use the vendored copy as before. The library is in Debian testing but not stable. Once it reaches stable, the vendored copy can be removed. Did not add it to debian/control because IIRC that's used to build git-annex on stable too, possibly. However, the Debian maintainer will probably want to make the package depend on libghc-http-client-restricted-dev This commit was sponsored by Ilya Shlyakhter on Patreon.
2020-06-22 15:30:33 +00:00
* Build with the http-client-restricted and git-lfs libraries when
available, otherwise use the vendored copy as before.
testremote: Fix over-allocation of resources and bad caching Including starting up a large number of external special remote processes. (Regression introduced in version 8.20200501)
2020-06-22 18:25:49 +00:00
* testremote: Fix over-allocation of resources and bad caching,
including starting up a large number of external special remote processes.
(Regression introduced in version 8.20200501)
test: Fix some test cases that assumed git's default branch name git is making that configurable, and configuring it globally would break the test suite in a few places. No other part of git-annex assumes any branch name. Renamed a few placeholders to make that clearer. This commit was sponsored by Jake Vosloo on Patreon.
2020-06-23 20:40:41 +00:00
* test: Fix some test cases that assumed git's default branch name.
importfeed: Added some additional --template variables for date and time This commit was sponsored by Ethan Aubin.
2020-06-24 18:24:50 +00:00
* importfeed: Added some additional --template variables:
itempubyear, itempubmonth, itempubday, itempubhour,
itempubminute, itempubsecond.
changelog wording
2020-06-26 18:27:42 +00:00
* Made several special remotes support locking content on them,
which allows dropping from other special remotes in some situations
import: Added --no-content option, which avoids downloading files from a special remote Only supported by some special remotes: directory I need to check the rest and they're currently missing methods until I do. git-annex sync --no-content does not yet use this to do imports
2020-07-03 17:41:57 +00:00
where it was not possible before. Supported special remotes:
lockContent for tahoe Trivial since git-annex cannot remove, but do an active checkKey verification anyway, in case the data was lost somehow. This commit was sponsored by Ryan Newton on Patreon.
2020-06-26 18:23:21 +00:00
S3 (with versioning=yes), git-lfs, tahoe
refix bug in a better way Always run Git.Config.store, so when the git config gets reloaded, the override gets re-added to it, and changeGitRepo then calls extractGitConfig on it and sees the annex.* settings from the override. Remove any prior occurance of -c v and add it to the end. This way, -c foo=1 -c foo=2 -c foo=1 will pass -c foo=1 to git, rather than -c foo=2 Note that, if git had some multiline config that got built up by multiple -c's, this would not work still. But it never worked because before the bug got fixed in the first place, the -c value was repeated many times, so the multivalue thing would have been wrong. I don't think -c can be used with multiline configs anyway, though git-config does talk about them?
2020-07-02 17:32:33 +00:00
* Fix reversion that broke passing annex.* and remote.*.annex-*
git configs with -c. (Since version 8.20200330.)
improve wording
2020-07-13 21:57:55 +00:00
* Bring back git-annex branch read cache. This speeds up some operations,
eg git-annex sync --content --all gets 20% faster.
Revert "async exception safety for coprocesses" This reverts commit 7013798df5a161f00962985ffaea613a87cc4fe4.
2020-07-06 19:11:28 +00:00
* Fix a recently introduced bug that could cause a "fork: resource exhausted"
after getting several thousand files.
sped up the --all option by 2x to 16x by using git cat-file --buffer This assumes that no location log files will have a newline or carriage return in their name. catObjectStream skips any such files due to cat-file not supporting them. Keys have been prevented from containing newlines since 2011, commit 480495beb4a3422f006ee529df807a20cc944727. If some old repo had a key with a newline in it, --all will just skip processing that key. Other things, like .git/annex/unused files certianly assume no newlines in keys too, and AFAICR, such keys never actually worked. Carriage return is escaped by preSanitizeKeyName since 2013. WORM keys generated before that point could perhaps contain a CR. (URL probably not, http probably doesn't support an URL with a raw CR in it.) So, added a warning in fsck about such keys. Although, fsck --all will naturally skip them, so won't be able to warn about them. Not entirely satisfactory, but I'll bet there are not really any such keys in existence. Thanks to Lukey for finding this optimisation.
2020-07-07 17:46:45 +00:00
* Sped up the --all option by 2x to 16x by using git cat-file --buffer.
Thanks to Lukey for finding this optimisation.
slower but sequential filtering of large files from pointer files There should still be a speedup seeking over pointer files, just not as large as the one seeking over symlinks.
2020-07-10 19:18:42 +00:00
* Sped up seeking for annexed files to operate on by a factor of nearly 2x.
improve wording
2020-07-13 21:57:55 +00:00
* Sped up sync --content by 2x and other commands like fsck --fast and
whereis by around 50%, by using git cat-file --buffer.
importfeed: stream metadata for 5% speedup On top of the 10% speedup from streaming url logs.
2020-07-14 18:35:26 +00:00
* importfeed: Made checking known urls step around 15% faster.
sped up the --all option by 2x to 16x by using git cat-file --buffer This assumes that no location log files will have a newline or carriage return in their name. catObjectStream skips any such files due to cat-file not supporting them. Keys have been prevented from containing newlines since 2011, commit 480495beb4a3422f006ee529df807a20cc944727. If some old repo had a key with a newline in it, --all will just skip processing that key. Other things, like .git/annex/unused files certianly assume no newlines in keys too, and AFAICR, such keys never actually worked. Carriage return is escaped by preSanitizeKeyName since 2013. WORM keys generated before that point could perhaps contain a CR. (URL probably not, http probably doesn't support an URL with a raw CR in it.) So, added a warning in fsck about such keys. Although, fsck --all will naturally skip them, so won't be able to warn about them. Not entirely satisfactory, but I'll bet there are not really any such keys in existence. Thanks to Lukey for finding this optimisation.
2020-07-07 17:46:45 +00:00
* fsck: Detect if WORM keys contain a carriage return, and recommend
make catObjectStream support newline and carriage return in filenames Turns out the %(rest) trick was not needed. Instead, just maintain a list of files we've asked for, and each cat-file response is for the next file in the list. This actually benchmarks 25% faster than before! Very surprising, but it must be due to needing to shove less data through the pipe, and parse less.
2020-07-08 16:34:56 +00:00
upgrading the key. (git-annex could have maybe created such keys back
in 2013).
fix reversion in skipping deleted files And add a test case for that. This certianly loses some of the 2x performance improvement in file seeking that seekFilteredKeys led to, because now it has to stat the worktree files again. Without benchmarking, I expect there will still be a sizable improvement, and also the git-annex branch precaching that seekFilteredKeys can do will still be a win of its approach. Also worth noting that lookupKey, when the file DNE, check if it's in an adjusted branch with hidden files, and if so, finds the key for the file anyway. That was intended to make git-annex sync --content be able to process those files, but a side effect was that, when a file was deleted but the deletion not yet staged, git-annex commands used to still list it. That was actually a bug. This commit fixes that bug too. (git-annex sync --content on such a branch does not use seekFilteredKeys so was not affected by the reversion or by this behavior change) This commit was sponsored by Jake Vosloo on Patreon.
2020-07-20 00:33:10 +00:00
* When on an adjust --hide-missing branch, fix handling of files that
have been deleted but the deletion is not yet staged.
fix deadlock Fix a deadlock that could occur after git-annex got an unlocked file, causing the command to hang indefinitely. Known to happen on vfat filesystems, possibly others. Note that a deadlock is still theoretically possible, if anything smudge --clean does causes it to run the git queue for some other reason. Apparently that doesn't happen, but will need to keep an eye on it.
2020-06-18 16:56:29 +00:00
releasing package git-annex version 8.20200720
2020-07-20 18:41:12 +00:00
-- Joey Hess <id@joeyh.name> Mon, 20 Jul 2020 14:40:51 -0400
fix deadlock Fix a deadlock that could occur after git-annex got an unlocked file, causing the command to hang indefinitely. Known to happen on vfat filesystems, possibly others. Note that a deadlock is still theoretically possible, if anything smudge --clean does causes it to run the git queue for some other reason. Apparently that doesn't happen, but will need to keep an eye on it.
2020-06-18 16:56:29 +00:00
releasing package git-annex version 8.20200617
2020-06-17 19:59:34 +00:00
git-annex (8.20200617) upstream; urgency=medium
export: Added options for json output Just worked, no need to do anything except add the options.
2020-05-26 14:31:10 +00:00
annex.skipunknown with transition plan Added annex.skipunknown git config, that can be set to false to change the behavior of commands like `git annex get foo*`, to not skip over files/dirs that are not checked into git and are explicitly listed in the command line. Significant complexity was needed to handle git-annex add, which uses some git ls-files calls, but needs to not use --error-unmatch because of course the files are not known to git. annex.skipunknown is planned to change to default to false in a git-annex release in early 2022. There's a todo for that.
2020-05-28 19:55:17 +00:00
* Added annex.skipunknown git config, that can be set to false to change
the behavior of commands like `git annex get foo*`, to not skip
over files/dirs that are not checked into git and are explicitly listed in
the command line.
* annex.skipunknown is planned to change to default to false in a
git-annex release in early 2022. If you prefer the current behavior,
you can explicitly set it to true.
auto-init autoenable=yes Try to enable special remotes configured with autoenable=yes when git-annex auto-initialization happens in a new clone of an existing repo. Previously, git-annex init had to be explicitly run to enable them. That was a bit of a wart of a special case for users to need to keep in mind. Special remotes cannot display anything when autoenabled this way, to avoid interfering with the output of git-annex query commands. Any error messages will be hidden, and if it fails, nothing is displayed. The user will realize the remote isn't enable when they try to use it, and can run git-annex init manually then to try the autoenable again and see what failed. That seems like a reasonable approach, and it's less complicated than communicating something across a pipe in order to display it as a side message. Other reason not to do that is that, if the first command the user runs is one like git-annex find that has machine readable output, any message about autoenable failing would need to not be displayed anyway. So better to not display a failure message ever, for consistency. (Had to split out Remote.List.Util to avoid an import cycle.)
2020-05-27 15:54:39 +00:00
* Try to enable special remotes configured with autoenable=yes
when git-annex auto-initialization happens in a new clone of an
existing repo. Previously, git-annex init had to be explicitly run to
enable them. Special remotes cannot display anything when autoenabled
this way, to avoid interfering with the output of git-annex query
commands.
export: Added options for json output Just worked, no need to do anything except add the options.
2020-05-26 14:31:10 +00:00
* export: Added options for json output.
import: Added --json-progress Already supported --json, but not that. Also checked all other commands that only support --json, and the only other one that does transfers is fsck (--from), which it did not seem worth adding --json-progress to really.
2020-05-26 15:27:47 +00:00
* import: Added --json-progress.
addurl: Make --preserve-filename also apply when eg a torrent contains multiple files Forgot to remove sanitizeFilePath after adding sanitizeOrPreserveFilePath here.
2020-05-26 14:45:57 +00:00
* addurl: Make --preserve-filename also apply when eg a torrent contains
multiple files.
disable buggy concurrency in Command.Export Fix a crash or potentially not all files being exported when sync -J --content is used with an export remote. Crash as described in fixed bug report. waitForAllRunningCommandActions inserted in several points where all the commandActions started before need to have finished before moving on to the next stage of the export. A race across those points could have maybe resulted in not all files being exported, or a wrong tree being export. For example, changeExport starting up an action like a rename of A to B. Then, with that action still running, fillExport uploading a new A, *before* the rename occurred. That race seems unlikely to have happened. There are some other ones that this also fixes.
2020-05-26 17:44:43 +00:00
* Fix a crash or potentially not all files being exported when
sync -J --content is used with an export remote.
export: Let concurrent transfers be done with -J or annex.jobs Tested working, although I did find this bug in my testing, which also afflicts sync -J to an export remote.
2020-05-26 15:43:58 +00:00
* export: Let concurrent transfers be done with -J or annex.jobs.
improve transfer stages for some commands move --to, copy --to, mirror --to: When concurrency is enabled, run cleanup actions in separate job pool from uploads. transferStages was confusingly named, it's only useful when doing downloads as then the verify actions can be run concurrently with other downloads. For commands that upload, there will be more concurrency from running cleanup actions in a separate job pool. As for sync, I left it using downloadStages although that's not optimal for the part of a sync that uploads. Perhaps it should use the union of both?
2020-05-26 15:55:50 +00:00
* move --to, copy --to, mirror --to: When concurrency is enabled, run
annex.skipunknown with transition plan Added annex.skipunknown git config, that can be set to false to change the behavior of commands like `git annex get foo*`, to not skip over files/dirs that are not checked into git and are explicitly listed in the command line. Significant complexity was needed to handle git-annex add, which uses some git ls-files calls, but needs to not use --error-unmatch because of course the files are not known to git. annex.skipunknown is planned to change to default to false in a git-annex release in early 2022. There's a todo for that.
2020-05-28 19:55:17 +00:00
cleanup actions in separate job pool from uploads.
init warning on stalled lock probe init: If lock probing stalls for a long time (eg a broken NFS server), display a message to let the user know what's taking so long.
2020-06-05 15:03:21 +00:00
* init: If lock probing stalls for a long time (eg a broken NFS server),
display a message to let the user know what's taking so long.
init: When annex.pidlock is set, skip lock probing.
2020-06-05 15:12:16 +00:00
* init: When annex.pidlock is set, skip lock probing.
fix file descriptor leak when importing from a directory special remote that is configured with exporttree=yes
2020-06-05 19:34:43 +00:00
* Fix file descriptor leak when importing from a directory special remote
that is configured with exporttree=yes.
async exception safety for external special remote processes Since an external process can be in the middle of some operation when an async exception is received, it has to be shut down then. Using cleanupProcess will close its IO handles and send it a SIGTERM. If a special remote choses to catch SIGTERM, it's fine for it to do some cleanup then, but until it finishes, git-annex will be blocked waiting for it. If a special remote blocked SIGTERM, it would cause a hang. Mentioned in docs. Also, in passing, fixed a FD leak, it was not closing the error handle when shutting down the external. In practice that didn't matter before because it was only run when git-annex was itself shutting down, but now that it can run on exception, it would have been a problem.
2020-06-09 16:13:06 +00:00
* Note that external special remote programs should not block SIGINT or
SIGTERM.
defer cleaning keys db of old data Avoid creating the keys database during init when there are no unlocked files, to prevent init failing when sqlite does not work in the filesystem.
2020-06-11 19:40:13 +00:00
* Avoid creating the keys database during init when there are no unlocked
files, to prevent init failing when sqlite does not work in the
filesystem.
confirmed fix
2020-06-17 16:12:41 +00:00
* import: Avoid using some strange names for temporary keys,
which broke importing from a directory special remote onto a vfat
filesystem.
S3: The REDUCED_REDUNDANCY storage class is no longer cheaper So stop documenting it, and stop offering it as a choice in the assistant. Removed the code that parses it into S3.ReducedRedundancy, because S3.OtherStorageClass with the value will work just the same and avoids a special case for a deprecated this.
2020-06-16 16:04:29 +00:00
* S3: The REDUCED_REDUNDANCY storage class is no longer cheaper so
stop documenting it, and stop offering it as a choice in the assistant.
local git remote autoinit improvements * Improve display of problems auto-initializing or upgrading local git remotes. * When a local git remote cannot be initialized because it has no git-annex branch or a .noannex file, avoid displaying a message about it.
2020-06-16 17:24:00 +00:00
* Improve display of problems auto-initializing or upgrading local git
remotes.
* When a local git remote cannot be initialized because it has no
git-annex branch or a .noannex file, avoid displaying a message about it.
checkpresentkey: fix behavior to match documentation checkpresentkey: When no remote is specified, try all remotes, not only ones that the location log says contain the key. This is what the documentation has always said it did. Still try the logged remotes first, because they are far more likely to have the key.
2020-06-16 17:54:26 +00:00
* checkpresentkey: When no remote is specified, try all remotes, not
only ones that the location log says contain the key. This is what
the documentation has always said it did.
analysis of regression and fix related less serious regression
2020-06-16 19:16:36 +00:00
* Fix regression in external special remote handling: GETCONFIG did not
return a value that was set with SETCONFIG immediately before.
(Regression introduced in version 7.20200202.7)
external: SETCREDS include creds in externalConfigChanges This makes the creds get saved, since only things recorded there will be saved. IIRC, unparsedRemoteConfig was not originally available when I implemented this; now that it is things get a bit simpler. More could probably be simplified, is externalConfigChanges needed at all? This does not entirely fix the bugs though, because creds are only embedded when embedcreds=yes, but not when encryption=pubkey is used without embedcreds=yes.
2020-06-16 21:24:24 +00:00
* Fix bug that made initremote of extrnal special remotes with
embedcreds=yes or gpg encryption not store the creds in the git-annex
branch. git-annex-remote-googledrive one was special remote affected by
this bug.
(Regression introduced in version 7.20200202.7)
fix implicit embedcreds regression Fix bug that made creds not be stored in git when a special remote was initialized with gpg encryption, but without an explicit embedcreds=yes. (Yet nother regression introduced in version 7.20200202.7. 5th so far.)
2020-06-16 21:59:55 +00:00
* Fix bug that made creds not be stored in git when a special remote
was initialized with gpg encryption, but without an explicit
embedcreds=yes.
(Regression introduced in version 7.20200202.7)
fix a annex.pidlock issue That made eg git-annex get of an unlocked file hang until the annex.pidlocktimeout and then fail. This fix should be fully thread safe no matter what else git-annex is doing. Only using runsGitAnnexChildProcess in the one place it's known to be a problem. Could audit for all places where git-annex runs itself as a child and add it to all of them, later.
2020-06-17 19:13:52 +00:00
* Fix a annex.pidlock issue that made eg git-annex get of an unlocked
file hang until the annex.pidlocktimeout and then fail.
export: Added options for json output Just worked, no need to do anything except add the options.
2020-05-26 14:31:10 +00:00
releasing package git-annex version 8.20200617
2020-06-17 19:59:34 +00:00
-- Joey Hess <id@joeyh.name> Wed, 17 Jun 2020 15:58:59 -0400
export: Added options for json output Just worked, no need to do anything except add the options.
2020-05-26 14:31:10 +00:00
releasing package git-annex version 8.20200522
2020-05-22 16:07:59 +00:00
git-annex (8.20200522) upstream; urgency=medium
avoid beware of the leopard situation * Display a warning message when a remote uses a protocol, such as git://, that git-annex does not support. Silently skipping such a remote was confusing behavior. It sets annex-ignore, so the warning is only displayed once. * Also display a warning message when a remote, without a known uuid, is located in a directory that does not currently exist, to avoid silently skipping such a remote. This is a bit more debatable, since git-annex get will say, try making repository available. And since it does not set annex-ignore, the warning will be displayed repeatedly. It's also an extreme edge case, I don't think I've ever seen it happen in real life.
2020-05-04 17:01:11 +00:00
releasing package git-annex version 8.20200522
2020-05-22 16:07:59 +00:00
* Fix bug that made enableremote of S3 and webdav remotes, that
have embedcreds=yes, fail to set up the embedded creds, so accessing
the remotes failed. (Regression introduced in version 7.20200202.7)
addurl --preserve-filename and a few related changes * addurl --preserve-filename: New option, uses server-provided filename without any sanitization, but with some security checking. Not yet implemented for remotes other than the web. * addurl, importfeed: Avoid adding filenames with leading '.', instead it will be replaced with '_'. This might be considered a security fix, but a CVE seems unwattanted. It was possible for addurl to create a dotfile, which could change behavior of some program. It was also possible for a web server to say the file name was ".git" or "foo/.git". That would not overrwrite the .git directory, but would cause addurl to fail; of course git won't add "foo/.git". sanitizeFilePath is too opinionated to remain in Utility, so moved it. The changes to mkSafeFilePath are because it used sanitizeFilePath. In particular: isDrive will never succeed, because "c:" gets munged to "c_" ".." gets sanitized now ".git" gets sanitized now It will never be null, because sanitizeFilePath keeps the length the same, and splitDirectories never returns a null path. Also, on the off chance a web server suggests a filename of "", ignore that, rather than trying to save to such a filename, which would fail in some way.
2020-05-08 20:09:29 +00:00
* addurl, importfeed: Avoid adding filenames with leading '.', instead
it will be replaced with '_'.
addurl, importfeed: Allow '-' in filenames, as long as it's not the first character
2020-05-11 17:50:49 +00:00
* addurl, importfeed: Allow '-' in filenames, as long as it's not the
first character.
addurl --preserve-filename and a few related changes * addurl --preserve-filename: New option, uses server-provided filename without any sanitization, but with some security checking. Not yet implemented for remotes other than the web. * addurl, importfeed: Avoid adding filenames with leading '.', instead it will be replaced with '_'. This might be considered a security fix, but a CVE seems unwattanted. It was possible for addurl to create a dotfile, which could change behavior of some program. It was also possible for a web server to say the file name was ".git" or "foo/.git". That would not overrwrite the .git directory, but would cause addurl to fail; of course git won't add "foo/.git". sanitizeFilePath is too opinionated to remain in Utility, so moved it. The changes to mkSafeFilePath are because it used sanitizeFilePath. In particular: isDrive will never succeed, because "c:" gets munged to "c_" ".." gets sanitized now ".git" gets sanitized now It will never be null, because sanitizeFilePath keeps the length the same, and splitDirectories never returns a null path. Also, on the off chance a web server suggests a filename of "", ignore that, rather than trying to save to such a filename, which would fail in some way.
2020-05-08 20:09:29 +00:00
* addurl --preserve-filename: New option, uses server-provided filename
addurl, importfeed: Allow '-' in filenames, as long as it's not the first character
2020-05-11 17:50:49 +00:00
without any sanitization, but will fail if the filename has an obvious
security problem like using an escape sequence or trying to escape
the current directory.
releasing package git-annex version 8.20200522
2020-05-22 16:07:59 +00:00
* whereis: Added --format option.
* S3: Support signature=v4, to use S3 Signature Version 4.
Some S3 services seem to require v4, while others may only
support v2, which remains the default.
* upgrade: When upgrade fails due to an exception, display it.
* repair: Improve fetching from a remote with an url in host:path format.
* git-lfs repos that encrypt the annexed content but not the git repo
only need --force passed to initremote, allow enableremote and
autoenable of such remotes without forcing again.
* When accessing a remote fails, always display a reason why.
* Display a warning message when a remote uses a protocol, such as
git://, that git-annex does not support. Silently skipping such a
remote was confusing behavior.
* Also display a warning message when a remote, without a known uuid,
is located in a directory that does not currently exist, to avoid
silently skipping such a remote.
* sync: Avoid an ugly error message when nothing has been committed to
master yet and there is a synced master branch to merge from.
Display a warning message when asked to operate on a file inside a directory that's a symbolic link to elsewhere This relicates git's behavior. It adds a few stat calls for the command line parameters, so there is some minor slowdown, but even with thousands of parameters it will not be very noticable, and git does the same statting in similar circumstances. Note that this does not prevent eg "git annex add symlink"; the symlink will be added to git as usual. And "git annex find symlink" will silently list nothing as well. It's only "symlink/foo" or "subdir/symlink/foo" that triggers the warning.
2020-05-11 19:03:35 +00:00
* Display a warning message when asked to operate on a file inside a
directory that's a symbolic link to elsewhere.
Support building with tasty-1.3 This commit was sponsored by Ethan Aubin.
2020-05-21 19:26:32 +00:00
* Support building with tasty-1.3.
avoid beware of the leopard situation * Display a warning message when a remote uses a protocol, such as git://, that git-annex does not support. Silently skipping such a remote was confusing behavior. It sets annex-ignore, so the warning is only displayed once. * Also display a warning message when a remote, without a known uuid, is located in a directory that does not currently exist, to avoid silently skipping such a remote. This is a bit more debatable, since git-annex get will say, try making repository available. And since it does not set annex-ignore, the warning will be displayed repeatedly. It's also an extreme edge case, I don't think I've ever seen it happen in real life.
2020-05-04 17:01:11 +00:00
releasing package git-annex version 8.20200522
2020-05-22 16:07:59 +00:00
-- Joey Hess <id@joeyh.name> Fri, 22 May 2020 12:05:01 -0400
avoid beware of the leopard situation * Display a warning message when a remote uses a protocol, such as git://, that git-annex does not support. Silently skipping such a remote was confusing behavior. It sets annex-ignore, so the warning is only displayed once. * Also display a warning message when a remote, without a known uuid, is located in a directory that does not currently exist, to avoid silently skipping such a remote. This is a bit more debatable, since git-annex get will say, try making repository available. And since it does not set annex-ignore, the warning will be displayed repeatedly. It's also an extreme edge case, I don't think I've ever seen it happen in real life.
2020-05-04 17:01:11 +00:00
releasing package git-annex version 8.20200501
2020-05-01 21:41:36 +00:00
git-annex (8.20200501) upstream; urgency=medium
use programPath consistently, not readProgramFile Improve git-annex's ability to find the path to its program, especially when it needs to run itself in another repo to upgrade it. Some parts of the code used readProgramFile, probably because I forgot that programPath exists. I noticed this when a git-annex auto-upgrade failed because it was running git-annex upgrade --autoonly, but the code to run git-annex used readProgramFile, which happened to point to an older build of git-annex.
2020-03-30 20:03:44 +00:00
* Improve git-annex's ability to find the path to its program,
especially when it needs to run itself in another repo to upgrade it.
adb: Better messages when the adb command is not installed After a user completely ignored the display of the exception probably because it didn't make sense.. This does make it a little bit slower since it checks adb is in path each time before running it. Also, it might display a lot of warnings about it not being installed. This commit was sponsored by Ilya Shlyakhter on Patreon.
2020-04-02 14:46:46 +00:00
* adb: Better messages when the adb command is not installed.
optimise catfile interface with ByteString and Attoparsec Around 3% total speedup. Profiling git annex find --not --in web, it's now bytestring end-to-end, and there is only a little added overhead in eg accessing the Annex state MVar (3%). The rest of the runtime is spent reading symlinks, and in attoparsec. This feels like the end of the optimisation road, without a major change like caching information for faster queries.
2020-04-10 18:03:40 +00:00
* Sped up query commands that read the git-annex branch by around 9%.
changelog for ByteString Ref
2020-04-08 18:10:29 +00:00
* Various speed improvements gained by using ByteStrings for git refs and
shas.
Fix a potential failure to parse git config Git has an obnoxious special case in git config, a line "foo" is the same as "foo = true". That means there is no way to examine the output of git config and tell if it was run with --null or not, since a "foo" in the first line could be such a boolean, or could be followed by its value on the next line if --null were used. So, rather than trying to do such a detection, track the style of config at all the points where it's generated.
2020-04-13 17:05:41 +00:00
* Fix a potential failure to parse git config.
support boolean git configs that are represented by the name of the setting with no value Eg"core.bare" is the same as "core.bare = true". Note that git treats "core.bare =" the same as "core.bare = false", so the code had to become more complicated in order to treat the absense of a value differently than an empty value. Ugh.
2020-04-13 17:35:22 +00:00
* Support boolean git configs that are represented by the name of the
setting with no value, eg "core.bare" is the same as "core.bare = true".
When parsing git configs, support all the documented ways to write true and false, including "yes", "on", "1", etc. This change does impact git-annex config eg "git annex config --set annex.addunlocked on" will store "on" and new git-annex will understand that value, while old git-annex will error: git-annex: bad annex.addunlocked configuration in git annex config: Parse failure: near "on" That seems acceptable. Not special remote configs that are only documented as =true or =false however. Having git-annex support other values for those would break backwards compatability when used with old versions of git-annex. And older versions ignore invalid special remote configs.. That would not be a good combination.
2020-04-13 17:45:40 +00:00
* When parsing git configs, support all the documented ways to write
true and false, including "yes", "on", "1", etc.
fix absolute filenames fed into --batch and git-annex info
2020-04-15 20:04:05 +00:00
* Fix --batch commands (and git-annex info) to accept absolute filenames
for unlocked files, which already worked for locked files.
fix a thundering herd problem Avoid repeatedly opening keys db when accessing a local git remote and -J is used. What was happening was that Remote.Git.onLocal created a new annex state as each thread started up. The way the MVar was used did not prevent that. And that, in turn, led to repeated opening of the keys db, as well as probably other extra work or resource use. Also managed to get rid of Annex.remoteannexstate, and it turned out there was an unncessary Maybe in the keysdbhandle, since the handle starts out closed.
2020-04-17 21:09:29 +00:00
* Avoid repeatedly opening keys db when accessing a local git remote
and -J is used.
cat-file resource pool Avoid running a large number of git cat-file child processes when run with a large -J value. This implementation takes care to avoid adding any overhead to git-annex when run without -J. When run with -J, there is a small bit of added overhead, to manipulate the resource pool. That optimisation added a fair bit of complexity.
2020-04-20 17:53:27 +00:00
* Avoid running a large number of git cat-file child processes when run
with a large -J value.
check-ignore resource pool Much like check-attr before.
2020-04-21 15:20:10 +00:00
* Avoid running with more git check-attr and check-ignore processes than
there are CPU cores when run with a large -J value.
fix bug involving local git remote and out of date location log get --from, move --from: When used with a local git remote, these used to silently skip files that the location log thought were present on the remote, when the remote actually no longer contained them. Since that behavior could be surprising, now instead display a warning. I got very confused when I encountered this behavior, since it was silently skipping a file I needed that whereis said was on the remote. get without --from already displayed a "unable to access these remotes" message, which while a bit misleading in that the remote is likely accessible, but just doesn't contain the file, at least indicated something went wrong. Having get --from display a warning makes it in line with get w/o --from, so seems certianly ok. It might be there are situations where move --from is used, on eg a whole directory, and the user only wants to move whatever is present in the remote, and is perfectly ok with files that are not present being skipped. So I'm less sure about the new warning being ok there. OTOH, only local git remotes avoiding displaying a warning in that case too, so this just brings them into line with other remotes. (Also note that this makes it a little bit faster when dealing with a lot of files, since it avoids a redundant stat of the file.)
2020-04-21 16:36:58 +00:00
* get --from, move --from: When used with a local git remote, these used
to silently skip files that the location log thought were present on the
remote, when the remote actually no longer contained them. Since that
behavior could be surprising, now instead display a warning.
stop using remote.name.annex-readonly for two distinct things
2020-04-23 18:56:03 +00:00
* external special remotes: remote.name.annex-readonly=true no longer
disables running the external special remote program. Instead, it just
makes the remote operate in a readonly mode, same as any remote.
To disable running the external special remote program, now need to set
remote.name.annex-externaltype=readonly. That is done when
git-annex enableremote is passed readonly=true.
external: stop storing readonly=true in remote.log readonly=true is used to make an external special remote that does not need the external program to be installed. It was stored in the remote.log by default, and so every time it was specified in an enableremote or initremote, whatever value was used became the new default for subsequent enableremotes of that remote. That was surprising, and I consider it to be a bug. It does not make much sense to pass it to initremote because then how would you populate that remote with anything? You would have to enableremote elsewhere, and store content there. I'm assuming nobody used it that way. Someone might rely on passing it to enableremote once, and then that being inherited in other clones. But that is not how it's documented to be used. It is barely documented in git-annex at all, only in the external special remote protocol, and the documentation there says to "Document that this external special remote can be used in readonly mode." (by the user of it passing readonly=true to enableremote). The one external special remote that I know of that does document that is <https://github.com/bgilbert/gcsannex> (the one that motivated adding it). That one's docs do say to pass it to enableremote. So, it seemed safe to make this behavior change. If someone was in fact relying on one of those behaviors, all their current repos will still work as they configured them (although they will need to deal with the related change in 9f3c2dfedae7ec840365f9578763209abae1005c). In new clones, they will find enableremote fails, complaining the external program is not in path. An easy enough problem to recover from.
2020-04-23 18:59:38 +00:00
* Stop storing readonly=true in remote.log of external special remotes;
it is a local setting only.
sync: When some remotes to sync with are specified, and --fast is too, pick the lowest cost of the specified remotes Do not sync with a faster remote that was not specified. That old behavior was only documented in the changelog, and was certianly surprising. It also meant adding --fast made it slower..
2020-04-23 20:08:45 +00:00
* sync: When some remotes to sync with are specified, and --fast is too,
pick the lowest cost of the specified remotes, do not sync with a
faster remote that was not specified.
addurl --fast error message improvement addurl: When run with --fast on an url that annex.security.allowed-ip-addresses prevents accessing, display a more useful message. (Also importfeed --fast potentially.)
2020-04-27 17:48:14 +00:00
* addurl: When run with --fast on an url that
annex.security.allowed-ip-addresses prevents accessing, display
a more useful message.
support required groupwanted When the required content is set to "groupwanted", use whatever expression has been set in groupwanted as the required content of the repo, similar to how setting required content to "standard" already worked.
2020-04-28 17:31:26 +00:00
* When the required content is set to "groupwanted", use whatever
expression has been set in groupwanted as the required content of the
repo, similar to how setting required content to "standard" already
worked.
Avoid a test suite failure when the environment does not let gpg be tested Due to eg, too long a path to the agent socket, caused by running gpg in a container where /run is not mounted, and/or some other gpg behavior like unnecessarily making relative paths to its home directory absolute.
2020-04-28 19:47:23 +00:00
* Avoid a test suite failure when the environment does not let gpg be
tested due to eg, too long a path to the agent socket.
wip This does not compile, and I hit a bad dead end. Wah.
2020-04-29 19:48:09 +00:00
* test: Include testremote tests, run on a directory special remote.
use programPath consistently, not readProgramFile Improve git-annex's ability to find the path to its program, especially when it needs to run itself in another repo to upgrade it. Some parts of the code used readProgramFile, probably because I forgot that programPath exists. I noticed this when a git-annex auto-upgrade failed because it was running git-annex upgrade --autoonly, but the code to run git-annex used readProgramFile, which happened to point to an older build of git-annex.
2020-03-30 20:03:44 +00:00
releasing package git-annex version 8.20200501
2020-05-01 21:41:36 +00:00
-- Joey Hess <id@joeyh.name> Fri, 01 May 2020 13:09:24 -0400
use programPath consistently, not readProgramFile Improve git-annex's ability to find the path to its program, especially when it needs to run itself in another repo to upgrade it. Some parts of the code used readProgramFile, probably because I forgot that programPath exists. I noticed this when a git-annex auto-upgrade failed because it was running git-annex upgrade --autoonly, but the code to run git-annex used readProgramFile, which happened to point to an older build of git-annex.
2020-03-30 20:03:44 +00:00
releasing package git-annex version 8.20200330
2020-03-30 17:46:24 +00:00
git-annex (8.20200330) upstream; urgency=medium
Fix a minor bug that caused options provided with -c to be passed multiple times to git.
2020-03-16 17:06:44 +00:00
fsck: Fix reversion in 8.20200226 that made it incorrectly warn that hashed keys with an extension should be upgraded.
2020-03-20 17:09:16 +00:00
* fsck: Fix reversion in 8.20200226 that made it incorrectly warn
that hashed keys with an extension should be upgraded.
changelog for Kyle's fix
2020-03-26 17:12:49 +00:00
* add --force-small: Fix a bug that, when adding a symbolic link,
checked in the content of the file the symlink pointed to.
patch applied
2020-03-26 19:18:47 +00:00
Thanks, Kyle Meyer for the patch.
changelog for Kyle's other fix, and close bug
2020-03-26 17:18:41 +00:00
* add --force-small: Fix failure when passed a modified submodule.
patch applied
2020-03-26 19:18:47 +00:00
Thanks, Kyle Meyer for the patch.
* When syncing changes back from an adjusted branch to the basis branch,
include changes to submodules.
Thanks, Kyle Meyer for the patch.
releasing package git-annex version 8.20200330
2020-03-30 17:46:24 +00:00
* webdav: Made exporttree remotes faster by caching connection to the
server.
* Fix a minor bug that caused options provided with -c to be passed
multiple times to git.
Fix a minor bug that caused options provided with -c to be passed multiple times to git.
2020-03-16 17:06:44 +00:00
releasing package git-annex version 8.20200330
2020-03-30 17:46:24 +00:00
-- Joey Hess <id@joeyh.name> Mon, 30 Mar 2020 12:14:50 -0400
Fix a minor bug that caused options provided with -c to be passed multiple times to git.
2020-03-16 17:06:44 +00:00
releasing package git-annex version 8.20200309
2020-03-09 21:08:16 +00:00
git-annex (8.20200309) upstream; urgency=medium
Bugfix: Don't ignore --debug when it is followed by -c
2020-02-27 04:52:37 +00:00
Fix bug that caused unlocked annexed dotfiles to be added to git by the smudge filter when annex.dotfiles was not set.
2020-03-09 18:20:02 +00:00
* Fix bug that caused unlocked annexed dotfiles to be added to git by the
smudge filter when annex.dotfiles was not set.
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
* Upgrade other repos than the current one by running git-annex upgrade
inside them, which avoids problems with upgrade code making assumptions
that the cwd will be inside the repo being upgraded. In particular,
this fixes a problem where upgrading a v7 repo to v8 caused an ugly
git error message.
Fix upgrade failure when a file has been deleted from the working tree
2020-03-09 20:59:18 +00:00
* Fix upgrade failure when a file has been deleted from the working tree.
releasing package git-annex version 8.20200309
2020-03-09 21:08:16 +00:00
* Fix regression 1 month ago that prevented external special remotes from
using GETCONFIG to query values like "name".
completed the createDirectoryIfMissing conversion Remaining calls in the assistant and Annex.Ssh have been audited and are ok.
2020-03-06 16:52:20 +00:00
* Improve behavior when a directory git-annex is writing to gets
unmounted. Previously it could in some cases re-create the mount point
and directory tree, and even write object contents to the wrong disk.
releasing package git-annex version 8.20200309
2020-03-09 21:08:16 +00:00
* Don't ignore --debug when it is followed by -c.
catch whereisKey exception and provide error messages when external programs neglect to * whereis: If a remote fails to report on urls where a key is located, display a warning, rather than giving up and not displaying any information. * When external special remotes fail but neglect to provide an error message, say what request failed, which is better than displaying an empty error message to the user.
2020-02-27 18:09:18 +00:00
* whereis: If a remote fails to report on urls where a key
is located, display a warning, rather than giving up and not displaying
any information.
* When external special remotes fail but neglect to provide an error
message, say what request failed, which is better than displaying an
empty error message to the user.
releasing package git-annex version 8.20200309
2020-03-09 21:08:16 +00:00
* git-annex config: Only allow configs be set that are ones git-annex
actually supports reading from repo-global config, to avoid confusion.
* Avoid converting .git file in a worktree or submodule to a symlink
when the repository is not a git-annex repository.
* Linux standalone: Use md5sum to shorten paths in .cache/git-annex/locales
changelog and minor style
2020-02-28 16:57:55 +00:00
* Fix build with ghc 8.8 (MonadFail)
Thanks, Peter Simons
stack.yaml: Updated to lts-14.27.
2020-03-02 17:25:51 +00:00
* stack.yaml: Updated to lts-14.27.
Bugfix: Don't ignore --debug when it is followed by -c
2020-02-27 04:52:37 +00:00
releasing package git-annex version 8.20200309
2020-03-09 21:08:16 +00:00
-- Joey Hess <id@joeyh.name> Mon, 09 Mar 2020 17:04:08 -0400
Bugfix: Don't ignore --debug when it is followed by -c
2020-02-27 04:52:37 +00:00
releasing package git-annex version 8.20200226
2020-02-26 23:09:23 +00:00
git-annex (8.20200226) upstream; urgency=medium
benchmark: Add --databases to benchmark sqlite databases Rescued from commit 11d6e2e260d70ba99e35464c19c2b2772ce9efaa which removed db benchmarks in favor of benchmarking arbitrary git-annex commands. Which is nice and general, but microbenchmarks are useful too.
2019-10-29 19:16:15 +00:00
Merge branch 'master' into sqlite
2019-11-07 17:20:04 +00:00
* New v8 repository version.
tighten up description of v8 upgrade
2020-02-19 19:07:48 +00:00
* v7 upgrades automatically to v8. The upgrade deletes old sqlite
databases, which may cause git-annex to need to do extra work to
regenerate the databases or due to not having the information from the
old databases available. Two notable cases are interrupted incremental
fscks and interrupted exports, both of which will restart from the
beginning.
* Improved serialization of filenames and keys to the sqlite databases,
mention speedup
2020-02-19 19:09:00 +00:00
avoiding encoding problems and speeding up operations on them.
update for things already fixed in this branch
2019-10-30 17:50:16 +00:00
* Add some missing indexes to sqlite databases. This will speed up
some things involving export and import remotes, and git-annex smudge.
Merge branch 'master' into sqlite
2019-11-07 17:20:04 +00:00
Microbenchmarks show around 10-25% speedup of sqlite database operations.
annex.dotfiles The git add behavior changes could be avoided if it turns out to be really annoying, but then it would need to behave the old way when annex.dotfiles=false and the new way when annex.dotfiles=true. I'd rather not have the config option result in such divergent behavior as `git annex add .` skipping a dotfile (old) vs adding to annex (new). Note that the assistant always adds dotfiles to the annex. This is surprising, but not new behavior. Might be worth making it also honor annex.dotfiles, but I wonder if perhaps some user somewhere uses it and keeps large files in a directory that happens to begin with a dot. Since dotfiles and dotdirs are a unix culture thing, and the assistant users may not be part of that culture, it seems best to keep its current behavior for now.
2019-12-26 20:24:40 +00:00
* add: When adding a whole directory, any dotfiles found in it will
not be skipped, but will be added to git by default. This behavior
can be configured with annex.dotfiles.
tighten up docs of dotfiles changes
2020-02-19 18:29:50 +00:00
* add: Removed special case for explicitly passing dotfiles,
that no longer adds them to the annex, but to git. This behavior
can be configured with annex.dotfiles.
annex.dotfiles The git add behavior changes could be avoided if it turns out to be really annoying, but then it would need to behave the old way when annex.dotfiles=false and the new way when annex.dotfiles=true. I'd rather not have the config option result in such divergent behavior as `git annex add .` skipping a dotfile (old) vs adding to annex (new). Note that the assistant always adds dotfiles to the annex. This is surprising, but not new behavior. Might be worth making it also honor annex.dotfiles, but I wonder if perhaps some user somewhere uses it and keeps large files in a directory that happens to begin with a dot. Since dotfiles and dotdirs are a unix culture thing, and the assistant users may not be part of that culture, it seems best to keep its current behavior for now.
2019-12-26 20:24:40 +00:00
* add: Removed the --include-dotfiles option.
automate remote.name.skipFetchAll initremote, enableremote: Set remote.name.skipFetchAll when the remote cannot be fetched from by git, so git fetch --all will not try to use it.
2020-02-19 17:58:26 +00:00
* initremote, enableremote: Set remote.name.skipFetchAll when
the remote cannot be fetched from by git, so git fetch --all
will not try to use it.
handle keys with extensions consistently in all locales Fix some cases where handling of keys with extensions varied depending on the locale. A filename with a unicode extension would before generate a key with an extension in a unicode locale, but not in LANG=C, because the extension was not all alphanumeric. Also the the length of the extension could be counted differently depending on the locale. In a non-unicode locale, git-annex migrate would see that the extension was not all alphanumeric and want to "upgrade" it. Now that doesn't happen. As far as backwards compatability, this does mean that unicode extensions are counted by the number of bytes, not number of characters. So, if someone is using unicode extensions, they may find git-annex stops using them when adding files, because their extensions are too long. Keys already in their repo with the "too long" extensions will still work though, so this only prevents adding the same content with the same extension generating the same key. Documented this by documenting that annex.maxextensionlength is a number of bytes. Also, if a filename has an extension that is not valid utf-8 and the locale is utf-8, the extension will be allowed now, and an old git-annex, in the same locale would not, and would also want to "upgrade" that.
2020-02-20 21:18:59 +00:00
* Fix some cases where handling of keys with extensions varied depending
on the locale.
* annex.maxextensionlength used to be the number of characters, not
bytes, when in a utf-8 locale. It's now always the number of bytes.
releasing package git-annex version 8.20200226
2020-02-26 23:09:23 +00:00
* Extended annex.security.allowed-ip-addresses to let specific ports
of an IP address to be used, while denying use of other ports.
jump right to v8, don't stop part way * init --version: When the version given is one that automatically upgrades to a newer version, use the newer version instead. * Auto upgrades from older repo versions, like v5, now jump right to v8.
2020-02-24 17:21:00 +00:00
* init --version: When the version given is one that automatically
upgrades to a newer version, use the newer version instead.
* Auto upgrades from older repo versions, like v5, now jump right to v8.
releasing package git-annex version 8.20200226
2020-02-26 23:09:23 +00:00
* Makefile: Support newer versions of cabal that use the new-build system.
Merge branch 'v7'
2020-02-26 22:15:18 +00:00
releasing package git-annex version 8.20200226
2020-02-26 23:09:23 +00:00
-- Joey Hess <id@joeyh.name> Wed, 26 Feb 2020 18:49:58 -0400
Merge branch 'v7'
2020-02-26 22:15:18 +00:00
fix encryption of content to gcrypt and git-lfs Fix serious regression in gcrypt and encrypted git-lfs remotes. Since version 7.20200202.7, git-annex incorrectly stored content on those remotes without encrypting it. Problem was, Remote.Git enumerates all git remotes, including git-lfs and gcrypt. It then dispatches to those. So, Remote.List used the RemoteConfigParser from Remote.Git, instead of from git-lfs or gcrypt, and that parser does not know about encryption fields, so did not include them in the ParsedRemoteConfig. (Also didn't include other fields specific to those remotes, perhaps chunking etc also didn't get through.) To fix, had to move RemoteConfig parsing down into the generate methods of each remote, rather than doing it in Remote.List. And a consequence of that was that ParsedRemoteConfig had to change to include the RemoteConfig that got parsed, so that testremote can generate a new remote based on an existing remote. (I would have rather fixed this just inside Remote.Git, but that was not practical, at least not w/o re-doing work that Remote.List already did. Big ugly mostly mechanical patch seemed preferable to making git-annex slower.)
2020-02-26 21:20:56 +00:00
git-annex (7.20200226) upstream; urgency=high
* Fix serious regression in gcrypt and encrypted git-lfs remotes.
Since version 7.20200202.7, git-annex incorrectly stored content
on those remotes without encrypting it.
If your remotes are affected, you will want to make sure to delete
any content that git-annex has stored on them that is not encrypted!
close already fixed bug and document it in the changelog for the release where it was fixed
2020-03-02 17:47:47 +00:00
* initremote: Fix regression in parsing the exporttree= parameter.
info: Fix display of the encryption value. (Some debugging junk had crept in.)
2020-02-26 19:02:23 +00:00
* info: Fix display of the encryption value.
(Some debugging junk had crept in.)
Bugfix to getting content from an export remote with -J, when the export database was not yet populated. (cherry picked from commit e520341500dce9f708a4de674f19047c5d325744)
2020-02-26 18:57:29 +00:00
* Bugfix to getting content from an export remote with -J, when the
export database was not yet populated.
Bugfix: export --tracking (a deprecated option) set annex-annex-tracking-branch, instead of annex-tracking-branch. (cherry picked from commit a3a674d15be01a2ee9d27339e9eeb88b21f559dc)
2020-02-19 17:34:24 +00:00
* Bugfix: export --tracking (a deprecated option) set
annex-annex-tracking-branch, instead of annex-tracking-branch.
Bugfix: export --tracking (a deprecated option) set annex-annex-tracking-branch, instead of annex-tracking-branch.
2020-02-19 17:34:24 +00:00
fix encryption of content to gcrypt and git-lfs Fix serious regression in gcrypt and encrypted git-lfs remotes. Since version 7.20200202.7, git-annex incorrectly stored content on those remotes without encrypting it. Problem was, Remote.Git enumerates all git remotes, including git-lfs and gcrypt. It then dispatches to those. So, Remote.List used the RemoteConfigParser from Remote.Git, instead of from git-lfs or gcrypt, and that parser does not know about encryption fields, so did not include them in the ParsedRemoteConfig. (Also didn't include other fields specific to those remotes, perhaps chunking etc also didn't get through.) To fix, had to move RemoteConfig parsing down into the generate methods of each remote, rather than doing it in Remote.List. And a consequence of that was that ParsedRemoteConfig had to change to include the RemoteConfig that got parsed, so that testremote can generate a new remote based on an existing remote. (I would have rather fixed this just inside Remote.Git, but that was not practical, at least not w/o re-doing work that Remote.List already did. Big ugly mostly mechanical patch seemed preferable to making git-annex slower.)
2020-02-26 21:20:56 +00:00
-- Joey Hess <id@joeyh.name> Wed, 26 Feb 2020 17:18:16 -0400
Bugfix: export --tracking (a deprecated option) set annex-annex-tracking-branch, instead of annex-tracking-branch.
2020-02-19 17:34:24 +00:00
releasing package git-annex version 7.20200219
2020-02-19 16:45:30 +00:00
git-annex (7.20200219) upstream; urgency=medium
annex.sshcaching warning improvement and allow overridding build time default * When git-annex is built with a ssh that does not support ssh connection caching, default annex.sshcaching to false, but let the user override it. * Improve warning messages further when ssh connection caching cannot be used, to clearly state why.
2020-02-14 18:21:03 +00:00
sync --only-annex and annex.synconlyannex * Added sync --only-annex, which syncs the git-annex branch and annexed content but leaves managing the other git branches up to you. * Added annex.synconlyannex git config setting, which can also be set with git-annex config to configure sync in all clones of the repo. Use case is then the user has their own git workflow, and wants to use git-annex without disrupting that, so they sync --only-annex to get the git-annex stuff in sync in addition to their usual git workflow. When annex.synconlyannex is set, --not-only-annex can be used to override it. It's not entirely clear what --only-annex --commit or --only-annex --push should do, and I left that combination not documented because I don't know if I might want to change the current behavior, which is that such options do not override the --only-annex. My gut feeling is that there is no good reasons to use such combinations; if you want to use your own git workflow, you'll be doing your own committing and pulling and pushing. A subtle question is, how should import/export special remotes be handled? Importing updates their remote tracking branch and merges it into master. If --only-annex prevented that git branch stuff, then it would prevent exporting to the special remote, in the case where it has changes that were not imported yet, because there would be a unresolved conflict. I decided that it's best to treat the fact that there's a remote tracking branch for import/export as an implementation detail in this case. The more important thing is that an import/export special remote is entirely annexed content, and so it makes a lot of sense that --only-annex will still sync with it.
2020-02-17 19:19:58 +00:00
* Added sync --only-annex, which syncs the git-annex branch and annexed
content but leaves managing the other git branches up to you.
* Added annex.synconlyannex git config setting, which can also be set with
git-annex config to configure sync in all clones of the repo.
fsck --from remote -J concurrency bug fsck --from remote: Fix a concurrency bug that could make it incorrectly detect that content in the remote is corrupt, and remove it, resulting in data loss.
2020-02-14 18:52:15 +00:00
* fsck --from remote: Fix a concurrency bug that could make it incorrectly
detect that content in the remote is corrupt, and remove it, resulting in
data loss.
annex.sshcaching warning improvement and allow overridding build time default * When git-annex is built with a ssh that does not support ssh connection caching, default annex.sshcaching to false, but let the user override it. * Improve warning messages further when ssh connection caching cannot be used, to clearly state why.
2020-02-14 18:21:03 +00:00
* When git-annex is built with a ssh that does not support ssh connection
caching, default annex.sshcaching to false, but let the user override it.
* Improve warning messages further when ssh connection caching cannot
be used, to clearly state why.
Avoid throwing fatal errors when asked to write to a readonly git remote on http Test suite found one of them, looking for giveup turned up several more.
2020-02-14 18:38:13 +00:00
* Avoid throwing fatal errors when asked to write to a readonly
git remote on http.
annex.tune.branchhash1=true bugfix Fix support for repositories tuned with annex.tune.branchhash1=true, including --all not working and git-annex log not displaying anything for annexed files.
2020-02-14 19:22:48 +00:00
* Fix support for repositories tuned with annex.tune.branchhash1=true,
including --all not working and git-annex log not displaying anything
for annexed files.
annex.sshcaching warning improvement and allow overridding build time default * When git-annex is built with a ssh that does not support ssh connection caching, default annex.sshcaching to false, but let the user override it. * Improve warning messages further when ssh connection caching cannot be used, to clearly state why.
2020-02-14 18:21:03 +00:00
releasing package git-annex version 7.20200219
2020-02-19 16:45:30 +00:00
-- Joey Hess <id@joeyh.name> Wed, 19 Feb 2020 12:44:43 -0400
annex.sshcaching warning improvement and allow overridding build time default * When git-annex is built with a ssh that does not support ssh connection caching, default annex.sshcaching to false, but let the user override it. * Improve warning messages further when ssh connection caching cannot be used, to clearly state why.
2020-02-14 18:21:03 +00:00
releasing package git-annex version 7.20200204
2020-02-04 18:33:03 +00:00
git-annex (7.20200204) upstream; urgency=medium
Fix build with newest version of persistent-template. This is untested because of rain, also I am operating from truncated copiler error messages in a bug report that also doesn't mention what the library version is. Still, it should work. May break builds with old ghc, in particular DerivingStrategies is I think fairly new? The pragmas could be ifdefed if necessary. Works with ghc 8.6.5.
2020-02-04 16:03:30 +00:00
ifdef persistent-template 2.8.0 fixes The i386ancient build has a ghc too old for these extensions. Build with persistent-template 2.8.0 tested.
2020-02-04 17:53:00 +00:00
* Fix build with persistent-template 2.8.0.
Makefile: Really move the fish completion to the vendor_completions.d directory.
2020-02-04 16:10:09 +00:00
* Makefile: Really move the fish completion to the
vendor_completions.d directory.
Fix build with newest version of persistent-template. This is untested because of rain, also I am operating from truncated copiler error messages in a bug report that also doesn't mention what the library version is. Still, it should work. May break builds with old ghc, in particular DerivingStrategies is I think fairly new? The pragmas could be ifdefed if necessary. Works with ghc 8.6.5.
2020-02-04 16:03:30 +00:00
releasing package git-annex version 7.20200204
2020-02-04 18:33:03 +00:00
-- Joey Hess <id@joeyh.name> Tue, 04 Feb 2020 14:30:55 -0400
Fix build with newest version of persistent-template. This is untested because of rain, also I am operating from truncated copiler error messages in a bug report that also doesn't mention what the library version is. Still, it should work. May break builds with old ghc, in particular DerivingStrategies is I think fairly new? The pragmas could be ifdefed if necessary. Works with ghc 8.6.5.
2020-02-04 16:03:30 +00:00
releasing package git-annex version 7.20200202.7
2020-02-02 20:55:38 +00:00
git-annex (7.20200202.7) upstream; urgency=medium
add --force-annex/--force-git options make it easier to override annex.largefiles configuration (and potentially safer as it avoids bugs like the smudge bug fixed in the last release) Deleted some old comments that were posted to the man page discussing such options. Updated docs that used -c annex.largefiles to use the options. Note that addSmallOverridden was needed to avoid the clean filter running on the file. It would be possible to make addFile also update the index directly, rather than going via git add. However, it was not necessary, and I want to avoid breaking on some edge case, particularly if the code in addSmallOverridden has some oversight. Also, when annex.addunlocked is set and annex.largefiles does not match a file, git annex add --force-large works, but git status will then show the file as added, with a unstaged modification. The unstaged modification adds the file to git. This is identical behavior to using -c annex.largefiles=nothing when annex.addunlocked is set. This does not prevent committing what was intended to be added. I have not gotten to the bottom of why git thinks the file is modified and runs it through the clean filter in this case.
2020-01-01 18:03:06 +00:00
document version for --force-large/--force-small also fix the wrong name in the changelog
2020-10-26 15:34:30 +00:00
* add: --force-large/--force-small options make it easier to override
add --force-annex/--force-git options make it easier to override annex.largefiles configuration (and potentially safer as it avoids bugs like the smudge bug fixed in the last release) Deleted some old comments that were posted to the man page discussing such options. Updated docs that used -c annex.largefiles to use the options. Note that addSmallOverridden was needed to avoid the clean filter running on the file. It would be possible to make addFile also update the index directly, rather than going via git add. However, it was not necessary, and I want to avoid breaking on some edge case, particularly if the code in addSmallOverridden has some oversight. Also, when annex.addunlocked is set and annex.largefiles does not match a file, git annex add --force-large works, but git status will then show the file as added, with a unstaged modification. The unstaged modification adds the file to git. This is identical behavior to using -c annex.largefiles=nothing when annex.addunlocked is set. This does not prevent committing what was intended to be added. I have not gotten to the bottom of why git thinks the file is modified and runs it through the clean filter in this case.
2020-01-01 18:03:06 +00:00
annex.largefiles configuration (and potentially safer as it avoids
bugs like the smudge bug fixed in the last release).
reinject --known: Fix bug that prevented it from working in a bare repo. ifAnnexed in a bare repo passes to git cat-file :./filename , which it refuses to do since the repo is bare. Note that, reinject somefile someannexedfile in a bare repo silently does nothing, because someannexedfile is never actually an annexed worktree file, because the repo is bare.
2020-01-06 18:22:22 +00:00
* reinject --known: Fix bug that prevented it from working in a bare repo.
support sha256 git repos Git will eventually switch to sha2 and there will not be one single shaSize anymore, but two (40 and 64). Changed all parsers for git plumbing output to support both sizes of shas. One potential problem this does not deal with is, if somewhere in git-annex it reads two shas from different sources, and compares them to see if they're the same sha, it would fail if they're sha1 and sha256 of the same value. I don't know if that will really be a concern.
2020-01-07 15:35:17 +00:00
* Support being used in a git repository that uses sha256 rather than sha1.
be stricter about rejecting invalid configurations for remotes This is a first step toward that goal, using the ProposedAccepted type in RemoteConfig lets initremote/enableremote reject bad parameters that were passed in a remote's configuration, while avoiding enableremote rejecting bad parameters that have already been stored in remote.log This does not eliminate every place where a remote config is parsed and a default value is used if the parse false. But, I did fix several things that expected foo=yes/no and so confusingly accepted foo=true but treated it like foo=no. There are still some fields that are parsed with yesNo but not not checked when initializing a remote, and there are other fields that are parsed in other ways and not checked when initializing a remote. This also lays groundwork for rejecting unknown/typoed config keys.
2020-01-10 18:10:20 +00:00
* initremote, enableremote: Be stricter about rejecting invalid
initremote --describe-other-params Does not yet include descriptions from external special remote programs.
2020-01-20 20:05:51 +00:00
configuration parameters for remotes, particularly things like foo=true
when foo=yes is expected.
* initremote, enableremote: Reject unknown configuration parameters
provided to these commands.
--whatelse is a better name than --describe-other-params The use case is basically the user having forgotten, so --help would be best, but it would be quite hard to include this in --help, since it may even have to spin up an external special remote program. I also considered --umm but typoed it the first time I tried it as --uum, and while memorable, it's too cutesy. --whatelse is good because it explicitly asks, what other params, besides the ones I've given?
2020-01-20 21:04:45 +00:00
* initremote: Added --whatelse option, to show additional
configuration parameters you might want to set. Eg:
git annex initremote type=directory encryption=none --whatelse
add LISTCONFIGS to external special remote protocol Special remote programs that use GETCONFIG/SETCONFIG are recommended to implement it. The description is not yet used, but will be useful later when adding a way to make initremote list all accepted configs. configParser now takes a RemoteConfig parameter. Normally, that's not needed, because configParser returns a parter, it does not parse it itself. But, it's needed to look at externaltype and work out what external remote program to run for LISTCONFIGS. Note that, while externalUUID is changed to a Maybe UUID, checkExportSupported used to use NoUUID. The code that now checks for Nothing used to behave in some undefined way if the external program made requests that triggered it. Also, note that in externalSetup, once it generates external, it parses the RemoteConfig strictly. That generates a ParsedRemoteConfig, which is thrown away. The reason it's ok to throw that away, is that, if the strict parse succeeded, the result must be the same as the earlier, lenient parse. initremote of an external special remote now runs the program three times. First for LISTCONFIGS, then EXPORTSUPPORTED, and again LISTCONFIGS+INITREMOTE. It would not be hard to eliminate at least one of those, and it should be possible to only run the program once.
2020-01-17 19:30:14 +00:00
* Added LISTCONFIGS to external special remote protocol. Special remote
programs that use GETCONFIG/SETCONFIG are recommended to implement it.
avoid ugly error message Http remotes that do expose a git config file, but are not initialized resulted in an ugly and unncessary error message, now sqelched. When git-annex-shell configlist is run w/o the autoinit field, it may not generate a uuid for the repository. So in that case, it's not unexpected for the config it does list to not include a UUID, and dumping out the config in a warning message is not needed. If configlist is asked to autoinit and we don't get back a config with a UUID in it, that suggests some problem, and what we got back may not be a config at all but some diagnostic message, so it does make sense to output it then.
2020-01-22 15:39:15 +00:00
* init: Avoid an ugly error message when http remote has no git-annex
uuid configured.
support git remotes that need http basic auth using git credential to get the password One thing this doesn't do is wrap the password prompting inside the prompt action. So with -J, the output can be a bit garbled.
2020-01-22 20:13:48 +00:00
* Support git remotes that need http basic auth to be accessed,
using git credential to get the password.
Display a warning when concurrency is enabled but ssh connection caching is not enabled or won't work due to a crippled filesystem A warning message is unsatisfying. But erroring out is too hard a failure, especially since it may well work fine if the user has enabled passwordless ssh. I did think about falling back to one ssh connection at a time in this case, but it would have needed a rework of every ssh call, which seems far overboard for such a niche problem. There's no single place where git-annex runs ssh, so no one place that it could block a concurrent call on a semaphore. And, even if it did fall back to one ssh connection at a time, it seems to me that doing so without warning the user about the problem just invites bug reports like "git-annex is ignoring my -J2 and only doing one download at a time". So a warning is needed, and I suppose is good enough.
2020-01-23 16:35:46 +00:00
* Display a warning when concurrency is enabled but ssh connection caching
is not enabled or won't work due to a crippled filesystem.
Makefile: Move the fish completion to the vendor_completions.d directory.
2020-01-23 20:42:08 +00:00
* Makefile: Move the fish completion to the vendor_completions.d directory.
releasing package git-annex version 7.20200202.7
2020-02-02 20:55:38 +00:00
* Fixed a test suite failure when run in the C locale.
add --force-annex/--force-git options make it easier to override annex.largefiles configuration (and potentially safer as it avoids bugs like the smudge bug fixed in the last release) Deleted some old comments that were posted to the man page discussing such options. Updated docs that used -c annex.largefiles to use the options. Note that addSmallOverridden was needed to avoid the clean filter running on the file. It would be possible to make addFile also update the index directly, rather than going via git add. However, it was not necessary, and I want to avoid breaking on some edge case, particularly if the code in addSmallOverridden has some oversight. Also, when annex.addunlocked is set and annex.largefiles does not match a file, git annex add --force-large works, but git status will then show the file as added, with a unstaged modification. The unstaged modification adds the file to git. This is identical behavior to using -c annex.largefiles=nothing when annex.addunlocked is set. This does not prevent committing what was intended to be added. I have not gotten to the bottom of why git thinks the file is modified and runs it through the clean filter in this case.
2020-01-01 18:03:06 +00:00
releasing package git-annex version 7.20200202.7
2020-02-02 20:55:38 +00:00
-- Joey Hess <id@joeyh.name> Sun, 02 Feb 2020 00:00:00 -0400
add --force-annex/--force-git options make it easier to override annex.largefiles configuration (and potentially safer as it avoids bugs like the smudge bug fixed in the last release) Deleted some old comments that were posted to the man page discussing such options. Updated docs that used -c annex.largefiles to use the options. Note that addSmallOverridden was needed to avoid the clean filter running on the file. It would be possible to make addFile also update the index directly, rather than going via git add. However, it was not necessary, and I want to avoid breaking on some edge case, particularly if the code in addSmallOverridden has some oversight. Also, when annex.addunlocked is set and annex.largefiles does not match a file, git annex add --force-large works, but git status will then show the file as added, with a unstaged modification. The unstaged modification adds the file to git. This is identical behavior to using -c annex.largefiles=nothing when annex.addunlocked is set. This does not prevent committing what was intended to be added. I have not gotten to the bottom of why git thinks the file is modified and runs it through the clean filter in this case.
2020-01-01 18:03:06 +00:00
releasing package git-annex version 7.20191230
2019-12-30 18:49:31 +00:00
git-annex (7.20191230) upstream; urgency=medium
stop displaying rsync progress, and use git-annex's own progress display for local-to-local repo transfers Reasons to do this include: 1. I've gotten pretty used to git-annex's own progress display, which is used for all transfers over ssh (except to old git-annex-shell), and for most special remote transfers. It's getting to seem weird to see the rsync progress display instead. 2. When -J was used, the rsync output could not be shown, and so there was no progress display. Now there will be. Progress will also be displayed now when cp CoW is used. But I'd expect a CoW copy to typically run so fast that the progress display will barely be noticable. This commit was sponsored by Peter on Patreon.
2019-11-15 16:01:30 +00:00
use filepath-bytestring for annex object manipulations git-annex find is now RawFilePath end to end, no string conversions. So is git-annex get when it does not need to get anything. So this is a major milestone on optimisation. Benchmarks indicate around 30% speedup in both commands. Probably many other performance improvements. All or nearly all places where a file is statted use RawFilePath now.
2019-12-11 18:12:22 +00:00
* Optimised processing of many files, especially by commands like find
and whereis that only report on the state of the repository. Commands
like get also sped up in cases where they have to check a lot of
files but only transfer a few files. Speedups range from 30-100%.
Merge branch 'master' into bs and update changelog
2019-12-18 19:13:30 +00:00
* Added build dependency on the filepath-bytestring library.
fixed an oversight that had always prevented annex.resolvemerge from being honored, when it was configured by git-annex config forgot to add it to the merge function
2019-12-20 14:55:23 +00:00
* Fixed an oversight that had always prevented annex.resolvemerge
from being honored, when it was configured by git-annex config.
git-annex config annex.largefiles annex.largefiles can be configured by git-annex config, to more easily set a default that will also be used by clones, without needing to shoehorn the expression into the gitattributes file. The git config and gitattributes override that. Whenever something is added to git-annex config, we have to consider what happens if a user puts a purposfully bad value in there. Or, if a new git-annex adds some new value that an old git-annex can't parse. In this case, a global annex.largefiles that can't be parsed currently makes an error be thrown. That might not be ideal, but the gitattribute behaves the same, and is almost equally repo-global. Performance notes: git-annex add and addurl construct a matcher once and uses it for every file, so the added time penalty for reading the global config log is minor. If the gitattributes annex.largefiles were deprecated, git-annex add would get around 2% faster (excluding hashing), because looking that up for each file is not fast. So this new way of setting it is progress toward speeding up add. git-annex smudge does need to load the log every time. As well as checking the git attribute. Not ideal. Setting annex.gitaddtoannex=false avoids both overheads.
2019-12-20 16:12:31 +00:00
* annex.largefiles can be configured by git-annex config,
to more easily set a default that will also be used by clones,
without needing to shoehorn the expression into the gitattributes file.
The git config and gitattributes override that.
annex.addunlocked expressions * annex.addunlocked can be set to an expression with the same format used by annex.largefiles, in case you want to default to unlocking some files but not others. * annex.addunlocked can be configured by git-annex config. Added a git-annex-matching-expression man page, broken out from tips/largefiles. A tricky consequence of this is that git-annex add --relaxed honors annex.addunlocked, but an expression might want to know the size or content of an url, which it's not going to download. I decided it was better not to fail, and just dummy up some plausible data in that case. Performance impact should be negligible. The global config is already loaded for annex.largefiles. The expression only has to be parsed once, and in the simple true/false case, it should not do any additional work matching it.
2019-12-20 19:01:34 +00:00
* annex.addunlocked can be set to an expression with the same format used by
annex.largefiles, when you want to default to unlocking some files but
not others.
* annex.addunlocked can be configured by git-annex config.
git-annex-config --set/--unset: No longer change the local git config setting e53070c1f quietly made it set the local git config too, but that was never documented anywhere, and it had surprising results. If I set annex.largefiles globally in a repo, I would expect to be able to change it in another repo, and the original repo would get the change and use it, rather than being stuck on the old value set there. And, if I have a local annex.largefiles and set a different global default, I'd be surprised to have my local setting overwritten. annex.securehashesonly does need to be set locally, since it's a security feature and the global is only a default until it gets set locally. So special cased.
2019-12-20 17:17:28 +00:00
* git-annex-config --set/--unset: No longer change the local git config
setting, except for in the special case of annex.securehashesonly.
Improve file ordering behavior when one parameter is "." and other parameters are other directories eg, `git-annex get . ..` used to order the files strangly, because it did not realize that when git ls-files output eg "foo", that should be grouped with the first set of files and not the second set. Fixed by making dirContains "." "./foo" = True which makes sense, because dirContains ".." "../foo" = True
2019-12-20 22:01:29 +00:00
* Improve file ordering behavior when one parameter is "." and other
parameters are other directories.
releasing package git-annex version 7.20191230
2019-12-30 18:49:31 +00:00
* smudge bugfix: When annex.largefiles=anything, files that were already
fix a case where file tracked by git unexpectedly becomes annex pointer file smudge: When annex.largefiles=anything, files that were already stored in git, and have not been modified could sometimes be converted to being stored in the annex. Changes in 7.20191024 made this more of a problem. This case is now detected and prevented.
2019-12-27 18:58:10 +00:00
stored in git, and have not been modified could sometimes be converted
to being stored in the annex. Changes in 7.20191024 made this more
of a problem. This case is now detected and prevented.
Merge branch 'master' into bs and update changelog
2019-12-18 19:13:30 +00:00
releasing package git-annex version 7.20191230
2019-12-30 18:49:31 +00:00
-- Joey Hess <id@joeyh.name> Mon, 30 Dec 2019 12:43:30 -0400
Merge branch 'master' into bs and update changelog
2019-12-18 19:13:30 +00:00
inprogress: Support --key
2019-12-18 18:04:14 +00:00
git-annex (7.20191218) upstream; urgency=medium
stop displaying rsync progress, and use git-annex's own progress display for local-to-local repo transfers Reasons to do this include: 1. I've gotten pretty used to git-annex's own progress display, which is used for all transfers over ssh (except to old git-annex-shell), and for most special remote transfers. It's getting to seem weird to see the rsync progress display instead. 2. When -J was used, the rsync output could not be shown, and so there was no progress display. Now there will be. Progress will also be displayed now when cp CoW is used. But I'd expect a CoW copy to typically run so fast that the progress display will barely be noticable. This commit was sponsored by Peter on Patreon.
2019-11-15 16:01:30 +00:00
git-lfs: remember urls, and autoenable remotes using known urls * git-lfs: The url provided to initremote/enableremote will now be stored in the git-annex branch, allowing enableremote to be used without an url. initremote --sameas can be used to add additional urls. * git-lfs: When there's a git remote with an url that's known to be used for git-lfs, automatically enable the special remote.
2019-11-18 20:09:09 +00:00
* git-lfs: The url provided to initremote/enableremote will now be
stored in the git-annex branch, allowing enableremote to be used without
an url. initremote --sameas can be used to add additional urls.
* git-lfs: When there's a git remote with an url that's known to be
used for git-lfs, automatically enable the special remote.
sync, assistant: Pull and push from git-lfs remotes. Oversight, forgot to add it to gitSyncableRemote
2019-11-18 20:13:21 +00:00
* sync, assistant: Pull and push from git-lfs remotes.
Fix bug that made bare repos be treated as non-bare when --git-dir was used. Eg: git clone url --bare r git --git-dir r annex init This resulted in worktree = Just "." and so several things that check worktree to determine when the repo is bare ran code paths intended for non-bare. One such code path[1] ran git checkout with --worktree=. which actually makes it ignore core.bare config, and so the current directory got populated with a checkout of the master branch in this example. There was probably also other breakage. The fix is a bit complicated because whether the repo is bare is not known until after Git.Config reads the config, but Git.Config handles setting the RepoLocations's worktree when core.worktree is set. So have to assume the worktree is the cwd, let core.worktree override that, and then if the repo turns out to be bare, it's set back to Nothing. (And then GIT_WORK_TREE can still override all of that.) [1] switchHEADBack, which runs even when the clone is not from a bare repo.
2019-11-21 17:16:42 +00:00
* Fix bug that made bare repos be treated as non-bare when --git-dir
was used.
inprogress: Support --key
2019-12-18 18:04:14 +00:00
* inprogress: Support --key.
* Sped up many git-annex commands that operate on many files, by
avoiding reserialization of keys.
find is 7% faster; whereis is 3% faster; and git-annex get when
all files are already present is 5% faster
* Stop displaying rsync progress, and use git-annex's own progress display
for local-to-local repo transfers.
improve benchmark --databases * benchmark: Changed --databases to take a parameter specifiying the size of the database to benchmark. * benchmark --databases: Display size of the populated database. * benchmark --databases: Improve the "addAssociatedFile to (new)" benchmark to really add new values, not overwriting old values.
2019-11-21 21:25:20 +00:00
* benchmark: Changed --databases to take a parameter specifiying the size
of the database to benchmark.
* benchmark --databases: Display size of the populated database.
simplify the name of the test cases
2019-11-21 21:38:58 +00:00
* benchmark --databases: Improve the "addAssociatedFile (new)"
improve benchmark --databases * benchmark: Changed --databases to take a parameter specifiying the size of the database to benchmark. * benchmark --databases: Display size of the populated database. * benchmark --databases: Improve the "addAssociatedFile to (new)" benchmark to really add new values, not overwriting old values.
2019-11-21 21:25:20 +00:00
benchmark to really add new values, not overwriting old values.
Windows: Fix handling of changes to time zone. Used to work but was broken in version 7.20181031, specifically commit 5ab0f48ffbd69029feb8cd101776c46ec86d6209. That this was not noticed over at least 1 daylight savings time zone changes makes me wonder if the TSDelta stuff is still needed. Perhaps the mtime on Windows no longer changes when the time zone is changed? (cherry picked from commit 09ee6b0ccb10059d44fd9a89f07646b35b718563)
2019-11-06 18:36:49 +00:00
* Windows: Fix handling of changes to time zone. (Used to work but was
broken in version 7.20181031.)
stop displaying rsync progress, and use git-annex's own progress display for local-to-local repo transfers Reasons to do this include: 1. I've gotten pretty used to git-annex's own progress display, which is used for all transfers over ssh (except to old git-annex-shell), and for most special remote transfers. It's getting to seem weird to see the rsync progress display instead. 2. When -J was used, the rsync output could not be shown, and so there was no progress display. Now there will be. Progress will also be displayed now when cp CoW is used. But I'd expect a CoW copy to typically run so fast that the progress display will barely be noticable. This commit was sponsored by Peter on Patreon.
2019-11-15 16:01:30 +00:00
inprogress: Support --key
2019-12-18 18:04:14 +00:00
-- Joey Hess <id@joeyh.name> Wed, 18 Dec 2019 13:53:51 -0400
stop displaying rsync progress, and use git-annex's own progress display for local-to-local repo transfers Reasons to do this include: 1. I've gotten pretty used to git-annex's own progress display, which is used for all transfers over ssh (except to old git-annex-shell), and for most special remote transfers. It's getting to seem weird to see the rsync progress display instead. 2. When -J was used, the rsync output could not be shown, and so there was no progress display. Now there will be. Progress will also be displayed now when cp CoW is used. But I'd expect a CoW copy to typically run so fast that the progress display will barely be noticable. This commit was sponsored by Peter on Patreon.
2019-11-15 16:01:30 +00:00
releasing package git-annex version 7.20191114
2019-11-15 01:58:23 +00:00
git-annex (7.20191114) upstream; urgency=medium
Added annex.allowsign option. This commit was sponsored by Ilya Shlyakhter on Patreon.
2019-11-11 20:15:05 +00:00
* Added annex.allowsign option.
make --json-error-messages capture url download errors Convert Utility.Url to return Either String so the error message can be displated in the annex monad and so captured. (When curl is used, its errors are still not caught.)
2019-11-12 17:33:41 +00:00
* Make --json-error-messages capture more errors,
particularly url download errors.
Fix a crash (STM deadlock) when -J is used with multiple files that point to the same key See the comment for a trace of the deadlock. Added a new StartStage. New worker threads begin in the StartStage. Once a thread is ready to do work, it moves away from the StartStage, and no thread will ever transition back to it. A thread that blocks waiting on another thread that is processing the same key will block while in the StartStage. That other thread will never switch back to the StartStage, and so the deadlock is avoided.
2019-11-14 15:31:43 +00:00
* Fix a crash (STM deadlock) when -J is used with multiple files
that point to the same key.
linuxstandalone: Fix a regression that broke git-remote-https.
2019-11-14 19:08:23 +00:00
* linuxstandalone: Fix a regression that broke git-remote-https.
OSX link libs into git-core directory So that binaries in that directory can find the library next to them, where they get modified to look. This is a hack; it would be better for OSXMkLibs to build a list of what libraries are needed where. Unsure if this is needed due to a recent reversion, or is an older problem, so updated changelog accordingly.
2019-11-14 22:31:58 +00:00
* OSX git-annex.app: Fix a problem that prevented using the bundled
git-remote-https, git-remote-http, and git-shell.
Added annex.allowsign option. This commit was sponsored by Ilya Shlyakhter on Patreon.
2019-11-11 20:15:05 +00:00
releasing package git-annex version 7.20191114
2019-11-15 01:58:23 +00:00
-- Joey Hess <id@joeyh.name> Thu, 14 Nov 2019 21:57:59 -0400
Added annex.allowsign option. This commit was sponsored by Ilya Shlyakhter on Patreon.
2019-11-11 20:15:05 +00:00
prep release
2019-11-06 16:21:02 +00:00
git-annex (7.20191106) upstream; urgency=medium
benchmark: Add --databases to benchmark sqlite databases Rescued from commit 11d6e2e260d70ba99e35464c19c2b2772ce9efaa which removed db benchmarks in favor of benchmarking arbitrary git-annex commands. Which is nice and general, but microbenchmarks are useful too.
2019-10-29 19:16:15 +00:00
init: fix data loss bug Fix bug that lost modifications to unlocked files when init is re-ran in an already initialized repo. In retrospect needing scanUnlockedFiles False in the direct mode upgrade path was a good hint that it was unsafe when used with True. However, this bug did not affect upgrade from v5. In such an upgrade, an unlocked file that is modified is left as-is. The only place scanUnlockedFiles True did overwrite modified unlocked files is during an git-annex init of a repo that was already initialized by git-annex. (I also tried a scenario where the repo had not been initialized by git-annex yet, but was cloned from a v7 repo with an unlocked file, and the pointer file replaced with some other content, and the data loss did not occur in that situation.) Since the fixed scanUnlockedFiles avoids overwriting non-pointer files, it should be safe to run in any situation, so there's no need any longer for the parameter.
2019-11-05 16:41:15 +00:00
* init: Fix bug that lost modifications to unlocked files when init is
re-ran in an already initialized repo.
benchmark: Add --databases to benchmark sqlite databases Rescued from commit 11d6e2e260d70ba99e35464c19c2b2772ce9efaa which removed db benchmarks in favor of benchmarking arbitrary git-annex commands. Which is nice and general, but microbenchmarks are useful too.
2019-10-29 19:16:15 +00:00
* benchmark: Add --databases to benchmark sqlite databases.
prep release
2019-11-06 16:21:02 +00:00
-- Joey Hess <id@joeyh.name> Wed, 06 Nov 2019 12:14:50 -0400
benchmark: Add --databases to benchmark sqlite databases Rescued from commit 11d6e2e260d70ba99e35464c19c2b2772ce9efaa which removed db benchmarks in favor of benchmarking arbitrary git-annex commands. Which is nice and general, but microbenchmarks are useful too.
2019-10-29 19:16:15 +00:00
releasing package git-annex version 7.20191024
2019-10-25 17:07:58 +00:00
git-annex (7.20191024) upstream; urgency=medium
init: Fix a failure when used in a submodule on a crippled filesystem. When the submodule's parent repo has an adjusted unlocked branch, it gets cloned by git, but git checks out master. git annex init then fails because it wants to enter the adjusted branch, but: adjusted branch adjusted/master(unlocked) already exists. Aborting because that branch may have changes that have not yet reached master Note that init actually then exits 0, leaving master checked out. This could also happen, absent submodules, if the parent repo has an adjusted unlocked branch, but it is not checked out. In the more common case where that branch is checked out, the clone uses the same branch, so no problem. The choices to fix this: * Init could delete the existing adjusted branch, and re-adjust. But then running init inside an adjusted branch on a crippled filesystem would lose any changes that have not been synced back to master. * Init could sync any changes back to master, but that would be very surprising behavior for it. * Init could simply check out the existing adjusted branch. If the branch is diverged from master, well, sync will sort that out later. This mirrors the behavior of cloning a repo that has an adjusted branch checked out that has not yet been synced back to master. Picked this choice.
2019-10-21 15:04:24 +00:00
documentation for making git add only annex when configured by annex.largefiles Code change should be trvial, but not yet implemented. This significantly complicated the task of documenting how git-annex works. I'm not sure how useful the annex.gitaddtoannex confguration is after this change; seems that if a user has an annex.largefiles they will want it applied consistently. But the last thing I want to hear is more complaining from users about git add doing something they don't want it to. There's a pretty high risk users who got used to the git add behavior and don't have annex.largefiles configured will miss the NEWS and complain bitterly about their suddenly bloated repositories. Oh well. Removed outdated comments about the old behavior to avoid confusion. I don't know if I've found all the places that griping spread to.
2019-10-24 17:50:44 +00:00
* Changed git add/git commit -a default behavior back to what it was
before v7; they add file contents to git, not to the annex.
(However, if a file was annexed before, they will still add it to
the annex, to avoid footgun.)
releasing package git-annex version 7.20191024
2019-10-25 17:07:58 +00:00
* Configuring annex.largefiles overrides that; once git-annex has
been told which files are large git add/git commit -a will annex them.
documentation for making git add only annex when configured by annex.largefiles Code change should be trvial, but not yet implemented. This significantly complicated the task of documenting how git-annex works. I'm not sure how useful the annex.gitaddtoannex confguration is after this change; seems that if a user has an annex.largefiles they will want it applied consistently. But the last thing I want to hear is more complaining from users about git add doing something they don't want it to. There's a pretty high risk users who got used to the git add behavior and don't have annex.largefiles configured will miss the NEWS and complain bitterly about their suddenly bloated repositories. Oh well. Removed outdated comments about the old behavior to avoid confusion. I don't know if I've found all the places that griping spread to.
2019-10-24 17:50:44 +00:00
* Added annex.gitaddtoannex configuration. Setting it to false prevents
git add from adding files to the annex even when annex.largefiles
is configured. (Unless the file was annexed before.)
releasing package git-annex version 7.20191024
2019-10-25 17:07:58 +00:00
* smudge: Made git add smarter about renamed annexed files. It can tell
when an annexed file was renamed, and will add it to the annex,
and not to git, unless annex.largefiles tells it to do otherwise.
init: Fix a failure when used in a submodule on a crippled filesystem. When the submodule's parent repo has an adjusted unlocked branch, it gets cloned by git, but git checks out master. git annex init then fails because it wants to enter the adjusted branch, but: adjusted branch adjusted/master(unlocked) already exists. Aborting because that branch may have changes that have not yet reached master Note that init actually then exits 0, leaving master checked out. This could also happen, absent submodules, if the parent repo has an adjusted unlocked branch, but it is not checked out. In the more common case where that branch is checked out, the clone uses the same branch, so no problem. The choices to fix this: * Init could delete the existing adjusted branch, and re-adjust. But then running init inside an adjusted branch on a crippled filesystem would lose any changes that have not been synced back to master. * Init could sync any changes back to master, but that would be very surprising behavior for it. * Init could simply check out the existing adjusted branch. If the branch is diverged from master, well, sync will sort that out later. This mirrors the behavior of cloning a repo that has an adjusted branch checked out that has not yet been synced back to master. Picked this choice.
2019-10-21 15:04:24 +00:00
* init: Fix a failure when used in a submodule on a crippled filesystem.
sync: Fix crash when there are submodules and an adjusted branch is checked out Reverse adjusting the branch uses treeItemToTreeContent, which was missed when adding submodule support earlier.
2019-10-23 15:52:56 +00:00
* sync: Fix crash when there are submodules and an adjusted branch is
checked out.
releasing package git-annex version 7.20191024
2019-10-25 17:07:58 +00:00
* enable-tor: Deal with pkexec changing to root's home directory
when running a command.
init: Fix a failure when used in a submodule on a crippled filesystem. When the submodule's parent repo has an adjusted unlocked branch, it gets cloned by git, but git checks out master. git annex init then fails because it wants to enter the adjusted branch, but: adjusted branch adjusted/master(unlocked) already exists. Aborting because that branch may have changes that have not yet reached master Note that init actually then exits 0, leaving master checked out. This could also happen, absent submodules, if the parent repo has an adjusted unlocked branch, but it is not checked out. In the more common case where that branch is checked out, the clone uses the same branch, so no problem. The choices to fix this: * Init could delete the existing adjusted branch, and re-adjust. But then running init inside an adjusted branch on a crippled filesystem would lose any changes that have not been synced back to master. * Init could sync any changes back to master, but that would be very surprising behavior for it. * Init could simply check out the existing adjusted branch. If the branch is diverged from master, well, sync will sort that out later. This mirrors the behavior of cloning a repo that has an adjusted branch checked out that has not yet been synced back to master. Picked this choice.
2019-10-21 15:04:24 +00:00
releasing package git-annex version 7.20191024
2019-10-25 17:07:58 +00:00
-- Joey Hess <id@joeyh.name> Fri, 25 Oct 2019 13:04:59 -0400
init: Fix a failure when used in a submodule on a crippled filesystem. When the submodule's parent repo has an adjusted unlocked branch, it gets cloned by git, but git checks out master. git annex init then fails because it wants to enter the adjusted branch, but: adjusted branch adjusted/master(unlocked) already exists. Aborting because that branch may have changes that have not yet reached master Note that init actually then exits 0, leaving master checked out. This could also happen, absent submodules, if the parent repo has an adjusted unlocked branch, but it is not checked out. In the more common case where that branch is checked out, the clone uses the same branch, so no problem. The choices to fix this: * Init could delete the existing adjusted branch, and re-adjust. But then running init inside an adjusted branch on a crippled filesystem would lose any changes that have not been synced back to master. * Init could sync any changes back to master, but that would be very surprising behavior for it. * Init could simply check out the existing adjusted branch. If the branch is diverged from master, well, sync will sort that out later. This mirrors the behavior of cloning a repo that has an adjusted branch checked out that has not yet been synced back to master. Picked this choice.
2019-10-21 15:04:24 +00:00
releasing package git-annex version 7.20191017
2019-10-17 22:19:47 +00:00
git-annex (7.20191017) upstream; urgency=medium
Added --mimetype and --mimeencoding file matching options. Already had these for largefiles matching, but I forgot to add them as command-line options.
2019-09-19 15:32:12 +00:00
Merge branch 'master' into sameas
2019-10-11 19:56:00 +00:00
* initremote: Added --sameas option, allows for two special remotes that
access the same data store.
add RemoteStateHandle This solves the problem of sameas remotes trampling over per-remote state. Used for: * per-remote state, of course * per-remote metadata, also of course * per-remote content identifiers, because two remote implementations could in theory generate the same content identifier for two different peices of content While chunk logs are per-remote data, they don't use this, because the number and size of chunks stored is a common property across sameas remotes. External special remote had a complication, where it was theoretically possible for a remote to send SETSTATE or GETSTATE during INITREMOTE or EXPORTSUPPORTED. Since the uuid of the remote is typically generate in Remote.setup, it would only be possible to pass a Maybe RemoteStateHandle into it, and it would otherwise have to construct its own. Rather than go that route, I decided to send an ERROR in this case. It seems unlikely that any existing external special remote will be affected. They would have to make up a git-annex key, and set state for some reason during INITREMOTE. I can imagine such a hack, but it doesn't seem worth complicating the code in such an ugly way to support it. Unfortunately, both TestRemote and Annex.Import needed the Remote to have a new field added that holds its RemoteStateHandle.
2019-10-14 16:33:27 +00:00
* Note that due to complications of the sameas feature, any external
special remotes that try to send SETSTATE or GETSTATE during INITREMOTE
or EXPORTSUPPORTED will now get back an ERROR. That would be a very
hackish thing for an external special remote to do, needing some kind
of hard-coded key value to be used, so probably nothing will be affected.
remove empty log files in transition forget --drop-dead: Remove several classes of git-annex log files when they become empty, further reducing the size of the git-annex branch. Noticed while testing sameas uuid removal, but it could happen other times too. An empty log file is always treated by git-annex the same as no file being present, and when the files are per-key, it can be a sizable space saving to exclude them from the tree.
2019-10-14 20:04:15 +00:00
* forget --drop-dead: Remove several classes of git-annex log files
when they become empty, further reducing the size of the git-annex branch.
OSX: Deal with symbolic link problem that caused git to not be included in the git-annex.dmg Homebrew now has eg: datalads-imac:~ joey$ ls -l /Users/joey/homebrew/Cellar/git/2.23.0/libexec/git-core total 36776 lrwxr-xr-x 1 joey staff 13 Aug 29 13:38 git -> ../../bin/git lrwxr-xr-x 1 joey staff 13 Aug 29 13:38 git-add -> ../../bin/git So the target of the symlink also needs to be installed now. Doing it in shell code was too hairy for my dentistry-addled brain, so reimplemented in haskell. Also using it for building linuxstandalone.
2019-10-17 15:01:41 +00:00
* OSX: Deal with symbolic link problem that caused git to not be included in
the git-annex.dmg.
Fix build with persistent-2.10. Added an additional constraint that persistent needs. This also builds with persistent-2.9.2 without needing any cpp.
2019-10-17 15:58:31 +00:00
* Fix build with persistent-2.10.
Merge branch 'master' into sameas
2019-10-11 19:56:00 +00:00
releasing package git-annex version 7.20191017
2019-10-17 22:19:47 +00:00
-- Joey Hess <id@joeyh.name> Thu, 17 Oct 2019 18:19:12 -0400
Merge branch 'master' into sameas
2019-10-11 19:56:00 +00:00
releasing package git-annex version 7.20191009
2019-10-09 16:33:09 +00:00
git-annex (7.20191009) upstream; urgency=medium
Added --mimetype and --mimeencoding file matching options. Already had these for largefiles matching, but I forgot to add them as command-line options.
2019-09-19 15:32:12 +00:00
releasing package git-annex version 7.20191009
2019-10-09 16:33:09 +00:00
* Fix bug in handling of annex.largefiles that use largerthan/smallerthan.
When adding a modified file, it incorrectly used the file size of the
old version of the file, not the current size.
Added --mimetype and --mimeencoding file matching options. Already had these for largefiles matching, but I forgot to add them as command-line options.
2019-09-19 15:32:12 +00:00
* Added --mimetype and --mimeencoding file matching options.
added --unlocked and --locked
2019-09-19 16:20:35 +00:00
* Added --unlocked and --locked file matching options.
releasing package git-annex version 7.20191009
2019-10-09 16:33:09 +00:00
* Added adjust --lock, to enter an adjusted branch where files are locked.
git-lfs: Added support for http basic auth
2019-09-24 18:46:20 +00:00
* git-lfs: Added support for http basic auth.
git-lfs: Only do endpoint discovery once when concurrency is enabled This avoids some extra work, but I don't think it was possible for two ssh endpoint discoveries run concurrently to both prompt for the ssh password; Annex.Ssh itself deals with concurrency. This is mostly groundwork for http password prompting.
2019-09-24 17:01:51 +00:00
* git-lfs: Only do endpoint discovery once when concurrency is enabled.
releasing package git-annex version 7.20191009
2019-10-09 16:33:09 +00:00
* fsck --incremental/--more: Fix bug that prevented the incremental fsck
information from being updated every 5 minutes as it was supposed to be;
it was only updated after 1000 files were checked, which may be more
files that are possible to fsck in a given fsck time window.
Thanks to Peter Simons for help with analysis of this bug.
Test: Use more robust directory removal method. I just had a test that crashed at cleanup on linux with: .t/gpgtest/12/S.gpg-agent.browser: removeDirectoryRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:getSymbolicLinkStatus: does not exist (No such file or directory) sleeping 10 seconds and will retry directory cleanup git-annex: .t/gpgtest/14/S.gpg-agent.browser: removeDirectoryRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:getSymbolicLinkStatus: does not exist (No such file or directory) removePathForcibly is supposed to be more robust to things in the directory vanishing while it's running, etc. Will probably avoid such crashes. It was added to directory-1.2.7, which comes with ghc since 8.0.2. Since base >= 4.11.1.0 means ghc 8.4.4, I expect all builds will have it, but I ifdefed it to be sure.
2019-09-24 20:59:37 +00:00
* Test: Use more robust directory removal when built with directory-1.2.7.
Close sqlite databases more robustly. Had a report of close throwing ErrorBusy on CIFS. Retrying up to 16 seconds is a balance between hopefully waiting long enough for the problem to clear up and waiting so long that git-annex seems to hang. The new dependency is free; persistent depends on unliftio-core.
2019-09-26 16:24:03 +00:00
* Close sqlite databases more robustly.
remotedaemon: Don't list --stop in help since it's not supported. Also, move out of plumbing section. When using tor, the remotedaemon is part of the user's workflow, as it runs the tor hidden service.
2019-09-30 18:40:46 +00:00
* remotedaemon: Don't list --stop in help since it's not supported.
enable-tor: Run kdesu with -c option.
2019-09-30 19:14:05 +00:00
* enable-tor: Run kdesu with -c option.
enable-tor: Use pkexec to run command as root when gksu and kdesu are not available. gksu is no longer in debian, even stable kdesu in debian is not installed in PATH any longer, though the executable is still present under /usr/lib pkexec is packagekit's replacement for those older commands.
2019-09-30 19:19:01 +00:00
* enable-tor: Use pkexec to run command as root when gksu and kdesu are not
available.
preserve unlocked file mtime when dropping When dropping an unlocked file, preserve its mtime, which avoids git status unncessarily running the clean filter on the file. If the index file has close to the same mtime as a work tree file, git will not trust the index to be up-to-date, and re-runs the clean filter unncessarily. Preserving the mtime when depopulating a pointer file avoids git status doing a little (or maybe a lot) of unncessary work. There are other places that the mtime could be preserved, including other places where pointer files are written perhaps, but also populatePointerFile. But, I don't know of cases where those lead to git status doing unncessary work, so I just fixed the one I'm aware of for now.
2019-10-08 18:01:12 +00:00
* When dropping an unlocked file, preserve its mtime, which avoids
git status unncessarily running the clean filter on the file.
update uninit to handle all the v7 stuff * uninit: Remove several git hooks that git-annex init sets up. * uninit: Remove the smudge and clean filters that git-annex init sets up.
2019-10-08 18:34:00 +00:00
* uninit: Remove several git hooks that git-annex init sets up.
* uninit: Remove the smudge and clean filters that git-annex init sets up.
work around windows having infected git's plumbing Work around git cat-file --batch's odd stripping of carriage return from the end of the line (some windows infection), avoiding crashing when the repo contains a filename ending in a carriage return.
2019-10-08 19:27:05 +00:00
* Work around git cat-file --batch's odd stripping of carriage return
from the end of the line (some windows infection), avoiding crashing
when the repo contains a filename ending in a carriage return.
releasing package git-annex version 7.20191009
2019-10-09 16:33:09 +00:00
* git-annex-standalone.rpm: Fix the git-annex-shell symlink.
Added --mimetype and --mimeencoding file matching options. Already had these for largefiles matching, but I forgot to add them as command-line options.
2019-09-19 15:32:12 +00:00
releasing package git-annex version 7.20191009
2019-10-09 16:33:09 +00:00
-- Joey Hess <id@joeyh.name> Wed, 09 Oct 2019 12:31:31 -0400
Added --mimetype and --mimeencoding file matching options. Already had these for largefiles matching, but I forgot to add them as command-line options.
2019-09-19 15:32:12 +00:00
releasing package git-annex version 7.20190912
2019-09-13 16:53:40 +00:00
git-annex (7.20190912) upstream; urgency=medium
info: error out when file matching options used on non-directory When file matching options are specified when getting info of something other than a directory, they won't have any effect, so error out to avoid confusion. This commit was sponsored by mo on Patreon.
2019-08-24 17:20:19 +00:00
v7 for all repositories * Default to v7 for new repositories. * Automatically upgrade v5 repositories to v7.
2019-08-30 17:54:57 +00:00
* Default to v7 for new repositories.
* Automatically upgrade v5 repositories to v7.
Automatically convert direct mode repositories to v7 with adjusted unlocked branches * Automatically convert direct mode repositories to v7 with adjusted unlocked branches and set annex.thin. * init: When run on a crippled filesystem with --version=5, will error out, since version 7 is needed for adjusted unlocked branch. * direct: This command always errors out as direct mode is no longer supported. * indirect: This command has become a deprecated noop. * proxy: This command is deprecated because it was only needed in direct mode. (But it continues to work.) Also removed mentions of direct mode throughough the documentation. I have not removed all the direct mode code yet.
2019-08-26 18:52:55 +00:00
* Automatically convert direct mode repositories to v7 with adjusted
unlocked branches and set annex.thin.
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
* Added annex.autoupgraderepository configuration that can be set to false
to prevent any automatic repository upgrades.
Refuse to upgrade direct mode repositories when git is older than 2.22 That git fixed a memory leak that could cause an OOM during the upgrade. Most git-annex builds have a new enough git already. OSX git was upgraded with brew. Linux i386ancient build's git was too old. Upgrading it to a fixed git didn't work (due to the newer git not working with the old ssh, https://bugs.chromium.org/p/git/issues/detail?id=7 ) Choices to deal with that were: * Somehow make direct mode upgrade work with the old git, avoiding its OOM problem. One way would be to switch the repo to indirect mode first, and so upgrade to a repo with locked files. Not good when the filesystem does not support symlinks. * backport the OOM fix from git 2.22 (And do what about the version number so git-annex knows it's fixed?) * backport openssh (and possibly more stuff) * move the i386ancient build to at least Debian stretch (still backporting git) But this will make it no longer work with some of the ancient kernels it targets. Of those, backporting the OOM fix seemed the best approach. Put "oomfix" in the git version number to indicate it. I have not automated building the git backport, so here's the patch I used: diff -ur orig/git-2.1.4/convert.c git-2.1.4/convert.c --- orig/git-2.1.4/convert.c 2014-12-18 18:42:18.000000000 +0000 +++ git-2.1.4/convert.c 2019-08-29 20:05:04.371872338 +0100 @@ -404,7 +404,7 @@ if (start_async(&async)) return 0; /* error was already reported */ - if (strbuf_read(&nbuf, async.out, len) < 0) { + if (strbuf_read(&nbuf, async.out, 0) < 0) { error("read from external filter %s failed", cmd); ret = 0; } diff -ur orig/git-2.1.4/GIT-VERSION-GEN git-2.1.4/GIT-VERSION-GEN --- orig/git-2.1.4/GIT-VERSION-GEN 2014-12-18 18:42:18.000000000 +0000 +++ git-2.1.4/GIT-VERSION-GEN 2019-08-29 20:06:39.132743228 +0100 @@ -1,7 +1,7 @@ #!/bin/sh GVF=GIT-VERSION-FILE -DEF_VER=v2.1.4 +DEF_VER=v2.1.4.oomfix LF=' ' diff -ur orig/git-2.1.4/configure git-2.1.4/configure --- orig/git-2.1.4/configure 2014-12-18 18:42:19.000000000 +0000 +++ git-2.1.4/configure 2019-08-29 20:27:45.896380015 +0100 @@ -580,8 +580,8 @@ # Identity of this package. PACKAGE_NAME='git' PACKAGE_TARNAME='git' -PACKAGE_VERSION='2.1.4' -PACKAGE_STRING='git 2.1.4' +PACKAGE_VERSION='2.1.4.oomfix' +PACKAGE_STRING='git 2.1.4.oomfix' PACKAGE_BUGREPORT='git@vger.kernel.org' PACKAGE_URL='' diff -ur orig/git-2.1.4/version git-2.1.4/version --- orig/git-2.1.4/version 2014-12-18 18:42:19.000000000 +0000 +++ git-2.1.4/version 2019-08-29 20:06:17.572545210 +0100 @@ -1 +1 @@ -2.1.4 +2.1.4.oomfix
2019-08-29 18:12:45 +00:00
* Refuse to upgrade direct mode repositories when git is older than 2.22,
which fixed a memory leak that could cause an OOM during the upgrade.
release prep
2019-09-12 17:53:22 +00:00
* Removed support for git versions older than 2.1.
Automatically convert direct mode repositories to v7 with adjusted unlocked branches * Automatically convert direct mode repositories to v7 with adjusted unlocked branches and set annex.thin. * init: When run on a crippled filesystem with --version=5, will error out, since version 7 is needed for adjusted unlocked branch. * direct: This command always errors out as direct mode is no longer supported. * indirect: This command has become a deprecated noop. * proxy: This command is deprecated because it was only needed in direct mode. (But it continues to work.) Also removed mentions of direct mode throughough the documentation. I have not removed all the direct mode code yet.
2019-08-26 18:52:55 +00:00
* assistant: When creating a new repository, no longer use direct
mode, instead use v7 adjusted branches with annex.thin.
* init: When run on a crippled filesystem with --version=5,
will error out, since version 7 is needed for adjusted unlocked branch.
* direct: This command always errors out as direct mode is no longer
supported.
* indirect: This command has become a deprecated noop.
* proxy: This command is deprecated because it was only needed in direct
mode. (But it continues to work.)
info: remove "repository mode" info: Removed the "repository mode" from its output (including the --json output) since with the removal of direct mode, there is no repository mode.
2019-08-29 18:12:22 +00:00
* info: Removed the "repository mode" from its output (including the
--json output) since with the removal of direct mode, there is no
repository mode.
info: error out when file matching options used on non-directory When file matching options are specified when getting info of something other than a directory, they won't have any effect, so error out to avoid confusion. This commit was sponsored by mo on Patreon.
2019-08-24 17:20:19 +00:00
* info: When file matching options are specified when getting
info of something other than a directory, they won't have any effect,
so error out to avoid confusion.
release prep
2019-09-12 17:53:22 +00:00
* info: Display trust level when getting info on a uuid, same as a remote.
fix hard links when upgrading from direct mode When upgrading a direct mode repo to v7 with adjusted unlocked branches, fix a bug that prevented annex.thin from taking effect for the files in working tree. The hard links used to be ok, but commit 8e22114735 accidentially broke them. It repopulates the worktree file, which is already a hard link, and when it's creating the new file, the link count is already 2, and so it doesn't make a hard link then.
2019-08-26 17:46:58 +00:00
* When upgrading a direct mode repo to v7 with adjusted unlocked branches,
fix a bug that prevented annex.thin from taking effect for the files
in working tree.
Avoid making a commit when upgrading from direct mode to v7 Three reasons: * Committing as part of an upgrade is very unusual and unexpected. * The commit was failing with a weird error message when done during an automatic upgrade. * Let me remove more of that sweet^Whorrible direct mode code.
2019-08-26 20:34:07 +00:00
* Avoid making a commit when upgrading from direct mode to v7.
init: Catch more exceptions when testing locking.
2019-08-29 16:19:07 +00:00
* init: Catch more exceptions when testing locking.
init: Fix a reversion that broke initialization on systems that need to use pid locking This brings back .git/annex/misctmp, but only for init. If an init is interrupted while probing using that temp directory, the files it left will get deleted 1 week later by a subsequent git-annex run.
2019-09-10 17:37:07 +00:00
* init: Fix a reversion that broke initialization on systems that
need to use pid locking.
docs for git-annex-standalone rpm
2019-09-13 16:18:36 +00:00
* A git-annex-standalone rpm is now distributed along with the standalone
tarball.
building a standalone rpm from the standalone tarball This allows the rpm to be built anywhere the necessary build deps are available (including on debian) and the resulting package will work on as broad a range of rpm distributions as the libc/kernel supports. The DistributionUpdate changes to use the new script have not yet been tested.
2019-09-12 23:02:59 +00:00
* Added standalone/rpm/rpmbuild-from-standalone-tarball script.
info: error out when file matching options used on non-directory When file matching options are specified when getting info of something other than a directory, they won't have any effect, so error out to avoid confusion. This commit was sponsored by mo on Patreon.
2019-08-24 17:20:19 +00:00
releasing package git-annex version 7.20190912
2019-09-13 16:53:40 +00:00
-- Joey Hess <id@joeyh.name> Fri, 13 Sep 2019 12:53:06 -0400
info: error out when file matching options used on non-directory When file matching options are specified when getting info of something other than a directory, they won't have any effect, so error out to avoid confusion. This commit was sponsored by mo on Patreon.
2019-08-24 17:20:19 +00:00
releasing package git-annex version 7.20190819
2019-08-19 16:26:45 +00:00
git-annex (7.20190819) upstream; urgency=medium
make --in=here fast Use the same optimisation for --in=here as has always been used for --in=. rather than the slow code path that unncessarily queries the git-annex branch. It looks like when "here" got added as an alias for "." back in 2012, I forgot about this place. Also sped up some very unlikely ways of referring to the current repository. Note that, this could in some rare corner case cause a behavior change, if the git-annex branch and inAnnex disagree about whether content is present in the local repository. But --in=. already behaved that way, and the truth on the ground should win also.
2019-08-01 04:17:02 +00:00
support using gcrypt with git-lfs special remote
2019-08-05 17:24:21 +00:00
* New git-lfs special remote, which can be used to store data on any git-lfs
server, including github, gitlab, and gogs.
* Support fully encrypting all data sent to a git-lfs special remote,
using a combination of gcrypt to encrypt the git data, and git-annex's
encryption of its data.
make --in=here fast Use the same optimisation for --in=here as has always been used for --in=. rather than the slow code path that unncessarily queries the git-annex branch. It looks like when "here" got added as an alias for "." back in 2012, I forgot about this place. Also sped up some very unlikely ways of referring to the current repository. Note that, this could in some rare corner case cause a behavior change, if the git-annex branch and inAnnex disagree about whether content is present in the local repository. But --in=. already behaved that way, and the truth on the ground should win also.
2019-08-01 04:17:02 +00:00
* Use the same optimisation for --in=here as has always been
used for --in=. rather than the slow code path that unncessarily
queries the git-annex branch.
Allow setting up a gcrypt special remote with encryption=shared It was documented to work, but seems it has been broken for a while/forever.
2019-08-05 16:41:05 +00:00
* Allow setting up a gcrypt special remote with encryption=shared.
Fix querying git for object type when operating on a file containing newlines This typo would make "git cat-file cat-file" fail, and the way it's used, I think it broke querying all info from filenames containing newlines, because the other queries are only run when it succeeds.
2019-08-07 17:35:42 +00:00
* Fix querying git for object type when operating on a file containing
newlines.
git-annex merge branch * merge: When run with a branch parameter, merges from that branch. This is especially useful when using an adjusted branch, because it applies the same adjustment to the branch before merging it.
2019-08-09 17:21:15 +00:00
* merge: When run with a branch parameter, merges from that branch.
This is especially useful when using an adjusted branch, because
it applies the same adjustment to the branch before merging it.
init: Install working hook scripts when run on a crippled filesystem and on Windows
2019-08-13 19:14:17 +00:00
* init: Install working hook scripts when run on a crippled filesystem
and on Windows. If your repository was set up by an old version
of git-annex that omitted the hooks, you can simply re-run git-annex init
to install them.
releasing package git-annex version 7.20190819
2019-08-19 16:26:45 +00:00
* init: When the repo is already initialized, and --version requests a
different version, error out rather than silently not changing the
version.
S3: Fix encoding when generating public urls of S3 objects. This code feels worryingly stringily typed, but using URI does not help because the uriPath still has to be constructed with the right uri-encoding.
2019-08-15 16:55:48 +00:00
* S3: Fix encoding when generating public urls of S3 objects.
Make git-annex-standalone.deb include the shell completions again Was lost when the install-completions target was added.
2019-08-16 17:47:48 +00:00
* Make git-annex-standalone.deb include the shell completions again.
Makefile: Changed default zsh completion location to zsh default fpath. Systems such as Debian that have overridden the default fpath will need to set ZSH_COMPLETIONS_PATH. I feel that Debian is causing unncessary complexity by making this change, and have filed a bug report about it. This also means that when git-annex is installed with PREFIX=/usr/local it will use /usr/local/share/zsh/site-functions which works with probably all versions of zsh.
2019-08-16 18:08:56 +00:00
* Makefile: Changed default zsh completion location to zsh default fpath.
Systems such as Debian that have overridden the default fpath will
need to set ZSH_COMPLETIONS_PATH.
releasing package git-annex version 7.20190819
2019-08-19 16:26:45 +00:00
* test: Add pass using adjusted unlocked branch.
* Fix intermittent failure of the test suite, where its repeated opening
and writing to the sqlite database somehow caused inode cache information
to occasionally be lost.
* Fix several test suite failures on Windows.
* Windows installer: Always install to 64 bit program files directory,
since it needs 64 bit git now.
* Build with silently-1.2.5.1 on Windows; the old one used "NUL" device
which is not supported with recent versions of ghc.
make --in=here fast Use the same optimisation for --in=here as has always been used for --in=. rather than the slow code path that unncessarily queries the git-annex branch. It looks like when "here" got added as an alias for "." back in 2012, I forgot about this place. Also sped up some very unlikely ways of referring to the current repository. Note that, this could in some rare corner case cause a behavior change, if the git-annex branch and inAnnex disagree about whether content is present in the local repository. But --in=. already behaved that way, and the truth on the ground should win also.
2019-08-01 04:17:02 +00:00
releasing package git-annex version 7.20190819
2019-08-19 16:26:45 +00:00
-- Joey Hess <id@joeyh.name> Mon, 19 Aug 2019 12:23:16 -0400
make --in=here fast Use the same optimisation for --in=here as has always been used for --in=. rather than the slow code path that unncessarily queries the git-annex branch. It looks like when "here" got added as an alias for "." back in 2012, I forgot about this place. Also sped up some very unlikely ways of referring to the current repository. Note that, this could in some rare corner case cause a behavior change, if the git-annex branch and inAnnex disagree about whether content is present in the local repository. But --in=. already behaved that way, and the truth on the ground should win also.
2019-08-01 04:17:02 +00:00
prep release
2019-07-30 16:47:33 +00:00
git-annex (7.20190730) upstream; urgency=medium
fixup after branch merge
2019-07-08 13:01:50 +00:00
CoW probing Improved probing when CoW copies can be made between files on the same drive. Now supports CoW between BTRFS subvolumes. And, falls back to rsync instead of using cp when CoW won't work, eg copies between repos on the same EXT4 filesystem. Rather than trying cp --reflink=always for each file copied to a remote, it's tried once and if it fails it falls back to using rsync thereafter for the lifetime of the Remote object. That avoids overhead of calling cp which while small, will add up over a large number of files. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2019-07-17 18:19:00 +00:00
* Improved probing when CoW copies can be made between files on the same
drive. Now supports CoW between BTRFS subvolumes. And, falls back to rsync
instead of using cp when CoW won't work, eg copies between repos on the
same EXT4 filesystem.
prep release
2019-07-30 16:47:33 +00:00
* Add BLAKE2BP512 and BLAKE2BP512E backends, using a blake2 variant
optimised for 4-way CPUs.
* Support running v7 upgrade in a repo where there is no branch checked
out, but HEAD is set directly to some other ref.
remove bundled rsync from windows build rsync is only needed for rsync special remotes and git-annex-shell from Debian oldstable. Since the library situation on windows for rsync required a particular 32 bit build of git for it to work, and may also somehow need git-annex to be 32 bit build, it's better to not include it. This commit was sponsored by Jake Vosloo on Patreon.
2019-07-22 13:37:42 +00:00
* Windows build no longer ships with a copy of rsync, since that is only
used any more to access rsync special remotes or remotes with a very
old version of git-annex-shell.
* Windows build is now 64 bit, and using it with the 64 bit git for
Windows is fully supported.
windows long filenames should be fixed now by new ghc
2019-07-22 13:44:09 +00:00
* Windows problems with long filenames should be fixed now,
since the Windows build is made with a newer ghc version that works
around the problems.
prep release
2019-07-30 16:47:33 +00:00
* stack.yaml: Build with http-client-0.5.14 to get a bug fix to http header
parsing.
* Drop support for building with ghc older than 8.4.4,
and with older versions of serveral haskell libraries.
Support building with socks-0.6 and persistant-template-2.7 persistent-template now needs UndecidableInstances. socks changed defaultSocksConf to take a SockAddr.
2019-07-30 16:49:37 +00:00
* Support building with socks-0.6 and persistant-template-2.7.
Corrected some license statements In 40ecf58d4ba33453f5c2eb077c780c7db278975d I changed the license of code I wrote from GPL to AGPL. But, two files containing code I wrote combined with code by others were updated to say their license is AGPL, while in fact part of it was (the code I wrote) but part remained under the original license (the code written by others). Remote/Ddar.hs is now changed entirely back to GPL 3. Annex/DirHashes.hs stays AGPL, but I broke out Utility/MD5.hs with the code not written by me, and corrected its license statement to GPL-2, which is the actual version of the GPL included with the code in its original distribution at http://www.cs.ox.ac.uk/people/ian.lynagh/md5/
2019-07-28 18:27:33 +00:00
* Corrected some license statements.
prep release
2019-07-30 16:47:33 +00:00
Thanks, Sean Whitton.
fixup after branch merge
2019-07-08 13:01:50 +00:00
prep release
2019-07-30 16:47:33 +00:00
-- Joey Hess <id@joeyh.name> Tue, 30 Jul 2019 12:22:25 -0400
fixup after branch merge
2019-07-08 13:01:50 +00:00
releasing package git-annex version 7.20190708
2019-07-08 12:58:44 +00:00
git-annex (7.20190708) upstream; urgency=medium
Fix find --json to output json once more. Reversion from commit 436f10771, CustomOutput was forcing quiet output which overrode the json setting. find happened to be the only command that uses CustomOutput and also outputs json. (metadata --get does also use CustomOutput and --json does not enable json output for that, which may be an oversight, but was already the behavior before this regression.)
2019-07-05 13:58:37 +00:00
* Fix find --json to output json once more.
releasing package git-annex version 7.20190708
2019-07-08 12:58:44 +00:00
-- Joey Hess <id@joeyh.name> Mon, 08 Jul 2019 08:58:16 -0400
Fix find --json to output json once more. Reversion from commit 436f10771, CustomOutput was forcing quiet output which overrode the json setting. find happened to be the only command that uses CustomOutput and also outputs json. (metadata --get does also use CustomOutput and --json does not enable json output for that, which may be an oversight, but was already the behavior before this regression.)
2019-07-05 13:58:37 +00:00
releasing package git-annex version 7.20190626
2019-06-26 16:30:03 +00:00
git-annex (7.20190626) upstream; urgency=medium
Merge branch 'starting'
2019-06-15 16:42:10 +00:00
run download checksum verification in separate job pool get, move, copy, sync: When -J or annex.jobs has enabled concurrency, checksum verification uses a separate job pool than is used for downloads, to keep bandwidth saturated. Not yet done for upload checksum verification, but that only affects remotes on local disks.
2019-06-17 18:58:02 +00:00
* get, move, copy, sync: When -J or annex.jobs has enabled concurrency,
checksum verification uses a separate job pool than is used for
downloads, to keep bandwidth saturated.
* Other commands also run their cleanup phase using a separate job pool
than their perform phase, which may make some of them somewhat faster
when running concurrently as well.
avoid url resume from 0 When downloading an url and the destination file exists but is empty, avoid using http range to resume, since a range "bytes=0-" is an unusual edge case that it's best to avoid relying on working. This is known to fix a case where importfeed downloaded a partial feed from such a server. Since importfeed uses withTmpFile, the destination always exists empty, so it would particularly tickle such problem servers. Resuming from 0 is otherwise possible, but unlikely.
2019-06-20 16:26:17 +00:00
* When downloading an url and the destination file exists but is empty,
avoid using http range to resume, since a range "bytes=0-" is an unusual
edge case that it's best to avoid relying on working. This is known to
fix a case where importfeed downloaded a partial feed from such a server.
importfeed: When there's a problem parsing the feed, --debug will output the feed content that was downloaded. And let the user know about it in the failure messages.
2019-06-20 16:37:07 +00:00
* importfeed: When there's a problem parsing the feed, --debug will
output the feed content that was downloaded.
fix init default description reversion init: Fix a reversion in the last release that prevented automatically generating and setting a description for the repository. Seemed best to factor out uuidDescMapRaw that does not have the default mempty descrition behavior. I don't much like that behavior, but I know things depend on it. One thing in particular is `git annex info` which lists the uuids and descriptions; if the current repo has been initialized in some way that means it does not have a description, it would not show up w/o that. (Not only repos created due to this bug might lack that. For example a repo that was marked dead and had --drop-dead delete its git-annex branch info, and then came back from the dead would similarly not be in the uuid.log. Also there have been other versions of git-annex that didn't set a default description; for years there was no default description.)
2019-06-21 00:30:24 +00:00
* init: Fix a reversion in the last release that prevented automatically
generating and setting a description for the repository.
add: Display progress meter when hashing files. * add: Display progress meter when hashing files. * add: Support --json-progress option.
2019-06-25 17:12:47 +00:00
* add: Display progress meter when hashing files.
* add: Support --json-progress option.
arm ghc bug fixed
2019-06-26 04:55:05 +00:00
* The Linux standalone arm build now works again on CPU versions below
arm6. Thanks to Emanuele Olivetti, Ilias Tsitsimpis, Bernhard
Übelacker, and Adrian Bunk for fixing ghc in Debian (bug #928882).
OSX dmg: Put git-annex's version in the Info.plist file.
2019-06-26 16:10:35 +00:00
* OSX dmg: Put git-annex's version in the Info.plist file.
Merge branch 'starting'
2019-06-15 16:42:10 +00:00
releasing package git-annex version 7.20190626
2019-06-26 16:30:03 +00:00
-- Joey Hess <id@joeyh.name> Wed, 26 Jun 2019 12:29:46 -0400
Merge branch 'starting'
2019-06-15 16:42:10 +00:00
releasing package git-annex version 7.20190615
2019-06-15 16:39:16 +00:00
git-annex (7.20190615) upstream; urgency=medium
Merge branch 'wip'
2019-05-07 17:07:45 +00:00
* Fixed bug that caused git-annex to fail to add a file when another
git-annex process cleaned up the temp directory it was using.
Makefile: Added install-completions to install target.
2019-05-08 14:48:38 +00:00
* Makefile: Added install-completions to install target.
annex.jobs=cpus etc Added the ability to run one job per CPU (core), by setting annex.jobs=cpus, or using option --jobs=cpus or -Jcpus. Built with future expansion in mind, including not defaulting matching on Concurrency so more constructors can later be added, and using "cpu" instead of "0".
2019-05-10 17:24:31 +00:00
* Added the ability to run one job per CPU (core), by setting
annex.jobs=cpus, or using option --jobs=cpus or -Jcpus.
docs for export preferred content This includes a note about how include= and exclude= match when exporting a subtree. I don't know if the note is prominent enough, but the behavior seems unsurprising enough.
2019-05-20 16:01:37 +00:00
* Honor preferred content of a special remote when exporting trees to it;
unwanted files are filtered out of the tree that is exported.
honor preferred content when importing Importing from a special remote honors its preferred content too; unwanted files are not imported. But, some preferred content expressions can't be checked before files are imported, and trying to import with such an expression will fail. Tested this with scenarios including changing the preferred content expression and making sure merging the import didn't delete files that were no longer wanted. There was one minor inefficiency mentioned in the todo that I punted on.
2019-05-21 18:38:00 +00:00
* Importing from a special remote honors its preferred content too;
unwanted files are not imported. But, some preferred content
expressions can't be checked before files are imported, and trying to
import with such an expression will fail.
Don't try to import .git directories from special remotes Because git does not support storing git repositories inside a git repository.
2019-06-04 19:14:20 +00:00
* Don't try to import .git directories from special remotes, because
git does not support storing git repositories inside a git repository.
Improve shape of commit tree when importing from unversioned special remotes Make the import have the previous import as a parent, so eg `git log --stat` displays a useful diff. Also a minor optimisation, only calculate the depth of the imported history once.
2019-05-21 15:32:54 +00:00
* Improve shape of commit tree when importing from unversioned special
remotes.
fix repo description setting bugs * init: When the repository already has a description, don't change it. * describe: When run with no description parameter it used to set the description to "", now it will error out.
2019-05-23 16:51:01 +00:00
* init: When the repository already has a description, don't change it.
* describe: When run with no description parameter it used to set
the description to "", now it will error out.
Android: Improve installation process when the user's login shell is not bash. ~/.profile works for bash, but not all other login shells. This setting PATH is a minor convenience for users, particuarly since typing on android is so much harder. The usual linux standalone bundle just expects the user to know how to add it to PATH. I don't want this code to grow special cases for every possible login shell. So displaying a message to the presumably minority who don't use bash seems like the best choice. Longer term, I'd hope termux gets some way to set an environment variable for all login shells. Systems using PAM can, via ~/.pam_environment. Or alternatively, add a git-annex package to termux, even if just an installer package. I'd rather spend time on either of those than on making this minor thing support more login shells. This commit was sponsored by mo on Patreon.
2019-05-23 17:06:20 +00:00
* Android: Improve installation process when the user's login shell is not
bash.
make readonly export remotes really be readonly When a remote is configured to be readonly, don't allow changing what's exported to it. This was missed in the original export remote implementation, but it makes sense for a readonly export remote to not be allowed to change.
2019-05-28 15:04:28 +00:00
* When a remote is configured to be readonly, don't allow changing
what's exported to it.
rename annex.security.allowed-http-addresses Renamed annex.security.allowed-http-addresses to annex.security.allowed-ip-addresses because it is not really specific to the http protocol, also limiting eg, git-annex's use of ftp and via youtube-dl, several other protocols. The old name for the config will still work. If both old and new name are set, the new name will win.
2019-05-30 16:43:40 +00:00
* Renamed annex.security.allowed-http-addresses to
annex.security.allowed-ip-addresses because it is not really specific
to the http protocol, also limiting eg, git-annex's use of ftp.
The old name for the config will still work.
add back support for ftp urls Add back support for ftp urls, which was disabled as part of the fix for security hole CVE-2018-10857 (except for configurations which enabled curl and bypassed public IP address restrictions). Now it will work if allowed by annex.security.allowed-ip-addresses.
2019-05-30 18:51:34 +00:00
* Add back support for ftp urls, which was disabled as part of the fix for
security hole CVE-2018-10857 (except for configurations which enabled curl
and bypassed public IP address restrictions). Now it will work
if allowed by annex.security.allowed-ip-addresses.
avoid rsync/gcrypt ssh startup delay with -J Avoid a delay at startup when concurrency is enabled and there are rsync or gcrypt special remotes, which was caused by git-annex opening a ssh connection to the remote too early. sshOptions makes a connection to the ssh server if one is not already open, when concurrency is enabled. Avoid doing that at startup, when the remote list is being built, but the remote may not be used at all. Instead, rsync/gcrypt now runs sshOptions once per ssh connection to the server. This should not be significant overhead since Remote.Git already has the same overhead (as do Bup and Ddar).
2019-06-13 15:09:55 +00:00
* Avoid a delay at startup when concurrency is enabled and there are
rsync or gcrypt special remotes, which was caused by git-annex
opening a ssh connection to the remote too early.
Merge branch 'wip'
2019-05-07 17:07:45 +00:00
releasing package git-annex version 7.20190615
2019-06-15 16:39:16 +00:00
-- Joey Hess <id@joeyh.name> Sat, 15 Jun 2019 12:38:25 -0400
Merge branch 'wip'
2019-05-07 17:07:45 +00:00
releasing package git-annex version 7.20190507
2019-05-07 17:05:52 +00:00
git-annex (7.20190507) upstream; urgency=medium
fix export subtree reversion Fix reversion in last release that caused wrong tree to be written to remote tracking branch after an export of a subtree. The invariant "commitsha should have the treesha as its tree" was not met due to a bug. Guarantee it's met by catting the commitsha to find its actual tree. A little bit slower, but this is not run often.
2019-05-06 17:56:39 +00:00
* Fix reversion in last release that caused wrong tree to be written
to remote tracking branch after an export of a subtree.
Improved locking when multiple git-annex processes are writing to the .git/index file
2019-05-06 19:15:12 +00:00
* Improved locking when multiple git-annex processes are writing to
the .git/index file
fix export subtree reversion Fix reversion in last release that caused wrong tree to be written to remote tracking branch after an export of a subtree. The invariant "commitsha should have the treesha as its tree" was not met due to a bug. Guarantee it's met by catting the commitsha to find its actual tree. A little bit slower, but this is not run often.
2019-05-06 17:56:39 +00:00
releasing package git-annex version 7.20190507
2019-05-07 17:05:52 +00:00
-- Joey Hess <id@joeyh.name> Tue, 07 May 2019 13:05:33 -0400
fix export subtree reversion Fix reversion in last release that caused wrong tree to be written to remote tracking branch after an export of a subtree. The invariant "commitsha should have the treesha as its tree" was not met due to a bug. Guarantee it's met by catting the commitsha to find its actual tree. A little bit slower, but this is not run often.
2019-05-06 17:56:39 +00:00
releasing package git-annex version 7.20190503
2019-05-03 16:48:28 +00:00
git-annex (7.20190503) upstream; urgency=medium
adb import As well as adding the necessary methods, a few other changes to the adb remote: * Use ".annextmp" extension for temp files, to avoid conflict with other temp files. * Stop using "echo $?" to get exit status of command inside adb. There were two problems; first the "echo" just before it meant it was always 0! And secondly, it seems kind of random on my phone whether it's 1 or 0, not dependant on whether the command seems to have succeeded.
2019-04-09 21:52:41 +00:00
fix typos
2019-05-01 18:43:35 +00:00
* adb special remote supports being configured with importtree=yes,
adb import As well as adding the necessary methods, a few other changes to the adb remote: * Use ".annextmp" extension for temp files, to avoid conflict with other temp files. * Stop using "echo $?" to get exit status of command inside adb. There were two problems; first the "echo" just before it meant it was always 0! And secondly, it seems kind of random on my phone whether it's 1 or 0, not dependant on whether the command seems to have succeeded.
2019-04-09 21:52:41 +00:00
to allow git-annex import of files from an Android device. This can be
combined with exporttree=yes and git-annex export used to send changes
back to the Android device.
fix typos
2019-05-01 18:43:35 +00:00
* S3 special remote supports being configured with importtree=yes,
versioned import from S3 is working Still some bugs and two stubbed methods to implement though.
2019-04-19 19:10:31 +00:00
to allow git-annex import of files from a S3 bucket. This can be
combined with exporttree=yes and git-annex export used to send changes
back to the S3 bucket.
* S3: When versioning is enabled on a bucket, importing from it will
import old versions of files that were written to the bucket as well
as the current versions. A git history is synthesized to reflect the way
the bucket changed over time.
remove wrong uniqueness constraint from ContentIdentifier db Fix bug that caused importing from a special remote to repeatedly download unchanged files when multiple files in the remote have the same content. Unfortunately, there's really no good way to remove a uniqueness constraint from a sqlite database. The best that can be done is to make a new table and copy the data over. But that would require using persistent's migrations or raw sql, and I don't want to do either. Instead, a sledgehammer approach: Renamed .git/annex/cid to .git/annex/cids. When the new database doesn't exist, it will be populated from the git-annex branch. Noting deletes the old database. Don't want to delete it out from under some long-running git-annex process that might be using it. It could eventually be deleted. But this is such a new feature, probably few repos have the database in any case.
2019-04-09 23:58:24 +00:00
* Fix bug that caused importing from a special remote to repeatedly
download unchanged files when multiple files in the remote have the same
content.
Made git-annex sync --content much faster when all the remotes it's syncing with are export/import remotes It was unnecessarily going over all files and checking preferred content against no remotes.
2019-04-10 16:42:10 +00:00
* Made git-annex sync --content much faster when all the remotes it's
syncing with are export/import remotes.
sync: When listing contents on an import remote fails, proceed with other syncing instead of aborting Switch listContents to being a proper CommandStart, so if it throws an exception, it will be treated like any other command action that fails. downloadImport apparently does not ever throw an exception, and itself uses commandAction, so it can't be a CommandStart.
2019-04-10 21:02:56 +00:00
* sync: When listing contents on an import remote fails, proceed with
other syncing instead of aborting.
added renameremote command
2019-04-15 17:05:44 +00:00
* renameremote: New command, changes the name that is used to enable
a special remote. Especially useful when you want to reuse the name
of an old remote for something new.
Drop support for building with aws older than 0.14. debian stable has 0.14 so lose the complexity for old versions
2019-04-19 18:27:59 +00:00
* Drop support for building with aws older than 0.14.
info: Show when a remote is configured with importtree
2019-04-23 18:27:43 +00:00
* info: Show when a remote is configured with importtree.
Added mimeencoding= term to annex.largefiles expressions. * Added mimeencoding= term to annex.largefiles expressions. This is probably mostly useful to match non-text files with eg "mimeencoding=binary" * git-annex matchexpression: Added --mimeencoding option.
2019-04-30 15:58:06 +00:00
* Added mimeencoding= term to annex.largefiles expressions.
This is probably mostly useful to match non-text files with eg
"mimeencoding=binary"
* git-annex matchexpression: Added --mimeencoding option.
adb import As well as adding the necessary methods, a few other changes to the adb remote: * Use ".annextmp" extension for temp files, to avoid conflict with other temp files. * Stop using "echo $?" to get exit status of command inside adb. There were two problems; first the "echo" just before it meant it was always 0! And secondly, it seems kind of random on my phone whether it's 1 or 0, not dependant on whether the command seems to have succeeded.
2019-04-09 21:52:41 +00:00
releasing package git-annex version 7.20190503
2019-05-03 16:48:28 +00:00
-- Joey Hess <id@joeyh.name> Fri, 03 May 2019 12:47:41 -0400
adb import As well as adding the necessary methods, a few other changes to the adb remote: * Use ".annextmp" extension for temp files, to avoid conflict with other temp files. * Stop using "echo $?" to get exit status of command inside adb. There were two problems; first the "echo" just before it meant it was always 0! And secondly, it seems kind of random on my phone whether it's 1 or 0, not dependant on whether the command seems to have succeeded.
2019-04-09 21:52:41 +00:00
releasing package git-annex version 7.20190322
2019-03-22 17:57:17 +00:00
git-annex (7.20190322) upstream; urgency=medium
fix encoding of metadata containing newlines This fixes a reversion in the ByteString conversion. The old code used isSpace to decide when the metadata value needs to be base64 encoded, and that incorrectly changed to only checking if it contained ' '. Note that only '\n' and '\r' were added and not other sorts of whitespace that isSpace matches, like '\t' and '\v'. Only the former would cause problems.
2019-02-20 18:22:31 +00:00
make export and sync update special remote tracking branch The branch is only updated once the export is 100% complete. This way, if an export is started but interrupted and so the remote does not yet contain some of the files, an import will make a commit on the old branch, and so won't delete the missing files.
2019-03-01 20:08:18 +00:00
* New feature allows importing from special remotes, using
git annex import branch:subdir --from remote
docs for importtree config
2019-03-04 19:39:19 +00:00
* Directory special remote supports being configured with importree=yes,
to allow git-annex import of files from the directory. This can be
combined with exporttree=yes and git-annex export used to send changes
back to the same directory.
update changelog
2019-03-11 18:17:49 +00:00
* Remote tracking branches are updated when importing and exporting to
special remotes, in ways analagous to how git fetch and git push do.
make export and sync update special remote tracking branch The branch is only updated once the export is 100% complete. This way, if an export is started but interrupted and so the remote does not yet contain some of the files, an import will make a commit on the old branch, and so won't delete the missing files.
2019-03-01 20:08:18 +00:00
* export: Deprecated the --tracking option.
Instead, users can configure remote.<name>.annex-tracking-branch
themselves.
update changelog
2019-03-11 18:17:49 +00:00
* sync --content: When remote.<name>.annex-tracking-branch is configured,
import from special remotes.
* sync, assistant: --no-push and remote.<name>.annex-push prevent exporting
--no-push and remote.name.annex-push prevent exporting trees to special remotes Users may want sync to only export, or only import and this is broadly analagous to push and pull, so it makes sense to use the same configuration for it.
2019-03-09 17:21:49 +00:00
trees to special remotes.
fix encoding of metadata containing newlines This fixes a reversion in the ByteString conversion. The old code used isSpace to decide when the metadata value needs to be base64 encoded, and that incorrectly changed to only checking if it contained ' '. Note that only '\n' and '\r' were added and not other sorts of whitespace that isSpace matches, like '\t' and '\v'. Only the former would cause problems.
2019-02-20 18:22:31 +00:00
* Fix storage of metadata values containing newlines.
(Reversion introduced in version 7.20190122.)
optimisation: use graftTree to remember the export branch Sped up git-annex export in repositories with lots of keys. Old method read whole git-annex branch tree into memory.
2019-02-22 15:16:22 +00:00
* Sped up git-annex export in repositories with lots of keys.
aws-0.21.1 * S3: Support enabling bucket versioning when built with aws-0.21.1. * stack.yaml: Build with aws-0.21.1
2019-02-24 16:45:09 +00:00
* S3: Support enabling bucket versioning when built with aws-0.21.1.
* stack.yaml: Build with aws-0.21.1
Fix cleanup of git-annex:export.log after git-annex forget --drop-dead This log, unlike all other current top-level logs, is a new format log. I have not checked what throwing it at the old log parser did, but it seems likely it ignored unparsable lines, and so perhaps deleted all lines from the log.
2019-02-23 01:34:31 +00:00
* Fix cleanup of git-annex:export.log after git-annex forget --drop-dead.
Makefile: Added install-home target which installs git-annex into the HOME directory
2019-03-18 16:33:56 +00:00
* Makefile: Added install-home target which installs git-annex into
the HOME directory.
addurl --file: Fix a bug that made youtube-dl be used unneccessarily when adding an html url that does not contain any media.
2019-03-18 17:34:29 +00:00
* addurl --file: Fix a bug that made youtube-dl be used unneccessarily
when adding an html url that does not contain any media.
Add -- before %f in the smudge/clean filter configuration To support filenames starting with dashes. To update the config of existing repositories, you can re-run git-annex init. Perhaps it should check every time for the old config and update it, but that has several problems: - read-only repos - unexpected commands like `git annex find` changing git configs might be surprising behavior Since filenames starting with dashes are not super common and the user can re-init easily enough if their repo needs fixed, I went for the simplest fix.
2019-03-18 18:12:13 +00:00
* Add -- before %f in the smudge/clean filter configuration,
to support filenames starting with dashes.
(To update the config of existing repositories, you can
re-run git-annex init.)
fsck: Detect situations where annex.thin has caused data loss to the content of locked files. In particular, when two files had the same content, and one was unlocked and modified, with annex.thin that can corrupt the content of the annex object, and so fsck on the other file should detect that. getKeyStatus was relying on Database.Keys.getAssociatedFiles to tell when a file is unlocked, but that can false positive because the database can list old associated files. Instead, separate out the case of unlocked object which has multiple hardlinks when annex.thin is in use.
2019-03-18 19:53:54 +00:00
* fsck: Detect situations where annex.thin has caused data loss
to the content of locked files.
Removed bundled gpg from the Linux standalone build and OSX dmg Because gpg now always wants to use gpg-agent, and shipping such a daemon in those is not a good idea.
2019-03-18 20:31:07 +00:00
* Removed bundled gpg from the Linux standalone build and OSX dmg,
because gpg now always wants to use gpg-agent, and shipping such a daemon
in those is not a good idea.
import: Let --force overwrite symlinks, not only regular files The docs already implied this should work.
2019-03-18 20:40:15 +00:00
* import: Let --force overwrite symlinks, not only regular files.
Android: Fix typo of name of armv7l in installation script. Thanks, 4omecha.
2019-03-22 13:39:18 +00:00
* Android: Fix typo of name of armv7l in installation script.
Thanks, 4omecha.
S3: Added protocol= initremote setting, to allow https to be used on a non-standard port protocol=https implies port=443 and port=443 implies protocol=https -- this was necessary because the existing configs set port=443, but with a protocol setting, users will naturally want to use it, and then there's no need for them to supply the default https port. So we keep back-compat, add a nicer way to enable https, and also add support for non-standard https ports.
2019-03-22 16:17:05 +00:00
* S3: Added protocol= initremote setting, to allow https to be used
on a non-standard port.
fix encoding of metadata containing newlines This fixes a reversion in the ByteString conversion. The old code used isSpace to decide when the metadata value needs to be base64 encoded, and that incorrectly changed to only checking if it contained ' '. Note that only '\n' and '\r' were added and not other sorts of whitespace that isSpace matches, like '\t' and '\v'. Only the former would cause problems.
2019-02-20 18:22:31 +00:00
releasing package git-annex version 7.20190322
2019-03-22 17:57:17 +00:00
-- Joey Hess <id@joeyh.name> Fri, 22 Mar 2019 13:56:51 -0400
fix encoding of metadata containing newlines This fixes a reversion in the ByteString conversion. The old code used isSpace to decide when the metadata value needs to be base64 encoded, and that incorrectly changed to only checking if it contained ' '. Note that only '\n' and '\r' were added and not other sorts of whitespace that isSpace matches, like '\t' and '\v'. Only the former would cause problems.
2019-02-20 18:22:31 +00:00
releasing package git-annex version 7.20190219
2019-02-19 16:19:53 +00:00
git-annex (7.20190219) upstream; urgency=medium
fix false positive in export conflict detection Like the earlier fixed one in Command.Export, it occurred when the same tree was exported by multiple clones. Previous fix was incomplete since several other places looked at the list of exported trees to detect when there was an export conflict. Added a single unified function to avoid missing any places it needed to be fixed. This commit was sponsored by mo on Patreon.
2019-01-30 16:36:30 +00:00
releasing package git-annex version 7.20190219
2019-02-19 16:19:53 +00:00
* init: Fix bug when direct mode needs to be enabled on a crippled
filesystem, that left the repository in indirect mode.
fix false positive in export conflict detection Like the earlier fixed one in Command.Export, it occurred when the same tree was exported by multiple clones. Previous fix was incomplete since several other places looked at the list of exported trees to detect when there was an export conflict. Added a single unified function to avoid missing any places it needed to be fixed. This commit was sponsored by mo on Patreon.
2019-01-30 16:36:30 +00:00
* Fix false positive in export conflict detection, that occurred
when the same tree was exported by multiple clones. Previous fix was
incomplete.
export retrieval fallback to handle S3 remote with partially missing version IDs When key-based retrieval from a S3 remote with exporttree=yes appendonly=yes fails, fall back to trying to retrieve from the exported tree. This allows downloads of files that were exported to such a remote before versioning was enabled on it. This is useful at least for a transition for users who got into that situation, so they can download content from their S3 remote. May want to remove this in the future though, since normally trying to download the second time is only extra work. This commit was sponsored by Brock Spratlen on Patreon.
2019-01-30 17:23:03 +00:00
* When key-based retrieval from a S3 remote with exporttree=yes appendonly=yes
fails, fall back to trying to retrieve from the exported tree.
This allows downloads of files that were exported to such a remote
before versioning was enabled on it.
releasing package git-annex version 7.20190219
2019-02-19 16:19:53 +00:00
* Fix path separator bug on Windows that completely broke git-annex
since version 7.20190122.
Improved speed of S3 remote by only loading S3 creds once This gets back any speed lost in commit 9cebfd7002d1022b08fec083b6531742010b18d1, and speeds up all uses of S3 remotes that operate on them more than once. This commit was sponsored by Brett Eisenberg on Patreon.
2019-01-30 20:04:16 +00:00
* Improved speed of S3 remote by only loading S3 creds once.
Display progress bar when getting files from export remotes And moved the progress bar display into storeExport as well. This commit was sponsored by John Pellman on Patreon.
2019-01-31 17:34:12 +00:00
* Display progress bar when getting files from export remotes.
Fix race in cleanup of othertmp directory that could result in a failure attempting to access it. Need to create the directory after the lock is held, not before. The other racing process would need to shut down at just the wrong time, running cleanupOtherTmp. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2019-02-02 17:56:31 +00:00
* Fix race in cleanup of othertmp directory that could result in a failure
attempting to access it.
fromkey: Made idempotent If the worktree file already exists, and is annexed and uses the same key, avoid failing, nothing needs to be done. Had to add lookupFileNotHidden to handle the case where an adjust --hide-missing is in use, and the worktree file was hidden due to the object content being missing. lookupFile would return the key of the hidden file, but it makes sense that after fromkey succeeds, the worktree must contain the file it was supposed to set up.
2019-02-05 17:13:09 +00:00
* fromkey: Made idempotent.
fromkey --json * fromkey: Added --json. * fromkey --batch output changed to support using it with --json. The old output was not parseable for any useful information, so this is not expected to break anything.
2019-02-05 18:03:29 +00:00
* fromkey: Added --json.
* fromkey --batch output changed to support using it with --json.
The old output was not parseable for any useful information, so
this is not expected to break anything.
make .noannex file prevent repo fixups Avoid performing repository fixups for submodules and git-worktrees when there's a .noannex file that will prevent git-annex from being used in the repository. This change is ok as long as the .noannex file is really going to prevent git-annex from being used. But, init --force could override the file. Which would result in the repo being initialized without the fixups having run. To avoid that situation decided to change init, to not let --force be used to override a .noannex file. Instead the user can just delete the file.
2019-02-05 18:43:23 +00:00
* Avoid performing repository fixups for submodules and git-worktrees
when there's a .noannex file that will prevent git-annex from being
used in the repository.
* init: Don't let --force be used to override a .noannex file,
instead the user can just delete the file.
deal with attempt to export filename with # or ? to webdav xporting files with '#' or '?' in their name won't work because urls get truncated on those. Fail in a better way in this case, and avoid failing when removing such files from the export, so after the user has renamed the problem files the export will succeed.
2019-02-07 17:47:57 +00:00
* webdav: Exporting files with '#' or '?' in their name won't work because
urls get truncated on those. Fail in a better way in this case,
and avoid failing when removing such files from the export, so
after the user has renamed the problem files the export will succeed.
improve wording
2019-02-18 21:52:18 +00:00
* On Windows, avoid using rsync for local copies, since rsync is not
On Windows, avoid using rsync for file-to-file copies, since rsync is not always available there. Installing git-annex with stack rsync won't be available. Also, using the git-annex installer with 64 bit git installs a non-working rsync binary because it's linked with libraries provided by 32 bit git.
2019-02-18 21:27:34 +00:00
always available there.
releasing package git-annex version 7.20190219
2019-02-19 16:19:53 +00:00
* Added NetworkBSD build flag to deal with Network.BSD moving to a new
package.
fix false positive in export conflict detection Like the earlier fixed one in Command.Export, it occurred when the same tree was exported by multiple clones. Previous fix was incomplete since several other places looked at the list of exported trees to detect when there was an export conflict. Added a single unified function to avoid missing any places it needed to be fixed. This commit was sponsored by mo on Patreon.
2019-01-30 16:36:30 +00:00
releasing package git-annex version 7.20190219
2019-02-19 16:19:53 +00:00
-- Joey Hess <id@joeyh.name> Tue, 19 Feb 2019 11:12:45 -0400
fix false positive in export conflict detection Like the earlier fixed one in Command.Export, it occurred when the same tree was exported by multiple clones. Previous fix was incomplete since several other places looked at the list of exported trees to detect when there was an export conflict. Added a single unified function to avoid missing any places it needed to be fixed. This commit was sponsored by mo on Patreon.
2019-01-30 16:36:30 +00:00
releasing package git-annex version 7.20190129
2019-01-29 19:21:44 +00:00
git-annex (7.20190129) upstream; urgency=medium
S3: Send a Content-Type header when storing objects in S3 So exports to public buckets can be linked to from web pages. (When git-annex is built with MagicMime support.) Thanks to Jared Cosulich for the idea.
2019-01-23 17:08:47 +00:00
releasing package git-annex version 7.20190129
2019-01-29 19:21:44 +00:00
* initremote S3: When configured with versioning=yes, either ask the user
to enable bucket versioning, or auto-enable it when built with aws-0.22.
enableremote S3: Do not let versioning=yes be set on existing remote Because when git-annex lacks S3 version IDs for files stored in the bucket, deleting them would cause data loss. Also because git-annex is not able to download unversioned objects from a bucket when versioning=yes. This also prevents setting versioning=no. While that would perhaps be possible to do safely, it would add complexity, and would mean that if the user accidentially did enableremote versioning=no, they would not be able to undo it. This commit was sponsored by Trenton Cronholm on Patreon.
2019-01-29 18:08:42 +00:00
* enableremote S3: Do not let versioning=yes be set on existing remote,
because when git-annex lacks S3 version IDs for files stored in
the bucket, deleting them would cause data loss.
releasing package git-annex version 7.20190129
2019-01-29 19:21:44 +00:00
* S3: Detect when version=yes but an exported file lacks a S3 version ID,
S3: Detect when version=yes but an exported file lacks versioning, and refuse to delete it, to avoid data loss. This commit was sponsored by Denis Dzyubenko on Patreon.
2019-01-29 19:07:27 +00:00
and refuse to delete it, to avoid data loss.
releasing package git-annex version 7.20190129
2019-01-29 19:21:44 +00:00
* S3: Send a Content-Type header when storing objects in S3,
so exports to public buckets can be linked to from web pages.
(When git-annex is built with MagicMime support.)
S3: Send a Content-Type header when storing objects in S3 So exports to public buckets can be linked to from web pages. (When git-annex is built with MagicMime support.) Thanks to Jared Cosulich for the idea.
2019-01-23 17:08:47 +00:00
releasing package git-annex version 7.20190129
2019-01-29 19:21:44 +00:00
-- Joey Hess <id@joeyh.name> Tue, 29 Jan 2019 15:09:20 -0400
S3: Send a Content-Type header when storing objects in S3 So exports to public buckets can be linked to from web pages. (When git-annex is built with MagicMime support.) Thanks to Jared Cosulich for the idea.
2019-01-23 17:08:47 +00:00
releasing package git-annex version 7.20190122
2019-01-22 16:28:14 +00:00
git-annex (7.20190122) upstream; urgency=medium
sync --content: Fix dropping unwanted content from the local repository This fixes a bug with the numcopies counting when using sync --content. It did not always pass the local repo uuid to handleDropsFrom, and so the numcopies counting was off by one, and unwanted local content would only be dropped when there were numcopies+1 remote copies. Also, support dropping local content that has reached an exporttree remote that is not untrusted (currently only S3 remotes with versioning).
2018-12-18 17:58:12 +00:00
* sync --content: Fix dropping unwanted content from the local repository.
* sync --content: Support dropping local content that has reached an
exporttree remote that is not untrusted (currently only S3 remotes
with versioning).
honor init --version=5 on crippled filesystem init: When --version=5 is passed on a crippled filesystem, use a v5 direct mode repo as requested, rather than upgrading to v7 adjusted unlocked. Fixed test suite on crippled filesystems, making it request --version=5 to test direct mode.
2018-12-19 17:07:19 +00:00
* init: When --version=5 is passed on a crippled filesystem,
use a v5 direct mode repo as requested, rather than upgrading to v7
adjusted unlocked. (Fixes test suite on crippled filesystems.)
new improved benchmark command that can benchmark anything git-annex does
2019-01-04 17:43:53 +00:00
* Some optimisations, including a 10x faster timestamp parser,
make everything build again after ByteString Key changes
2019-01-11 20:34:04 +00:00
a 7x faster key parser, and improved parsing and serialization of
devblog
2019-01-14 23:00:38 +00:00
git-annex branch data. Many commands will run 5-15% faster.
make everything build again after ByteString Key changes
2019-01-11 20:34:04 +00:00
* Stricter parser for keys doesn't allow doubled fields or out of order fields.
new improved benchmark command that can benchmark anything git-annex does
2019-01-04 17:43:53 +00:00
* The benchmark command, which only had some old benchmarking of the sqlite
databases before, now allows benchmarking any other git-annex commands.
correct ghc version number
2019-01-05 20:07:53 +00:00
* Support being built with ghc 8.6.3 (MonadFail).
convert all per-uuid log files to use Builder Mostly didn't push the ByteStrings down very deep, but all of these log files are not written to frequently at all, so slight remaining innefficiency doesn't matter. In Logs.UUID, removed the fixBadUUID code that cleaned up after a bug in git-annex versions 3.20111105-3.20111110. In the unlikely event that a repo was last touched by that ancient git-annex version, the descriptions of remotes would appear missing when used with this version of git-annex. That is such minor breakage, and so unlikely to still be a problem for any repos, that it was not worth forward-porting that code to ByteString.
2019-01-09 18:00:35 +00:00
* Removed old code that cleaned up after a bug in git-annex versions
3.20111105-3.20111110. In the unlikely event that a repo was
last touched by that ancient git-annex version, the descriptions
of remotes would appear missing when used with this version of
git-annex.
convert old uuid-based log parsers to attoparsec This preserves the workaround for the old bug that caused NoUUID items to be stored in the log, prefixing log lines with " ". It's now handled implicitly, by using takeWhile1 (/= ' ') to get the uuid. There is a behavior change from the old parser, which split the value into words and then recombined it. That meant that "foo bar" and "foo\tbar" came out as "foo bar". That behavior was not documented, and seems surprising; it meant that after a git-annex describe here "foo bar", you wouldn't get that same string back out when git-annex displayed repo descriptions. Otoh, some other parsers relied on the old behavior, and the attoparsec rewrites had to deal with the issue themselves... For group.log, there are some edge cases around the user providing a group name with a leading or trailing space. The old parser would ignore such excess whitespace. The new parser does too, because the alternative is to refuse to parse something like " group1 group2 " due to excess whitespace, which would be even more confusing behavior. The only git-annex branch log file that is not converted to attoparsec and bytestring-builder now is transitions.log.
2019-01-10 18:39:36 +00:00
* Improve uuid.log parser to preserve whitespace in repo descriptions.
convert transitions.log to attoparsec and bytestring-builder Not likely to be any speed gain here, but this completes porting every log file over. And, it let me get rid of code copied from ghc and modified, so simplifying the licensing.
2019-01-10 21:13:30 +00:00
* Improve activity.log parser to not remove unknown values,
convert old uuid-based log parsers to attoparsec This preserves the workaround for the old bug that caused NoUUID items to be stored in the log, prefixing log lines with " ". It's now handled implicitly, by using takeWhile1 (/= ' ') to get the uuid. There is a behavior change from the old parser, which split the value into words and then recombined it. That meant that "foo bar" and "foo\tbar" came out as "foo bar". That behavior was not documented, and seems surprising; it meant that after a git-annex describe here "foo bar", you wouldn't get that same string back out when git-annex displayed repo descriptions. Otoh, some other parsers relied on the old behavior, and the attoparsec rewrites had to deal with the issue themselves... For group.log, there are some edge cases around the user providing a group name with a leading or trailing space. The old parser would ignore such excess whitespace. The new parser does too, because the alternative is to refuse to parse something like " group1 group2 " due to excess whitespace, which would be even more confusing behavior. The only git-annex branch log file that is not converted to attoparsec and bytestring-builder now is transitions.log.
2019-01-10 18:39:36 +00:00
allowing for future expansion.
avoid creating work tree files in subdirectories in an edge case A keyName could contain "/", though this is unlikely and certianly only ever could happen with WORM keys. The change to addunused to escape that is no problem at all. The change to VariantFile to escape it means that different versions of git-annex could resolve a merge conflict differently in this case, which is unfortunate. There would be different .variant files used, so the two resolutions would themselves merge together without additional conflicts, but the user would have to clean up the extra .variant files.
2019-01-14 17:07:18 +00:00
* addunused, merge, assistant: Avoid creating work tree files in
subdirectories in an edge case where the key contains "/".
testremote: Support testing readonly remotes with the --test-readonly option This commit was sponsored by Ilya Shlyakhter on Patreon.
2019-01-17 16:39:29 +00:00
* testremote: Support testing readonly remotes with the --test-readonly option.
misctmp cleanup * Switch to using .git/annex/othertmp for tmp files other than partial downloads, and make stale files left in that directory when git-annex is interrupted be cleaned up promptly by subsequent git-annex processes. * The .git/annex/misctmp directory is no longer used and git-annex will delete anything lingering in there after it's 1 week old. Also, in Annex.Ingest, made the filename it uses in the tmp dir be prefixed with "ingest-" to avoid potentially using a filename used by some other code.
2019-01-17 19:40:44 +00:00
* Switch to using .git/annex/othertmp for tmp files other than partial
downloads, and make stale files left in that directory when git-annex
is interrupted be cleaned up promptly by subsequent git-annex processes.
* The .git/annex/misctmp directory is no longer used and git-annex will
delete anything lingering in there after it's 1 week old.
Estimated time to completion display shortened from eg "1h1m1s" to "1h1m" Because seconds accuracy over such a time is unlikely to be accurate. Also, it was possible to get a ridiculous "1y1d1h1m1s" if stalled or very slow.
2019-01-21 00:00:05 +00:00
* Estimated time to completion display shortened from eg "1h1m1s" to "1h1m".
releasing package git-annex version 7.20190122
2019-01-22 16:28:14 +00:00
* Fix doubled progress display when downloading an url when -J is used.
* unused: Update suggested git log message to see where data was previously
used so it will also work with v7 unlocked pointer files.
* importfeed: Better error message when downloading the feed fails.
* Make test suite work better when the temp directory is on NFS.
webapp: remove configurators for obsolste cloud storage services * webapp: Remove configurator for box.com repository, since their webdav support is going away at the end of this January. * webapp: Remove configurator for gitlab, which stopped supporting git-annex some time ago. This commit was sponsored by Brock Spratlen on Patreon.
2019-01-22 15:48:35 +00:00
* webapp: Remove configurator for box.com repository, since their
webdav support is going away at the end of this January.
* webapp: Remove configurator for gitlab, which stopped supporting git-annex
some time ago.
Android: For armv71 architecture, use the armel build This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2019-01-22 15:50:29 +00:00
* Android: For armv71 architecture, use the armel build.
releasing package git-annex version 7.20190122
2019-01-22 16:28:14 +00:00
* Windows: If 64 bit git is installed, use it when installing git-annex.
(However, rsync still won't work and this is still not the documented way
to install it.)
sync --content: Fix dropping unwanted content from the local repository This fixes a bug with the numcopies counting when using sync --content. It did not always pass the local repo uuid to handleDropsFrom, and so the numcopies counting was off by one, and unwanted local content would only be dropped when there were numcopies+1 remote copies. Also, support dropping local content that has reached an exporttree remote that is not untrusted (currently only S3 remotes with versioning).
2018-12-18 17:58:12 +00:00
releasing package git-annex version 7.20190122
2019-01-22 16:28:14 +00:00
-- Joey Hess <id@joeyh.name> Tue, 22 Jan 2019 12:25:26 -0400
sync --content: Fix dropping unwanted content from the local repository This fixes a bug with the numcopies counting when using sync --content. It did not always pass the local repo uuid to handleDropsFrom, and so the numcopies counting was off by one, and unwanted local content would only be dropped when there were numcopies+1 remote copies. Also, support dropping local content that has reached an exporttree remote that is not untrusted (currently only S3 remotes with versioning).
2018-12-18 17:58:12 +00:00
releasing package git-annex version 7.20181211
2018-12-11 20:33:30 +00:00
git-annex (7.20181211) upstream; urgency=medium
S3: Improve diagnostics when a remote is configured with exporttree and versioning, but no S3 version id has been recorded for a key. When public access is used for the remote, it complained that the user needed to set creds to use it, which was just wrong. When creds were being used, it fell back from trying to use the version ID to just accessing the key in the bucket, which was ok for non-export remotes, but wrong for buckets. In both cases, display a hopefully useful warning. This should only come up when an existing S3 remote has been exported to, and then later versioning was enabled. Note that it would perhaps be possible to fall back from trying to use retrieveKeyFile when it fails and instead use retrieveKeyFileFromExport, which may work when S3 version ID is missing. But there are problems with that approach; how to tell when retrieveKeyFile has failed due to this rather than a network problem etc? Anyway, that approach would only work until the file in the export got overwritten, and then it would no longer be accessible. And with versioning enabled, the user wants old versions of objects to remain accessible, so it seems better to warn about the problem as soon as possible, so they can go back and add S3 version IDs. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-12-06 17:43:18 +00:00
* S3: Improve diagnostics when a remote is configured with exporttree and
versioning, but no S3 version id has been recorded for a key.
support findred and --branch with file matching options * findref: Support file matching options: --include, --exclude, --want-get, --want-drop, --largerthan, --smallerthan, --accessedwithin * Commands supporting --branch now apply file matching options --include, --exclude, --want-get, --want-drop to filenames from the branch. Previously, combining --branch with those would fail to match anything. * add, import, findref: Support --time-limit. This commit was sponsored by Jake Vosloo on Patreon.
2018-12-09 17:38:35 +00:00
* findref: Support file matching options: --include, --exclude,
--want-get, --want-drop, --largerthan, --smallerthan, --accessedwithin
* Commands supporting --branch now apply file matching options --include,
--exclude, --want-get, --want-drop to filenames from the branch.
Previously, combining --branch with those would fail to match anything.
* add, import, findref: Support --time-limit.
add --branch option to git-annex find and mildly deprecate findref in favor of it No deprecation warning at run time, just one on the man page. One thing findref remains able to do that find cannot is to run in a bare repo. Find was made to refuse to run in a bare repo because it seemed confusing for it to not list any files ever in that situation. It would be better for find --branch to work in a bare repo but not without --branch but I don't currently have a way to do that. Probably a better solution would be to make git-annex in a bare repo default to --branch master or something like that instead of --all. This commit was sponsored by Denis Dzyubenko on Patreon.
2018-12-09 18:10:37 +00:00
* Add --branch option to git-annex find and mildly deprecate findref in
favor of it.
fix webdav reversion webdav: When initializing, avoid trying to make a directory at the top of the webdav server, which could never accomplish anything and failed on nextcloud servers. (Reversion introduced in version 6.20170925.) This commit was sponsored by mo on patreon.
2018-12-10 16:49:51 +00:00
* webdav: When initializing, avoid trying to make a directory at the top of
the webdav server, which could never accomplish anything and failed on
nextcloud servers. (Reversion introduced in version 6.20170925.)
Fix a case where upgrade to v7 caused git to think that unlocked files were modified When a file was already unlocked, but the annex object was present, the upgrade process populated the unlocked file, but neglected to update the index. This commit was sponsored by Jochen Bartl on Patreon.
2018-12-11 17:05:03 +00:00
* Fix a case where upgrade to v7 caused git to think that unlocked files
were modified.
fix bugs involving v7 unlocked files and direct mode * Fix bug upgrading from direct mode to v7: when files in the repository were already committed as v7 unlocked files elsewhere, and the content was present in the direct mode repository, the annexed files got their full content checked into git. * Fix bug that caused v7 unlocked files in a direct mode repository to get locked when committing. This commit was sponsored by Nick Piper on Patreon.
2018-12-11 17:46:22 +00:00
* Fix bug upgrading from direct mode to v7: when files in the repository
were already committed as v7 unlocked files elsewhere, and the
content was present in the direct mode repository, the annexed files
got their full content checked into git.
* Fix bug that caused v7 unlocked files in a direct mode repository
to get locked when committing.
S3: Improve diagnostics when a remote is configured with exporttree and versioning, but no S3 version id has been recorded for a key. When public access is used for the remote, it complained that the user needed to set creds to use it, which was just wrong. When creds were being used, it fell back from trying to use the version ID to just accessing the key in the bucket, which was ok for non-export remotes, but wrong for buckets. In both cases, display a hopefully useful warning. This should only come up when an existing S3 remote has been exported to, and then later versioning was enabled. Note that it would perhaps be possible to fall back from trying to use retrieveKeyFile when it fails and instead use retrieveKeyFileFromExport, which may work when S3 version ID is missing. But there are problems with that approach; how to tell when retrieveKeyFile has failed due to this rather than a network problem etc? Anyway, that approach would only work until the file in the export got overwritten, and then it would no longer be accessible. And with versioning enabled, the user wants old versions of objects to remain accessible, so it seems better to warn about the problem as soon as possible, so they can go back and add S3 version IDs. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-12-06 17:43:18 +00:00
releasing package git-annex version 7.20181211
2018-12-11 20:33:30 +00:00
-- Joey Hess <id@joeyh.name> Tue, 11 Dec 2018 16:33:10 -0400
S3: Improve diagnostics when a remote is configured with exporttree and versioning, but no S3 version id has been recorded for a key. When public access is used for the remote, it complained that the user needed to set creds to use it, which was just wrong. When creds were being used, it fell back from trying to use the version ID to just accessing the key in the bucket, which was ok for non-export remotes, but wrong for buckets. In both cases, display a hopefully useful warning. This should only come up when an existing S3 remote has been exported to, and then later versioning was enabled. Note that it would perhaps be possible to fall back from trying to use retrieveKeyFile when it fails and instead use retrieveKeyFileFromExport, which may work when S3 version ID is missing. But there are problems with that approach; how to tell when retrieveKeyFile has failed due to this rather than a network problem etc? Anyway, that approach would only work until the file in the export got overwritten, and then it would no longer be accessible. And with versioning enabled, the user wants old versions of objects to remain accessible, so it seems better to warn about the problem as soon as possible, so they can go back and add S3 version IDs. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-12-06 17:43:18 +00:00
releasing package git-annex version 7.20181205
2018-12-05 20:19:11 +00:00
git-annex (7.20181205) upstream; urgency=medium
Make bittorrent special remote work w/o btshowmetainfo installed when it was build with torrentparser. Thanks, Robert Schütz
2018-11-27 16:31:06 +00:00
* Make bittorrent special remote work w/o btshowmetainfo installed
when it was build with torrentparser.
Thanks, Robert Schütz
When running youtube-dl to get a filename, pass --no-playlist Seems that youtube-dl --get-filename on a playlist lists all the filenames for the playlist, which can take quite some time. The code already only took the first name, so --no-playlist can speed it up a lot. This commit was sponsored by Brett Eisenberg on Patreon.
2018-11-28 21:14:47 +00:00
* When running youtube-dl to get a filename, pass --no-playlist.
Fix build without concurrent-output. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2018-12-03 16:33:00 +00:00
* Fix build without concurrent-output.
fix init in cripped filesystem version issues * init: When a crippled filesystem causes an adjusted unlocked branch to be used, set repo version to 7, which it neglected to do before. * init: When on a crippled filesystem, and the git version is too old to use an adjusted unlocked branch, fall back to using direct mode. This commit was sponsored by Ilya Shlyakhter on Patreon.
2018-12-03 16:57:23 +00:00
* init: When a crippled filesystem causes an adjusted unlocked branch to
be used, set repo version to 7, which it neglected to do before.
* init: When on a crippled filesystem, and the git version is too old
to use an adjusted unlocked branch, fall back to using direct mode.
add exportedtree to info info: When used with an exporttree remote, includes an "exportedtree" info, which is the tree last exported to the remote. During an export conflict, multiple values will be listed. This commit was sponsored by John Pellman on Patreon.
2018-12-03 18:15:15 +00:00
* info: When used with an exporttree remote, includes an "exportedtree"
info, which is the tree last exported to the remote. During an export
conflict, multiple values will be listed.
dropunused edge case when annex.thin caused unused object to be modified dropunused: When an unused object file has gotten modified, eg due to annex.thin being set, don't silently skip it, but display a warning and let --force drop it. This commit was sponsored by Ethan Aubin.
2018-12-04 16:20:34 +00:00
* dropunused: When an unused object file has gotten modified, eg due to
annex.thin being set, don't silently skip it, but display a warning
and let --force drop it.
annex.cachecreds: New config to allow disabling of credentials caching for special remotes. Note that it does not prevent storing p2p access tokens or multicast encryption keys, since those are not cached; the previous commit established the distinction. How well this works depends on how often getRemoteCredPair is called and how expensive it is. In some cases setting this will result in an annoying number of gpg password prompts and/or slowdowns due to reading creds from the git-annex branch and decrypting, which could be improved by calling getRemoteCredPair less often. This commit was sponsored by Ilya Shlyakhter on Patreon.
2018-12-04 18:16:56 +00:00
* annex.cachecreds: New config to allow disabling of credentials caching
for special remotes.
Make bittorrent special remote work w/o btshowmetainfo installed when it was build with torrentparser. Thanks, Robert Schütz
2018-11-27 16:31:06 +00:00
releasing package git-annex version 7.20181205
2018-12-05 20:19:11 +00:00
-- Joey Hess <id@joeyh.name> Wed, 05 Dec 2018 14:02:27 -0400
Make bittorrent special remote work w/o btshowmetainfo installed when it was build with torrentparser. Thanks, Robert Schütz
2018-11-27 16:31:06 +00:00
releasing package git-annex version 7.20181121
2018-11-21 18:24:04 +00:00
git-annex (7.20181121) upstream; urgency=medium
git-annex-shell: fix transfer hang Fix hang when transferring the same objects to two different clients at the same time. (Or when annex.pidlock is used, two different objects to the same or different clients.) Could also potentially occur if a client was downloading an object and somehow lost connection but that git-annex-shell was still running and holding the transfer lock. This does not guarantee that, if `transfer` fails for some other reason, a DATA response will be made. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-06 17:00:25 +00:00
* git-annex-shell: Fix hang when transferring the same objects to two
different clients at the same time. (Or when annex.pidlock is used,
two different objects.)
Fixed some other potential hangs in the P2P protocol Finishes the start made in 983c9d5a53189f71797591692c0ed675f5bd1c16, by handling the case where `transfer` fails for some other reason, and so the ReadContent callback does not get run. I don't know of a case where `transfer` does fail other than the locking dealt with in that commit, but it's good to have a guarantee. StoreContent and StoreContentTo had a similar problem. Things like `getViaTmp` may decide not to run the transfer action. And `transfer` could certianly fail, if another transfer of the same object was in progress. (Or a different object when annex.pidlock is set.) If the transfer action was not run, the content of the object would not all get consumed, and so would get interpreted as protocol commands, which would not go well. My approach to fixing all of these things is to set a TVar only once all the data in the transfer is known to have been read/written. This way the internals of `transfer`, `getViaTmp` etc don't matter. So in ReadContent, it checks if the transfer completed. If not, as long as it didn't throw an exception, send empty and Invalid data to the callback. On an exception the state of the protocol is unknown so it has to raise ProtoFailureException and close the connection, same as before. In StoreContent, if the transfer did not complete some portion of the DATA has been read, so the protocol is in an unknown state and it has to close the conection as well. (The ProtoFailureMessage used here matches the one in Annex.Transfer, which is the most likely reason. Not ideal to duplicate it..) StoreContent did not ever close the protocol connection before. So this is a protocol change, but only in an exceptional circumstance, and it's not going to break anything, because clients already need to deal with the connection breaking at any point. The way this new behavior looks (here origin has annex.pidlock = true so will only accept one upload to it at a time): git annex copy --to origin -J2 copy x (to origin...) ok copy y (to origin...) Lost connection (fd:25: hGetChar: end of file) This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-06 18:44:00 +00:00
* Fixed some other potential hangs in the P2P protocol.
bash completion fix Fix bash completion of "git annex" to propertly handle files with spaces and other problem characters. (Completion of "git-annex" already did.) This commit was sponsored by Jake Vosloo on Patreon.
2018-11-12 17:23:05 +00:00
* Fix bash completion of "git annex" to propertly handle files with
spaces and other problem characters. (Completion of "git-annex"
already did.)
Fix resume of download of url when the whole file content is already actually downloaded Don't much like that there's no way to distinguish between having the whole content and having an old version of the file that's bigger, but of course resuming a http transfer can always yield the wrong result if the file on the http server is changing, and git-annex will detect that when it verifies the downloaded content. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-12 20:08:47 +00:00
* Fix resume of download of url when the whole file content is
already actually downloaded.
improve messages around export conflicts When an export conflict prevents accessing a special remote, be clearer about what the problem is and how to resolve it. This commit was sponsored by Trenton Cronholm on Patreon.
2018-11-13 19:50:06 +00:00
* When an export conflict prevents accessing a special remote,
be clearer about what the problem is and how to resolve it.
avoid redundant export uploads export, sync --content: Avoid unnecessarily trying to upload files to an exporttree remote that already contains the files. When the export was origianly made in one repo and now git-annex is running in a different repo, the export database is not yet populated with information about the exportLocation of files. So, it was trying to upload the files to the export, even when it already contained them. sync --content would first download the content from the export, and then re-upload the content back. And this also led to "not available" failures for each file that was not locally present yet. Fix: Just use checkPresentExport before uploading; if it succeeds update the database. This is a surprising oversight, it's possible it fixes a reversion because I would have thought I'd have noticed this problem when originally developing exporttree remotes. This commit was sponsored by Jochen Bartl on Patreon.
2018-11-14 15:47:40 +00:00
* export, sync --content: Avoid unnecessarily trying to upload files
to an exporttree remote that already contains the files.
improve smudge --clean behavior on outside work tree files smudge: When passed a file located outside the working tree, eg by git diff, avoid erroring out. This commit was sponsored by Ewen McNeill on Patreon.
2018-11-15 17:04:40 +00:00
* smudge: When passed a file located outside the working tree, eg by git
diff, avoid erroring out.
check onlyActionOn in Drop * drop -J: Avoid processing the same key twice at the same time when multiple annexes files use it. This prevents a drop of a key conflicting with another drop of the same key. This commit was sponsored by Brock Spratlen on Patreon.
2018-11-15 19:43:51 +00:00
* drop -J: Avoid processing the same key twice at the same time when
typo
2018-11-20 16:02:21 +00:00
multiple annexed files use it.
catch all (non-async) exceptions when running a commandAction When a command is operating on multiple files and there's an error with one, try harder to continue to the rest. (As was already done for many types of errors including IO errors.) This handles cases like lockContentForRemoval throwing an exception when the content is already locked. Just because a drop of one file fails, does not mean it shouldn't go on to try to drop other files. I looked over uses of `giveup` in Command/*; there are too many to check them all extensively, but none stood out as being problems that should let one commandAction stop running other commandActions. Worst case, something bad will happen and rather than stopping right away with an error, git-annex will display multiple errors as it fails over and over on each file. I don't think I ever really intended `error`/`giveup` to stop other commandActions; this was a relic of old confusion over haskell exception handling. Test suite passes. This commit was sponsored by Ethan Aubin.
2018-11-15 19:59:43 +00:00
* When a command is operating on multiple files and there's an error
with one, try harder to continue to the rest. (As was already done
for many types of errors including IO errors.)
changelog for now fixed crash
2018-11-19 22:59:45 +00:00
* Fixed a crash when using -J with ssh password prompts in
--quiet/--json mode.
Meld ReproNim into Yarik/DataLad's identity
2018-11-21 16:33:38 +00:00
Thanks to Yaroslav Halchenko and the DataLad&ReproNim team for
helping to track down this bug.
update
2018-11-10 16:30:39 +00:00
* Remove esqueleto dependency to allow upgrading other dependencies to
newer versions.
Thanks Sean Parsons.
Merge branch 'remove-esqueleto'
2018-11-20 15:50:04 +00:00
* Fix build with persistent-sqlite older than 2.6.3.
Updated stack.yaml to lts-12.19 And added stack-lts-9.9.yaml to support old versions of stack. The i386 ancient autobuilder needs stack-lts-9.9.yaml; the OSX autobuilder may also use it for a while, and it's needed to build on eg debian stable.
2018-11-20 18:00:02 +00:00
* Updated stack.yaml to lts-12.19; added stack-lts-9.9.yaml
to support old versions of stack.
git-annex-shell: fix transfer hang Fix hang when transferring the same objects to two different clients at the same time. (Or when annex.pidlock is used, two different objects to the same or different clients.) Could also potentially occur if a client was downloading an object and somehow lost connection but that git-annex-shell was still running and holding the transfer lock. This does not guarantee that, if `transfer` fails for some other reason, a DATA response will be made. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-06 17:00:25 +00:00
releasing package git-annex version 7.20181121
2018-11-21 18:24:04 +00:00
-- Joey Hess <id@joeyh.name> Wed, 21 Nov 2018 14:22:47 -0400
git-annex-shell: fix transfer hang Fix hang when transferring the same objects to two different clients at the same time. (Or when annex.pidlock is used, two different objects to the same or different clients.) Could also potentially occur if a client was downloading an object and somehow lost connection but that git-annex-shell was still running and holding the transfer lock. This does not guarantee that, if `transfer` fails for some other reason, a DATA response will be made. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-06 17:00:25 +00:00
releasing package git-annex version 7.20181105
2018-11-05 17:33:36 +00:00
git-annex (7.20181105) upstream; urgency=medium
Increase minimum QuickCheck version.
2018-10-31 19:53:22 +00:00
Fix test suite failure when git-annex test is not run inside a git repository Not the first time this kind of test suite breakage has happened.. It would be good to avoid somehow it looking up from .t and finding a git repo. But just running the test suite from time to time outside of git-annex would also let me notice these before the distribution packagers do. This commit was sponsored by mo on Patreon.
2018-11-05 17:27:16 +00:00
* Fix test suite failure when git-annex test is not run inside a git
repository.
Fix a P2P protocol hang When readContent got Nothing from prepSendAnnex, it did not run its callback, and the callback is what sends the DATA reply. sendContent checks with contentSize that the object file is present, but that doesn't really guarantee that prepSendAnnex won't return Nothing. So, it was possible for a P2P protocol GET to not receive a response, and appear to hang. When what it's really doing is waiting for the next protocol command. This seems most likely to happen when the annex is in direct mode, and the file being requested has been modified. It could also happen in an indirect mode repository if genInodeCache somehow failed. Perhaps due to a race with a drop of the content file. Fixed by making readContent behave the way its spec said it should, and run the callback with L.empty in this case. Note that, it's finee for readContent to send any amount of data to the callback, including L.empty. sendBytes deals with that by making sure it sends exactly the specified number of bytes, aborting the protocol if it's too short. So, when L.empty is sent, the protocol will end up aborting. This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-02 17:41:50 +00:00
* Fix a P2P protocol hang.
importfeed: Avoid erroring out when a feed has been repeatedly broken That can leave other imported files not checked into git, because the git command queue is not flushed when git-annex errors out. And since it only happens once git-annex has concluded a feed is broken, it's an intermittent bug, worst kind. Been seeing it for a while, only tracked down today. Instead, by returning False, git-annex importfeed will cleanly shutdown and still exit nonzero. This commit was sponsored by Denis Dzyubenko on Patreon.
2018-11-04 21:41:49 +00:00
* importfeed: Avoid erroring out when a feed has been repeatedly broken,
as that can leave other imported files not checked into git.
releasing package git-annex version 7.20181105
2018-11-05 17:33:36 +00:00
* Increase minimum QuickCheck version.
Increase minimum QuickCheck version.
2018-10-31 19:53:22 +00:00
releasing package git-annex version 7.20181105
2018-11-05 17:33:36 +00:00
-- Joey Hess <id@joeyh.name> Mon, 05 Nov 2018 13:31:09 -0400
Increase minimum QuickCheck version.
2018-10-31 19:53:22 +00:00
releasing package git-annex version 7.20181031
2018-10-31 19:46:57 +00:00
git-annex (7.20181031) upstream; urgency=medium
removed the old Android app Running git-annex linux builds in termux seems to work well enough that the only reason to keep the Android app would be to support Android 4-5, which the old Android app supported, and which I don't know if the termux method works on (although I see no reason why it would not). According to [1], Android 4-5 remains on around 29% of devices, down from 51% one year ago. [1] https://www.statista.com/statistics/271774/share-of-android-platforms-on-mobile-devices-with-android-os/ This is a rather large commit, but mostly very straightfoward removal of android ifdefs and patches and associated cruft. Also, removed support for building with very old ghc < 8.0.1, and with yesod < 1.4.3, and without concurrent-output, which were only being used by the cross build. Some documentation specific to the Android app (screenshots etc) needs to be updated still. This commit was sponsored by Brett Eisenberg on Patreon.
2018-10-13 05:36:06 +00:00
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
* Added v7 repository mode. v6 upgrades automatically to v7, but
v5 is still the default for now. While v6 was always experimental
to some degree, its successor v7 is ready for production use!
releasing package git-annex version 7.20181031
2018-10-31 19:46:57 +00:00
http://git-annex.branchable.com/tips/unlocked_files/
fix autoupgrade from v6 to go to v7, not v5 v3 and v4 still autoupgrade to v5 And a few more upgrade doc updates.
2018-10-25 22:33:34 +00:00
* Direct mode repositories are deprecated; they have many problems
that v7 fixes, so upgrading them now is recommended (but not yet
required): git annex upgrade --version=7
default to v7 adjusted unlocked in crippled filesystem init: When in a crippled filesystem, initialize a v7 repository using an adjusted unlocked branch, instead of a direct mode repository. Direct mode is deprecated, so this makes sense to do already I hope. This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-10-25 22:49:57 +00:00
* init: When in a crippled filesystem, initialize a v7 repository
using an adjusted unlocked branch, instead of a direct mode repository.
adjust --hide-missing * At long last there's a way to hide annexed files whose content is missing from the working tree: git-annex adjust --hide-missing * When already in an adjusted branch, running git-annex adjust again will update the branch as needed. This is mostly useful with --hide-missing to hide/unhide files after their content has been dropped or received. Still needs integration with sync and the assistant, and not as fast as it could be, but already usable. This commit was sponsored by Ethan Aubin.
2018-10-18 19:32:42 +00:00
* At long last there's a way to hide annexed files whose content
is missing from the working tree: git-annex adjust --hide-missing
link to tip
2018-10-20 18:25:03 +00:00
See https://git-annex.branchable.com/tips/hiding_missing_files/
adjust --hide-missing * At long last there's a way to hide annexed files whose content is missing from the working tree: git-annex adjust --hide-missing * When already in an adjusted branch, running git-annex adjust again will update the branch as needed. This is mostly useful with --hide-missing to hide/unhide files after their content has been dropped or received. Still needs integration with sync and the assistant, and not as fast as it could be, but already usable. This commit was sponsored by Ethan Aubin.
2018-10-18 19:32:42 +00:00
* When already in an adjusted branch, running git-annex adjust
again will update the branch as needed. This is mostly
useful with --hide-missing to hide/unhide files after their content
has been dropped or received.
sync --content now supports --hide-missing adjusted branches This relies on git ls-files --with-tree, which I'm using in a way that its man page does not document. Hm. I emailed the git list to try to get the docs improved, but at least the git test suite does test the same kind of use case I'm using here. Performance impact when not in an adjusted branch is limited to some additional MVar accesses, and a single git call to determine the name of the current branch. So very minimal. When in an adjusted branch, the performance impact is in Annex.WorkTree.lookupFile, which starts doing an equal amount of work for files that didn't exist as it already did for files that were unlocked. This commit was sponsored by Jochen Bartl on Patreon.
2018-10-19 21:51:25 +00:00
* git-annex sync --content supports --hide-missing; it can
make sync update adjusted branch to hide/unhide This completes initial support for --hide-missing, although the assistant still needs to be updated and it perhaps needs to be sped up, and maybe there needs to be a way for git-annex get to operate on missing files. Opened some more todos for those things. This commit was sponsored by Henrik Riomar.
2018-10-20 18:12:04 +00:00
be used to get the content of hidden files, and it updates the
adjusted branch to hide/unhide files as necessary.
defer updating unlocked files until after smudge filter The smuge filter no longer provides git with annexed file content, to avoid a git memory leak, and because that did not honor annex.thin. git annex smudge --update has to be run after a checkout to update unlocked files in the working tree with annexed file contents. No hooks yet to run it. This commit was sponsored by Nick Piper on Patreon.
2018-10-25 18:43:13 +00:00
* smudge: The smudge filter no longer provides git with annexed
file content, to avoid a git memory leak, and because that did not
honor annex.thin. Now git annex smudge --update has to be run
after a checkout to update unlocked files in the working tree
with annexed file contents.
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
* v7 init, upgrade: Install git post-checkout and post-merge hooks that run
git post-checkout and post-merge hooks * init, upgrade: Install git post-checkout and post-merge hooks that run git annex smudge --update. * precommit: Run git annex smudge --update, because the post-merge hook is not run when there is a merge conflict. So the work tree will be updated when a commit is made to resolve the merge conflict. * precommit: Run git annex smudge --update, because the post-merge hook is not run when there is a merge conflict. So the work tree will be updated when a commit is made to resolve the merge conflict. * Note that git has no hooks run after git stash or git cherry-pick, so the user will have to manually run git annex smudge --update after such commands. Nothing currently installs the hooks into v6 repos that already exist. Something will need to be done about that, either move this behavior to v7, or document that the user will need to manually fix up their v6 repos. This commit was sponsored by Eric Drechsel on Patreon.
2018-10-25 19:40:12 +00:00
git annex smudge --update.
* precommit: Run git annex smudge --update, because the post-merge
hook is not run when there is a merge conflict. So the work tree will
be updated when a commit is made to resolve the merge conflict.
* Note that git has no hooks run after git stash or git cherry-pick,
so the user will have to manually run git annex smudge --update
after such commands.
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
* Removed the old Android app.
* Removed support for building with very old ghc < 8.0.1,
and with yesod < 1.4.3, and without concurrent-output,
which were only being used for the Android cross build.
* Webapp: Fix termux detection.
* runshell: Use system locales when built with
GIT_ANNEX_PACKAGE_INSTALL set. (For Neurodebian packages.)
* Fix database inconsistency that could cause git-annex to
get confused about whether a locked file's content was present.
* Fix concurrency bug that occurred on the first download from an
exporttree remote.
snap --version= to auto-upgrade This makes --version=6 still work, despite v6 not being in supportedVersions. Which is useful for scripts that use it. I didn't document it on the man page, because it's indistinguishable from an automatic upgrade after initting as v6.
2018-10-26 15:44:05 +00:00
* init --version=6 will still work, but the repository is auto-upgraded
immediately to v7.
When annex.thin is set, allow hard links to be made between executable work tree files and annex objects. This is safe, because while the annex object ends up executable, there were already at least two other cases where it ended up executable: 1. git add an an executable file 2. chmod +x of a a non-executable worktree file that was hard linked to the annex object After copy/hard link, it always fixes up the permissions to match the mode of the worktree file, so when an executable annex object gets hard linked to a non-executable worktree file, its execute bit gets removed. Commit b7c8bf5274a64389ac87d6ce0388b8708c261971 already *said* it would do this; I suspect the line of code I've removed was included in that commit accidentially. Also improves annex.thin documentation. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2018-10-26 17:35:13 +00:00
* When annex.thin is set, allow hard links to be made between executable
work tree files and annex objects.
remove CHECKURL-MULTI single url response special case Removed undocumented special case in handling of a CHECKURL-MULTI response with only a single file listed. Rather than ignoring the url that was in the response, use it. This allows external special remotes that want to provide some better url to do so, although I don't entirely agree with using CHECKURL-MULTI to accomplish that. I'm more of the feeling that an undocumented special case that throws data away is just not a good idea. This could in theory break some external special remote program that relied on the current behavior, but its seems unlikely that it would because such a program must already handle the multiple url case, unless it only ever provides a single url response to CHECKURL-MULTI. Make addurl --file work with a single item CHECKURL-MULTI response. It already did for external special remotes due to the special case, but now it also will for builtin ones like the BitTorrent special remote. This commit was sponsored by Ilya Shlyakhter on Patron.
2018-10-29 18:41:41 +00:00
* addurl: Removed undocumented special case in handling of a CHECKURL-MULTI
response with only a single file listed. Rather than ignoring the url that
was in the response, use it.
webapp: Fixed a crash when adding a git remote. Reversion introduced in 2b66492d6e which added a new cache that needs to be cleared.
2018-10-29 20:01:08 +00:00
* webapp: Fixed a crash when adding a git remote.
(Reversion introduced in version 6.20180112)
migrate: Fix failure to migrate from URL keys. (Reversion introduced in version 6.20180926)
2018-10-29 20:26:43 +00:00
* migrate: Fix failure to migrate from URL keys.
(Reversion introduced in version 6.20180926)
high-res mtimes Cache high-resolution mtimes for improved detection of modified files in v7 (and direct mode). Including on Windows. With back-compat support so old low-res mtimes won't break anything, and so the new information also won't break old versions of git-annex.
2018-10-30 04:40:17 +00:00
* Cache high-resolution mtimes for improved detection of modified files
in v7 (and direct mode).
removed the old Android app Running git-annex linux builds in termux seems to work well enough that the only reason to keep the Android app would be to support Android 4-5, which the old Android app supported, and which I don't know if the termux method works on (although I see no reason why it would not). According to [1], Android 4-5 remains on around 29% of devices, down from 51% one year ago. [1] https://www.statista.com/statistics/271774/share-of-android-platforms-on-mobile-devices-with-android-os/ This is a rather large commit, but mostly very straightfoward removal of android ifdefs and patches and associated cruft. Also, removed support for building with very old ghc < 8.0.1, and with yesod < 1.4.3, and without concurrent-output, which were only being used by the cross build. Some documentation specific to the Android app (screenshots etc) needs to be updated still. This commit was sponsored by Brett Eisenberg on Patreon.
2018-10-13 05:36:06 +00:00
releasing package git-annex version 7.20181031
2018-10-31 19:46:57 +00:00
-- Joey Hess <id@joeyh.name> Wed, 31 Oct 2018 09:21:50 -0400
removed the old Android app Running git-annex linux builds in termux seems to work well enough that the only reason to keep the Android app would be to support Android 4-5, which the old Android app supported, and which I don't know if the termux method works on (although I see no reason why it would not). According to [1], Android 4-5 remains on around 29% of devices, down from 51% one year ago. [1] https://www.statista.com/statistics/271774/share-of-android-platforms-on-mobile-devices-with-android-os/ This is a rather large commit, but mostly very straightfoward removal of android ifdefs and patches and associated cruft. Also, removed support for building with very old ghc < 8.0.1, and with yesod < 1.4.3, and without concurrent-output, which were only being used by the cross build. Some documentation specific to the Android app (screenshots etc) needs to be updated still. This commit was sponsored by Brett Eisenberg on Patreon.
2018-10-13 05:36:06 +00:00
releasing package git-annex version 6.20181011
2018-10-11 17:50:53 +00:00
git-annex (6.20181011) upstream; urgency=medium
sync: Warn when a remote's export is not updated to the current tree because export tracking is not configured. Only display the warning when the current branch has a tree that is not the same as the tree in the export. Note that it doesn't check to see if the current tree is in incompleteExportedTreeish; it might be worth checking that and reminding the user about an incomplete export, but when export tracking is not configured, they are probably not in the right clone of the repository to resolve the incomplete export. This commit was sponsored by Ethan Aubin.
2018-09-27 19:35:46 +00:00
* sync: Warn when a remote's export is not updated to the current
tree because export tracking is not configured.
Improve display when git config download from a http remote fails. The error message displayed used to only come from curl/wget and perhaps was clearer than the one displayed now that http-client is used. In any case, it does make sense to hide it because git-annex prints its own warning message. This commit was sponsored by Jake Vosloo on Patreon.
2018-10-03 16:31:09 +00:00
* Improve display when git config download from a http remote fails.
annex.jobs Added annex.jobs setting, which is like using the -J option. Of course, -J overrides annex.jobs. This commit was sponsored by Trenton Cronholm on Patreon.
2018-10-04 16:47:27 +00:00
* Added annex.jobs setting, which is like using the -J option.
Fix reversion in support of annex.web-options Inverted logic added as part of the url security fix made it always use curl when annex.security.allowed-http-addresses=all unless annex.web-options was set. That nobody noticed kind of makes me wonder if anyone uses annex.web-options.. This commit was sponsored by Denis Dzyubenko on Patreon.
2018-10-04 17:43:29 +00:00
* Fix reversion in support of annex.web-options.
clean up url removal presence update * rmurl: Fix a case where removing the last url left git-annex thinking content was still present in the web special remote. * SETURLPRESENT, SETURIPRESENT, SETURLMISSING, and SETURIMISSING used to update the presence information of the external special remote that called them; this was not documented behavior and is no longer done. Done by making setUrlPresent and setUrlMissing only update presence info for the web, and only when the url is a web url. See the comment for reasoning about why that's the right thing to do. In AddUrl, had to make it update location tracking, to handle the non-web-url case. This commit was sponsored by Ewen McNeill on Patreon.
2018-10-04 21:33:25 +00:00
* rmurl: Fix a case where removing the last url left git-annex thinking
content was still present in the web special remote.
* SETURLPRESENT, SETURIPRESENT, SETURLMISSING, and SETURIMISSING
used to update the presence information of the external special remote
that called them; this was not documented behavior and is no longer done.
export: Fix false positive in export conflict detection It occurred when the same tree was exported by multiple clones. nub out identical trees. This commit was sponsored by Jochen Bartl on Patreon.
2018-10-09 19:54:12 +00:00
* export: Fix false positive in export conflict detection, that occurred
when the same tree was exported by multiple clones.
Fix potential crash in exporttree database due to failure to honor uniqueness constraint I don't know the circumstances, but have a report of this: git-annex: failed to commit changes to sqlite database: Just SQLite3 returned ErrorConstraint while attempting to perform step. All 3 tables in the export db have uniqueness constraints on them, insertUnique is used for all the rest, but this use of insertMany means it doesn't check the constraint. I guess that's what caused the crash, but I have not been able to test it yet. Use putMany when available, as it should be faster than mapM of insertMany. This commit was sponsored by Brock Spratlen on Patreon.
2018-10-09 20:56:33 +00:00
* Fix potential crash in exporttree database due to failure to honor
uniqueness constraint.
Fix crash when exporttree is set to a bad value. Made it impossible to recover from setting a bad value since enableremote to change it would crash. This commit was sponsored by Henrik Riomar on Patreon.
2018-10-10 14:44:54 +00:00
* Fix crash when exporttree is set to a bad value.
adding arm64 build, and improved termux installation process * Added arm64 Linux standalone build. (No autobuilder yet.) * Improved termux installation process. Added git-annex-install.sh script to avoid user needing to type as much in termux. The scope of this script is limited; runshell handles the rest. Runshell runs termux-fix-shebang on the shell scripts. The problem is the bundled bin/sh script, deleting that script also works, but then the others probably use the system Android /bin/sh, which could be old or broken or not posix or whatever. Using termux sh to run the scripts is better. This commit was sponsored by Eric Drechsel on Patreon.
2018-10-11 17:22:02 +00:00
* Linux standalone: Avoid using bundled cp before envionment is fully set up.
* Added arm64 Linux standalone build.
* Improved termux installation process.
sync: Warn when a remote's export is not updated to the current tree because export tracking is not configured. Only display the warning when the current branch has a tree that is not the same as the tree in the export. Note that it doesn't check to see if the current tree is in incompleteExportedTreeish; it might be worth checking that and reminding the user about an incomplete export, but when export tracking is not configured, they are probably not in the right clone of the repository to resolve the incomplete export. This commit was sponsored by Ethan Aubin.
2018-09-27 19:35:46 +00:00
releasing package git-annex version 6.20181011
2018-10-11 17:50:53 +00:00
-- Joey Hess <id@joeyh.name> Thu, 11 Oct 2018 13:41:10 -0400
sync: Warn when a remote's export is not updated to the current tree because export tracking is not configured. Only display the warning when the current branch has a tree that is not the same as the tree in the export. Note that it doesn't check to see if the current tree is in incompleteExportedTreeish; it might be worth checking that and reminding the user about an incomplete export, but when export tracking is not configured, they are probably not in the right clone of the repository to resolve the incomplete export. This commit was sponsored by Ethan Aubin.
2018-09-27 19:35:46 +00:00
releasing package git-annex version 6.20180926
2018-09-26 17:16:54 +00:00
git-annex (6.20180926) upstream; urgency=medium
S3: Fix url construction bug When the publicurl has been set to an url that does not end with a slash, we need to add one in between it and the rest of the url. As far as I can see, git-annex does not default to such publicurls; it's careful to end them with slashes. But this was observed in the wild, and there may be documentation that doesn't include the slash. And it's an easy mistake to make in any case. This commit was sponsored by Eric Drechsel on Patreon.
2018-09-14 16:25:23 +00:00
BF: add netbase to Depends: see https://github.com/nipy/heudiconv/issues/260 for more context, but it seems to be required on a lean docker instances for git annex to be usable
2018-09-19 17:11:25 +00:00
[ Joey Hess ]
Revert "clean P2P protocol shutdown on EOF" This reverts commit b18fb1e343e9654207fbebacf686659c75d0fb4c. That broke support for old git-annex-shell before p2pstdio was added. The immediate problem is that postAuth had a fallthrough case that sent an error back to the peer, but sending an error back when the connection is closed is surely not going to work. But thinking about it some more, making every function that uses receiveMessage need to handle ProtocolEOF adds a lot of complication, so I don't want to do that. The commit only cleaned up the test suite output a tiny bit, so I'm just gonna revert it for now.
2018-09-25 18:04:12 +00:00
* Fixes a reversion in the last release that broke interoperation with
older versions of git-annex-shell.
improve back-compat of post-receive hook * init: Improve generated post-receive hook, so it won't fail when run on a system whose git-annex is too old to support git-annex post-receive * init: Update the post-receive hook when re-run in an existing repository. This commit was sponsored by Jack Hill on Patreon.
2018-09-25 19:02:12 +00:00
* init: Improve generated post-receive hook, so it won't fail when
run on a system whose git-annex is too old to support git-annex post-receive
* init: Update the post-receive hook when re-run in an existing repository.
S3: Fix url construction bug When the publicurl has been set to an url that does not end with a slash, we need to add one in between it and the rest of the url. As far as I can see, git-annex does not default to such publicurls; it's careful to end them with slashes. But this was observed in the wild, and there may be documentation that doesn't include the slash. And it's an easy mistake to make in any case. This commit was sponsored by Eric Drechsel on Patreon.
2018-09-14 16:25:23 +00:00
* S3: Fix url construction bug when the publicurl has been set to an url
that does not end with a slash.
--debug urls When git-annex used wget and curl, --debug would show urls. So there can't be any new security problem with doing so. This commit was sponsored by John Pellman on Patreon.
2018-09-14 16:46:39 +00:00
* --debug shows urls accessed by git-annex, like it used to do when
git-annex used wget and curl.
added -z Added -z option to git-annex commands that use --batch, useful for supporting filenames containing newlines. It only controls input to --batch, the output will still be line delimited unless --json or etc is used to get some other output. While git often makes -z affect both input and output, I don't like trying them together, and making it affect output would have been a significant complication, and also git-annex output is generally not intended to be machine parsed, unless using --json or a format option. Commands that take pairs like "file key" still separate them with a space in --batch mode. All such commands take care to support filenames with spaces when parsing that, so there was no need to change it, and it would have needed significant changes to the batch machinery to separate tose with a null. To make fromkey and registerurl support -z, I had to give them a --batch option. The implicit batch mode they enter when not provided with input parameters does not support -z as that would have complicated option parsing. Seemed better to move these toward using the same --batch as everything else, though the implicit batch mode can still be used. This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
* Fix support for filenames containing newlines when querying git
cat-file, though less efficiently than other filenames.
This should make git-annex fully support filenames containing newlines
as the rest of git's interface is used in newline-safe ways.
* Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
annex.maxextensionlength Added annex.maxextensionlength for use cases where extensions longer than 4 characters are needed. This commit was sponsored by Henrik Riomar on Patreon.
2018-09-24 16:07:46 +00:00
* Added annex.maxextensionlength for use cases where extensions longer
than 4 characters are needed.
remote.name.annex-security-allow-unverified-downloads Added remote.name.annex-security-allow-unverified-downloads, a per-remote setting for annex.security.allow-unverified-downloads. This commit was sponsored by Brock Spratlen on Patreon.
2018-09-25 19:32:50 +00:00
* Added remote.name.annex-security-allow-unverified-downloads, a
per-remote setting for annex.security.allow-unverified-downloads.
More FreeBSD build fixes. Untested, on FreeBSD but enough to fix the listed build errors. Seems that System.Posix.Files must have used to export this stuff and it was split. This commit was sponsored by Peter on Patreon.
2018-09-24 15:25:51 +00:00
* More FreeBSD build fixes.
S3: Fix url construction bug When the publicurl has been set to an url that does not end with a slash, we need to add one in between it and the rest of the url. As far as I can see, git-annex does not default to such publicurls; it's careful to end them with slashes. But this was observed in the wild, and there may be documentation that doesn't include the slash. And it's an easy mistake to make in any case. This commit was sponsored by Eric Drechsel on Patreon.
2018-09-14 16:25:23 +00:00
BF: add netbase to Depends: see https://github.com/nipy/heudiconv/issues/260 for more context, but it seems to be required on a lean docker instances for git annex to be usable
2018-09-19 17:11:25 +00:00
[ Yaroslav Halchenko ]
* debian/control
+ add netbase to Depends: since required for basic tcp interactions
(see e.g. https://github.com/nipy/heudiconv/issues/260)
releasing package git-annex version 6.20180926
2018-09-26 17:16:54 +00:00
-- Joey Hess <id@joeyh.name> Wed, 26 Sep 2018 12:56:49 -0400
S3: Fix url construction bug When the publicurl has been set to an url that does not end with a slash, we need to add one in between it and the rest of the url. As far as I can see, git-annex does not default to such publicurls; it's careful to end them with slashes. But this was observed in the wild, and there may be documentation that doesn't include the slash. And it's an easy mistake to make in any case. This commit was sponsored by Eric Drechsel on Patreon.
2018-09-14 16:25:23 +00:00
releasing package git-annex version 6.20180913
2018-09-13 19:50:50 +00:00
git-annex (6.20180913) upstream; urgency=medium
make --batch honor matching options When --batch is used with matching options like --in, --metadata, etc, only operate on the provided files when they match those options. Otherwise, a blank line is output in the batch protocol. Affected commands: find, add, whereis, drop, copy, move, get In the case of find, the documentation for --batch already said it honored the matching options. The docs for the rest didn't, but it makes sense to have them honor them. While this is a behavior change, why specify the matching options with --batch if you didn't want them to apply? Note that the batch output for all of the affected commands could already output a blank line in other cases, so batch users should already be prepared to deal with it. git-annex metadata didn't seem worth making support the matching options, since all it does is output metadata or set metadata, the use cases for using it in combination with the martching options seem small. Made it refuse to run when they're combined, leaving open the possibility for later support if a use case develops. This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
* When --batch is used with matching options like --in, --metadata,
etc, only operate on the provided files when they match those options.
Otherwise, a blank line is output in the batch protocol.
Affected commands: find, add, whereis, drop, copy, move, get
* Make metadata --batch combined with matching options refuse to run,
since it does not seem worth supporting that combination.
massive v6 add speed/memory improvement v6 add: Take advantage of improved SIGPIPE handler in git 2.5 to speed up the clean filter by not reading the file content from the pipe. This also avoids git buffering the whole file content in memory. When built with an older git, still consumes stdin. If built with a newer git and used with an older one, it breaks, but that's acceptable -- checking the git version every time would make repeated smudge runs slow. This commit was supported by the NSF-funded DataLad project.
2018-08-09 22:17:46 +00:00
* v6 add: Take advantage of improved SIGPIPE handler in git 2.5 to
speed up the clean filter by not reading the file content from the
pipe. This also avoids git buffering the whole file content in memory.
finally fixed v6 get/drop git status After updating the worktree for an add/drop, update git's index, so git status will not show the files as modified. What actually happens is that the index update removes the inode information from the index. The next git status (or similar) run then has to do some work. It runs the clean filter. So, this depends on the clean filter being reasonably fast and on git not leaking memory when running it. Both problems were fixed in a96972015dd76271b46432151e15d5d38d7151ff, but only for git 2.5. Anyone using an older git will see very expensive git status after an add/drop. This uses the same git update-index queue as other parts of git-annex, so the actual index update is fairly efficient. Of course, updating the index does still have some overhead. The annex.queuesize config will control how often the index gets updated when working on a lot of files. This is an imperfect workaround... Added several todos about new problems this workaround causes. Still, this seems a lot better than the old behavior. This commit was supported by the NSF-funded DataLad project.
2018-08-14 18:22:23 +00:00
* v6: After updating the worktree for an add/drop, update git's index,
so git status will not show the files as modified.
v6: avoid accidental conversion when annex.largefiles is not configured v6: When annex.largefiles is not configured for a file, running git add or git commit, or otherwise using git to stage a file will add it to the annex if the file was in the annex before, and to git otherwise. This is to avoid accidental conversion. Note that git-annex add's behavior has not changed, for reasons explained in the added comment. Performance: No added overhead when annex.largefiles is configured. When not configured, there is an added call to catObjectMetaData, which involves a round trip through git cat-file --batch. However, the earlier catKeyFile primes the cache for it. This commit was supported by the NSF-funded DataLad project.
2018-08-27 18:47:17 +00:00
* v6: When annex.largefiles is not configured for a file, running git
add or git commit, or otherwise using git to stage a file
will add it to the annex if the file was in the annex before,
and to git otherwise. This is to avoid accidental conversion.
Note that git-annex add's behavior has not changed.
v6: Update associated files database when git has staged changes to pointer files This commit was supported by the NSF-funded DataLad project.
2018-08-21 20:48:20 +00:00
* v6: Update associated files database when git has staged changes
to pointer files.
v6: recover from race between git mv and git-annex get/drop Update pointer file next time reconcileStaged is run to recover from the race. Note that restagePointerFile causes git to run the clean filter, and that will run reconcileStaged. So, normally by the time the git annex get/drop command finishes, the race has already been dealt with. It may be that, in some case, that won't happen and the race will be dealt with at a later point. git-annex could run reconcileStaged at shutdown if that becomes a problem. This does not handle the situation where the git mv is committed before git-annex gets a chance to run again. git commit does run the clean filter, and that happens to re-inject the content if it was supposed to be dropped but is still populated. But, the case where the file was supposed to be gotten but is not populated is not handled yet. This commit was supported by the NSF-funded DataLad project.
2018-08-22 19:28:57 +00:00
* v6: Fix some race conditions.
make linkToAnnex freezeContent the object file v6: Fix annex object file permissions when git-annex add is run on a modified unlocked file, and in some related cases. If a hard link is made, don't freeze it; annex.thin uses writable object files. Also: For some reason, linkToAnnex used to thawContent src. I can see no reason why it needed to do that, so I eliminated that. This commit was sponsored by Brock Spratlen on Patreon.
2018-09-05 19:27:22 +00:00
* v6: Fix annex object file permissions when git-annex add is run
on a modified unlocked file, and in some related cases.
change v6 git-annex add of staged unmodified unlocked file v6: When a file is unlocked but has not been modified, and the unlocking is only staged, git-annex add did not lock it. Now it will, for consistency with how modified files are handled and with v5. Note the removal of the sameInodeCache check. Otherwise it would see that the unmodified file is unmodified and stop there. That check seems to have been copied from the direct mode branch. But, direct mode had a specific reason to check for unmodified content, that does not apply to v6. The second pass means there is potential for a race, eg the unlocked file could be modified in between the first and second passes. No problem with that, since both passes do the same thing. This commit was sponsored by Jake Vosloo on Patreon.
2018-09-12 17:53:03 +00:00
* v6: When a file is unlocked but has not been modified,
and the unlocking is only staged, git-annex add did not lock it.
Now it will, for consistency with how modified files are handled and
with v5.
fix git command queue to be concurrency safe Probably not noticed until now because the queue is large enough that two threads each filling theirs at the same time and flushing is unlikely to happen. Also made explicit that each worker thread gets its own queue. I think that was the case before, but if something was put in the queue before worker threads were forked off, they could have each inherited the same queue. Could have gone with a single shared queue, but per-worker queues is more efficient, because a worker can add lots of stuff to its own queue without any locking. This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-08-28 17:14:44 +00:00
* Fix git command queue to be concurrency safe.
linux standalone: When LOCPATH is already set, use it instead of the bundled locales. It can be set to an empty string to use the system locales too. Of course whether that will work depends on the amount of divergence. This commit was supported by the NSF-funded DataLad project.
2018-08-20 16:20:54 +00:00
* linux standalone: When LOCPATH is already set, use it instead of the
bundled locales. It can be set to an empty string to use the system
locales too.
stop using external hash programs, since cryptonite is faster In 2013, I wrote "Cryptohash benchmarks 90 to 101% faster than external hashers". Re-benchmarking today, I found cryptonite's sha256 consistently outperformed coreutils by 10% for large files. Tested 10 mb, 100 mb, 1 gb files with both sha256 and sha512. And for smaller files, the external process startup time swamps the hash time. Perhaps cryptonite has improved. Or it could just do better on my current CPU Intel(R) Pentium(R) CPU 4410Y @ 1.50GHz). Anyway, even if cryptonite is slower in some situations, seems likely it would only be marginally slower; it's got the same class of highly optimised C code under the hood as coreutils. The main difference between the two sha256 implementations seems to be how much of the inner loop they unroll.. This commit was sponsored by Henrik Riomar on Patreon.
2018-08-28 22:01:13 +00:00
* Stop using external hash programs, since cryptonite is faster.
Fix build on FreeBSD This must have been broken for years.. This commit was sponsored by Jack Hill on Patreon.
2018-08-29 16:09:00 +00:00
* Fix build on FreeBSD.
devblog
2018-08-30 19:52:44 +00:00
* S3: Support buckets with versioning enabled. When a remote
is configured with exporttree=yes versioning=yes, git-annex can
download past versions of exported files from it.
fix build with older aws S3: Multipart uploads are now only supported when git-annex is built with aws-0.16.0 or later, as earlier versions of the library don't support versioning with multipart uploads. This will affect the android build, and debian stable also has a too old aws to support both features at the same time. This commit was sponsored by Nick Piper on Patreon.
2018-09-13 13:58:36 +00:00
* S3: Multipart uploads are now only supported when git-annex is built
with aws-0.16.0 or later, as earlier versions of the library don't
support versioning with multipart uploads.
S3: Support AWS_SESSION_TOKEN This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2018-09-05 19:53:57 +00:00
* S3: Support AWS_SESSION_TOKEN.
fix reversions due to undocumented and buggy git behavior * Don't use GIT_PREFIX when GIT_WORK_TREE=. because it seems git does not intend GIT_WORK_TREE to be relative to GIT_PREFIX in that case, despite GIT_WORK_TREE=.. being relative to GIT_PREFIX. * Don't use GIT_PREFIX to fix up a relative GIT_DIR, because git 2.11 sets GIT_PREFIX set to a path it's not relative to. and apparently GIT_DIR is never relative to GIT_PREFIX. Commit e50ed4ba48f93cf0addb3638a4a9605a10f17976 led us down this path by working around a git bug by relying on the barely documented GIT_PREFIX. This commit was sponsored by Trenton Cronholm on Patreon.
2018-09-11 19:53:48 +00:00
* Don't use GIT_PREFIX when GIT_WORK_TREE=. because it seems git
does not intend GIT_WORK_TREE to be relative to GIT_PREFIX in that
case, despite GIT_WORK_TREE=.. being relative to GIT_PREFIX.
* Don't use GIT_PREFIX to fix up a relative GIT_DIR, because
wording
2018-09-11 20:03:58 +00:00
git 2.11 sets GIT_PREFIX to a path it's not relative to.
fix reversions due to undocumented and buggy git behavior * Don't use GIT_PREFIX when GIT_WORK_TREE=. because it seems git does not intend GIT_WORK_TREE to be relative to GIT_PREFIX in that case, despite GIT_WORK_TREE=.. being relative to GIT_PREFIX. * Don't use GIT_PREFIX to fix up a relative GIT_DIR, because git 2.11 sets GIT_PREFIX set to a path it's not relative to. and apparently GIT_DIR is never relative to GIT_PREFIX. Commit e50ed4ba48f93cf0addb3638a4a9605a10f17976 led us down this path by working around a git bug by relying on the barely documented GIT_PREFIX. This commit was sponsored by Trenton Cronholm on Patreon.
2018-09-11 19:53:48 +00:00
and apparently GIT_DIR is never relative to GIT_PREFIX.
revert a few problem areas of git-annex.cabal patch
2018-09-05 15:47:00 +00:00
* git-annex.cabal: Fix build without assistant, and some other refinements.
Thanks fftehnik.
make --batch honor matching options When --batch is used with matching options like --in, --metadata, etc, only operate on the provided files when they match those options. Otherwise, a blank line is output in the batch protocol. Affected commands: find, add, whereis, drop, copy, move, get In the case of find, the documentation for --batch already said it honored the matching options. The docs for the rest didn't, but it makes sense to have them honor them. While this is a behavior change, why specify the matching options with --batch if you didn't want them to apply? Note that the batch output for all of the affected commands could already output a blank line in other cases, so batch users should already be prepared to deal with it. git-annex metadata didn't seem worth making support the matching options, since all it does is output metadata or set metadata, the use cases for using it in combination with the martching options seem small. Made it refuse to run when they're combined, leaving open the possibility for later support if a use case develops. This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
releasing package git-annex version 6.20180913
2018-09-13 19:50:50 +00:00
-- Joey Hess <id@joeyh.name> Thu, 13 Sep 2018 15:50:38 -0400
make --batch honor matching options When --batch is used with matching options like --in, --metadata, etc, only operate on the provided files when they match those options. Otherwise, a blank line is output in the batch protocol. Affected commands: find, add, whereis, drop, copy, move, get In the case of find, the documentation for --batch already said it honored the matching options. The docs for the rest didn't, but it makes sense to have them honor them. While this is a behavior change, why specify the matching options with --batch if you didn't want them to apply? Note that the batch output for all of the affected commands could already output a blank line in other cases, so batch users should already be prepared to deal with it. git-annex metadata didn't seem worth making support the matching options, since all it does is output metadata or set metadata, the use cases for using it in combination with the martching options seem small. Made it refuse to run when they're combined, leaving open the possibility for later support if a use case develops. This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
prep release
2018-08-07 00:30:38 +00:00
git-annex (6.20180807) upstream; urgency=medium
Fix reversion in display of http 404 errors. Switch to using http-client for large file downloads caused the reversion; the code for displaying a 404 response was instead displaying the raw html document, which is not useful. This commit was sponsored by Ryan Newton on Patreon.
2018-07-31 16:15:26 +00:00
S3: Support credential-less download from remotes configured with public=yes exporttree=yes. This commit was supported by the NSF-funded DataLad project.
2018-07-31 20:29:11 +00:00
* S3: Support credential-less download from remotes configured
with public=yes exporttree=yes.
Fix reversion in display of http 404 errors. Switch to using http-client for large file downloads caused the reversion; the code for displaying a 404 response was instead displaying the raw html document, which is not useful. This commit was sponsored by Ryan Newton on Patreon.
2018-07-31 16:15:26 +00:00
* Fix reversion in display of http 404 errors.
cache remotes via annex-speculate-present Added remote.name.annex-speculate-present config that can be used to make cache remotes. Implemented it in Remote.keyPossibilities, which is used by the get/move/copy/mirror commands, and nothing else. This way, things like whereis will not show content that's speculatively present. The assistant and sync --content were not using Remote.keyPossibilities, and were changed to use it. The efficiency hit should be small; Remote.keyPossibilities is only used before transferring a file, which is the expensive operation. And, it's only doing one lookup of the remoteList and a very cheap filter over it. Note that, git-annex still updates the location log when copying content to a remote with annex-speculate-present set. In this case, the location tracking will indicate that content is present in the remote. This may not be wanted for caches, or may not be a real problem for them. TBD. This commit was supported by the NSF-funded DataLad project.
2018-08-01 18:22:52 +00:00
* Added remote.name.annex-speculate-present config that can be used to
make cache remotes.
Added --accessedwithin matching option. Useful for dropping old objects from cache repositories. But also, quite a genrally useful thing to have.. Rather than imitiating find's -atime and other options, all of which are pretty horrible to use, I made this match files accessed within a time period, using the same duration format used by git-annex schedule and --limit-time In passing, changed the --limit-time option parser to parse the duration, instead of having it later throw an error. This commit was supported by the NSF-funded DataLad project.
2018-08-01 19:20:18 +00:00
* Added --accessedwithin matching option.
added annex.commitmessage Added annex.commitmessage config that can specify a commit message for the git-annex branch instead of the usual "update". This commit was supported by the NSF-funded DataLad project.
2018-08-02 18:06:06 +00:00
* Added annex.commitmessage config that can specify a commit message
for the git-annex branch instead of the usual "update".
Fix wrong sorting of remotes when using -J It was sorting by uuid, rather than cost! Avoid future bugs of this kind by changing the Ord to primarily compare by cost, with uuid only used when the cost is the same. This commit was supported by the NSF-funded DataLad project.
2018-08-03 17:06:06 +00:00
* Fix wrong sorting of remotes when using -J, it was sorting by uuid,
rather than cost.
addurl: Include filename in --json-progress output when known.
2018-08-06 16:52:09 +00:00
* addurl: Include filename in --json-progress output.
Fix git-annex branch data loss that could occur after git-annex forget --drop-dead Added getStaged, to get the versions of git-annex branch files staged in its index, and use during transitions so the result of merging sibling branches is used. The catFileStop in performTransitionsLocked is absolutely necessary, without that the bug still occurred, because git cat-file was already running and was looking at the old index file. Note that getLocal still has cat-file look at the git-annex branch, not the index. It might be faster if it looked at the index, but probably only marginally so, and I've not benchmarked it to see if it's faster at all. I didn't want to change unrelated behavior as part of this bug fix. And as the need for catFileStop shows, using the index file has added complications. Anyway, it still seems fine for getLocal to look at the git-annex branch, because normally the index file is updated just before the git-annex branch is committed, and so they'll contain the same information. It's only during a transition that the two diverge. This commit was sponsored by Paul Walmsley in honor of Mark Phillips.
2018-08-06 21:22:12 +00:00
* Fix git-annex branch data loss that could occur after
git-annex forget --drop-dead.
Fix reversion in display of http 404 errors. Switch to using http-client for large file downloads caused the reversion; the code for displaying a 404 response was instead displaying the raw html document, which is not useful. This commit was sponsored by Ryan Newton on Patreon.
2018-07-31 16:15:26 +00:00
releasing package git-annex version 6.20180807
2018-08-07 20:22:16 +00:00
-- Joey Hess <id@joeyh.name> Tue, 07 Aug 2018 16:22:05 -0400
Fix reversion in display of http 404 errors. Switch to using http-client for large file downloads caused the reversion; the code for displaying a 404 response was instead displaying the raw html document, which is not useful. This commit was sponsored by Ryan Newton on Patreon.
2018-07-31 16:15:26 +00:00
releasing package git-annex version 6.20180719
2018-07-19 17:53:59 +00:00
git-annex (6.20180719) upstream; urgency=medium
Merge branch 'later'
2018-06-26 01:59:20 +00:00
Support working trees set up by git-worktree. Support working trees set up by git-worktree, by setting up some symlinks such that git-annex links work right. Also improved support for repositories created with --separate-git-dir. At least recent git makes a .git file for those (older may have used a symlink?), so that also needs to be converted to a symlink. This commit was sponsored by Nick Piper on Patreon.
2018-07-18 18:25:03 +00:00
* Support working trees set up by git-worktree.
* Improve support for repositories created with --separate-git-dir.
Support configuring remote.web.annex-cost and remote.bittorrent.annex-cost Seems that has never worked before due to oversight.
2018-06-23 22:16:37 +00:00
* Support configuring remote.web.annex-cost and remote.bittorrent.annex-cost
make addurl of media url fail when youtube-dl is disabled addurl: When security configuration prevents downloads with youtube-dl, still check if the url is one that it supports, and fail downloading it, instead of downloading the raw web page.
2018-06-28 17:01:18 +00:00
* addurl: When security configuration prevents downloads with youtube-dl,
still check if the url is one that it supports, and fail downloading
it, instead of downloading the raw web page.
reorder
2018-07-18 18:48:06 +00:00
* Send User-Agent and any configured annex.http-headers when downloading
with http, fixes reversion introduced when switching to http-client.
fix nested exception bug Fix reversion introduced in version 6.20180316 that caused git-annex to stop processing files when unable to contact a ssh remote. The bug was not in any of the changed lines, but this one in inAnnex: P2PHelper.checkpresent (Ssh.runProto rmt connpool (cantCheck rmt) fallback) key cantCheck throws an exception, but that parameter to runProto expects a value, which it returns. So, inAnnex is returning a Bool containing an exception. This defeats the usual checks for checkPresent throwing an exception, crashing git-annex. Fixed by making runProto take an `Annex a` instead of an `a`, so passing cantCheck to it doesn't nest exceptions. This commit was sponsored by andrea rota.
2018-07-03 17:09:04 +00:00
* Fix reversion introduced in version 6.20180316 that caused git-annex to
stop processing files when unable to contact a ssh remote.
reorder
2018-07-18 18:48:06 +00:00
* v6: Work around git bug that runs smudge/clean filters at the top of the
repository while passing them a relative GIT_WORK_TREE that may point
outside of the repository, by using GIT_PREFIX to get back to the
subdirectory where a relative GIT_WORK_TREE is valid.
fix p2p --pair p2p --pair: Fix interception of the magic-wormhole pairing code, which since 0.8.2 it has sent to stderr rather than stdout. This is highly annoying because I had asked the magic wormhole developers for a machine-readable way to get the data, and instead they changed how the data was output, and didn't even mention this in my issue, or in the changelog. Seems this needs to be tested periodically to make sure it's still working. This commit was sponsored by Ethan Aubin.
2018-07-04 19:14:03 +00:00
* p2p --pair: Fix interception of the magic-wormhole pairing code,
which since 0.8.2 it has sent to stderr rather than stdout.
reorder
2018-07-18 18:48:06 +00:00
* info: Display uuid and description when a repository is identified by
uuid, and for "here".
* unused --from: Allow specifiying a repository by uuid or description.
deal with the persistent locpath issue linux standalone: Generate locale files in ~/.cache/git-annex/locales/ so they're available even when the standalone tarball is installed in a directory owned by root. This avoids a full-on reference counting cleanup hell, by letting old locale caches linger as long as the standalone bundle directory associated with them is still around. Old ones get cleaned up. In the case where the directory has a new bundle unpacked over top of it, the old locale cache is invalidated and rebuilt. Of course, running programs using that may get confused, but this was already the case, and unpacking over top of a bundle is probably not a good idea anyhow. To support that, added a buildid file, which only needs to be unique across builds of git-annex with different libc versions. sha1sum of git-annex seems good enough for that. Removed debian/patches/standalone-no-LOCPATH as it's no longer necessary. This commit was supported by the NSF-funded DataLad project.
2018-07-10 16:13:14 +00:00
* linux standalone: Generate locale files in ~/.cache/git-annex/locales/
so they're available even when the standalone tarball is installed
update re writable HOME with standalone bundle
2018-07-10 18:22:37 +00:00
in a directory owned by root. Note that this prevents using the
standalone bundle in environments where HOME is not writable.
reorder
2018-07-18 18:48:06 +00:00
* Include uname command in standalone builds since git-annex uses it.
* git-annex.cabal: Fix network version.
make addurl of media url fail when youtube-dl is disabled addurl: When security configuration prevents downloads with youtube-dl, still check if the url is one that it supports, and fail downloading it, instead of downloading the raw web page.
2018-06-28 17:01:18 +00:00
releasing package git-annex version 6.20180719
2018-07-19 17:53:59 +00:00
-- Joey Hess <id@joeyh.name> Thu, 19 Jul 2018 13:53:45 -0400
git-annex version 6.20180626 -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEKKUAw1IH6rcvbA8l2xLbD/BfjzgFAlstCaQACgkQ2xLbD/Bf jzh5nxAAn7D9soTI0ex6AVDDo2CjOyTTDVrIcl2h5XizfuUD3ev5P0TR3BZmzpAb MI6uaZ8kxqZ/eGAsBTyH9PsV7QVYIdht9t89ytP4xWyTQiOgjyJeA6PnJl4zVK9z Y8Of3mlylaz+97+sndljpsvy/KHENrHI7HHd+qxAu7wKysJxG6fJB7CjremkjaCI zAwg3mIy72ZKyuR/8hL9puJN9fdfw1ulkzQR+he007e/HkurPCwgRAOYW/Aa2tpY Oigdb9a6/0nl/VnOS8ZyHrSPRrhLH9c4IBmsdC1Xt5NDVmID/sWgD9uPF9dsHSMF OM25QdSlJ5cSNg+/XCpmmhC9MjgKkuVNpZ/fWBaHFs6KYgGhtZcAayQdz5AmMS2N HTPWB1IxZiV5TQHQpLbdH/q3RfNtRq1G1tc24zpd/zdhzijeTM6D8n4No6LXNq8X 7U0qcrp9TdLOpBCTf6Jrg/7qFaXddHoEW1e3KrsOmB0hlYHuNxfY4bs0+ROeXGOT 00koezcbF8kEI0ekoDvJjtVqaUq+608YjJZ5v7dE0vbtTj0KGbl5EHwC9atUluCX MHyTDY89uq68g4HIDytL001ZLvE3EUGJc4jh3+OMDzuZSKB5uwJIIky+qIaQu34K QJrZuyAIY0sVFV6LUX9nwqTW6Nnx/bB+kZ6k0+gx+Lpf7pUpE+o= =kex4 -----END PGP SIGNATURE----- gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEKKUAw1IH6rcvbA8l2xLbD/BfjzgFAlsxnX4ACgkQ2xLbD/Bf jzjK1xAAnJ58ZxLyTYlCZRcKiR81UHS/Mk6+SDAjRIRbT0SsY+6gSP55XKjrcuOb Jatp+6cNNSgk2lBpn37mq+rYIqboFh9moDRK7JSh1mDHCVtIwdARGblFRfuwaWPi xHnu+Pj43+SP7OF+8qP8/kDM+js3iMS+0gvBBz8pQN/yJDROXii6u0eONOd7vbER iRY9QpJdj5lp3hjaWfXt5iJC0re0eOAY4eUSHPsFIASysShnn33dFPOZ2hbhRKjR unQHUVIUE+ehmW3w9qIqn+9v2kca7laGK11cvzYRpmu/9rrvpf+RF1h42S8822dP CKHvxDkBGbyqTA+F9/6zpU1i9/ARgHFDpScRcdq7ZJi9FbWabKDklHCsgxwrkdXb +FXgb7N5Sa4+eVDNUf4rxldtLPX53nrtZ3IqrGiCWApCvbysNyP5kE0nix02l9z2 xzY2vlpicx7TOMoO9mZesSFNgRzuFAbbya/zDJrz+xfgSRYXRYg58yTpmhpTFvSI h3Fw6+MYvehvRdAweLtoQt2p/UV2MAWrTpNzFoqgf2OCQOiH97ACDHn8Yki9rnQi NuMsqv9WOYQs4SaygDZMKemgAxftf3uaXiBW0RzHHwwWnDjHhqsEioOvOhNNyZbz U3OjKrH1JZlkNHlIBQD4BsWGLlIct66ZTU3k2OxPEp+mpEG/Xi4= =p+cW -----END PGP SIGNATURE----- Merge tag '6.20180626' - previously embargoed security release
2018-06-26 01:56:43 +00:00
update version
2018-06-21 18:56:04 +00:00
git-annex (6.20180626) upstream; urgency=high
finalize changelog for release
2018-06-19 15:41:50 +00:00
enforce retrievalSecurityPolicy Leveraged the existing verification code by making it also check the retrievalSecurityPolicy. Also, prevented getViaTmp from running the download action at all when the retrievalSecurityPolicy is going to prevent verifying and so storing it. Added annex.security.allow-unverified-downloads. A per-remote version would be nice to have too, but would need more plumbing, so KISS. (Bill the Cat reference not too over the top I hope. The point is to make this something the user reads the documentation for before using.) A few calls to verifyKeyContent and getViaTmp, that don't involve downloads from remotes, have RetrievalAllKeysSecure hard-coded. It was also hard-coded for P2P.Annex and Command.RecvKey, to match the values of the corresponding remotes. A few things use retrieveKeyFile/retrieveKeyFileCheap without going through getViaTmp. * Command.Fsck when downloading content from a remote to verify it. That content does not get into the annex, so this is ok. * Command.AddUrl when using a remote to download an url; this is new content being added, so this is ok. This commit was sponsored by Fernando Jimenez on Patreon.
2018-06-21 17:34:11 +00:00
Security fix release for CVE-2018-10857 and CVE-2018-10859
add link to advistory
2018-06-22 14:27:22 +00:00
https://git-annex.branchable.com/security/CVE-2018-10857_and_CVE-2018-10859/
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
enforce retrievalSecurityPolicy Leveraged the existing verification code by making it also check the retrievalSecurityPolicy. Also, prevented getViaTmp from running the download action at all when the retrievalSecurityPolicy is going to prevent verifying and so storing it. Added annex.security.allow-unverified-downloads. A per-remote version would be nice to have too, but would need more plumbing, so KISS. (Bill the Cat reference not too over the top I hope. The point is to make this something the user reads the documentation for before using.) A few calls to verifyKeyContent and getViaTmp, that don't involve downloads from remotes, have RetrievalAllKeysSecure hard-coded. It was also hard-coded for P2P.Annex and Command.RecvKey, to match the values of the corresponding remotes. A few things use retrieveKeyFile/retrieveKeyFileCheap without going through getViaTmp. * Command.Fsck when downloading content from a remote to verify it. That content does not get into the annex, so this is ok. * Command.AddUrl when using a remote to download an url; this is new content being added, so this is ok. This commit was sponsored by Fernando Jimenez on Patreon.
2018-06-21 17:34:11 +00:00
* Refuse to download content, that cannot be verified with a hash,
from encrypted special remotes (for CVE-2018-10859),
note that glacier was also limited
2018-06-21 20:40:31 +00:00
and from all external special remotes and glacier (for CVE-2018-10857).
enforce retrievalSecurityPolicy Leveraged the existing verification code by making it also check the retrievalSecurityPolicy. Also, prevented getViaTmp from running the download action at all when the retrievalSecurityPolicy is going to prevent verifying and so storing it. Added annex.security.allow-unverified-downloads. A per-remote version would be nice to have too, but would need more plumbing, so KISS. (Bill the Cat reference not too over the top I hope. The point is to make this something the user reads the documentation for before using.) A few calls to verifyKeyContent and getViaTmp, that don't involve downloads from remotes, have RetrievalAllKeysSecure hard-coded. It was also hard-coded for P2P.Annex and Command.RecvKey, to match the values of the corresponding remotes. A few things use retrieveKeyFile/retrieveKeyFileCheap without going through getViaTmp. * Command.Fsck when downloading content from a remote to verify it. That content does not get into the annex, so this is ok. * Command.AddUrl when using a remote to download an url; this is new content being added, so this is ok. This commit was sponsored by Fernando Jimenez on Patreon.
2018-06-21 17:34:11 +00:00
In particular, URL and WORM keys stored on such remotes won't
be downloaded. If this affects your files, you can run
`git-annex migrate` on the affected files, to convert them
to use a hash.
* Added annex.security.allow-unverified-downloads, which can override
the above.
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
* Added annex.security.allowed-url-schemes setting, which defaults
allow ftp urls by default They're no worse than http certianly. And, the backport of these security fixes has to deal with wget, which supports http https and ftp and has no way to turn off individual schemes, so this will make that easier.
2018-06-18 19:36:12 +00:00
to only allowing http, https, and ftp URLs. Note especially that file:/
enforce retrievalSecurityPolicy Leveraged the existing verification code by making it also check the retrievalSecurityPolicy. Also, prevented getViaTmp from running the download action at all when the retrievalSecurityPolicy is going to prevent verifying and so storing it. Added annex.security.allow-unverified-downloads. A per-remote version would be nice to have too, but would need more plumbing, so KISS. (Bill the Cat reference not too over the top I hope. The point is to make this something the user reads the documentation for before using.) A few calls to verifyKeyContent and getViaTmp, that don't involve downloads from remotes, have RetrievalAllKeysSecure hard-coded. It was also hard-coded for P2P.Annex and Command.RecvKey, to match the values of the corresponding remotes. A few things use retrieveKeyFile/retrieveKeyFileCheap without going through getViaTmp. * Command.Fsck when downloading content from a remote to verify it. That content does not get into the annex, so this is ok. * Command.AddUrl when using a remote to download an url; this is new content being added, so this is ok. This commit was sponsored by Fernando Jimenez on Patreon.
2018-06-21 17:34:11 +00:00
is no longer enabled by default.
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
* Removed annex.web-download-command, since its interface does not allow
supporting annex.security.allowed-url-schemes across redirects.
If you used this setting, you may want to instead use annex.web-options
to pass options to curl.
prevent http connections to localhost and private ips by default Security fix! * git-annex will refuse to download content from http servers on localhost, or any private IP addresses, to prevent accidental exposure of internal data. This can be overridden with the annex.security.allowed-http-addresses setting. * Since curl's interface does not have a way to prevent it from accessing localhost or private IP addresses, curl defaults to not being used for url downloads, even if annex.web-options enabled it before. Only when annex.security.allowed-http-addresses=all will curl be used. Since S3 and WebDav use the Manager, the same policies apply to them too. youtube-dl is not handled yet, and a http proxy configuration can bypass these checks too. Those cases are still TBD. This commit was sponsored by Jeff Goeke-Smith on Patreon.
2018-06-17 17:05:30 +00:00
* git-annex will refuse to download content from http servers on
localhost, or any private IP addresses, to prevent accidental
exposure of internal data. This can be overridden with the
annex.security.allowed-http-addresses setting.
prevent using local http proxies per annex.security.allowed-http-addresses A local http proxy would bypass the security configuration. So, the security configuration has to be applied when choosing whether to use the proxy. While http rebinding attacks against the dns lookup of the proxy IP address seem very unlikely, this implementation does prevent them, since it resolves the IP address once, checks it, and then reconfigures http-client's proxy using the resolved address. This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-06-18 17:32:20 +00:00
* Local http proxies will not be used unless allowed by the
annex.security.allowed-http-addresses setting.
default to not using youtube-dl, for security Pity, but same reasoning as curl applies to it. This commit was sponsored by Peter on Patreon.
2018-06-17 18:46:22 +00:00
* Since the interfaces to curl and youtube-dl do not have a way to
prevent them from accessing localhost or private IP addresses,
they default to not being used for url downloads.
Only when annex.security.allowed-http-addresses=all will curl and
youtube-dl be used.
limit url downloads to whitelisted schemes Security fix! Allowing any schemes, particularly file: and possibly others like scp: allowed file exfiltration by anyone who had write access to the git repository, since they could add an annexed file using such an url, or using an url that redirected to such an url, and wait for the victim to get it into their repository and send them a copy. * Added annex.security.allowed-url-schemes setting, which defaults to only allowing http and https URLs. Note especially that file:/ is no longer enabled by default. * Removed annex.web-download-command, since its interface does not allow supporting annex.security.allowed-url-schemes across redirects. If you used this setting, you may want to instead use annex.web-options to pass options to curl. With annex.web-download-command removed, nearly all url accesses in git-annex are made via Utility.Url via http-client or curl. http-client only supports http and https, so no problem there. (Disabling one and not the other is not implemented.) Used curl --proto to limit the allowed url schemes. Note that this will cause git annex fsck --from web to mark files using a disallowed url scheme as not being present in the web. That seems acceptable; fsck --from web also does that when a web server is not available. youtube-dl already disabled file: itself (probably for similar reasons). The scheme check was also added to youtube-dl urls for completeness, although that check won't catch any redirects it might follow. But youtube-dl goes off and does its own thing with other protocols anyway, so that's fine. Special remotes that support other domain-specific url schemes are not affected by this change. In the bittorrent remote, aria2c can still download magnet: links. The download of the .torrent file is otherwise now limited by annex.security.allowed-url-schemes. This does not address any external special remotes that might download an url themselves. Current thinking is all external special remotes will need to be audited for this problem, although many of them will use http libraries that only support http and not curl's menagarie. The related problem of accessing private localhost and LAN urls is not addressed by this commit. This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
finalize changelog for release
2018-06-19 15:41:50 +00:00
Non-security fix changes:
Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change https://prime.haskell.org/wiki/Libraries/Proposals/SemigroupMonoid I am not happy with the fragile pile of CPP boilerplate required to support ghc back to 7.0, which git-annex still targets for both the android build and the standalone build targeting old linux kernels. It makes me unlikely to want to use Semigroup more in git-annex, because the benefit of the abstraction is swamped by the ugliness. I actually considered ripping out all the Semigroup instances, but some are needed to use optparse-applicative. The problem, I think, is they made this transaction on too fast a timeline. (Although ironically, work on it started in 2015 or earlier!) In particular, Debian oldstable is not out of security support, and it's not possible to follow the simpler workarounds documented on the wiki and have it build on oldstable (because the semigroups package in it is too old). I have only tested this build with ghc 8.2.2, not the newer and older versions that branches of the CPP support. So there could be typoes, we'll see. This commit was sponsored by Brock Spratlen on Patreon.
2018-05-30 16:28:43 +00:00
* Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change.
avoid unncessary version output differences in different contexts Show operating system and repository version list when run outside a git repo too. Also made it only display the local repository version when in a git-annex repo. Before it showed "unknown" when run in a git repo that was not git-annex initialized. That seemed like confusing behavior. This commit was sponsored by Jochen Bartl on Patreon.
2018-06-04 16:26:18 +00:00
* version: Show operating system and repository version list
when run outside a git repo too.
removal of the rest of remoteGitConfig In keyUrls, the GitConfig is used only by annexLocations to support configured Differences. Since such configurations affect all clones of a repository, the local repo's GitConfig must have the same information as the remote's GitConfig would have. So, used getGitConfig to get the local GitConfig, which is cached and so available cheaply. That actually fixed a bug noone had ever noticed: keyUrls is used for remotes accessed over http. The full git config of such a remote is normally not available, so the remoteGitConfig that keyUrls used would not have the necessary information in it. In copyFromRemoteCheap', it uses gitAnnexLocation, which does need the GitConfig of the remote repo itself in order to check if it's crippled, supports symlinks, etc. So, made the State include that GitConfig, cached. The use of gitAnnexLocation is within a (not $ Git.repoIsUrl repo) guard, so it's local, and so its git config will always be read and available. (Note that gitAnnexLocation in turn calls annexLocations, so the Differences config it uses in this case comes from the remote repo's GitConfig and not from the local repo's GitConfig. As explained above this is ok since they must have the same value.) Not very happy with this mess of different GitConfigs not type-safe and some read only sometimes etc. Very hairy. Think I got it this change right. Test suite passes.. This commit was sponsored by Ethan Aubin.
2018-06-05 18:23:34 +00:00
* Fix annex-checkuuid implementation, so that remotes configured that
fix annex-checkuuid Fixed annex-checkuuid implementation, so that remotes configured that way can be used. This was 100% broken from the first commit of it, oops. This commit was sponsored by Øyvind Andersen Holm.
2018-06-04 20:48:26 +00:00
way can be used.
removal of the rest of remoteGitConfig In keyUrls, the GitConfig is used only by annexLocations to support configured Differences. Since such configurations affect all clones of a repository, the local repo's GitConfig must have the same information as the remote's GitConfig would have. So, used getGitConfig to get the local GitConfig, which is cached and so available cheaply. That actually fixed a bug noone had ever noticed: keyUrls is used for remotes accessed over http. The full git config of such a remote is normally not available, so the remoteGitConfig that keyUrls used would not have the necessary information in it. In copyFromRemoteCheap', it uses gitAnnexLocation, which does need the GitConfig of the remote repo itself in order to check if it's crippled, supports symlinks, etc. So, made the State include that GitConfig, cached. The use of gitAnnexLocation is within a (not $ Git.repoIsUrl repo) guard, so it's local, and so its git config will always be read and available. (Note that gitAnnexLocation in turn calls annexLocations, so the Differences config it uses in this case comes from the remote repo's GitConfig and not from the local repo's GitConfig. As explained above this is ok since they must have the same value.) Not very happy with this mess of different GitConfigs not type-safe and some read only sometimes etc. Very hairy. Think I got it this change right. Test suite passes.. This commit was sponsored by Ethan Aubin.
2018-06-05 18:23:34 +00:00
* Fix problems accessing repositories over http when annex.tune.*
is configured.
add GETINFO to external protocol (for ronnypfa) External special remotes can now add info to `git annex info $remote`, by replying to the GETINFO message. Had to generalize some helpers to allow consuming multiple messages from the remote. The code added to Remote/* here is AGPL licensed, thus changed the license of the files. This commit was sponsored by Jake Vosloo on Patreon.
2018-06-08 15:52:20 +00:00
* External special remotes can now add info to `git annex info $remote`,
by replying to the GETINFO message.
adb: Android serial numbers are not all 16 characters long, so accept other lengths. I can't find any documentation of how long it should be. Hard to imagine it being shorter than 4 characters though, so put that in as a conservative lower bound. This commit was sponsored by Nick Piper on Patreon.
2018-06-12 17:56:01 +00:00
* adb: Android serial numbers are not all 16 characters long, so accept
other lengths.
display p2pstdio stderr after auth Display error messages that come from git-annex-shell when the p2p protocol is used, so that diskreserve messages, IO errors, etc from the remote side are visible again. Felt like it should perhaps use outputError, so --json-error-messages would include these, but as an async IO action, it can't, and this would need MessageState to be converted to a tvar. Anyway, when not using p2pstdio, that's not done; nor is it done for stderr from external special remotes or other commands, so punted on the idea for now. This commit was sponsored by mo on Patreon.
2018-06-12 18:57:45 +00:00
* Display error messages that come from git-annex-shell when the p2p
protocol is used, so that diskreserve messages, IO errors, etc from
the remote side are visible again.
export: re-send lost exported files after fsck notices they're gone When content has been lost from an export remote and git-annex fsck --from remote has noticed it's gone, re-running git-annex export or git-annex sync --content will re-upload it. Note that normally there's no way to remove a single file from an export. doc/design/exporting_trees_to_special_remotes.mdwn talks about this in the section "dropping from exports and copying to exports". But, if a file is somehow deleted or corrupted on the export, and fsck notices this, it will update the location log to say it's missing. So, checking the location log when determining if a file needs to be sent to the export will let such missing files be added back in. There's otherwise no way to do so. It does not fall afoul of the races documented in the abovementioned section, I think. This commit was sponsored by Ryan Newton on Patreon.
2018-06-14 16:22:12 +00:00
* When content has been lost from an export remote and
git-annex fsck --from remote has noticed it's gone, re-running
git-annex export or git-annex sync --content will re-upload it.
Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change https://prime.haskell.org/wiki/Libraries/Proposals/SemigroupMonoid I am not happy with the fragile pile of CPP boilerplate required to support ghc back to 7.0, which git-annex still targets for both the android build and the standalone build targeting old linux kernels. It makes me unlikely to want to use Semigroup more in git-annex, because the benefit of the abstraction is swamped by the ugliness. I actually considered ripping out all the Semigroup instances, but some are needed to use optparse-applicative. The problem, I think, is they made this transaction on too fast a timeline. (Although ironically, work on it started in 2015 or earlier!) In particular, Debian oldstable is not out of security support, and it's not possible to follow the simpler workarounds documented on the wiki and have it build on oldstable (because the semigroups package in it is too old). I have only tested this build with ghc 8.2.2, not the newer and older versions that branches of the CPP support. So there could be typoes, we'll see. This commit was sponsored by Brock Spratlen on Patreon.
2018-05-30 16:28:43 +00:00
finalize release
2018-06-22 14:36:44 +00:00
-- Joey Hess <id@joeyh.name> Fri, 22 Jun 2018 10:36:22 -0400
Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change https://prime.haskell.org/wiki/Libraries/Proposals/SemigroupMonoid I am not happy with the fragile pile of CPP boilerplate required to support ghc back to 7.0, which git-annex still targets for both the android build and the standalone build targeting old linux kernels. It makes me unlikely to want to use Semigroup more in git-annex, because the benefit of the abstraction is swamped by the ugliness. I actually considered ripping out all the Semigroup instances, but some are needed to use optparse-applicative. The problem, I think, is they made this transaction on too fast a timeline. (Although ironically, work on it started in 2015 or earlier!) In particular, Debian oldstable is not out of security support, and it's not possible to follow the simpler workarounds documented on the wiki and have it build on oldstable (because the semigroups package in it is too old). I have only tested this build with ghc 8.2.2, not the newer and older versions that branches of the CPP support. So there could be typoes, we'll see. This commit was sponsored by Brock Spratlen on Patreon.
2018-05-30 16:28:43 +00:00
releasing package git-annex version 6.20180529
2018-05-29 17:06:56 +00:00
git-annex (6.20180529) upstream; urgency=medium
reuse hashes of dotfiles/dirs/submodules when entering view This fixes a crash when a git submodule has a name starting with a dot. Such a submodule might contain dotfiles that are intended to be used when inside the view (since a dot-directory that's not a submodule was already preserved when entering a view). So, rather than eliminating the submodule from the view, its git ls-files --stage hash is copied over into the view. dotfiles/dirs have their git ls-files --stage hashes similarly copied over to the view. This is more efficient and simpler than the old method, and also won't break if git ever adds a new type of tree item, like was done with submodules. Since the content of dotfiles in the working tree is no longer hashed when entering a view, when there are unstaged modifications, they are not included in the view branch. Entering the view branch still works, but git checkout shows "M .dotfile", and git diff will show the unstaged changes. This seems like an improvement over the old behavior. Also made Command.View not delete empty directories that are submodules when entering a view, while still deleting other empty directories. This commit was supported by the NSF-funded DataLad project.
2018-05-14 18:58:13 +00:00
reorder
2018-05-25 16:10:49 +00:00
* Prevent haskell http-client from decompressing gzip files, so downloads
of such files works the same as it used to with wget and curl.
Workaround for bug in an old version of cryptonite that broke https downloads, by using curl for downloads when git-annex is built with it. This commit was supported by the NSF-funded DataLad project.
2018-05-20 18:12:18 +00:00
* Workaround for bug in an old version of cryptonite that broke https
downloads, by using curl for downloads when git-annex is built with it.
reorder
2018-05-25 16:10:49 +00:00
* view, vadd: Fix crash when a git submodule has a name starting with a dot.
* Don't allow entering a view with staged or unstaged changes.
remove the older move --force, which never behaved as documented and seems useless * move: --force was accidentially enabling two unrelated behaviors since 6.20180427. The older behavior, which has never been well documented and seems almost entirely useless, has been removed. * copy: --force no longer does anything. This commit was sponsored by Øyvind Andersen Holm.
2018-05-21 17:20:40 +00:00
* move: --force was accidentially enabling two unrelated behaviors
since 6.20180427. The older behavior, which has never been well
documented and seems almost entirely useless, has been removed.
* copy: --force no longer does anything.
fix migration bug and make fsck warn * migrate: Fix bug in migration between eg SHA256 and SHA256E, that caused the extension to be included in SHA256 keys, and omitted from SHA256E keys. (Bug introduced in version 6.20170214) * migrate: Check for above bug when migrating from SHA256 to SHA256 (and same for SHA1 to SHA1 etc), and remove the extension that should not be in the SHA256 key. * fsck: Detect and warn when keys need an upgrade, either to fix up from the above migrate bug, or to add missing size information (a long ago transition), or because of a few other past key related bugs. This commit was sponsored by Henrik Riomar on Patreon.
2018-05-23 18:07:51 +00:00
* migrate: Fix bug in migration between eg SHA256 and SHA256E,
that caused the extension to be included in SHA256 keys,
and omitted from SHA256E keys.
(Bug introduced in version 6.20170214)
* migrate: Check for above bug when migrating from SHA256 to SHA256
(and same for SHA1 to SHA1 etc), and remove the extension that should
not be in the SHA256 key.
* fsck: Detect and warn when keys need an upgrade, either to fix up
from the above migrate bug, or to add missing size information
(a long ago transition), or because of a few other past key related
bugs.
GIT_ANNEX_SHELL_APPENDONLY Makes it allow writes, but not deletion of annexed content. Note that securing pushes to the git repository is left up to the user. This commit was sponsored by Jack Hill on Patreon.
2018-05-25 17:17:56 +00:00
* git-annex-shell: GIT_ANNEX_SHELL_APPENDONLY makes it allow writes,
but not deletion of annexed content. Note that securing pushes to
the git repository is left up to the user.
setpresentkey: Added --batch support (for ronnypfa) This commit was sponsored by Peter on Patreon.
2018-05-27 18:56:14 +00:00
* setpresentkey: Added --batch support.
reuse hashes of dotfiles/dirs/submodules when entering view This fixes a crash when a git submodule has a name starting with a dot. Such a submodule might contain dotfiles that are intended to be used when inside the view (since a dot-directory that's not a submodule was already preserved when entering a view). So, rather than eliminating the submodule from the view, its git ls-files --stage hash is copied over into the view. dotfiles/dirs have their git ls-files --stage hashes similarly copied over to the view. This is more efficient and simpler than the old method, and also won't break if git ever adds a new type of tree item, like was done with submodules. Since the content of dotfiles in the working tree is no longer hashed when entering a view, when there are unstaged modifications, they are not included in the view branch. Entering the view branch still works, but git checkout shows "M .dotfile", and git diff will show the unstaged changes. This seems like an improvement over the old behavior. Also made Command.View not delete empty directories that are submodules when entering a view, while still deleting other empty directories. This commit was supported by the NSF-funded DataLad project.
2018-05-14 18:58:13 +00:00
releasing package git-annex version 6.20180529
2018-05-29 17:06:56 +00:00
-- Joey Hess <id@joeyh.name> Tue, 29 May 2018 13:05:26 -0400
reuse hashes of dotfiles/dirs/submodules when entering view This fixes a crash when a git submodule has a name starting with a dot. Such a submodule might contain dotfiles that are intended to be used when inside the view (since a dot-directory that's not a submodule was already preserved when entering a view). So, rather than eliminating the submodule from the view, its git ls-files --stage hash is copied over into the view. dotfiles/dirs have their git ls-files --stage hashes similarly copied over to the view. This is more efficient and simpler than the old method, and also won't break if git ever adds a new type of tree item, like was done with submodules. Since the content of dotfiles in the working tree is no longer hashed when entering a view, when there are unstaged modifications, they are not included in the view branch. Entering the view branch still works, but git checkout shows "M .dotfile", and git diff will show the unstaged changes. This seems like an improvement over the old behavior. Also made Command.View not delete empty directories that are submodules when entering a view, while still deleting other empty directories. This commit was supported by the NSF-funded DataLad project.
2018-05-14 18:58:13 +00:00
releasing package git-annex version 6.20180509
2018-05-09 20:20:43 +00:00
git-annex (6.20180509) upstream; urgency=medium
back out incorrect IO interleaving change Fix regression in last release that crashes when using --all or running git-annex in a bare repository. May have also affected git-annex unused and git-annex info. Reversed the order of the (++) in Annex.Branch.files so --all will stream lazily still when there are not a bunch of uncommitted journal files. Added a todo to maybe improve this later. This commit was sponsored by Trenton Cronholm on Patreon.
2018-05-08 17:54:42 +00:00
close almost all old Android app bug reports The old git-annex Android app is now deprecated in favor of running git-annex in termux. I suspect all or nearly all of these no longer apply. This commit was sponsored by Jochen Bartl on Patreon.
2018-05-08 18:59:00 +00:00
* The old git-annex Android app is now deprecated in favor of running
git-annex in termux.
use proot to support Android 8 runshell: Use proot when running on Android, to work around Android 8's ill-advised seccomp filtering of system calls, including ones crucial for reliable thread locking. (This will only work with termux's version of proot.) See https://github.com/termux/termux-packages/issues/420#issuecomment-386636938 This commit was sponsored by andrea rota.
2018-05-05 12:55:50 +00:00
* runshell: Use proot when running on Android, to work around
Android 8's ill-advised seccomp filtering of system calls,
including ones crucial for reliable thread locking.
(This will only work with termux's version of proot.)
Support building with hinotify-0.3.10. Kept backwards compat with old versions via a shim. This commit was sponsored by mo on Patreon.
2018-05-08 18:26:57 +00:00
* Fix bug in last release that crashes when using
--all or running git-annex in a bare repository. May have also
affected git-annex unused and git-annex info.
avoid uname -o on !linux and catch any exception from it Fix bug in last release that prevented the webapp opening on non-Linux systems. This commit was sponsored by Jake Vosloo on Patreon.
2018-05-08 18:06:19 +00:00
* Fix bug in last release that prevented the webapp opening on
non-Linux systems.
Support building with hinotify-0.3.10. Kept backwards compat with old versions via a shim. This commit was sponsored by mo on Patreon.
2018-05-08 18:26:57 +00:00
* Support building with hinotify-0.3.10.
Display error message when http download fails. * Display error message when http download fails. There's nothing in the http-client library to nicely format a http exception, so in some cases it has to fall back to using show on it. Seems better than just saying "it failed" or only showing the http status code. * Avoid forward retry when 0 bytes were received. forwardRetry was comparing Nothing to Just 0, and so thought there had been progress made when 0 bytes were received. This commit was supported by the NSF-funded DataLad project.
2018-05-08 20:11:45 +00:00
* Display error message when http download fails.
* Avoid forward retry when 0 bytes were received.
back out incorrect IO interleaving change Fix regression in last release that crashes when using --all or running git-annex in a bare repository. May have also affected git-annex unused and git-annex info. Reversed the order of the (++) in Annex.Branch.files so --all will stream lazily still when there are not a bunch of uncommitted journal files. Added a todo to maybe improve this later. This commit was sponsored by Trenton Cronholm on Patreon.
2018-05-08 17:54:42 +00:00
releasing package git-annex version 6.20180509
2018-05-09 20:20:43 +00:00
-- Joey Hess <id@joeyh.name> Wed, 09 May 2018 16:20:26 -0400
back out incorrect IO interleaving change Fix regression in last release that crashes when using --all or running git-annex in a bare repository. May have also affected git-annex unused and git-annex info. Reversed the order of the (++) in Annex.Branch.files so --all will stream lazily still when there are not a bunch of uncommitted journal files. Added a todo to maybe improve this later. This commit was sponsored by Trenton Cronholm on Patreon.
2018-05-08 17:54:42 +00:00
releasing package git-annex version 6.20180427
2018-04-27 16:37:01 +00:00
git-annex (6.20180427) upstream; urgency=medium
move: --safe/--unsafe and potential drop race fix move: Added --safe option, which makes move honor numcopies settings. Also --unsafe enables the default behavior, anticipating that the default may one day change. This commit was sponsored by Ethan Aubin.
2018-04-09 20:09:00 +00:00
move: Avoid drops that make bad situations worse, but otherwise allow See the big comment at the bottom of Command.Drop for the full details. (The --safe/--unsafe options were never released.) This commit was sponsored by Jake Vosloo on Patreon.
2018-04-13 18:06:25 +00:00
* move: Now takes numcopies configuration, and required content
configuration into account, and refuses to reduce the current
number of copies of a file, or remove content that a repository
requires. --force can override these checks.
Note that it's still allowed to move the content of a file
from one repository to another when numcopies is not satisfied, as long
as the move does not result in there being fewer copies.
Fix mangling of --json output of utf-8 characters when not running in a utf-8 locale As long as all code imports Utility.Aeson rather than Data.Aeson, and no Strings that may contain utf-8 characters are used for eg, object keys via T.pack, this is guaranteed to fix the problem everywhere that git-annex generates json. It's kind of annoying to need to wrap ToJSON with a ToJSON', especially since every data type that has a ToJSON instance has to be ported over. However, that only took 50 lines of code, which is worth it to ensure full coverage. I initially tried an alternative approach of a newtype FileEncoded, which had to be used everywhere a String was fed into aeson, and chasing down all the sites would have been far too hard. Did consider creating an intentionally overlapping instance ToJSON String, and letting ghc fail to build anything that passed in a String, but am not sure that wouldn't pollute some library that git-annex depends on that happens to use ToJSON String internally. This commit was supported by the NSF-funded DataLad project.
2018-04-16 19:42:45 +00:00
* Fix mangling of --json output of utf-8 characters when not
running in a utf-8 locale.
Fix build with yesod 1.6. Also avoid some depreaction warnings.
2018-04-22 17:56:35 +00:00
* Fix build with yesod 1.6.
update
2018-04-22 17:58:18 +00:00
* Clean up some build warnings with newer versions of ghc and haskell
libraries.
runshell: Unset LD_PRELOAD Preloaded libraries from the host system may not get along with the bundled linker. This was observed by users in termux: ERROR: ld.so: object '/data/data/com.termux/files/usr/lib/libtermux-exec.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS64): ignored. Bad system call But it could also affect more usual systems; the preloaded library might rely on symbols from the host libc that are not available or have the wrong versions in the bundled libc. Unsetting LD_PRELOAD entirely seems safest.
2018-04-25 17:40:48 +00:00
* runshell: Unset LD_PRELOAD since preloaded libraries from the host
system may not get along with the bundled linker.
runshell: hacks for termux; add tip Added some tweaks to make git-annex work in termux on Android. The regular arm standalone tarball now works in termux. I guess the test for "$base/bin/git" is not really necessary, since it tests for git-annex. Since that gets deleted on android, removed that test. These are pretty hackish hacks, especially adding it to PATH. The goal is to make it work well enough out of the box on Android. This commit was sponsored by Eric Drechsel on Patreon.
2018-04-25 17:47:43 +00:00
* runshell: Added some tweaks to make git-annex work in termux on
Android. The regular arm standalone tarball now works in termux.
Termux:Boot integration Assistant: Integrate with Termux:Boot, so when it's installed, the assistant is autostarted on boot. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2018-04-25 19:31:25 +00:00
* Webapp: Support being run inside termux on Android, and offer to set up
a repository on the sdcard.
* Assistant: Integrate with Termux:Boot, so when it's installed, the
assistant is autostarted on boot.
Assistant: Fix installation of menus, icons, etc when run from within runshell. runshell followed by git annex webapp didn't install that stuff, because GIT_ANNEX_APP_BASE is not set. Running git-annex.linux/git-annex-webapp did install that stuff, since that script set the env var. I noticed this with the termux port whose instructions currently go that way. Seems the right thing to do is to move the env var setting to runshell.
2018-04-25 21:58:00 +00:00
* Assistant: Fix installation of menus, icons, etc when run
from within runshell.
import: Avoid buffering all filenames to be imported in memory. Test case is 24 directories each containing files named 1..10000. The concat and filterM destroyed what laziness there is in dirContentsRecursive, making it buffer all the filenames. Memory use was around 300 mb (possibly growing slightly as it progressed). After this fix, memory use drops to a constant 59 mb. Note that dirContentsRecursive still buffers the entire content of a directory (not subdirectories) so this is still not optimal.
2018-04-26 16:06:12 +00:00
* import: Avoid buffering all filenames to be imported in memory.
avoid git annex info remote buffering list of keys This leaves git annex unused --from remote still using loggedKeysFor and buffering more than ought to be necessary, but I can't see a way to improve that.
2018-04-26 20:13:05 +00:00
* Improve memory use and speed of --all and git-annex info remote,
by not buffering list of all keys.
move: --safe/--unsafe and potential drop race fix move: Added --safe option, which makes move honor numcopies settings. Also --unsafe enables the default behavior, anticipating that the default may one day change. This commit was sponsored by Ethan Aubin.
2018-04-09 20:09:00 +00:00
releasing package git-annex version 6.20180427
2018-04-27 16:37:01 +00:00
-- Joey Hess <id@joeyh.name> Fri, 27 Apr 2018 12:36:20 -0400
move: --safe/--unsafe and potential drop race fix move: Added --safe option, which makes move honor numcopies settings. Also --unsafe enables the default behavior, anticipating that the default may one day change. This commit was sponsored by Ethan Aubin.
2018-04-09 20:09:00 +00:00
releasing package git-annex version 6.20180409
2018-04-09 17:24:45 +00:00
git-annex (6.20180409) upstream; urgency=medium
Fix calculation of estimated completion for progress meter. Was estimating transfer of whole file, not remaining part of it.
2018-03-20 03:26:41 +00:00
Added adb special remote which allows exporting files to Android devices. git annex testremote passes. exportree not implemented yet, although the documentation talks about it, since it will be the main way this remote will be used. The adb push/pull progress is displayed for now; it would be better to consume it and use it to update the git-annex progress bar. This commit was sponsored by andrea rota.
2018-03-27 16:41:57 +00:00
* Added adb special remote which allows exporting files to Android devices.
Use http-conduit for url downloads by default, annex.web-options enables curl * For url downloads, git-annex now defaults to using a http library, rather than wget or curl. But, if annex.web-options is set, it will use curl. To use the .netrc file, run: git config annex.web-options --netrc * git-annex no longer uses wget (and wget is no longer shipped with git-annex builds). Note that curl is always run in silent mode, since the new API for download has a MeterUpdate and doesn't make way for curl progress output. It might be worth writing a parser for curl's progress output to update the meter when using it, but I didn't bother with this edge case for now. This commit was supported by the NSF-funded DataLad project.
2018-04-06 21:00:46 +00:00
* For url downloads, git-annex now defaults to using a http library,
rather than wget or curl. But, if annex.web-options is set, it will
use curl. To use the .netrc file, run:
git config annex.web-options --netrc
* git-annex no longer uses wget (and wget is no longer shipped with
git-annex builds).
* Enable HTTP connection reuse across multiple files for improved speed.
Fix calculation of estimated completion for progress meter. Was estimating transfer of whole file, not remaining part of it.
2018-03-20 03:26:41 +00:00
* Fix calculation of estimated completion for progress meter.
OSX app: Work around libz/libPng/ImageIO.framework version skew By not bundling libz, assuming OSX includes a suitable libz.1.dylib. This commit was supported by the NSF-funded DataLad project.
2018-03-22 15:50:42 +00:00
* OSX app: Work around libz/libPng/ImageIO.framework version skew
by not bundling libz, assuming OSX includes a suitable libz.1.dylib.
implement annex.retry et al Added annex.retry, annex.retry-delay, and per-remote versions to configure transfer retries. This commit was supported by the NSF-funded DataLad project.
2018-03-29 17:04:07 +00:00
* Added annex.retry, annex.retry-delay, and per-remote versions
to configure transfer retries.
Also do forward retrying in cases where no exception is thrown, but the transfer failed. I think this used to be the case, but it was accidentially lost way back in commit 3887432c54e1da6d66d364bb7f153a3c6a1cace8. Normally, transfers do not throw exceptions, so probably forward retrying was rarely done due to that oversight. This also affects the new annex.retry etc configuration. If a transfer fails, without making any progress, eg because the file is not present on the remote or the remote is not accessible, it will now retry when configuration calls for it. In some cases such a retry is not desirable, for example the remote could be accessible and not have a copy of the file that the local repo thinks it has. I see no way to distinguish such cases from cases where a retry should really be done. So, it'll be up to the user to configure it to work for them.
2018-03-29 17:22:49 +00:00
* Also do forward retrying in cases where no exception is thrown,
but the transfer failed.
don't copy old date metadata when adding new version of a file When adding a new version of a file, and annex.genmetadata is enabled, don't copy the data metadata from the old version of the file, instead use the mtime of the file. Rationalle being that the user has requested to generate metadata and so would expect to get the new mtime into metadata. Also, avoid warning about copying metadata when all the old metadata is date metadata. Which was rather the harder part. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2018-04-04 17:42:15 +00:00
* When adding a new version of a file, and annex.genmetadata is enabled,
don't copy the data metadata from the old version of the file,
instead use the mtime of the file.
Avoid running annex.http-headers-command more than once.
2018-04-04 19:00:51 +00:00
* Avoid running annex.http-headers-command more than once.
info: Added "combined size of repositories containing these files" stat when run on a directory This commit was sponsored by andrea rota.
2018-04-05 18:44:58 +00:00
* info: Added "combined size of repositories containing these files"
stat when run on a directory.
info: Changed sorting of numcopies stats table, so it's ordered by the variance from the desired number of copies. Compare these... numcopies stats: numcopies -1: 1986 numcopies +0: 1170 numcopies -2: 769 numcopies +1: 716 numcopies -4: 696 numcopies -3: 485 numcopies -6: 230 numcopies -5: 111 numcopies -7: 91 numcopies -9: 9 numcopies stats: numcopies +1: 716 numcopies +0: 1170 numcopies -1: 1986 numcopies -2: 769 numcopies -3: 485 numcopies -4: 696 numcopies -5: 111 numcopies -6: 230 numcopies -7: 91 numcopies -9: 9 I feel that the former is a jumbled mess that doesn't tell much overall, while the second shows pretty clearly that most files are within 1 degree of the desired number of copies, with some outliers without enough.
2018-04-05 18:54:39 +00:00
* info: Changed sorting of numcopies stats table, so it's ordered
by the variance from the desired number of copies.
Fix resuming a download when using curl. Noticed a bug; when using curl a workaround for its empty file behavior overwrote the file content, so it never resumed and always started over.
2018-04-06 20:09:53 +00:00
* Fix resuming a download when using curl.
Fix calculation of estimated completion for progress meter. Was estimating transfer of whole file, not remaining part of it.
2018-03-20 03:26:41 +00:00
releasing package git-annex version 6.20180409
2018-04-09 17:24:45 +00:00
-- Joey Hess <id@joeyh.name> Mon, 09 Apr 2018 13:03:15 -0400
Fix calculation of estimated completion for progress meter. Was estimating transfer of whole file, not remaining part of it.
2018-03-20 03:26:41 +00:00
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
git-annex (6.20180316) upstream; urgency=medium
Support exporttree=yes for rsync special remotes. Renaming is not supported; it might be possible to use --fuzzy to get rsync to notice the file is being renamed, but that is a bit ..fuzzy. On the other hand, interrupted transfers of an exported file are resumed, since rsync is great at that. Had to adjust the exporttree docs, which said interrupted transfers would restart. Note that remove no longer makes the empty directory dummy, instead sending the top-level empty directory. This works just as well and I noticed the dummy was unncessary when refactoring it into removeGeneric. Verified that behavior of remove is not changed, and git annex testremote does pass. This commit was sponsored by Brock Spratlen on Patreon.
2018-02-28 16:09:03 +00:00
use P2P protocol for checkpresent, retrieve, and store Note that, due to not using rsync to transfer files to ssh remotes any longer, permissions and other file metadata of annexed files will no longer be preserved when copying them to ssh remotes. Other remotes never supported preserving that information, so this is not considered a regression. Added NEWS item about this. Another significant side effect of this is that, even when rsync is run to retrieve a file, its progress display will no longer be shown, and instead the native git-annex progress display will appear. It would be possible to use the rsync process display when rsync is used (old git-annex-shell and also retrieval from a local repository), but it would have complicated the code unncessarily, and been inconsistent behavior. (I'd been thinking for a while about eliminating the rsync progress display, since it's got some annoying verbosities, including display of the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat inconsistent.) retrieveKeyFileCheap still uses rsync, since that ensures that it gets the actual file content from the remote. Using the P2P protocol would use the local content, as long as the local and remote size are the same. This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
* New protocol for communicating with git-annex-shell increases speed
of operations involving ssh remotes. When not transferring large files,
avoid double-connect to unreachable ssh remote When git-annex-shell p2pstdio fails with 255, it's because the ssh server is not reachable. Avoid running the fallback action in this case, since it would just try a second time to connect, and presumably fail. Note that the closed P2PSshConnection will not be stored in the pool, so the next request tries again to connect. This is just the right behavior; when the remote becomes reachable again, the same git-annex process will start using it. This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-03-12 20:50:21 +00:00
git-annex is between 200% and 400% faster using the new protocol,
and it's just as fast as before when transferring large files.
use P2P protocol for checkpresent, retrieve, and store Note that, due to not using rsync to transfer files to ssh remotes any longer, permissions and other file metadata of annexed files will no longer be preserved when copying them to ssh remotes. Other remotes never supported preserving that information, so this is not considered a regression. Added NEWS item about this. Another significant side effect of this is that, even when rsync is run to retrieve a file, its progress display will no longer be shown, and instead the native git-annex progress display will appear. It would be possible to use the rsync process display when rsync is used (old git-annex-shell and also retrieval from a local repository), but it would have complicated the code unncessarily, and been inconsistent behavior. (I'd been thinking for a while about eliminating the rsync progress display, since it's got some annoying verbosities, including display of the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat inconsistent.) retrieveKeyFileCheap still uses rsync, since that ensures that it gets the actual file content from the remote. Using the P2P protocol would use the local content, as long as the local and remote size are the same. This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
(When the remote has an old git-annex-shell, git-annex falls back
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
to the old slower code. This fallback is planned to be removed
after 5 years or so.)
wording
2018-03-09 17:56:21 +00:00
* Note that, due to not using rsync to transfer files over ssh
use P2P protocol for checkpresent, retrieve, and store Note that, due to not using rsync to transfer files to ssh remotes any longer, permissions and other file metadata of annexed files will no longer be preserved when copying them to ssh remotes. Other remotes never supported preserving that information, so this is not considered a regression. Added NEWS item about this. Another significant side effect of this is that, even when rsync is run to retrieve a file, its progress display will no longer be shown, and instead the native git-annex progress display will appear. It would be possible to use the rsync process display when rsync is used (old git-annex-shell and also retrieval from a local repository), but it would have complicated the code unncessarily, and been inconsistent behavior. (I'd been thinking for a while about eliminating the rsync progress display, since it's got some annoying verbosities, including display of the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat inconsistent.) retrieveKeyFileCheap still uses rsync, since that ensures that it gets the actual file content from the remote. Using the P2P protocol would use the local content, as long as the local and remote size are the same. This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to and from ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression.
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
* Fix data loss bug in content locking over tor, when the remote
repository is in direct mode, it neglected to check that the content
was actually present when locking it. This could cause git annex drop
to remove the only copy of a file when it thought the tor remote had
a copy.
* Fix data loss bug when the local repository uses direct mode, and a
locally modified file is dropped from a remote repsitory. The bug
caused the modified file to be counted as a copy of the original file.
(This is not a severe bug because in such a situation, dropping
from the remote and then modifying the file is allowed and has the same
end result.)
force verification when resuming download When resuming a download and not using a rolling checksummer like rsync, the partial file we start with might contain garbage, in the case where a file changed as it was being downloaded. So, disabling verification on resumes risked a bad object being put into the annex. Even downloads with rsync are currently affected. It didn't seem worth the added complexity to special case those to prevent verification, especially since git-annex is using rsync less often now. This commit was sponsored by Brock Spratlen on Patreon.
2018-03-13 18:50:49 +00:00
* Some downloads will be verified, even when annex.verify=false.
This is done in some edge cases where there's a likelyhood than an
object was downloaded incorrectly.
Support exporttree=yes for rsync special remotes. Renaming is not supported; it might be possible to use --fuzzy to get rsync to notice the file is being renamed, but that is a bit ..fuzzy. On the other hand, interrupted transfers of an exported file are resumed, since rsync is great at that. Had to adjust the exporttree docs, which said interrupted transfers would restart. Note that remove no longer makes the empty directory dummy, instead sending the top-level empty directory. This works just as well and I noticed the dummy was unncessary when refactoring it into removeGeneric. Verified that behavior of remove is not changed, and git annex testremote does pass. This commit was sponsored by Brock Spratlen on Patreon.
2018-02-28 16:09:03 +00:00
* Support exporttree=yes for rsync special remotes.
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
* Added backends for the BLAKE2 family of hashes, when built with
a new enough version of cryptonite.
Improve SHA*E extension extraction code Do not treat parts of the filename that contain punctuation or other non-alphanumeric characters as extensions. Before, such characters were filtered out. Note that in 45308ec78b759e207b6624157a4947ee8cd00464 "foo.ba__________r" was munged to ".bar" and so incorrectly treated as an extension. That was fixed by changing the filter order, but not allowing punctuation seems a better fix. This assumes that extensions containing punctuation are rare. "_" seems the most likely character; I used it in ikiwiki "._comment" files. But I can't recall seeing it anywhere else. It certianly seems that no commonly used extensions contain punctuation. If git-annex doesn't treat "._comment" as an extension, it's not likely to break software that expects to see that extension like some software expects to see .epub or .mp3. This commit was sponsored by Jack Hill on Patreon.
2018-03-05 15:25:01 +00:00
* Improve SHA*E extension extraction code to not treat parts of the
filename that contain punctuation or other non-alphanumeric characters
as extensions. Before, such characters were filtered out.
Better ssh connection warmup when using -J for concurrency. Avoids ugly messages when forced ssh command is not git-annex-shell. This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-03-07 21:25:42 +00:00
* Better ssh connection warmup when using -J for concurrency.
Avoids ugly messages when forced ssh command is not git-annex-shell.
fix ssh warmup hang Fix race condition in ssh warmup that caused git-annex to get stuck and never process some while when run with high levels of concurrency. So far, I've isolated the problem to processTranscript, which hangs reading output from ssh in this situation. I don't yet understand why processTranscript behaves that way. Since here we don't care about the ssh output, and only want to /dev/null it, changed to not use processTranscript, avoiding its problem. This commit was supported by the NSF-funded DataLad project.
2018-03-15 19:00:19 +00:00
* Fix race condition in ssh warmup that caused git-annex to get
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
stuck and never process some files when run with high levels of
fix ssh warmup hang Fix race condition in ssh warmup that caused git-annex to get stuck and never process some while when run with high levels of concurrency. So far, I've isolated the problem to processTranscript, which hangs reading output from ssh in this situation. I don't yet understand why processTranscript behaves that way. Since here we don't care about the ssh output, and only want to /dev/null it, changed to not use processTranscript, avoiding its problem. This commit was supported by the NSF-funded DataLad project.
2018-03-15 19:00:19 +00:00
concurrency.
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
* Fix reversion introduced in 6.20171214 that caused concurrent
transfers to incorrectly fail with "transfer already in progress".
going AGPL Remote/Git.hs now contains AGPL licensed code, thus the license of git-annex as a whole is AGPL. This was already the case when git-annex was built with the webapp enabled. The AGPL license will apply to all code added to Remote/Git.hs in the future, which is going to include support for using `git-annex-shell p2pstdio`.
2018-03-09 05:03:11 +00:00
* Note that Remote/Git.hs now contains AGPL licensed code,
thus the license of git-annex as a whole is AGPL. This was already
the case when git-annex was built with the webapp enabled.
Include amount of data transferred in progress display.
2018-03-14 17:39:14 +00:00
* Include amount of data transferred in progress display.
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
* Dial back optimisation when building on arm, which prevents
ghc and llc from running out of memory when optimising some files.
cabal sdist refuses to build with -O1 Sigh, disabled the full fix. What we have now will work once the ghc bug is fixed, but not currently.
2018-03-16 16:56:28 +00:00
(Unfortunately this fix is incomplete due to a ghc bug.)
Support exporttree=yes for rsync special remotes. Renaming is not supported; it might be possible to use --fuzzy to get rsync to notice the file is being renamed, but that is a bit ..fuzzy. On the other hand, interrupted transfers of an exported file are resumed, since rsync is great at that. Had to adjust the exporttree docs, which said interrupted transfers would restart. Note that remove no longer makes the empty directory dummy, instead sending the top-level empty directory. This works just as well and I noticed the dummy was unncessary when refactoring it into removeGeneric. Verified that behavior of remove is not changed, and git annex testremote does pass. This commit was sponsored by Brock Spratlen on Patreon.
2018-02-28 16:09:03 +00:00
releasing package git-annex version 6.20180316
2018-03-16 16:40:27 +00:00
-- Joey Hess <id@joeyh.name> Fri, 16 Mar 2018 12:10:40 -0400
Support exporttree=yes for rsync special remotes. Renaming is not supported; it might be possible to use --fuzzy to get rsync to notice the file is being renamed, but that is a bit ..fuzzy. On the other hand, interrupted transfers of an exported file are resumed, since rsync is great at that. Had to adjust the exporttree docs, which said interrupted transfers would restart. Note that remove no longer makes the empty directory dummy, instead sending the top-level empty directory. This works just as well and I noticed the dummy was unncessary when refactoring it into removeGeneric. Verified that behavior of remove is not changed, and git annex testremote does pass. This commit was sponsored by Brock Spratlen on Patreon.
2018-02-28 16:09:03 +00:00
releasing package git-annex version 6.20180227
2018-02-27 16:05:35 +00:00
git-annex (6.20180227) upstream; urgency=medium
inprogress: Avoid showing failures for files not in progress.
2018-01-25 00:43:19 +00:00
* inprogress: Avoid showing failures for files not in progress.
Added INFO to external special remote protocol. It's left up to the special remote to detect when git-annex is new enough to support the message; an old git-annex will blow up. This commit was supported by the NSF-funded DataLad project.
2018-02-06 17:03:55 +00:00
* Added INFO to external special remote protocol.
Added EXTENSIONS to external special remote protocol. Allows using new special remote messages when git-annex supports them, and avoiding using them when git-annex is too old. The new INFO is one such message. There's also the possibility, currently unused, for the special remote's reply to include some kind of extensions of its own. Merging this is blocked by https://github.com/datalad/datalad/issues/2124 since it seems it will break datalad. I checked all the other special remotes and they will be ok. This commit was supported by the NSF-funded DataLad project.
2018-02-07 19:02:12 +00:00
* Added EXTENSIONS to external special remote protocol.
datalad < 0.9.1 had a problem in its special remote protocol handling which is broken by EXTENSIONS. Make the debian git-annex package conflict with the problem version of datalad.
2018-02-07 20:16:33 +00:00
* datalad < 0.9.1 had a problem in its special remote protocol handling
which is broken by EXTENSIONS. Make the debian git-annex package
conflict with the problem version of datalad.
fsck: Warn when required content is not present in the repository that requires it. This commit was sponsored by Jack Hill on Patreon.
2018-02-08 18:08:41 +00:00
* fsck: Warn when required content is not present in the repository that
requires it.
Add gpg-agent to Build-Depends. Needed to run the test suite. Merged from Debian. I think what this actually deals with is the case where gpg is installed, but gpg-agent is not, since Utility.Gpg.stdParams enables --use-agent when GPG_BATCH is set, and the test suite enables GPG_BATCH. So, test suite will work with gpg not installed, or with both gpg and gpg-agent installed, but not with only gpg. For this reason, I've also put in an explicit dep on gnupg, although dpkg-dev recommends it and all debian package builds tend to have it available implicitly.
2018-02-14 21:32:44 +00:00
* Add gpg-agent to Build-Depends.
Needed to run the test suite.
--json: multi-line notes --json: When there are multiple lines of notes about a file, make the note field multiline, rather than the old behavior of only including the last line. Using newlines in the note is perhaps not ideal, but upgrading it to an array in this case would be an annoying inconsistency to need to deal with. This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-02-16 17:27:17 +00:00
* --json: When there are multiple lines of notes about a file, make the note
field multiline, rather than the old behavior of only including the
last line.
git-annex.cabal: Once more try to not build the assistant on the hurd, hopefully hackage finally recognises that OS. Last try was in 2015 https://github.com/haskell/hackage-server/issues/269
2018-02-18 15:09:16 +00:00
* git-annex.cabal: Once more try to not build the assistant on the hurd,
hopefully hackage finally recognises that OS.
Split Test.hs and avoid optimising it much, to need less memory to compile. The ghc options were found by Sean Whitton; the debian arm autobuilders need those to build w/o OOM, and it seems to involve llvm using too much memory to optimize Test. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2018-02-18 15:48:48 +00:00
* Split Test.hs and avoid optimising it much, to need less memory to
compile.
wording
2018-02-26 18:36:40 +00:00
* Fix behavior of --json-progress followed by --json, the latter option
used to disable the former.
wording
2018-02-19 19:40:44 +00:00
* Added --json-error-messages option, which makes messages
that would normally be output to standard error be included in
the json output.
Remove temporary code added in 6.20160619 to prime the mergedrefs log. Repositories that are upgraded from before that version to this one will not break, but will just not see the benefit of the mergedrefs log speeding things up, until one new ref gets merged in.
2018-02-22 16:31:27 +00:00
* Remove temporary code added in 6.20160619 to prime the mergedrefs
log.
importfeed: Fix a failure when downloading with youtube-dl and the destination subdirectory does not exist yet. Noticed while running this (which a user posted in a comment they deleted for some reason): git-annex importfeed https://vimeo.com/logiingimars/videos/rss The filename that youtube-dl suggests included a subdirectory, which didn't exist, so renaming to it failed. This commit was sponsored by mo on Patreon.
2018-02-22 17:20:19 +00:00
* importfeed: Fix a failure when downloading with youtube-dl
and the destination subdirectory does not exist yet.
annex.merge-annex-branches Added annex.merge-annex-branches config setting which can be used to disable automatic merge of git-annex branches. I wonder if git-annex merge/sync/assistant should disable this setting? Not sure yet, so have not done so. May be that users will not set it in git config, but pass it via -c to commands that need it. Checking the config setting adds a very small overhead, but it's only checked once per command so should be insignificant. This commit was supported by the NSF-funded DataLad project.
2018-02-22 18:25:32 +00:00
* Added annex.merge-annex-branches config setting which
can be used to disable automatic merge of git-annex branches.
Fix metadata hook script to support non-ascii characters tips/automatically_adding_metadata/pre-commit-annex: Fix to not silently skip filenames containing non-ascii characters. git diff-index defaults to munging non-ascii characters. Using -z makes it not do that, and then we just change the nulls to newlines. This commit was sponsored by Jochen Bartl on Patreon.
2018-02-26 16:58:37 +00:00
* tips/automatically_adding_metadata/pre-commit-annex: Fix to not
silently skip filenames containing non-ascii characters.
fix sync bug in direct mode sync: Fix bug that prevented pulling changes into direct mode repositories that were committed to remotes using git commit rather than git-annex sync. This commit was supported by the NSF-funded DataLad project.
2018-02-26 18:10:03 +00:00
* sync: Fix bug that prevented pulling changes into direct mode
repositories that were committed to remotes using git commit
rather than git-annex sync.
remove chrpath hack Makefile: Remove chrpath workaround for bug in cabal, which is no longer needed. https://github.com/haskell/cabal/issues/2717 says it uses RUNPATH instead of RPATH now, but I don't even see that for statically linked libraries; the bug with that appears to be fixed. cabal-install version 1.24.0.2 compiled using version 1.24.2.0 of the Cabal library I left the rpath removal using otool on OSX because those straight up broke the linker, and I don't know if the OSX autobuilder is updated to a new enough cabal to not need it. This commit was sponsored by Ewen McNeill on Patreon.
2018-02-27 15:59:45 +00:00
* Makefile: Remove chrpath workaround for bug in cabal,
which is no longer needed.
inprogress: Avoid showing failures for files not in progress.
2018-01-25 00:43:19 +00:00
releasing package git-annex version 6.20180227
2018-02-27 16:05:35 +00:00
-- Joey Hess <id@joeyh.name> Tue, 27 Feb 2018 12:04:52 -0400
inprogress: Avoid showing failures for files not in progress.
2018-01-25 00:43:19 +00:00
releasing package git-annex version 6.20180112
2018-01-12 19:47:32 +00:00
git-annex (6.20180112) upstream; urgency=medium
Removed the testsuite build flag Test suite is always included. Building with this flag disabled has actually been broken for some time, since Command.TestRemote uses tasty. Fewer build flags are better, so good time to drop it. This commit was sponsored by Thomas Hochstein on Patreon.
2017-12-20 16:25:03 +00:00
Added inprogress command for accessing files as they are being downloaded. Chose to make this only handle files actively being downloaded, not temp files for downloads that were interrupted or files that have been fully downloaded. This commit was sponsored by Ole-Morten Duesund on Patreon.
2017-12-28 15:46:39 +00:00
* Added inprogress command for accessing files as they are being
downloaded.
Fix bug introduced in version 6.20171018 that caused some commands to print out "ok" twice after processing a file. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon
2017-12-28 21:29:38 +00:00
* Fix bug introduced in version 6.20171018 that caused some commands
to print out "ok" twice after processing a file.
repeated addurl behavior reversion fix addurl: When the file youtube-dl will download is already an annexed file, don't download it again and fail to overwrite it, instead just do nothing, like it used to when quvi was used. This commit was sponsored by Anthony DeRobertis on Patreon.
2017-12-31 18:55:51 +00:00
* addurl: When the file youtube-dl will download is already an annexed
file, don't download it again and fail to overwrite it, instead just do
nothing, like it used to when quvi was used.
addurl: Fix encoding of filename queried from youtube-dl when in --fast mode. And also now in non-fast mode, since it was just changed to query for the filename separately. And avoid processTranscript which mixed up stdout and stderr and could have led to weirdness if there were warnings that didn't get suppressed.
2017-12-31 19:19:01 +00:00
* addurl: Fix encoding of filename queried from youtube-dl when in
--fast mode.
Fix several places where files in .git/annex/ were written with modes that did not take the core.sharedRepository config into account. git grep writeFile finds some more that might also be problems, but for now I've concentrated on .git/annex/ log files. There are certianly cases where writeFile is not a problem too. This commit was sponsored by mo on Patreon.
2018-01-02 21:17:10 +00:00
* Fix several places where files in .git/annex/ were written with modes
that did not take the core.sharedRepository config into account.
Improve startup time for commands that do not operate on remotes And for tab completion, by not unnessessarily statting paths to remotes, which used to cause eg, spin-up of removable drives. Got rid of the remotes member of Git.Repo. This was a bit painful. Remote.Git modifies the list of remotes as it reads their configs, so still need a persistent list of remotes. So, put it in as Annex.gitremotes. It's only populated by getGitRemotes, so commands like examinekey that don't care about remotes won't do so. This commit was sponsored by Jake Vosloo on Patreon.
2018-01-09 19:36:56 +00:00
* Improve startup time for commands that do not operate on remotes,
and for tab completion, by not unnessessarily statting paths to
remotes, which used to cause eg, spin-up of removable drives.
add remote.<name>.annex-checkuuid Added remote.<name>.annex-checkuuid config, which can be set to false to disable the default checking of the uuid of remotes that point to directories. This can be useful to avoid unncessary drive spin-ups and automounting. Note that the UUID check is still done before writing to the repository, to avoid writing to the wrong repository if it got relocated. Check is also done before checkPresent to avoid getting confused about what is in which repo. This is effectively the same as the use of git-annex-shell with a uuid to check that the remote repository is the expected one. Did not bother with the check for retrieveKeyFile because it doesn't matter if the wrong repo is used then. This commit was sponsored by Trenton Cronholm on Patreon.
2018-01-10 18:21:18 +00:00
* Added remote.<name>.annex-checkuuid config, which can be set to false
to disable the default checking of the uuid of remotes that point to
directories. This can be useful to avoid unncessary drive spin-ups and
automounting.
releasing package git-annex version 6.20180112
2018-01-12 19:47:32 +00:00
* git-annex.cabal: Add back custom-setup stanza, so cabal new-build works.
* git-annex.cabal: Removed the testsuite build flag; test suite is always
included.
Removed the testsuite build flag Test suite is always included. Building with this flag disabled has actually been broken for some time, since Command.TestRemote uses tasty. Fewer build flags are better, so good time to drop it. This commit was sponsored by Thomas Hochstein on Patreon.
2017-12-20 16:25:03 +00:00
releasing package git-annex version 6.20180112
2018-01-12 19:47:32 +00:00
-- Joey Hess <id@joeyh.name> Fri, 12 Jan 2018 15:45:48 -0400
Removed the testsuite build flag Test suite is always included. Building with this flag disabled has actually been broken for some time, since Command.TestRemote uses tasty. Fewer build flags are better, so good time to drop it. This commit was sponsored by Thomas Hochstein on Patreon.
2017-12-20 16:25:03 +00:00
switch distribution name to "upstream" to not imply that this changelog has anything to do with debian unstable
2018-01-05 17:35:35 +00:00
git-annex (6.20171214) upstream; urgency=medium
documentation update for youtube-dl Code not updated yet. This commit was sponsored by Thomas Hochstein on Patreon.
2017-11-28 18:05:58 +00:00
* Use youtube-dl rather than quvi to download media from web pages,
since quvi is not being actively developed and youtube-dl supports
many more sites.
rethought --relaxed change Better to make it not be surprising and slow, than surprising and fast. --raw can be used when it needs to be really fast. Implemented adding a youtube-dl supported url to an existing file. This commit was sponsored by andrea rota.
2017-11-30 17:45:43 +00:00
* addurl --relaxed got slower, since youtube-dl has to hit the network
to check for embedded media. If you relied on --relaxed not hitting the
network for speed reasons, using --relaxed --raw will get the old level
of speed, but can't be used for urls with embedded videos.
convert importfeed to youtube-dl Fully working, including --fast/--relaxed. Note that, while git-annex addurl --relaxed is not going to check youtube-dl, I kept git annex importfeed --relaxed checking it. Thinking is that, let's not break people's importfeed cron jobs, and importfeed does not typically have to check a large number of new items, so it's ok if it's a little bit slower when used with youtube playlist feeds. importfeed's behavior is also improved (?) when a feed has links in it to non-media files. Before, those were skipped. Now, the content of the link is downloaded. This had to be done, because trying to use youtube-dl is slow, and if those were skipped, it would have to check every time importfeed was run. While this behavior change may not be desirable for some feeds, that intersperse links to web pages with enclosures, it will be desirable for other feeds, that have non-enclosure directy links to media files. Remove old quvi modules. This commit was sponsored by Øyvind Andersen Holm.
2017-11-29 21:05:27 +00:00
* importfeed now downloads things linked to by feeds, even when they are
not media files.
Removed no longer needed dependency on yesod-default. This commit was sponsored by Nick Daly on Patreon.
2017-12-05 17:18:25 +00:00
* Removed no longer needed dependency on yesod-default.
Allow exporttree remotes to be marked as dead. Union with max so that DeadTrusted wins over UnTrusted. This commit was sponsored by Trenton Cronholm on Patreon.
2017-12-05 17:46:55 +00:00
* Allow exporttree remotes to be marked as dead.
fix gpg subkey support typo initremote, enableremote: Really support gpg subkeys suffixed with an exclamation mark, which forces gpg to use a specific subkey. (Previous try had a bug.) This commit was sponsored by Jake Vosloo on Patreon.
2017-12-05 17:58:53 +00:00
* initremote, enableremote: Really support gpg subkeys suffixed with an
exclamation mark, which forces gpg to use a specific subkey.
(Previous try had a bug.)
lookupkey absolute path support lookupkey: Support being given an absolute filename to a file within the current git repository. This commit was supported by the NSF-funded DataLad project.
2017-12-08 19:35:02 +00:00
* lookupkey: Support being given an absolute filename to a file
within the current git repository.
.noannex file A top-level .noannex file will prevent git-annex init from being used in a repository. This is useful for repositories that have a policy reason not to use git-annex. The content of the file will be displayed to the user who tries to run git-annex init. This also affects git annex reinit and initialization via the webapp. It does not affect automatic inits, when there's a sibling git-annex branch already. This commit was supported by the NSF-funded DataLad project.
2017-12-13 18:34:32 +00:00
* A top-level .noannex file will prevent git-annex init from being used
in a repository. This is useful for repositories that have a policy
reason not to use git-annex. The content of the file will be displayed
to the user who tries to run git-annex init.
documentation update for youtube-dl Code not updated yet. This commit was sponsored by Thomas Hochstein on Patreon.
2017-11-28 18:05:58 +00:00
releasing package git-annex version 6.20171214
2017-12-14 15:54:36 +00:00
-- Joey Hess <id@joeyh.name> Thu, 14 Dec 2017 11:50:48 -0400
documentation update for youtube-dl Code not updated yet. This commit was sponsored by Thomas Hochstein on Patreon.
2017-11-28 18:05:58 +00:00
releasing package git-annex version 6.20171124
2017-11-24 14:49:57 +00:00
git-annex (6.20171124) unstable; urgency=medium
Display progress meter when uploading a key without size information Getting the size by statting the content file. This commit was supported by the NSF-funded DataLad project.
2017-11-14 20:27:39 +00:00
* Display progress meter when uploading a key without size information,
getting the size by statting the content file.
Fix build with dns-3.0. This commit was sponsored by Henrik Riomar on Patreon.
2017-11-24 14:49:31 +00:00
* Fix build with dns-3.0.
Display progress meter when uploading a key without size information Getting the size by statting the content file. This commit was supported by the NSF-funded DataLad project.
2017-11-14 20:27:39 +00:00
releasing package git-annex version 6.20171124
2017-11-24 14:49:57 +00:00
-- Joey Hess <id@joeyh.name> Fri, 24 Nov 2017 10:49:36 -0400
Display progress meter when uploading a key without size information Getting the size by statting the content file. This commit was supported by the NSF-funded DataLad project.
2017-11-14 20:27:39 +00:00
releasing package git-annex version 6.20171109
2017-11-09 16:23:34 +00:00
git-annex (6.20171109) unstable; urgency=medium
Fix export of subdir of a branch. Seems I forgot to fully test that feature when documenting it. git rev-parse needs a colon after a branch to de-reference the tree it points to, rather than the commit. But that had it adding an extra colon when the user specified "branch:subdir". So, check if there is a colon before adding one. This commit was sponsored by Francois Marier on Patreon.
2017-10-30 16:02:18 +00:00
* Fix export of subdir of a branch.
Fix exporting of non-annexed files to external special remotes. Actual problem is the keyName was set to "Ref \"sha\"", which led to this follow-on failure since it contained a space. The bad data would also get into the export database when exporting to a non-external special remote. Looking briefly at that, I don't think the bad data will lead to anything more than a re-upload of the file content now that the problem has been fixed. This commit was sponsored by Peter Hogg on Patreon.
2017-10-30 16:47:13 +00:00
* Fix exporting of non-annexed files to external special remotes.
unlock, lock: Support --json.
2017-10-30 18:44:11 +00:00
* unlock, lock: Support --json.
Web.checkKey: Fix handling of multiple urls When there are multiple urls for a file, still treat it as being present in the web when some urls don't work, as long as at least one url does work. This is consistent with the other web methods handling of multiple urls. This commit was sponsored by Ole-Morten Duesund on Patreon.
2017-11-07 20:15:44 +00:00
* When there are multiple urls for a file, still treat it as being present
in the web when some urls don't work, as long as at least one url does
work.
Makefile improvement for sudo make install. Thanks, Eric Siegerman
2017-11-07 20:29:58 +00:00
* Makefile improvement for sudo make install.
Thanks, Eric Siegerman
Makefile improvement for BUILDER=stack, use stack to run ghc.
2017-11-07 20:56:48 +00:00
* Makefile improvement for BUILDER=stack, use stack to run ghc.
testremote: Test exporttree. As long as the class of remotes supports exporting, it's tested whether or not the remote is configured with exporttree=yes. Also, made testremote of a remote configured with exporttree=yes disable that configuration for testing non-export storage. This commit was supported by the NSF-funded DataLad project.
2017-11-08 18:22:05 +00:00
* testremote: Test exporttree.
Fix directory special remote's cleanup of empty export directories. Was trying to rmdir the file, which had already been deleted, and when that failed, it skipped trying to delete the parent directories. Noticed the bug through testremote, but it can't itself detect such problems as there is no enumeration in the API. This commit was sponsored by Brock Spratlen on Patreon.
2017-11-08 18:38:24 +00:00
* Fix directory special remote's cleanup of empty export directories.
Fix export of subdir of a branch. Seems I forgot to fully test that feature when documenting it. git rev-parse needs a colon after a branch to de-reference the tree it points to, rather than the commit. But that had it adding an extra colon when the user specified "branch:subdir". So, check if there is a colon before adding one. This commit was sponsored by Francois Marier on Patreon.
2017-10-30 16:02:18 +00:00
releasing package git-annex version 6.20171109
2017-11-09 16:23:34 +00:00
-- Joey Hess <id@joeyh.name> Thu, 09 Nov 2017 12:21:49 -0400
Fix export of subdir of a branch. Seems I forgot to fully test that feature when documenting it. git rev-parse needs a colon after a branch to de-reference the tree it points to, rather than the commit. But that had it adding an extra colon when the user specified "branch:subdir". So, check if there is a colon before adding one. This commit was sponsored by Francois Marier on Patreon.
2017-10-30 16:02:18 +00:00
releasing package git-annex version 6.20171026
2017-10-26 18:02:40 +00:00
git-annex (6.20171026) unstable; urgency=medium
temporary hack to get windows build working Code for terminating processes on Windows is not linking anymore; made a warning be displayed instead. This breaks restarting the assistant and git annex assistant --stop. I hope to see the code added to the Win32 library, where it should fit better and should avoid whatever problem is making the linker not like it when included in git-annex. I opened an issue requesting its addition, here: https://github.com/haskell/win32/issues/91 This commit was sponsored by Thomas Hochstein on Patreon.
2017-10-24 17:16:40 +00:00
releasing package git-annex version 6.20171026
2017-10-26 18:02:40 +00:00
* Windows: Fix reversion that caused the path used to link
to annexed content to include the drive letter and full path, rather
than being relative. (`git annex fix` will fix up after this problem).
* Windows build fixed, and changed to use stack for more reliable build
temporary hack to get windows build working Code for terminating processes on Windows is not linking anymore; made a warning be displayed instead. This breaks restarting the assistant and git annex assistant --stop. I hope to see the code added to the Win32 library, where it should fit better and should avoid whatever problem is making the linker not like it when included in git-annex. I opened an issue requesting its addition, here: https://github.com/haskell/win32/issues/91 This commit was sponsored by Thomas Hochstein on Patreon.
2017-10-24 17:16:40 +00:00
environment.
Windows: Remove wget from bundle; it needs libraries that are not included, and git for windows includes curl which git-annex will use instead. wget was broken even in the previous old release of the windows bundle, this is not new breakage. msys-idn-11.dll and probably more would be needed to use it. git for windows includes msys-idn2-0.dll instead.
2017-10-25 17:51:31 +00:00
* Windows: Remove wget from bundle; it needs libraries that are not
included, and git for windows includes curl which git-annex will use
instead.
Add day to metadata when annex.genmetadata is enabled. Thanks, Sean T Parsons
2017-10-25 19:11:12 +00:00
* Add day to metadata when annex.genmetadata is enabled.
Thanks, Sean T Parsons
releasing package git-annex version 6.20171026
2017-10-26 18:02:40 +00:00
* stack.yaml: Added nix packages section.
Thanks, Sean T Parsons
temporary hack to get windows build working Code for terminating processes on Windows is not linking anymore; made a warning be displayed instead. This breaks restarting the assistant and git annex assistant --stop. I hope to see the code added to the Win32 library, where it should fit better and should avoid whatever problem is making the linker not like it when included in git-annex. I opened an issue requesting its addition, here: https://github.com/haskell/win32/issues/91 This commit was sponsored by Thomas Hochstein on Patreon.
2017-10-24 17:16:40 +00:00
releasing package git-annex version 6.20171026
2017-10-26 18:02:40 +00:00
-- Joey Hess <id@joeyh.name> Thu, 26 Oct 2017 13:56:18 -0400
temporary hack to get windows build working Code for terminating processes on Windows is not linking anymore; made a warning be displayed instead. This breaks restarting the assistant and git annex assistant --stop. I hope to see the code added to the Win32 library, where it should fit better and should avoid whatever problem is making the linker not like it when included in git-annex. I opened an issue requesting its addition, here: https://github.com/haskell/win32/issues/91 This commit was sponsored by Thomas Hochstein on Patreon.
2017-10-24 17:16:40 +00:00
releasing package git-annex version 6.20171018
2017-10-18 19:41:08 +00:00
git-annex (6.20171018) unstable; urgency=medium
webdav: Make --debug show all webdav operations.
2017-10-07 18:11:32 +00:00
reorder
2017-10-18 17:36:43 +00:00
* add: Replace work tree file atomically on systems supporting hard
links. Avoids a window where interrupting an add could result in
the file being moved into the annex, with no symlink yet created.
webdav: Avoid unncessisarily creating the collection at the top of the repo when storing files there, since that collection is created by initremote. (This seems to work around some brokenness of the box.com webdav server which was entering a redirect loop.) Note that the fix makes locationParent return Nothing instead of "." when there's no parent directory between the path and the top of the webdav repo. This commit was sponsored by André Pereira on Patreon.
2017-10-11 15:09:27 +00:00
* webdav: Avoid unncessisarily creating the collection at the top
of the repository when storing files there, since that collection
is created by initremote.
(This seems to work around some brokenness of the box.com webdav
close bug
2017-10-16 18:50:30 +00:00
server, which caused uploads to be very slow or sometimes fail.)
reorder
2017-10-18 17:36:43 +00:00
* webdav: Make --debug show all webdav operations.
better dup key with -J fix This avoids all the complication about redundant work discussed in the previous try at fixing this. At the expense of needing each command that could have the problem to be patched to simply wrap the action in onlyActionOn once the key is known. But there do not seem to be many such commands. onlyActionOn' should not be used with a CommandStart (or CommandPerform), although the types do allow it. onlyActionOn handles running the whole CommandStart chain. I couldn't immediately see a way to avoid mistken use of onlyActionOn'. This commit was supported by the NSF-funded DataLad project.
2017-10-17 21:54:38 +00:00
* get -J/move -J/copy -J/mirror -J/sync -J: Avoid "transfer already in
progress" errors when two files use the same key.
changelog
2017-10-18 17:34:53 +00:00
* Konqueror desktop file location changed to one used by plasma 5.
Thanks, Félix Sipma.
reorder
2017-10-18 17:36:43 +00:00
* Avoid repeated checking that files passed on the command line exist.
* Fix build with aws-0.17.
* stack.yaml: Update to lts-9.9.
webdav: Make --debug show all webdav operations.
2017-10-07 18:11:32 +00:00
releasing package git-annex version 6.20171018
2017-10-18 19:41:08 +00:00
-- Joey Hess <id@joeyh.name> Wed, 18 Oct 2017 15:40:06 -0400
webdav: Make --debug show all webdav operations.
2017-10-07 18:11:32 +00:00
releasing package git-annex version 6.20171003
2017-10-03 17:18:47 +00:00
git-annex (6.20171003) unstable; urgency=medium
webdav: Improve error message for failed request to include the request method and path.
2017-09-28 16:01:58 +00:00
* webdav: Improve error message for failed request to include the request
method and path.
metadata: Added --remove-all. Motivation is to remove all metadata when it gets copied from a previous version of the file, and that is not deisrable. This commit was supported by the NSF-funded DataLad project.
2017-09-28 16:36:10 +00:00
* metadata: Added --remove-all.
Warn when metadata is inherited from a previous version of a file to avoid the user being surprised in cases where that behavior is not desired or expected This commit was supported by the NSF-funded DataLad project.
2017-09-28 16:56:35 +00:00
* Warn when metadata is inherited from a previous version of a file,
to avoid the user being surprised in cases where that behavior is not
desired or expected.
sync: Added --cleanup, which removes local and remote synced/ branches. Also deletes any tagged pushes that the assistant might have done, since those would also prevent resetting a branch back. This commit was sponsored by andrea rota.
2017-09-28 18:14:07 +00:00
* sync: Added --cleanup, which removes local and remote synced/ branches.
external crash fixes When the external special remote program crashed, a newline could be output, which messed up the expected output for --batch mode. Avoid checking EXPORTSUPPORTED for special remotes that are not configured to use exports. The datalad special remote apparently is/was buggy and crashed on EXPORTSUPPORTED. Anyway, there's no need to send it when the configuration doesn't need it. This commit was supported by the NSF-funded DataLad project.
2017-09-28 19:44:45 +00:00
* external: When the external special remote program crashed, a newline
could be output, which messed up the expected output for --batch mode.
* external: Avoid checking EXPORTSUPPORTED for special remotes that are
not configured to use exports.
test: Fix reversion that made it only run inside a git repository. Using annexeval to run probeCrippledFileSystem' caused Git.CurrentRepo.get to be run. Fixed easily since probeCrippledFileSystem' had no need to use the Annex monad. This commit was sponsored by Ethan Aubin.
2017-09-29 18:58:23 +00:00
* test: Fix reversion that made it only run inside a git repository.
copy, move: Behave same with --fast when sending to remotes located on a local disk as when sending to other remotes. Let --fast override use of hasKey even when hasKeyCheap.
2017-09-29 20:30:43 +00:00
* copy, move: Behave same with --fast when sending to remotes located
on a local disk as when sending to other remotes.
fix process and FD leak Fix process and file descriptor leak that was exposed when git-annex was built with ghc 8.2.1. Apparently ghc has changed its behavior of GC of open file handles that are pipes to running processes. That broke git-annex test on OSX due to running out of FDs. Audited for all uses of Annex.new and made stopCoProcesses be called once it's done with the state. Fixed several places that might have leaked in other situations than running the test suite. This commit was sponsored by Ewen McNeill.
2017-09-30 02:36:08 +00:00
* Fix process and file descriptor leak that was exposed when
git-annex was built with ghc 8.2.1. Broke git-annex test on OSX
due to running out of FDs, and may have also leaked in other situations.
info: Improve cleanup of stale transfer info files. In my git-annex repos, I found some stale transfer info files without lock files. Pass a mode to tryLockExclusive, so it will create the lock file if not present, and so not fail to clean up such transfer info files. Normally, transfer info files are accompanied by a lock file. But, when alwaysRunTransfer is used, the locking can fail and it will still write the transfer info file. Perhaps there are other cases too? Note that mkProgressUpdater's meter writes to the transfer info file too, and it might be possible for that meter to fire after runTransfer has cleaned up. This commit was sponsored by andrea rota.
2017-10-02 17:55:26 +00:00
* info: Improve cleanup of stale transfer info files.
webdav: Improve error message for failed request to include the request method and path.
2017-09-28 16:01:58 +00:00
releasing package git-annex version 6.20171003
2017-10-03 17:18:47 +00:00
-- Joey Hess <id@joeyh.name> Tue, 03 Oct 2017 13:18:15 -0400
webdav: Improve error message for failed request to include the request method and path.
2017-09-28 16:01:58 +00:00
releasing package git-annex version 6.20170925
2017-09-25 15:34:55 +00:00
git-annex (6.20170925) unstable; urgency=medium
Support building with feed-1.0, while still supporting older versions. This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-08-28 16:29:00 +00:00
update for export
2017-09-04 18:25:00 +00:00
* git-annex export: New command, can create and efficiently update
exports of trees to special remotes.
* Use git-annex initremote with exporttree=yes to set up a special remote
for use by git-annex export.
export to webdav This basically works, but there's a bug when renaming a file that leaves a .git-annex-temp-content-key file in the webdav store, that never gets cleaned up. Also, exporting files with spaces to box.com seems to fail; perhaps it does not support it? This commit was supported by the NSF-funded DataLad project.
2017-09-12 18:08:00 +00:00
* Implemented export to directory, S3, and webdav special remotes.
External special remote protocol extended to support export. Also updated example.sh to support export. This commit was supported by the NSF-funded DataLad project.
2017-09-08 18:24:05 +00:00
* External special remote protocol extended to support export.
releasing package git-annex version 6.20170925
2017-09-25 15:34:55 +00:00
Developers of external special remotes should consider if export makes
sense for them and add support.
* sync, assistant: Update tracking exports.
Support building with feed-1.0, while still supporting older versions. This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-08-28 16:29:00 +00:00
* Support building with feed-1.0, while still supporting older versions.
init: Display an additional message when it detects a filesystem that allows writing to files whose write bit is not set.
2017-08-28 17:21:18 +00:00
* init: Display an additional message when it detects a filesystem that
allows writing to files whose write bit is not set.
stop warning about removals from IA In a test, I uploaded a pdf, and several files were derived from it. After removing the pdf, the derived files went away after approximatly half an hour. This window does not seem worth warning about every time. Documented it in the tip.
2017-09-12 16:45:33 +00:00
* S3: Allow removing files from IA.
deal with box.com horrible infinite redirect behavior webdav: Checking if a non-existent file is present on Box.com triggered a bug in its webdav support that generates an infinite series of redirects. It seems to redirect foo to foo/ to foo/index.php to foo/index.php/index.php ... Why a webdav endpoint would behave this way who knows. Deal with such problems by assuming such behavior means the file is not present. Can't simply disable following redirects, because the webdav endpoint could legitimately be redirected to a new endpoint. So, when this happens 10 redirects have to be followed, before it gives up and assumes this means the file does not exist. This commit was supported by the NSF-funded DataLad project.
2017-09-12 19:13:42 +00:00
* webdav: Checking if a non-existent file is present on Box.com
triggered a bug in its webdav support that generates an infinite series
of redirects. Deal with such problems by assuming such behavior means
the file is not present.
webdav: Fix lack of url-escaping of filenames. inDAVLocation does not url-escape, and so exporting a filename with spaces to box.com at least resulted in a error 400. It might also have affected storing keys on a webdav remote, if the key contained a space or other problem character. Pretty unlikely. I emailed Clint about the inDAVLocation gotcha, but seems best to fix it here. This commit was supported by the NSF-funded DataLad project.
2017-09-12 19:45:03 +00:00
* webdav: Fix lack of url-escaping of filenames. Mostly impacted exports
of filenames containing eg spaces.
webdav: Changed path used on webdav server for temporary files. Done to avoid a "tmp" directory appearing in webdav exports. Also affects non-export webdav remotes, so interrupted uploads using the old path will not overwrite it. However, PUT is quite likely to be implemented atomically on web servers anyway, so I doubt this will cause problems.
2017-09-15 19:52:31 +00:00
* webdav: Changed path used on webdav server for temporary files.
Support building with feed-1.0, while still supporting older versions. This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-08-28 16:29:00 +00:00
releasing package git-annex version 6.20170925
2017-09-25 15:34:55 +00:00
-- Joey Hess <id@joeyh.name> Mon, 25 Sep 2017 11:13:58 -0400
Support building with feed-1.0, while still supporting older versions. This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-08-28 16:29:00 +00:00
releasing package git-annex version 6.20170818
2017-08-18 15:19:14 +00:00
git-annex (6.20170818) unstable; urgency=high
Fix build with QuickCheck 2.10. QuickCheck added an Arbitrary instance for CTime aka EpochTime. However, while git-annex's instance disallowed times before the epoch, QuickCheck's does not. So, rather than using its instance, convert from an Integer. This commit was sponsored by Thomas Hochstein on Patreon.
2017-06-17 17:04:48 +00:00
avoid the dashed ssh hostname class of security holes Security fix: Disallow hostname starting with a dash, which would get passed to ssh and be treated an option. This could be used by an attacker who provides a crafted ssh url (for eg a git remote) to execute arbitrary code via ssh -oProxyCommand. No CVE has yet been assigned for this hole. The same class of security hole recently affected git itself, CVE-2017-1000117. Method: Identified all places where ssh is run, by git grep '"ssh"' Converted them all to use a SshHost, if they did not already, for specifying the hostname. SshHost was made a data type with a smart constructor, which rejects hostnames starting with '-'. Note that git-annex already contains extensive use of Utility.SafeCommand, which fixes a similar class of problem where a filename starting with a dash gets passed to a program which treats it as an option. This commit was sponsored by Jochen Bartl on Patreon.
2017-08-18 02:11:31 +00:00
* Security fix: Disallow hostname starting with a dash, which
would get passed to ssh and be treated an option. This could
releasing package git-annex version 6.20170818
2017-08-18 15:19:14 +00:00
be used by an attacker who provides a crafted repository url
to cause the victim to execute arbitrary code via -oProxyCommand.
CVE-2017-12976
2017-08-20 20:50:53 +00:00
CVE-2017-12976
avoid the dashed ssh hostname class of security holes Security fix: Disallow hostname starting with a dash, which would get passed to ssh and be treated an option. This could be used by an attacker who provides a crafted ssh url (for eg a git remote) to execute arbitrary code via ssh -oProxyCommand. No CVE has yet been assigned for this hole. The same class of security hole recently affected git itself, CVE-2017-1000117. Method: Identified all places where ssh is run, by git grep '"ssh"' Converted them all to use a SshHost, if they did not already, for specifying the hostname. SshHost was made a data type with a smart constructor, which rejects hostnames starting with '-'. Note that git-annex already contains extensive use of Utility.SafeCommand, which fixes a similar class of problem where a filename starting with a dash gets passed to a program which treats it as an option. This commit was sponsored by Jochen Bartl on Patreon.
2017-08-18 02:11:31 +00:00
(The same class of security hole recently affected git itself.)
releasing package git-annex version 6.20170818
2017-08-18 15:19:14 +00:00
* git-annex.cabal: Deal with breaking changes in Cabal 2.0.
Fix build with QuickCheck 2.10. QuickCheck added an Arbitrary instance for CTime aka EpochTime. However, while git-annex's instance disallowed times before the epoch, QuickCheck's does not. So, rather than using its instance, convert from an Integer. This commit was sponsored by Thomas Hochstein on Patreon.
2017-06-17 17:04:48 +00:00
* Fix build with QuickCheck 2.10.
fsck: Support --json. One use case is to get a list of files that fsck fails on, in order to eg, drop them from a remote. This commit was sponsored by Nick Daly on Patreon.
2017-06-26 17:40:57 +00:00
* fsck: Support --json.
releasing package git-annex version 6.20170818
2017-08-18 15:19:14 +00:00
* move, copy: Support --batch.
Added GIT_ANNEX_VECTOR_CLOCK environment variable Can be used to override the default timestamps used in log files in the git-annex branch. This is a dangerous environment variable; use with caution. Note that this only affects writing to the logs on the git-annex branch. It is not used for metadata in git commits (other env vars can be set for that). There are many other places where timestamps are still used, that don't get committed to git, but do touch disk. Including regular timestamps of files, and timestamps embedded in some files in .git/annex/, including the last fsck timestamp and timestamps in transfer log files. A good way to find such things in git-annex is to get for getPOSIXTime and getCurrentTime, although some of the results are of course false positives that never hit disk (unless git-annex gets swapped out..) So this commit does NOT necessarily make git-annex comply with some HIPPA privacy regulations; it's up to the user to determine if they can use it in a way compliant with such regulations. Benchmarking: It takes 0.00114 milliseconds to call getEnv "GIT_ANNEX_VECTOR_CLOCK" when that env var is not set. So, 100 thousand log files can be written with an added overhead of only 0.114 seconds. That should be by far swamped by the actual overhead of writing the log files and making the commit containing them. This commit was supported by the NSF-funded DataLad project.
2017-08-14 17:55:38 +00:00
* Added GIT_ANNEX_VECTOR_CLOCK environment variable, which can be used to
override the default timestamps used in log files in the git-annex
branch. This is a dangerous environment variable; use with caution.
Fix a git-annex test failure when run on NFS due to NFS lock files preventing directory removal. Should fix this: lock (v6 --force): FAIL Exception: .git/annex/keys: removeDirectoryRecursive: unsatisfied constraints (Directory not empty) Verified that the test case still catches the regression it's meant to. This commit was supported by the NSF-funded DataLad project.
2017-08-14 19:11:42 +00:00
* Fix a git-annex test failure when run on NFS due to NFS lock files
preventing directory removal.
test: Avoid most situations involving failure to delete test directories By forking a worker process and only deleting the test directory once it exits. This way, if a test leaves files open, they'll get closed when the worker exits, so avoiding failure to delete open files on Windows, and failure to delete directories due to NFS lock files. If a test leaves a git worker process running, the closed pipes should cause the worker to exit too, also avoiding the problem there. The 10 second sleep ought to give plenty of time for such worker processes to exit, although this is of course a race. Finally, even if test directory fails to be deleted still, it won't appear as if the last test in the test suite failed; the error will be displayed at the very end. This commit was supported by the NSF-funded DataLad project.
2017-08-14 20:27:35 +00:00
* test: Avoid most situations involving failure to delete test
directories, by forking a worker process and only deleting the test
directory once it exits.
Disable http-client's default 30 second response timeout when HEADing an url to check if it exists. Some web servers take quite a long time to answer a HEAD request.
2017-08-15 17:56:12 +00:00
* Disable http-client's default 30 second response timeout when HEADing
an url to check if it exists. Some web servers take quite a long time
to answer a HEAD request.
add annex-ignore-command and annex-sync-command configs Added remote configuration settings annex-ignore-command and annex-sync-command, which are dynamic equivilants of the annex-ignore and annex-sync configurations. For this I needed a new DynamicConfig infrastructure. Its implementation should be as fast as before when there is no dynamic config, and it caches so shell commands are only run once. Note that annex-ignore-command exits nonzero when the remote should be ignored. While that may seem backwards, it allows using the same command for it as for annex-sync-command when you want to disable both. This commit was sponsored by Trenton Cronholm on Patreon.
2017-08-17 16:26:14 +00:00
* Added remote configuration settings annex-ignore-command and
annex-sync-command, which are dynamic equivilants of the annex-ignore
and annex-sync configurations.
Prevent spaces from being embedded in the name of new WORM keys, as that handing spaces in keys would complicate things like the external special remote protocol.
2017-08-17 18:46:33 +00:00
* Prevent spaces from being embedded in the name of new WORM keys,
as that handing spaces in keys would complicate things like the
external special remote protocol.
migrate: WORM keys containing spaces will be migrated to not contain spaces anymore To work around the problem that the external special remote protocol does not support keys containing spaces. This commit was sponsored by Denis Dzyubenko on Patreon.
2017-08-17 19:09:38 +00:00
* migrate: WORM keys containing spaces will be migrated to not contain
spaces anymore.
external: nice error message for keys with spaces in their name External special remotes will refuse to operate on keys with spaces in their names. That has never worked correctly due to the design of the external special remote protocol. Display an error message suggesting migration. Not super happy with this, but it's a pragmatic solution. Better than complicating the external special remote interface and all external special remotes. Note that I only made it use SafeKey in Request, not Response. git-annex does not construct a Response, so that would not add any safety. And presumably, if git-annex avoids feeding any such keys to an external special remote, it will never have a reason to make a Response using such a key. If it did, it would result in a protocol error anyway. There's still a Serializeable instance for Key; it's used by P2P.Protocol. There, the Key is always in the final position, so it's ok if it contains spaces. Note that the protocol documentation has been fixed to say that the File may contain spaces. One way that can happen, even though the Key can't, is when using direct mode, and the work tree filename contains spaces. When sending such a file to the external special remote the worktree filename is used. This commit was sponsored by Thom May on Patreon.
2017-08-17 20:08:35 +00:00
* External special remotes will refuse to operate on keys with spaces in
their names. That has never worked correctly due to the design of the
external special remote protocol. Display an error message suggesting
migration.
* Fix incorrect external special remote documentation, which said that
the filename parameter to the TRANSFER command could not contain
spaces. It can in fact contain spaces. Special remotes implementors
that relied on that may need to fix bugs in their special remotes.
fix external script for filenames with spaces from protocol Fix the external special remotes git-annex-remote-ipfs, git-annex-remote-torrent and the example.sh template to correctly support filenames with spaces. This commit was sponsored by John Peloquin on Patreon.
2017-08-17 20:20:09 +00:00
* Fix the external special remotes git-annex-remote-ipfs,
git-annex-remote-torrent and the example.sh template to correctly
support filenames with spaces.
releasing package git-annex version 6.20170818
2017-08-18 15:19:14 +00:00
* Windows: Win32 package has subsumed Win32-extras; update dependency.
Fix build with QuickCheck 2.10. QuickCheck added an Arbitrary instance for CTime aka EpochTime. However, while git-annex's instance disallowed times before the epoch, QuickCheck's does not. So, rather than using its instance, convert from an Integer. This commit was sponsored by Thomas Hochstein on Patreon.
2017-06-17 17:04:48 +00:00
releasing package git-annex version 6.20170818
2017-08-18 15:19:14 +00:00
-- Joey Hess <id@joeyh.name> Fri, 18 Aug 2017 11:19:06 -0400
Fix build with QuickCheck 2.10. QuickCheck added an Arbitrary instance for CTime aka EpochTime. However, while git-annex's instance disallowed times before the epoch, QuickCheck's does not. So, rather than using its instance, convert from an Integer. This commit was sponsored by Thomas Hochstein on Patreon.
2017-06-17 17:04:48 +00:00
releasing package git-annex version 6.20170520
2017-06-12 17:55:00 +00:00
git-annex (6.20170520) unstable; urgency=medium
initremote, enableremote: Support gpg subkeys suffixed with an exclamation mark, which forces gpg to use a specific subkey. This commit was sponsored by Peter Hogg on Patreon.
2017-05-24 18:08:02 +00:00
move --to=here * move --to=here moves from all reachable remotes to the local repository. The output of move --from remote is changed slightly, when the remote and local both have the content. It used to say: move foo ok Now: move foo (from theremote...) ok That was done so that, when move --to=here is used and the content is locally present and also in several remotes, it's clear which remotes the content gets dropped from. Note that move --to=here will report an error if a non-reachable remote contains the file, even if the local repository also contains the file. I think that's reasonable; the user may be intending to move all other copies of the file from remotes. OTOH, if a copy of the file is believed to be present in some repository that is not a configured remote, move --to=here does not report an error. So a little bit inconsistent, but erroring in this case feels wrong. copy --to=here came along for free, but it's basically the same behavior as git-annex get, and probably with not as good messages in edge cases (especially on failure), so I've not documented it. This commit was sponsored by Anthony DeRobertis on Patreon.
2017-05-31 20:57:27 +00:00
* move --to=here moves from all reachable remotes to the local repository.
initremote, enableremote: Support gpg subkeys suffixed with an exclamation mark, which forces gpg to use a specific subkey. This commit was sponsored by Peter Hogg on Patreon.
2017-05-24 18:08:02 +00:00
* initremote, enableremote: Support gpg subkeys suffixed with an
exclamation mark, which forces gpg to use a specific subkey.
Improve progress display when watching file size, in cases where a transfer does not resume. This commit was supported by the NSF-funded DataLad project.
2017-05-25 18:30:18 +00:00
* Improve progress display when watching file size, in cases where
a transfer does not resume.
Fix transfer log file locking problem when running concurrent transfers. orElse is great, but was not the right thing to use here because waitTakeLock could retry for other reasons than the lock being held, which made tryTakeLock fail when it shouldn't. Instead, move the code to tryTakeLock and implement waitTakeLock using tryTakeLock and retry. (Also, in runTransfer, when checkSaneLock fails, dropLock to avoid leaking a lock handle.) This commit was supported by the NSF-funded DataLad project.
2017-05-25 20:02:17 +00:00
* Fix transfer log file locking problem when running concurrent
transfers.
Avoid concurrent git-config setting problem when running concurrent threads. See my comment. This only avoids the problem for -J; two git-annex processes started at the same time could still both try to write to .git/config and one fail. That would be very unlikely though, and it doesn't really seem worth adding an additional layer of locking around .git/config. This commit was supported by the NSF-funded DataLad project.
2017-05-25 22:27:01 +00:00
* Avoid concurrent git-config setting problem when running concurrent
threads.
error when metadata set is used with file that does not exist When setting metadata of a file that did not exist, no error message was displayed, unlike getting metadata and most other git-annex commands. Fixed this oversight. Note that, if the file exists but is not annexed, there's no error. This is the same behavior as other git-annex commands. This commit was supported by the NSF-funded DataLad project.
2017-06-01 15:40:47 +00:00
* metadata: When setting metadata of a file that did not exist,
no error message was displayed, unlike getting metadata and most other
git-annex commands. Fixed this oversight.
configuration to disable automatic merge conflict resolution * Added annex.resolvemerge configuration, which can be set to false to disable the usual automatic merge conflict resolution done by git-annex sync and the assistant. * sync: Added --no-resolvemerge option. Note that disabling merge conflict resolution is probably not a good idea in a direct mode repo or adjusted branch. Since updates to both are done outside the usual work tree, if it fails the tree is not left in a conflicted state, and it would be hard to manually resolve the conflict. Still, made annex.resolvemerge be supported in those cases for consistency. This commit was sponsored by Riku Voipio.
2017-06-01 16:46:36 +00:00
* Added annex.resolvemerge configuration, which can be set to false to
disable the usual automatic merge conflict resolution done by git-annex
sync and the assistant.
* sync: Added --no-resolvemerge option.
Avoid error about git-annex-shell not being found when syncing with -J with a git remote where git-annex-shell is not installed. This commit was sponsored by andrea rota.
2017-06-06 16:57:27 +00:00
* Avoid error about git-annex-shell not being found when
syncing with -J with a git remote where git-annex-shell is not
installed.
Fix bug that prevented transfer locks from working when run on SMB or other filesystem that does not support fcntl locks and hard links. This commit was sponsored by Ethan Aubin.
2017-06-06 18:21:43 +00:00
* Fix bug that prevented transfer locks from working when
run on SMB or other filesystem that does not support fcntl locks
and hard links.
assistant: Merge changes from refs/remotes/foo/master into master. Previously, only sync branches were merged. This makes regular git push into a repository watched by the assistant auto-merge. While this does hardcode an assumption about what the remote tracking branch is named, which some unusual git configurations won't match, git-annex sync already made the same assumption. Also, changed behavior when a tracking branch like refs/remotes/synced/not/master is received. When on the master branch, that used to get merged into it, but it's the tracking branch for not/master, so should only be merged in when on the not/master branch. This commit was sponsored by Ewen McNeill.
2017-06-07 20:16:22 +00:00
* assistant: Merge changes from refs/remotes/foo/master into master.
Previously, only sync branches were merged. This makes regular git push
into a repository watched by the assistant auto-merge.
zsh and fish completions optparse-applicative-0.14.0.0 adds support for these, so have the Makefile install their scripts when built with it. CmdLine/GitAnnex/Options.hs now uses action "file" in cmdParams, which affects the bash and zsh completions, letting them complete filenames for subcommands that use that. This is not needed for bash, since bash-completion.bash enables -o bashdefault, which lets it complete filenames too. But it does not seem to break the bash completions. It is needed for zsh; the zsh completion otherwise does not complete filenames. The fish completion will always complete filenames no matter what. Messy. This commit was sponsored by Denis Dzyubenko on Patreon.
2017-06-09 15:38:20 +00:00
* Makefile: Install completions for the fish and zsh shells
when git-annex is built with optparse-applicative-0.14.
disable closingTracked on OSX Don't trust OSX FSEvents's eventFlagItemModified to be called when the last writer of a file closes it; apparently that sometimes does not happen, which prevented files from being quickly added. This commit was sponsored by John Peloquin on Patreon.
2017-06-09 18:18:58 +00:00
* assistant: Don't trust OSX FSEvents's eventFlagItemModified to be called
when the last writer of a file closes it; apparently that sometimes
does not happen, which prevented files from being quickly added.
initremote, enableremote: Support gpg subkeys suffixed with an exclamation mark, which forces gpg to use a specific subkey. This commit was sponsored by Peter Hogg on Patreon.
2017-05-24 18:08:02 +00:00
releasing package git-annex version 6.20170520
2017-06-12 17:55:00 +00:00
-- Joey Hess <id@joeyh.name> Mon, 12 Jun 2017 13:37:16 -0400
initremote, enableremote: Support gpg subkeys suffixed with an exclamation mark, which forces gpg to use a specific subkey. This commit was sponsored by Peter Hogg on Patreon.
2017-05-24 18:08:02 +00:00
releasing package git-annex version 6.20170519
2017-05-19 14:58:03 +00:00
git-annex (6.20170519) unstable; urgency=medium
Ssh password prompting improved when using -J When ssh connection caching is enabled (and when GIT_ANNEX_USE_GIT_SSH is not set), only one ssh password prompt will be made per host, and only one ssh password prompt will be made at a time. This also fixes a race in prepSocket's stale ssh connection stopping when run with -J. It was possible for one thread to start a cached ssh connection, and another thread to immediately stop it, resulting in excess connections being made. This commit was supported by the NSF-funded DataLad project.
2017-05-11 21:33:18 +00:00
* Ssh password prompting improved when using -J for concurrency.
When ssh connection caching is enabled (and when GIT_ANNEX_USE_GIT_SSH
is not set), only one ssh password prompt will be made per host, and
only one ssh password prompt will be made at a time.
clear regions before ssh prompt When built with concurrent-output 1.9, ssh password prompts will no longer interfere with the -J display. To avoid flicker, only done when ssh actually does need to prompt; ssh is first run in batch mode and if that succeeds the connection is up and no need to clear regions. This commit was supported by the NSF-funded DataLad project.
2017-05-16 19:28:06 +00:00
* When built with concurrent-output 1.9, ssh password prompts will no
longer interfere with the -J display.
adeiu, MissingH Removed dependency on MissingH, instead depending on the split library. After laying groundwork for this since 2015, it was mostly straightforward. Added Utility.Tuple and Utility.Split. Eyeballed System.Path.WildMatch while implementing the same thing. Since MissingH's progress meter display was being used, I re-implemented my own. Bonus: Now progress is displayed for transfers of files of unknown size. This commit was sponsored by Shane-o on Patreon.
2017-05-16 03:32:17 +00:00
* Removed dependency on MissingH, instead depending on the split library.
* Progress is displayed for transfers of files of unknown size.
Work around bug in git 2.13.0 involving GIT_COMMON_DIR that broke merging changes into adjusted branches. Might want to remove this when it gets fixed, in case adjusted branches are used in a repo with a great many refs, which would become unnecessarily slow. This commit was supported by the NSF-funded DataLad project.
2017-05-16 18:35:37 +00:00
* Work around bug in git 2.13.0 involving GIT_COMMON_DIR that broke
merging changes into adjusted branches.
Ssh password prompting improved when using -J When ssh connection caching is enabled (and when GIT_ANNEX_USE_GIT_SSH is not set), only one ssh password prompt will be made per host, and only one ssh password prompt will be made at a time. This also fixes a race in prepSocket's stale ssh connection stopping when run with -J. It was possible for one thread to start a cached ssh connection, and another thread to immediately stop it, resulting in excess connections being made. This commit was supported by the NSF-funded DataLad project.
2017-05-11 21:33:18 +00:00
releasing package git-annex version 6.20170519
2017-05-19 14:58:03 +00:00
-- Joey Hess <id@joeyh.name> Fri, 19 May 2017 10:37:57 -0400
Ssh password prompting improved when using -J When ssh connection caching is enabled (and when GIT_ANNEX_USE_GIT_SSH is not set), only one ssh password prompt will be made per host, and only one ssh password prompt will be made at a time. This also fixes a race in prepSocket's stale ssh connection stopping when run with -J. It was possible for one thread to start a cached ssh connection, and another thread to immediately stop it, resulting in excess connections being made. This commit was supported by the NSF-funded DataLad project.
2017-05-11 21:33:18 +00:00
releasing package git-annex version 6.20170510
2017-05-10 19:37:16 +00:00
git-annex (6.20170510) unstable; urgency=medium
stack.yaml: Update to lts-8.6.
2017-03-28 00:01:46 +00:00
When a http remote does not expose an annex.uuid config, only warn about it once, not every time git-annex is run. Same behavior as for a ssh remote.
2017-03-29 16:43:47 +00:00
* When a http remote does not expose an annex.uuid config, only warn
about it once, not every time git-annex is run.
multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting. This commit was supported by the NSF-funded DataLad project.
2017-03-30 23:32:58 +00:00
* multicast: New command, uses uftp to multicast annexed files, for eg
a classroom setting.
Added remote.<name>.annex-push and remote.<name>.annex-pull The former can be useful to make remotes that don't get fully synced with local changes, which comes up in a lot of situations. The latter was mostly added for symmetry, but could be useful (though less likely to be). Implementing `remote.<name>.annex-pull` was a bit tricky, as there's no one place where git-annex pulls/fetches from remotes. I audited all instances of "fetch" and "pull". A few cases were left not checking this config: * Git.Repair can try to pull missing refs from a remote, and if the local repo is corrupted, that seems a reasonable thing to do even though the config would normally prevent it. * Assistant.WebApp.Gpg and Remote.Gcrypt and Remote.Git do fetches as part of the setup process of a remote. The config would probably not be set then, and having the setup fail seems worse than honoring it if it is already set. I have not prevented all the code that does a "merge" from merging branches from remotes with remote.<name>.annex-pull=false. That could perhaps be done, but it would need a way to map from branch name to remote name, and the way refspecs work makes that hard to get really correct. So if the user fetches manually, the git-annex branch will get merged, for example. Anther way of looking at/justifying this is that the setting is called "annex-pull", not "annex-merge". This commit was supported by the NSF-funded DataLad project.
2017-04-05 17:04:02 +00:00
* Added remote.<name>.annex-push and remote.<name>.annex-pull
which can be useful to make remotes that don't get fully synced with
local changes.
Disable git-annex's support for GIT_SSH and GIT_SSH_COMMAND, unless GIT_ANNEX_USE_GIT_SSH=1 is also set in the environment. This is necessary because as feared, the extra -n parameter that git-annex passes breaks uses of these environment variables that expect exactly the parameters that git passes. For example, see https://github.com/datalad/datalad/issues/1456 It would of course be possible to pre-close stdin before running ssh so not needing the -n, and I think that would not even break ssh's password caching. But it would probably involve a lot of work, possibly would need to deal with some layering violations, and would be error-prone. The really clean fix would be to make all the ssh stuff return a CreateProcess, which could have the handle closed when appropriate, but that would be a large reworing of the code base. This commit was supported by the NSF-funded DataLad project.
2017-04-07 15:35:20 +00:00
* Disable git-annex's support for GIT_SSH and GIT_SSH_COMMAND, unless
GIT_ANNEX_USE_GIT_SSH=1 is also set in the environment. This is
necessary because as feared, the extra -n parameter that git-annex
passes breaks uses of these environment variables that expect exactly
the parameters that git passes.
enableremote: When enabling a non-special remote, param=value parameters can't be used, so error out if any are provided. This commit was sponsored by Riku Voipio.
2017-04-07 17:14:53 +00:00
* enableremote: When enabling a non-special remote, param=value
parameters can't be used, so error out if any are provided.
gcrypt: Support re-enabling to change eg, encryption parameters. This was never supported before. And it doesn't re-encrypt the gcrypt repo to the new gcrypt-participants, but it does at least now not crash, and set gcrypt-participants. This commit was sponsored by andrea rota.
2017-04-07 18:10:21 +00:00
* enableremote: Fix re-enabling of special remotes that have a git
url, so that eg, encryption key changes take effect. They were silently
enableremote: Fix re-enabling of existing gcrypt remotes, so that eg, encryption key changes take effect. They were silently ignored, a reversion introduced in 6.20160527. I don't like this regular git remote special case in enableremote, but I can't see a way to get rid of it. So, check if the existing remote is a Remote.Git This commit was sponsored by Trenton Cronholm on Patreon.
2017-04-07 17:51:06 +00:00
ignored, a reversion introduced in 6.20160527.
gcrypt: Support re-enabling to change eg, encryption parameters. This was never supported before. And it doesn't re-encrypt the gcrypt repo to the new gcrypt-participants, but it does at least now not crash, and set gcrypt-participants. This commit was sponsored by andrea rota.
2017-04-07 18:10:21 +00:00
* gcrypt: Support re-enabling to change eg, encryption parameters.
This was never supported before.
git annex add -u now supported, analagous to git add -u Unlike git add -u, git annex add -u does not update the index for files removed from the working tree. But then, "git add ." stages removals, and "git annex add ." does not, so that's an existing divergence. Seems that --update --batch would need to run git ls-files once per line of batch input, which would surely be too slow, so just throw an error for that. This commit was supported by the NSF-funded DataLad project.
2017-04-07 19:55:34 +00:00
* git annex add -u now supported, analagous to git add -u
version: Added "dependency versions" line. This commit was sponsored by Anthony DeRobertis on Patreon.
2017-04-07 22:03:09 +00:00
* version: Added "dependency versions" line.
Keys marked as dead are now skipped by --all. fsck already special-cased dead keys to make --all not report errors with them, and it makes sense to also expand that to whereis. I think it makes sense for dead keys to be skipped by all uses of --all, so mistakes can be completely forgotten about and not come back to haunt us. The speed impact of testing if the key is dead is negligible for fsck and whereis, since they use the location log anyway and it gets cached. This does slow down a few commands that support --all, in particular metadata --all runs around 2x as slow. I don't think metadata --all is often used though. It might slow down copy/move/mirror --all and get --all. log --all is not affected (does not use the normal --all machinery). Dead keys will still be processed by --incomplete, --branch, --failed, and --key. Although it would be unlikely for a dead key to ave in incomplete or failed transfer. It seems to make perfect sense for --branch to process keys on the branch, even if dead. (fsck's special-casing of dead keys was left in, so if one of these options causes a dead key to be fscked, there will be a nice message.) This commit was supported by the NSF-funded DataLad project.
2017-05-09 16:55:21 +00:00
* Keys marked as dead are now skipped by --all.
annex.backend is the new name for what was annex.backends It takes a single key-value backend, rather than the unncessary and confusing list. The old option still works if set. Simplified some old old code too. This commit was sponsored by Thomas Hochstein on Patreon.
2017-05-09 19:04:07 +00:00
* annex.backend is the new name for what was annex.backends, and takes
a single key-value backend, rather than the unncessary and confusing
list. The old option still works if set.
stack.yaml: Update to lts-8.6.
2017-03-28 00:01:46 +00:00
releasing package git-annex version 6.20170510
2017-05-10 19:37:16 +00:00
-- Joey Hess <id@joeyh.name> Wed, 10 May 2017 15:05:22 -0400
stack.yaml: Update to lts-8.6.
2017-03-28 00:01:46 +00:00
releasing package git-annex version 6.20170321
2017-03-21 17:46:20 +00:00
git-annex (6.20170321) unstable; urgency=medium
Bugfix: Passing a command a filename that does not exist sometimes did not display an error, when a path to a directory was also passed. It was relying on segmentPaths to work correctly, so when it didn't, sometimes the file that did not exist got matched up with a non-null list of results. Fixed by always checking if each parameter exists. There are two reason segmentPaths might not work correctly. For one, it assumes that when the original list of paths has more than 100 paths, it's not worth paying the CPU cost to preserve input orders. And then, it fails when a directory such as "." or ".." or /path/to/repo is in the input list, and the list of found paths does not start with that same thing. It should probably not be using dirContains, but something else. But, it's not clear how to handle this fully. Consider when [".", "subdir"] has been expanded by git ls-files to ["subdir/1", "subdir/2"] -- Both of the inputs contained those results, so there's no one right answer for segmentPaths. All these would be equally valid: [["subdir/1", "subdir/2"], []] [[], ["subdir/1", "subdir/2"]] [["subdir/1"], [""subdir/2"]] So I've not tried to improve segmentPaths.
2017-03-02 17:06:20 +00:00
* Bugfix: Passing a command a filename that does not exist sometimes
did not display an error, when a path to a directory was also passed.
status: Propigate nonzero exit code from git status.
2017-03-02 18:09:42 +00:00
* status: Propigate nonzero exit code from git status.
Linux standalone builds put the bundled ssh last in PATH, so any system ssh will be preferred over it. This commit was sponsored by Denis Dzyubenko on Patreon.
2017-03-02 21:40:40 +00:00
* Linux standalone builds put the bundled ssh last in PATH,
so any system ssh will be preferred over it.
assistant: Add 1/200th second delay between checking each file in the full transfer scan, to avoid using too much CPU. The slowdown is not going to be large in typical small-ish repos. And it does not seem to matter if the assistant reacts a little bit slower in situations involving the expensive scan, since: a) Those situations typically involve getting back in sync after something has changed on a remote, often after a disconnect of some duration. So taking a few seconds more is not noticable. b) If the scan finds things that it needs to do, it will start blocking anyway after 10 transfers are queued (due to use of queueTransferWhenSmall). So, only the speed of finding the first 10 transfers will be impacted by this change. This commit was sponsored by Jochen Bartl on Patreon.
2017-03-06 17:32:47 +00:00
* assistant: Add 1/200th second delay between checking each file
in the full transfer scan, to avoid using too much CPU.
get -J: Improve distribution of jobs amoung remotes when there are more jobs than remotes. It was distributing jobs to remotes that were not being used by any other job. But, suppose that there are only 2 remotes, and -J10. In such a case, the first 2 downloads would be distributed amoung the 2 remotes, but the other 8 would all go to remote #1. Improved by keeping a counter of how many jobs are assigned to a remote, and prefer remotes with fewer jobs. Note use of Data.Map.Strict to avoid blowing up space. I kept the bang-patterns as-is, although probably not needed with Data.Map.Strict. This commit was sponsored by Jack Hill on Patreon.
2017-03-08 18:49:30 +00:00
* get -J: Improve distribution of jobs amoung remotes when there are more
jobs than remotes.
fsck -q: When a file has bad content, include the name of the file in the warning message. This commit was sponsored by Alexander Thompson on Patreon.
2017-03-08 19:15:20 +00:00
* fsck -q: When a file has bad content, include the name of the file
in the warning message.
Windows: Improve handling of shebang in external special remote program, searching for the program in the PATH. findShellCommand needs a full path to a file in order to check it for a shebang on Windows. It was being run with only the base name of the external special remote program, which would only work when it was in the current directory. This is why users in https://github.com/DanielDent/git-annex-remote-rclone/pull/10 and elsewhere were complaining that the previous improvements to git-annex didn't make git-remote-rclone work on Windows. Also, reworked checkearlytermination, which while it worked, seemed to rely on a race condition. And, improved its error messages. This commit was sponsored by Shane-o on Patreon.
2017-03-08 19:56:55 +00:00
* Windows: Improve handling of shebang in external special remote
program, searching for the program in the PATH.
Drop support for building with old versions of directory, feed, and http-types.
2017-03-10 19:57:41 +00:00
* Drop support for building with old versions of dns, http-conduit,
directory, feed, and http-types.
Windows: Fix bug in shell script shebang lookup code that caused a "delayed read on closed handle" error. The bug was that withFile closes the handle afterwards, but the content of the file was not read due to laziness. Using readFile avoids it. This commit was sponsored by Nick Daly on Patreon.
2017-03-13 20:20:52 +00:00
* Windows: Fix bug in shell script shebang lookup code that
caused a "delayed read on closed handle" error.
git-annex-shell: run all commands with noMessages Fix bug when used with a recently cloned repository, where "merging" messages were included in the output of configlist (and perhaps other commands) and caused a "Failed to get annex.uuid configuration" error. This does not seem to have been a reversion. I saw this with configlist, but it seems possible for other commands to be effected, and it might not always happen only after a fresh clone. Eg, if a foo/git-annex branch is pushed to the remote, the next git-annex-shell will auto-merge it and display the message. Decided to run all git-annex-shell commands with noMessages, even ones that don't currently use stdout for structured communication. Better to keep open the possibility for using stdout in the future. This commit was supported by the NSF-funded DataLad project
2017-03-17 16:28:08 +00:00
* git-annex-shell: Fix bug when used with a recently cloned repository,
where "merging" messages were included in the output of configlist
(and perhaps other commands) and caused a "Failed to get annex.uuid
configuration" error.
Support GIT_SSH and GIT_SSH_COMMAND They are handled close the same as they are by git. However, unlike git, git-annex sometimes needs to pass the -n parameter when using these. So, this has the potential for breaking some setup, and perhaps there ought to be a ANNEX_USE_GIT_SSH=1 needed to use these. But I'd rather avoid that if possible, so let's see if anyone complains. Almost all places where "ssh" was run have been changed to support the env vars. Anything still calling sshOptions does not support them. In particular, rsync special remotes don't. Seems that annex-rsync-transport already gives sufficient control there. (Fixed in passing: Remote.Helper.Ssh.toRepo used to extract remoteAnnexSshOptions and pass them to sshOptions, which was redundant since sshOptions also extracts those.) This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-03-17 20:02:47 +00:00
* Support GIT_SSH and GIT_SSH_COMMAND, which are handled close the same
as they are by git. However, unlike git, git-annex sometimes needs to
pass the -n parameter when using these.
sync --content-of=path For when you want to sync only some files' contents, not the whole working tree. This commit was sponsored by Anthony DeRobertis on Patreon.
2017-03-20 20:00:06 +00:00
* sync --content-of=path (-C path) added for when you want to sync
only some files' contents, not the whole working tree.
Bugfix: Passing a command a filename that does not exist sometimes did not display an error, when a path to a directory was also passed. It was relying on segmentPaths to work correctly, so when it didn't, sometimes the file that did not exist got matched up with a non-null list of results. Fixed by always checking if each parameter exists. There are two reason segmentPaths might not work correctly. For one, it assumes that when the original list of paths has more than 100 paths, it's not worth paying the CPU cost to preserve input orders. And then, it fails when a directory such as "." or ".." or /path/to/repo is in the input list, and the list of found paths does not start with that same thing. It should probably not be using dirContains, but something else. But, it's not clear how to handle this fully. Consider when [".", "subdir"] has been expanded by git ls-files to ["subdir/1", "subdir/2"] -- Both of the inputs contained those results, so there's no one right answer for segmentPaths. All these would be equally valid: [["subdir/1", "subdir/2"], []] [[], ["subdir/1", "subdir/2"]] [["subdir/1"], [""subdir/2"]] So I've not tried to improve segmentPaths.
2017-03-02 17:06:20 +00:00
releasing package git-annex version 6.20170321
2017-03-21 17:46:20 +00:00
-- Joey Hess <id@joeyh.name> Tue, 21 Mar 2017 11:27:38 -0400
Bugfix: Passing a command a filename that does not exist sometimes did not display an error, when a path to a directory was also passed. It was relying on segmentPaths to work correctly, so when it didn't, sometimes the file that did not exist got matched up with a non-null list of results. Fixed by always checking if each parameter exists. There are two reason segmentPaths might not work correctly. For one, it assumes that when the original list of paths has more than 100 paths, it's not worth paying the CPU cost to preserve input orders. And then, it fails when a directory such as "." or ".." or /path/to/repo is in the input list, and the list of found paths does not start with that same thing. It should probably not be using dirContains, but something else. But, it's not clear how to handle this fully. Consider when [".", "subdir"] has been expanded by git ls-files to ["subdir/1", "subdir/2"] -- Both of the inputs contained those results, so there's no one right answer for segmentPaths. All these would be equally valid: [["subdir/1", "subdir/2"], []] [[], ["subdir/1", "subdir/2"]] [["subdir/1"], [""subdir/2"]] So I've not tried to improve segmentPaths.
2017-03-02 17:06:20 +00:00
releasing package git-annex version 6.20170301.1
2017-03-01 16:46:26 +00:00
git-annex (6.20170301.1) unstable; urgency=medium
Fix reversion in yesterday's release that made SHA1E and MD5E backends not work.
2017-03-01 16:43:15 +00:00
* Fix reversion in yesterday's release that made SHA1E and MD5E backends
not work.
releasing package git-annex version 6.20170301.1
2017-03-01 16:46:26 +00:00
-- Joey Hess <id@joeyh.name> Wed, 01 Mar 2017 12:46:03 -0400
Fix reversion in yesterday's release that made SHA1E and MD5E backends not work.
2017-03-01 16:43:15 +00:00
6.20170301 version for hackage No changes from 6.20170228; a new version number was needed due to a problem with Hackage.
2017-03-01 16:05:42 +00:00
git-annex (6.20170301) unstable; urgency=medium
* No changes from 6.20170228; a new version number was needed due
to a problem with Hackage.
-- Joey Hess <id@joeyh.name> Wed, 01 Mar 2017 12:06:02 -0400
releasing package git-annex version 6.20170228
2017-02-28 18:41:57 +00:00
git-annex (6.20170228) unstable; urgency=medium
sync, merge: Fail when the current branch has no commits yet, instead of not merging in anything from remotes and appearing to succeed. At first I wanted to make it go ahead and merge into the newborn branch, so made it use Git.Branch.currentUnsafe to get the current branch. But that failed: fatal: ambiguous argument 'refs/heads/master..refs/heads/synced/master': unknown revision or path not in the working tree. A whole nother code path to handle merging into newborn branches seemed excessive, so went with displaying a warning and propigating failure status. This commit was sponsored by Brock Spratlen on Patreon.
2017-02-14 19:58:05 +00:00
annex.securehashesonly Cryptographically secure hashes can be forced to be used in a repository, by setting annex.securehashesonly. This does not prevent the git repository from containing files with insecure hashes, but it does prevent the content of such files from being pulled into .git/annex/objects from another repository. We want to make sure that at no point does git-annex accept content into .git/annex/objects that is hashed with an insecure key. Here's how it was done: * .git/annex/objects/xx/yy/KEY/ is kept frozen, so nothing can be written to it normally * So every place that writes content must call, thawContent or modifyContent. We can audit for these, and be sure we've considered all cases. * The main functions are moveAnnex, and linkToAnnex; these were made to check annex.securehashesonly, and are the main security boundary for annex.securehashesonly. * Most other calls to modifyContent deal with other files in the KEY directory (inode cache etc). The other ones that mess with the content are: - Annex.Direct.toDirectGen, in which content already in the annex directory is moved to the direct mode file, so not relevant. - fix and lock, which don't add new content - Command.ReKey.linkKey, which manually unlocks it to make a copy. * All other calls to thawContent appear safe. Made moveAnnex return a Bool, so checked all callsites and made them deal with a failure in appropriate ways. linkToAnnex simply returns LinkAnnexFailed; all callsites already deal with it failing in appropriate ways. This commit was sponsored by Riku Voipio.
2017-02-27 17:01:32 +00:00
* Cryptographically secure hashes can be forced to be used in a
repository, by setting annex.securehashesonly.
make fsck check annex.securehashesonly, and new tip for working around SHA1 collisions with git-annex This commit was sponsored by andrea rota.
2017-02-27 17:50:00 +00:00
This does not prevent the git repository from containing links
to insecure hashes, but it does prevent the content of such files
from being added to .git/annex/objects by any method.
reorg
2017-02-27 19:04:03 +00:00
* Tighten key parser to prevent SHA1 collision attacks generating
two keys that have the same SHA1. (Only done for keys that contain
a hash). This ensures that signed git commits of annexed files
will remain secure, as long as git-annex is using a secure hashing
backend.
make fsck check annex.securehashesonly, and new tip for working around SHA1 collisions with git-annex This commit was sponsored by andrea rota.
2017-02-27 17:50:00 +00:00
* fsck: Warn about any files whose content is present, that don't
use secure hashes, when annex.securehashesonly is set.
inheritable annex.securehashesonly * init: When annex.securehashesonly has been set with git-annex config, copy that value to the annex.securehashesonly git config. * config --set: As well as setting value in git-annex branch, set local gitconfig. This is needed especially for annex.securehashesonly, which is read only from local gitconfig and not the git-annex branch. doc/todo/sha1_collision_embedding_in_git-annex_keys.mdwn has the rationalle for doing it this way. There's no perfect solution; this seems to be the least-bad one. This commit was supported by the NSF-funded DataLad project.
2017-02-27 20:08:16 +00:00
* init: When annex.securehashesonly has been set with git-annex config,
copy that value to the annex.securehashesonly git config.
securehash matching Added --securehash option to match files using a secure hash function, and corresponding securehash preferred content expression. This commit was sponsored by Ethan Aubin.
2017-02-27 19:02:38 +00:00
* Added --securehash option to match files using a secure hash function,
and corresponding securehash preferred content expression.
sync, merge: Fail when the current branch has no commits yet, instead of not merging in anything from remotes and appearing to succeed. At first I wanted to make it go ahead and merge into the newborn branch, so made it use Git.Branch.currentUnsafe to get the current branch. But that failed: fatal: ambiguous argument 'refs/heads/master..refs/heads/synced/master': unknown revision or path not in the working tree. A whole nother code path to handle merging into newborn branches seemed excessive, so went with displaying a warning and propigating failure status. This commit was sponsored by Brock Spratlen on Patreon.
2017-02-14 19:58:05 +00:00
* sync, merge: Fail when the current branch has no commits yet, instead
of not merging in anything from remotes and appearing to succeed.
Run ssh with -n whenever input is not being piped into it ... to avoid it consuming stdin that it shouldn't. This fixes git-annex-checkpresentkey --batch remote, which didn't output results for all keys passed into it. Other git-annex commands that communicate with a remote over ssh may also have been consuming stdin that they shouldn't have, which could have impacted using them in eg, shell scripts. For example, a shell script reading files from stdin and passing them to git annex drop would be impacted by this bug, whenever git annex drop ran git-annex-shell checkpresent, it would consume part/all of the stdin that the shell script was supposed to consume. Fixed by adding a ConsumeStdin parameter to Annex.Ssh.sshOptions, which is used throughout git-annex to run ssh (in order for ssh connection caching to work). Every call site was checked to see if it used CreatePipe for stdin, and if not was marked NoConsumeStdin.
2017-02-15 19:08:46 +00:00
* Run ssh with -n whenever input is not being piped into it,
to avoid it consuming stdin that it shouldn't.
This fixes git-annex-checkpresentkey --batch remote,
which didn't output results for all keys passed into it. Other
git-annex commands that communicate with a remote over ssh may also
have been consuming stdin that they shouldn't have, which could have
impacted using them in eg, shell scripts.
sync: Improve integration with receive.denyCurrentBranch=updateInstead By displaying error messages from the remote then it fails to update its checked out branch. Error messages in the default receive.denyCurrentBranch are still suppressed, which matches user expectations. This commit was sponsored by Nick Daly on Patreon.
2017-02-15 20:13:30 +00:00
* sync: Improve integration with receive.denyCurrentBranch=updateInstead,
displaying error messages from the remote then it fails to update
its checked out branch.
post-recive hook to make updateInstead work in direct mode and adjusted branches * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 18:04:43 +00:00
* Added post-recieve hook, which makes updateInstead work with direct
mode and adjusted branches.
* init: Set up the post-receive hook.
sync hack to make updateInstead work on eg FAT sync: When syncing with a local repository located on a crippled filesystem, run the post-receive hook there, since it wouldn't get run otherwise. This makes pushing to repos on FAT-formatted removable drives update them when receive.denyCurrentBranch=updateInstead. Made Remote.Git export onLocal, which was cleaned up to not have so many caveats about its use. This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-02-17 19:21:39 +00:00
* sync: When syncing with a local repository located on a crippled
filesystem, run the post-receive hook there, since it wouldn't get run
otherwise. This makes pushing to repos on FAT-formatted removable
drives update them when receive.denyCurrentBranch=updateInstead.
make query commands not output extraneous messages config group groupwanted numcopies schedule wanted required: Avoid displaying extraneous messages about repository auto-init, git-annex branch merging, etc, when being used to get information.
2017-02-16 17:24:16 +00:00
* config group groupwanted numcopies schedule wanted required:
Avoid displaying extraneous messages about repository auto-init,
git-annex branch merging, etc, when being used to get information.
adjust: Fix behavior when used in a repository that contains submodules. Also fixed the LsFiles parser to not assume its output has a fixed width type field.
2017-02-20 17:44:55 +00:00
* adjust: Fix behavior when used in a repository that contains
submodules.
Run wget with -nv instead of -q, so it will display HTTP errors. This adds one extra line of output when a download is successful, after the progress bar. I don't much like that, but wget does not provide a way to show HTTP errors without it.
2017-02-20 19:14:56 +00:00
* Run wget with -nv instead of -q, so it will display HTTP errors.
make curl show http errors to stderr * Run curl with -S, so HTTP errors are displayed, even when it's otherwise silent. * When downloading in --json or --quiet mode, use curl in preference to wget, since curl is able to display only errors to stderr, unlike wget. This does mean that downloadQuiet is only silent on stdout, not necessarily on stderr, which affects a couple other calls of it. For example, downloading the .git/config of a http remote may show an error message now, perhaps with slightly suboptimal formatting due to other output.
2017-02-20 19:59:55 +00:00
* Run curl with -S, so HTTP errors are displayed, even when
it's otherwise silent.
* When downloading in --json or --quiet mode, use curl in preference
to wget, since curl is able to display only errors to stderr, unlike
wget.
status: Pass --ignore-submodules=when option on to git status. Didn't make --ignore-submodules without a value be handled because I can't see a way to make optparse-applicative parse that. I've opened a bug requesting a way to do that: https://github.com/pcapriotti/optparse-applicative/issues/243
2017-02-20 20:37:04 +00:00
* status: Pass --ignore-submodules=when option on to git status.
inheritable annex.securehashesonly * init: When annex.securehashesonly has been set with git-annex config, copy that value to the annex.securehashesonly git config. * config --set: As well as setting value in git-annex branch, set local gitconfig. This is needed especially for annex.securehashesonly, which is read only from local gitconfig and not the git-annex branch. doc/todo/sha1_collision_embedding_in_git-annex_keys.mdwn has the rationalle for doing it this way. There's no perfect solution; this seems to be the least-bad one. This commit was supported by the NSF-funded DataLad project.
2017-02-27 20:08:16 +00:00
* config --set: As well as setting value in git-annex branch,
set local gitconfig. This is needed especially for
annex.securehashesonly, which is read only from local gitconfig and not
the git-annex branch.
release prep
2017-02-28 17:28:34 +00:00
* Removed support for building with the old cryptohash library.
Building with that library made git-annex not support SHA3; it's time
for that to always be supported in case SHA2 dominoes.
* git-annex.cabal: Make crypto-api a dependency even when built w/o
webapp and test suite.
sync, merge: Fail when the current branch has no commits yet, instead of not merging in anything from remotes and appearing to succeed. At first I wanted to make it go ahead and merge into the newborn branch, so made it use Git.Branch.currentUnsafe to get the current branch. But that failed: fatal: ambiguous argument 'refs/heads/master..refs/heads/synced/master': unknown revision or path not in the working tree. A whole nother code path to handle merging into newborn branches seemed excessive, so went with displaying a warning and propigating failure status. This commit was sponsored by Brock Spratlen on Patreon.
2017-02-14 19:58:05 +00:00
releasing package git-annex version 6.20170228
2017-02-28 18:41:57 +00:00
-- Joey Hess <id@joeyh.name> Tue, 28 Feb 2017 14:39:47 -0400
sync, merge: Fail when the current branch has no commits yet, instead of not merging in anything from remotes and appearing to succeed. At first I wanted to make it go ahead and merge into the newborn branch, so made it use Git.Branch.currentUnsafe to get the current branch. But that failed: fatal: ambiguous argument 'refs/heads/master..refs/heads/synced/master': unknown revision or path not in the working tree. A whole nother code path to handle merging into newborn branches seemed excessive, so went with displaying a warning and propigating failure status. This commit was sponsored by Brock Spratlen on Patreon.
2017-02-14 19:58:05 +00:00
releasing package git-annex version 6.20170214
2017-02-14 18:56:11 +00:00
git-annex (6.20170214) unstable; urgency=medium
Increase default cost for p2p remotes from 200 to 1000. This makes git-annex prefer transferring data from special remotes when possible.
2017-01-06 19:23:30 +00:00
* Increase default cost for p2p remotes from 200 to 1000.
This makes git-annex prefer transferring data from special
remotes when possible.
Remove -j short option for --json-progress; that option was already taken for --json. This commit was sponsored by Trenton Cronholm.
2017-01-30 16:46:42 +00:00
* Remove -j short option for --json-progress; that option was already
taken for --json.
vicfg: Include the numcopies configuation. Docs say vicfg can configure everything from git-annex branch, so it ought to configure numcopies. Note that commenting out existing numcopies does not unset it. This commit was sponsored by Thom May on Patreon.
2017-01-30 19:11:26 +00:00
* vicfg: Include the numcopies configuation.
config: New command for storing configuration in the git-annex branch. Any config names can be set using this; git-annex commands will only look at specific ones that make sense and are worth the overhead of querying the branch. This might also be useful for storing whatever other config-type stuff the user might want to shove into the git-annex branch. This commit was sponsored by Jochen Bartl on Patreon.
2017-01-30 20:41:29 +00:00
* config: New command for storing configuration in the git-annex branch.
lost a changelog entry; put back
2017-02-03 18:40:53 +00:00
* annex.autocommit can be configured via git-annex config, to control
the default behavior in all clones of a repository.
New annex.synccontent config setting .. which can be set to true to make git annex sync default to --content. This may become the default at some point in the future. As well as being configuable by git config, it can be configured by git-annex config to control the default behavior in all clones of a repository. Had to add a separate --no-content switch to we can tell if it's been explicitly set, and should override annex.synccontent. If --content was the default, this complication would not be necessary. This commit was sponsored by Jake Vosloo on Patreon.
2017-02-03 18:31:17 +00:00
* New annex.synccontent config setting, which can be set to true to make
git annex sync default to --content. This may become the default at
some point in the future. As well as being configuable by git config,
it can be configured by git-annex config to control the default
behavior in all clones of a repository.
lts-7.18 finally! esqueleto finally got fixed, thanks to @bitemyapp Since XMPP was removed, the previous build failures related to it should no longer be a problem either. Meanwhile, lts-5.18 fails to build anymore on Debian due to linker hardening breaking the version of ghc stack uses with that version. This commit was sponsored by Francois Marier on Patreon.
2017-01-31 16:27:08 +00:00
* stack.yaml: Update to lts-7.18.
Some optimisations to string splitting code. Turns out that Data.List.Utils.split is slow and makes a lot of allocations. Here's a much simpler single character splitter that behaves the same (even in wacky corner cases) while running in half the time and 75% the allocations. As well as being an optimisation, this helps move toward eliminating use of missingh. (Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and allocates even more.) I have not benchmarked the effect on git-annex, but would not be surprised to see some parsing of eg, large streams from git commands run twice as fast, and possibly in less memory. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2017-01-31 22:40:42 +00:00
* Some optimisations to string splitting code.
unused: When large files are checked right into git, avoid buffering their contents in memory. This makes it a little bit slower since it has to check file size, but worth it to fix a potential memory use problem. This commit was sponsored by Fernando Jimenez on Patreon.
2017-01-31 23:07:24 +00:00
* unused: When large files are checked right into git, avoid buffering
their contents in memory.
unused: Improved memory use significantly when there are a lot of differences between branches. Argh, didn't need an accumulator here! I think I use accumulators a lot more than I need to when recusively processing lists.. This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-01-31 23:42:00 +00:00
* unused: Improved memory use significantly when there are a lot
of differences between branches.
wormhole pairing appid flag day 2021-12-31 Wormhole pairing will start to provide an appid to wormhole on 2021-12-31. An appid can't be provided now because Debian stable is going to ship a older version of git-annex that does not provide an appid. Assumption is that by 2021-12-31, this version of git-annex will be shipped in a Debian stable release. If that turns out to not be the case, this change will need to be cherry-picked into the git-annex in Debian stable, or its wormhole pairing will break. This commit was sponsored by Thomas Hochstein on Patreon.
2017-02-03 19:06:17 +00:00
* Wormhole pairing will start to provide an appid to wormhole on
2021-12-31. An appid can't be provided now because Debian stable is going
to ship a older version of git-annex that does not provide an appid.
Assumption is that by 2021-12-31, this version of git-annex will be
shipped in a Debian stable release. If that turns out to not be the
case, this change will need to be cherry-picked into the git-annex in
Debian stable, or its wormhole pairing will break.
Fix build with aws 0.16. Thanks, aristidb.
2017-02-07 17:01:57 +00:00
* Fix build with aws 0.16. Thanks, aristidb.
assistant: Make --autostart --foreground wait for the children it starts. Before, the --foreground was ignored when autostarting. This commit was sponsored by Denis Dzyubenko on Patreon.
2017-02-07 17:31:45 +00:00
* assistant: Make --autostart --foreground wait for the children it
starts. Before, the --foreground was ignored when autostarting.
initremote: When a uuid= parameter is passed, use the specified UUID for the new special remote, instead of generating a UUID. This can be useful in some situations, eg when the same data can be accessed via two different special remote backends.
2017-02-07 19:10:41 +00:00
* initremote: When a uuid= parameter is passed, use the specified
UUID for the new special remote, instead of generating a UUID.
This can be useful in some situations, eg when the same data can be
accessed via two different special remote backends.
import: Changed how --deduplicate, --skip-duplicates, and --clean-duplicates determine if a file is a duplicate Before, only content known to be present somewhere was considered a duplicate. Now, any content that has been annexed before will be considered a duplicate, even if all annexed copies of the data have been lost. Note that --clean-duplicates and --deduplicate still check numcopies, so won't delete duplicate files unless there's an annexed copy. This makes import use the same method as reinject --known. The man page already said that duplicate meant "its content is either present in the local repository already, or git-annex knows of another repository that contains it, or it was present in the annex before but has been removed now". So, this is really only bringing the implementation into line with the man page. This commit was sponsored by Jochen Bartl on Patreon.
2017-02-07 21:35:51 +00:00
* import: Changed how --deduplicate, --skip-duplicates, and
--clean-duplicates determine if a file is a duplicate.
Before, only content known to be present somewhere was considered
a duplicate. Now, any content that has been annexed before will be
considered a duplicate, even if all annexed copies of the data have
been lost.
Note that --clean-duplicates and --deduplicate still check
numcopies, so won't delete duplicate files unless there's an annexed
copy.
Make import --deduplicate and --skip-duplicates only hash once, not twice import: --deduplicate and --skip-duplicates were implemented inneficiently; they unncessarily hashed each file twice. They have been improved to only hash once. The new approach is to lock down (minimally) and hash files, and then reuse that information when importing them. This was rather tricky, especially in detecting changes to files while they are being imported. The output of import changed slightly. While before it silently skipped over files with eg --skip-duplicates, now it shows each file as it starts to act on it. Since every file is hashed first thing, it would otherwise not be clear what file import is chewing on. (Actually, it wasn't clear before when any of the duplicates switches were used.) This commit was sponsored by Alexander Thompson on Patreon.
2017-02-09 19:32:22 +00:00
* import: --deduplicate and --skip-duplicates were implemented
inneficiently; they unncessarily hashed each file twice. They have
been improved to only hash once.
import --reinject-duplicates This is the same as running git annex reinject --known, followed by git-annex import. The advantage to having it in one command is that it only has to hash each file once; the two commands have to hash the imported files a second time. This commit was sponsored by Shane-o on Patreon.
2017-02-09 19:40:44 +00:00
* import: Added --reinject-duplicates.
Added git template directory to Linux standalone tarball and OSX app bundle. Git does not provide a switch to find out where this directory is, and while the git-init man page says it will always be in /usr/share/git-core/templates, that's not the case on OSX with git installed from homebrew. So, I used a hack taking the --man-path and constructing a path from that. Works on both Debian and OSX at least.
2017-02-10 17:55:54 +00:00
* Added git template directory to Linux standalone tarball and OSX
app bundle.
Improve pid locking code to work on filesystems that don't support hard links. Probing for hard link support in the pid locking code is redundant since git-annex init already probes that. But, it didn't seem worth threading that data through; the pid locking code runs at most once per git-annex process, and only on unusual filesystems. Optimising a single hard link and unlink isn't worth it. This commit was sponsored by Francois Marier on Patreon.
2017-02-10 19:21:58 +00:00
* Improve pid locking code to work on filesystems that don't support hard
links.
S3: Fix check of uuid file stored in bucket, which was not working. The check was broken in two ways.. First, nowhere did it error out when checkUUIDFile found a different UUID already in the file. Instead, it overwrote the uuid file. And, checkUUIDFile's implementation was for some reason always failing with a ConnectionClosed exception. Apparently something to do with using two different runResourceT's and a response getting GCed inbetween. I'm pretty sure that used to work, but changed to a more obviously correct implementation. This commit was sponsored by Peter Hogg on Patreon.
2017-02-13 19:35:24 +00:00
* S3: Fix check of uuid file stored in bucket, which was not working.
Work around sqlite's incorrect handling of umask when creating databases. Refactored some common code into initDb. This only deals with the problem when creating new databases. If a repo got bad permissions into it, it's up to the user to deal with it. This commit was sponsored by Ole-Morten Duesund on Patreon.
2017-02-13 21:30:28 +00:00
* Work around sqlite's incorrect handling of umask when creating
databases.
Increase default cost for p2p remotes from 200 to 1000. This makes git-annex prefer transferring data from special remotes when possible.
2017-01-06 19:23:30 +00:00
releasing package git-annex version 6.20170214
2017-02-14 18:56:11 +00:00
-- Joey Hess <id@joeyh.name> Tue, 14 Feb 2017 14:22:00 -0400
Increase default cost for p2p remotes from 200 to 1000. This makes git-annex prefer transferring data from special remotes when possible.
2017-01-06 19:23:30 +00:00
releasing package git-annex version 6.20170101
2017-01-01 03:59:56 +00:00
git-annex (6.20170101) unstable; urgency=medium
Debian: Build webapp on armel.
2016-12-12 01:30:07 +00:00
webapp: full wormhole pairing UI (untested) This commit was sponsored by Riku Voipio.
2016-12-27 20:36:05 +00:00
* XMPP support has been removed from the assistant in this release.
If your repositories used XMPP to keep in sync, that will no longer
work, and you should enable some other remote to keep them in sync.
A ssh server is one way, or use the new Tor pairing feature.
releasing package git-annex version 6.20170101
2017-01-01 03:59:56 +00:00
* p2p --pair makes it easy to pair repositories, over Tor, using
p2p --pair with magic wormhole (untested) It builds. I have not tried to run it yet. :) This commit was sponsored by Jake Vosloo on Patreon.
2016-12-18 20:50:58 +00:00
Magic Wormhole codes to find the other repository.
releasing package git-annex version 6.20170101
2017-01-01 03:59:56 +00:00
See http://git-annex.branchable.com/tips/peer_to_peer_network_with_tor/
webapp: full wormhole pairing UI (untested) This commit was sponsored by Riku Voipio.
2016-12-27 20:36:05 +00:00
* webapp: The "Share with a friend" and "Share with your other devices"
pages have been changed to pair repositories using Tor and Magic Wormhole.
metadata --batch: Fix bug when conflicting metadata changes were made in the same batch run. 1 microsecond delay is ugly.. but, maintaining an queue of a list of timestamps and taking a new one from the queue each time around, or maintaining a timestamp counter, would probably be slower.
2016-12-13 15:07:49 +00:00
* metadata --batch: Fix bug when conflicting metadata changes were
made in the same batch run.
Pass annex.web-options to wget and curl after other options, so that eg --no-show-progress can be set by the user to disable the default --show-progress.
2016-12-13 15:56:23 +00:00
* Pass annex.web-options to wget and curl after other options, so that
eg --no-show-progress can be set by the user to disable the default
--show-progress.
Revert ServerAliveInterval Revert ServerAliveInterval change in 6.20161111, which caused problems with too many old versions of ssh and unusual ssh configurations. It should have not been needed anyway since ssh is supposted to have TCPKeepAlive enabled by default.
2016-12-13 16:05:05 +00:00
* Revert ServerAliveInterval change in 6.20161111, which caused problems
with too many old versions of ssh and unusual ssh configurations.
It should have not been needed anyway since ssh is supposted to
have TCPKeepAlive enabled by default.
Make all --batch input, as well as fromkey and registerurl stdin be processed without requiring it to be in the current encoding.
2016-12-13 19:35:04 +00:00
* Make all --batch input, as well as fromkey and registerurl stdin
be processed without requiring it to be in the current encoding.
p2p: --link no longer takes a remote name, instead the --name option can be used.
2016-12-16 19:36:59 +00:00
* p2p: --link no longer takes a remote name, instead the --name
option can be used.
Linux standalone: Improve generation of locale definition files, supporting locales such as, en_GB.UTF-8.
2016-12-19 21:03:52 +00:00
* Linux standalone: Improve generation of locale definition files,
releasing package git-annex version 6.20170101
2017-01-01 03:59:56 +00:00
supporting locales such as en_GB.UTF-8.
rekey --force: Incorrectly marked the new key's content as being present in the local repo even when it was not.
2016-12-19 22:18:57 +00:00
* rekey --force: Incorrectly marked the new key's content as being
present in the local repo even when it was not.
relocate tor socket out of /etc weasel explained that apparmor limits on what files tor can read do not apply to sockets (because they're not files). And apparently the problems I was seeing with hidden services not being accessible had to do with onion address propigation and not the location of the socket file. remotedaemon looks up the HiddenServicePort in torrc, so if it was previously configured with the socket in /etc, that will still work. This commit was sponsored by Denis Dzyubenko on Patreon.
2016-12-20 20:01:10 +00:00
* enable-tor: Put tor sockets in /var/lib/tor-annex/, rather
than in /etc/tor/hidden_service/.
enable-tor: No longer needs to be run as root. When run by not root, su's to root automatically. This commit was sponsored by Brock Spratlen on Patreon.
2016-12-20 21:40:36 +00:00
* enable-tor: No longer needs to be run as root.
releasing package git-annex version 6.20170101
2017-01-01 03:59:56 +00:00
* enable-tor: When run as a regular user, also tests a connection back to
enable-tor: When run as a regular user, test a connection back to the hidden service over tor. This way we know that after enable-tor, the tor hidden service is fully published and working, and so there should be no problems with it at pairing time. It has to start up its own temporary listener on the hidden service. It would be nice to have it start the remotedaemon running, so that extra step is not needed afterwards. But, there may already be a remotedaemon running, in communication with the assistant and we don't want to start another one. I thought about trying to HUP any running remotedaemon, but Windows does not make it easy to do that. In any case, having the user start the remotedaemon themselves lets them know it needs to be running to serve the hidden service. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2016-12-24 16:49:28 +00:00
the hidden service over tor.
Support all common locations of the torrc file.
2016-12-28 19:12:31 +00:00
* Support all common locations of the torrc file.
Always use filesystem encoding for all file and handle reads and writes. This is a big scary change. I have convinced myself it should be safe. I hope!
2016-12-24 18:46:31 +00:00
* Always use filesystem encoding for all file and handle reads and
writes.
Fix build with directory-1.3. See https://github.com/haskell/directory/issues/66
2016-12-20 19:23:59 +00:00
* Fix build with directory-1.3.
Debian: Suggest tor and magic-wormhole. Suggests, not recommends, because tor is not for everyone.
2016-12-20 19:26:14 +00:00
* Debian: Suggest tor and magic-wormhole.
* Debian: Build webapp on armel.
Debian: Build webapp on armel.
2016-12-12 01:30:07 +00:00
releasing package git-annex version 6.20170101
2017-01-01 03:59:56 +00:00
-- Joey Hess <id@joeyh.name> Sat, 31 Dec 2016 15:11:04 -0400
Debian: Build webapp on armel.
2016-12-12 01:30:07 +00:00
releasing package git-annex version 6.20161210
2016-12-10 16:23:18 +00:00
git-annex (6.20161210) unstable; urgency=medium
remotedaemon: Fork to background by default. Added --foreground switch to enable old behavior. Groundwork for tor hidden services, which the remotedaemon will serve.
2016-11-20 18:39:26 +00:00
linux standalone builds should have "unable to decommit memory" bug fixed
2016-12-11 19:37:52 +00:00
* Linux standalone: Updated ghc to fix its "unable to decommit memory"
bug, which may have resulted in data loss when these builds were used
with Linux kernels older than 4.5.
remotedaemon: serve tor hidden service
2016-11-20 19:45:01 +00:00
* enable-tor: New command, enables tor hidden service for P2P syncing.
implement p2p command
2016-11-30 18:35:24 +00:00
* p2p: New command, allows linking repositories using a P2P network.
remotedaemon: serve tor hidden service
2016-11-20 19:45:01 +00:00
* remotedaemon: Serve tor hidden service.
Added git-remote-tor-annex, which allows git pull and push to the tor hidden service. Almost working, but there's a bug in the relaying. Also, made tor hidden service setup pick a random port, to make it harder to port scan. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2016-11-21 21:27:38 +00:00
* Added git-remote-tor-annex, which allows git pull and push to the tor
hidden service.
remotedaemon: Fork to background by default. Added --foreground switch to enable old behavior. Groundwork for tor hidden services, which the remotedaemon will serve.
2016-11-20 18:39:26 +00:00
* remotedaemon: Fork to background by default. Added --foreground switch
to enable old behavior.
addurl: Fix bug in checking annex.largefiles expressions using largerthan, mimetype, and smallerthan; the first two always failed to match, and the latter always matched.
2016-11-21 15:30:53 +00:00
* addurl: Fix bug in checking annex.largefiles expressions using
largerthan, mimetype, and smallerthan; the first two always failed
to match, and the latter always matched.
Relicense 5 source files that are not part of the webapp from AGPL to GPL. Building w/o the webapp is not supposed to pull in any AGPLed files. I appear to have written all the code in these files; the only commit by anyone else is 64e844e1fefca513d143e3f74a54dfc6d526a2ff and is a spelling fix that is not copyrightable.
2016-11-22 03:46:59 +00:00
* Relicense 5 source files that are not part of the webapp from AGPL to GPL.
prefer xdot over dot * map: Run xdot if it's available in PATH. On OSX, the dot command does not support graphical display, while xdot does. * Debian: xdot is a better interactive viewer than dot, so Suggest xdot, rather than graphviz.
2016-11-30 16:50:49 +00:00
* map: Run xdot if it's available in PATH. On OSX, the dot command
does not support graphical display, while xdot does.
* Debian: xdot is a better interactive viewer than dot, so Suggest
xdot, rather than graphviz.
rmurl: --batch * rmurl: Multiple pairs of files and urls can be provided on the command line. * rmurl: Added --batch mode. This commit was sponsored by Trenton Cronholm on Patreon.
2016-12-05 16:10:07 +00:00
* rmurl: Multiple pairs of files and urls can be provided on the
command line.
* rmurl: Added --batch mode.
changelog
2016-12-05 16:16:35 +00:00
* fromkey: Accept multiple pairs of files and keys.
Thanks, Daniel Brooks.
rekey: Added --batch mode. Would have liked to make the Parser parse the file and key pairs, but it seems that optparse-applicative is unable to handle eg: many ((,) <$> argument <*> argument) This commit was sponsored by Thomas Hochstein on Patreon.
2016-12-05 16:55:50 +00:00
* rekey: Added --batch mode.
add: Stage modified non-large files when running in indirect mode. (This was already done in v6 mode and direct mode.)
2016-12-05 18:02:11 +00:00
* add: Stage modified non-large files when running in indirect mode.
(This was already done in v6 mode and direct mode.)
git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks. The attacker could just send a very lot of data, with no \n and it would all be buffered in memory until the kernel killed git-annex or perhaps OOM killed some other more valuable process. This is a low impact security hole, only affecting communication between local git-annex and git-annex-shell on the remote system. (With either able to be the attacker). Only those with the right ssh key can do it. And, there are probably lots of ways to construct git repositories that make git use a lot of memory in various ways, which would have similar impact as this attack. The fix in P2P/IO.hs would have been higher impact, if it had made it to a released version, since it would have allowed DOSing the tor hidden service without needing to authenticate. (The LockContent and NotifyChanges instances may not be really exploitable; since the line is read and ignored, it probably gets read lazily and does not end up staying buffered in memory.)
2016-12-09 17:34:00 +00:00
* git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks.
fixed
2016-12-10 14:46:51 +00:00
* Fix build with http-client 0.5.
Thanks, Alper Nebi Yasak.
remotedaemon: Fork to background by default. Added --foreground switch to enable old behavior. Groundwork for tor hidden services, which the remotedaemon will serve.
2016-11-20 18:39:26 +00:00
releasing package git-annex version 6.20161210
2016-12-10 16:23:18 +00:00
-- Joey Hess <id@joeyh.name> Sat, 10 Dec 2016 11:56:25 -0400
remotedaemon: Fork to background by default. Added --foreground switch to enable old behavior. Groundwork for tor hidden services, which the remotedaemon will serve.
2016-11-20 18:39:26 +00:00
releasing package git-annex version 6.20161118
2016-11-18 15:59:49 +00:00
git-annex (6.20161118) unstable; urgency=medium
unbreak all the autobuilders git-annex.cabal: Loosen bounds on persistent to allow 2.5, which on Debian has been patched to work with esqueleto. This may break cabal's resolver on non-Debian systems; if so, either use stack to build, or run cabal with --constraint='persistent ==2.2.4.1' Hopefully this mess with esqueleto will be resolved soon. https://github.com/prowdsponsor/esqueleto/issues/137
2016-11-15 15:19:57 +00:00
* git-annex.cabal: Loosen bounds on persistent to allow 2.5, which
on Debian has been patched to work with esqueleto.
This may break cabal's resolver on non-Debian systems;
if so, either use stack to build, or run cabal with
--constraint='persistent ==2.2.4.1'
Hopefully this mess with esqueleto will be resolved soon.
sync: Pass --allow-unrelated-histories to git merge when used with git git 2.9.0 or newer. This makes merging a remote into a freshly created direct mode repository work the same as it works in indirect mode. The git-annex branches would get merged in any case by a sync, since that doesn't use git merge. This might need to be revisited later to better mirror git's behavior.
2016-11-15 22:26:17 +00:00
* sync: Pass --allow-unrelated-histories to git merge when used with git
git 2.9.0 or newer. This makes merging a remote into a freshly created
direct mode repository work the same as it works in indirect mode.
Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. ghc 8 added backtraces on uncaught errors. This is great, but git-annex was using error in many places for a error message targeted at the user, in some known problem case. A backtrace only confuses such a message, so omit it. Notably, commands like git annex drop that failed due to eg, numcopies, used to use error, so had a backtrace. This commit was sponsored by Ethan Aubin.
2016-11-16 01:29:54 +00:00
* Avoid backtraces on expected failures when built with ghc 8;
only use backtraces for unexpected errors.
fsck --all --from was checking the content of files in the local repository, rather than on the special remote. Straight up forgot to handle this case! This commit was sponsored by Fernando Jimenez on Patreon.
2016-11-16 19:32:49 +00:00
* fsck --all --from was checking the existence and content of files
in the local repository, rather than on the special remote. Oops.
arm build uses 32kb page size (Change was made in gitannexbuilder scripts not here.)
2016-11-16 22:05:42 +00:00
* Linux arm standalone: Build with a 32kb page size, which is needed
on several ARM NAS devices, including Drobo 5N, and WD NAS.
unbreak all the autobuilders git-annex.cabal: Loosen bounds on persistent to allow 2.5, which on Debian has been patched to work with esqueleto. This may break cabal's resolver on non-Debian systems; if so, either use stack to build, or run cabal with --constraint='persistent ==2.2.4.1' Hopefully this mess with esqueleto will be resolved soon. https://github.com/prowdsponsor/esqueleto/issues/137
2016-11-15 15:19:57 +00:00
releasing package git-annex version 6.20161118
2016-11-18 15:59:49 +00:00
-- Joey Hess <id@joeyh.name> Fri, 18 Nov 2016 11:43:14 -0400
unbreak all the autobuilders git-annex.cabal: Loosen bounds on persistent to allow 2.5, which on Debian has been patched to work with esqueleto. This may break cabal's resolver on non-Debian systems; if so, either use stack to build, or run cabal with --constraint='persistent ==2.2.4.1' Hopefully this mess with esqueleto will be resolved soon. https://github.com/prowdsponsor/esqueleto/issues/137
2016-11-15 15:19:57 +00:00
releasing package git-annex version 6.20161111
2016-11-11 18:47:31 +00:00
git-annex (6.20161111) unstable; urgency=medium
restart coprocess in raw mode Restarting a crashing git process could result in filename encoding issues when not in a unicode locale, as the restarted processes's handles were not read in raw mode. Since rawMode is always used when starting a coprocess, didn't bother to parameterise it and just always enable it for simplicity. This commit was sponsored by Jake Vosloo on Patreon.
2016-11-01 18:03:55 +00:00
* Restarting a crashing git process could result in filename encoding
issues when not in a unicode locale, as the restarted processes's
handles were not read in raw mode.
Make .git/annex/ssh.config file work with versions of ssh older than 7.3, which don't support Include. When used with an older version of ssh, any ServerAliveInterval in ~/.ssh/config will be overridden by .git/annex/ssh.config. This commit was sponsored by Josh Taylor on Patreon.
2016-11-07 14:32:57 +00:00
* Make .git/annex/ssh.config file work with versions of ssh older than
7.3, which don't support Include. When used with an older version
of ssh, any ServerAliveInterval in ~/.ssh/config will be overridden
by .git/annex/ssh.config.
S3: Support the special case endpoint needed for the cn-north-1 region. * S3: Support the special case endpoint needed for the cn-north-1 region. * Webapp: Don't list the Frankfurt region, as this (and some other new regions) need V4 authorization which the aws library does not yet use. This commit was sponsored by Nick Daly on Patreon.
2016-11-07 15:49:34 +00:00
* S3: Support the special case endpoint needed for the cn-north-1 region.
releasing package git-annex version 6.20161111
2016-11-11 18:47:31 +00:00
* Webapp: Don't list the Frankfurt S3 region, as this (and some other new
S3: Support the special case endpoint needed for the cn-north-1 region. * S3: Support the special case endpoint needed for the cn-north-1 region. * Webapp: Don't list the Frankfurt region, as this (and some other new regions) need V4 authorization which the aws library does not yet use. This commit was sponsored by Nick Daly on Patreon.
2016-11-07 15:49:34 +00:00
regions) need V4 authorization which the aws library does not yet use.
reinject --known: Avoid second, unncessary checksum of file.
2016-11-07 16:07:36 +00:00
* reinject --known: Avoid second, unncessary checksum of file.
OSX: Remove RPATHs from git-annex binary, which are not needed, slow down startup, and break the OSX Sierra linker. ghc 8.0.2 may make this unncessary, but it's not in a stackage version yet, so put in a workaround. Note that the linux builds already delete the RPATHs for similar reasons. This commit was sponsored by Josh Taylor on Patreon.
2016-11-07 18:21:30 +00:00
* OSX: Remove RPATHs from git-annex binary, which are not needed,
slow down startup, and break the OSX Sierra linker.
webapp: Explicitly avoid checking for auth in static subsite requests. Yesod didn't used to do auth checks for that, but this may have changed. I don't have a way to reproduce the reported problem yet, but this change certianly won't hurt anything. This commit was sponsored by Thom May on Patreon.
2016-11-10 17:48:54 +00:00
* webapp: Explicitly avoid checking for auth in static subsite
requests. Yesod didn't used to do auth checks for that, but this may
have changed.
Linux standalone: Avoid using hard links in the tarball so it can be untarred on eg, afs which does not support them.
2016-11-10 19:12:08 +00:00
* Linux standalone: Avoid using hard links in the tarball so it can be
untarred on eg, afs which does not support them.
restart coprocess in raw mode Restarting a crashing git process could result in filename encoding issues when not in a unicode locale, as the restarted processes's handles were not read in raw mode. Since rawMode is always used when starting a coprocess, didn't bother to parameterise it and just always enable it for simplicity. This commit was sponsored by Jake Vosloo on Patreon.
2016-11-01 18:03:55 +00:00
releasing package git-annex version 6.20161111
2016-11-11 18:47:31 +00:00
-- Joey Hess <id@joeyh.name> Fri, 11 Nov 2016 14:46:39 -0400
restart coprocess in raw mode Restarting a crashing git process could result in filename encoding issues when not in a unicode locale, as the restarted processes's handles were not read in raw mode. Since rawMode is always used when starting a coprocess, didn't bother to parameterise it and just always enable it for simplicity. This commit was sponsored by Jake Vosloo on Patreon.
2016-11-01 18:03:55 +00:00
releasing package git-annex version 6.20161031
2016-10-31 22:56:18 +00:00
git-annex (6.20161031) unstable; urgency=medium
Assistant, repair: Fix ignoring of git fsck errors due to duplicate file entries in tree objects.
2016-10-31 18:00:37 +00:00
* Assistant, repair: Fix ignoring of git fsck errors due to
duplicate file entries in tree objects.
Linux standalone: Fix location of locale files in the bundle. The Makefile was putting them in git-annex.linux/i18n/i18n, and so I18NPATH did not point to the files. I think that on close enough to Debian systems, localedef then fell back to using the system-wide locale files, while on other systems it would fail to generate locales.
2016-10-31 18:31:46 +00:00
* Linux standalone: Fix location of locale files in the bundle.
Fix reversion in 6.20161012 that prevented adding files with a space in their name.
2016-10-31 22:35:38 +00:00
* Fix reversion in 6.20161012 that prevented adding files with a space
in their name.
Assistant, repair: Fix ignoring of git fsck errors due to duplicate file entries in tree objects.
2016-10-31 18:00:37 +00:00
releasing package git-annex version 6.20161031
2016-10-31 22:56:18 +00:00
-- Joey Hess <id@joeyh.name> Mon, 31 Oct 2016 18:55:59 -0400
Assistant, repair: Fix ignoring of git fsck errors due to duplicate file entries in tree objects.
2016-10-31 18:00:37 +00:00
prep release
2016-10-27 19:25:05 +00:00
git-annex (6.20161027) unstable; urgency=medium
lock: Fix edge cases where data loss could occur in v6 mode. In the case where the pointer file is in place, and not the content of the object, lock's performNew was called with filemodified=True, which caused it to try to repopulate the object from an unmodified associated file, of which there were none. So, the content of the object got thrown away incorrectly. This was the cause (although not the root cause) of data loss in https://github.com/datalad/datalad/issues/1020 The same problem could also occur when the work tree file is modified, but the object is not, and lock is called with --force. Added a test case for this, since it's excercising the same code path and is easier to set up than the problem above. Note that this only occurred when the keys database did not have an inode cache recorded for the annex object. Normally, the annex object would be in there, but there are of course circumstances where the inode cache is out of sync with reality, since it's only a cache. Fixed by checking if the object is unmodified; if so we don't need to try to repopulate it. This does add an additional checksum to the unlock path, but it's already checksumming the worktree file in another case, so it doesn't slow it down overall. Further investigation found a similar problem occurred when smudge --clean is called on a file and the inode cache is not populated. cleanOldKeys deleted the unmodified old object file in this case. This was also fixed by checking if the object is unmodified. In general, use of getInodeCaches and sameInodeCache is potentially dangerous if the inode cache has not gotten populated for some reason. Better to use isUnmodified. I breifly auited other places that check the inode cache, and did not see any immediate problems, but it would be easy to miss this kind of problem.
2016-10-17 16:56:26 +00:00
* lock, smudge: Fix edge cases where data loss could occur in v6 mode
when the keys database was not populated.
upgrade: Handle upgrade to v6 when the repository already contains v6 unlocked files whose content is already present. Closes https://github.com/datalad/datalad/issues/1020 The use of runWriter in scanUnlockedFiles broke due to this change; it failed with blocked indefinitely in mvar, because the database write handle was taken while linkFromAnnex needed to also write to it (to update the inode cache). So, switched to using a separate runWriter for each call to addAssociatedFileFast. A little less efficient, but not greatly; the writes should all still be cached.
2016-10-17 19:19:47 +00:00
* upgrade: Handle upgrade to v6 when the repository already contains
v6 unlocked files whose content is already present.
Improve style of offline html build of website.
2016-10-17 19:55:49 +00:00
* Improve style of offline html build of website.
importfeed: Drop URL parameters from file extension. Thanks, James MacMahon.
2016-10-17 20:02:05 +00:00
* importfeed: Drop URL parameters from file extension.
Thanks, James MacMahon.
Assistant, repair: Improved filtering out of git fsck lines about duplicate file entries in tree objects.
2016-10-18 15:19:41 +00:00
* Assistant, repair: Improved filtering out of git fsck lines about
duplicate file entries in tree objects.
test: Deal with gpg-agent behavior change that broke the test suite. gpg-agent started deleting its socket file on shutdown, and this tickled an ugly behavior in removeDirectoryRecursive, https://github.com/haskell/directory/issues/60 Running removeDirectoryRecursive again on exception avoids the problem.
2016-10-18 20:56:38 +00:00
* test: Deal with gpg-agent behavior change that broke the test suite.
Improve ssh socket cleanup code to skip over the cruft that NFS sometimes puts in a directory when a file is being deleted.
2016-10-26 17:16:41 +00:00
* Improve ssh socket cleanup code to skip over the cruft that
NFS sometimes puts in a directory when a file is being deleted.
enable forwardRetry for command-line transfers If a transfer fails for some reason, but some data managed to be sent, the transfer will be retried. (The assistant already did this.) Possible impacts: * More ssh prompts if ssh needs to prompt for a password to connect to a host, or is prompting about some other problem like a ssh key mismatch. * More data transfer due to retrying, epecially when a remote does not support resuming a transfer. In the worst case, a lot of data will be transferred but it fails before the end, and then all that data gets transferred again plus one byte more; repeat until it manages to get the whole file.
2016-10-26 19:38:22 +00:00
* If a transfer fails for some reason, but some data managed to be sent,
the transfer will be retried. (The assistant already did this.)
Run ssh with ServerAliveInterval 60 So that stalled transfers will be noticed within about 3 minutes, even if TCPKeepAlive is disabled or doesn't work. Rather than setting with -o, use -F with another config file, so that any settings in ~/.ssh/config or /etc/ssh/ssh_config overrides this.
2016-10-26 20:41:34 +00:00
* Run ssh with ServerAliveInterval 60, so that stalled transfers will
be noticed within about 3 minutes.
(Any setting in your ~/.ssh/config or /etc/ssh/ssh_config
overrides this.)
lock: Fix edge cases where data loss could occur in v6 mode. In the case where the pointer file is in place, and not the content of the object, lock's performNew was called with filemodified=True, which caused it to try to repopulate the object from an unmodified associated file, of which there were none. So, the content of the object got thrown away incorrectly. This was the cause (although not the root cause) of data loss in https://github.com/datalad/datalad/issues/1020 The same problem could also occur when the work tree file is modified, but the object is not, and lock is called with --force. Added a test case for this, since it's excercising the same code path and is easier to set up than the problem above. Note that this only occurred when the keys database did not have an inode cache recorded for the annex object. Normally, the annex object would be in there, but there are of course circumstances where the inode cache is out of sync with reality, since it's only a cache. Fixed by checking if the object is unmodified; if so we don't need to try to repopulate it. This does add an additional checksum to the unlock path, but it's already checksumming the worktree file in another case, so it doesn't slow it down overall. Further investigation found a similar problem occurred when smudge --clean is called on a file and the inode cache is not populated. cleanOldKeys deleted the unmodified old object file in this case. This was also fixed by checking if the object is unmodified. In general, use of getInodeCaches and sameInodeCache is potentially dangerous if the inode cache has not gotten populated for some reason. Better to use isUnmodified. I breifly auited other places that check the inode cache, and did not see any immediate problems, but it would be easy to miss this kind of problem.
2016-10-17 16:56:26 +00:00
prep release
2016-10-27 19:25:05 +00:00
-- Joey Hess <id@joeyh.name> Thu, 27 Oct 2016 15:21:58 -0400
lock: Fix edge cases where data loss could occur in v6 mode. In the case where the pointer file is in place, and not the content of the object, lock's performNew was called with filemodified=True, which caused it to try to repopulate the object from an unmodified associated file, of which there were none. So, the content of the object got thrown away incorrectly. This was the cause (although not the root cause) of data loss in https://github.com/datalad/datalad/issues/1020 The same problem could also occur when the work tree file is modified, but the object is not, and lock is called with --force. Added a test case for this, since it's excercising the same code path and is easier to set up than the problem above. Note that this only occurred when the keys database did not have an inode cache recorded for the annex object. Normally, the annex object would be in there, but there are of course circumstances where the inode cache is out of sync with reality, since it's only a cache. Fixed by checking if the object is unmodified; if so we don't need to try to repopulate it. This does add an additional checksum to the unlock path, but it's already checksumming the worktree file in another case, so it doesn't slow it down overall. Further investigation found a similar problem occurred when smudge --clean is called on a file and the inode cache is not populated. cleanOldKeys deleted the unmodified old object file in this case. This was also fixed by checking if the object is unmodified. In general, use of getInodeCaches and sameInodeCache is potentially dangerous if the inode cache has not gotten populated for some reason. Better to use isUnmodified. I breifly auited other places that check the inode cache, and did not see any immediate problems, but it would be easy to miss this kind of problem.
2016-10-17 16:56:26 +00:00
releasing package git-annex version 6.20161012
2016-10-12 13:38:03 +00:00
git-annex (6.20161012) unstable; urgency=medium
Optimisations to time it takes git-annex to walk working tree and find files to work on. Sped up by around 18%. key2file and file2key were top cost centers according to profiling. The repeated use of replace was not efficient. This new approach is quite a lot more efficient. This commit was sponsored by Denis Dzyubenko on Patreon.
2016-09-26 20:47:59 +00:00
* Optimisations to time it takes git-annex to walk working tree and find
files to work on. Sped up by around 18%.
Optimisations to git-annex branch query and setting, avoiding repeated copies of the environment. Speeds up commands like "git-annex find --in remote" by over 50%. Profiling showed that adjustGitEnv was 21% of the time and 37% of the allocations of that command. It copied the environment each time with getEnvironment. The only repeated use of adjustGitEnv is in withIndexFile, which tends to be run at least once per file. So, it was optimised by keeping a cache of the environment, which can be reused. There could be other better ways to optimise this. Maybe get the while environment once at startup. But, then it would have to be serialized back out each time running a child process, so I doubt that would be a net win. It might be better to cache a version of the environment that is pre-modified to use .git-annex/index. But, profiling doesn't show that modifying the enviroment is taking any significant time.
2016-09-29 17:36:48 +00:00
* Optimisations to git-annex branch query and setting, avoiding repeated
copies of the environment. Speeds up commands like
"git-annex find --in remote" by over 50%.
Optimised git-annex branch log file timestamp parsing. 10% speedup This sped up git annex find --not --in web from 6.64s to 5.69s. The optimised parser is probably more like 50% faster than the general one it replaced.
2016-09-29 18:04:53 +00:00
* Optimised git-annex branch log file timestamp parsing.
Add "total-size" field to --json-progress output.
2016-09-29 20:29:54 +00:00
* Add "total-size" field to --json-progress output.
Make --json-progress output be shown even when the size of a object is not known.
2016-09-29 20:59:48 +00:00
* Make --json-progress output be shown even when the size of a object
is not known.
allow multiple concurrent external special remote processes Multiple external special remote processes for the same remote will be started as needed when using -J. This should not beak any existing external special remotes, because running multiple git-annex commands at the same time could already start multiple processes for the same external special remotes.
2016-09-30 18:29:02 +00:00
* Multiple external special remote processes for the same remote will be
started as needed when using -J. This should not beak any existing
external special remotes, because running multiple git-annex commands
at the same time could already start multiple processes for the same
external special remotes.
Linux standalone: Include locale files in the bundle, and generate locale definition files for the locales in use when starting runshell. Currently only done for utf-8 locales because the charset can easily be told for those. Other locales don't include the charset in their name. The locale definition is generated under git-annex.linux/locales. So, this only works if the user can write there. If locale generation fails for any reason, it's silently skipped. The git-annex-standalone.deb installs the bundle under /usr, so this locale generation won't work for non-root users.
2016-10-04 20:37:43 +00:00
* Linux standalone: Include locale files in the bundle, and generate
locale definition files for the locales in use when starting runshell.
(Currently only done for utf-8 locales.)
Avoid using a lot of memory when large objects are present in the git repository .. and have to be checked to see if they are a pointed to an annexed file. Cases where such memory use could occur included, but were not limited to: - git commit -a of a large unlocked file (in v5 mode) - git-annex adjust when a large file was checked into git directly Generally, any use of catKey was a potential problem. Fix by using git cat-file --batch-check to check size before catting. This adds another git batch process, which is included in the CatFileHandle for simplicity. There could be performance impact, anywhere catKey is used. Particularly likely to affect adjusted branch generation speed, and operations on unlocked files in v6 mode. Hopefully since the --batch-check and --batch read the same data, disk buffering will avoid most overhead. Leaving only the overhead of talking to the process over the pipe and whatever computation --batch-check needs to do. This commit was sponsored by Bruno BEAUFILS on Patreon.
2016-10-05 19:21:36 +00:00
* Avoid using a lot of memory when large objects are present in the git
repository and have to be checked to see if they are a pointed to an
annexed file. Cases where such memory use could occur included, but
were not limited to:
- git commit -a of a large unlocked file (in v5 mode)
- git-annex adjust when a large file was checked into git directly
When auto-upgrading a v3 remote, avoid upgrading to version 6, instead keep it at version 5. Fixes a bug introduced with v6 mode that I didn't notice until now. Probably not many v3 repos left out there, and upgrading them to v6 mode is not disastrous, only a little premature. This commit was sponsored by Riku Voipio
2016-10-05 20:23:09 +00:00
* When auto-upgrading a v3 remote, avoid upgrading to version 6,
instead keep it at version 5.
Support using v3 repositories without upgrading them to v5. An easy change now that supportedVersions is a list. Since v3 and v5 are identical other than version number, just add v3 to the list. This commit was sponsored by andrea rota.
2016-10-05 20:29:32 +00:00
* Support using v3 repositories without upgrading them to v5.
sync: Fix bug in adjusted branch merging that could cause recently added files to be lost when updating the adjusted branch. The modification flag was not being set when making modifications deep in a tree, so parent trees were not updated to contain the modified tree. Seems to have exposed another bug where the wrong filename gets grafted in. This commit was sponsored by Brock Spratlen on Patreon.
2016-10-10 19:00:45 +00:00
* sync: Fix bug in adjusted branch merging that could cause recently
added files to be lost when updating the adjusted branch.
Optimisations to time it takes git-annex to walk working tree and find files to work on. Sped up by around 18%. key2file and file2key were top cost centers according to profiling. The repeated use of replace was not efficient. This new approach is quite a lot more efficient. This commit was sponsored by Denis Dzyubenko on Patreon.
2016-09-26 20:47:59 +00:00
releasing package git-annex version 6.20161012
2016-10-12 13:38:03 +00:00
-- Joey Hess <id@joeyh.name> Wed, 12 Oct 2016 09:37:41 -0400
Optimisations to time it takes git-annex to walk working tree and find files to work on. Sped up by around 18%. key2file and file2key were top cost centers according to profiling. The repeated use of replace was not efficient. This new approach is quite a lot more efficient. This commit was sponsored by Denis Dzyubenko on Patreon.
2016-09-26 20:47:59 +00:00
prep release
2016-09-23 13:45:46 +00:00
git-annex (6.20160923) unstable; urgency=medium
Rate limit console progress display updates to 10 per second. Was updating as frequently as changes were reported, up to hundreds of times per second, which used unncessary bandwidth when running git-annex over ssh etc.
2016-09-08 17:17:43 +00:00
* Rate limit console progress display updates to 10 per second.
Was updating as frequently as changes were reported, up to hundreds of
times per second, which used unncessary bandwidth when running git-annex
over ssh etc.
buffer json output until done when in concurrent mode
2016-09-09 17:21:38 +00:00
* Make --json and --quiet work when used with -J.
Previously, -J override the other options.
addurl, get: Added --json-progress option, which adds progress objects to the json output. This doesn't work right when used with -J yet, and there is some really ugly hand-crafting of part of the json output.
2016-09-09 19:06:54 +00:00
* addurl, get: Added --json-progress option, which adds progress
objects to the json output.
Remove key:null from git-annex add --json output.
2016-09-09 18:26:34 +00:00
* Remove key:null from git-annex add --json output.
copy, move, mirror: Support --json and --json-progress.
2016-09-09 20:24:26 +00:00
* copy, move, mirror: Support --json and --json-progress.
Improve gpg secret key list parser to deal with changes in gpg 2.1.15. Fixes key name display in webapp. gpg 2.1.15 (or so) seems to have added some new fields to the --with-colons --list-secret-keys output. These include "fpr" and "grp", and come before the "uid" line. So, the parser was giving up before it saw the name. Fix by continuing to look for the uid line until the next "sec" line. This commit was sponsored by Ole-Morten,Duesund on Patreon.
2016-09-14 17:30:50 +00:00
* Improve gpg secret key list parser to deal with changes in gpg 2.1.15.
Fixes key name display in webapp.
info: Support being passed a treeish, and show info about the annexed files in it similar to how a directory is handled.
2016-09-15 16:51:00 +00:00
* info: Support being passed a treeish, and show info about the annexed
files in it similar to how a directory is handled.
fix bugs in handing of deep branches with sync and adjusted branches * sync: Previously, when run in a branch with a slash in its name, such as "foo/bar", the sync branch was "synced/bar". That conflicted with the sync branch used for branch "bar", so has been changed to "synced/foo/bar". * adjust: Previously, when adjusting a branch with a slash in its name, such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)". That conflicted with the adjusted branch used for branch "bar", so has been changed to "adjusted/foo/bar(unlocked)" * Also, running sync in an adjusted branch did not correctly sync changes back to the parent branch when it had a slash in its name. This bug has been fixed. Eliminate use of Git.Ref.under and Git.Ref.basename; using Git.Ref.underBase and Git.Ref.base make everything handle deep branches correctly. Probably noone was adjusting deep branches, and v6 is still experimental anyway, so I'm not going to worry about the mess that was left by that bug. In the case of git-annex sync, using a fixed git-annex with an old unfixed one will mean they use different sync branches for a deep branch, and so they may stop syncing until the old one is upgraded. However, that's only a problem when syncing between repositories without going via a central bare repository. Added a warning about this to the CHANGELOG, but it's probably not going to affect many people at all. This commit was sponsored by Riku Voipio.
2016-09-21 19:18:51 +00:00
* sync: Previously, when run in a branch with a slash in its name,
such as "foo/bar", the sync branch was "synced/bar". That conflicted
with the sync branch used for branch "bar", so has been changed to
"synced/foo/bar".
* Note that if you're using an old version of git-annex to sync with
a branch with a slash in its name, it won't see some changes synced by
this version, and this version won't see some changes synced by the older
version. This is not a problem if there's a central bare repository,
but may impact other configurations until git-annex is upgraded to this
version.
* adjust: Previously, when adjusting a branch with a slash in its name,
such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)".
That conflicted with the adjusted branch used for branch "bar",
so has been changed to "adjusted/foo/bar(unlocked)"
* Also, running sync in an adjusted branch did not correctly sync
changes back to the parent branch when it had a slash in its name.
This bug has been fixed.
addurl, importfeed: Improve behavior when file being added is gitignored.
2016-09-21 21:21:48 +00:00
* addurl, importfeed: Improve behavior when file being added is gitignored.
Rate limit console progress display updates to 10 per second. Was updating as frequently as changes were reported, up to hundreds of times per second, which used unncessary bandwidth when running git-annex over ssh etc.
2016-09-08 17:17:43 +00:00
prep release
2016-09-23 13:45:46 +00:00
-- Joey Hess <id@joeyh.name> Fri, 23 Sep 2016 09:43:26 -0400
Rate limit console progress display updates to 10 per second. Was updating as frequently as changes were reported, up to hundreds of times per second, which used unncessary bandwidth when running git-annex over ssh etc.
2016-09-08 17:17:43 +00:00
prep release
2016-09-07 15:12:33 +00:00
git-annex (6.20160907) unstable; urgency=medium
Windows: Handle shebang in external special remote program.
2016-09-05 16:09:23 +00:00
* Windows: Handle shebang in external special remote program.
Fix formatting of git-annex-smudge man page, and improve mdwn2man. Thanks, Jim Paris.
2016-09-05 16:16:03 +00:00
* Fix formatting of git-annex-smudge man page, and improve mdwn2man.
Thanks, Jim Paris.
examimekey: Allow being run in a git repo that is not initialized by git-annex yet. No reason not to; indeed there's no real reason to need a git repository at all except the implementation uses the Annex monad.
2016-09-05 16:26:59 +00:00
* examimekey: Allow being run in a git repo that is not initialized by
git-annex yet.
Android: Fix disabling use of cp --reflink=auto, curl, sha224, and sha384. This was originally done in a7ef05a9, but got lost in some change to the Makefile. Use CROSS_COMPILE=Android to tell configure that it's configuring for android instead of passing it a parameter.
2016-09-05 18:11:35 +00:00
* Android: Fix disabling use of cp --reflink=auto, curl, sha224, and sha384.
Make --json and --quiet suppress automatic init messages And any other messages that might be output before a command starts. Fixes a reversion introduced in version 5.20150727. During the optparse-applicative conversion, I needed a place to run per-command global option setters, and I made it get run during the seek stage. But that is too late to have --json and --quiet disable output produced in the check stage. Fix is just to run those per-command global option setters at the same time as the all-command global option setters. This commit was sponsored by Thom May.
2016-09-05 19:32:59 +00:00
* Make --json and --quiet suppress automatic init messages, and any
other messages that might be output before a command starts.
Fixes a reversion introduced in version 5.20150727.
Assistant, repair: Filter out git fsck lines about duplicate file entries in tree objects.
2016-09-05 20:08:49 +00:00
* Assistant, repair: Filter out git fsck lines about duplicate file
entries in tree objects.
get -J: Download different files from different remotes when the remotes have the same costs. Only done in -J mode because only if there's concurrency can downloading from two remotes be faster. Without concurrency, it's likely the case that sequential downloads from the same remote are faster than switching back and forth between two remotes. There is some hairy MVar code here, but basically it just keeps the activeremotes MVar full except when deciding which remote to assign to a thread. Also affects gets by sync --content -J This commit was sponsored by Jochen Bartl.
2016-09-06 16:42:50 +00:00
* get -J, sync --content -J: Download different files from different
remotes when the remotes have the same costs.
Windows: Handle shebang in external special remote program.
2016-09-05 16:09:23 +00:00
prep release
2016-09-07 15:12:33 +00:00
-- Joey Hess <id@joeyh.name> Wed, 07 Sep 2016 11:12:11 -0400
Windows: Handle shebang in external special remote program.
2016-09-05 16:09:23 +00:00
releasing package git-annex version 6.20160808
2016-08-08 15:57:09 +00:00
git-annex (6.20160808) unstable; urgency=medium
--branch, stage 1 Added --branch option to copy, drop, fsck, get, metadata, mirror, move, and whereis commands. This option makes git-annex operate on files that are included in a specified branch (or other treeish). The names of the files from the branch that are being operated on are not displayed yet; only the keys. Displaying the filenames will need changes to every affected command. Also, note that --branch can be specified repeatedly. This is not really documented, but seemed worth supporting, especially since we may later want the ability to operate on all branches matching a refspec. However, when operating on two branches that contain the same key, that key will be operated on twice.
2016-07-20 16:05:22 +00:00
saner format for metadata --json metadata --json output format has changed, adding a inner json object named "fields" which contains only the fields and their values. This should be easier to parse than the old format, which mixed up metadata fields with other keys in the json object. Any consumers of the old format will need to be updated. This adds a dependency on unordered-containers for parsing MetaData from JSON, but it's a free dependency; aeson pulls in that library.
2016-07-26 18:53:00 +00:00
* metadata --json output format has changed, adding a inner json object
named "fields" which contains only the fields and their values.
This should be easier to parse than the old format, which mixed up
metadata fields with other keys in the json object.
Any consumers of the old format will need to be updated.
Added metadata --batch option, which allows getting, setting, deleting, and modifying metadata for multiple files/keys.
2016-07-27 14:46:25 +00:00
* Added metadata --batch option, which allows getting, setting, deleting,
and modifying metadata for multiple files/keys.
--branch, stage 1 Added --branch option to copy, drop, fsck, get, metadata, mirror, move, and whereis commands. This option makes git-annex operate on files that are included in a specified branch (or other treeish). The names of the files from the branch that are being operated on are not displayed yet; only the keys. Displaying the filenames will need changes to every affected command. Also, note that --branch can be specified repeatedly. This is not really documented, but seemed worth supporting, especially since we may later want the ability to operate on all branches matching a refspec. However, when operating on two branches that contain the same key, that key will be operated on twice.
2016-07-20 16:05:22 +00:00
* Added --branch option to copy, drop, fsck, get, metadata, mirror, move,
and whereis commands. This option makes git-annex operate on files that
are included in a specified branch (or other treeish).
cabal constraints for aws and esqueleto closes https://github.com/joeyh/git-annex/pull/55 * git-annex.cabal: Temporarily limit to http-conduit <2.2.0 since aws 0.14.0 is not compatible with the newer version. * git-annex.cabal: Temporarily limit to persistent <2.5 since esqueleto 2.4.3 is not compatible with the newer version.
2016-07-22 16:41:28 +00:00
* git-annex.cabal: Temporarily limit to http-conduit <2.2.0
since aws 0.14.0 is not compatible with the newer version.
* git-annex.cabal: Temporarily limit to persistent <2.5
since esqueleto 2.4.3 is not compatible with the newer version.
Removed dependency on json library; all JSON is now handled by aeson. I've eyeballed all --json commands, and the only difference should be that some fields are re-ordered.
2016-07-26 23:15:34 +00:00
* Removed dependency on json library; all JSON is now handled by aeson.
When built with ut uid-1.3.12, generate more random UUIDs than before Use nextRandom to generate the random UUID, rather than using randomIO. This gets fixes for the following two bugs in the uuid library. However, this did not impact git-annex much, so a hard depedency has not been added on uuid-1.3.12. https://github.com/aslatter/uuid/issues/15 "v4 UUIDs are not that random" This doesn't greatly affect git-annex, because even with only 2^64 possible UUIDs, the chance that two git-annex repositories that are clones of the same git repo get the same UUID is miniscule. And, git-annex generates only one UUID per run, so preducting subsequent UUIDs is not a problem. https://github.com/aslatter/uuid/issues/16 "Remove Random instance for UUID, or mark it as deprecated" git-annex was using that instance; let's stop before it gets deprecated or removed.
2016-07-27 11:38:11 +00:00
* When built with uuid-1.3.12, generate more random UUIDs than before.
(However, this did not impact git-annex much, so a hard depedency has
not been added on uuid-1.3.12.)
info: When run on a file now includes an indication of whether the content is present locally.
2016-07-30 16:29:59 +00:00
* info: When run on a file now includes an indication of whether
the content is present locally.
get, move, copy, mirror: Added --failed switch which retries failed copies/moves Note that get --from foo --failed will get things that a previous get --from bar tried and failed to get, etc. I considered making --failed only retry transfers from the same remote, but it was easier, and seems more useful, to not have the same remote requirement. Noisy due to some refactoring into Types/
2016-08-03 16:37:12 +00:00
* get, move, copy, mirror: Added --failed switch which retries
failed copies/moves.
Re-enable accumulating transfer failure log files for command-line actions This was disabled in commit 61ccf95004cc1ffaadba9a495f3d4d5e0c106043, because only the assistant used them, and they were clutter. But, now --failed also uses them. Remove the failure log files after successful transfers. Should avoid most of the clutter problems. Commit 61ccf95004cc1ffaadba9a495f3d4d5e0c106043 mentions a subtle behavior change, which has now been reverted: There is one behavior change from this. If glacier is being used, and a manual git annex get --from glacier fails because the file isn't available yet, the assistant will no longer later see that failed transfer file and retry the get.
2016-08-03 17:41:07 +00:00
* Re-enable accumulating transfer failure log files for command-line
actions (disabled in 5.20150522), and remove the log files after
successful transfers.
--branch, stage 1 Added --branch option to copy, drop, fsck, get, metadata, mirror, move, and whereis commands. This option makes git-annex operate on files that are included in a specified branch (or other treeish). The names of the files from the branch that are being operated on are not displayed yet; only the keys. Displaying the filenames will need changes to every affected command. Also, note that --branch can be specified repeatedly. This is not really documented, but seemed worth supporting, especially since we may later want the ability to operate on all branches matching a refspec. However, when operating on two branches that contain the same key, that key will be operated on twice.
2016-07-20 16:05:22 +00:00
releasing package git-annex version 6.20160808
2016-08-08 15:57:09 +00:00
-- Joey Hess <id@joeyh.name> Mon, 08 Aug 2016 11:42:17 -0400
--branch, stage 1 Added --branch option to copy, drop, fsck, get, metadata, mirror, move, and whereis commands. This option makes git-annex operate on files that are included in a specified branch (or other treeish). The names of the files from the branch that are being operated on are not displayed yet; only the keys. Displaying the filenames will need changes to every affected command. Also, note that --branch can be specified repeatedly. This is not really documented, but seemed worth supporting, especially since we may later want the ability to operate on all branches matching a refspec. However, when operating on two branches that contain the same key, that key will be operated on twice.
2016-07-20 16:05:22 +00:00
prep release
2016-07-19 18:18:16 +00:00
git-annex (6.20160619) unstable; urgency=medium
merged patch closes https://github.com/joeyh/git-annex/pull/54
2016-06-14 01:53:43 +00:00
drop: Add --batch and --json options.
2016-07-06 15:54:46 +00:00
* get, drop: Add --batch and --json options.
testremote: Fix crash when testing a freshly made external special remote. Ignore exceptions when getting the cost and availability for the remote, and return sane defaults. These defaults are not cached, so if a special remote program has a transient problem, it will re-query it later.
2016-07-05 20:34:39 +00:00
* testremote: Fix crash when testing a freshly made external special remote.
Remove unnecessary rpaths in the git-annex binary, but only when it's built using make, not cabal. This speeds up git-annex statup time by around 50%.
2016-07-06 18:40:18 +00:00
* Remove unnecessary rpaths in the git-annex binary, but only when
it's built using make, not cabal.
Speed up startup time by caching the refs that have been merged into the git-annex branch. This can speed up git-annex commands by as much as a second, depending on the number of remotes.
2016-07-17 16:11:05 +00:00
This speeds up git-annex startup time by around 50%.
* Speed up startup time by caching the refs that have been merged into
the git-annex branch.
This can speed up git-annex commands by as much as a second,
depending on the number of remotes.
fsck: Fix a reversion in direct mode fsck of a file that is present when the location log thinks it is not. Reversion introduced in version 5.20151208.
2016-07-12 17:41:03 +00:00
* fsck: Fix a reversion in direct mode fsck of a file that is
present when the location log thinks it is not. Reversion introduced
in version 5.20151208.
uninit: Fix crash due to trying to write to deleted keys db. Reversion introduced by v6 mode support, affects v5 too. Also fix a similar crash when the webapp is used to delete a repository.
2016-07-12 18:18:35 +00:00
* uninit: Fix crash due to trying to write to deleted keys db.
Reversion introduced by v6 mode support, affects v5 too.
* Fix a similar crash when the webapp is used to delete a repository.
Support checking presence of content at a http url that redirects to a ftp url.
2016-07-12 20:30:36 +00:00
* Support checking presence of content at a http url that redirects to
a ftp url.
log: Added --all option.
2016-07-17 19:15:08 +00:00
* log: Added --all option.
prep release
2016-07-19 18:18:16 +00:00
* New url for git-remote-gcrypt, now maintained by spwhitton.
* webapp: Don't allow deleting a remote that has syncing disabled,
as such a deletion will never finish.
Thanks, Farhan Kathawala.
webapp: Escape unusual characters in ssh hostnames when generating mangled hostnames. This allows IPv6 addresses to be used on filesystems not supporting : in filenames.
2016-07-19 15:37:03 +00:00
* webapp: Escape unusual characters in ssh hostnames when generating
mangled hostnames. This allows IPv6 addresses to be used on filesystems
not supporting : in filenames.
Avoid any access to keys database in v5 mode repositories, which are not supposed to use that database.
2016-07-19 16:12:19 +00:00
* Avoid any access to keys database in v5 mode repositories, which
are not supposed to use that database.
prep release
2016-07-19 18:18:16 +00:00
* Remove the EKG build flag, since Gentoo for some reason decided to
enable this flag, depsite it not being intended for production use and
so disabled by default.
merged patch closes https://github.com/joeyh/git-annex/pull/54
2016-06-14 01:53:43 +00:00
prep release
2016-07-19 18:18:16 +00:00
-- Joey Hess <id@joeyh.name> Tue, 19 Jul 2016 14:17:54 -0400
merged patch closes https://github.com/joeyh/git-annex/pull/54
2016-06-14 01:53:43 +00:00
prep release
2016-06-13 18:57:52 +00:00
git-annex (6.20160613) unstable; urgency=medium
Improve SHA*E extension extraction code. Filter out over-long "extensions" before stripping out non-alphanumerics from them, so that eg "foo.ba__________r" is not considered a .bar extension.
2016-05-27 17:14:51 +00:00
* Improve SHA*E extension extraction code.
Windows: Avoid terminating git-annex branch lines with \r\n when union merging.
2016-05-27 19:22:29 +00:00
* Windows: Avoid terminating git-annex branch lines with \r\n when
union merging and performing transitions.
Remove Makefile from cabal tarball; man page building is now handled by a small haskell program. This actually runs faster than building the man pages from the makefile did. But the main purpose is to let Setup.hs import Build.Mans and so not need the makefile.
2016-05-31 17:58:13 +00:00
* Remove Makefile from cabal tarball; man page building is now handled by
a small haskell program.
sync --content: Fix bug that caused transfers of files to be made to a git remote that does not have a UUID. This particularly impacted clones from gcrypt repositories. Added guard in Annex.Transfer to prevent this problem at a deeper level. I'm unhappy ith NoUUID, but having Maybe UUID instead wouldn't help either if nothing checked that there was a UUID. Since there legitimately need to be Remotes that do not have a UUID, I can't see a way to fix it at the type level, short making there be two separate types of Remotes.
2016-06-02 17:50:05 +00:00
* sync --content: Fix bug that caused transfers of files to be made
to a git remote that does not have a UUID. This particularly impacted
clones from gcrypt repositories.
Pass -S to git commit-tree when commit.gpgsign is set and when making a non-automatic commit, in order to preserve current behavior when used with git 1.9, which has stopped doing this itself.
2016-06-02 19:07:20 +00:00
* Pass -S to git commit-tree when commit.gpgsign is set and when
making a non-automatic commit, in order to preserve current behavior
fix git version
2016-06-02 19:55:11 +00:00
when used with git 2.9, which has stopped doing this itself.
remotedaemon: Fixed support for notifications of changes to gcrypt remotes, which was never tested and didn't quite work before.
2016-06-02 20:19:31 +00:00
* remotedaemon: Fixed support for notifications of changes to gcrypt
remotes, which was never tested and didn't quite work before.
list: Do not include dead repositories.
2016-06-04 18:33:31 +00:00
* list: Do not include dead repositories.
move --to: Better behavior when system is completely out of disk space; drop content from disk before writing location log. I noticed move --to failing when there was no disk space. The file was sent to the remote, but it crashed before it could be dropped locally. This could fix that.
2016-06-05 17:51:22 +00:00
* move --to: Better behavior when system is completely out of disk space;
drop content from disk before writing location log.
prep release
2016-06-13 18:57:52 +00:00
* Avoid a crash if getpwuid does not work, when querying the user's full
name.
* Automatically enable v6 mode when initializing in a clone from a repo
that has an adjusted branch checked out.
* v6: Fix initialization of a bare clone of a repo that has an adjusted
branch checked out.
highlight v6 changes
2016-06-09 19:17:54 +00:00
* v6: Fix bad automatic merge conflict resolution between an annexed file
Fix bad automatic merge conflict resolution between an annexed file and a directory with the same name when in an adjusted branch. When running in an overlay work tree, all unchanged files show as deleted, so this code that stages deletions should not run.
2016-06-07 16:53:35 +00:00
and a directory with the same name when in an adjusted branch.
v6: Fix bad merge in an adjusted branch that resulted in an empty tree.
2016-06-13 18:18:22 +00:00
* v6: Fix bad merge in an adjusted branch that resulted in an empty tree.
highlight v6 changes
2016-06-09 19:17:54 +00:00
* v6: Fix bug in initialization of clone from a repo with an adjusted branch
Fix bug in initialization of clone from a repo with an adjusted branch that had not been synced back to master. This bug caused broken tree objects to get built by a later git annex sync. This is a somewhat unlikely but not impossible situation, and the test suite's union_merge_regression test tickled it when it was run on FAT.
2016-06-09 18:11:00 +00:00
that had not been synced back to master.
(This bug caused broken tree objects to get built by a later git annex
sync.)
highlight v6 changes
2016-06-09 19:17:54 +00:00
* v6: Make lock and unlock work on files whose content is not present.
* v6: Fix update of associated files db when unlocking a file.
* v6: Make git clean filter preserve the backend that was used for a file.
Improve SHA*E extension extraction code. Filter out over-long "extensions" before stripping out non-alphanumerics from them, so that eg "foo.ba__________r" is not considered a .bar extension.
2016-05-27 17:14:51 +00:00
prep release
2016-06-13 18:57:52 +00:00
-- Joey Hess <id@joeyh.name> Mon, 13 Jun 2016 14:57:38 -0400
Improve SHA*E extension extraction code. Filter out over-long "extensions" before stripping out non-alphanumerics from them, so that eg "foo.ba__________r" is not considered a .bar extension.
2016-05-27 17:14:51 +00:00
prep release
2016-05-27 15:50:27 +00:00
git-annex (6.20160527) unstable; urgency=medium
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
Split lines in the git-annex branch on \r as well as \n, to deal with \r\n terminated lines written by some versions of git-annex on Windows. This fixes strange displays in some cases, including whereis showing many duplicate locations, and showing more total copies than actually exist. It's unknown if that lead to data loss when eg, dropping. At the moment, it seems unlikely it could, since the UUID with \r's appended is not the same as a UUID without, and so no remote matches it. It's also unknown if \r's can leak in on windows, perhaps when merging the git-annex branch.
2016-05-27 15:45:13 +00:00
* Split lines in the git-annex branch on \r as well as \n, to deal
with \r\n terminated lines written by some versions of git-annex on
Windows. This fixes strange displays in some cases.
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
* assistant: Fix bug that caused v6 pointer files to be annexed by the
assistant.
* assistant: Fix race in v6 mode that caused downloaded file content to
sometimes not replace pointer files.
* add: Adding a v6 pointer file used to annex it; now the pointer file is
added to git as-is. (git add of a pointer file already did the right
thing)
enableremote: Can now be used to explicitly enable git-annex to use git remotes. Using the command this way prevents other git-annex commands from probing new git remotes to auto-enable them.
2016-05-24 19:24:38 +00:00
* enableremote: Can now be used to explicitly enable git-annex to use
git remotes. Using the command this way prevents other git-annex
commands from probing new git remotes to auto-enable them.
enableremote: Remove annex-ignore configuration from a remote.
2016-05-24 19:48:22 +00:00
* enableremote: Remove annex-ignore configuration from a remote.
prep release
2016-05-27 15:50:27 +00:00
* Change git annex info remote encryption description to use wording
closer to what's used in initremote.
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
* Pass the various gnupg-options configs to gpg in several cases where
they were not before. Most notably, gnupg-decrypt-options is now
passed when decrypting an encrypted cipher.
prep release
2016-05-27 15:50:27 +00:00
* adjust: Add --fix adjustment, which is useful when the git directory
is in a nonstandard place.
* adjust: If the adjusted branch already exists, avoid overwriting it,
since it might contain changes that have not yet been propigated to the
original branch.
* Work around git weirdness in handling of relative path to GIT_INDEX_FILE
when in a subdirectory of the repository. This affected git annex view.
* Fix crash when entering/changing view in a subdirectory of a repo that
has a dotfile in its root.
* webapp: Avoid confusing display of dead remotes.
* Support building with ghc 8.0.1.
releasing package git-annex version 6.20160527
2016-05-27 16:35:18 +00:00
* Updated cabal file explicitly lists source files. The tarball
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
on hackage will include only the files needed for cabal install;
it is NOT the full git-annex source tree.
* debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks
to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead.
prep release
2016-05-27 15:50:27 +00:00
-- Joey Hess <id@joeyh.name> Fri, 27 May 2016 11:48:36 -0400
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
git-annex (6.20160511) unstable; urgency=medium
* Fix bug that sometimes prevented git-annex smudge --clean from consuming
all its input, which resulted in git add bypassing git-annex.
* Fix build with directory-1.2.6.2.
* Improve behavior when a just added http remote is not available
during uuid probe. Do not mark it as annex-ignore, so it will be tried
again later.
* Android: Icon refresh.
Thanks, freewheelinfranks.
* Added DIRHASH-LOWER to external special remote protocol.
* git-annex.cabal: Add Setup-Depends.
* stack.yaml: Enable explicit-setup-deps.
* Windows: Fix several bugs in propigation of changes from the adjusted
branch back to the master branch.
* Windows: Fix an over-long temp directory name.
* map: Hide dead repositories that are not connected to the graph.
* map: Changed colors; red is used for untrusted repositories and grey
for dead.
* version: Display OS version and architecture too.
* Propigate GIT_DIR and GIT_WORK_TREE environment to external special
remotes.
* Added annex.gnupg-decrypt-options and
remote.<name>.annex-gnupg-decrypt-options, which are passed to gpg
when it's decrypting data.
* fsck: When a key is not previously known in the location log,
record something so that reinject --known will work.
* In the unusual configuration where annex.crippledfilesystem=true but
core.symlinks=true, store object contents in mixed case hash
directories so that symlinks will point to them.
* Added new encryption=sharedpubkey mode for special remotes.
This is useful for makking a special remote that anyone with a clone
of the repo and your public keys can upload files to, but only you can
decrypt the files stored in it.
-- Joey Hess <id@joeyh.name> Wed, 11 May 2016 12:41:42 -0400
git-annex (6.20160419) unstable; urgency=medium
* Fix bug that prevented resuming of uploads to encrypted special remotes
that used chunking.
* That bug could also expose the names of keys to such remotes, so it is a
minor security issue.
* Fix duplicate progress meter display when downloading from a git remote
over http with -J.
* reinject: When src file's content cannot be verified, leave it alone,
instead of deleting it.
* reinject: Added new mode which can reinject known files into the annex.
For example: git-annex reinject --known /mnt/backup/*
* calckey: New plumbing command, calculates the key that would be used
to refer to a file.
* Fix bug that prevented annex.sshcaching=false configuration from taking
effect when on a crippled filesystem. Thanks, divergentdave.
* git 2.9.0 is going to prevent git merge from merging in unrelated
branches. Since the webapp's pairing etc features often combine
together repositories with unrelated histories, work around
this behavior change when the assistant merges, by passing
--allow-unrelated-histories. Note though that this is not done
for git annex sync's merges, so it will follow git's default or
configured behavior.
* When git-annex is used with a git version older than 2.2.0, disable
support for adjusted branches, since GIT_COMMON_DIR is needed to update
them and was first added in that version of git.
* Avoid setting LOCPATH in linux standalone builds that are built with
a ghc that has been fixed to not hang when it cannot find locale files.
* Isolate test suite from global git config settings.
-- Joey Hess <id@joeyh.name> Thu, 28 Apr 2016 09:31:14 -0400
git-annex (6.20160418) unstable; urgency=medium
* smudge: Print a warning when annex.thin is set, as git's smudge
interface does not allow honoring that configuration.
* webapp: When $HOME is a git repository, and has been initialized for
use by git-annex, opening the webapp went ahead and ran the assistant
there, annexing all files. Since this is almost certianly not
desirable, especially when the user is just opening the webapp from
a dekstop menu which happens to run it in $HOME, the webapp will now not
treat such a $HOME git repository as a git-annex repository.
* webapp: Update url to add gitlab.com ssh key.
* Fix bug in v6 mode that prevented treating unlocked executable files
as annexed. If you have such files, run git annex init --version=6
to update the cache after upgrading to this version of git-annex.
* Preserve execute bits of unlocked files in v6 mode.
* fsck: Warn when core.sharedRepository is set and an annex object file's
write bit is not set and cannot be set due to the file being owned
by a different user.
* Fix hang when dropping content needs to lock the content on a
ssh remote, which occurred when the remote has git-annex version
5.20151019 or newer. (The bug was in the client side; the remote
git-annex-shell does not need to be upgraded.)
-- Joey Hess <id@joeyh.name> Mon, 18 Apr 2016 18:33:52 -0400
git-annex (6.20160412) unstable; urgency=medium
* adjust --unlock: Enters an adjusted branch in which all annexed files
are unlocked. The v6 equivilant of direct mode, but much cleaner!
* Upgrading a direct mode repository to v6 has changed to enter
an adjusted unlocked branch. This makes the direct mode to v6 upgrade
able to be performed in one clone of a repository without affecting
other clones, which can continue using v5 and direct mode.
* init --version=6: Automatically enter the adjusted unlocked branch
when filesystem doesn't support symlinks.
* ddar remote: fix ssh calls
Thanks, Robie Basak
* log: Display time with time zone.
* log --raw-date: Use to display seconds from unix epoch.
* v6: Close pointer file handles more quickly, to avoid problems on Windows.
* sync: Show output of git commit.
* annex.thin and annex.hardlink are now supported on Windows.
* unannex --fast now makes hard links on Windows.
* Fix bug in annex.largefiles mimetype= matching when git-annex
is run in a subdirectory of the repository.
* Fix build with ghc v7.11. Thanks, Gabor Greif.
-- Joey Hess <id@joeyh.name> Tue, 12 Apr 2016 14:53:22 -0400
git-annex (6.20160318) unstable; urgency=medium
* metadata: Added -r to remove all current values of a field.
* Fix data loss that can occur when annex.pidlock is set in a repository.
* Fix bug preventing moving files to/from a repository with annex.pidlock set.
* Fix shared lock file FD leak.
* Fix metadata hook behavior when multiple files are added at once.
Thanks, Klaus Ethgen.
* Added dependencies on haskell mountpoints and disk-free-space
libraries, removing FFI code from git-annex.
* dropkey: Add --batch and --json.
* Fix OSX dmg to include libraries needed by bundled gpg,
lost in last release.
* Always try to thaw content, even when annex.crippledfilesystem is set.
* Correct git-annex info to include unlocked files in v6 repository.
* Sped up git-annex add in direct mode and v6 by using
git hash-object --stdin-paths.
* Sped up git-annex merge by using git hash-object --stdin-paths.
-- Joey Hess <id@joeyh.name> Fri, 18 Mar 2016 11:30:36 -0400
git-annex (6.20160229) unstable; urgency=medium
* Update perlmagick build dependency. Closes: #789225
* Fix memory leak in last release, which affected commands like
git-annex status when a large non-annexed file is present in the work
tree.
* fsck: When the only copy of a file is in a dead repository, mention
the repository.
* info: Mention when run in a dead repository.
* Linux and OSX standalone builds put the bundled gpg last in PATH,
so any system gpg will be preferred over it.
* Avoid crashing when built with MagicMime support, but when the magic
database cannot be loaded.
* Include magic database in the linux and OSX standalone builds.
* Fix memory leak when hashing files, which triggered during fsck
when an external hash program was not used.
(This leak was introduced in version 6.20160114.)
* Support --metadata field<number, --metadata field>number etc
to match ranges of numeric values.
* Similarly, support preferred content expressions like
metadata=field<number and metadata=field>number
* The pre-commit-annex hook script that automatically extracts
metadata has been updated to also use exiftool.
Thanks, Klaus Ethgen.
-- Joey Hess <id@joeyh.name> Mon, 29 Feb 2016 12:41:49 -0400
git-annex (6.20160217) unstable; urgency=medium
* Support getting files from read-only repositories.
* checkpresentkey: Allow to be run without an explicit remote.
* checkpresentkey: Added --batch.
* Work around problem with concurrent-output when in a non-unicode locale
by avoiding use of it in such a locale. Instead -J will behave as if
it was built without concurrent-output support in this situation.
* Fix storing of filenames of v6 unlocked files when the filename is not
representable in the current locale.
* fsck: Detect and fix missing associated file mappings in v6 repositories.
* fsck: Populate unlocked files in v6 repositories whose content is
present in annex/objects but didn't reach the work tree.
* When initializing a v6 repo on a crippled filesystem, don't force it
into direct mode.
* Windows: Fix v6 unlocked files to actually work.
* add, addurl, import, importfeed: When in a v6 repository on a crippled
filesystem, add files unlocked.
* annex.addunlocked: New configuration setting, makes files always be
added unlocked. (v6 only)
* Improve format of v6 unlocked pointer files to support keys containing
slashes.
-- Joey Hess <id@joeyh.name> Wed, 17 Feb 2016 14:48:51 -0400
git-annex (6.20160211) unstable; urgency=medium
* annex.addsmallfiles: New option controlling what is done when
adding files not matching annex.largefiles.
* Fix reversion in lookupkey, contentlocation, and examinekey which
caused them to sometimes output side messages.
* webapp: Fix deletion of current repository directory.
* Added "nothing" to preferred content expression syntax.
* annex.largefiles can be configured in .gitattributes too;
this is particulary useful for v6 repositories, since the
.gitattributes configuration will apply in all clones of the
repository.
* Limit annex.largefiles parsing to the subset of preferred content
expressions that make sense in its context. So, not "standard"
or "lackingcopies", etc.
* annex.largefiles: Add support for mimetype=text/* etc, when git-annex
is linked with libmagic.
* matchexpression: Added --largefiles option to parse an annex.largefiles
expression.
* Brought back the dbus and xmpp build flags, so build from source can be
done without C libraries that may be hard to install.
* init: Fix bugs in submodule .git symlink fixup, that occurred when
initializing in a subdirectory of a submodule and a submodule of a
submodule.
* WebDAV: Set depth 1 in PROPFIND request, for better compatibility with
some servers. Thanks, wzhd.
* WebDAV: Remove a bogus trailing slash from the end of the url to the
temporary store location for a key. Thanks, wzhd.
* S3: Allow configuring with requeststyle=path to use path-style bucket
access instead of the default DNS-style access.
-- Joey Hess <id@joeyh.name> Thu, 11 Feb 2016 11:42:19 -0400
git-annex (6.20160126) unstable; urgency=medium
* Fix nasty reversion in the last release that broke sync --content's
handling of many preferred content expressions.
* whereis --json: Urls are now listed inside the remote that claims them,
rather than all together at the end.
* info, add, whereis, find: Support --batch mode.
* Force output to be line-buffered, even when it's not connected to the
terminal. This is particuarly important for commands with --batch
output, which was not always being flushed at an appropriate time.
* add, import: Support --json output.
* addurl --json: Include field for added key (unless the file was
added directly to git due to annex.largefiles configuration.)
(Also done by add --json and import --json)
* registerurl: Check if a remote claims the url, same as addurl does.
* Bug fix: Git config settings passed to git-annex -c did not always take
effect.
* assistant: Use udisks2 dbus events to detect when disks are mounted,
instead of relying on gnome/kde stuff that is not stable.
* Fix build with QuickCheck 2.8.2
* matchexpression: New plumbing command to check if a preferred content
expression matches some data.
* Removed the webapp-secure build flag, rolling it into the webapp build
flag.
* Removed the quvi, tahoe, feed, and tfds build flags, adding
aeson feed and regex-tdfa to the core dependencies.
* Roll the dns build flag into the assistant build flag.
* Debian: Avoid building debug package, since gdb is not often useful
to debug haskell programs.
-- Joey Hess <id@joeyh.name> Tue, 26 Jan 2016 14:57:42 -0400
git-annex (6.20160114) unstable; urgency=medium
"hexapodia as the key insight"
* Added v6 repository mode, but v5 is still the default for now.
* unlock, lock: In v6 mode, unlocking a file changes it from a symlink to a
pointer file, and this change can be committed to the git repository.
For details, see http://git-annex.branchable.com/tips/unlocked_files/
* The upgrade to version 6 is not done fully automatically yet, because
upgrading a direct mode repository to version 6 will prevent old
versions of git-annex from working in other clones of that repository.
For details, see http://git-annex.branchable.com/upgrades/
* init: --version parameter added to control which supported repository
version to use.
* init, upgrade: Configure .git/info/attributes to use git-annex
as a smudge filter. In v6 repository mode, this makes git add
add files to the annex in unlocked mode, unless overridden by
annex.largefiles configuration.
* assistant: In v6 mode, adds files in unlocked mode, so they can
continue to be modified.
* Added annex.thin setting, which makes unlocked files in v6 repositories
be hard linked to their content, instead of a copy. This saves disk
space but means any modification of an unlocked file will lose the local
(and possibly only) copy of the old version.
* Enable annex.thin by default on upgrade from direct mode to v6, since
direct mode made the same tradeoff.
* fix: Adjusts unlocked files as configured by annex.thin.
* persistent-sqlite is now a hard build dependency, since v6 repository
mode needs it.
* status: On crippled filesystems, was displaying M for all annexed files
that were present. Probably caused by a change to what git status
displays in this situation. Fixed by treating files git thinks are
modified the same as typechanged files.
* addurl: Added --batch and --with-files options.
* addurl: Support --json, particularly useful in --batch mode.
* addurl: Refuse to overwrite any existing, non-annexed file.
* Debian: Adjust build dependencies for webapp, DAV. Now available on
mips, mipsel, but temporarily removed armel since build is failing
there.
* info: Fix "backend usage" numbers, which were counting present keys
twice.
* info --json: Improve json for "backend usage", using a nested object
with fields for each backend instead of the previous weird nested lists.
This may break existing parsers of this json output, if there were any.
* whereis --json: Make url list be included in machine-parseable form.
* test: Added --keep-failures option.
* unused: Bug fix when a new file was added to the annex, and then
removed (but not git rmed). git still has the add staged in this case,
so the content should not be unused and was wrongly treated as such.
* migrate: Copy over metadata to new key.
* rekey: No longer copies over urls from the old to the new key.
It makes sense for migrate to do that, but not for this low-level
(and little used) plumbing command to.
* view: Fix crash in non-unicode capable locale when entering a view
of metadata containing a slash or backslash.
* When annex.http-headers is used to set the User-Agent header, avoid
sending User-Agent: git-annex
* Windows: Fix rsync cross-drive hack to work with msys2 rsync.
Thanks, Pieter Kitslaar.
-- Joey Hess <id@joeyh.name> Thu, 14 Jan 2016 10:14:19 -0400
git-annex (5.20151218) unstable; urgency=medium
* Add S3 features to git-annex version output.
* webdav: When testing the WebDAV server, send a file with content.
The empty file it was sending tickled bugs in some php WebDAV server.
* fsck: Failed to honor annex.diskreserve when checking a remote.
* Debian: Build depend on concurrent-output.
* Fix insecure temporary permissions when git-annex repair is used in
in a corrupted git repository.
* Fix potential denial of service attack when creating temp dirs.
-- Joey Hess <id@joeyh.name> Fri, 18 Dec 2015 12:09:33 -0400
git-annex (5.20151208) unstable; urgency=medium
* Build with -j1 again to get reproducible build.
* Display progress meter in -J mode when copying from a local git repo,
to a local git repo, and from a remote git repo.
* Display progress meter in -J mode when downloading from the web.
* map: Improve display of git remotes with non-ssh urls, including http
and gcrypt.
* When core.sharedRepository is set, annex object files are not made mode
444, since that prevents a user other than the file owner from locking
them. Instead, a mode such as 664 is used in this case.
* tahoe: Include tahoe capabilities in whereis display.
* import: Changed to honor annex.largefiles settings.
* addurl, importfeed: Changed to honor annex.largefiles settings,
when the content of the url is downloaded. (Not when using --fast or
--relaxed.)
* webapp: Fix bugs that could result in a relative path such as "."
being written to ~/.config/git-annex/autostart, and ignore any such
relative paths in the file.
This was a reversion caused by the relative path changes in 5.20150113.
* dropunused: Make more robust when trying to drop an object that has
already been dropped.
* Fix reversion in handling of long filenames, particularly when using
addurl/importfeed, which was introduced in the previous release.
-- Joey Hess <id@joeyh.name> Tue, 08 Dec 2015 11:14:03 -0400
git-annex (5.20151116) unstable; urgency=medium
* Use concurrent-output library when configured with -fConcurrentOutput.
This allows nicely displayed messages when using the -J flag.
* Additional commands now support the -J flag:
fsck, drop, add, addurl, import
* import: Avoid very ugly error messages when the directory files
are imported to is not a directort, but perhaps an annexed file.
* Concurrent progress bars are now displayed when using -J with a command
that moves file contents around.
* Fix race that could result in an annexed file's symlink not being
created, when eg, running concurrent git-annex adds.
* add: Fix error recovery rollback to not move the injested file content
out of the annex back to the file, because other files may point to
that same content. Instead, copy the injected file content out to
recover.
* quvi may output utf-8 encoded data when the conifigured locale doesn't
support that; avoid crashing on such invalid encoding.
* runshell: Avoid failing when $HOME/.ssh does not exist and cannot be
created.
* Make the git-annex-standalone.deb prevent runshell from installing
wrappers into $HOME/.ssh
* Make git-annex-standalone.deb include the git-annex html documentation,
desktop file, and base completion file, same as the regular git-annex.deb.
* fsck: When fscking a dead repo, avoid incorrect "fixing location log"
message, and display a warning about it being dead, since it's unusual
to have access to a dead repo.
* assistant: Pass ssh-options through 3 more git pull/push calls
that were missed before.
* Added annex.pidlock and annex.pidlocktimeout configuration to support
filesystems where POSIX fcntl locks cannot be used.
* init: Automatically enable annex.pidlock when necessary.
-- Joey Hess <id@joeyh.name> Mon, 16 Nov 2015 14:17:40 -0400
git-annex (5.20151102.1) unstable; urgency=medium
* Avoid installing desktop file and program file if cabal install
git-annex is run as root, since that is not a systemwide install,
but to /root, and so generating a systemwide desktop file is not right.
* When cabal install is run with the desktop file location not writable,
display a warning, but continue successfully.
-- Joey Hess <id@joeyh.name> Tue, 03 Nov 2015 12:08:38 -0400
git-annex (5.20151102) unstable; urgency=medium
* Use statvfs on OSX.
* Symlink timestamp preservation code uses functions
from unix-2.7.0 when available, which should be more portable.
* enableremote: List uuids and descriptions of remotes that can be
enabled, and accept either the uuid or the description in leu if the
name.
* Catch up with current git behavior when both repo and repo.git exist;
it seems it now prefers repo in this case, although historically it may
have preferred repo.git.
* Fix failure to build with aws-0.13.0.
* When built with aws-0.13.0, the S3 special remote can be used to create
google nearline buckets, by setting storageclass=NEARLINE.
-- Joey Hess <id@joeyh.name> Mon, 02 Nov 2015 12:41:20 -0400
git-annex (5.20151019) unstable; urgency=medium
* Fix a longstanding, but unlikely to occur bug, where dropping
a file from a remote could race with other drops of the same file,
and result in all copies of its content being lost.
* git-annex-shell: Added lockcontent command, to prevent dropping of
a key's content. This is necessary due to the above bugfix.
* In some cases, the above bugfix changes what git-annex allows you to
drop:
- When a file is present in several special remotes,
but not in any accessible git repositories, dropping it from one of
the special remotes will now fail. Instead, the file has to be
moved from one of the special remotes to the git repository, and can
then safely be dropped from the git repository.
- If a git remote has too old a version of git-annex-shell installed,
git-annex won't trust it to hold onto a copy of a file when dropping
that file from the local git repository.
* Changed drop ordering when using git annex sync --content or the
assistant, to drop from remotes first and from the local repo last.
This works better with the behavior changes to drop in many cases.
* Do verification of checksums of annex objects downloaded from remotes.
* When annex objects are received into git repositories from other git
repos, their checksums are verified then too.
* To get the old, faster, behavior of not verifying checksums, set
annex.verify=false, or remote.<name>.annex-verify=false.
* setkey, rekey: These commands also now verify that the provided file
matches the expected checksum of the key, unless annex.verify=false.
* reinject: Already verified content; this can now be disabled by
setting annex.verify=false.
* sync, merge, assistant: When git merge failed for a reason other
than a conflicted merge, such as a crippled filesystem not allowing
particular characters in filenames, git-annex would make a merge commit
that could omit such files or otherwise be bad. Fixed by aborting the
whole merge process when git merge fails for any reason other than a
merge conflict.
* Allow building with S3 disabled again.
* Ported disk free space checking code to work on Solaris.
* Windows webapp: Fix support for entering password when setting
up a ssh remote.
* copy --auto was checking the wrong repo's preferred content.
(--from was checking what --to should, and vice-versa.)
Fixed this bug, which was introduced in version 5.20150727.
* Avoid unncessary write to the location log when a file is unlocked
and then added back with unchanged content.
* S3: Fix support for using https.
* Avoid displaying network transport warning when a ssh remote
does not yet have an annex.uuid set.
* Debian: Add torrent library to build-depends as it's packaged now,
and stop recommending bittornado | bittorrent.
* Debian: Remove build dependency on transformers library, as it is now
included in ghc.
* Debian: Remove menu file, since a desktop file is provided and
lintian says there can be only one.
-- Joey Hess <id@joeyh.name> Mon, 19 Oct 2015 13:59:01 -0400
git-annex (5.20150930) unstable; urgency=medium
* Added new linux standalone "ancient" build to support kernels
like 2.6.32.
* info: Don't allow use in a non-git-annex repository, since it
uses the git-annex branch and would create it if it were missing.
* assistant: When updating ~/.ssh/config, preserve any symlinks.
* webapp: Remove the "disable remote" feature from the UI.
* S3: When built with aws-0.13.0, supports using more storage classes.
In particular, storageclass=STANDARD_IA to use Amazon's
new Infrequently Accessed storage, and storageclass=NEARLINE
to use Google's NearLine storage.
* Improve ~/.ssh/config modification code to not add trailing spaces
to lines it cannot parse.
* Fix a crash at direct mode merge time when .git/index doesn't exist
yet. Triggered by eg, git-annex sync --no-commit in a fresh clone of
a repository.
* status: Show added but not yet committed files.
* Added stack.yaml to support easy builds from source with stack.
-- Joey Hess <id@joeyh.name> Wed, 30 Sep 2015 14:31:52 -0400
git-annex (5.20150916) unstable; urgency=medium
* Fix Windows build to work with ghc 7.10.
* init: Fix reversion in detection of repo made with git clone --shared
* info: Support querying info of individual files in direct mode.
* unused: Fix reversion in 5.20150727 that broke parsing of the
--unused-refspec option. Thanks, Øyvind A. Holm.
* Make full option parsing be done when not in a git repo, so --help
can be displayed for commands that require a git repo, etc.
* fsck: Work around bug in persistent that broke display of
problematically encoded filenames on stderr when using --incremental.
* When gpg.program is configured, it's used to get the command to run
for gpg. Useful on systems that have only a gpg2 command or want to
use it instead of the gpg command.
* Windows: Switched to using git for Windows, rather than msysgit.
Using msysgit with git-annex is no longer supported.
* Windows: Even when the user neglects to tell the git installer to
add git to PATH, git-annex will still work from within the git bash
shell, and the webapp can be used too.
* sync: Add --no-commit, --no-pull, --no-push options to turn off parts of
the sync process, as well as supporting --commit, --pull, --push, and
--no-content options to specify the (current) default behavior.
* annex.hardlink extended to also try to use hard links when copying from
the repository to a remote.
* Improve bash completion, so it completes names of remotes and backends
in appropriate places.
* Special remotes configured with autoenable=true will be automatically
enabled when git-annex init is run.
* Fix bug in combination of preferred and required content settings.
When one was set to the empty string and the other set to some expression,
this bug caused all files to be wanted, instead of only files matching
the expression.
-- Joey Hess <id@joeyh.name> Wed, 16 Sep 2015 10:31:24 -0400
git-annex (5.20150824) unstable; urgency=medium
* Sped up downloads of files from ssh remotes, reducing the
non-data-transfer overhead 6x.
* sync: Support --jobs
* sync --content: Avoid unnecessary second pull from remotes when
no file transfers are made.
* External special remotes can now be built that can be used in readonly
mode, where git-annex downloads content from the remote using regular
http.
* Added WHEREIS to external special remote protocol.
* importfeed --relaxed: Avoid hitting the urls of items in the feed.
* Fix reversion in init when ran as root, introduced in version 5.20150731.
* Reorder declaration to fix build with yesod-core > 1.4.13.
Thanks, Michael Alan Dorman.
* Fix building without quvi and without database.
Thanks, Ben Boeckel.
* Avoid building the assistant on the hurd, since an inotify equivalent
is not yet implemented in git-annex for the hurd.
* --debug log messages are now timestamped with fractional seconds.
* --debug is passed along to git-annex-shell when git-annex is in debug mode.
* Makefile: Pass LDFLAGS, CFLAGS, and CPPFLAGS through ghc and on to
ld, cc, and cpp.
* As a result of the Makefile changes, the Debian package is built
with various hardening options. Although their benefit to a largely
haskell program is unknown.
-- Joey Hess <id@joeyh.name> Mon, 24 Aug 2015 14:11:05 -0700
git-annex (5.20150812) unstable; urgency=medium
* Added support for SHA3 hashed keys (in 8 varieties), when git-annex is
built using the cryptonite library.
* metadata: Fix reversion introduced in 5.20150727 that caused recursive
display of metadata to not work.
* Windows: Fix bug that caused git-annex sync to fail due to missing
environment variable.
* Fix setting/setting/viewing metadata that contains unicode or other
special characters, when in a non-unicode locale.
* Simplify setup process for a ssh remote. Now it suffices to run git
remote add, followed by git-annex sync. Now the remote is automatically
initialized for use by git-annex, where before the git-annex branch had
to manually be pushed before using git-annex sync. Note that this
involved changes to git-annex-shell, so if the remote is using an old
version, the manual push is still needed.
* git-annex-shell: Don't let configlist auto-init repository when in
readonly mode.
* Perform a clean shutdown when --time-limit is reached.
This includes running queued git commands, and cleanup actions normally
run when a command is finished.
* fsck: Commit incremental fsck database when --time-limit is reached.
Previously, some of the last files fscked did not make it into the
database when using --time-limit.
* fsck: Commit incremental fsck database after every 1000 files
fscked, or every 5 minutes, whichever comes first. Previously,
commits were made every 1000 files fscked.
* Linux standalone: Work around problem that prevented it from working
properly if unpacked into a directory that contains ":" or ";" in its
name.
* proxy: Fix proxy git commit of non-annexed files in direct mode.
* proxy: If a non-proxied git command, such as git revert
would normally fail because of unstaged files in the work tree,
make the proxied command fail the same way.
* proxy: Fix removal of files deleted by the proxied command.
* proxy: Fix behavior when run in subdirectory of git repo.
* Improve Setup.hs file so that cabal copy --destdir works.
Thanks, Magnus Therning.
* Tighten dependency on optparse-applicative to 0.11.0.
* Added back debian/cabal-wrapper, since it still seems needed after all.
-- Joey Hess <id@joeyh.name> Wed, 12 Aug 2015 11:14:58 -0400
git-annex (5.20150731) unstable; urgency=medium
* webapp: Support enabling known gitlab.com remotes.
* Fix rsync special remote to work when -Jn is used for concurrent
uploads.
correct spelling mistakes
2017-02-11 09:38:49 +00:00
* The last release accidentally removed a number of options from the
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
copy command. (-J, file matching options, etc). These have been added
back.
* init: Detect when the filesystem is crippled such that it ignores
attempts to remove the write bit from a file, and enable direct mode.
Seen with eg, NTFS fuse on linux.
* Fix man page installation by cabal install; all the new man pages are
now installed.
-- Joey Hess <id@joeyh.name> Fri, 31 Jul 2015 11:34:36 -0400
git-annex (5.20150727) unstable; urgency=medium
* Fix bug that prevented uploads to remotes using new-style chunking
from resuming after the last successfully uploaded chunk.
* Switched option parsing to use optparse-applicative. This was a very large
and invasive change, and may have caused some minor behavior changes to
edge cases of option parsing. (For example, the metadata command no
longer accepts the combination of --get and --set, which never actually
worked.)
* Bash completion file is now included in the git-annex source tree,
and installed into Debian package (and any other packages built using make
install). This bash completion is generated by the option parser, so it
covers all commands, all options, and will never go out of date!
* As well as tab completing "git-annex" commands, "git annex" will also tab
complete. However, git's bash completion script needs a patch,
which I've submitted, for this to work prefectly.
* version --raw now works when run outside a git repository.
* assistant --startdelay now works when run outside a git repository.
* dead now accepts multiple --key options.
* addurl now accepts --prefix and --suffix options to adjust the
filenames used.
* sync --content: Fix bug that caused files to be uploaded to eg,
more archive remotes than wanted copies, only to later be dropped
to satisfy the preferred content settings.
* importfeed: Improve detection of known items whose url has changed,
and avoid adding redundant files. Where before this only looked at
permalinks in rss feeds, it now also looks at guids.
* importfeed: Look at not only permalinks, but now also guids
to identify previously downloaded files.
* Webapp: Now features easy setup of git-annex repositories on gitlab.com.
* Adjust debian build deps: The webapp can now build on arm64, s390x
and hurd-i386. WebDAV support is also available on those architectures.
* Debian package now maintained by Richard Hartmann.
* Support building without persistent database on for systems that
lack TH. This removes support for incremental fsck.
-- Joey Hess <id@joeyh.name> Mon, 27 Jul 2015 12:24:49 -0400
git-annex (5.20150710) unstable; urgency=medium
* add: Stage symlinks the same as git add would, even if they are not a
link to annexed content.
* sync: When annex.autocommit=false, avoid making any commit of local
changes, while still merging with remote to the extent possible.
* unused: --used-refspec can now be configured to look at refs in the
reflog. This provides a way to not consider old versions of files to be
unused after they have reached a specified age, when the old refs in
the reflog expire.
* log: Fix reversion introduced in version 5.20150528 that broke this command.
* assistant --autostart: First stop any daemons that are already running,
which might be left over from a previous login session and so unable to
use the ssh agent of a new login session.
* assistant: Fix local pairing to not include newline in ssh pubkey,
which is rejected on the other end for security reasons.
* assistant: Fix ANNEX_SHELL_DIR written to ~/.ssh/authorized_keys
in local pairing to be the absolute path to the repository, not "."
This was a reversion caused by the relative path changes in 5.20150113.
* Brought back the setkey plumbing command that was removed in 2011, since
we found a use case for it. Note that the command's syntax was changed
for consistency.
* bugfix: Pass --full-tree when using git ls-files to get a list of files
on the git-annex branch, so it works when run in a subdirectory.
This bug affected git-annex unused, and potentially also transitions
running code and other things.
* Support git's undocumented core.sharedRepository=2 value, which
is equivalent to "world", and is set when a repo was created using
git init --shared=world.
* When building on linux, pass --as-needed to linker to avoid linking
with unused shared libraries including libyaml.
* import: Fix failure of cross-device import on Windows.
* merge: Avoid creating the synced/master branch.
* Removed support for optparse-applicative versions older than 0.10.
-- Joey Hess <id@joeyh.name> Fri, 10 Jul 2015 16:36:42 -0400
git-annex (5.20150617) unstable; urgency=medium
* Now supports git annex sync --all --content to sync all versions of all
files with all repos that want them.
* Added new "anything" preferred content expression, which matches all
versions of all files.
* Standard preferred content for client, backup, incremental backup,
and unwanted groups have been adjusted to work better when used
with git annex sync --all --content.
* fromkey, registerurl: Improve handling of urls that happen to also
be parsable as strange keys.
* sync, remotedaemon: Pass configured ssh-options even when
annex.sshcaching is disabled.
* assistant: Consume systemd-networkd dbus events to learn about
changes to network connections, as was already done with
network-manager and wicd.
Thanks to Sebastian Reuße for the patches.
* get --incomplete: New option to resume any interrupted downloads.
* dead --key: Can be used to mark a key as dead.
* fsck: Ignore keys that are known to be dead when running in
--all/--unused/--key mode or a in a bare repo. Closes: #753888
Otherwise, still reports files with lost contents, even if the content
is dead.
* S3: Special remotes can be configured with public=yes to allow
the public to access the bucket's content.
* S3: Publically accessible buckets can be used without creds.
* import --clean-duplicates: Fix bug that didn't count local or trusted
repo's copy of a file as one of the necessary copies to allow removing
it from the import location.
* tahoe: Use ~/.tahoe-git-annex/ rather than ~/.tahoe/git-annex/
when setting up a tahoe special remote to avoid old versions of
tahoe create-client choking.
* Fix bug that prevented enumerating locally present objects in repos
tuned with annex.tune.objecthash1=true.
Fixes: unused, object count in info, unannex.
* Improve url parsing to handle some urls containing illegal []
characters in their paths.
* info: Added json output for "backend usage", "numcopies stats",
"repositories containing these files", and "transfers in progress".
* Fix incremental backup standard preferred content expression to match
its documentation, which says it does not want files that have reached
a backup repository.
* Increased the default annex.bloomaccuracy from 1000 to 10000000.
This makes git annex unused use up to 16 mb more memory than it did
before, but the massive increase in accuracy makes this worthwhile
for all but the smallest systems.
* Build documentation with deterministic=1 for reproducible builds.
(A new ikiwiki feature.) Closes: #785736
* Re-remove dependency on obsolete hamlet package. Closes: #786659
* debian/cabal-wrapper: Removed this hack which should not be needed anymore.
-- Joey Hess <id@joeyh.name> Wed, 17 Jun 2015 13:50:35 -0400
git-annex (5.20150528) unstable; urgency=medium
* fromkey, registerurl: Allow urls to be specified instead of keys,
and generate URL keys.
* Linux standalone, OSX app: Improve runshell script to always quote
shell vars, so that it will work when eg, untarred into a directory
path with spaces in its name.
* Revert removal dependency on obsolete hamlet package, since the
autobuilders are not ready for this change yet and it prevented them
from building the webapp. Reopens: #786659
* fsck: When checksumming a file fails due to a hardware fault,
the file is now moved to the bad directory, and the fsck proceeds.
Before, the fsck immediately failed.
* Linux standalone: The webapp was not built in the previous release,
this release fixes that oversight.
-- Joey Hess <id@joeyh.name> Thu, 28 May 2015 10:48:03 -0400
git-annex (5.20150522) unstable; urgency=medium
* import: Refuse to import files that are within the work tree, as that
does not make sense and could cause data loss.
* drop: Now supports --all, --unused, and --key.
* drop: Now defaults to --all when run in a bare repository.
(Previously, did nothing when run in a bare repository.)
* get, move, copy, mirror: Concurrent transfers are now supported!
For example: git-annex get -J10
However, progress bars are not yet displayed for concurrent transfers,
pending an updated version of the ascii-progress library.
* --quiet now makes progress output by rsync, wget, etc be quiet too.
* Take space that will be used by other running downloads into account when
checking annex.diskreserve.
* Avoid accumulating transfer failure log files unless the assistant is
being used.
* Fix an unlikely race that could result in two transfers of the same key
running at once.
* Stale transfer lock and info files will be cleaned up automatically
when get/unused/info commands are run.
* unused: Add --used-refspec option and annex.used-refspec, which can
specify a set of refs to consider used, rather than the default of
considering all refs used.
* webapp: Fix zombie xdg-open process left when opening file browser.
Closes: #785498
* Safer posix fctnl locking implementation, using lock pools and STM.
* Build documentation with TZ=UTC for reproducible builds. See #785736.
* OSX: Corrected the location of trustedkeys.gpg, so the built-in
upgrade code will find it. Fixes OSX upgrade going forward, but
older versions won't upgrade themselves due to this problem.
* Remove dependency on obsolete hamlet package. Closes: #786659
-- Joey Hess <id@joeyh.name> Fri, 22 May 2015 14:20:18 -0400
git-annex (5.20150508.1) unstable; urgency=medium
* Now builds cleanly using ghc 7.10 (as well as ghc back to 7.6).
* Imrovements to the git-annex-standalone.deb build process.
(Thanks, Yaroslav Halchenko)
-- Joey Hess <id@joeyh.name> Mon, 11 May 2015 12:08:58 -0400
git-annex (5.20150508) unstable; urgency=medium
* Improve behavior when a git-annex command is told to operate
on a file that doesn't exist. It will now continue to other
files specified after that on the command line, and only error out at
the end.
* S3: Enable debug logging when annex.debug or --debug is set.
* S3: git annex info will show additional information about a S3 remote
(endpoint, port, storage class)
* S3: Let git annex enableremote be used, without trying to recreate
a bucket that should already exist.
correct spelling mistakes
2017-02-11 09:38:49 +00:00
* S3: Fix incompatibility with bucket names used by hS3; the aws library
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
cannot handle upper-case bucket names. git-annex now converts them to
lower case automatically.
* import: Check for gitignored files before moving them into the tree.
(Needs git 1.8.4 or newer.)
* import: Don't stop entire import when one file fails due to being
gitignored or conflicting with something in the work tree.
* import: Before removing a duplicate file in --deduplicate or
--clean-duplicates mode, verify that enough copies of its content still
exist.
* Improve integration with KDE's file manager to work with dolphin
version 14.12.3 while still being compatable with 4.14.2.
Thanks, silvio.
* assistant: Added --autostop to complement --autostart.
* Work around wget bug #784348 which could cause it to clobber git-annex
symlinks when downloading from ftp.
* Support checking ftp urls for file presence.
* Fix bogus failure of fsck --fast.
* fsck: Ignore error recording the fsck in the activity log,
which can happen when running fsck in a read-only repository.
Closes: #698559
(fsck can still need to write to the repository if it find problems,
but a successful fsck can be done read-only)
* Improve quvi 0.4 output parsing to handle cases wher there is no known
filename extension. This is currently the case when using quvi with
youtube. In this case, the extension ".m" will be used.
* Dropped support for older versions of yesod, warp, and dbus than the ones
in Debian Jessie.
* Switch from the obsolete dataenc library for base64 encoding to sandi.
(Thanks, Magnus Therning)
* Debian's ghc now supports TH on arm! Adjust build dependencies
to build the webapp on arm, and enable DAV support on arm. \o/
* Adjust some other arch specific build dependencies that are now
available on more architectures in Devian unstable.
* Windows: Remove cygwin ssh, the newer version of which has stopped
honoring the setting of HOME. Instead, copy msysgit's ssh into PATH.
Note that setting up a remote ssh server using password authentication
is known to be broken in this release on Windows.
* Windows: Roll back to an older version of rsync from cygwin.
The newer version has some dependency on a newer ssh from cygwin.
-- Joey Hess <id@joeyh.name> Fri, 08 May 2015 13:42:30 -0400
git-annex (5.20150420) unstable; urgency=medium
* Fix activity log parsing, which caused the log to not retain
activity from other uuids.
* Union merge could fall over if there was a file in the repository
with the same name as a git ref. Now fixed.
* info dir: Added information about repositories that
contain files in the specified directory.
* info: Added --bytes option.
* bittorrent: Fix handling of magnet links.
* When a key's size is unknown, still check the annex.diskreserve,
and avoid getting content if the disk is too full.
* Fix fsck --from a git remote in a local directory, and from
a directory special remote.
This was a reversion caused by the relative path changes in 5.20150113.
* fsck --from remote: When bad content is found in the remote,
and the local repo does not have a copy of the content, preserve
the bad content in .git/annex/bad/ to avoid further data loss.
* fsck --from remote: Avoid downloading a key if it would go over
the annex.diskreserve limit.
* required: New command, like wanted, but for required content.
* Removed dependency on haskell SHA library,
instead using cryptohash >= 0.11.0.
* Make repo init more robust.
* New debian/rules build-standalone target, which generates a
git-annex-standalone.deb that should work on many old Debian etc
systems. Thanks, Yaroslav Halchenko.
* Windows: Renamed start menu file to avoid loop in some versions
of Windows where the menu file is treated as a git-annex program.
* Windows: Fixed support of remotes on other drives.
(A reversion introduced in version 5.20150113.)
* Windows: Bundled versions of rsync, wget, ssh, and gpg from
cygwin all updated. Thanks, Yury V. Zaytsev.
-- Joey Hess <id@joeyh.name> Mon, 20 Apr 2015 14:44:04 -0400
git-annex (5.20150409) unstable; urgency=medium
* This fixes a bug in the assistant introduced by the literal pathspec
changes in version 5.20150406.
* --quiet now suppresses progress displays from eg, rsync.
(Second time's the charm..)
* fromkey, registerurl: When reading from stdin, allow the
filename and url, respectively, to contain whitespace.
* add: If annex.largefiles is set and does not match a file that's being
added, the file will be checked into git rather than being added to the
annex. Previously, git annex add skipped over such files; this new
behavior is more useful in direct mode.
* proxy: Made it work when run in a new repository before initial
commit.
* info: Display repository mode: bare when in a bare (non-direct mode)
repo.
* importfeed: Fix feed download when curl is used.
* importfeed: Error out when passed a non-url.
* webapp: When adding another local repository, and combining it
with the current repository, the new repository's remote path
was set to "." rather than the path to the current repository.
This was a reversion caused by the relative path changes in 5.20150113.
* contentlocationn: New plumbing command.
-- Joey Hess <id@joeyh.name> Thu, 09 Apr 2015 15:06:38 -0400
git-annex (5.20150406.1) unstable; urgency=medium
* Fixes a bug in the last release that caused rsync and possibly
other commands to hang at the end of a file transfer.
(--quiet is back to not blocking progress displays until
that code can be fixed properly.)
-- Joey Hess <id@joeyh.name> Mon, 06 Apr 2015 17:13:13 -0400
git-annex (5.20150406) unstable; urgency=medium
* Prevent git-ls-files from double-expanding wildcards when an
unexpanded wildcard is passed to a git-annex command like add or find.
* Fix make build target. Thanks, Justin Geibel.
* Fix GETURLS in external special remote protocol to strip
downloader prefix from logged url info before checking for the
specified prefix.
* importfeed: Avoid downloading a redundant item from a feed whose
permalink has been seen before, even when the url has changed.
* importfeed: Always store itemid in metadata; before this was only
done when annex.genmetadata was set.
* Relax debian package dependencies to git >= 1:1.8.1 rather
than needing >= 1:2.0.
* test: Fix --list-tests
* addurl --file: When used with a special remote that claims
urls and checks their contents, don't override the user's provided
filename with filenames that the special remote suggests. Also,
don't allow adding the url if the special remote says it contains
multiple files.
* import: --deduplicate and --cleanduplicates now output the keys
corresponding to duplicated files they process.
* expire: New command, for expiring inactive repositories.
* fsck: Record fsck activity for use by expire command.
* Fix truncation of parameters that could occur when using xargs git-annex.
* Significantly sped up processing of large numbers of directories
passed to a single git-annex command.
* version: Add --raw
* init: Improve fifo test to detect NFS systems that support fifos
but not well enough for sshcaching.
* --quiet now suppresses progress displays from eg, rsync.
(The option already suppressed git-annex's own built-in progress
displays.)
-- Joey Hess <id@joeyh.name> Mon, 06 Apr 2015 12:48:48 -0400
git-annex (5.20150327) unstable; urgency=medium
* readpresentkey: New plumbing command for checking location log.
* checkpresentkey: New plumbing command to check if a key can be verified
to be present on a remote.
* Added a post-update-annex hook, which is run after the git-annex branch
is updated. Needed for git update-server-info.
* migrate: --force will force migration of keys already using the
destination backend. Useful in rare cases.
* Man pages for individual commands now available, and can be
opened using "git annex help <command>"
* --auto is no longer a global option; only get, drop, and copy
accept it. (Not a behavior change unless you were passing it to a
command that ignored it.)
* Improve error message when --in @date is used and there is no
reflog for the git-annex branch.
* assistant: Committing a whole lot of files at once could overflow
command-line length limits and cause the commit to fail. This
only happened when using the assistant in an indirect mode repository.
* Work around curl bug when asked to download an empty url to a file.
* Fix bug introduced in the last release that broke git-annex sync
when git-annex was installed from the standalone tarball.
-- Joey Hess <id@joeyh.name> Fri, 27 Mar 2015 13:10:59 -0400
git-annex (5.20150317) unstable; urgency=medium
* fsck: Incremental fsck uses sqlite to store its records, instead
of abusing the sticky bit. Existing sticky bits are ignored;
incremental fscks started by old versions won't be resumed by
this version.
* fsck: Multiple incremental fscks of different repos (including remotes)
can now be running at the same time in the same repo without it
getting confused about which files have been checked for which remotes.
* unannex: Refuse to unannex when repo is too new to have a HEAD,
since in this case there must be staged changes in the index
(if there is anything to unannex), and the unannex code path
needs to run with a clean index.
* Linux standalone: Set LOCPATH=/dev/null to work around
https://ghc.haskell.org/trac/ghc/ticket/7695
This prevents localization from working, but git-annex
is not localized anyway.
* sync: As well as the synced/git-annex push, attempt a
git-annex:git-annex push, as long as the remote branch
is an ancestor of the local branch, to better support bare git repos.
(This used to be done, but it forgot to do it since version 4.20130909.)
* When re-execing git-annex, use current program location, rather than
~/.config/git-annex/program, when possible.
* Submodules are now supported by git-annex!
* metadata: Fix encoding problem that led to mojibake when storing
metadata strings that contained both unicode characters and a space
(or '!') character.
* Also potentially fixes encoding problem when embedding credentials
that contain unicode characters.
* sync: Fix committing when in a direct mode repo that has no HEAD ref.
(For example, a newly checked out git submodule.)
* Added SETURIPRESENT and SETURIMISSING to external special remote protocol,
useful for things like ipfs that don't use regular urls.
* addurl: Added --raw option, which bypasses special handling of quvi,
bittorrent etc urls.
* git-annex-shell: Improve error message when the specified repository
doesn't exist or git config fails for some reason.
* fromkey --force: Skip test that the key has its content in the annex.
* fromkey: Add stdin mode.
* registerurl: New plumbing command for mass-adding urls to keys.
* remotedaemon: Fixed support for notifications of changes to gcrypt
remotes, which was never tested and didn't quite work before.
-- Joey Hess <id@joeyh.name> Tue, 17 Mar 2015 13:02:36 -0400
git-annex (5.20150219) unstable; urgency=medium
* glacier: Detect when the glacier command in PATH is the wrong one,
from boto, rather than from glacier-cli, and refuse to use it,
since the boto program fails to fail when passed
parameters it does not understand.
* groupwanted: New command to set the groupwanted preferred content
expression.
* import: Support file matching options such as --exclude, --include,
--smallerthan, --largerthan
* The file matching options are now only accepted by commands that
can actually use them, instead of by all commands.
* import: Avoid checksumming file twice when run in the default
or --duplicate mode.
* Windows: Fix bug in dropping an annexed file, which
caused a symlink to be staged that contained backslashes.
* webapp: Fix reversion in opening webapp when starting it manually
inside a repository.
* assistant: Improve sanity check for control characters when pairing.
* Improve race recovery code when committing to git-annex branch.
* addurl: Avoid crash if quvi is not installed, when git-annex was
built with process-1.2
* bittorrent: Fix mojibake introduced in parsing arai2c progress output.
* fsck --from: If a download from a remote fails, propagate the failure.
* metadata: When setting metadata, do not recurse into directories by
default, since that can be surprising behavior and difficult to recover
from. The old behavior is available by using --force.
* sync, assistant: Include repository name in head branch commit message.
* The ssh-options git config is now used by gcrypt, rsync, and ddar
special remotes that use ssh as a transport.
* sync, assistant: Use the ssh-options git config when doing git pull
and push.
* remotedaemon: Use the ssh-options git config.
* Linux standalone: Improved process names of linker shimmed programs.
-- Joey Hess <id@joeyh.name> Thu, 19 Feb 2015 14:16:03 -0400
git-annex (5.20150205) unstable; urgency=medium
* info: Can now display info about a given uuid.
* Added to remote/uuid info: Count of the number of keys present
on the remote, and their size. This is rather expensive to calculate,
so comes last and --fast will disable it.
* info remote: Include the date of the last sync with the remote.
* sync: Added --message/-m option like git commit.
* remotedaemon: Fix problem that could prevent ssh connections being
made after two LOSTNET messages were received in a row (perhaps due to
two different network interfaces being brought down).
* Fix build failure when wget is not installed.
* Fix wording of message displayed when unable to get a file that
is available in untrusted repositories.
* addurl: When a Content-Disposition header suggests a filename to use,
addurl will consider using it, if it's reasonable and doesn't conflict
with an existing file. (--file overrides this)
* Fix default repository description created by git annex init,
which got broken by the relative path changes in the last release.
* init: Repository tuning parameters can now be passed when initializing a
repository for the first time. For details, see
http://git-annex.branchable.com/tuning/
* merge: Refuse to merge changes from a git-annex branch of a repo
that has been tuned in incompatible ways.
* Support annex.tune.objecthash1, annex.tune.objecthashlower, and
annex.tune.branchhash1.
* Remove support for building without cryptohash.
* Added MD5 and MD5E backends.
* assistant: Fix local pairing when ssh pubkey comment contains spaces.
* Avoid using fileSize which maxes out at just 2 gb on Windows.
Instead, use hFileSize, which doesn't have a bounded size.
Fixes support for files > 2 gb on Windows.
* Windows: Fix running of the pre-commit-annex hook.
* Windows: Fix S3 special remote; need to call withSocketsDo. Thanks, Trent.
-- Joey Hess <id@joeyh.name> Thu, 05 Feb 2015 14:08:33 -0400
git-annex (5.20150113) unstable; urgency=medium
* unlock: Don't allow unlocking files that have never been committed to git
before, to avoid an intractable problem that prevents the pre-commit
hook from telling if such a file is intended to be an annexed file or not.
* Avoid re-checksumming when migrating from hash to hashE backend.
Closes: #774494
* Fix build with process 1.2.1.0.
* Android: Provide a version built with -fPIE -pie to support Android 5.0.
* sync: Fix an edge case where syncing in a bare repository would try to
merge and so fail.
* Check git version at runtime, rather than assuming it will be the same
as the git version used at build time when running git-checkattr and
git-branch remove.
* Switch to using relative paths to the git repository.
- This allows the git repository to be moved while git-annex is running in
it, with fewer problems.
- On Windows, this avoids some of the problems with the absurdly small
MAX_PATH of 260 bytes. In particular, git-annex repositories should
work in deeper/longer directory structures than before.
* Generate shorter keys for WORM and URL, avoiding keys that are longer
than used for SHA256, so as to not break on systems like Windows that
have very small maximum path length limits.
* Bugfix: A file named HEAD in the work tree could confuse some git commands
run by git-annex.
-- Joey Hess <id@joeyh.name> Tue, 13 Jan 2015 12:10:08 -0400
git-annex (5.20141231) unstable; urgency=medium
* vicfg: Avoid crashing on badly encoded config data.
* Work around statfs() overflow on some XFS systems.
* sync: Now supports remote groups, the same way git remote update does.
* setpresentkey: A new plumbing-level command.
* Run shutdown cleanup actions even if there were failures processing
the command. Among other fixes, this means that addurl will stage
added files even if adding one of the urls fails.
* bittorrent: Fix locking problem when using addurl file://
* Windows: Fix local rsync filepath munging (fixes 26 test suite failures).
* Windows: Got the rsync special remote working.
* Windows: Fix handling of views of filenames containing '%'
* OSX: Switched away from deprecated statfs64 interface.
-- Joey Hess <id@joeyh.name> Wed, 31 Dec 2014 15:15:46 -0400
git-annex (5.20141219) unstable; urgency=medium
* Webapp: When adding a new box.com remote, use the new style chunking.
Thanks, Jon Ander Peñalba.
* External special remote protocol now includes commands for setting
and getting the urls associated with a key.
* Urls can now be claimed by remotes. This will allow creating,
for example, a external special remote that handles magnet: and
*.torrent urls.
* Use wget -q --show-progress for less verbose wget output,
when built with wget 1.16.
* Added bittorrent special remote.
* addurl behavior change: When downloading an url ending in .torrent,
it will download files from bittorrent, instead of the old behavior
of adding the torrent file to the repository.
* Added Recommends on aria2.
* When possible, build with the haskell torrent library for parsing
torrent files. As a fallback, can instead use btshowmetainfo from
bittornado | bittorrent.
* Fix build with -f-S3.
-- Joey Hess <id@joeyh.name> Fri, 19 Dec 2014 16:53:26 -0400
git-annex (5.20141203) unstable; urgency=medium
* proxy: New command for direct mode repositories, allows bypassing
the direct mode guard in a safe way to do all sorts of things
including git revert, git mv, git checkout ...
* undo: New command to undo the most recent change to a file
or to the contents of a directory.
* Add undo action to nautilus and konqueror integration.
* diffdriver: New git-annex command, to make git external diff drivers
work with annexed files.
* pre-commit: Block partial commit of unlocked annexed file, since
that left a typechange staged in index due to some infelicity of git's
handling of partial commits.
* Work around behavior change in lsof 4.88's -F output format.
* S3: Switched to using the haskell aws library.
* S3: No longer buffers entire files in memory when uploading without
chunking.
* S3: When built with a new enough version of the haskell aws library,
supports doing multipart uploads, in order to store extremely large
files in S3 when not using chunking.
* Don't show "(gpg)" when decrypting the remote encryption cipher,
since this could be taken to read that's the only time git-annex
runs gpg, which is not the case.
* Debian package is now maintained by Gergely Nagy.
* Windows: Remove Alt+A keyboard shortcut, which turns out to have scope
outside the menus.
* Windows: Install ssh and other bundled programs to Git/cmd,
instead of Git/bin, since the latter is not in the default msysgit PATH.
-- Joey Hess <id@joeyh.name> Wed, 03 Dec 2014 15:16:52 -0400
git-annex (5.20141125) unstable; urgency=medium
* Remove fixup code for bad bare repositories created by
versions 5.20131118 through 5.20131127. That fixup code would
correct spelling mistakes
2017-02-11 09:38:49 +00:00
accidentally fire when --git-dir was incorrectly
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
pointed at the working tree of a git-annex repository,
possibly resulting in data loss. Closes: #768093
* Windows: Fix crash when user.name is not set in git config.
-- Joey Hess <joeyh@debian.org> Wed, 05 Nov 2014 11:41:51 -0400
git-annex (5.20141024) unstable; urgency=medium
* vicfg: Deleting configurations now resets to the default, where
before it has no effect.
* Remove hurd stuff from cabal file, since hackage currently rejects
it, and the test suite fails on hurd.
* initremote: Don't allow creating a special remote that has the same
name as an existing git remote.
* Windows: Use haskell setenv library to clean up several ugly workarounds
for inability to manipulate the environment on windows. This includes
making git-annex not re-exec itself on start on windows, and making the
test suite on Windows run tests without forking.
* glacier: Fix pipe setup when calling glacier-cli to retrieve an object.
* info: When run on a single annexed file, displays some info about the
file, including its key and size.
* info: When passed the name or uuid of a remote, displays info about that
remote. Remotes that support encryption, chunking, or embedded
creds will include that in their info.
* enableremote: When the remote has creds, update the local creds cache
file. Before, the old version of the creds could be left there, and
would continue to be used.
-- Joey Hess <joeyh@debian.org> Fri, 24 Oct 2014 13:03:29 -0400
git-annex (5.20141013) unstable; urgency=medium
* Adjust cabal file to support building w/o assistant on the hurd.
* Support building with yesod 1.4.
* S3: Fix embedcreds=yes handling for the Internet Archive.
* map: Handle .git prefixed remote repos. Closes: #614759
* repair: Prevent auto gc from happening when fetching from a remote.
-- Joey Hess <joeyh@debian.org> Mon, 13 Oct 2014 10:13:06 -0400
git-annex (5.20140927) unstable; urgency=medium
* Really depend (not just build-depend) on new enough git for --no-gpg-sign
to work. Closes: #763057
* Add temporary workaround for bug #763078 which broke building on armel
and armhf.
-- Joey Hess <joeyh@debian.org> Sat, 27 Sep 2014 14:25:09 -0400
git-annex (5.20140926) unstable; urgency=high
* Depend on new enough git for --no-gpg-sign to work. Closes: #762446
* Work around failure to build on mips by using cabal, not Setup,
to build in debian/rules.
-- Joey Hess <joeyh@debian.org> Fri, 26 Sep 2014 15:09:02 -0400
git-annex (5.20140919) unstable; urgency=high
* Security fix for S3 and glacier when using embedcreds=yes with
encryption=pubkey or encryption=hybrid. CVE-2014-6274
The creds embedded in the git repo were *not* encrypted.
git-annex enableremote will warn when used on a remote that has
this problem. For details, see:
https://git-annex.branchable.com/upgrades/insecure_embedded_creds/
* assistant: Detect when repository has been deleted or moved, and
automatically shut down the assistant. Closes: #761261
* Windows: Avoid crashing trying to list gpg secret keys, for gcrypt
which is not yet supported on Windows.
* WebDav: Fix enableremote crash when the remote already exists.
(Bug introduced in version 5.20140817.)
* add: In direct mode, adding an annex symlink will check it into git,
as was already done in indirect mode.
-- Joey Hess <joeyh@debian.org> Fri, 19 Sep 2014 12:53:42 -0400
git-annex (5.20140915) unstable; urgency=medium
* New annex.hardlink setting. Closes: #758593
* init: Automatically detect when a repository was cloned with --shared,
and set annex.hardlink=true, as well as marking the repository as
untrusted.
* Fix parsing of ipv6 address in git remote address when it was not
formatted as an url.
* The annex-rsync-transport configuration is now also used when checking
if a key is present on a rsync remote, and when dropping a key from
the remote.
* Promote file not found warning message to an error.
* Fix transfer lock file FD leak that could occur when two separate
git-annex processes were both working to perform the same set of
transfers.
* sync: Ensure that pending changes to git-annex branch are committed
before push when in direct mode. (Fixing a very minor reversion.)
* WORM backend: Switched to include the relative path to the file inside
the repository, rather than just the file's base name. Note that if you're
relying on such things to keep files separate with WORM, you should really
be using a better backend.
* Rather than crashing when there's a problem with the requested bloomfilter
capacity/accuracy, fall back to a reasonable default bloom filter size.
* Fix build with optparse-applicative 0.10. Closes: #761484
* webapp: Fixed visual glitch in xmpp pairing that was reported live by a
user who tracked me down in front of a coffee cart in Portland.
(New bug reporting method of choice?)
-- Joey Hess <joeyh@debian.org> Mon, 15 Sep 2014 10:45:00 -0400
git-annex (5.20140831) unstable; urgency=medium
* Make --help work when not in a git repository. Closes: #758592
* Ensure that all lock fds are close-on-exec, fixing various problems with
them being inherited by child processes such as git commands.
* When accessing a local remote, shut down git-cat-file processes
afterwards, to ensure that remotes on removable media can be unmounted.
Closes: #758630
* Fix handing of autocorrection when running outside a git repository.
* Fix stub git-annex test support when built without tasty.
* Do not preserve permissions and acls when copying files from
one local git repository to another. Timestamps are still preserved
as long as cp --preserve=timestamps is supported. Closes: #729757
-- Joey Hess <joeyh@debian.org> Sun, 31 Aug 2014 12:30:08 -0700
git-annex (5.20140817) unstable; urgency=medium
* New chunk= option to chunk files stored in special remotes.
Supported by: directory, S3, webdav, gcrypt, rsync, and all external
and hook special remotes.
* Partially transferred files are automatically resumed when using
chunked remotes!
* The old chunksize= option is deprecated. Do not use for new remotes.
* Legacy code for directory remotes using the old chunksize= option
will keep them working, but more slowly than before.
* webapp: Automatically install Konqueror integration scripts
to get and drop files.
* repair: Removing bad objects could leave fsck finding no more
unreachable objects, but some branches no longer accessible.
Fix this, including support for fixing up repositories that
were incompletely repaired before.
* Fix cost calculation for non-encrypted remotes.
* Display exception message when a transfer fails due to an exception.
* WebDAV: Sped up by avoiding making multiple http connections
when storing a file.
* WebDAV: Avoid buffering whole file in memory when uploading and
downloading.
* WebDAV: Dropped support for DAV before 1.0.
* testremote: New command to test uploads/downloads to a remote.
* Dropping an object from a bup special remote now deletes the git branch
for the object, although of course the object's content cannot be deleted
due to the nature of bup.
* unlock: Better error handling; continue past files that are not available
or cannot be unlocked due to disk space, and try all specified files.
* Windows: Now uses actual inode equivilants in new direct mode
repositories, for safer detection of eg, renaming of files with the same
size and mtime.
* direct: Fix ugly warning messages.
* WORM backend: When adding a file in a subdirectory, avoid including the
subdirectory in the key name.
* S3, Glacier, WebDAV: Fix bug that prevented accessing the creds
when the repository was configured with encryption=shared embedcreds=yes.
* direct: Avoid leaving file content in misctemp if interrupted.
* git-annex-shell sendkey: Don't fail if a remote asks for a key to be sent
that already has a transfer lock file indicating it's being sent to that
remote. The remote may have moved between networks, or reconnected.
* Switched from the old haskell HTTP library to http-conduit.
-- Joey Hess <joeyh@debian.org> Sun, 17 Aug 2014 10:30:58 -0400
git-annex (5.20140717) unstable; urgency=high
* Fix minor FD leak in journal code. Closes: #754608
* direct: Fix handling of case where a work tree subdirectory cannot
be written to due to permissions.
* migrate: Avoid re-checksumming when migrating from hashE to hash backend.
* uninit: Avoid failing final removal in some direct mode repositories
due to file modes.
* S3: Deal with AWS ACL configurations that do not allow creating or
checking the location of a bucket, but only reading and writing content to
it.
* resolvemerge: New plumbing command that runs the automatic merge conflict
resolver.
* Deal with change in git 2.0 that made indirect mode merge conflict
resolution leave behind old files.
* sync: Fix git sync with local git remotes even when they don't have an
annex.uuid set. (The assistant already did so.)
* Set gcrypt-publish-participants when setting up a gcrypt repository,
to avoid unncessary passphrase prompts.
This is a security/usability tradeoff. To avoid exposing the gpg key
ids who can decrypt the repository, users can unset
gcrypt-publish-participants.
* Install nautilus hooks even when ~/.local/share/nautilus/ does not yet
exist, since it is not automatically created for Gnome 3 users.
* Windows: Move .vbs files out of git\bin, to avoid that being in the
PATH, which caused some weird breakage. (Thanks, divB)
* Windows: Fix locking issue that prevented the webapp starting
(since 5.20140707).
-- Joey Hess <joeyh@debian.org> Thu, 17 Jul 2014 11:27:25 -0400
git-annex (5.20140709) unstable; urgency=medium
* Fix race in direct mode merge code that could cause all files in the
repository to be removed. It should be able to recover repositories
experiencing this bug without data loss. See:
http://git-annex.branchable.com/bugs/bad_merge_commit_deleting_all_files/
* Fix git version that supported --no-gpg-sign.
* Fix bug in automatic merge conflict resolution, when one side is an
annexed symlink, and the other side is a non-annexed symlink.
* Really fix bug that caused the assistant to make many unncessary
empty merge commits.
-- Joey Hess <joeyh@debian.org> Wed, 09 Jul 2014 15:28:03 -0400
git-annex (5.20140707) unstable; urgency=medium
* assistant: Fix bug, introduced in last release, that caused the assistant
to make many unncessary empty merge commits.
* assistant: Fix one-way assistant->assistant sync in direct mode.
* Fix bug in annex.queuesize calculation that caused much more
queue flushing than necessary.
* importfeed: When annex.genmetadata is set, metadata from the feed
is added to files that are imported from it.
* Support users who have set commit.gpgsign, by disabling gpg signatures
for git-annex branch commits and commits made by the assistant.
* Fix memory leak when committing millions of changes to the git-annex
branch, eg after git-annex add has run on 2 million files in one go.
* Support building with bloomfilter 2.0.0.
* Run standalone install process when the assistant is started
(was only being run when the webapp was opened).
* Android: patch git to avoid fchmod, which fails on /sdcard.
* Windows: Got rid of that pesky DOS box when starting the webapp.
* Windows: Added Startup menu item so assistant starts automatically
on login.
* Windows: Fix opening file browser from webapp when repo is in a
directory with spaces.
* Windows: Assistant now logs to daemon.log.
-- Joey Hess <joeyh@debian.org> Mon, 07 Jul 2014 12:24:13 -0400
git-annex (5.20140613) unstable; urgency=medium
* Ignore setsid failures.
* Avoid leaving behind .tmp files when failing in some cases, including
importing files to a disk that is full.
* Avoid bad commits after interrupted direct mode sync (or merge).
* Fix build with wai 0.3.0.
* Deal with FAT's low resolution timestamps, which in combination with
Linux's caching of higher res timestamps while a FAT is mounted, caused
direct mode repositories on FAT to seem to have modified files after
they were unmounted and remounted.
* Windows: Fix opening webapp when repository is in a directory with
spaces in the path.
* Detect when Windows has lost its mind in a timezone change, and
automatically apply a delta to the timestamps it returns, to get back to
sane values.
-- Joey Hess <joeyh@debian.org> Fri, 13 Jun 2014 09:58:07 -0400
git-annex (5.20140606) unstable; urgency=medium
* webapp: When adding a new local repository, fix bug that caused its
group and preferred content to be set in the current repository,
even when not combining.
* webapp: Avoid stomping on existing description, group and
preferred content settings when enabling or combining with
an already existing remote.
* assistant: Make sanity checker tmp dir cleanup code more robust.
* unused: Avoid checking view branches for unused files.
* webapp: Include ssh port in mangled hostname.
* Windows: Fix bug introduced in last release that caused files
in the git-annex branch to have lines teminated with \r.
* Windows: Fix retrieving of files from local bare git repositories.
-- Joey Hess <joeyh@debian.org> Fri, 06 Jun 2014 12:54:06 -0400
git-annex (5.20140529) unstable; urgency=medium
* Fix encoding of data written to git-annex branch. Avoid truncating
unicode characters to 8 bits. Allow any encoding to be used, as with
filenames (but utf8 is the sane choice). Affects metadata and repository
descriptions, and preferred content expressions.
* assistant: When there are multiple remotes giving different ways
to access the same repository, honor remote cost settings and use
the cheapest available.
* webapp: More robust startup when annex directory is not a git repo.
* initremote/enableremote: Basic support for using with regular git remotes;
initremote stores the location of an already existing git remote,
and enableremote setups up a remote using its stored location.
* webapp: Support for enabling known git repositories on ssh servers.
The repository must have been added using initremote.
* webapp: When setting up a ssh remote, record it using initremote,
so that it can be easily enabled elsewhere.
* webapp: When setting up a ssh remote, if the user inputs ~/foo,
normalize that to foo, since it's in the home directory by default.
* Use exceptions in place of deprecated MonadCatchIO-transformers
Thanks, Ben Gamari.
* android: Run busybox install with -s, since some versions of Android
prohibit making hard links.
* Android webapp: Fix EvilSplicer bugs that mangled the css files,
preventing icons from displaying, and also slightly broke the js files.
-- Joey Hess <joeyh@debian.org> Thu, 29 May 2014 14:41:56 -0400
git-annex (5.20140517) unstable; urgency=medium
* webapp: Switched to bootstrap 3.
Thanks, Sören Brunk.
* Standalone builds now check gpg signatures before upgrading.
* Simplified repository description line format. The remote name,
if any, is always in square brackets after the description.
* assistant: Clean up stale tmp files on startup.
* webapp: Better ssh password prompting.
* Depend on git-remote-gcrypt 0.20130908-6. Older versions
fail when the assistant is run with no controlling tty.
* Added ddar special remote.
Thanks, Robie Basak.
* webapp: Fixed drag and drop to reorder the list of remotes.
* group: When no groups are specified to set, lists the current groups
of a repository.
* Add remote.$name.annex-shell configuration.
Thanks, Fraser Tweedale
* Support symlinking git-annex and git-annex-shell
from the Linux standalone bundle into PATH.
Thanks, jlebar.
-- Joey Hess <joeyh@debian.org> Sat, 17 May 2014 13:30:39 -0400
git-annex (5.20140421) unstable; urgency=medium
* assistant: Now detects immediately when other repositories push
changes to a ssh remote, and pulls.
** XMPP is no longer needed in this configuration! **
This requires the remote server have git-annex-shell with
notifychanges support (>= 5.20140405)
* webapp: Show a network signal icon next to ssh and xmpp remotes that
it's currently connected with.
* webapp: Rework xmpp nudge to prompt for either xmpp or a ssh remote
to be set up.
* sync, assistant, remotedaemon: Use ssh connection caching for git pushes
and pulls.
* remotedaemon: When network connection is lost, close all cached ssh
connections.
* Improve handling of monthly/yearly scheduling.
* Avoid depending on shakespeare except for when building the webapp.
* uninit: Avoid making unncessary copies of files.
* info: Allow use in a repository where annex.uuid is not set.
* reinit: New command that can initialize a new repository using
the configuration of a previously known repository.
Useful if a repository got deleted and you want
to clone it back the way it was.
* drop --from: When local repository is untrusted, its copy of a file does
not count.
* Bring back rsync -p, but only when git-annex is running on a non-crippled
file system. This is a better approach to fix #700282 while not
unncessarily losing file permissions on non-crippled systems.
* webapp: Start even if the current directory is listed in
~/.config/git-annex/autostart but no longer has a git repository in it.
* findref: New command, like find but shows files in a specified git ref.
* webapp: Fix UI for removing XMPP connection.
* When init detects that git is not configured to commit, and sets
user.email to work around the problem, also make it set user.name.
* webapp: Support using git-annex on a remote server, which was installed
from the standalone tarball or OSX app, and so does not have
git-annex in PATH (and may also not have git or rsync in PATH).
* standalone tarball, OSX app: Install a ~/.ssh/git-annex-wrapper, which
can be used to run git-annex, git, rsync, etc.
-- Joey Hess <joeyh@debian.org> Sun, 20 Apr 2014 19:43:14 -0400
git-annex (5.20140412) unstable; urgency=high
* Last release didn't quite fix the high cpu issue in all cases, this should.
-- Joey Hess <joeyh@debian.org> Fri, 11 Apr 2014 17:14:38 -0400
git-annex (5.20140411) unstable; urgency=high
* importfeed: Filename template can now contain an itempubdate variable.
Needs feed 0.3.9.2.
* Fix rsync progress parsing in locales that use comma in number display.
Closes: #744148
* assistant: Fix high CPU usage triggered when a monthly fsck is scheduled,
and the last time the job ran was a day of the month > 12. This caused a
runaway loop. Thanks to Anarcat for his assistance, and to Maximiliano
Curia for identifying the cause of this bug.
* Remove wget from OSX dmg, due to issues with cert paths that broke
git-annex automatic upgrading. Instead, curl is used, unless the
OSX system has wget installed, which will then be used.
-- Joey Hess <joeyh@debian.org> Fri, 11 Apr 2014 14:59:49 -0400
git-annex (5.20140405) unstable; urgency=medium
* git-annex-shell: Added notifychanges command.
* Improve display of dbus notifications. Thanks, Johan Kiviniemi.
* Fix nautilus script installation to not crash when the nautilus script dir
does not exist. Instead, only install scripts when the directory already
exists.
-- Joey Hess <joeyh@debian.org> Sat, 05 Apr 2014 16:54:33 -0400
git-annex (5.20140402) unstable; urgency=medium
* unannex, uninit: Avoid committing after every file is unannexed,
for massive speedup.
* --notify-finish switch will cause desktop notifications after each
file upload/download/drop completes
(using the dbus Desktop Notifications Specification)
* --notify-start switch will show desktop notifications when each
file upload/download starts.
* webapp: Automatically install Nautilus integration scripts
to get and drop files.
* tahoe: Pass -d parameter before subcommand; putting it after
the subcommand no longer works with tahoe-lafs version 1.10.
(Thanks, Alberto Berti)
* forget --drop-dead: Avoid removing the dead remote from the trust.log,
so that if git remotes for it still exist anywhere, git annex info
will still know it's dead and not show it.
* git-annex-shell: Make configlist automatically initialize
a remote git repository, as long as a git-annex branch has
been pushed to it, to simplify setup of remote git repositories,
including via gitolite.
* add --include-dotfiles: New option, perhaps useful for backups.
* Version 5.20140227 broke creation of glacier repositories,
not including the datacenter and vault in their configuration.
This bug is fixed, but glacier repositories set up with the broken
version of git-annex need to have the datacenter and vault set
in order to be usable. This can be done using git annex enableremote
to add the missing settings. For details, see
http://git-annex.branchable.com/bugs/problems_with_glacier/
* Added required content configuration.
* assistant: Improve ssh authorized keys line generated in local pairing
or for a remote ssh server to set environment variables in an
alternative way that works with the non-POSIX fish shell, as well
as POSIX shells.
-- Joey Hess <joeyh@debian.org> Wed, 02 Apr 2014 16:42:53 -0400
git-annex (5.20140320) unstable; urgency=medium
* Fix zombie leak and general inneficiency when copying files to a
local git repo.
* Fix ssh connection caching stop method to work with openssh 6.5p1,
which broke the old method.
* webapp: Added a "Sync now" item to each repository's menu.
* webapp: Use securemem for constant time auth token comparisons.
* copy --fast --to remote: Avoid printing anything for files that
are already believed to be present on the remote.
* Commands that allow specifying which repository to act on using
the repository's description will now fail when multiple repositories
match, rather than picking a repository at random.
(So will --in=)
* Better workaround for problem umasks when eg, setting up ssh keys.
* "standard" can now be used as a first-class keyword in preferred content
expressions. For example "standard or (include=otherdir/*)"
* groupwanted can be used in preferred content expressions.
* vicfg: Allows editing preferred content expressions for groups.
* Improve behavior when unable to parse a preferred content expression
(thanks, ion).
* metadata: Add --get
* metadata: Support --key option (and some other ones like --all)
* For each metadata field, there's now an automatically maintained
"$field-lastchanged" that gives the date of the last change to that
field. Also the "lastchanged" field for the date of the last change
to any of a file's metadata.
* unused: In direct mode, files that are deleted from the work tree
and so have no content present are no longer incorrectly detected as
unused.
* Avoid encoding errors when using the unused log file.
* map: Fix crash when one of the remotes of a repo is a local directory
that does not exist, or is not a git repo.
* repair: Improve memory usage when git fsck finds a great many broken
objects.
* Windows: Fix some filename encoding bugs.
* rsync special remote: Fix slashes when used on Windows.
-- Joey Hess <joeyh@debian.org> Thu, 20 Mar 2014 13:21:12 -0400
git-annex (5.20140306) unstable; urgency=high
* sync: Fix bug in direct mode that caused a file that was not
checked into git to be deleted when there was a conflicting
merge with a remote.
* webapp: Now supports HTTPS.
* webapp: No longer supports a port specified after --listen, since
it was buggy, and that use case is better supported by setting up HTTPS.
* annex.listen can be configured, instead of using --listen
* annex.startupscan can be set to false to disable the assistant's startup
scan.
* Probe for quvi version at run time.
* webapp: Filter out from Switch Repository list any
repositories listed in autostart file that don't have a
git directory anymore. (Or are bare)
* webapp: Refuse to start in a bare git repository.
* assistant --autostart: Refuse to start in a bare git repository.
* webapp: Don't list the public repository group when editing a
git repository; it only makes sense for special remotes.
* view, vfilter: Add support for filtering tags and values out of a view,
using !tag and field!=value.
* vadd: Allow listing multiple desired values for a field.
* view: Refuse to enter a view when no branch is currently checked out.
* metadata: To only set a field when it's not already got a value, use
-s field?=value
* Run .git/hooks/pre-commit-annex whenever a commit is made.
* sync: Automatically resolve merge conflict between and annexed file
and a regular git file.
* glacier: Pass --region to glacier checkpresent.
* webdav: When built with a new enough haskell DAV (0.6), disable
the http response timeout, which was only 5 seconds.
* webapp: Include no-pty in ssh authorized_keys lines.
* assistant: Smarter log file rotation, which takes free disk space
into account.
-- Joey Hess <joeyh@debian.org> Thu, 06 Mar 2014 12:28:04 -0400
git-annex (5.20140227) unstable; urgency=medium
* metadata: Field names limited to alphanumerics and a few whitelisted
punctuation characters to avoid issues with views, etc.
* metadata: Field names are now case insensative.
* When constructing views, metadata is available about the location of the
file in the view's reference branch. Allows incorporating parts of the
directory hierarchy in a view.
For example `git annex view tag=* podcasts/=*` makes a view in the form
tag/showname.
* --metadata field=value can now use globs to match, and matches
case insensatively, the same as git annex view field=value does.
* annex.genmetadata can be set to make git-annex automatically set
metadata (year and month) when adding files.
* Make annex.web-options be used in several places that call curl.
* Fix handling of rsync remote urls containing a username,
including rsync.net.
* Preserve metadata when staging a new version of an annexed file.
* metadata: Support --json
* webapp: Fix creation of box.com and Amazon S3 and Glacier
repositories, broken in 5.20140221.
* webdav: When built with DAV 0.6.0, use the new DAV monad to avoid
locking files, which is not needed by git-annex's use of webdav, and
does not work on Box.com.
* webdav: Fix path separator bug when used on Windows.
* repair: Optimise unpacking of pack files, and avoid repeated error
messages about corrupt pack files.
* Add build dep on regex-compat to fix build on mipsel, which lacks
regex-tdfa.
* Disable test suite on sparc, which is missing optparse-applicative.
* Put non-object tmp files in .git/annex/misctmp, leaving .git/annex/tmp
for only partially transferred objects.
-- Joey Hess <joeyh@debian.org> Thu, 27 Feb 2014 11:34:19 -0400
git-annex (5.20140221) unstable; urgency=medium
* metadata: New command that can attach metadata to files.
* --metadata can be used to limit commands to acting on files
that have particular metadata.
* Preferred content expressions can use metadata=field=value
to limit them to acting on files that have particular metadata.
* view: New command that creates and checks out a branch that provides
a structured view of selected metadata.
* vfilter, vadd, vpop, vcycle: New commands for operating within views.
* pre-commit: Update metadata when committing changes to locations
of annexed files within a view.
* Add progress display for transfers to/from external special remotes.
* unused: Fix to actually detect unused keys when in direct mode.
* fsck: When run with --all or --unused, while .gitattributes
annex.numcopies cannot be honored since it's operating on keys
instead of files, make it honor the global numcopies setting,
and the annex.numcopies git config setting.
* trust, untrust, semitrust, dead: Warn when the trust level is
overridden in .git/config.
* glacier: Do not try to run glacier value create when an existing glacier
remote is enabled.
* fsck: Refuse to do anything if more than one of --incremental, --more,
and --incremental-schedule are given, since it's not clear which option
should win.
* Windows webapp: Can set up box.com, Amazon S3, and rsync.net remotes
* Windows webapp: Can create repos on removable drives.
* Windows: Ensure HOME is set, as needed by bundled cygwin utilities.
-- Joey Hess <joeyh@debian.org> Fri, 21 Feb 2014 11:23:59 -0400
git-annex (5.20140210) unstable; urgency=medium
* --in can now refer to files that were located in a repository at
some past date. For example, --in="here@{yesterday}"
* Fixed direct mode annexed content locking code, which is used to
guard against recursive file drops.
* This is the first beta-level release of the Windows port with important
fixes (see below).
(The webapp and assistant are still alpha-level on Windows.)
* sync --content: Honor annex-ignore configuration.
* sync: Don't try to sync with xmpp remotes, which are only currently
supported when using the assistant.
* sync --content: Re-pull from remotes after downloading content,
since that can take a while and other changes may be pushed in the
meantime.
* sync --content: Reuse smart copy code from copy command, including
handling and repairing out of date location tracking info.
Closes: #737480
* sync --content: Drop files from remotes that don't want them after
getting them.
* sync: Fix bug in automatic merge conflict resolution code when used
on a filesystem not supporting symlinks, which resulted in it losing
track of the symlink bit of annexed files.
* Added ways to configure rsync options to be used only when uploading
or downloading from a remote. Useful to eg limit upload bandwidth.
* Fix initremote with encryption=pubkey to work with S3, glacier, webdav,
and external special remotes.
* Avoid building with DAV 0.6 which is badly broken (see #737902).
* Fix dropping of unused keys with spaces in their name.
* Fix build on platforms not supporting the webapp.
* Document in man page that sshcaching uses ssh ControlMaster.
Closes: #737476
* Windows: It's now safe to run multiple git-annex processes concurrently
on Windows; the lock files have been sorted out.
* Windows: Avoid using unix-compat's rename, which refuses to rename
directories.
* Windows: Fix deletion of repositories by test suite and webapp.
* Windows: Test suite 100% passes again.
* Windows: Fix bug in symlink calculation code.
* Windows: Fix handling of absolute unix-style git repository paths.
* Android: Avoid crashing when unable to set file mode for ssh config file
due to Android filesystem horribleness.
-- Joey Hess <joeyh@debian.org> Mon, 10 Feb 2014 12:54:57 -0400
git-annex (5.20140127) unstable; urgency=medium
* sync --content: New option that makes the content of annexed files be
transferred. Similar to the assistant, this honors any configured
preferred content expressions.
* Remove --json option from commands not supporting it.
* status: Support --json.
* list: Fix specifying of files to list.
* Allow --all to be mixed with matching options like --copies and --in
(but not --include and --exclude).
* numcopies: New command, sets global numcopies value that is seen by all
clones of a repository.
* The annex.numcopies git config setting is deprecated. Once the numcopies
command is used to set the global number of copies, any annex.numcopies
git configs will be ignored.
* assistant: Make the prefs page set the global numcopies.
* Add lackingcopies, approxlackingcopies, and unused to
preferred content expressions.
* Client, transfer, incremental backup, and archive repositories
now want to get content that does not yet have enough copies.
* Client, transfer, and source repositories now do not want to retain
unused file contents.
* assistant: Checks daily for unused file contents, and when possible
moves them to a repository (such as a backup repository) that
wants to retain them.
* assistant: annex.expireunused can be configured to cause unused
file contents to be deleted after some period of time.
* webapp: Nudge user to see if they want to expire old unused file
contents when a lot of them seem to be piling up in the repository.
* repair: Check git version at run time.
* assistant: Run the periodic git gc in batch mode.
* added annex.secure-erase-command config option.
* test suite: Use tasty-rerun, and expose tasty command-line options.
* Optimise non-bare http remotes; no longer does a 404 to the wrong
url every time before trying the right url. Needs annex-bare to be
set to false, which is done when initially probing the uuid of a
http remote.
* webapp: After upgrading a git repository to git-annex, fix
bug that made it temporarily not be synced with.
* whereis: Support --all.
* All commands that support --all also support a --key option,
which limits them to acting on a single key.
-- Joey Hess <joeyh@debian.org> Mon, 27 Jan 2014 13:43:28 -0400
git-annex (5.20140117) unstable; urgency=medium
* Really fix FTBFS on mipsel and sparc due to test suite not being available
on those architectures.
-- Joey Hess <joeyh@debian.org> Fri, 17 Jan 2014 14:46:27 -0400
git-annex (5.20140116) unstable; urgency=medium
* Added tahoe special remote.
* external special remote protocol: Added GETGITDIR, and GETAVAILABILITY.
* Refuse to build with git older than 1.7.1.1, which is needed for
git checkout -B
* map: Fix display of v5 direct mode repos.
* repair: Support old git versions from before git fsck --no-dangling was
implemented.
* Fix a long-standing bug that could cause the wrong index file to be used
when committing to the git-annex branch, if GIT_INDEX_FILE is set in the
environment. This typically resulted in git-annex branch log files being
committed to the master branch and later showing up in the work tree.
(These log files can be safely removed.)
* assistant: Detect if .git/annex/index is corrupt at startup, and
recover.
* repair: Fix bug in packed refs file exploding code that caused a .gitrefs
directory to be created instead of .git/refs
* Fix FTBFS on mipsel and sparc due to test suite not being available
on those architectures.
* Android: Avoid passing --clobber to busybox wget.
-- Joey Hess <joeyh@debian.org> Thu, 16 Jan 2014 11:34:54 -0400
git-annex (5.20140107) unstable; urgency=medium
* mirror: Support --all (and --unused).
* external special remote protocol: Added GETUUID, GETWANTED, SETWANTED,
SETSTATE, GETSTATE, DEBUG.
* Windows: Fix bug in direct mode merge code that could cause files
in subdirectories to go missing.
* Windows: Avoid eating stdin when running ssh to add a authorized key,
since this is used for password prompting.
* Avoid looping if long-running git cat-file or git hash-object crashes
and keeps crashing when restarted.
* Assistant: Remove stale MERGE_HEAD files in lockfile cleanup.
* Remotes can now be made read-only, by setting remote.<name>.annex-readonly
* wanted, schedule: Avoid printing "ok" after requested value.
* assistant: Ensure that .ssh/config and .ssh/authorized_keys are not
group or world writable when writing to those files, as that can make
ssh refuse to use them, if it allows another user to write to them.
* addurl, importfeed: Honor annex.diskreserve as long as the size of the
url can be checked.
* add: Fix rollback when disk is completely full.
* assistant: Fixed several minor memory leaks that manifested when
adding a large number of files.
* assistant: Start a new git-annex transferkeys process
after a network connection change, so that remotes that use a persistent
network connection are restarted.
* Adjust Debian build deps to match current state of sparc, mipsel.
-- Joey Hess <joeyh@debian.org> Tue, 07 Jan 2014 12:22:18 -0400
git-annex (5.20131230) unstable; urgency=medium
* Added new external special remote interface.
* importfeed: Support youtube playlists.
* Add tasty to build-depends, so that test suite builds again.
(tasty was stuck in incoming.)
* Fix typo in test suite.
* Fix bug in Linux standalone build's shimming that broke git-annex-shell.
* Include git-receive-pack, git-upload-pack, git, and git-shell wrappers
in the Linux standalone build, and OSX app, so they will be available
when it's added to PATH.
* addurl, importfeed: Sanitize | and some other symbols and special
characters.
* Auto-upgrade v3 indirect repos to v5 with no changes.
This also fixes a problem when a direct mode repo was somehow set to v3
rather than v4, and so the automatic direct mode upgrade to v5 was not
done.
* Android: Avoid trying to use Android's own ionice, which does not
allow specifying a command to run. Fixes transferring files to/from
android and probably a few other things.
-- Joey Hess <joeyh@debian.org> Mon, 30 Dec 2013 14:13:40 -0400
git-annex (5.20131221) unstable; urgency=low
* assistant: Fix OSX-specific bug that caused the startup scan to try to
follow symlinks to other directories, and add their contents to the annex.
* assistant: Set StrictHostKeyChecking yes when creating ssh remotes,
and add it to the configuration for any ssh remotes previously created
by the assistant. This avoids repeated prompts by ssh if the host key
changes, instead syncing with such a remote will fail. Closes: #732602
* Fix test suite to cover lock --force change.
* Add plumbing-level lookupkey and examinekey commands.
* find --format: Added hashdirlower, hashdirmixed, keyname, and mtime
format variables.
* assistant: Always batch changes found in startup scan.
* An armel Linux standalone build is now available, which includes the
webapp.
* Programs from Linux and OSX standalone builds can now be symlinked
into a directory in PATH as an alternative installation method, and will
use readlink to find where the build was unpacked.
* Include man pages in Linux and OSX standalone builds.
* Linux standalone build now includes its own glibc and forces the linker to
use it, to remove dependence on the host glibc.
-- Joey Hess <joeyh@debian.org> Sat, 21 Dec 2013 12:00:17 -0400
git-annex (5.20131213) unstable; urgency=low
* Avoid using git commit in direct mode, since in some situations
it will read the full contents of files in the tree.
* assistant: Batch jobs are now run with ionice and nocache, when
those commands are available.
* assistant: Run transferkeys as batch jobs.
* Automatically fix up bad bare repositories created by
versions 5.20131118 through 5.20131127.
* rsync special remote: Fix fallback mode for rsync remotes that
use hashDirMixed. Closes: #731142
* copy --from, get --from: When --force is used, ignore the
location log and always try to get the file from the remote.
* Deal with box.com changing the url of their webdav endpoint.
* Android: Fix SRV record lookups for XMPP to use android getprop
command to find DNS server, since there is no resolv.conf.
* import: Add --skip-duplicates option.
* lock: Require --force. Closes: #731606
* import: better handling of overwriting an existing file/directory/broken
link when importing
* Windows: assistant and webapp work! (very experimental)
* Windows: Support annex.diskreserve.
* Fix bad behavior in Firefox, which was caused by an earlier fix to
bad behavior in Chromium.
* repair: Improve repair of git-annex index file.
* repair: Remove damaged git-annex sync branches.
* status: Ignore new files that are gitignored.
* Fix direct mode's handling when modifications to non-annexed files
are pulled from a remote. A bug prevented the files from being updated
in the work tree, and this caused the modification to be reverted.
* OSX: Remove ssh and ssh-keygen from dmg as they're included in OSX by
default.
-- Joey Hess <joeyh@debian.org> Fri, 13 Dec 2013 14:20:32 -0400
git-annex (5.20131130) unstable; urgency=low
* init: Fix a bug that caused git annex init, when run in a bare
repository, to set core.bare=false.
-- Joey Hess <joeyh@debian.org> Sat, 30 Nov 2013 16:32:35 -0400
git-annex (5.20131127.1) unstable; urgency=low
* Rebuild that does not try to use quvi 0.9 from experimental.
-- Joey Hess <joeyh@debian.org> Thu, 28 Nov 2013 07:57:36 -0400
git-annex (5.20131127) unstable; urgency=low
* webapp: Detect when upgrades are available, and upgrade if the user
desires.
(Only when git-annex is installed using the prebuilt binaries
from git-annex upstream, not from eg Debian.)
* assistant: Detect when the git-annex binary is modified or replaced,
and either prompt the user to restart the program, or automatically
restart it.
* annex.autoupgrade configures both the above upgrade behaviors.
* Added support for quvi 0.9. Slightly suboptimal due to limitations in its
interface compared with the old version.
* Bug fix: annex.version did not get set on automatic upgrade to v5 direct
mode repo, so the upgrade was performed repeatedly, slowing commands down.
* webapp: Fix bug that broke switching between local repositories
that use the new guarded direct mode.
* Android: Fix stripping of the git-annex binary.
* Android: Make terminal app show git-annex version number.
* Android: Re-enable XMPP support.
* reinject: Allow to be used in direct mode.
* Futher improvements to git repo repair. Has now been tested in tens
of thousands of intentionally damaged repos, and successfully
repaired them all.
* Allow use of --unused in bare repository.
-- Joey Hess <joeyh@debian.org> Wed, 27 Nov 2013 18:41:44 -0400
git-annex (5.20131120) unstable; urgency=low
* Fix Debian package to not try to run test suite, since haskell-tasty
is not out of new or in Build-Depends yet.
* dropunused, addunused: Allow "all" instead of a range to
act on all unused data.
* Ensure execute bit is set on directories when core.sharedrepository is set.
* Ensure that core.sharedrepository is honored when creating the .git/annex
directory.
* Improve repair code in the case where the index file is corrupt,
and this hides other problems from git fsck.
-- Joey Hess <joeyh@debian.org> Wed, 20 Nov 2013 12:54:18 -0400
git-annex (5.20131118) unstable; urgency=low
* Direct mode repositories now have core.bare=true set, to prevent
accidentally running git commands that try to operate on the work tree,
and so do the wrong thing in direct mode.
* annex.version is now set to 5 for direct mode repositories.
This upgrade is handled fully automatically, no need to run
git annex upgrade
* The "status" command has been renamed to "info", to allow
"git annex status" to be used in direct mode repositories, now that
"git status" won't work in them.
* The -c option now not only modifies the git configuration seen by
git-annex, but it is passed along to every git command git-annex runs.
* watcher: Avoid loop when adding a file owned by someone else fails
in indirect mode because its permissions cannot be modified.
* webapp: Avoid encoding problems when displaying the daemon log file.
* webapp: Improve UI around remote that have no annex.uuid set,
either because setup of them is incomplete, or because the remote
git repository is not a git-annex repository.
* Include ssh-keygen in standalone bundle.
* Allow optionally configuring git-annex with -fEKG to enable awesome
remote monitoring interfaceat http://localhost:4242/
* Fix bug that caused bad information to be written to the git-annex branch
when running describe or other commands with a remote that has no uuid.
* Work around Android linker problem that had prevented git-annex from
running on Android 4.3 and 4.4.
* repair: Handle case where index file is corrupt, but all objects are ok.
* assistant: Notice on startup when the index file is corrupt, and
auto-repair.
* Fix direct mode merge bug when a direct mode file was deleted and replaced
with a directory. An ordering problem caused the directory to not get
created in this case.
Thanks to Tim for the test case.
* Direct mode .git/annex/objects directories are no longer left writable,
because that allowed writing to symlinks of files that are not present,
which followed the link and put bad content in an object location.
Thanks to Tim for the test case.
* fsck: Fix up .git/annex/object directory permissions.
* Switched to the tasty test framework.
* Android: Adjust default .gitignore to ignore .thumbnails at any location
in the tree, not just at its top.
* webapp: Check annex.version.
-- Joey Hess <joeyh@debian.org> Mon, 18 Nov 2013 10:45:43 -0400
git-annex (4.20131106) unstable; urgency=low
* Improve local pairing behavior when two computers both try to start
the pairing process separately.
* sync: Work even when the local git repository is new and empty,
with no master branch.
* gcrypt, bup: Fix bug that prevented using these special remotes
with encryption=pubkey.
* Fix enabling of gcrypt repository accessed over ssh;
git-annex-shell gcryptsetup had a bug that caused it to fail
with permission denied.
* Fix zombie process that occurred when switching between repository
views in the webapp.
* map: Work when there are gcrypt remotes.
* Fix build w/o webapp.
* Fix exception handling bug that could cause .git/annex/index to be used
for git commits outside the git-annex branch. Known to affect git-annex
when used with the git shipped with Ubuntu 13.10.
-- Joey Hess <joeyh@debian.org> Wed, 06 Nov 2013 11:17:47 -0400
git-annex (4.20131101) unstable; urgency=low
* The "git annex content" command is renamed to "git annex wanted".
* New --want-get and --want-drop options which can be used to
test preferred content settings.
For example, "git annex find --in . --want-drop"
* assistant: When autostarted, wait 5 seconds before running the startup
scan, to avoid contending with the user's desktop login process.
* webapp: When setting up a bare shared repository, enable non-fast-forward
pushes.
* sync: Show a hint about receive.denyNonFastForwards when a push fails.
* directory, webdav: Fix bug introduced in version 4.20131002 that
caused the chunkcount file to not be written. Work around repositories
without such a file, so files can still be retreived from them.
* assistant: Automatically repair damanged git repository, if it can
be done without losing data.
* assistant: Support repairing git remotes that are locally accessible
(eg, on removable drives).
* add: Fix reversion in 4.20130827 when adding unlocked files that have
not yet been committed.
* unannex: New, much slower, but more safe behavior: Copies files out of
the annex. This avoids an unannex of one file breaking other files that
link to the same content. Also, it means that the content
remains in the annex using up space until cleaned up with
"git annex unused".
(The behavior of unannex --fast has not changed; it still hard links
to content in the annex. --fast was not made the default because it is
potentially unsafe; editing such a hard linked file can unexpectedly
change content stored in the annex.)
-- Joey Hess <joeyh@debian.org> Fri, 01 Nov 2013 11:34:27 -0400
git-annex (4.20131024) unstable; urgency=low
* webapp: Fix bug when adding a remote and git-remote-gcrypt
is not installed.
* The assitant can now run scheduled incremental fsck jobs on the local
repository and remotes. These can be configured using vicfg or with the
webapp.
* repair: New command, which can repair damaged git repositories
(even ones not using git-annex).
* webapp: When git repository damange is detected, repairs can be
done using the webapp UI.
* Automatically and safely detect and recover from dangling
.git/annex/index.lock files, which would prevent git from
committing to the git-annex branch, eg after a crash.
* assistant: Detect stale git lock files at startup time, and remove them.
* addurl: Better sanitization of generated filenames.
* Better sanitization of problem characters when generating URL and WORM
keys.
* The control socket path passed to ssh needs to be 17 characters
shorter than the maximum unix domain socket length, because ssh
appends stuff to it to make a temporary filename. Closes: #725512
* status: Fix space leak in local mode, introduced in version 4.20130920.
* import: Skip .git directories.
* Remove bogus runshell loop check.
* addurl: Improve message when adding url with wrong size to existing file.
* Fixed handling of URL keys that have no recorded size.
* status: Fix a crash if a temp file went away while its size was
being checked for status.
* Deal with git check-attr -z output format change in git 1.8.5.
* Work around sed output difference that led to version containing a newline
on OSX.
* sync: Fix automatic resolution of merge conflicts where one side is an
annexed file, and the other side is a non-annexed file, or a directory.
* S3: Try to ensure bucket name is valid for archive.org.
* assistant: Bug fix: When run in a subdirectory, files from incoming merges
were wrongly added to that subdirectory, and removed from their original
locations.
* Windows: Deal with strange msysgit 1.8.4 behavior of not understanding
DOS formatted paths for --git-dir and --work-tree.
* Removed workaround for bug in git 1.8.4r0.
* Added git-recover-repository command to git-annex source
(not built by default; this needs to move to someplace else).
* webapp: Move sidebar to the right hand side of the screen.
-- Joey Hess <joeyh@debian.org> Thu, 24 Oct 2013 12:59:55 -0400
git-annex (4.20131002) unstable; urgency=low
* Note that the layout of gcrypt repositories has changed, and
if you created one you must manually upgrade it.
See http://git-annex.branchable.com/upgrades/gcrypt/
* webapp: Support setting up and using encrypted git repositories on
any ssh server, as well as on rsync.net.
* git-annex-shell: Added support for operating inside gcrypt repositories.
* Disable receive.denyNonFastForwards when setting up a gcrypt special
remote, since gcrypt needs to be able to fast-forward the master branch.
* import: Preserve top-level directory structure.
* Use cryptohash rather than SHA for hashing when no external hash program
is available. This is a significant speedup for SHA256 on OSX, for
example.
* Added SKEIN256 and SKEIN512 backends.
* Android build redone from scratch, many dependencies updated,
and entire build can now be done using provided scripts.
* assistant: Clear the list of failed transfers when doing a full transfer
scan. This prevents repeated retries to download files that are not
available, or are not referenced by the current git tree.
* indirect, direct: Better behavior when a file is not owned by
the user running the conversion.
* add, import, assistant: Better preserve the mtime of symlinks,
when when adding content that gets deduplicated.
* Send a git-annex user-agent when downloading urls.
Overridable with --user-agent option.
(Not yet done for S3 or WebDAV due to limitations of libraries used.)
* webapp: Fixed a bug where when a new remote is added, one file
may fail to sync to or from it due to the transferrer process not
yet knowing about the new remote.
* OSX: Bundled gpg upgraded, now compatible with config files
written by MacGPG.
* assistant: More robust inotify handling; avoid crashing if a directory
cannot be read.
* Moved list of backends and remote types from status to version
command.
-- Joey Hess <joeyh@debian.org> Wed, 02 Oct 2013 16:00:39 -0400
git-annex (4.20130920) unstable; urgency=low
* webapp: Initial support for setting up encrypted removable drives.
* Recommend using my patched gcrypt, which fixes some bugs:
https://github.com/joeyh/git-remote-gcrypt
* Support hot-swapping of removable drives containing gcrypt repositories.
* list: New command, displays a compact table of remotes that
contain files.
(Thanks, anarcat for display code and mastensg for inspiration.)
* fsck: Fix detection and fixing of present direct mode files that are
wrongly represented as standin symlinks on crippled filesystems.
* sync: Fix bug that caused direct mode mappings to not be updated
when merging files into the tree on Windows.
* sync: Don't fail if the directory it is run in gets removed by the
sync.
* addurl: Fix quvi audodetection, broken in last release.
* status: In local mode, displays information about variance from configured
numcopies levels. (--fast avoids calculating these)
* gcrypt: Ensure that signing key is set to one of the participants keys.
* webapp: Show encryption information when editing a remote.
* Avoid unnecessarily catting non-symlink files from git, which can be
so large it runs out of memory.
-- Joey Hess <joeyh@debian.org> Fri, 20 Sep 2013 10:34:51 -0400
git-annex (4.20130911) unstable; urgency=low
* Fix problem with test suite in non-unicode locale.
-- Joey Hess <joeyh@debian.org> Wed, 11 Sep 2013 12:14:16 -0400
git-annex (4.20130909) unstable; urgency=low
* initremote: Syntax change when setting up an encrypted special remote.
Now use keyid=$KEYID rather than the old encryption=$KEYID
* forget: New command, causes git-annex branch history to be forgotten
in a way that will spread to other clones of the repository.
(As long as they're running this version or newer of git-annex.)
* forget --drop-dead: Completely removes mentions of repositories that
have been marked as dead from the git-annex branch.
* sync, assistant: Force push of the git-annex branch. Necessary
to ensure it gets pushed to remotes after being rewritten by forget.
* Added gcrypt support. This combines a fully encrypted git
repository (using git-remote-gcrypt) with an encrypted git-annex special
remote.
* sync: Support syncing with gcrypt remotes.
* importfeed: Also ignore transient problems with downloading content
from feeds.
* Honor core.sharedrepository when receiving and adding files in direct
mode.
* enableremote: gpg keys can be removed from those a remote encrypts
to by passing "keyid-=$KEYID". keyid+= is also provided.
(Thanks, guilhem for the patch.)
* Added encryption=pubkey scheme, which encrypts to public keys directly
rather than the hybrid approach. See documentation for advantages
and disadvantages, but encryption=hybrid is the recommended scheme still.
(Thanks, guilhem for the patch.)
* Fix Feeds display in build flags.
* Remind user when annex-ignore is set for some remotes, if unable to
get or drop a file, possibly because it's on an ignored remote.
* gpg: Force --no-textmode in case the user has it turned on in config.
* webapp: Improve javascript's handling of longpolling connection
failures, by reloading the current page in this case.
Works around chromium behavior where ajax connections to urls
that were already accessed are denied after navigating back to
a previous page.
* Allow building without quvi support.
-- Joey Hess <joeyh@debian.org> Mon, 09 Sep 2013 09:47:02 -0400
git-annex (4.20130827) unstable; urgency=low
* Youtube support! (And 53 other video hosts). When quvi is installed,
git-annex addurl automatically uses it to detect when an page is
a video, and downloads the video file.
* web special remote: Also support using quvi, for getting files,
or checking if files exist in the web.
* unused: Is now a minimum of 30 times faster, and typically many
more times than that (when a repository has several branches).
(Thanks, guilhem for the patch.)
* unused: Fix bugs in two edge cases involving manually staged changes.
(Thanks, guilhem for the patch.)
* Android: Fix bug in terminal app that caused it to spin using much
CPU and battery. This problem was introduced in version 4.20130601.
* sync, merge: Bug fix: Don't try to merge into master when in a bare repo.
* import: Add options to control handling of duplicate files:
--duplicate, --deduplicate, and --clean-duplicates
* mirror: New command, makes two repositories contain the same set of files.
* Set --clobber when running wget to ensure resuming works properly.
* Unescape characters in 'file://...' URIs. (Thanks, guilhem for the patch.)
* Better error message when trying to use a git remote that has annex.ignore
set.
* Fix bug that caused typechanged symlinks to be assumed to be unlocked
files, so they were added to the annex by the pre-commit hook.
* Debian: Run the builtin test suite as an autopkgtest.
* Debian: Recommend ssh-askpass, which ssh will use when the assistant
is run w/o a tty. Closes: #719832
-- Joey Hess <joeyh@debian.org> Tue, 27 Aug 2013 11:03:00 -0400
git-annex (4.20130815) unstable; urgency=low
* assistant, watcher: .gitignore files and other git ignores are now
honored, when git 1.8.4 or newer is installed.
(Thanks, Adam Spiers, for getting the necessary support into git for this.)
* importfeed: Ignores transient problems with feeds. Only exits nonzero
when a feed has repeatedly had a problems for at least 1 day.
* importfeed: Fix handling of dots in extensions.
* Windows: Added support for encrypted special remotes.
* Windows: Fixed permissions problem that prevented removing files
from directory special remote. Directory special remotes now fully usable.
-- Joey Hess <joeyh@debian.org> Thu, 15 Aug 2013 10:14:33 +0200
git-annex (4.20130802) unstable; urgency=low
* dropunused behavior change: Now refuses to drop the last copy of a
file, unless you use the --force.
This was the last place in git-annex that could remove data referred
to by the git history, without being forced.
Like drop, dropunused checks remotes, and honors the global
annex.numcopies setting. (However, .gitattributes settings cannot
apply to unused files.)
* Fix inverted logic in last release's fix for data loss bug,
that caused git-annex sync on FAT or other crippled filesystems to add
symlink standin files to the annex.
* importfeed can be used to import files from podcast feeds.
* webapp: When setting up a dedicated ssh key to access the annex
on a host, set IdentitiesOnly to prevent the ssh-agent from forcing
use of a different ssh key. That could result in unncessary password
prompts, or prevent git-annex-shell from being run on the remote host.
* webapp: Improve handling of remotes whose setup has stalled.
* Add status message to XMPP presence tag, to identify to others that
the client is a git-annex client. Closes: #717652
* webapp: When creating a repository on a removable drive, set
core.fsyncobjectfiles, to help prevent data loss when the drive is yanked.
* Always build with -threaded, to avoid a deadlock when communicating with
gpg.
* unused: No longer shows as unused tmp files that are actively being
transferred.
* assistant: Fix NetWatcher to not sync with remotes that have
remote.<name>.annex-sync set to false.
* assistant: Fix deadlock that could occur when adding a lot of files
at once in indirect mode.
* assistant: Fix bug that caused it to stall when adding a very large
number of files at once (around 5 thousand).
* OSX: Make git-annex-webapp run in the background, so that the app icon
can be clicked on the open a new webapp when the assistant is already
running.
* Improve test suite on Windows; now tests git annex sync.
* Fix a few bugs involving filenames that are at or near the filesystem's
maximum filename length limit.
* find: Avoid polluting stdout with progress messages. Closes: #718186
* Escape ':' in file/directory names to avoid it being treated
as a pathspec by some git commands. Closes: #718185
* Slow and ugly work around for bug #718517 in git 1.8.4~rc0, which broke
git-cat-file --batch for filenames containing spaces.
(Will be reverted after next git pre-release fixes the problem.)
-- Joey Hess <joeyh@debian.org> Fri, 02 Aug 2013 11:35:16 -0400
git-annex (4.20130723) unstable; urgency=low
* Fix data loss bug when adding an (uncompressed) tarball of a
git-annex repository, or other file that begins with something
that can be mistaken for a git-annex link. Closes: #717456
* New improved version of the git-annex logo, contributed by
John Lawrence.
* Rsync.net have committed to support git-annex and offer a special
discounted rate for git-annex users. Updated the webapp to reflect this.
http://www.rsync.net/products/git-annex-pricing.html
* Install XDG desktop icon files.
* Support unannex and uninit in direct mode.
* Support import in direct mode.
* webapp: Better display of added files.
* fix: Preserve the original mtime of fixed symlinks.
* uninit: Preserve .git/annex/objects at the end, if it still
has content, so that old versions of files and deleted files
are not deleted. Print a message with some suggested actions.
* When a transfer is already being run by another process,
proceed on to the next file, rather than dying.
* Fix checking when content is present in a non-bare repository
accessed via http.
* Display byte sizes with more precision.
* watcher: Fixed a crash that could occur when a directory was renamed
or deleted before it could be scanned.
* watcher: Partially worked around a bug in hinotify, no longer crashes
if hinotify cannot process a directory (but can't detect changes in it)
* directory special remote: Fix checking that there is enough disk space
to hold an object, was broken when using encryption.
* webapp: Differentiate between creating a new S3/Glacier/WebDav remote,
and initializing an existing remote. When creating a new remote, avoid
conflicts with other existing (or deleted) remotes with the same name.
* When an XMPP server has SRV records, try them, but don't then fall
back to the regular host if they all fail.
* For long hostnames, use a hash of the hostname to generate the socket
file for ssh connection caching.
-- Joey Hess <joeyh@debian.org> Tue, 23 Jul 2013 10:46:05 -0400
git-annex (4.20130709) unstable; urgency=low
* --all: New switch that makes git-annex operate on all data stored
in the git annex, including old versions of files. Supported by
fsck, get, move, copy.
* --unused: New switch that makes git-annex operate on all data found
by the last run of git annex unused. Supported by fsck, move, copy.
* get, move, copy: Can now be run in a bare repository,
like fsck already could. --all is enabled automatically in this case.
* merge: Now also merges synced/master or similar branches, which
makes it useful to put in a post-receive hook to make a repository
automatically update its working copy when git annex sync or the assistant
sync with it.
* webapp: Fix ssh setup with nonstandard port, broken in last release.
* init: Detect systems on which git commit fails due to not being able to
determine the FQDN, and put in a workaround so committing to the git-annex
branch works.
* addurl --pathdepth: Fix failure when the pathdepth specified is deeper
than the urls's path.
* Windows: Look for .exe extension when searching for a command in path.
* Pass -f to curl when downloading a file with it, so it propigates failure.
* Windows: Fix url to object when using a http remote.
* webapp: Fix authorized_keys line added when setting up a rsync remote
on a server that also supports git-annex, to not force running
git-annex-shell.
* OSX Mountain Lion: Fixed gpg bundled in dmg to not fail due to a missing
gpg-agent.
* Android: gpg is built without --enable-minimal, so it interoperates
better with other gpg builds that may default to using other algorithms
for encryption.
* dropunused, addunused: Complain when asked to operate on a number that
does not correspond to any unused key.
* fsck: Don't claim to fix direct mode when run on a symlink whose content
is not present.
* Make --numcopies override annex.numcopies set in .gitattributes.
-- Joey Hess <joeyh@debian.org> Tue, 09 Jul 2013 13:55:39 -0400
git-annex (4.20130627) unstable; urgency=low
* assistant --autostart: Automatically ionices the daemons it starts.
* assistant: Daily sanity check thread is run niced.
* bup: Handle /~/ in bup remote paths.
Thanks, Oliver Matthews
* fsck: Ensures that direct mode is used for files when it's enabled.
* webapp: Fix bug when setting up a remote ssh repo repeatedly on the same
server.
* webapp: Ensure that ssh keys generated for different directories
on a server are always different.
* webapp: Fix bug setting up ssh repo if the user enters "~/" at the start
of the path.
* assistant: Fix bug that prevented adding files written by gnucash,
and more generally support adding hard links to files. However,
other operations on hard links are still unsupported.
* webapp: Fix bug that caused the webapp to hang when built with yesod 1.2.
-- Joey Hess <joeyh@debian.org> Thu, 27 Jun 2013 14:21:55 -0400
git-annex (4.20130621) unstable; urgency=low
* Supports indirect mode on encfs in paranoia mode, and other
filesystems that do not support hard links, but do support
symlinks and other POSIX filesystem features.
* Android: Add .thumbnails to .gitignore when setting up a camera
repository.
* Android: Make the "Open webapp" menu item open the just created
repository when a new repo is made.
* webapp: When the user switches to display a different repository,
that repository becomes the default repository to be displayed next time
the webapp gets started.
* glacier: Better handling of the glacier inventory, which avoids
duplicate uploads to the same glacier repository by `git annex copy`.
* Direct mode: No longer temporarily remove write permission bit of files
when adding them.
* sync: Better support for bare git remotes. Now pushes directly to the
master branch on such a remote, instead of to synced/master. This
makes it easier to clone from a bare git remote that has been populated
with git annex sync or by the assistant.
* Android: Fix use of cp command to not try to use features present
only on build system.
* Windows: Fix hang when adding several files at once.
* assistant: In direct mode, objects are now only dropped when all
associated files are unwanted. This avoids a repreated drop/get loop
of a file that has a copy in an archive directory, and a copy not in an
archive directory. (Indirect mode still has some buggy behavior in this
area, since it does not keep track of associated files.)
Closes: #712060
* status: No longer shows dead repositories.
* annex.debug can now be set to enable debug logging by default.
The webapp's debugging check box does this.
* fsck: Avoid getting confused by Windows path separators
* Windows: Multiple bug fixes, including fixing the data written to the
git-annex branch.
* Windows: The test suite now passes on Windows (a few broken parts are
disabled).
* assistant: On Linux, the expensive transfer scan is run niced.
* Enable assistant and WebDAV support on powerpc and sparc architectures,
which now have the necessary dependencies built.
-- Joey Hess <joeyh@debian.org> Fri, 21 Jun 2013 10:18:41 -0400
git-annex (4.20130601) unstable; urgency=medium
* XMPP: Git push over xmpp made much more robust.
* XMPP: Avoid redundant and unncessary pushes. Note that this breaks
compatibility with previous versions of git-annex, which will refuse
to accept any XMPP pushes from this version.
* XMPP: Send pings and use them to detect when contact with the server
is lost.
* hook special remote: Added combined hook program support.
* Android app: Avoid using hard links to app's lib directory, which
is sometimes on a different filesystem than the data directory.
* Fix bug in parsing of parens in some preferred content expressions.
This fixes the behavior of the manual mode group.
* assistant: Work around git-cat-file's not reloading the index after files
are staged.
* Improve error handling when getting uuid of http remotes to auto-ignore,
like with ssh remotes.
* content: New command line way to view and configure a repository's
preferred content settings.
* sync: Fix double merge conflict resolution handling.
* XMPP: Fix a file descriptor leak.
* Android: Added an "Open WebApp" item to the terminal's menu.
* Android: Work around Android devices where the `am` command doesn't work.
* Can now restart certain long-running git processes if they crash, and
continue working.
-- Joey Hess <joeyh@debian.org> Sat, 01 Jun 2013 19:16:04 -0400
git-annex (4.20130521) unstable; urgency=low
* Sanitize debian changelog version before putting it into cabal file.
Closes: #708619
* Switch to MonadCatchIO-transformers for better handling of state while
catching exceptions.
* Fix a zombie that could result when running a process like gpg to
read and write to it.
* Allow building with gpg2.
* Disable building with the haskell threaded runtime when the webapp
is not built. This may fix builds on mips, s390x and sparc, which are
failing to link -lHSrts_thr
* Temporarily build without webapp on kfreebsd-i386, until yesod is
installable there again.
* Direct mode bug fix: After a conflicted merge was automatically resolved,
the content of a file that was already present could incorrectly
be replaced with a symlink.
* Fix a bug in the git-annex branch handling code that could
cause info from a remote to not be merged and take effect immediately.
* Direct mode is now fully tested by the test suite.
* Detect bad content in ~/.config/git-annex/program and look in PATH instead.
* OSX: Fixed gpg included in dmg.
* Linux standalone: Back to being built with glibc 2.13 for maximum
portability.
-- Joey Hess <joeyh@debian.org> Tue, 21 May 2013 13:10:26 -0400
git-annex (4.20130516) unstable; urgency=low
* Android: The webapp is ported and working.
* Windows: There is a very rough Windows port. Do not trust it with
important data.
* git-annex-shell: Ensure that received files can be read. Files
transferred from some Android devices may have very broken permissions
as received.
* direct mode: Direct mode commands now work on files staged in the index,
they do not need to be committed to git.
* Temporarily add an upper bound to the version of yesod that can be built
with, since yesod 1.2 has a great many changes that will require extensive
work on the webapp.
* Disable building with the haskell threaded runtime when the assistant
is not built. This may fix builds on s390x and sparc, which are failing
to link -lHSrts_thr
* Avoid depending on regex-tdfa on mips, mipsel, and s390, where it fails
to build.
* direct: Fix a bug that could cause some files to be left in indirect mode.
* When initializing a directory special remote with a relative path,
the path is made absolute.
* SHA: Add a runtime sanity check that sha commands output something
that appears to be a real sha.
* configure: Better checking that sha commands output in the desired format.
* rsync special remotes: When sending from a crippled filesystem, use
the destination's default file permissions, as the local ones can
be arbitrarily broken. (Ie, ----rwxr-x for files on Android)
* migrate: Detect if a file gets corrupted while it's being migrated.
* Debian: Add a menu file.
-- Joey Hess <joeyh@debian.org> Thu, 16 May 2013 11:03:35 -0400
git-annex (4.20130501) unstable; urgency=low
* sync, assistant: Behavior changes: Sync with remotes that have
annex-ignore set, so that git remotes on servers without git-annex
installed can be used to keep clients' git repos in sync.
* assistant: Work around misfeature in git 1.8.2 that makes
`git commit --alow-empty -m ""` run an editor.
* sync: Bug fix, avoid adding to the annex the
dummy symlinks used on crippled filesystems.
* Add public repository group.
(And inpreferreddir to preferred content expressions.)
* webapp: Can now set up Internet Archive repositories.
* S3: Dropping content from the Internet Archive doesn't work, but
their API indicates it does. Always refuse to drop from there.
* Automatically register public urls for files uploaded to the
Internet Archive.
* To enable an existing special remote, the new enableremote command
must be used. The initremote command now is used only to create
new special remotes.
* initremote: If two existing remotes have the same name,
prefer the one with a higher trust level.
* assistant: Improved XMPP protocol to better support multiple repositories
using the same XMPP account. Fixes bad behavior when sharing with a friend
when you or the friend have multiple reposotories on an XMPP account.
Note that XMPP pairing with your own devices still pairs with all
repositories using your XMPP account.
* assistant: Fix bug that could cause incoming pushes to not get
merged into the local tree. Particularly affected XMPP pushes.
* webapp: Display some additional information about a repository on
its edit page.
* webapp: Install FDO desktop menu file when started in standalone mode.
* webapp: Don't default to making repository in cwd when started
from within a directory containing a git-annex file (eg, standalone
tarball directory).
* Detect systems that have no user name set in GECOS, and also
don't have user.name set in git config, and put in a workaround
so that commits to the git-annex branch (and the assistant)
will still succeed despite git not liking the system configuration.
* webapp: When told to add a git repository on a remote server, and
the repository already exists as a non-bare repository, use it,
rather than initializing a bare repository in the same directory.
* direct, indirect: Refuse to do anything when the assistant
or git-annex watch daemon is running.
* assistant: When built with git before 1.8.0, use `git remote rm`
to delete a remote. Newer git uses `git remote remove`.
* rmurl: New command, removes one of the recorded urls for a file.
* Detect when the remote is broken like bitbucket is, and exits 0 when
it fails to run git-annex-shell.
* assistant: Several improvements to performance and behavior when
performing bulk adds of a large number of files (tens to hundreds
of thousands).
* assistant: Sanitize XMPP presence information logged for debugging.
* webapp: Now automatically fills in any creds used by an existing remote
when creating a new remote of the same type. Done for Internet Archive,
S3, Glacier, and Box.com remotes.
* Store an annex-uuid file in the bucket when setting up a new S3 remote.
* Support building with DAV 0.4.
-- Joey Hess <joeyh@debian.org> Wed, 01 May 2013 01:42:46 -0400
git-annex (4.20130417) unstable; urgency=low
* initremote: Generates encryption keys with high quality entropy.
This can be disabled using --fast to get the old behavior.
The assistant still uses low-quality entropy when creating encrypted
remotes, to avoid delays. (Thanks, guilhem for the patch.)
* Bugfix: Direct mode no longer repeatedly checksums duplicated files.
* assistant: Work around horrible, terrible, very bad behavior of
gnome-keyring, by not storing special-purpose ssh keys in ~/.ssh/*.pub.
Apparently gnome-keyring apparently will load and indiscriminately use
such keys in some cases, even if they are not using any of the standard
ssh key names. Instead store the keys in ~/.ssh/annex/,
which gnome-keyring will not check.
* addurl: Bugfix: Did not properly add file in direct mode.
* assistant: Bug fix to avoid annexing the files that git uses
to stand in for symlinks on FAT and other filesystem not supporting
symlinks.
* Adjust preferred content expressions so that content in archive
directories is preferred until it has reached an archive or smallarchive
repository.
* webapp: New --listen= option allows running the webapp on one computer
and connecting to it from another. (Note: Does not yet use HTTPS.)
* Added annex.web-download-command setting.
* Added per-remote annex-rsync-transport option. (guilhem again)
* Ssh connection caching is now also used by rsync special remotes.
(guilhem yet again)
* The version number is now derived from git, unless built with
VERSION_FROM_CHANGELOG.
* assistant: Stop any transfers the assistant initiated on shutdown.
* assistant: Added sequence numbers to XMPP git push packets. (Not yet used.)
* addurl: Register transfer so the webapp can see it.
* addurl: Automatically retry downloads that fail, as long as some
additional content was downloaded.
* webapp: Much improved progress bar display for downloads from encrypted
remotes.
* Avoid using runghc, as that needs ghci.
* webapp: When a repository's group is changed, rescan for transfers.
* webapp: Added animations.
* webapp: Include the repository directory in the mangled hostname and
ssh key name, so that a locked down ssh key for one repository is not
re-used when setting up additional repositories on the same server.
* Fall back to internal url downloader when built without curl.
* fsck: Check content of direct mode files (only when the inode cache
thinks they are unmodified).
-- Joey Hess <joeyh@debian.org> Wed, 17 Apr 2013 09:07:38 -0400
git-annex (4.20130405) unstable; urgency=low
* Group subcommands into sections in usage. Closes: #703797
* Per-command usage messages.
* webapp: Fix a race that sometimes caused alerts or other notifications
to be missed if they occurred while a page was loading.
* webapp: Progess bar fixes for many types of special remotes.
* Build debian package without using cabal, which writes to HOME.
Closes: #704205
* webapp: Run ssh server probes in a way that will work when the
login shell is a monstrosity that should have died 25 years ago,
such as csh.
* New annex.largefiles setting, which configures which files
`git annex add` and the assistant add to the annex.
* assistant: Check small files into git directly.
* Remotes can be configured to use other MAC algorithms than HMACSHA1
to encrypt filenames.
Thanks, guilhem for the patch.
* git-annex-shell: Passes rsync --bwlimit options on rsync.
Thanks, guilhem for the patch.
* webapp: Added UI to delete repositories. Closes: #689847
* Adjust built-in preferred content expressions to make most types
of repositories want content that is only located on untrusted, dead,
and unwanted repositories.
* drop --auto: Fix bug that prevented dropping files from untrusted
repositories.
* assistant: Fix bug that could cause direct mode files to be unstaged
from git.
* Update working tree files fully atomically.
* webapp: Improved transfer queue management.
* init: Probe whether the filesystem supports fifos, and if not,
disable ssh connection caching.
* Use lower case hash directories for storing files on crippled filesystems,
same as is already done for bare repositories.
-- Joey Hess <joeyh@debian.org> Fri, 05 Apr 2013 10:42:18 -0400
git-annex (4.20130323) unstable; urgency=low
* webapp: Repository list is now included in the dashboard, and other
UI tweaks.
* webapp: Improved UI for pairing your own devices together using XMPP.
* webapp: Display an alert when there are XMPP remotes, and a cloud
transfer repository needs to be configured.
* Add incrementalbackup repository group.
* webapp: Encourage user to install git-annex on a server when adding
a ssh server, rather than just funneling them through to rsync.
* xmpp: --debug now enables a sanitized dump of the XMPP protocol
* xmpp: Try harder to detect presence of clients when there's a git push
to send.
* xmpp: Re-enable XA flag, since disabling it did not turn out to help
with the problems Google Talk has with not always sending presence
messages to clients.
* map: Combine duplicate repositories, for a nicer looking map.
* Fix several bugs caused by a bad Ord instance for Remote.
* webapp: Switch all forms to POST.
* assistant: Avoid syncing with annex-ignored remotes when reconnecting
to the network, or connecting a drive.
* assistant: Fix OSX bug that prevented committing changed files to a
repository when in indirect mode.
* webapp: Improved alerts displayed when syncing with remotes, and
when syncing with a remote fails.
* webapp: Force wrap long filenames in transfer display.
* assistant: The ConfigMonitor left one zombie behind each time
it checked for changes, now fixed.
* get, copy, move: Display an error message when an identical transfer
is already in progress, rather than failing with no indication why.
* assistant: Several optimisations to file transfers.
* OSX app and standalone Linux tarball now both support being added to
PATH; no need to use runshell to start git-annex.
* webapp: When adding a removable drive, you can now specify the
directory inside it to use.
* webapp: Confirm whether user wants to combine repositories when
adding a removable drive that already has a repository on it.
-- Joey Hess <joeyh@debian.org> Fri, 22 Mar 2013 18:54:05 -0400
git-annex (4.20130314) unstable; urgency=low
* Bugfix: git annex add, when ran without any file or directory specified,
should add files in the current directory, but not act on unlocked files
elsewhere in the tree.
* Bugfix: drop --from an unavailable remote no longer updates the location
log, incorrectly, to say the remote does not have the key.
* Bugfix: If the UUID of a remote is not known, prevent --from, --to,
and other ways of specifying remotes by name from selecting it,
since it is not possible to sanely use it.
* Bugfix: Fix bug in inode cache sentinal check, which broke
copying to local repos if the repo being copied from had moved
to a different filesystem or otherwise changed all its inodes
* Switch from using regex-compat to regex-tdfa, as the C regex library
is rather buggy.
* status: Can now be run with a directory path to show only the
status of that directory, rather than the whole annex.
* Added remote.<name>.annex-gnupg-options setting.
Thanks, guilhem for the patch.
* addurl: Add --relaxed option.
* addurl: Escape invalid characters in urls, rather than failing to
use an invalid url.
* addurl: Properly handle url-escaped characters in file:// urls.
* assistant: Fix dropping content when a file is moved to an archive
directory, and getting contennt when a file is moved back out.
* assistant: Fix bug in direct mode that could occur when a symlink is
moved out of an archive directory, and resulted in the file not being
set to direct mode when it was transferred.
* assistant: Generate better commits for renames.
* assistant: Logs are rotated to avoid them using too much disk space.
* assistant: Avoid noise in logs from git commit about typechanged
files in direct mode repositories.
* assistant: Set gc.auto=0 when creating repositories to prevent
automatic commits from causing git-gc runs.
* assistant: If gc.auto=0, run git-gc once a day, packing loose objects
very non-aggressively.
* assistant: XMPP git pull and push requests are cached and sent when
presence of a new client is detected.
* assistant: Sync with all git remotes on startup.
* assistant: Get back in sync with XMPP remotes after network reconnection,
and on startup.
* assistant: Fix syncing after XMPP pairing.
* assistant: Optimised handling of renamed files in direct mode,
avoiding re-checksumming.
* assistant: Detects most renames, including directory renames, and
combines all their changes into a single commit.
* assistant: Fix ~/.ssh/git-annex-shell wrapper to work when the
ssh key does not force a command.
* assistant: Be smarter about avoiding unncessary transfers.
* webapp: Work around bug in Warp's slowloris attack prevention code,
that caused regular browsers to stall when they reuse a connection
after leaving it idle for 30 seconds.
(See https://github.com/yesodweb/wai/issues/146)
* webapp: New preferences page allows enabling/disabling debug logging
at runtime, as well as configuring numcopies and diskreserve.
* webapp: Repository costs can be configured by dragging repositories around
in the repository list.
* webapp: Proceed automatically on from "Configure jabber account"
to pairing.
* webapp: Only show up to 10 queued transfers.
* webapp: DTRT when told to create a git repo that already exists.
* webapp: Set locally paired repositories to a lower cost than other
network remotes.
* Run ssh with -T to avoid tty allocation and any login scripts that
may do undesired things with it.
* Several improvements to Makefile and cabal file. Thanks, Peter Simmons
* Stop depending on testpack.
* Android: Enable test suite.
-- Joey Hess <joeyh@debian.org> Thu, 14 Mar 2013 15:29:20 -0400
git-annex (4.20130227) unstable; urgency=low
* annex.version is now set to 4 for direct mode repositories.
* Should now fully support git repositories with core.symlinks=false;
always using git's pseudosymlink files in such repositories.
* webapp: Allow creating repositories on filesystems that lack support for
symlinks.
* webapp: Can now add a new local repository, and make it sync with
the main local repository.
* Android: Bundle now includes openssh.
* Android: Support ssh connection caching.
* Android: Assistant is fully working. (But no webapp yet.)
* Direct mode: Support filesystems like FAT which can change their inodes
each time they are mounted.
* Direct mode: Fix support for adding a modified file.
* Avoid passing -p to rsync, to interoperate with crippled filesystems.
Closes: #700282
* Additional GIT_DIR support bugfixes. May actually work now.
* webapp: Display any error message from git init if it fails to create
a repository.
* Fix a reversion in matching globs introduced in the last release,
where "*" did not match files inside subdirectories. No longer uses
the Glob library.
* copy: Update location log when no copy was performed, if the location
log was out of date.
* Makefile now builds using cabal, taking advantage of cabal's automatic
detection of appropriate build flags.
* test: The test suite is now built into the git-annex binary, and can
be run at any time.
-- Joey Hess <joeyh@debian.org> Wed, 27 Feb 2013 14:07:24 -0400
git-annex (3.20130216) unstable; urgency=low
* Now uses the Haskell uuid library, rather than needing a uuid program.
* Now uses the Haskell Glob library, rather than pcre-light, avoiding
the need to install libpcre. Currently done only for Cabal or when
the Makefile is made to use -DWITH_GLOB
* Android port now available (command-line only).
* New annex.crippledfilesystem setting, allows use of git-annex
repositories on FAT and even worse filesystems; avoiding use of
hard links and locked down permissions settings. (Support is incomplete.)
* init: Detect when the repository is on a filesystem that does not
support hard links, or symlinks, or unix permissions, and set
annex.crippledfilesystem, as well as annex.direct.
* add: Improved detection of files that are modified while being added.
* Fix a bug in direct mode, introduced in the previous release, where
if a file was dropped and then got back, it would be stored in indirect
mode.
-- Joey Hess <joeyh@debian.org> Sat, 16 Feb 2013 10:03:26 -0400
git-annex (3.20130207) unstable; urgency=low
* webapp: Now allows restarting any threads that crash.
* Adjust debian package to only build-depend on DAV on architectures
where it is available.
* addurl --fast: Use curl, rather than haskell HTTP library, to support https.
* annex.autocommit: New setting, can be used to disable autocommit
of changed files by the assistant, while it still does data syncing
and other tasks.
* assistant: Ignore .DS_Store on OSX.
* assistant: Fix location log when adding new file in direct mode.
* Deal with stale mappings for deleted file in direct mode.
* pre-commit: Update direct mode mappings.
* uninit, unannex --fast: If hard link creation fails, fall back to slow
mode.
* Clean up direct mode cache and mapping info when dropping keys.
* dropunused: Clean up stale direct mode cache and mapping info not
removed before.
-- Joey Hess <joeyh@debian.org> Thu, 07 Feb 2013 12:45:25 -0400
git-annex (3.20130124) unstable; urgency=low
* Added source repository group, that only retains files until they've
been transferred to another repository. Useful for things like
repositories on cameras.
* Added manual repository group. Use to prevent the assistant from
downloading any file contents to keep things in sync. Instead
`git annex get`, `git annex drop` etc can be used manually as desired.
* webapp: More adjustments to longpoll code to deal with changes in
variable quoting in different versions of shakespeare-js.
* webapp: Avoid an error if a transfer is stopped just as it finishes.
Closes: #698184
* webapp: Now always logs to .git/annex/daemon.log
* webapp: Has a page to view the log, accessed from the control menu.
* webapp: Fix crash adding removable drive that has an annex directory
in it that is not a git repository.
correct spelling mistakes
2017-02-11 09:38:49 +00:00
* Deal with incompatibility in gpg2, which caused prompts for encryption
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
passphrases rather than using the supplied --passphrase-fd.
* bugfix: Union merges involving two or more repositories could sometimes
result in data from one repository getting lost. This could result
in the location log data becoming wrong, and fsck being needed to fix it.
* sync: Automatic merge conflict resolution now stages deleted files.
* Depend on git 1.7.7.6 for --no-edit. Closes: #698399
* Fix direct mode mapping code to always store direct mode filenames
relative to the top of the repository, even when operating inside a
subdirectory.
* fsck: Detect and fix consistency errors in direct mode mapping files.
* Avoid filename encoding errors when writing direct mode mappings.
-- Joey Hess <joeyh@debian.org> Tue, 22 Jan 2013 07:11:59 +1100
git-annex (3.20130114) unstable; urgency=low
* Now handles the case where a file that's being transferred to a remote
is modified in place, which direct mode allows. When this
happens, the transfer now fails, rather than allow possibly corrupt
data into the remote.
* fsck: Better checking of file content in direct mode.
* drop: Suggest using git annex move when numcopies prevents dropping a file.
* webapp: Repo switcher filters out repos that do not exist any more
(or are on a drive that's not mounted).
* webapp: Use IP address, rather than localhost, since some systems may
have configuration problems or other issues that prevent web browsers
from connecting to the right localhost IP for the webapp.
* webapp: Adjust longpoll code to work with recent versions of
shakespeare-js.
* assistant: Support new gvfs dbus names used in Gnome 3.6.
* In direct mode, files with the same key are no longer hardlinked, as
that would cause a surprising behavior if modifying one, where the other
would also change.
* webapp: Avoid illegal characters in hostname when creating S3 or
Glacier remote.
* assistant: Avoid committer crashing if a file is deleted at the wrong
instant.
-- Joey Hess <joeyh@debian.org> Mon, 14 Jan 2013 15:25:18 -0400
git-annex (3.20130107) unstable; urgency=low
* webapp: Add UI to stop and restart assistant.
* committer: Fix a file handle leak.
* assistant: Make expensive transfer scan work fully in direct mode.
* More commands work in direct mode repositories: find, whereis, move, copy,
drop, log, fsck, add, addurl.
* sync: No longer automatically adds files in direct mode.
* assistant: Detect when system is not configured with a user name,
and set environment to prevent git from failing.
* direct: Avoid hardlinking symlinks that point to the same content
when the content is not present.
* Fix transferring files to special remotes in direct mode.
-- Joey Hess <joeyh@debian.org> Mon, 07 Jan 2013 01:01:41 -0400
git-annex (3.20130102) unstable; urgency=low
* direct, indirect: New commands, that switch a repository to and from
direct mode. In direct mode, files are accessed directly, rather than
via symlinks. Note that direct mode is currently experimental. Many
git-annex commands do not work in direct mode. Some git commands can
cause data loss when used in direct mode repositories.
* assistant: Now uses direct mode by default when setting up a new
local repository.
* OSX assistant: Uses the FSEvents API to detect file changes.
This avoids issues with running out of file descriptors on large trees,
as well as allowing detection of modification of files in direct mode.
Other BSD systems still use kqueue.
* kqueue: Fix bug that made broken symlinks not be noticed.
* vicfg: Quote filename. Closes: #696193
* Bugfix: Fixed bug parsing transfer info files, where the newline after
the filename was included in it. This was generally benign, but in
the assistant, it caused unexpected dropping of preferred content.
* Bugfix: Remove leading \ from checksums output by sha*sum commands,
when the filename contains \ or a newline. Closes: #696384
* fsck: Still accept checksums with a leading \ as valid, now that
above bug is fixed.
* SHA*E backends: Exclude non-alphanumeric characters from extensions.
* migrate: Remove leading \ in SHA* checksums, and non-alphanumerics
from extensions of SHA*E keys.
-- Joey Hess <joeyh@debian.org> Wed, 02 Jan 2013 13:21:34 -0400
git-annex (3.20121211) unstable; urgency=low
* webapp: Defaults to sharing box.com account info with friends, allowing
one-click enabling of the repository.
* Fix broken .config/git-annex/program installed by standalone tarball.
* assistant: Retrival from glacier now handled.
* Include ssh in standalone tarball and OSX app.
* watch: Avoid leaving hard links to files behind in .git/annex/tmp
if a file is deleted or moved while it's being quarantined in preparation
to being added to the annex.
* Allow `git annex drop --from web`; of course this does not remove
any file from the web, but it does make git-annex remove all urls
associated with a file.
* webapp: S3 and Glacier forms now have a select list of all
currently-supported AWS regions.
correct spelling mistakes
2017-02-11 09:38:49 +00:00
* webdav: Avoid trying to set props, avoiding incompatibility with
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
livedrive.com. Needs DAV version 0.3.
* webapp: Prettify error display.
* webapp: Fix bad interaction between required fields and modals.
* webapp: Added help buttons and links next to fields that require
explanations.
* webapp: Encryption can be disabled when setting up remotes.
* assistant: Avoid trying to drop content from remotes that don't have it.
* assistant: Allow periods in ssh key comments.
* get/copy --auto: Transfer data even if it would exceed numcopies,
when preferred content settings want it.
* drop --auto: Fix dropping content when there are no preferred content
settings.
* webapp: Allow user to specify the port when setting up a ssh or rsync
remote.
* assistant: Fix syncing to just created ssh remotes.
* Enable WebDAV support in Debian package. Closes: #695532
-- Joey Hess <joeyh@debian.org> Tue, 11 Dec 2012 11:25:03 -0400
git-annex (3.20121127) unstable; urgency=low
* Fix dirContentsRecursive, which had missed some files in deeply nested
subdirectories. Could affect various parts of git-annex.
* rsync: Fix bug introduced in last release that broke encrypted rsync
special remotes.
* The standalone builds now unset their special path and library path
variables before running the system web browser.
-- Joey Hess <joeyh@debian.org> Tue, 27 Nov 2012 17:07:32 -0400
git-annex (3.20121126) unstable; urgency=low
* New webdav and Amazon glacier special remotes.
* Display a warning when a non-existing file or directory is specified.
* webapp: Added configurator for Box.com.
* webapp: Show error messages to user when testing XMPP creds.
* Fix build of assistant without yesod.
* webapp: The list of repositiories refreshes when new repositories are
added, including when new repository configurations are pushed in from
remotes.
* OSX: Fix RunAtLoad value in plist file.
* Getting a file from chunked directory special remotes no longer buffers
it all in memory.
* S3: Added progress display for uploading and downloading.
* directory special remote: Made more efficient and robust.
* Bugfix: directory special remote could loop forever storing a key
when a too small chunksize was configured.
* Allow controlling whether login credentials for S3 and webdav are
committed to the repository, by setting embedcreds=yes|no when running
initremote.
* Added smallarchive repository group, that only archives files that are
in archive directories. Used by default for glacier when set up in the
webapp.
* assistant: Fixed handling of toplevel archive directory and
client repository group.
* assistant: Apply preferred content settings when a new symlink
is created, or a symlink gets renamed. Made archive directories work.
-- Joey Hess <joeyh@debian.org> Mon, 26 Nov 2012 11:37:49 -0400
git-annex (3.20121112) unstable; urgency=low
* assistant: Can use XMPP to notify other nodes about pushes made to other
repositories, as well as pushing to them directly over XMPP.
* wepapp: Added an XMPP configuration interface.
* webapp: Supports pairing over XMPP, with both friends, and other repos
using the same account.
* assistant: Drops non-preferred content when possible.
* assistant: Notices, and applies config changes as they are made to
the git-annex branch, including config changes pushed in from remotes.
* git-annex-shell: GIT_ANNEX_SHELL_DIRECTORY can be set to limit it
to operating on a specified directory.
* webapp: When setting up authorized_keys, use GIT_ANNEX_SHELL_DIRECTORY.
* Preferred content path matching bugfix.
* Preferred content expressions cannot use "in=".
* Preferred content expressions can use "present".
* Fix handling of GIT_DIR when it refers to a git submodule.
* Depend on and use the Haskell SafeSemaphore library, which provides
exception-safe versions of SampleVar and QSemN.
Thanks, Ben Gamari for an excellent patch set.
* file:/// URLs can now be used with the web special remote.
* webapp: Allow dashes in ssh key comments when pairing.
* uninit: Check and abort if there are symlinks to annexed content that
are not checked into git.
* webapp: Switched to using the same multicast IP address that avahi uses.
* bup: Don't pass - to bup-split to make it read stdin; bup 0.25
does not accept that.
* bugfix: Don't fail transferring content from read-only repos.
Closes: #691341
* configure: Check that checksum programs produce correct checksums.
* Re-enable dbus, using a new version of the library that fixes the memory
leak.
* NetWatcher: When dbus connection is lost, try to reconnect.
* Use USER and HOME environment when set, and only fall back to getpwent,
which doesn't work with LDAP or NIS.
* rsync special remote: Include annex-rsync-options when running rsync
to test a key's presence.
* The standalone tarball's runshell now takes care of installing a
~/.ssh/git-annex-shell wrapper the first time it's run.
* webapp: Make an initial, empty commit so there is a master branch
* assistant: Fix syncing local drives.
* webapp: Fix creation of rsync.net repositories.
* webapp: Fix renaming of special remotes.
* webapp: Generate better git remote names.
* webapp: Ensure that rsync special remotes are enabled using the same
name they were originally created using.
* Bugfix: Fix hang in webapp when setting up a ssh remote with an absolute
path.
-- Joey Hess <joeyh@debian.org> Mon, 12 Nov 2012 10:39:47 -0400
git-annex (3.20121017) unstable; urgency=low
* Fix zombie cleanup reversion introduced in 3.20121009.
* Additional fix to support git submodules.
-- Joey Hess <joeyh@debian.org> Tue, 16 Oct 2012 21:10:14 -0400
git-annex (3.20121016) unstable; urgency=low
* vicfg: New file format, avoids ambiguity with repos that have the same
description, or no description.
* Bug fix: A recent change caused git-annex-shell to crash.
* Better preferred content expression for transfer repos.
* webapp: Repository edit form can now edit the name of a repository.
* webapp: Make bare repositories on removable drives, as there is nothing
to ensure non-bare repos get updated when syncing.
* webapp: Better behavior when pausing syncing to a remote when a transfer
scan is running and queueing new transfers for that remote.
* The standalone binaries are now built to not use ssh connection caching,
in order to work with old versions of ssh.
* A relative core.worktree is relative to the gitdir. Now that this is
handled correctly, git-annex can be used in git submodules.
* Temporarily disable use of dbus, as the haskell dbus library blows up
when losing connection, which will need to be fixed upstream.
-- Joey Hess <joeyh@debian.org> Tue, 16 Oct 2012 15:25:22 -0400
git-annex (3.20121010) unstable; urgency=low
* Renamed --ingroup to --inallgroup.
* Standard groups changed to client, transfer, archive, and backup.
Each of these has its own standard preferred content setting.
* dead: Remove dead repository from all groups.
* Avoid unsetting HOME when running certian git commands. Closes: #690193
* test: Fix threaded runtime hang.
* Makefile: Avoid building with -threaded if the ghc threaded runtime does
not exist.
* webapp: Improve wording of intro display. Closes: #689848
* webapp: Repositories can now be configured, to change their description,
their group, or even to disable syncing to them.
* git config remote.name.annex-sync can be used to control whether
a remote gets synced.
* Fix a crash when merging files in the git-annex branch that contain
invalid utf8.
* Automatically detect when a ssh remote does not have git-annex-shell
installed, and set annex-ignore.
-- Joey Hess <joeyh@debian.org> Fri, 12 Oct 2012 13:45:21 -0400
git-annex (3.20121009) unstable; urgency=low
* watch, assistant: It's now safe to git annex unlock files while
the watcher is running, as well as modify files checked into git
as normal files. Additionally, .gitignore settings are now honored.
Closes: #689979
* group, ungroup: New commands to indicate groups of repositories.
* webapp: Adds newly created repositories to one of these groups:
clients, drives, servers
* vicfg: New command, allows editing (or simply viewing) most
of the repository configuration settings stored in the git-annex branch.
* Added preferred content expressions, configurable using vicfg.
* get --auto: If the local repository has preferred content
configured, only get that content.
* drop --auto: If the repository the content is dropped from has
preferred content configured, drop only content that is not preferred.
* copy --auto: Only transfer content that the destination repository prefers.
* assistant: Now honors preferred content settings when deciding what to
transfer.
* --copies=group:number can now be used to match files that are present
in a specified number of repositories in a group.
* Added --smallerthan, --largerthan, and --inall limits.
* Only build-depend on libghc-clientsession-dev on arches that will have
the webapp.
* uninit: Unset annex.version. Closes: #689852
-- Joey Hess <joeyh@debian.org> Tue, 09 Oct 2012 15:13:23 -0400
git-annex (3.20121001) unstable; urgency=low
* fsck: Now has an incremental mode. Start a new incremental fsck pass
with git annex fsck --incremental. Now the fsck can be interrupted
as desired, and resumed with git annex fsck --more.
Thanks, Justin Azoff
* New --time-limit option, makes long git-annex commands stop after
a specified amount of time.
* fsck: New --incremental-schedule option which is nice for scheduling
eg, monthly incremental fsck runs in cron jobs.
* Fix fallback to ~/Desktop when xdg-user-dir is not available.
Closes: #688833
* S3: When using a shared cipher, S3 credentials are not stored encrypted
in the git repository, as that would allow anyone with access to
the repository access to the S3 account. Instead, they're stored
in a 600 mode file in the local git repo.
* webapp: Avoid crashing when ssh-keygen -F chokes on an invalid known_hosts
file.
* Always do a system wide installation when DESTDIR is set. Closes: #689052
* The Makefile now builds with the new yesod by default.
Systems like Debian that have the old yesod 1.0.1 should set
GIT_ANNEX_LOCAL_FEATURES=-DWITH_OLD_YESOD
* copy: Avoid updating the location log when no copy is performed.
* configure: Test that uuid -m works, falling back to plain uuid if not.
* Avoid building the webapp on Debian architectures that do not yet
have template haskell and thus yesod. (Should be available for arm soonish
I hope).
-- Joey Hess <joeyh@debian.org> Mon, 01 Oct 2012 13:56:55 -0400
git-annex (3.20120924) unstable; urgency=low
* assistant: New command, a daemon which does everything watch does,
as well as automatically syncing file contents between repositories.
* webapp: An interface for managing and configuring the assistant.
* The default backend used when adding files to the annex is changed
from SHA256 to SHA256E, to simplify interoperability with OSX, media
players, and various programs that needlessly look at symlink targets.
To get old behavior, add a .gitattributes containing: * annex.backend=SHA256
* init: If no description is provided for a new repository, one will
automatically be generated, like "joey@gnu:~/foo"
* test: Set a lot of git environment variables so testing works in strange
environments that normally need git config to set names, etc.
Closes: #682351 Thanks, gregor herrmann
* Disable ssh connection caching if the path to the control socket would be
too long (and use relative path to minimise path to the control socket).
* migrate: Check content before generating the new key, to avoid generating
a key for corrupt data.
* Support repositories created with --separate-git-dir. Closes: #684405
* reinject: When the provided file doesn't match, leave it where it is,
rather than moving to .git/annex/bad/
* Avoid crashing on encoding errors in filenames when writing transfer info
files and reading from checksum commands.
* sync: Pushes the git-annex branch to remote/synced/git-annex, rather
than directly to remote/git-annex.
* Now supports matching files that are present on a number of remotes
with a specified trust level. Example: --copies=trusted:2
Thanks, Nicolas Pouillard
-- Joey Hess <joeyh@debian.org> Mon, 24 Sep 2012 13:47:48 -0400
git-annex (3.20120825) unstable; urgency=low
* S3: Add fileprefix setting.
* Pass --use-agent to gpg when in no tty mode. Thanks, Eskild Hustvedt.
* Bugfix: Fix fsck in SHA*E backends, when the key contains composite
extensions, as added in 3.20120721.
-- Joey Hess <joeyh@debian.org> Sat, 25 Aug 2012 10:00:10 -0400
git-annex (3.20120807) unstable; urgency=low
* initremote: Avoid recording remote's description before checking
that its config is valid.
* unused, status: Avoid crashing when ran in bare repo.
* Avoid crashing when "git annex get" fails to download from one
location, and falls back to downloading from a second location.
-- Joey Hess <joeyh@debian.org> Tue, 07 Aug 2012 13:35:07 -0400
git-annex (3.20120721) unstable; urgency=low
* get, move, copy: Now refuse to do anything when the requested file
transfer is already in progress by another process.
* status: Lists transfers that are currently in progress.
* Fix passing --uuid to git-annex-shell.
* When shaNsum commands cannot be found, use the Haskell SHA library
(already a dependency) to do the checksumming. This may be slower,
but avoids portability problems.
* Use SHA library for files less than 50 kb in size, at which point it's
faster than forking the more optimised external program.
* SHAnE backends are now smarter about composite extensions, such as
.tar.gz Closes: #680450
* map: Write map.dot to .git/annex, which avoids watch trying to annex it.
-- Joey Hess <joeyh@debian.org> Sat, 21 Jul 2012 16:52:48 -0400
git-annex (3.20120629) unstable; urgency=low
* cabal: Only try to use inotify on Linux.
* Version build dependency on STM, and allow building without it,
which disables the watch command.
* Avoid ugly failure mode when moving content from a local repository
that is not available.
* Got rid of the last place that did utf8 decoding.
* Accept arbitrarily encoded repository filepaths etc when reading
git config output. This fixes support for remotes with unusual characters
in their names.
* sync: Automatically resolves merge conflicts.
-- Joey Hess <joeyh@debian.org> Fri, 29 Jun 2012 10:17:49 -0400
git-annex (3.20120624) unstable; urgency=low
* watch: New subcommand, a daemon which notices changes to
files and automatically annexes new files, etc, so you don't
need to manually run git commands when manipulating files.
Available on Linux, BSDs, and OSX!
* Enable diskfree on kfreebsd, using kqueue.
* unused: Fix crash when key names contain invalid utf8.
* sync: Avoid recent git's interactive merge.
-- Joey Hess <joeyh@debian.org> Sun, 24 Jun 2012 12:36:50 -0400
git-annex (3.20120614) unstable; urgency=medium
* addurl: Was broken by a typo introduced 2 released ago, now fixed.
Closes: #677576
* Install man page when run by cabal, in a location where man will
find it, even when installing under $HOME. Thanks, Nathan Collins
-- Joey Hess <joeyh@debian.org> Thu, 14 Jun 2012 20:21:29 -0400
git-annex (3.20120611) unstable; urgency=medium
* add: Prevent (most) modifications from being made to a file while it
is being added to the annex.
* initremote: Automatically describe a remote when creating it.
* uninit: Refuse to run in a subdirectory. Closes: #677076
-- Joey Hess <joeyh@debian.org> Mon, 11 Jun 2012 10:32:01 -0400
git-annex (3.20120605) unstable; urgency=low
* sync: Show a nicer message if a user tries to sync to a special remote.
* lock: Reset unlocked file to index, rather than to branch head.
* import: New subcommand, pulls files from a directory outside the annex
and adds them.
* Fix display of warning message when encountering a file that uses an
unsupported backend.
* Require that the SHA256 backend can be used when building, since it's the
default.
* Preserve parent environment when running hooks of the hook special remote.
-- Joey Hess <joeyh@debian.org> Tue, 05 Jun 2012 14:03:39 -0400
git-annex (3.20120522) unstable; urgency=low
* Pass -a to cp even when it supports --reflink=auto, to preserve
permissions.
* Clean up handling of git directory and git worktree.
* Add support for core.worktree, and fix support for GIT_WORK_TREE and
GIT_DIR.
-- Joey Hess <joeyh@debian.org> Tue, 22 May 2012 11:16:13 -0400
git-annex (3.20120511) unstable; urgency=low
* Rsync special remotes can be configured with shellescape=no
to avoid shell quoting that is normally done when using rsync over ssh.
This is known to be needed for certian rsync hosting providers
(specificially hidrive.strato.com) that use rsync over ssh but do not
pass it through the shell.
* dropunused: Allow specifying ranges to drop.
* addunused: New command, the opposite of dropunused, it relinks unused
content into the git repository.
* Fix use of several config settings: annex.ssh-options,
annex.rsync-options, annex.bup-split-options. (And adjust types to avoid
the bugs that broke several config settings.)
-- Joey Hess <joeyh@debian.org> Fri, 11 May 2012 12:29:30 -0400
git-annex (3.20120430) unstable; urgency=low
* Fix use of annex.diskreserve config setting.
* Directory special remotes now check annex.diskreserve.
* Support git's core.sharedRepository configuration.
* Add annex.http-headers and annex.http-headers-command config
settings, to allow custom headers to be sent with all HTTP requests.
(Requested by the Internet Archive)
* uninit: Clear annex.uuid from .git/config. Closes: #670639
* Added shared cipher mode to encryptable special remotes. This option
minor typo fixes throughout problematic flexibility
2016-06-02 01:46:58 +00:00
avoids gpg key distribution, at the expense of flexibility, and with
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
the requirement that all clones of the git repository be equally trusted.
-- Joey Hess <joeyh@debian.org> Mon, 30 Apr 2012 13:16:10 -0400
git-annex (3.20120418) unstable; urgency=low
* bugfix: Adding a dotfile also caused all non-dotfiles to be added.
* bup: Properly handle key names with spaces or other things that are
not legal git refs.
* git-annex (but not git-annex-shell) supports the git help.autocorrect
configuration setting, doing fuzzy matching using the restricted
Damerau-Levenshtein edit distance, just as git does. This adds a build
dependency on the haskell edit-distance library.
* Renamed diskfree.c to avoid OSX case insensativity bug.
* cabal now installs git-annex-shell as a symlink to git-annex.
* cabal file now autodetects whether S3 support is available.
-- Joey Hess <joeyh@debian.org> Wed, 18 Apr 2012 12:11:32 -0400
git-annex (3.20120406) unstable; urgency=low
* Disable diskfree on kfreebsd, as I have a build failure on kfreebsd-i386
that is quite likely caused by it.
-- Joey Hess <joeyh@debian.org> Sat, 07 Apr 2012 15:50:36 -0400
git-annex (3.20120405) unstable; urgency=low
* Rewrote free disk space checking code, moving the portability
handling into a small C library.
* status: Display amount of free disk space.
-- Joey Hess <joeyh@debian.org> Thu, 05 Apr 2012 16:19:10 -0400
git-annex (3.20120315) unstable; urgency=low
* fsck: Fix up any broken links and misplaced content caused by the
directory hash calculation bug fixed in the last release.
* sync: Sync to lower cost remotes first.
* status: Fixed to run in constant space.
* status: More accurate display of sizes of tmp and bad keys.
* unused: Now uses a bloom filter, and runs in constant space.
Use of a bloom filter does mean it will not notice a small
number of unused keys. For repos with up to half a million keys,
it will miss one key in 1000.
* Added annex.bloomcapacity and annex.bloomaccuracy, which can be
adjusted as desired to tune the bloom filter.
* status: Display amount of memory used by bloom filter, and
detect when it's too small for the number of keys in a repository.
* git-annex-shell: Runs hooks/annex-content after content is received
or dropped.
* Work around a bug in rsync (IMHO) introduced by openSUSE's SIP patch.
* git-annex now behaves as git-annex-shell if symlinked to and run by that
name. The Makefile sets this up, saving some 8 mb of installed size.
* git-union-merge is a demo program, so it is no longer built by default.
-- Joey Hess <joeyh@debian.org> Thu, 15 Mar 2012 11:05:28 -0400
git-annex (3.20120309) unstable; urgency=low
* Fix key directory hash calculation code to behave as it did before
version 3.20120227 when a key contains non-ascii characters (only
WORM backend is likely to have been affected).
-- Joey Hess <joeyh@debian.org> Fri, 09 Mar 2012 20:05:09 -0400
git-annex (3.20120230) unstable; urgency=low
* "here" can be used to refer to the current repository,
which can read better than the old "." (which still works too).
* Directory special remotes now support chunking files written to them,
avoiding writing files larger than a specified size.
* Add progress bar display to the directory special remote.
* Add configurable hooks that are run when git-annex starts and stops
using a remote: remote.name.annex-start-command and
remote.name.annex-stop-command
* Fix a bug in symlink calculation code, that triggered in rare
cases where an annexed file is in a subdirectory that nearly
matched to the .git/annex/object/xx/yy subdirectories.
-- Joey Hess <joeyh@debian.org> Mon, 05 Mar 2012 13:38:13 -0400
git-annex (3.20120229) unstable; urgency=low
* Fix test suite to not require a unicode locale.
* Fix cabal build failure. Thanks, Sergei Trofimovich
-- Joey Hess <joeyh@debian.org> Wed, 29 Feb 2012 02:31:31 -0400
git-annex (3.20120227) unstable; urgency=low
* Modifications to support ghc 7.4's handling of filenames.
This version can only be built with ghc 7.4 or newer. See the ghc7.0
branch for older ghcs.
* S3: Fix irrefutable pattern failure when accessing encrypted S3
credentials.
* Use the haskell IfElse library.
* Fix teardown of stale cached ssh connections.
* Fixed to use the strict state monad, to avoid leaking all kinds of memory
due to lazy state update thunks when adding/fixing many files.
* Fixed some memory leaks that occurred when committing journal files.
* Added a annex.queuesize setting, useful when adding hundreds of thousands
of files on a system with plenty of memory.
* whereis: Prints the urls of files that the web special remote knows about.
* addurl --fast: Verifies that the url can be downloaded (only getting
its head), and records the size in the key.
* When checking that an url has a key, verify that the Content-Length,
if available, matches the size of the key.
* addurl: Added a --file option, which can be used to specify what
file the url is added to. This can be used to override the default
filename that is used when adding an url, which is based on the url.
Or, when the file already exists, the url is recorded as another
location of the file.
* addurl: Normalize badly encoded urls.
* addurl: Add --pathdepth option.
* rekey: New plumbing level command, can be used to change the keys used
for files en masse.
* Store web special remote url info in a more efficient location.
(Urls stored with this version will not be visible to older versions.)
* Deal with NFS problem that caused a failure to remove a directory
when removing content from the annex.
* Make a single location log commit after a remote has received or
dropped files. Uses a new "git-annex-shell commit" command when available.
* To avoid commits of data to the git-annex branch after each command
is run, set annex.alwayscommit=false. Its data will then be committed
less frequently, when a merge or sync is done.
* configure: Check if ssh connection caching is supported by the installed
version of ssh and default annex.sshcaching accordingly.
* move --from, copy --from: Now 10 times faster when scanning to find
files in a remote on a local disk; rather than go through the location log
to see which files are present on the remote, it simply looks at the
disk contents directly.
-- Joey Hess <joeyh@debian.org> Mon, 27 Feb 2012 12:58:21 -0400
git-annex (3.20120123) unstable; urgency=low
* fsck --from: Fscking a remote is now supported. It's done by retrieving
the contents of the specified files from the remote, and checking them,
so can be an expensive operation. Still, if the remote is a special
remote, or a git repository that you cannot run fsck in locally, it's
nice to have the ability to fsck it.
* If you have any directory special remotes, now would be a good time to
fsck them, in case you were hit by the data loss bug fixed in the
previous release!
* fsck --from remote --fast: Avoids expensive file transfers, at the
expense of not checking file size and/or contents.
* Ssh connection caching is now enabled automatically by git-annex.
Only one ssh connection is made to each host per git-annex run, which
can speed some things up a lot, as well as avoiding repeated password
prompts. Concurrent git-annex processes also share ssh connections.
Cached ssh connections are shut down when git-annex exits.
* To disable the ssh caching (if for example you have your own broader
ssh caching configuration), set annex.sshcaching=false.
-- Joey Hess <joeyh@debian.org> Mon, 23 Jan 2012 13:48:48 -0400
git-annex (3.20120116) unstable; urgency=medium
* Fix data loss bug in directory special remote, when moving a file
to the remote failed, and partially transferred content was left
behind in the directory, re-running the same move would think it
succeeded and delete the local copy.
-- Joey Hess <joeyh@debian.org> Mon, 16 Jan 2012 16:43:45 -0400
git-annex (3.20120115) unstable; urgency=low
* Add a sanity check for bad StatFS results. On architectures
where StatFS does not currently work (s390, mips, powerpc, sparc),
this disables the diskreserve checking code, and attempting to
configure an annex.diskreserve will result in an error.
* Fix QuickCheck dependency in cabal file.
* Minor optimisations.
-- Joey Hess <joeyh@debian.org> Sun, 15 Jan 2012 13:54:20 -0400
git-annex (3.20120113) unstable; urgency=low
* log: Add --gource mode, which generates output usable by gource.
* map: Fix display of remote repos
* Add annex-trustlevel configuration settings, which can be used to
override the trust level of a remote.
* git-annex, git-union-merge: Support GIT_DIR and GIT_WORK_TREE.
* Add libghc-testpack-dev to build depends on all arches.
-- Joey Hess <joeyh@debian.org> Fri, 13 Jan 2012 15:35:17 -0400
git-annex (3.20120106) unstable; urgency=low
* Support unescaped repository urls, like git does.
* log: New command that displays the location log for files,
showing each repository they were added to and removed from.
* Fix overbroad gpg --no-tty fix from last release.
-- Joey Hess <joeyh@debian.org> Sat, 07 Jan 2012 13:16:23 -0400
git-annex (3.20120105) unstable; urgency=low
* Added annex-web-options configuration settings, which can be
used to provide parameters to whichever of wget or curl git-annex uses
(depends on which is available, but most of their important options
suitable for use here are the same).
* Dotfiles, and files inside dotdirs are not added by "git annex add"
unless the dotfile or directory is explicitly listed. So "git annex add ."
will add all untracked files in the current directory except for those in
dotdirs.
* Added quickcheck to build dependencies, and fail if test suite cannot be
built.
* fsck: Do backend-specific check before checking numcopies is satisfied.
* Run gpg with --no-tty. Closes: #654721
-- Joey Hess <joeyh@debian.org> Thu, 05 Jan 2012 13:44:12 -0400
git-annex (3.20111231) unstable; urgency=low
* sync: Improved to work well without a central bare repository.
Thanks to Joachim Breitner.
* Rather than manually committing, pushing, pulling, merging, and git annex
merging, we encourage you to give "git annex sync" a try.
* sync --fast: Selects some of the remotes with the lowest annex.cost
and syncs those, in addition to any specified at the command line.
* Union merge now finds the least expensive way to represent the merge.
* reinject: Add a sanity check for using an annexed file as the source file.
* Properly handle multiline git config values.
* Fix the hook special remote, which bitrotted a while ago.
* map: --fast disables use of dot to display map
* Test suite improvements. Current top-level test coverage: 75%
* Improve deletion of files from rsync special remotes. Closes: #652849
* Add --include, which is the same as --not --exclude.
* Format strings can be specified using the new --format option, to control
what is output by git annex find.
* Support git annex find --json
* Fixed behavior when multiple insteadOf configs are provided for the
same url base.
* Can now be built with older git versions (before 1.7.7); the resulting
binary should only be used with old git.
* Updated to build with monad-control 0.3.
-- Joey Hess <joeyh@debian.org> Sat, 31 Dec 2011 14:55:29 -0400
git-annex (3.20111211) unstable; urgency=medium
* Fix bug in last version in getting contents from bare repositories.
* Ensure that git-annex branch changes are merged into git-annex's index,
which fixes a bug that could cause changes that were pushed to the
git-annex branch to get reverted. As a side effect, it's now safe
for users to check out and commit changes directly to the git-annex
branch.
* map: Fix a failure to detect a loop when both repositories are local
and refer to each other with relative paths.
* Prevent key names from containing newlines.
* add: If interrupted, add can leave files converted to symlinks but not
yet added to git. Running the add again will now clean up this situtation.
* Fix caching of decrypted ciphers, which failed when drop had to check
multiple different encrypted special remotes.
* unannex: Can be run on files that have been added to the annex, but not
yet committed.
* sync: New command that synchronises the local repository and default
remote, by running git commit, pull, and push for you.
* Version monad-control dependency in cabal file.
-- Joey Hess <joeyh@debian.org> Sun, 11 Dec 2011 21:24:39 -0400
git-annex (3.20111203) unstable; urgency=low
* The VFAT filesystem on recent versions of Linux, when mounted with
shortname=mixed, does not get along well with git-annex's mixed case
.git/annex/objects hash directories. To avoid this problem, new content
is now stored in all-lowercase hash directories. Except for non-bare
repositories which would be a pain to transition and cannot be put on FAT.
(Old mixed-case hash directories are still tried for backwards
compatibility.)
* Flush json output, avoiding a buffering problem that could result in
doubled output.
* Avoid needing haskell98 and other fixes for new ghc. Thanks, Mark Wright.
* Bugfix: dropunused did not drop keys with two spaces in their name.
* Support for storing .git/annex on a different device than the rest of the
git repository.
* --inbackend can be used to make git-annex only operate on files
whose content is stored using a specified key-value backend.
* dead: A command which says that a repository is gone for good
and you don't want git-annex to mention it again.
-- Joey Hess <joeyh@debian.org> Sat, 03 Dec 2011 21:01:45 -0400
git-annex (3.20111122) unstable; urgency=low
* merge: Improve commit messages to mention what was merged.
* Avoid doing auto-merging in commands that don't need fully current
information from the git-annex branch. In particular, git annex add
no longer needs to auto-merge.
* init: When run in an already initalized repository, and without
a description specified, don't delete the old description.
* Optimised union merging; now only runs git cat-file once, and runs
in constant space.
* status: Now displays trusted, untrusted, and semitrusted repositories
separately.
* status: Include all special remotes in the list of repositories.
* status: Fix --json mode.
* status: --fast is back
* Fix support for insteadOf url remapping. Closes: #644278
* When not run in a git repository, git-annex can still display a usage
message, and "git annex version" even works.
* migrate: Don't fall over a stale temp file.
* Avoid excessive escaping for rsync special remotes that are not accessed
over ssh.
* find: Support --print0
-- Joey Hess <joeyh@debian.org> Tue, 22 Nov 2011 14:31:45 -0400
git-annex (3.20111111) unstable; urgency=low
* Handle a case where an annexed file is moved into a gitignored directory,
by having fix --force add its change.
* Avoid cyclic drop problems.
* Optimized copy --from and get --from to avoid checking the location log
for files that are already present.
* Automatically fix up badly formatted uuid.log entries produced by
3.20111105, whenever the uuid.log is changed (ie, by init or describe).
* map: Support remotes with /~/ and /~user/
-- Joey Hess <joeyh@debian.org> Fri, 11 Nov 2011 13:44:18 -0400
git-annex (3.20111107) unstable; urgency=low
* merge: Use fast-forward merges when possible.
Thanks Valentin Haenel for a test case showing how non-fast-forward
merges could result in an ongoing pull/merge/push cycle.
* Don't try to read config from repos with annex-ignore set.
* Bugfix: In the past two releases, git-annex init has written the uuid.log
in the wrong format, with the UUID and description flipped.
-- Joey Hess <joeyh@debian.org> Mon, 07 Nov 2011 12:47:44 -0400
git-annex (3.20111105) unstable; urgency=low
* The default backend used when adding files to the annex is changed
from WORM to SHA256.
To get old behavior, add a .gitattributes containing: * annex.backend=WORM
* Sped up some operations on remotes that are on the same host.
* copy --to: Fixed leak when copying many files to a remote on the same
host.
* uninit: Add guard against being run with the git-annex branch checked out.
* Fail if --from or --to is passed to commands that do not support them.
* drop --from is now supported to remove file content from a remote.
* status: Now always shows the current repository, even when it does not
appear in uuid.log.
* fsck: Now works in bare repositories. Checks location log information,
and file contents. Does not check that numcopies is satisfied, as
.gitattributes information about numcopies is not available in a bare
repository.
* unused, dropunused: Now work in bare repositories.
* Removed the setkey command, and added a reinject command with a more
useful interface.
* The fromkey command now takes the key as its first parameter. The --key
option is no longer used.
* Built without any filename containing .git being excluded. Closes: #647215
* Record uuid when auto-initializing a remote so it shows in status.
* Bugfix: Fixed git-annex init crash in a bare repository when there was
already an existing git-annex branch.
* Pass -t to rsync to preserve timestamps.
-- Joey Hess <joeyh@debian.org> Sat, 05 Nov 2011 15:47:52 -0400
git-annex (3.20111025) unstable; urgency=low
* A remote can have a annexUrl configured, that is used by git-annex
instead of its usual url. (Similar to pushUrl.)
* migrate: Copy url logs for keys when migrating.
* git-annex-shell: GIT_ANNEX_SHELL_READONLY and GIT_ANNEX_SHELL_LIMITED
environment variables can be set to limit what commands can be run.
This is used by gitolite's new git-annex support!
-- Joey Hess <joeyh@debian.org> Tue, 25 Oct 2011 13:03:08 -0700
git-annex (3.20111011) unstable; urgency=low
* This version of git-annex only works with git 1.7.7 and newer.
The breakage with old versions is subtle, and affects the
annex.numcopies settings in .gitattributes, so be sure to upgrade git
to 1.7.7. (Debian package now depends on that version.)
* Don't pass absolute paths to git show-attr, as it started following
symlinks when that's done in 1.7.7. Instead, use relative paths,
which show-attr only handles 100% correctly in 1.7.7. Closes: #645046
* Fix referring to remotes by uuid.
* New or changed repository descriptions in uuid.log now have a timestamp,
which is used to ensure the newest description is used when the uuid.log
has been merged.
* Note that older versions of git-annex will display the timestamp as part
of the repository description, which is ugly but otherwise harmless.
* Add timestamps to trust.log and remote.log too.
* git-annex-shell: Added the --uuid option.
* git-annex now asks git-annex-shell to verify that it's operating in
the expected repository.
* Note that this git-annex will not interoperate with remotes using
older versions of git-annex-shell.
* Now supports git's insteadOf configuration, to modify the url
used to access a remote. Note that pushInsteadOf is not used;
that and pushurl are reserved for actual git pushes. Closes: #644278
* status: List all known repositories.
* When displaying a list of repositories, show git remote names
in addition to their descriptions.
* Add locking to avoid races when changing the git-annex branch.
* Various speed improvements gained by using ByteStrings.
* Contain the zombie hordes.
-- Joey Hess <joeyh@debian.org> Tue, 11 Oct 2011 23:00:02 -0400
git-annex (3.20110928) unstable; urgency=low
* --in can be used to make git-annex only operate on files
believed to be present in a given repository.
* Arbitrarily complex expressions can be built to limit the files git-annex
operates on, by combining the options --not --and --or -( and -)
Example: git annex get --exclude '*.mp3' --and --not -( --in usbdrive --or --in archive -)
* --copies=N can be used to make git-annex only operate on files with
the specified number of copies. (And --not --copies=N for the inverse.)
* find: Rather than only showing files whose contents are present,
when used with --exclude --copies or --in, displays all files that
match the specified conditions.
* Note that this is a behavior change for git-annex find! Old behavior
can be gotten by using: git-annex find --in .
* status: Massively sped up; remove --fast mode.
* unused: File contents used by branches and tags are no longer
considered unused, even when not used by the current branch. This is
the final piece of the puzzle needed for git-annex to to play nicely
with branches.
-- Joey Hess <joeyh@debian.org> Wed, 28 Sep 2011 18:14:02 -0400
git-annex (3.20110915) unstable; urgency=low
* whereis: Show untrusted locations separately and do not include in
location count.
* Fix build without S3.
* addurl: Always use whole url as destination filename, rather than
only its file component.
* get, drop, copy: Added --auto option, which decides whether
to get/drop content as needed to work toward the configured numcopies.
* bugfix: drop and fsck did not honor --exclude
-- Joey Hess <joeyh@debian.org> Thu, 15 Sep 2011 22:25:46 -0400
git-annex (3.20110906) unstable; urgency=low
* Improve display of newlines around error and warning messages.
* Fix Makefile to work with cabal again.
-- Joey Hess <joeyh@debian.org> Tue, 06 Sep 2011 13:45:16 -0400
git-annex (3.20110902) unstable; urgency=low
* Set EMAIL when running test suite so that git does not need to be
configured first. Closes: #638998
* The wget command will now be used in preference to curl, if available.
* init: Make description an optional parameter.
* unused, status: Sped up by avoiding unnecessary stats of annexed files.
* unused --remote: Reduced memory use to 1/4th what was used before.
* Add --json switch, to produce machine-consumable output.
-- Joey Hess <joeyh@debian.org> Fri, 02 Sep 2011 21:20:37 -0400
git-annex (3.20110819) unstable; urgency=low
* Now "git annex init" only has to be run once, when a git repository
is first being created. Clones will automatically notice that git-annex
is in use and automatically perform a basic initalization. It's
still recommended to run "git annex init" in any clones, to describe them.
* Added annex-cost-command configuration, which can be used to vary the
cost of a remote based on the output of a shell command.
* Fix broken upgrade from V1 repository. Closes: #638584
-- Joey Hess <joeyh@debian.org> Fri, 19 Aug 2011 20:34:09 -0400
git-annex (3.20110817) unstable; urgency=low
* Fix shell escaping in rsync special remote.
* addurl: --fast can be used to avoid immediately downloading the url.
* Added support for getting content from git remotes using http (and https).
* Added curl to Debian package dependencies.
-- Joey Hess <joeyh@debian.org> Wed, 17 Aug 2011 01:29:02 -0400
git-annex (3.20110719) unstable; urgency=low
* add: Be even more robust to avoid ever leaving the file seemingly deleted.
Closes: #634233
* Bugfix: Make add ../ work.
* Support the standard git -c name=value
* unannex: Clean up use of git commit -a.
-- Joey Hess <joeyh@debian.org> Tue, 19 Jul 2011 23:39:53 -0400
git-annex (3.20110707) unstable; urgency=low
* Fix sign bug in disk free space checking.
* Bugfix: Forgot to de-escape keys when upgrading. Could result in
bad location log data for keys that contain [&:%] in their names.
(A workaround for this problem is to run git annex fsck.)
* add: Avoid a failure mode that resulted in the file seemingly being
deleted (content put in the annex but no symlink present).
-- Joey Hess <joeyh@debian.org> Thu, 07 Jul 2011 19:29:39 -0400
git-annex (3.20110705) unstable; urgency=low
* uninit: Delete the git-annex branch and .git/annex/
* unannex: In --fast mode, file content is left in the annex, and a
hard link made to it.
* uninit: Use unannex in --fast mode, to support unannexing multiple
files that link to the same content.
* Drop the dependency on the haskell curl bindings, use regular haskell HTTP.
* Fix a pipeline stall when upgrading (caused by #624389).
-- Joey Hess <joeyh@debian.org> Tue, 05 Jul 2011 14:37:39 -0400
git-annex (3.20110702) unstable; urgency=low
* Now the web can be used as a special remote.
This feature replaces the old URL backend.
* addurl: New command to download an url and store it in the annex.
* Sped back up fsck, copy --from, and other commands that often
have to read a lot of information from the git-annex branch. Such
commands are now faster than they were before introduction of the
git-annex branch.
* Always ensure git-annex branch exists.
* Modify location log parser to allow future expansion.
* --force will cause add, etc, to operate on ignored files.
* Avoid mangling encoding when storing the description of repository
and other content.
* cabal can now be used to build git-annex. This is substantially
slower than using make, does not build or install documentation,
does not run the test suite, and is not particularly recommended,
but could be useful to some.
-- Joey Hess <joeyh@debian.org> Sat, 02 Jul 2011 15:00:18 -0400
git-annex (3.20110624) experimental; urgency=low
* New repository format, annex.version=3. Use `git annex upgrade` to migrate.
* git-annex now stores its logs in a git-annex branch.
* merge: New subcommand. Auto-merges the new git-annex branch.
* Improved handling of bare git repos with annexes. Many more commands will
work in them.
* git-annex is now more robust; it will never leave state files
uncommitted when some other git process comes along and locks the index
at an inconvenient time.
* rsync is now used when copying files from repos on other filesystems.
cp is still used when copying file from repos on the same filesystem,
since --reflink=auto can make it significantly faster on filesystems
such as btrfs.
* Allow --trust etc to specify a repository by name, for temporarily
trusting repositories that are not configured remotes.
* unlock: Made atomic.
* git-union-merge: New git subcommand, that does a generic union merge
operation, and operates efficiently without touching the working tree.
-- Joey Hess <joeyh@debian.org> Fri, 24 Jun 2011 14:32:18 -0400
git-annex (0.20110610) unstable; urgency=low
* Add --numcopies option.
* Add --trust, --untrust, and --semitrust options.
* get --from is the same as copy --from
* Bugfix: Fix fsck to not think all SHAnE keys are bad.
-- Joey Hess <joeyh@debian.org> Fri, 10 Jun 2011 11:48:40 -0400
git-annex (0.20110601) unstable; urgency=low
* Minor bugfixes and error message improvements.
* Massively sped up `git annex lock` by avoiding use of the uber-slow
`git reset`, and only running `git checkout` once, even when many files
are being locked.
* Fix locking of files with staged changes.
* Somewhat sped up `git commit` of modifications to unlocked files.
* Build fix for older ghc.
-- Joey Hess <joeyh@debian.org> Wed, 01 Jun 2011 11:50:47 -0400
git-annex (0.20110522) unstable; urgency=low
* Closer emulation of git's behavior when told to use "foo/.git" as a
git repository instead of just "foo". Closes: #627563
* Fix bug in --exclude introduced in 0.20110516.
-- Joey Hess <joeyh@debian.org> Fri, 27 May 2011 20:20:41 -0400
git-annex (0.20110521) unstable; urgency=low
* status: New subcommand to show info about an annex, including its size.
* --backend now overrides any backend configured in .gitattributes files.
* Add --debug option. Closes: #627499
-- Joey Hess <joeyh@debian.org> Sat, 21 May 2011 11:52:53 -0400
git-annex (0.20110516) unstable; urgency=low
* Add a few tweaks to make it easy to use the Internet Archive's variant
of S3. In particular, munge key filenames to comply with the IA's filename
limits, disable encryption, support their nonstandard way of creating
buckets, and allow x-archive-* headers to be specified in initremote to
set item metadata.
* Added filename extension preserving variant backends SHA1E, SHA256E, etc.
* migrate: Use current filename when generating new key, for backends
where the filename affects the key name.
* Work around a bug in Network.URI's handling of bracketed ipv6 addresses.
-- Joey Hess <joeyh@debian.org> Mon, 16 May 2011 14:16:52 -0400
git-annex (0.20110503) unstable; urgency=low
* Fix hasKeyCheap setting for bup and rsync special remotes.
* Add hook special remotes.
* Avoid crashing when an existing key is readded to the annex.
* unused: Now also lists files fsck places in .git/annex/bad/
* S3: When encryption is enabled, the Amazon S3 login credentials
are stored, encrypted, in .git-annex/remotes.log, so environment
variables need not be set after the remote is initialized.
-- Joey Hess <joeyh@debian.org> Tue, 03 May 2011 20:56:01 -0400
git-annex (0.20110427) unstable; urgency=low
* Switch back to haskell SHA library, so git-annex remains buildable on
Debian stable.
* Added rsync special remotes. This could be used, for example, to
store annexed content on rsync.net (encrypted naturally). Or anywhere else.
* Bugfix: Avoid pipeline stall when running git annex drop or fsck on a
lot of files. Possibly only occured with ghc 7.
-- Joey Hess <joeyh@debian.org> Wed, 27 Apr 2011 22:50:26 -0400
git-annex (0.20110425) unstable; urgency=low
* Use haskell Crypto library instead of haskell SHA library.
* Remove testpack from build depends for non x86 architectures where it
is not available. The test suite will not be run if it cannot be compiled.
* Avoid using absolute paths when staging location log, as that can
confuse git when a remote's path contains a symlink. Closes: #621386
-- Joey Hess <joeyh@debian.org> Mon, 25 Apr 2011 15:47:00 -0400
git-annex (0.20110420) unstable; urgency=low
* Update Debian build dependencies for ghc 7.
* Debian package is now built with S3 support.
Thanks Joachim Breitner for making this possible.
* Somewhat improved memory usage of S3, still work to do.
Thanks Greg Heartsfield for ongoing work to improve the hS3 library
for git-annex.
-- Joey Hess <joeyh@debian.org> Thu, 21 Apr 2011 15:00:48 -0400
git-annex (0.20110419) unstable; urgency=low
* Don't run gpg in batch mode, so it can prompt for passphrase when
there is no agent.
* Add missing build dep on dataenc.
* S3: Fix stalls when transferring encrypted data.
* bup: Avoid memory leak when transferring encrypted data.
-- Joey Hess <joeyh@debian.org> Tue, 19 Apr 2011 21:26:51 -0400
git-annex (0.20110417) unstable; urgency=low
* bup is now supported as a special type of remote.
* The data sent to special remotes (Amazon S3, bup, etc) can be encrypted
using GPG for privacy.
* Use lowercase hash directories for locationlog files, to avoid
some issues with git on OSX with the mixed-case directories.
No migration is needed; the old mixed case hash directories are still
read; new information is written to the new directories.
* Unused files on remotes, particulary special remotes, can now be
identified and dropped, by using "--from remote" with git annex unused
and git annex dropunused.
* Clear up short option confusion between --from and --force (-f is now
--from, and there is no short option for --force).
* Add build depend on perlmagick so docs are consistently built.
Closes: #621410
* Add doc-base file. Closes: #621408
* Periodically flush git command queue, to avoid boating memory usage
too much.
* Support "sha1" and "sha512" commands on FreeBSD, and allow building
if any/all SHA commands are not available. Thanks, Fraser Tweedale
-- Joey Hess <joeyh@debian.org> Sun, 17 Apr 2011 12:00:24 -0400
git-annex (0.20110401) experimental; urgency=low
* Amazon S3 is now supported as a special type of remote.
Warning: Encrypting data before sending it to S3 is not yet supported.
* Note that Amazon S3 support is not built in by default on Debian yet,
as hS3 is not packaged.
* fsck: Ensure that files and directories in .git/annex/objects
have proper permissions.
* Added a special type of remote called a directory remote, which
simply stores files in an arbitrary local directory.
* Bugfix: copy --to --fast never really copied, fixed.
-- Joey Hess <joeyh@debian.org> Fri, 01 Apr 2011 21:27:22 -0400
git-annex (0.20110328) experimental; urgency=low
* annex.diskreserve can be given in arbitrary units (ie "0.5 gigabytes")
* Generalized remotes handling, laying groundwork for remotes that are
not regular git remotes. (Think Amazon S3.)
* Provide a less expensive version of `git annex copy --to`, enabled
via --fast. This assumes that location tracking information is correct,
rather than contacting the remote for every file.
* Bugfix: Keys could be received into v1 annexes from v2 annexes, via
v1 git-annex-shell. This results in some oddly named keys in the v1
annex. Recognise and fix those keys when upgrading, instead of crashing.
-- Joey Hess <joeyh@debian.org> Mon, 28 Mar 2011 10:47:29 -0400
git-annex (0.20110325) experimental; urgency=low
* Free space checking is now done, for transfers of data for keys
that have free space metadata. (Notably, not for SHA* keys generated
with git-annex 0.2x or earlier.) The code is believed to work on
Linux, FreeBSD, and OSX; check compile-time messages to see if it
is not enabled for your OS.
* Add annex.diskreserve config setting, to control how much free space
to reserve for other purposes and avoid using (defaults to 1 mb).
* Add --fast flag, that can enable less expensive, but also less thorough
versions of some commands.
* fsck: In fast mode, avoid checking checksums.
* unused: In fast mode, just show all existing temp files as unused,
and avoid expensive scan for other unused content.
* migrate: Support migrating v1 SHA keys to v2 SHA keys with
size information that can be used for free space checking.
* Fix space leak in fsck and drop commands.
* migrate: Bugfix for case when migrating a file results in a key that
is already present in .git/annex/objects.
* dropunused: Significantly sped up; only read unused log file once.
-- Joey Hess <joeyh@debian.org> Fri, 25 Mar 2011 00:47:37 -0400
git-annex (0.20110320) experimental; urgency=low
* Fix dropping of files using the URL backend.
* Fix support for remotes with '.' in their names.
* Add version command to show git-annex version as well as repository
version information.
* No longer auto-upgrade to repository format 2, to avoid accidental
upgrades, etc. Use git-annex upgrade when you're ready to run this
version.
-- Joey Hess <joeyh@debian.org> Sun, 20 Mar 2011 16:36:33 -0400
git-annex (0.20110316) experimental; urgency=low
* New repository format, annex.version=2.
* The first time git-annex is run in an old format repository, it
will automatically upgrade it to the new format, staging all
necessary changes to git. Also added a "git annex upgrade" command.
* Colons are now avoided in filenames, so bare clones of git repos
can be put on USB thumb drives formatted with vFAT or similar
filesystems.
* Added two levels of hashing to object directory and .git-annex logs,
to improve scalability with enormous numbers of annexed
objects. (With one hundred million annexed objects, each
directory would contain fewer than 1024 files.)
* The setkey, fromkey, and dropkey subcommands have changed how
the key is specified. --backend is no longer used with these.
-- Joey Hess <joeyh@debian.org> Wed, 16 Mar 2011 16:20:23 -0400
git-annex (0.24) unstable; urgency=low
Branched the 0.24 series, which will be maintained for a while to
support v1 git-annex repos, while main development moves to the 0.2011
series, with v2 git-annex repos.
* Add Suggests on graphviz. Closes: #618039
* When adding files to the annex, the symlinks pointing at the annexed
content are made to have the same mtime as the original file.
While git does not preserve that information, this allows a tool
like metastore to be used with annexed files.
(Currently this is only done on systems supporting POSIX 200809.)
-- Joey Hess <joeyh@debian.org> Wed, 16 Mar 2011 18:35:13 -0400
git-annex (0.23) unstable; urgency=low
* Support ssh remotes with a port specified.
* whereis: New subcommand to show where a file's content has gotten to.
* Rethink filename encoding handling for display. Since filename encoding
may or may not match locale settings, any attempt to decode filenames
will fail for some files. So instead, do all output in binary mode.
-- Joey Hess <joeyh@debian.org> Sat, 12 Mar 2011 15:02:49 -0400
git-annex (0.22) unstable; urgency=low
* Git annexes can now be attached to bare git repositories.
(Both the local and remote host must have this version of git-annex
installed for it to work.)
* Support filenames that start with a dash; when such a file is passed
to a utility it will be escaped to avoid it being interpreted as an
option. (I went a little overboard and got the type checker involved
in this, so such files are rather comprehensively supported now.)
* New backends: SHA512 SHA384 SHA256 SHA224
(Supported on systems where corresponding shaNsum commands are available.)
* describe: New subcommand that can set or change the description of
a repository.
* Fix test suite to reap zombies.
(Zombies can be particularly annoying on OSX; thanks to Jimmy Tang
for his help eliminating the infestation... for now.)
* Make test suite not rely on a working cp -pr.
(The Unix wars are still ON!)
* Look for dir.git directories the same as git does.
* Support remote urls specified as relative paths.
* Support non-ssh remote paths that contain tilde expansions.
* fsck: Check for and repair location log damage.
* Bugfix: When fsck detected and moved away corrupt file content, it did
not update the location log.
-- Joey Hess <joeyh@debian.org> Fri, 04 Mar 2011 15:10:57 -0400
git-annex (0.21) unstable; urgency=low
* test: Don't rely on chmod -R working.
* unannex: Fix recently introduced bug when attempting to unannex more
than one file at a time.
* test: Set git user name and email in case git can't guess values.
* Fix display of unicode filenames.
-- Joey Hess <joeyh@debian.org> Fri, 11 Feb 2011 23:21:08 -0400
git-annex (0.20) unstable; urgency=low
* Preserve specified file ordering when instructed to act on multiple
files or directories. For example, "git annex get a b" will now always
get "a" before "b". Previously it could operate in either order.
* unannex: Commit staged changes at end, to avoid some confusing behavior
with the pre-commit hook, which would see some types of commits after
an unannex as checking in of an unlocked file.
* map: New subcommand that uses graphviz to display a nice map of
the git repository network.
* Deal with the mtl/monads-fd conflict.
* configure: Check for sha1sum.
-- Joey Hess <joeyh@debian.org> Tue, 08 Feb 2011 18:57:24 -0400
git-annex (0.19) unstable; urgency=low
* configure: Support using the uuidgen command if the uuid command is
not available.
* Allow --exclude to be specified more than once.
* There are now three levels of repository trust.
* untrust: Now marks the current repository as untrusted.
* semitrust: Now restores the default trust level. (What untrust used to do.)
* fsck, drop: Take untrusted repositories into account.
* Bugfix: Files were copied from trusted remotes first even if their
annex.cost was higher than other remotes.
* Improved temp file handling. Transfers of content can now be resumed
from temp files later; the resume does not have to be the immediate
next git-annex run.
* unused: Include partially transferred content in the list.
* Bugfix: Running a second git-annex while a first has a transfer in
progress no longer deletes the first processes's temp file.
-- Joey Hess <joeyh@debian.org> Fri, 28 Jan 2011 14:31:37 -0400
git-annex (0.18) unstable; urgency=low
* Bugfix: `copy --to` and `move --to` forgot to stage location log changes
after transferring the file to the remote repository.
(Did not affect ssh remotes.)
* fsck: Fix bug in moving of corrupted files to .git/annex/bad/
* migrate: Fix support for --backend option.
* unlock: Fix behavior when file content is not present.
* Test suite improvements. Current top-level test coverage: 80%
-- Joey Hess <joeyh@debian.org> Fri, 14 Jan 2011 14:17:44 -0400
git-annex (0.17) unstable; urgency=low
* unannex: Now skips files whose content is not present, rather than
it being an error.
* New migrate subcommand can be used to switch files to using a different
backend, safely and with no duplication of content.
* bugfix: Fix crash caused by empty key name. (Thanks Henrik for reporting.)
-- Joey Hess <joeyh@debian.org> Sun, 09 Jan 2011 10:04:11 -0400
git-annex (0.16) unstable; urgency=low
* git-annex-shell: Avoid exposing any git repo config except for the
annex.uuid when doing configlist.
* bugfix: Running `move --to` with a remote whose UUID was not yet known
could result in git-annex not recording on the local side where the
file was moved to. This could not result in data loss, or even a
significant problem, since the remote *did* record that it had the file.
* Also, add a general guard to detect attempts to record information
about repositories with missing UUIDs.
* bugfix: Running `move --to` with a non-ssh remote failed.
* bugfix: Running `copy --to` with a non-ssh remote actually did a move.
* Many test suite improvements. Current top-level test coverage: 65%
-- Joey Hess <joeyh@debian.org> Fri, 07 Jan 2011 14:33:13 -0400
git-annex (0.15) unstable; urgency=low
* Support scp-style urls for remotes (host:path).
* Support ssh urls containing "~".
* Add trust and untrust subcommands, to allow configuring repositories
that are trusted to retain files without explicit checking.
* Fix bug in numcopies handling when multiple remotes pointed to the
same repository.
* Introduce the git-annex-shell command. It's now possible to make
a user have it as a restricted login shell, similar to git-shell.
* Note that git-annex will always use git-annex-shell when accessing
a ssh remote, so all of your remotes need to be upgraded to this
version of git-annex at the same time.
* Now rsync is exclusively used for copying files to and from remotes.
scp is not longer supported.
-- Joey Hess <joeyh@debian.org> Fri, 31 Dec 2010 22:00:52 -0400
git-annex (0.14) unstable; urgency=low
* Bugfix to git annex unused in a repository with nothing yet annexed.
* Support upgrading from a v0 annex with nothing in it.
* Avoid multiple calls to git ls-files when passed eg, "*".
-- Joey Hess <joeyh@debian.org> Fri, 24 Dec 2010 17:38:48 -0400
git-annex (0.13) unstable; urgency=low
* Makefile: Install man page and html (when built).
* Makefile: Add GHCFLAGS variable.
* Fix upgrade from 0.03.
* Support remotes using git+ssh and ssh+git as protocol.
Closes: #607056
-- Joey Hess <joeyh@debian.org> Tue, 14 Dec 2010 13:05:10 -0400
git-annex (0.12) unstable; urgency=low
* Add --exclude option to exclude files from processing.
* mwdn2man: Fix a bug in newline supression. Closes: #606578
* Bugfix to git annex add of an unlocked file in a subdir. Closes: #606579
* Makefile: Add PREFIX variable.
-- Joey Hess <joeyh@debian.org> Sat, 11 Dec 2010 17:32:00 -0400
git-annex (0.11) unstable; urgency=low
* If available, rsync will be used for file transfers from remote
repositories. This allows resuming interrupted transfers.
* Added remote.annex-rsync-options.
* Avoid deleting temp files when rsync fails.
* Improve detection of version 0 repos.
* Add uninit subcommand. Closes: #605749
-- Joey Hess <joeyh@debian.org> Sat, 04 Dec 2010 17:27:42 -0400
git-annex (0.10) unstable; urgency=low
* In .gitattributes, the annex.numcopies attribute can be used
to control the number of copies to retain of different types of files.
* Bugfix: Always correctly handle gitattributes when in a subdirectory of
the repository. (Had worked ok for ones like "*.mp3", but failed for
ones like "dir/*".)
* fsck: Fix warning about not enough copies of a file, when locations
are known, but are not available in currently configured remotes.
* precommit: Optimise to avoid calling git-check-attr more than once.
* The git-annex-backend attribute has been renamed to annex.backend.
-- Joey Hess <joeyh@debian.org> Sun, 28 Nov 2010 19:28:05 -0400
git-annex (0.09) unstable; urgency=low
* Add copy subcommand.
* Fix bug in setkey subcommand triggered by move --to.
-- Joey Hess <joeyh@debian.org> Sat, 27 Nov 2010 17:14:59 -0400
git-annex (0.08) unstable; urgency=low
* Fix `git annex add ../foo` (when ran in a subdir of the repo).
* Add configure step to build process.
* Only use cp -a if it is supported, falling back to cp -p or plain cp
as needed for portability.
* cp --reflink=auto is used if supported, and will make git annex unlock
much faster on filesystems like btrfs that support copy on write.
-- Joey Hess <joeyh@debian.org> Sun, 21 Nov 2010 13:45:44 -0400
git-annex (0.07) unstable; urgency=low
* find: New subcommand.
* unused: New subcommand, finds unused data. (Split out from fsck.)
* dropunused: New subcommand, provides for easy dropping of unused keys
by number, as listed by the unused subcommand.
* fsck: Print warnings to stderr; --quiet can now be used to only see
problems.
-- Joey Hess <joeyh@debian.org> Mon, 15 Nov 2010 18:41:50 -0400
git-annex (0.06) unstable; urgency=low
* fsck: Check if annex.numcopies is satisfied.
* fsck: Verify the sha1 of files when the SHA1 backend is used.
* fsck: Verify the size of files when the WORM backend is used.
* fsck: Allow specifying individual files if fscking everything
is not desired.
* fsck: Fix bug, introduced in 0.04, in detection of unused data.
-- Joey Hess <joeyh@debian.org> Sat, 13 Nov 2010 16:24:29 -0400
git-annex (0.05) unstable; urgency=low
* Optimize both pre-commit and lock subcommands to not call git diff
on every file being committed/locked.
(This actually also works around a bug in ghc, that caused
git-annex 0.04 pre-commit to sometimes corrupt filename being read
from git ls-files and fail.
See <http://hackage.haskell.org/trac/ghc/ticket/4493>
The excessive number of calls made by pre-commit exposed the ghc bug.
Thanks Josh Triplett for the debugging.)
* Build with -O2.
-- Joey Hess <joeyh@debian.org> Thu, 11 Nov 2010 18:31:09 -0400
git-annex (0.04) unstable; urgency=low
* Add unlock subcommand, which replaces the symlink with a copy of
the file's content in preparation of changing it. The "edit" subcommand
is an alias for unlock.
* Add lock subcommand.
* Unlocked files will now automatically be added back into the annex when
committed (and the updated symlink committed), by some magic in the
pre-commit hook.
* The SHA1 backend is now fully usable.
* Add annex.version, which will be used to automate upgrades
between incompatible versions.
* Reorganised the layout of .git/annex/
* The new layout will be automatically upgraded to the first time
git-annex is used in a repository with the old layout.
* Note that git-annex 0.04 cannot transfer content from old repositories
that have not yet been upgraded.
* Annexed file contents are now made unwritable and put in unwriteable
correct spelling mistakes
2017-02-11 09:38:49 +00:00
directories, to avoid them accidentally being removed or modified.
Updated cabal file explictly lists source files. The tarball on hackage will include only the files needed for cabal install; it is NOT the full git-annex source tree. While it's totally obnoxious that cabal files need every file listed out when basic wildcard support could avoid hundreds of lines, and have to be maintained when files are added, this does get the tarball size back down to 1 mb. This also stops stack from complaining that it found modules not listed in the cabal file. debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead. This avoids needing to include debian/ in the hackage tarball. Setup.hs: Build man pages at install time using make and mdwn2man. If it fails, which it probably will on windows, just skip installing them.
2016-05-24 05:00:06 +00:00
(Thanks Josh Triplett for the idea.)
* Add build dep on libghc6-testpack-dev. Closes: #603016
* Avoid using runghc to run test suite as it is not available on all
architectures. Closes: #603006
-- Joey Hess <joeyh@debian.org> Wed, 10 Nov 2010 14:23:23 -0400
git-annex (0.03) unstable; urgency=low
* Fix support for file:// remotes.
* Add --verbose
* Fix SIGINT handling.
* Fix handling of files with unusual characters in their name.
* Fixed memory leak; git-annex no longer reads the whole file list
from git before starting, and will be much faster with large repos.
* Fix crash on unknown symlinks.
* Added remote.annex-scp-options and remote.annex-ssh-options.
* The backends to use when adding different sets of files can be configured
via gitattributes.
* In .gitattributes, the git-annex-backend attribute can be set to the
names of backends to use when adding different types of files.
* Add fsck subcommand. (For now it only finds unused key contents in the
annex.)
-- Joey Hess <joeyh@debian.org> Sun, 07 Nov 2010 18:26:04 -0400
git-annex (0.02) unstable; urgency=low
* Can scp annexed files from remote hosts, and check remote hosts for
file content when dropping files.
* New move subcommand, that makes it easy to move file contents from
or to a remote.
* New fromkey subcommand, for registering urls, etc.
* git-annex init will now set up a pre-commit hook that fixes up symlinks
before they are committed, to ensure that moving symlinks around does not
break them.
* More intelligent and fast staging of modified files; git add coalescing.
* Add remote.annex-ignore git config setting to allow completly disabling
a given remote.
* --from/--to can be used to control the remote repository that git-annex
uses.
* --quiet can be used to avoid verbose output
* New plumbing-level dropkey and addkey subcommands.
* Lots of bug fixes.
-- Joey Hess <joeyh@debian.org> Wed, 27 Oct 2010 16:39:29 -0400
git-annex (0.01) unstable; urgency=low
* First prerelease.
-- Joey Hess <joeyh@debian.org> Wed, 20 Oct 2010 12:54:24 -0400
Reference in a new issue Copy permalink