mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
37002 commits 13 branches 445 tags 110 MiB
Commit graph

3014 commits

Author SHA1 Message Date
Ilya_Shlyakhter
d4dafa0671 Added a comment: special remote that restores contents by running a command 2019-10-04 18:16:39 +00:00
Joey Hess
e4e4777a9b
Merge branch 'master' of ssh://git-annex.branchable.com 2019-10-03 09:54:24 -04:00
tobiasBora
d42b117f71 Added a comment 2019-10-03 00:48:23 +00:00
Ilya_Shlyakhter
7dce92bfca Added a comment 2019-10-03 00:36:54 +00:00
tobiasBora
1a2a0707a6 2019-10-02 21:21:01 +00:00
tobiasBora
71405c1c85 2019-10-02 21:20:36 +00:00
Joey Hess
4967f8f0ab
update 2019-10-02 11:24:30 -04:00
Ilya_Shlyakhter
f35b130f59 Added a comment: different repos with same uuid 2019-10-02 15:13:56 +00:00
Joey Hess
d7f4bb5462
Merge branch 'master' of ssh://git-annex.branchable.com 2019-10-01 16:05:15 -04:00
Joey Hess
3229cde14f
comments 2019-10-01 16:04:44 -04:00
Joey Hess
7245a86587
my, this is looking complicated 2019-10-01 13:11:49 -04:00
Joey Hess
61b384d2b7
add --sameas option, not yet used 2019-10-01 12:36:25 -04:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
115886df9f Added a comment: There are still benefits to commit throttling 2019-09-30 22:16:10 +00:00
Joey Hess
7b5ce2b330
expand 2019-09-30 17:35:38 -04:00
Joey Hess
ccf9f03c84
Merge branch 'master' of ssh://git-annex.branchable.com 2019-09-30 17:33:08 -04:00
Joey Hess
7302f12bf6
todo 2019-09-30 17:32:41 -04:00
yarikoptic
47245d7f4b Added a comment 2019-09-30 20:36:00 +00:00
Joey Hess
4200d2933b
comment 2019-09-30 15:31:55 -04:00
Joey Hess
2b55a2b882
remotedaemon: Don't list --stop in help since it's not supported. 
Also, move out of plumbing section. When using tor, the remotedaemon is
part of the user's workflow, as it runs the tor hidden service.
 2019-09-30 14:40:46 -04:00
Joey Hess
5fbf30301a
response 2019-09-30 14:27:23 -04:00
Joey Hess
a9f078da43
Merge branch 'master' of ssh://git-annex.branchable.com 2019-09-30 14:21:23 -04:00
Joey Hess
e52fb641a0
comment 2019-09-30 14:19:35 -04:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
2e1d2bc6d5 Added a comment: .gitattributes could solve this 2019-09-29 22:59:48 +00:00
yarikoptic
e51697fe06 initial idea on separating our hint about remotes 2019-09-27 20:37:40 +00:00
Joey Hess
090898a138
adjust --lock: This enters an adjusted branch where files are locked. 
Straightforward, except for the issue of how to reverse LockAdjustment.

With --unlock, a commit that modifies/adds unlocked files gets reverse
adjusted to use locked files. That's fairly reasonable, I think.

But reversing --lock by unlocking all modified files feels wrong. Maybe
that's just because repositories typically seem to still have mostly
locked files in them (unless one is in an adjusted unlocked branch of
course!)

It may be that eventually how to reverse both will need to be configurable,
I don't know.
 2019-09-27 14:23:25 -04:00
Joey Hess
3d5c1a4912
slightly improve design 2019-09-27 12:46:14 -04:00
Joey Hess
e5cb4f9dee
thoughts 2019-09-27 12:17:16 -04:00
Ilya_Shlyakhter
f9aae586a8 Added a comment 2019-09-25 17:34:33 +00:00
Joey Hess
789721b65f
Merge branch 'master' of ssh://git-annex.branchable.com 2019-09-24 14:46:43 -04:00
Joey Hess
6ae0a44c64
git-lfs: Added support for http basic auth 2019-09-24 14:46:20 -04:00
kyle
b2a2bc9597 Added a comment: git docs on "%f" 2019-09-24 18:10:16 +00:00
Ilya_Shlyakhter
dd99dfa301 added suggestion to only enable clean/smudge filter for unlocked annexed files 2019-09-24 17:33:06 +00:00
Joey Hess
9b5f014238
todo 2019-09-24 11:42:20 -04:00
Joey Hess
5e787b936b
Merge branch 'master' of ssh://git-annex.branchable.com 2019-09-21 21:27:06 -04:00
Yaroslav Halchenko
6eb8784e7c
Embed projects/{datalad,repronim} tags for the past todo/bugs 
Assignment between datalad and repronim might not be 100% right but
approx there depending on the date

Here is the command and its output:

	$> git grep -l '\[\[!meta author=\(yoh\|ben\|yarikoptic\|kyle\)\]\]' | grep -v comment | grep -v assure_correct_names___40 | while read f; do d=$(git log --date=iso $f| awk '/^Date/{print $2;}' | tail -n 1 ); p=datalad; if [[ "$d" =~ 2019-.* ]] || [[ "$d" =~ 2018-1.* ]]; then p=repronim; fi; echo $d $p: $f; sed -i -e "s,\(\[\[\!meta author.*\),\1\n\[\[\!tag projects/$p\]\],g" $f; done
	2017-10-10 datalad: doc/bugs/--shared_setting_of_git_causes_annex__39__ed_files_to_be_writeable__33__.mdwn
	2017-08-12 datalad: doc/bugs/2_ssh_connection_prompts_for_password.mdwn
	2018-10-22 repronim: doc/bugs/3_tests_fail_while_trying_to_build_fresh_git_annex_6.20181011+git109-gff9ba1f4d.mdwn
	2018-12-20 repronim: doc/bugs/7.20181211+git29-gab4a1bed9_fails_tests_during_neurodebian_build.mdwn
	2016-03-12 datalad: doc/bugs/OSX__58___addurl_--batch_--json_spits_out_shortened_output_string__dies_off_with_4.mdwn
	2018-11-16 repronim: doc/bugs/Please_include_version_of_used_to_build_ghc_within___34__git_annex_version__34__.mdwn
	2016-03-10 datalad: doc/bugs/R__47__O_permissions_are_stalking_me_somehow.mdwn
	2019-07-03 repronim: doc/bugs/Regression_in___96__find_--json__96___output.mdwn
	2017-01-27 datalad: doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes.mdwn
	2018-12-11 repronim: doc/bugs/Unable_to_addurl_file__58____47____47____47___on_Windows.mdwn
	2019-04-10 repronim: doc/bugs/Unable_to_get__47__addurl_to_http_link__58___download_failed__58___InvalidHeader___34__preload__34__.mdwn
	2019-09-19 repronim: doc/bugs/add_--json_should_provide_a___34__note__34___with_an_error_message_in_case_of_an_error.mdwn
	2016-11-20 datalad: doc/bugs/adds_file_destined_for_annex_into_git_in___39__addurl__39__.mdwn
	2016-01-12 datalad: doc/bugs/addurl_--batch__--with-files_doesn__39__t_add_file_into_git_until_pipe_is_closed.mdwn
	2016-01-15 datalad: doc/bugs/addurl_--batch___40__--json_or_not__41___doesn__39__t_report_failure_correctly_if_non-annexed_file_exists.mdwn
	2016-05-23 datalad: doc/bugs/addurl_--batch_decides_to_talk_to_ssh_remotes_for_some_reason.mdwn
	2016-01-11 datalad: doc/bugs/addurl_--file__causes_file_redownload_even_if_it_already_present.mdwn
	2019-03-18 repronim: doc/bugs/addurl_fails_with_non-youtube_url_with___34__This_url_is_supported_by_youtube-dl__44___but_youtube-dl_could_potentially_access_any_address__34__.mdwn
	2016-12-04 datalad: doc/bugs/annex_add_ignores_.-prefixed_directories.mdwn
	2017-03-22 datalad: doc/bugs/annex_doesn__39__t_fixup_symlinks_when___34__git_commit_path__95__to__95__repo__34___is_used.mdwn
	2016-07-12 datalad: doc/bugs/annex_drop_fails_to_determine_availability_on_a_http_url_redirecting_to_ftp.mdwn
	2016-04-26 datalad: doc/bugs/annex_drop_is_not___34__in_effect__34___for_load_which_was___34__addurl_--batch__34__ed_but_not_yet_committed.mdwn
	2018-09-25 datalad: doc/bugs/annex_get_-J_16_via_ssh_stalls_.mdwn
	2019-06-05 repronim: doc/bugs/annex_init_no_longer_generates_default_description.mdwn
	2018-03-27 datalad: doc/bugs/annex_metadata___40__not_--batch__39__ed__41___is_not_aware_of_files_added_via_addurls_--batch.mdwn
	2018-11-12 repronim: doc/bugs/annex_sets_remote___40__regular_http__41___to_be_ignored_while_running_in_docker_container.mdwn
	2018-02-23 datalad: doc/bugs/annex_sync___40__in_direct_mode_only__41___does_not_actually_update_local_branch__47__tree.mdwn
	2018-05-04 datalad: doc/bugs/annex_view_barfs__fatal__58___Unable_to_add___40__null__41___to_database.mdwn
	2016-09-09 datalad: doc/bugs/autoenable__61__true_seems_to_not_work_any_longer.mdwn
	2015-11-20 datalad: doc/bugs/autostart__47__git-annex.desktop_gets_resurrected.mdwn
	2016-03-29 datalad: doc/bugs/awkward_error_from_annex_whenever_operating_on_a_clone_with_submodules.mdwn
	2016-11-08 datalad: doc/bugs/cannot___34__install__34___standalone_git_annex_within_afs_mount.mdwn
	2017-03-17 datalad: doc/bugs/cannot___40__or_how__63____41___to_pass_socket_path_with_a_space_in_its_path_via_annex-ssh-options.mdwn
	2018-08-08 datalad: doc/bugs/cannot_commit___34__annex_add__34__ed_modified_file_which_switched_its_largefile_status_to_be_committed_to_git_now.mdwn
	2017-01-20 datalad: doc/bugs/clash_of_-j__in_copy_for_--json_--json-progress.mdwn
	2017-03-01 datalad: doc/bugs/copy_does_not_reflect_some_failed_copies_in_--json_output.mdwn
	2017-02-10 datalad: doc/bugs/could_standalone_copy_of_git_ship__47__deploy_hooks_samples_as_well_as_stock_git__63__.mdwn
	2015-11-12 datalad: doc/bugs/does_not_complain__47__fail_if_by_mistake_option_value_passed_within_arg_for__-c.mdwn
	2016-07-14 datalad: doc/bugs/drop_blows_on_lustre__58___SQLite3_returned_ErrorIO.mdwn
	2016-04-29 datalad: doc/bugs/dropkey_--batch_--json_--force_is_always_succesfull.mdwn
	2016-02-24 datalad: doc/bugs/duplicate_progress_reports_in_parallel___39__get__39__.mdwn
	2016-04-27 datalad: doc/bugs/encfs_support_--_shouldn__39__t_it_be_treated_as_crippled_already__63__.mdwn
	2019-08-03 repronim: doc/bugs/fails_to___96__get__96___in_parallel_for_a_freshly_clone_from_public_s3_bucket_where_versioning_info_was_forgotten.mdwn
	2019-09-02 repronim: doc/bugs/fails_to_init_under_a_directory_with_a___34__tricky__34___name.mdwn
	2017-07-26 datalad: doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine.mdwn
	2019-08-13 repronim: doc/bugs/fresh_build_for_neurodebian__58___test_failure.mdwn
	2017-09-13 datalad: doc/bugs/fsck_does_not_detect__47__fix_some_key_directories_correctly.mdwn
	2017-07-30 datalad: doc/bugs/get_--json_fails_whenever_plain_get_works___40__with_https_urls__41__.mdwn
	2018-03-09 datalad: doc/bugs/get_-JX__58____transfer_already_in_progress_..._for_some_files.mdwn
	2017-08-30 datalad: doc/bugs/get_-J___34__fails__34___to_get_files_with_the_same_key.mdwn
	2017-03-15 datalad: doc/bugs/get_-J_cannot_be_used_with_password-based_authentication.mdwn
	2018-02-10 datalad: doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__.mdwn
	2019-07-11 repronim: doc/bugs/get_fails_to_place_v7_unlocked_file_content_into_the_file_tree_in_v7_in_repo_with_detached_HEAD.mdwn
	2018-05-06 datalad: doc/bugs/get_from_the_web_remote_fails___40__redirect__63____41___-_lack_of_further_debug_info.mdwn
	2018-09-22 datalad: doc/bugs/get_over_ssh_fails_with___fd__58__19__58___hClose__58___resource_vanished.mdwn
	2016-03-12 datalad: doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock.mdwn
	2019-03-18 repronim: doc/bugs/git-annex_precommit_.__gets_stuck_after_likely_interrupted___96__annex_add__96___call.mdwn
	2016-06-02 datalad: doc/bugs/git_annex_adjust_--unlock_seems_to_cause_migration_of_a_file_to_another_backend.mdwn
	2017-02-06 datalad: doc/bugs/git_annex_info_fails_on_NFS__58___waitToSetLock__58___resource_exhausted___40__No_locks_available__41__.mdwn
	2017-10-25 datalad: doc/bugs/graft__47__graft_cleanup_commits_--_really_needed__63__.mdwn
	2018-03-05 datalad: doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__.mdwn
	2018-09-25 datalad: doc/bugs/impossible_to_login_to_the_website_at_times.mdwn
	2018-02-20 datalad: doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions.mdwn
	2016-01-11 datalad: doc/bugs/inconsistent_output_upon_addurl_--batch_complicates_if_not_forbids_reliable_parsing_of_output.mdwn
	2015-12-21 datalad: doc/bugs/info_--json_lists_backend_usage_stats_as_a_list_of_lists.mdwn
	2018-10-11 repronim: doc/bugs/initial_get_-J_on_exported_S3_bucket_fails.mdwn
	2016-01-28 datalad: doc/bugs/lookupkey_started_to_spit_out___34__debug__34___messages_to_stdout.mdwn
	2016-02-09 datalad: doc/bugs/man_page_for_command_misses_actual_command_in_the_synopsis_for_git-annex-checkpresentkey.mdwn
	2018-11-08 repronim: doc/bugs/multiple_ssh_prompts__44___and_thread_blocked_indefinitely_in_an___63____63____63___transaction.mdwn
	2017-02-17 datalad: doc/bugs/mysterious_dependency_of_git_annex_status_output_of_the_added_file.mdwn
	2016-01-15 datalad: doc/bugs/new_whereis_--json_lost_information_about_web_urls_if_other_special_remotes_provide_them.mdwn
	2017-05-25 datalad: doc/bugs/parallel_get_can_fail_some_downloads_and_require_re-getting_.mdwn
	2016-05-01 datalad: doc/bugs/prematurely___40__can__39__t_check_offline__41___marks_remote_as_annex-ignore.mdwn
	2018-05-08 datalad: doc/bugs/regression_-_fails_to_drop_._Exit_code_11_wo_--debug__44___and_1_with_--debug.mdwn
	2017-12-06 datalad: doc/bugs/regression_-_yt__58___prefix_for___34__regular__34___urls.mdwn
	2019-08-23 repronim: doc/bugs/regression__58___fails_to_detect_need_for_pidlock_on_an_NSF_mount.mdwn
	2019-05-09 repronim: doc/bugs/regression__58___http_downloads_redirecting_to_ftp_are_no_longer_supported.mdwn
	2016-10-17 datalad: doc/bugs/remote_repository_must_be_version_6_as_well_to_have_copied_files_appear_correctly.mdwn
	2016-08-05 datalad: doc/bugs/reports_success_when_addurl_--batch__a_file_which_is_.gitignore__39__d.mdwn
	2017-09-26 datalad: doc/bugs/set_metadata_leaks_from_one___40__staged__41___key_to_another_during_rename_of_file.mdwn
	2016-02-15 datalad: doc/bugs/shouldn__39__t_keep_permissions_of_the_ssh_remote__63__.mdwn
	2016-03-09 datalad: doc/bugs/some_tests_fail_while_running_under_NFS.mdwn
	2018-09-18 datalad: doc/bugs/some_transfers_for_get_-J4_fail_over_ssh__47__rsync_and_require_retry.mdwn
	2018-09-25 datalad: doc/bugs/standalone_build_is_slightly_out_of_date.mdwn
	2015-10-28 datalad: doc/bugs/standalone_builds_shouldn__39__t_pollute___126____47__.ssh_with_helpers_merely_upon_annex_init.mdwn
	2016-05-25 datalad: doc/bugs/strips___95___from_extensions_in_E_backends__63__.mdwn
	2016-03-01 datalad: doc/bugs/thread_blocked_indefinitely_in_an_STM_transaction__while_moving_within__a_local_clone.mdwn
	2016-01-15 datalad: doc/bugs/treatment_of_largefiles_is_not_working_for_addurl_--fast___40__or_--relaxed__41__.mdwn
	2018-09-11 datalad: doc/bugs/unable_to_get_from_public_S3_remote_without_clear_reasoning_why.mdwn
	2018-09-25 datalad: doc/bugs/unclear_why_annex_fails_to_get_a_file_from_a_public_S3_bucket_where_it_exported_before.mdwn
	2016-02-08 datalad: doc/bugs/use_of_annex_in_submodule_replaces_.git_with_incorrect_symlink.mdwn
	2016-02-26 datalad: doc/bugs/using_regular_magic_file__warning_pollutes_stderr.mdwn
	2018-09-06 datalad: doc/bugs/v6_-_under_subdir__58___git_add___34__whines__34____44___git_commit___34__blows__34__.mdwn
	2018-09-12 datalad: doc/bugs/whereis_shouldn__39__t_just_crash_if_one_of_the_special_remotes___34__drivers__34___is_not_available.mdwn
	2019-01-21 repronim: doc/bugs/windows__58___autostart_for_annex_64bit_looks_under_x86.mdwn
	2016-01-18 datalad: doc/todo/--batch_for_add.mdwn
	2016-01-18 datalad: doc/todo/--batch_for_find.mdwn
	2016-01-05 datalad: doc/todo/--batch_for_info.mdwn
	2016-10-24 datalad: doc/todo/Allow_for_TRANSFER-SUCCESS_to_report_also_a_URL_where_key_could_now_be_obtained_from.mdwn
	2018-02-05 datalad: doc/todo/INFO_message_for_custom_special_remotes.mdwn
	2018-09-19 datalad: doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__.mdwn
	2016-07-28 datalad: doc/todo/__39__info_filename__39___to_provide_information_either_content_is_locally_present.mdwn
	2019-04-25 repronim: doc/todo/add_--json-progress_to___96__git_annex_add__96__.mdwn
	2016-04-22 datalad: doc/todo/add_option_to_whereis_to_avoid_network_interactions.mdwn
	2017-03-23 datalad: doc/todo/annex_add___40__-u__124__--update__41___mode.mdwn
	2016-08-05 datalad: doc/todo/annex_merge_--remotes.mdwn
	2017-02-06 datalad: doc/todo/be_able_to_specify_custom_commit_message_for_git-annex_branch_commit.mdwn
	2016-02-10 datalad: doc/todo/checkpresentkey_without_explicit_remote.mdwn
	2018-09-28 datalad: doc/todo/config_setting_to_force_--progress_reporting_for_some_git_calls.mdwn
	2018-09-27 datalad: doc/todo/configuration_option_for_default___34__mode__34___on_crippled_file_systems.mdwn
	2016-06-09 datalad: doc/todo/could_standalone___39__fixed__39___git-annex_binaries_be_prelinked__63__.mdwn
	2016-04-25 datalad: doc/todo/drop_--batch.mdwn
	2016-06-16 datalad: doc/todo/get_--batch.mdwn
	2017-07-21 datalad: doc/todo/git_annex_info___60__remote__62___does_not_list_all_the_parameters_for_the_remote.mdwn
	2016-06-09 datalad: doc/todo/git_annex_info_to_include_information_about_repo_version__63__.mdwn
	2019-04-05 repronim: doc/todo/globus_special_remote_as_a___34__transport__34___layer.mdwn
	2018-02-06 datalad: doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn
	2016-07-05 datalad: doc/todo/interface_to_the___34__progress__34___of_annex_operations.mdwn
	2018-05-22 datalad: doc/todo/machine_readable_information_about_reason_of_failure_in_enableremote.mdwn
	2015-11-02 datalad: doc/todo/make_addurl_respect_annex.largefiles_option.mdwn
	2016-02-15 datalad: doc/todo/make_annex_info_more_efficient.mdwn
	2016-05-19 datalad: doc/todo/make_copy_--fast__faster.mdwn
	2016-02-12 datalad: doc/todo/metadata_--batch.mdwn
	2017-02-16 datalad: doc/todo/output_of_wanted___40__and_possibly_group_etc__41___should_not_be_polluted_with___34__informational__34___messages.mdwn
	2014-12-18 datalad: doc/todo/parallel_get.mdwn
	2018-09-28 datalad: doc/todo/prevent_directly_printing_to_stderr_error_from_an_attempt_to_download_remote___47__config_file.mdwn
	2018-08-04 datalad: doc/todo/provide___39__file__39___in_--json-progress_record_for_addurl.mdwn
	2019-01-25 repronim: doc/todo/publicurl_config_for_all_special_remotes.mdwn
	2016-01-19 datalad: doc/todo/return___34__key__34___entry_in_--json_output_for_addurl___40__and_future_add__41___--batch.mdwn
	2019-01-25 repronim: doc/todo/some_way_to_get_a_list_of_options_for_a_special_remote_of_a_given_type.mdwn
	2018-04-28 datalad: doc/todo/support___126____47__.netrc_for_http_access_authentication_specs_as_was_done_when_wget_was_used.mdwn
	2018-01-12 datalad: doc/todo/support_ssh__58____47____47___or_sftp__58____47____47___urls_via___34__built-in__34___ssh_support.mdwn
	2016-08-05 datalad: doc/todo/sync_--branches__to_sync_only_specified_branches___40__e.g._git-annex__41__.mdwn
	2016-10-28 datalad: doc/todo/unlock_--json.mdwn
	2016-10-17 datalad: doc/todo/unlock_--read-only.mdwn
 2019-09-21 21:26:15 -04:00
yarikoptic
191dcb7c8e change from using !meta to !tag for project specification 2019-09-22 00:07:46 +00:00
Ilya_Shlyakhter
2c774b910c Added a comment 2019-09-20 20:46:41 +00:00
Ilya_Shlyakhter
a9d5c08b6a Added a comment 2019-09-20 20:41:25 +00:00
yarikoptic
fccbe038f3 Added a comment: found it 2019-09-20 18:56:52 +00:00
Joey Hess
a3f5158021
responses 2019-09-20 13:01:22 -04:00
Ilya_Shlyakhter
3fadb8e20f Added a comment 2019-09-20 15:58:08 +00:00
Ilya_Shlyakhter
0558c6de79 added suggestion to prevent pushed branches from referencing unavailable files 2019-09-19 21:03:30 +00:00
Ilya_Shlyakhter
f64f6f82d3 removed 2019-09-19 20:17:10 +00:00
Ilya_Shlyakhter
dc4d34355b added todo for optionally auto-locking unlocked files after one edit 2019-09-19 19:57:04 +00:00
Ilya_Shlyakhter
96e4c10cf9 Added a comment 2019-09-19 18:10:36 +00:00
Ilya_Shlyakhter
4f2e58b5cd added suggestion to auto-configure git diff driver 2019-09-19 17:43:41 +00:00
Joey Hess
b13a350556
added --unlocked and --locked 2019-09-19 12:33:13 -04:00
Ilya_Shlyakhter
d1ec2a1faa added todo for finding unlocked files 2019-09-19 01:47:10 +00:00
Ilya_Shlyakhter
31a218a6a3 Added a comment: git-annex REST API 2019-09-18 22:55:06 +00:00
Ilya_Shlyakhter
1fc9cae9fe Added a comment 2019-09-18 21:30:46 +00:00
Ilya_Shlyakhter
177b3f9dce added suggestion to split off clean and smudge filter into separate lightweight executables 2019-09-18 20:59:15 +00:00
Joey Hess
7722125e97
Merge branch 'master' of ssh://git-annex.branchable.com 2019-09-18 13:49:00 -04:00
Joey Hess
31ead4f0f3
comment 2019-09-18 13:17:39 -04:00
yarikoptic
fd3423deba initial TODO on special remotes parameter checking 2019-09-18 17:09:11 +00:00
Joey Hess
10b63050eb
add 2019-09-17 11:56:56 -04:00
Ilya_Shlyakhter
24a256d402 Added a comment 2019-09-16 23:44:48 +00:00
Ilya_Shlyakhter
d98e6a211a Added a comment 2019-09-16 21:12:21 +00:00
Joey Hess
15e35a80f5
comment 2019-09-16 13:26:16 -04:00
Joey Hess
81b77dd73d
comment and todo 2019-09-16 13:17:15 -04:00
forums+git-annex@d2ec818a5e37440c45fa5f3cbd82fe5110b32020
3c6ebf69ae Added a comment 2019-09-13 23:17:37 +00:00
Joey Hess
91caba25be
comment 2019-09-13 18:21:21 -04:00
forums+git-annex@d2ec818a5e37440c45fa5f3cbd82fe5110b32020
378974d502 2019-09-13 20:37:03 +00:00
Joey Hess
eae5e96473
Merge branch 'v7-default' 2019-09-12 13:47:55 -04:00
Joey Hess
5ba16c06ed
re-investigated sqlite encoding issues 
It's really persistent causing the problem, and BLOB really seems the
only way around it.
 2019-09-12 13:06:39 -04:00
Joey Hess
23ca1bcd99
close 2019-09-12 11:52:22 -04:00
Joey Hess
9317c4a5b3
close all old direct mode issues 
except for one issue that was not really specific to direct mode after
all
 2019-09-11 13:44:17 -04:00
Ilya_Shlyakhter
664897e3bf reinit repo with new uuid 2019-09-10 21:14:55 +00:00
Joey Hess
70f02df599
link to datalad issue that has some data collected 2019-09-10 14:50:58 -04:00
Ilya_Shlyakhter
1a7c06ea9d added suggestion for S3 export remote redirecting to key-value store 2019-09-02 17:41:46 +00:00
Joey Hess
31a6086a9a
v7-default branch 2019-08-30 14:14:42 -04:00
Joey Hess
6b27fc58d0
v7 bug review and cleanup 2019-08-28 12:20:30 -04:00
Joey Hess
a648c22b9b
clarification 2019-08-27 16:18:29 -04:00
Joey Hess
c21c0cd885
update with WSL findings 2019-08-27 15:45:43 -04:00
Joey Hess
56370c009c
Merge branch 'master' of ssh://git-annex.branchable.com 2019-08-27 14:14:52 -04:00
Joey Hess
45663f6f35
planning 2019-08-27 14:14:16 -04:00
Joey Hess
6b56c3ae11
close 2019-08-27 11:39:52 -04:00
Joey Hess
0b10d10753
add 2019-08-26 17:00:06 -04:00
Ilya_Shlyakhter
637ced91b9 Added a comment 2019-08-26 20:58:14 +00:00
Joey Hess
20741b1eb4
Automatically convert direct mode repositories to v7 with adjusted unlocked branches 
* Automatically convert direct mode repositories to v7 with adjusted
  unlocked branches and set annex.thin.
* init: When run on a crippled filesystem with --version=5,
  will error out, since version 7 is needed for adjusted unlocked branch.
* direct: This command always errors out as direct mode is no longer
  supported.
* indirect: This command has become a deprecated noop.
* proxy: This command is deprecated because it was only needed in direct
  mode. (But it continues to work.)

Also removed mentions of direct mode throughough the documentation.

I have not removed all the direct mode code yet.
 2019-08-26 15:05:25 -04:00
Joey Hess
2fd27c6df5
assistant: When creating a new repository use v7 adjusted branches with annex.thin 
Rather than direct mode, which this is a small step on the path to
removing.

Init on a crippled filesystem already used v7 adjusted branches,
and like that, this doesn't pose any interoperability issues with old
versions of git-annex that clone the same repo, because files are only
unlocked on the adjusted branch.
 2019-08-26 12:54:14 -04:00
Joey Hess
0c718763fd
planning 
several entangled things
 2019-08-26 12:29:43 -04:00
Joey Hess
9bf3a5c405
close as probably fixed in recent test suite improvements 2019-08-26 12:27:27 -04:00
Joey Hess
b627c5b4d0
followup 2019-08-26 12:20:57 -04:00
Joey Hess
cb318d4404
todo 2019-08-23 16:52:08 -04:00
Joey Hess
0dd6ba302e
close 2019-08-05 13:44:24 -04:00
Joey Hess
74a36b8262
comment 2019-08-01 12:36:06 -04:00
Joey Hess
3d301dfb9f
comment 2019-08-01 12:26:28 -04:00
Joey Hess
018b5b8173
Support building with socks-0.6 and persistant-template-2.7 
persistent-template now needs UndecidableInstances.

socks changed defaultSocksConf to take a SockAddr.
 2019-07-30 12:50:48 -04:00
Joey Hess
c634dae66b
wacky ideas 2019-07-30 12:13:00 -04:00
Joey Hess
ee3885d152
git-lfs branch 2019-07-29 15:48:52 -04:00
Joey Hess
724ebec3f0
lfs deep dive 2019-07-29 12:28:07 -04:00
Ilya_Shlyakhter
5759065e18 added suggestion to let git-annex-export export non-annexed files 2019-07-26 00:36:18 +00:00
Dan
8fcf9ff7c6 Added a comment: git annex sync commits changes, too? 2019-07-22 17:25:39 +00:00
Dan
adc2ac1a89 Added a comment: Still Wanted 2019-07-22 17:06:21 +00:00
Joey Hess
875c7b5cc9
windows long filenames should be fixed now by new ghc 2019-07-22 09:44:09 -04:00
Joey Hess
7ea98e00d0
todo 2019-07-19 14:49:26 -04:00
Joey Hess
05e3b93b89
comment 2019-07-19 13:22:09 -04:00
Joey Hess
cf6152575a
comment, open related todo 2019-07-19 13:10:17 -04:00
Ilya_Shlyakhter
8ba9aae481 added a request for more extensive retrying of transiently-failing operations 2019-07-08 16:30:37 +00:00
Joey Hess
8f26422943
Merge branch 'master' of ssh://git-annex.branchable.com 2019-07-05 09:49:57 -04:00
Ilya_Shlyakhter
1c25bfc88c Added a comment: status of v7 2019-07-02 18:45:52 +00:00
Ilya_Shlyakhter
47f16c8915 Added a comment 2019-07-02 18:36:16 +00:00
yarikoptic
067050c5c7 Added a comment: I wonder if "thin mode" could generalize beyond hardlinks 2019-07-01 16:19:47 +00:00
Ilya_Shlyakhter
607bb3c208 Added a comment: verifying and external backends 2019-06-28 17:14:18 +00:00
Ilya_Shlyakhter
3388722110 Added a comment: dockerized external backends 2019-06-28 16:36:10 +00:00
Ilya_Shlyakhter
24378d94e6 Added a comment: read-only unlock of only some files 2019-06-28 15:57:05 +00:00
Joey Hess
dfe30c8a6c
todo 2019-06-28 10:59:20 -04:00
Joey Hess
c05c73657c
Merge branch 'master' of ssh://git-annex.branchable.com 2019-06-27 19:08:46 -04:00
Joey Hess
a9f7e013a0
todo 2019-06-27 19:08:27 -04:00
Joey Hess
fb72f4db64
comment 2019-06-27 10:44:35 -04:00
Ilya_Shlyakhter
b168a06b82 Added a comment 2019-06-26 20:38:19 +00:00
Ilya_Shlyakhter
8ae3197940 Added a comment: some todos that external backends would obviate (fix formatting, add one) 2019-06-26 20:25:51 +00:00
Ilya_Shlyakhter
13794595d6 Added a comment 2019-06-26 20:15:19 +00:00
Joey Hess
732f03b202
comment 2019-06-26 11:58:53 -04:00
Joey Hess
3bb8b62699
comments 2019-06-26 11:23:41 -04:00
Joey Hess
42c386fc47
add: Display progress meter when hashing files. 
* add: Display progress meter when hashing files.
* add: Support --json-progress option.
 2019-06-25 13:12:47 -04:00
Joey Hess
191bdaafc5
comment 2019-06-25 11:08:45 -04:00
Joey Hess
9d36c826c0
use fine-grained WorkerStages when transferring and verifying 
This means that Command.Move and Command.Get don't need to
manually set the stage, and is a lot cleaner conceptually.

Also, this makes Command.Sync.syncFile use the worker pool better.
In the scenario where it first downloads content and then uploads it to
some other remotes, it will start in TransferStage, then enter VerifyStage
and then go back to TransferStage for each transfer to the remotes.
Before, it entered CleanupStage after the download, and stayed in it for
the upload, so too many transfer jobs could run at the same time.

Note that, in Remote.Git, it uses runTransfer and also verifyKeyContent
inside onLocal. That has a Annex state for the remote, with no worker pool.
So the resulting calls to enteringStage won't block in there.

While Remote.Git.copyToRemote does do checksum verification, I
realized that should not use a verification slot in the WorkerPool
to do it. Because, it's reading back from eg, a removable disk to checksum.
That will contend with other writes to that disk. It's best to treat
that checksum verification as just part of the transer. So, removed the todo
item about that, as there's nothing needing to be done.
 2019-06-19 13:24:20 -04:00
Joey Hess
e19408ed9d
Merge branch 'master' of ssh://git-annex.branchable.com 2019-06-17 15:26:57 -04:00
Joey Hess
04cc470201
run download checksum verification in separate job pool 
get, move, copy, sync: When -J or annex.jobs has enabled concurrency,
checksum verification uses a separate job pool than is used for
downloads, to keep bandwidth saturated.

Not yet done for upload checksum verification, but that only affects
remotes on local disks.
 2019-06-17 14:58:02 -04:00
Joey Hess
1a8d06d251
thought 2019-06-17 11:50:18 -04:00
jsag@f84637fe752e0235291a118b1cd007bafad0997e
ae9f2d5e6a 2019-06-17 12:43:17 +00:00
Joey Hess
e589a9b3fc
moving this to a bug 2019-06-12 15:00:14 -04:00
Joey Hess
8e5ea28c26
finish CommandStart transition 
The hoped for optimisation of CommandStart with -J did not materialize.
In fact, not runnign CommandStart in parallel is slower than -J3.
So, CommandStart are still run in parallel.

(The actual bad performance I've been seeing with -J in my big repo
has to do with building the remoteList.)

But, this is still progress toward making -J faster, because it gets rid
of the onlyActionOn roadblock in the way of making CommandCleanup jobs
run separate from CommandPerform jobs.

Added OnlyActionOn constructor for ActionItem which fixes the
onlyActionOn breakage in the last commit.

Made CustomOutput include an ActionItem, so even things using it can
specify OnlyActionOn.

In Command.Move and Command.Sync, there were CommandStarts that used
includeCommandAction, so output messages, which is no longer allowed.
Fixed by using startingCustomOutput, but that's still not quite right,
since it prevents message display for the includeCommandAction run
inside it too.
 2019-06-12 13:24:01 -04:00
Joey Hess
3893d84764
todo 2019-06-06 12:02:27 -04:00
Joey Hess
3eac4e01a4
idea 2019-06-05 19:43:01 -04:00
Joey Hess
659640e224
separate queue for cleanup actions 
When running multiple concurrent actions, the cleanup phase is run in a
separate queue than the main action queue. This can make some commands
faster, because less time is spent on bookkeeping in between each file
transfer.

But as far as I can see, nothing will be sped up much by this yet, because
all the existing cleanup actions are very light-weight. This is just groundwork
for deferring checksum verification to cleanup time.

This change does mean that if the user expects -J2 will mean that they see no
more than 2 jobs running at a time, they may be surprised to see 4 in some
cases (if the cleanup actions are slow enough to notice).

It might also make sense to enable background cleanup without the -J,
for at least one cleanup action. Indeed, that's the behavior that -J1
has now. At some point in the future, it make make sense to make the
behavior with no -J the same as -J1. The only reason it's not currently
is that git-annex can build w/o concurrent-output, and also any bugs
in concurrent-output (such as perhaps misbehaving on non-VT100 compatible
terminals) are avoided by default by only using it when -J is used.
 2019-06-05 17:54:35 -04:00
Joey Hess
7dcc815c29
more thoughts 2019-06-04 14:38:55 -04:00
Joey Hess
cd20dc4158
thoughts 2019-06-04 14:13:15 -04:00
Joey Hess
cd5e8be2dc
comment 2019-05-23 13:36:59 -04:00
Joey Hess
e06feb7316
honor preferred content when importing 
Importing from a special remote honors its preferred content too; unwanted
files are not imported. But, some preferred content expressions can't be
checked before files are imported, and trying to import with such an
expression will fail.

Tested this with scenarios including changing the preferred content
expression and making sure merging the import didn't delete files that were
no longer wanted.

There was one minor inefficiency mentioned in the todo that I punted on.
 2019-05-21 14:38:06 -04:00
Joey Hess
ec11575d17
hairyness 2019-05-21 12:54:57 -04:00
Joey Hess
3b9a19171a
Merge branch 'master' into preferred 2019-05-21 11:34:45 -04:00
Joey Hess
5e1221ad53
Improve shape of commit tree when importing from unversioned special remotes 
Make the import have the previous import as a parent, so eg `git log --stat`
displays a useful diff.

Also a minor optimisation, only calculate the depth of the imported history
once.
 2019-05-21 11:32:54 -04:00
Joey Hess
5af9e7f3d0
break out a todo 2019-05-21 11:10:13 -04:00
Joey Hess
97fd9da6e7
add back non-preferred files to imported tree 
Prevents merging the import from deleting the non-preferred files from
the branch it's merged into.

adjustTree previously appended the new list of items to the old, which
could result in it generating a tree with multiple files with the same
name. That is not good and confuses some parts of git. Gave it a
function to resolve such conflicts.

That allowed dealing with the problem of what happens when the import
contains some files (or subtrees) with the same name as files that were
filtered out of the export. The files from the import win.
 2019-05-20 16:43:52 -04:00
Joey Hess
7d177b78e4
docs for export preferred content 
This includes a note about how include= and exclude= match when exporting
a subtree. I don't know if the note is prominent enough, but the
behavior seems unsurprising enough.
 2019-05-20 12:06:02 -04:00
Joey Hess
12451ea010
Merge branch 'master' into preferred 2019-05-20 10:00:03 -04:00
Joey Hess
8958556fe3
thought 2019-05-16 20:41:17 -04:00
Joey Hess
24c8b1b15a
update 2019-05-14 15:25:09 -04:00
Joey Hess
9411a7c93c
matching preferred content before key is known 
This will let import try to match preferred content expressions before
downloading the content and generating its key.

If an expression needs a key, it preferredContentParser with
preferredContentKeylessTokens will fail to parse it.

standard and groupwanted are not in preferredContentKeylessTokens
because they may refer to an expression that refers to a key.
That needs further work to support them.
 2019-05-14 14:28:23 -04:00
Joey Hess
a3e24ed533
more design work 2019-05-14 11:49:23 -04:00
Joey Hess
c5a61ee808
closing in on final design for this 2019-05-14 10:52:00 -04:00
Ilya_Shlyakhter
0610789285 Added a comment: checksums of remote data 2019-05-13 22:03:38 +00:00
Joey Hess
0c7569bb6f
close 2019-05-10 13:59:39 -04:00
Joey Hess
c77d79d343
close old todo 2019-05-10 13:54:32 -04:00
Joey Hess
ae562ad4d7
update old todo item with what still needs doing 
removed old comments that are no longer relevant
 2019-05-10 13:52:40 -04:00
Joey Hess
daa0c6c1c6
close old todo 2019-05-10 13:35:55 -04:00
Joey Hess
d32143e7ad
close 2019-05-10 13:34:44 -04:00
Joey Hess
ccfb800fa6
Merge branch 'master' of ssh://git-annex.branchable.com 2019-05-10 13:31:49 -04:00
Joey Hess
82186ca58f
annex.jobs=cpus etc 
Added the ability to run one job per CPU (core), by setting annex.jobs=cpus,
or using option --jobs=cpus or -Jcpus.

Built with future expansion in mind, including not defaulting matching on
Concurrency so more constructors can later be added, and using "cpu"
instead of "0".
 2019-05-10 13:27:08 -04:00
Ilya_Shlyakhter
e0c73c7f29 Added a comment 2019-05-09 21:07:39 +00:00
Ilya_Shlyakhter
5638ae9688 Added a comment 2019-05-07 00:59:31 +00:00
Joey Hess
b03e65d260
Improved locking when multiple git-annex processes are writing to the .git/index file 2019-05-06 15:15:12 -04:00
Joey Hess
4bc99e4c21
add todo 2019-05-06 14:58:59 -04:00
Joey Hess
6845c1e020
comment 2019-05-06 12:16:19 -04:00
Ilya_Shlyakhter
437fa438e3 Added a comment 2019-05-03 16:31:18 +00:00
Ilya_Shlyakhter
6535d0c1b2 Added a comment 2019-05-03 16:26:40 +00:00
Ilya_Shlyakhter
03a20b225a Added a comment 2019-05-03 16:11:06 +00:00
Joey Hess
40c749387f
comment 2019-05-03 11:53:03 -04:00
Ilya_Shlyakhter
64bcaff016 added todo for speculate-can-get : extension of speculate-present 2019-05-03 15:34:41 +00:00
Joey Hess
700a3f2787
Merge branch 'master' into import-from-s3 2019-05-01 14:30:52 -04:00
Joey Hess
a405ae015d
remove simple fast-forward todo 
I think the history looks nice enough without that special case.
 2019-05-01 14:29:52 -04:00
Joey Hess
a32f31235a
reuse old imported commits 
This avoids proliferation of different import commits for the same
trees, and makes the resulting git history nice.
 2019-05-01 14:20:26 -04:00
Joey Hess
83a420dd66
update todo 2019-04-30 16:31:46 -04:00
Joey Hess
1503b86a14
make import tree from remote generate a merge commit 
This way no history is lost, neither what was exported to the remote,
or the history of changes that is imported from it. No complicated
correlation of two possibly very different histories is needed, just
record what we know and then git merge will do a good job.

Also, it notices when the remote tracking branch doesn't need to be updated,
and avoids doing anything, so noop remotes are super cheap.

The only catch here is that, since the commits generated for imports
from the remote don't have a stable date or author/committer, each
(non-noop) import generates different commits for the same imported
trees. So, when the imported remote tracking branch is merged into master
and then a change is imported again, there will be an extra series of
commits, which will get more and more expensive each time.

This seems to call for making stable commits for imports. Also that
seems a good idea to make importing in several repositories have the
same result.
 2019-04-30 16:13:21 -04:00
Joey Hess
cd5e685fd1
comment 2019-04-26 10:18:55 -04:00
Joey Hess
3e85707ccf
Merge branch 'master' of ssh://git-annex.branchable.com 2019-04-26 10:17:15 -04:00
yarikoptic
b71a1b5483 wishlist for add --json-progress 2019-04-25 16:33:19 +00:00
Joey Hess
2a6824bf9d
close 2019-04-25 10:49:55 -04:00
Joey Hess
ca385a09c1
rename problem 2019-04-24 15:52:05 -04:00
Joey Hess
5b09b016fe
update 2019-04-24 15:22:02 -04:00
Ilya_Shlyakhter
ae04ab3b91 re: backend variants that compute checksum of chunk checksums 2019-04-24 17:40:13 +00:00
Joey Hess
2d0dd34916
initial work toward correctly merging deeper import histories 
Pure code is tested working, including with even histories that merge
several lines of development. Needs to be hooked up to git histories
next.
 2019-04-23 16:34:19 -04:00
Joey Hess
48d30d8753
Merge branch 'master' into import-from-s3 2019-04-23 15:34:26 -04:00
Joey Hess
c3f5e7863c
some more todos 2019-04-23 15:34:11 -04:00
Joey Hess
8d01b00507
update status 2019-04-23 14:50:33 -04:00
Joey Hess
a42e7a012a
refuse unsafe store to unversioned exporttree with old aws version 
I've developed a patch to aws, once it gets merged, the real version
number of aws can be filled in.
 2019-04-23 14:39:30 -04:00
Joey Hess
ae21c88640
tested S3 import/export with versioned bucket 
rename and delete working
 2019-04-23 13:43:41 -04:00
Joey Hess
0c878899ea
update status 2019-04-23 13:21:38 -04:00
Ilya_Shlyakhter
4c79f2b4ac added suggestion to use git-replace for better implement git-annex-migrate 2019-04-22 01:26:55 +00:00
Joey Hess
2f79cb4b45
versioned import from S3 is working 
Still some bugs and two stubbed methods to implement though.
 2019-04-19 15:13:49 -04:00
Joey Hess
55a5d9679a
implemented mkImportableContentsVersioned 2019-04-19 13:39:33 -04:00
Joey Hess
1968f6d9c6
designing S3 GetBucketObjectVersions to ImportableContents algo 
I think I have a good algo now, at least poorly explained in English..
 2019-04-18 16:25:04 -04:00
Joey Hess
2f740d14da
hmm 2019-04-16 13:18:59 -04:00
Joey Hess
a474304f1d
Merge branch 'master' of ssh://git-annex.branchable.com 2019-04-15 13:49:16 -04:00
Joey Hess
c0c38e986d
added renameremote command 2019-04-15 13:49:03 -04:00
Joey Hess
de7a510da1
update 2019-04-15 13:00:46 -04:00
Joey Hess
00b1943927
close 2019-04-15 12:59:39 -04:00
Joey Hess
72b01b0faf
todo 2019-04-15 12:55:56 -04:00
Joey Hess
40fe5e8927
todo 2019-04-12 11:49:38 -04:00
Ilya_Shlyakhter
9a7cef06e3 added suggestion for git-annex-get --batch --key 2019-04-11 23:41:17 +00:00
Joey Hess
d3d6a45918
thoughts 2019-04-10 12:01:52 -04:00
Joey Hess
7b6d0da9b8
adb import 
As well as adding the necessary methods, a few other changes to the adb
remote:

* Use ".annextmp" extension for temp files, to avoid conflict with other
  temp files.
* Stop using "echo $?" to get exit status of command inside adb.
  There were two problems; first the "echo" just before it meant it was
  always 0! And secondly, it seems kind of random on my phone whether it's
  1 or 0, not dependant on whether the command seems to have succeeded.
 2019-04-09 17:52:41 -04:00
Joey Hess
7bf18f23e5
todo 2019-04-09 14:07:47 -04:00
Joey Hess
0a14dfd383
comment 2019-04-09 11:08:18 -04:00
Joey Hess
4af55c1f30
Merge branch 'master' of ssh://git-annex.branchable.com 2019-04-05 11:41:46 -04:00
yarikoptic
7db8eaf512 initial question about possible "globus" special remote 2019-04-05 02:25:12 +00:00
Joey Hess
1f3245ddf5
close as basis of this is wrong 2019-04-04 12:50:55 -04:00
Joey Hess
727ac0451a
comment 2019-04-03 13:14:54 -04:00
Joey Hess
bc302b56ae
test patch 2019-03-28 16:16:28 -04:00
Joey Hess
c68ae14268
further thought 2019-03-28 15:46:14 -04:00
Joey Hess
b09c6e3016
todo item based on behavior yoh showed me 2019-03-28 14:04:20 -04:00
Joey Hess
e035bc5324
minor typos 2019-03-27 11:15:20 -04:00
Ilya_Shlyakhter
1c334f74d6 Added a comment 2019-03-26 18:27:04 +00:00
Ilya_Shlyakhter
9b4e06d8c2 fixed a typo 2019-03-26 17:51:19 +00:00
Ilya_Shlyakhter
38669f0817 Added a comment: simplifying the interface 2019-03-26 17:40:33 +00:00
Ilya_Shlyakhter
72b788dfaf re: documenting git-annex dependencies 2019-03-24 18:47:28 +00:00
Joey Hess
9ada4b38c1
comment 2019-03-22 10:30:22 -04:00
Joey Hess
59c8119b2a
comment 2019-03-22 10:18:07 -04:00
Joey Hess
5fea7efee7
comment and toddo 2019-03-22 09:23:31 -04:00
Ilya_Shlyakhter
438ff50013 Added a comment 2019-03-19 20:40:50 +00:00
Joey Hess
caa5cca47f
Merge branch 'master' of ssh://git-annex.branchable.com 2019-03-19 15:57:19 -04:00
Ilya_Shlyakhter
10291efa55 added suggestion for git-annex-test to do simple sanity checks on external utils before running tests that depend on them 2019-03-19 18:55:59 +00:00
Joey Hess
2118006711
comment 2019-03-19 13:53:32 -04:00
Joey Hess
1c2f6aee11
comment 2019-03-19 13:45:28 -04:00
Joey Hess
738325ffb7
comment 2019-03-19 13:40:46 -04:00
Joey Hess
cce81f9b0c
comment 2019-03-19 13:34:45 -04:00
Joey Hess
059ff44e8b
comment 2019-03-19 13:28:08 -04:00
Ilya_Shlyakhter
56fd24de0d added suggestion for importing files from special remotes without downloading them 2019-03-12 17:18:28 +00:00
Joey Hess
91f0f76252
need to write a tip 
deferred because a tip that is limited to the directory special remote
is not very interesting to most
 2019-03-11 14:30:05 -04:00
Joey Hess
ffec06aaec
update 2019-03-11 14:13:58 -04:00
Joey Hess
057999f0fc
fix sync --content with remote.name.annex-tracking-branch=master:subdir 
It was exporting the whole tree not just the subdir. Now tested fully
working in both directions.
 2019-03-11 14:07:52 -04:00
Joey Hess
e1fdc8b374
record only subtree in export db and log after importing a subtree 2019-03-11 13:45:01 -04:00
Joey Hess
e46e40bf05
fix update of export tracking branch when exporting branch:subdir 2019-03-11 13:44:12 -04:00
Joey Hess
2912429640
better indicate when special remotes do not support renameExport 
Avoid a warning message when renameExport is not supported, and just
fallback to deleting with a subsequent re-upload. Especially needed for
importtree remotes, where renameExport needs to be disabled.

This changes the external special remote protocol, but in a
backwards-compatible way. A reply of UNSUPPORTED-REQUEST to an older
version of git-annex will cause it to make renameExport return False.
 2019-03-11 12:53:24 -04:00
Joey Hess
c755788256
sync: import when annex-tracking-branch is configured 
This works, and tested syncing both gets changes from a special remote
and sends changes to it, keeping it fully in sync nicely!

But have not tried it with a subdir configured.
 2019-03-09 13:57:49 -04:00
Joey Hess
6149a3b9bd
added progress bars 
clean up old todo item I checked earlier, see commit
dec30d2b14
 2019-03-08 12:43:03 -04:00
Joey Hess
e412129523
concurrency and status messages when downloading from import 2019-03-08 12:33:44 -04:00
Joey Hess
ee5f1422df
remove debug print 2019-03-07 16:08:58 -04:00
Joey Hess
7e35c81ada
locking problem 2019-03-07 15:22:23 -04:00
Joey Hess
9a72785307
fixes to export db lookup when accessing importtree=yes 
Now in a fresh clone with a importtree=yes remote enabled,
git annex fsck --from the remote works.
 2019-03-07 14:10:56 -04:00
Joey Hess
93025dd59f
add missing locking of ContentIdentifier database when writing 
This is not super efficient; it would be better to lock the database
once and build up a queue of changes and flush once.

But, storeExportWithContentIdentifier is likely going to be the really
expensive part, so let's do the simple thing and only optimise later if
needed.
 2019-03-07 13:32:33 -04:00
Joey Hess
3f449f845e
update 2019-03-07 13:28:18 -04:00
Joey Hess
68d1661251
cross-repo import now working correctly 2019-03-07 12:31:35 -04:00
Joey Hess
ee251b2e2e
implement updating the ContentIdentifier db with info from the git-annex branch 
untested

This won't be super slow, but it does need to diff two likely large
trees, and since the git-annex branch rarely sits still, it will most
likely be run at the beginning of every import.

A possible speed improvement would be to only run this when the database
did not contain a ContentIdentifier. But that would only speed up
imports when there is no new version of a file on the special remote,
at most renames of existing files being imported.

A better speed improvement would be to record something in the git-annex
branch that indicates when an import has been run, and only do the diff
if the git-annex branch has record of a newer import than we've seen
before. Then, it would only run when there is in fact new
ContentIdentifier information available from a remote. Certianly doable,
but didn't want to complicate things yet.
 2019-03-06 18:04:30 -04:00
Joey Hess
dec30d2b14
updates 
Note that I tried an evil remote that lists ImportLocations with
../../../ in them and indeed this resulted in git blowing up and the
import failing, and not writing outside the repo.
 2019-03-06 17:07:36 -04:00
Joey Hess
8e9713b769
add export+import test case 2019-03-06 16:49:33 -04:00
Joey Hess
b3d30e7d70
remove unncessary locking of ContentIdentifier db 
Remote.Helper.ExportImport only reads from it, and locking is only
needed when writing.
 2019-03-06 14:36:57 -04:00
Joey Hess
f957f64278
add todo 2019-03-06 11:24:06 -04:00
Joey Hess
0db393d82f
add bug 2019-03-05 17:19:26 -04:00
Joey Hess
5767b1b00d
avoid updating tracking branch when transfer to export throws exception 2019-03-05 16:51:13 -04:00
Joey Hess
dc278c059c
fix STM crash 
git-annex: thread blocked indefinitely in an STM transaction
failed

git-annex: sqlite query crashed
CallStack (from HasCallStack):
  error, called at ./Database/Handle.hs:98:42 in main:Database.Handle
failed

This needs further investigation.
 2019-03-05 16:37:40 -04:00
Joey Hess
46d33e804a
added checkPresentExportWithContentIdentifier 
Ugh, don't like needing to add this, but I can't see a way around it.
 2019-03-05 16:03:03 -04:00
Joey Hess
fd2a1aaa17
avoid using renameExport on import remotes 2019-03-05 14:57:48 -04:00
Joey Hess
9df9a3f82b
more todo 2019-03-05 14:55:22 -04:00
Joey Hess
8c54604e67
import+export from directory special remote fully working 
Had to add two more API calls to override export APIs that are not safe
for use in combination with import.

It's unfortunate that removeExportDirectory is documented to be allowed
to remove non-empty directories. I'm not entirely sure why it's that
way, my best guess is it was intended to make it easy to implement with
just rm -rf.
 2019-03-05 14:20:14 -04:00
Joey Hess
554b7b7f3e
fix todo 2019-03-04 18:20:12 -04:00
Joey Hess
cd3a2b023a
initial try at using storeExportWithContentIdentifier 
Untested, and I'm not sure about the locking of the ContentIdentifier db.
 2019-03-04 17:50:41 -04:00
Joey Hess
aaacf431d8
handle importtree=yes config 
For now, it's only allowed when exporttree=yes is also set.
That simplified the implementation, but could later be changed if
there's a remote that makes sense to be an import but not an export.
However, it may work just as well to make a remote be readonly to
prevent export to it while still allowing import.
 2019-03-04 16:07:35 -04:00
Joey Hess
3cd19fb4d0
use InodeCache to avoid races in import from directory special remote 
This does not avoid all possible races, but it does avoid all likely
ones, and is demonstratably better than git's own handling of races
where files get modified at the same time as it's updating the working
tree.

The main thing this won't detect are not unlikely races where part
of a file gets changed while it's being copied and then the file is
restored to its original condition before the modification check.
No, it's more likely that the limitations of checking inode, size,
and mtime won't detect certian modifications, involving eg mmapped
files.
 2019-03-04 13:57:23 -04:00
Joey Hess
51fc969b66
notes 2019-03-01 16:44:34 -04:00
Joey Hess
18d7a1dbbb
make export and sync update special remote tracking branch 
The branch is only updated once the export is 100% complete. This way,
if an export is started but interrupted and so the remote does not yet
contain some of the files, an import will make a commit on the old
branch, and so won't delete the missing files.
 2019-03-01 16:35:48 -04:00
Joey Hess
d28b0a8bd0
use disconnected history for import tracking branch 
This avoids the first merge from it deleting all files in the current
branch, which was very surpring and unwanted behavior.
 2019-03-01 14:33:29 -04:00
Joey Hess
740f957cef
new problem 2019-03-01 13:49:26 -04:00
Joey Hess
1c8793691a
import: update location log for removed files 2019-03-01 13:26:59 -04:00
Joey Hess
5402c39882
plan for item 2019-03-01 12:56:20 -04:00
Joey Hess
a3f6e07fec
todo 2019-02-27 15:56:32 -04:00
Joey Hess
b1f10fbb4d
update location log during import 2019-02-27 13:58:03 -04:00
Joey Hess
e2e57f8556
initial export support for directory special remote 
This does not guard against race condition yet, it's only for testing
purposes.
 2019-02-27 13:42:34 -04:00
Joey Hess
45aacd888b
import downloader complete (untested) 
Made some api changes.

listImportableContents needs to provide the size
of the data, so the downloader can check disk free space.

retrieveExportWithContentIdentifier is passed the filepath to write to

Use temporary "CID" key during download of a ContentIdentifier from a
remote, so withTmp can be used and then move the content to the real key
once it's known.
 2019-02-27 13:15:02 -04:00
Joey Hess
f4b773e9a1
incomplete action to download files from import 2019-02-26 15:25:28 -04:00
Joey Hess
e4e464da65
import command is updating tracking branch 2019-02-26 13:15:48 -04:00
Joey Hess
7072f7a071
note 2019-02-23 15:57:18 -04:00
Joey Hess
4e0d08b66b
Merge branch 'master' into importtree 2019-02-22 21:18:13 -04:00
Joey Hess
8c836623b7
design work 2019-02-22 16:18:09 -04:00
Joey Hess
8fdea8f444
WIP 
Added graftTree but it's buggy.

Should use graftTree in Annex.Branch.graftTreeish; it will be faster
than the current implementation there.

Started Annex.Import, but untested and it doesn't yet handle tree
grafting.
 2019-02-21 17:32:59 -04:00
Joey Hess
7392304f1c
notes 2019-02-20 17:06:36 -04:00
Joey Hess
d128c8c3ec
add design document for import tree 2019-02-20 12:12:32 -04:00
Joey Hess
2f67c4ac87
minor tweaks to transition plan 2019-02-20 11:28:00 -04:00
Joey Hess
02c1542886
Merge branch 'master' of ssh://git-annex.branchable.com 2019-02-13 16:29:28 -04:00
Joey Hess
94d8bfb158
finally an API happy with 2019-02-13 16:28:02 -04:00
Ilya_Shlyakhter
3caae7ea81 alternate way to support batch operations by remotes 2019-02-13 16:55:34 +00:00
Ilya_Shlyakhter
1b279ec738 added suggestion for batch-mode operations for remotes 2019-02-13 16:52:09 +00:00
Ilya_Shlyakhter
23ef159352 Added a comment 2019-02-13 16:48:08 +00:00
Ilya_Shlyakhter
9d192e1cbb Added a comment 2019-02-13 16:46:26 +00:00
Grothausmann.Roman@343b033fef650d362015d5593ec935121340e082
a716b2cff9 Added a comment: Up-vote for sftp special remote 2019-02-13 12:11:05 +00:00
Grothausmann.Roman@343b033fef650d362015d5593ec935121340e082
bd06bcaebe 2019-02-13 11:21:10 +00:00
Joey Hess
2514c26ed8
update 2019-02-12 22:02:02 -04:00
Ilya_Shlyakhter
7e8016f104 added suggestion for creating simpler-to-use trusted export remotes 2019-02-12 21:09:22 +00:00
Ilya_Shlyakhter
024120065e universal batch mode -- clarified difference from current batch modes 2019-02-12 00:38:14 +00:00
Ilya_Shlyakhter
ea00337468 universal batch mode 2019-02-12 00:35:16 +00:00
Joey Hess
87987c78cf
starting api design 2019-02-11 15:47:18 -04:00
Joey Hess
b7991248db
simplify 2019-02-11 15:16:35 -04:00
Joey Hess
5a36f85c16
thoughts 2019-02-11 14:14:44 -04:00
Joey Hess
2f117ec7b7
thought 2019-02-09 14:03:59 -04:00
Joey Hess
b6df092f56
close 2019-02-07 16:12:07 -04:00
Joey Hess
d5c435d3dc
add 2019-02-07 12:16:23 -04:00
Ilya_Shlyakhter
c33793f983 Added a comment 2019-02-05 20:55:59 +00:00
Joey Hess
c3f47ba389
make .noannex file prevent repo fixups 
Avoid performing repository fixups for submodules and git-worktrees
when there's a .noannex file that will prevent git-annex from being
used in the repository.

This change is ok as long as the .noannex file is really going to prevent
git-annex from being used. But, init --force could override the file.
Which would result in the repo being initialized without the fixups
having run.

To avoid that situation decided to change init, to not let --force be used
to override a .noannex file. Instead the user can just delete the file.
 2019-02-05 14:43:23 -04:00
Joey Hess
b080699a95
fromkey --json 
* fromkey: Added --json.
* fromkey --batch output changed to support using it with --json.
  The old output was not parseable for any useful information, so
  this is not expected to break anything.
 2019-02-05 14:03:29 -04:00
Joey Hess
7b46b43c48
fromkey: Made idempotent 
If the worktree file already exists, and is annexed and uses the same
key, avoid failing, nothing needs to be done.

Had to add lookupFileNotHidden to handle the case where an adjust --hide-missing
is in use, and the worktree file was hidden due to the object content
being missing. lookupFile would return the key of the hidden file,
but it makes sense that after fromkey succeeds, the worktree must
contain the file it was supposed to set up.
 2019-02-05 13:13:13 -04:00
Ilya_Shlyakhter
1b3c4d9a56 git-annex-fromkey behavior when target file exists 2019-02-02 23:13:55 +00:00
Ilya_Shlyakhter
f0dea0ba06 reporting the status of each operation in a batch; completing doable ops even if others fail. 2019-01-30 22:20:10 +00:00
kyle
64ae689f21 2019-01-29 16:26:06 +00:00
yarikoptic
76652f0907 Added a comment 2019-01-26 00:46:04 +00:00
Ilya_Shlyakhter
973034eb6d Added a comment 2019-01-25 21:34:15 +00:00
yarikoptic
fd5bf09cd5 initial plea for publicurl for all special remotes 2019-01-25 03:08:39 +00:00
yarikoptic
55376d0a28 initial whining about absent way to get supported configuration options for a special remote 2019-01-25 02:58:52 +00:00
Joey Hess
ee1cbe98e3
thoughts 2019-01-22 13:25:54 -04:00
Joey Hess
fe424fd76c
comment 2019-01-22 11:56:25 -04:00
Joey Hess
e0b3ba3819
comment 2019-01-22 11:44:02 -04:00
lykos@d125a37d89b1cfac20829f12911656c40cb70018
cc3a70e7aa Added a comment 2019-01-21 11:57:07 +00:00
lykos@d125a37d89b1cfac20829f12911656c40cb70018
8961e9fd79 Added a comment 2019-01-21 11:40:07 +00:00
yarikoptic
aa8dbb3436 Added a comment: May be it is time? 2019-01-20 15:32:39 +00:00
Ilya_Shlyakhter
62a6dc6148 Added a comment 2019-01-18 20:57:36 +00:00
Joey Hess
d5f2463702
misctmp cleanup 
* Switch to using .git/annex/othertmp for tmp files other than partial
  downloads, and make stale files left in that directory when git-annex
  is interrupted be cleaned up promptly by subsequent git-annex processes.
* The .git/annex/misctmp directory is no longer used and git-annex will
  delete anything lingering in there after it's 1 week old.

Also, in Annex.Ingest, made the filename it uses in the tmp dir be
prefixed with "ingest-" to avoid potentially using a filename used by
some other code.
 2019-01-17 16:02:22 -04:00
Joey Hess
8555169e71
testremote: Support testing readonly remotes with the --test-readonly option 
This commit was sponsored by Ilya Shlyakhter on Patreon.
 2019-01-17 12:44:52 -04:00
Joey Hess
8230b62e06
add todo 2019-01-17 11:49:56 -04:00
Ilya_Shlyakhter
8c0a2a1155 added comments about testing addurl-related functions in testremote 2019-01-15 21:24:29 +00:00
Ilya_Shlyakhter
a1be80635f git-annex-testremote for read-only external special remotes 2019-01-15 16:00:45 +00:00
Joey Hess
745ecccd0e
Merge branch 'master' of ssh://git-annex.branchable.com 2019-01-14 19:00:56 -04:00
Joey Hess
918868915c
rename page 2019-01-14 15:57:04 -04:00
Ilya_Shlyakhter
9953e3d353 Added a comment 2019-01-14 16:11:39 +00:00
andrew
f605156e42 2019-01-13 18:11:07 +00:00
Ilya_Shlyakhter
15dd1a17a1 Added a comment 2019-01-11 22:23:06 +00:00
Joey Hess
c1c976d1fa
Merge branch 'master' of ssh://git-annex.branchable.com 2019-01-11 17:25:56 -04:00
Joey Hess
d12f4db54d
comment 2019-01-11 16:54:07 -04:00
git-annex.branchable.com@1c3a8a83c15a19620a0a1a2e653d7c662fc8fe50
c30045bf75 Added a comment: get dry-run-ish option 2019-01-11 16:18:13 +00:00
Ilya_Shlyakhter
15fddad749 added suggestion to support MD5E keys that omit file size, and/or support attaching checksums to URL/WORM keys in metadata 2019-01-08 20:12:38 +00:00
Ilya_Shlyakhter
6835cd3957 asked about per-branch git-annex branches 2019-01-02 21:12:13 +00:00
andrew
d849ee579f Added a comment 2019-01-01 20:02:58 +00:00
michael@ff03af62c7fd492c75066bda2fbf02370f5431f4
25cc890be1 Added a comment: Import --clean-duplicates 2018-12-31 10:40:04 +00:00
andrew
68a07151ef Added a comment 2018-12-30 23:29:16 +00:00
travis.a.everett@3055951666d9c7bb74d8826294a71629368f5b92
81cd3c7d2a 2018-12-27 19:22:45 +00:00
spwhitton
556c555aa9 Added a comment 2018-12-21 12:27:14 +00:00
Joey Hess
365286279f
unused: Update suggested git log message to see where data was previously used so it will also work with v7 unlocked pointer files. 2018-12-19 13:53:49 -04:00
Joey Hess
c53a168717
followup 2018-12-19 13:45:37 -04:00
spwhitton
68da029629 Added a comment 2018-12-18 21:56:24 +00:00
Joey Hess
e6886eb316
Merge branch 'master' of ssh://git-annex.branchable.com 2018-12-18 16:31:49 -04:00
Joey Hess
70ba925abc
followup 2018-12-18 16:30:41 -04:00
spwhitton
c5a7221b21 update bug report in light of attempt to reproduce 2018-12-18 18:37:21 +00:00
anarcat
636dea401b trivial documentation change... 2018-12-14 15:15:28 +00:00
spwhitton
a1df27a102 file test suite failure bug 2018-12-13 17:48:50 +00:00
Joey Hess
c1dd582f97
Merge branch 'master' of ssh://git-annex.branchable.com 2018-12-09 14:30:00 -04:00
Joey Hess
029ae8d4db
support findred and --branch with file matching options 
* findref: Support file matching options: --include, --exclude,
  --want-get, --want-drop, --largerthan, --smallerthan, --accessedwithin
* Commands supporting --branch now apply file matching options --include,
  --exclude, --want-get, --want-drop to filenames from the branch.
  Previously, combining --branch with those would fail to match anything.
* add, import, findref: Support --time-limit.

This commit was sponsored by Jake Vosloo on Patreon.
 2018-12-09 13:38:35 -04:00
Joey Hess
2d9b3e4510
comment 2018-12-09 11:11:53 -04:00
andrew
38a658753a Added a comment 2018-12-08 17:09:27 +00:00
andrew
666db40a32 Added a comment 2018-12-08 16:46:35 +00:00
Ilya_Shlyakhter
22b487e738 asked about adding tests under concurrency 2018-12-07 17:16:56 +00:00
Joey Hess
eb0db3d230
comments 2018-12-05 12:44:09 -04:00
michael@ff03af62c7fd492c75066bda2fbf02370f5431f4
d810efe844 Added a comment: Borg vs. restic, some design considerations 2018-12-05 14:36:45 +00:00
anarcat
fbd0c57e8a updated the upgrades page, thanks for the clarification! 2018-12-04 21:07:52 +00:00
Joey Hess
14e6d7cf2d
comment 2018-12-04 16:55:36 -04:00
anarcat
5e3bb14873 another v7 catch? 2018-12-04 20:41:08 +00:00
Ilya_Shlyakhter
8f87be6622 Added a comment 2018-12-04 19:56:36 +00:00
Joey Hess
8d0d00d926
fix typo 2018-12-04 15:23:04 -04:00
Joey Hess
78879b5b36
response 2018-12-04 15:20:40 -04:00
Ilya_Shlyakhter
5c510f6937 fixed markup 2018-12-04 18:58:04 +00:00
Ilya_Shlyakhter
00ccf952e4 added suggestion for encrypting URLs 2018-12-04 18:57:04 +00:00