git-annex/doc/todo/make_annex_info_more_efficient.mdwn
Yaroslav Halchenko 6eb8784e7c
Embed projects/{datalad,repronim} tags for the past todo/bugs
Assignment between datalad and repronim might not be 100% right but
approx there depending on the date

Here is the command and its output:

	$> git grep -l '\[\[!meta author=\(yoh\|ben\|yarikoptic\|kyle\)\]\]' | grep -v comment | grep -v assure_correct_names___40 | while read f; do d=$(git log --date=iso $f| awk '/^Date/{print $2;}' | tail -n 1 ); p=datalad; if [[ "$d" =~ 2019-.* ]] || [[ "$d" =~ 2018-1.* ]]; then p=repronim; fi; echo $d $p: $f; sed -i -e "s,\(\[\[\!meta author.*\),\1\n\[\[\!tag projects/$p\]\],g" $f; done
	2017-10-10 datalad: doc/bugs/--shared_setting_of_git_causes_annex__39__ed_files_to_be_writeable__33__.mdwn
	2017-08-12 datalad: doc/bugs/2_ssh_connection_prompts_for_password.mdwn
	2018-10-22 repronim: doc/bugs/3_tests_fail_while_trying_to_build_fresh_git_annex_6.20181011+git109-gff9ba1f4d.mdwn
	2018-12-20 repronim: doc/bugs/7.20181211+git29-gab4a1bed9_fails_tests_during_neurodebian_build.mdwn
	2016-03-12 datalad: doc/bugs/OSX__58___addurl_--batch_--json_spits_out_shortened_output_string__dies_off_with_4.mdwn
	2018-11-16 repronim: doc/bugs/Please_include_version_of_used_to_build_ghc_within___34__git_annex_version__34__.mdwn
	2016-03-10 datalad: doc/bugs/R__47__O_permissions_are_stalking_me_somehow.mdwn
	2019-07-03 repronim: doc/bugs/Regression_in___96__find_--json__96___output.mdwn
	2017-01-27 datalad: doc/bugs/Too_difficult_if_not_impossible_to_explicitly_add__47__keep_file_under_git___40__not_annex__41___in_v6_without_employing_.gitattributes.mdwn
	2018-12-11 repronim: doc/bugs/Unable_to_addurl_file__58____47____47____47___on_Windows.mdwn
	2019-04-10 repronim: doc/bugs/Unable_to_get__47__addurl_to_http_link__58___download_failed__58___InvalidHeader___34__preload__34__.mdwn
	2019-09-19 repronim: doc/bugs/add_--json_should_provide_a___34__note__34___with_an_error_message_in_case_of_an_error.mdwn
	2016-11-20 datalad: doc/bugs/adds_file_destined_for_annex_into_git_in___39__addurl__39__.mdwn
	2016-01-12 datalad: doc/bugs/addurl_--batch__--with-files_doesn__39__t_add_file_into_git_until_pipe_is_closed.mdwn
	2016-01-15 datalad: doc/bugs/addurl_--batch___40__--json_or_not__41___doesn__39__t_report_failure_correctly_if_non-annexed_file_exists.mdwn
	2016-05-23 datalad: doc/bugs/addurl_--batch_decides_to_talk_to_ssh_remotes_for_some_reason.mdwn
	2016-01-11 datalad: doc/bugs/addurl_--file__causes_file_redownload_even_if_it_already_present.mdwn
	2019-03-18 repronim: doc/bugs/addurl_fails_with_non-youtube_url_with___34__This_url_is_supported_by_youtube-dl__44___but_youtube-dl_could_potentially_access_any_address__34__.mdwn
	2016-12-04 datalad: doc/bugs/annex_add_ignores_.-prefixed_directories.mdwn
	2017-03-22 datalad: doc/bugs/annex_doesn__39__t_fixup_symlinks_when___34__git_commit_path__95__to__95__repo__34___is_used.mdwn
	2016-07-12 datalad: doc/bugs/annex_drop_fails_to_determine_availability_on_a_http_url_redirecting_to_ftp.mdwn
	2016-04-26 datalad: doc/bugs/annex_drop_is_not___34__in_effect__34___for_load_which_was___34__addurl_--batch__34__ed_but_not_yet_committed.mdwn
	2018-09-25 datalad: doc/bugs/annex_get_-J_16_via_ssh_stalls_.mdwn
	2019-06-05 repronim: doc/bugs/annex_init_no_longer_generates_default_description.mdwn
	2018-03-27 datalad: doc/bugs/annex_metadata___40__not_--batch__39__ed__41___is_not_aware_of_files_added_via_addurls_--batch.mdwn
	2018-11-12 repronim: doc/bugs/annex_sets_remote___40__regular_http__41___to_be_ignored_while_running_in_docker_container.mdwn
	2018-02-23 datalad: doc/bugs/annex_sync___40__in_direct_mode_only__41___does_not_actually_update_local_branch__47__tree.mdwn
	2018-05-04 datalad: doc/bugs/annex_view_barfs__fatal__58___Unable_to_add___40__null__41___to_database.mdwn
	2016-09-09 datalad: doc/bugs/autoenable__61__true_seems_to_not_work_any_longer.mdwn
	2015-11-20 datalad: doc/bugs/autostart__47__git-annex.desktop_gets_resurrected.mdwn
	2016-03-29 datalad: doc/bugs/awkward_error_from_annex_whenever_operating_on_a_clone_with_submodules.mdwn
	2016-11-08 datalad: doc/bugs/cannot___34__install__34___standalone_git_annex_within_afs_mount.mdwn
	2017-03-17 datalad: doc/bugs/cannot___40__or_how__63____41___to_pass_socket_path_with_a_space_in_its_path_via_annex-ssh-options.mdwn
	2018-08-08 datalad: doc/bugs/cannot_commit___34__annex_add__34__ed_modified_file_which_switched_its_largefile_status_to_be_committed_to_git_now.mdwn
	2017-01-20 datalad: doc/bugs/clash_of_-j__in_copy_for_--json_--json-progress.mdwn
	2017-03-01 datalad: doc/bugs/copy_does_not_reflect_some_failed_copies_in_--json_output.mdwn
	2017-02-10 datalad: doc/bugs/could_standalone_copy_of_git_ship__47__deploy_hooks_samples_as_well_as_stock_git__63__.mdwn
	2015-11-12 datalad: doc/bugs/does_not_complain__47__fail_if_by_mistake_option_value_passed_within_arg_for__-c.mdwn
	2016-07-14 datalad: doc/bugs/drop_blows_on_lustre__58___SQLite3_returned_ErrorIO.mdwn
	2016-04-29 datalad: doc/bugs/dropkey_--batch_--json_--force_is_always_succesfull.mdwn
	2016-02-24 datalad: doc/bugs/duplicate_progress_reports_in_parallel___39__get__39__.mdwn
	2016-04-27 datalad: doc/bugs/encfs_support_--_shouldn__39__t_it_be_treated_as_crippled_already__63__.mdwn
	2019-08-03 repronim: doc/bugs/fails_to___96__get__96___in_parallel_for_a_freshly_clone_from_public_s3_bucket_where_versioning_info_was_forgotten.mdwn
	2019-09-02 repronim: doc/bugs/fails_to_init_under_a_directory_with_a___34__tricky__34___name.mdwn
	2017-07-26 datalad: doc/bugs/fails_to_verify_presence_via_http_while_wget_fetches_it_just_fine.mdwn
	2019-08-13 repronim: doc/bugs/fresh_build_for_neurodebian__58___test_failure.mdwn
	2017-09-13 datalad: doc/bugs/fsck_does_not_detect__47__fix_some_key_directories_correctly.mdwn
	2017-07-30 datalad: doc/bugs/get_--json_fails_whenever_plain_get_works___40__with_https_urls__41__.mdwn
	2018-03-09 datalad: doc/bugs/get_-JX__58____transfer_already_in_progress_..._for_some_files.mdwn
	2017-08-30 datalad: doc/bugs/get_-J___34__fails__34___to_get_files_with_the_same_key.mdwn
	2017-03-15 datalad: doc/bugs/get_-J_cannot_be_used_with_password-based_authentication.mdwn
	2018-02-10 datalad: doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__.mdwn
	2019-07-11 repronim: doc/bugs/get_fails_to_place_v7_unlocked_file_content_into_the_file_tree_in_v7_in_repo_with_detached_HEAD.mdwn
	2018-05-06 datalad: doc/bugs/get_from_the_web_remote_fails___40__redirect__63____41___-_lack_of_further_debug_info.mdwn
	2018-09-22 datalad: doc/bugs/get_over_ssh_fails_with___fd__58__19__58___hClose__58___resource_vanished.mdwn
	2016-03-12 datalad: doc/bugs/git-annex__58___content_is_locked__while_trying_to_move_under_NFS_and_pidlock.mdwn
	2019-03-18 repronim: doc/bugs/git-annex_precommit_.__gets_stuck_after_likely_interrupted___96__annex_add__96___call.mdwn
	2016-06-02 datalad: doc/bugs/git_annex_adjust_--unlock_seems_to_cause_migration_of_a_file_to_another_backend.mdwn
	2017-02-06 datalad: doc/bugs/git_annex_info_fails_on_NFS__58___waitToSetLock__58___resource_exhausted___40__No_locks_available__41__.mdwn
	2017-10-25 datalad: doc/bugs/graft__47__graft_cleanup_commits_--_really_needed__63__.mdwn
	2018-03-05 datalad: doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__.mdwn
	2018-09-25 datalad: doc/bugs/impossible_to_login_to_the_website_at_times.mdwn
	2018-02-20 datalad: doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions.mdwn
	2016-01-11 datalad: doc/bugs/inconsistent_output_upon_addurl_--batch_complicates_if_not_forbids_reliable_parsing_of_output.mdwn
	2015-12-21 datalad: doc/bugs/info_--json_lists_backend_usage_stats_as_a_list_of_lists.mdwn
	2018-10-11 repronim: doc/bugs/initial_get_-J_on_exported_S3_bucket_fails.mdwn
	2016-01-28 datalad: doc/bugs/lookupkey_started_to_spit_out___34__debug__34___messages_to_stdout.mdwn
	2016-02-09 datalad: doc/bugs/man_page_for_command_misses_actual_command_in_the_synopsis_for_git-annex-checkpresentkey.mdwn
	2018-11-08 repronim: doc/bugs/multiple_ssh_prompts__44___and_thread_blocked_indefinitely_in_an___63____63____63___transaction.mdwn
	2017-02-17 datalad: doc/bugs/mysterious_dependency_of_git_annex_status_output_of_the_added_file.mdwn
	2016-01-15 datalad: doc/bugs/new_whereis_--json_lost_information_about_web_urls_if_other_special_remotes_provide_them.mdwn
	2017-05-25 datalad: doc/bugs/parallel_get_can_fail_some_downloads_and_require_re-getting_.mdwn
	2016-05-01 datalad: doc/bugs/prematurely___40__can__39__t_check_offline__41___marks_remote_as_annex-ignore.mdwn
	2018-05-08 datalad: doc/bugs/regression_-_fails_to_drop_._Exit_code_11_wo_--debug__44___and_1_with_--debug.mdwn
	2017-12-06 datalad: doc/bugs/regression_-_yt__58___prefix_for___34__regular__34___urls.mdwn
	2019-08-23 repronim: doc/bugs/regression__58___fails_to_detect_need_for_pidlock_on_an_NSF_mount.mdwn
	2019-05-09 repronim: doc/bugs/regression__58___http_downloads_redirecting_to_ftp_are_no_longer_supported.mdwn
	2016-10-17 datalad: doc/bugs/remote_repository_must_be_version_6_as_well_to_have_copied_files_appear_correctly.mdwn
	2016-08-05 datalad: doc/bugs/reports_success_when_addurl_--batch__a_file_which_is_.gitignore__39__d.mdwn
	2017-09-26 datalad: doc/bugs/set_metadata_leaks_from_one___40__staged__41___key_to_another_during_rename_of_file.mdwn
	2016-02-15 datalad: doc/bugs/shouldn__39__t_keep_permissions_of_the_ssh_remote__63__.mdwn
	2016-03-09 datalad: doc/bugs/some_tests_fail_while_running_under_NFS.mdwn
	2018-09-18 datalad: doc/bugs/some_transfers_for_get_-J4_fail_over_ssh__47__rsync_and_require_retry.mdwn
	2018-09-25 datalad: doc/bugs/standalone_build_is_slightly_out_of_date.mdwn
	2015-10-28 datalad: doc/bugs/standalone_builds_shouldn__39__t_pollute___126____47__.ssh_with_helpers_merely_upon_annex_init.mdwn
	2016-05-25 datalad: doc/bugs/strips___95___from_extensions_in_E_backends__63__.mdwn
	2016-03-01 datalad: doc/bugs/thread_blocked_indefinitely_in_an_STM_transaction__while_moving_within__a_local_clone.mdwn
	2016-01-15 datalad: doc/bugs/treatment_of_largefiles_is_not_working_for_addurl_--fast___40__or_--relaxed__41__.mdwn
	2018-09-11 datalad: doc/bugs/unable_to_get_from_public_S3_remote_without_clear_reasoning_why.mdwn
	2018-09-25 datalad: doc/bugs/unclear_why_annex_fails_to_get_a_file_from_a_public_S3_bucket_where_it_exported_before.mdwn
	2016-02-08 datalad: doc/bugs/use_of_annex_in_submodule_replaces_.git_with_incorrect_symlink.mdwn
	2016-02-26 datalad: doc/bugs/using_regular_magic_file__warning_pollutes_stderr.mdwn
	2018-09-06 datalad: doc/bugs/v6_-_under_subdir__58___git_add___34__whines__34____44___git_commit___34__blows__34__.mdwn
	2018-09-12 datalad: doc/bugs/whereis_shouldn__39__t_just_crash_if_one_of_the_special_remotes___34__drivers__34___is_not_available.mdwn
	2019-01-21 repronim: doc/bugs/windows__58___autostart_for_annex_64bit_looks_under_x86.mdwn
	2016-01-18 datalad: doc/todo/--batch_for_add.mdwn
	2016-01-18 datalad: doc/todo/--batch_for_find.mdwn
	2016-01-05 datalad: doc/todo/--batch_for_info.mdwn
	2016-10-24 datalad: doc/todo/Allow_for_TRANSFER-SUCCESS_to_report_also_a_URL_where_key_could_now_be_obtained_from.mdwn
	2018-02-05 datalad: doc/todo/INFO_message_for_custom_special_remotes.mdwn
	2018-09-19 datalad: doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__.mdwn
	2016-07-28 datalad: doc/todo/__39__info_filename__39___to_provide_information_either_content_is_locally_present.mdwn
	2019-04-25 repronim: doc/todo/add_--json-progress_to___96__git_annex_add__96__.mdwn
	2016-04-22 datalad: doc/todo/add_option_to_whereis_to_avoid_network_interactions.mdwn
	2017-03-23 datalad: doc/todo/annex_add___40__-u__124__--update__41___mode.mdwn
	2016-08-05 datalad: doc/todo/annex_merge_--remotes.mdwn
	2017-02-06 datalad: doc/todo/be_able_to_specify_custom_commit_message_for_git-annex_branch_commit.mdwn
	2016-02-10 datalad: doc/todo/checkpresentkey_without_explicit_remote.mdwn
	2018-09-28 datalad: doc/todo/config_setting_to_force_--progress_reporting_for_some_git_calls.mdwn
	2018-09-27 datalad: doc/todo/configuration_option_for_default___34__mode__34___on_crippled_file_systems.mdwn
	2016-06-09 datalad: doc/todo/could_standalone___39__fixed__39___git-annex_binaries_be_prelinked__63__.mdwn
	2016-04-25 datalad: doc/todo/drop_--batch.mdwn
	2016-06-16 datalad: doc/todo/get_--batch.mdwn
	2017-07-21 datalad: doc/todo/git_annex_info___60__remote__62___does_not_list_all_the_parameters_for_the_remote.mdwn
	2016-06-09 datalad: doc/todo/git_annex_info_to_include_information_about_repo_version__63__.mdwn
	2019-04-05 repronim: doc/todo/globus_special_remote_as_a___34__transport__34___layer.mdwn
	2018-02-06 datalad: doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn
	2016-07-05 datalad: doc/todo/interface_to_the___34__progress__34___of_annex_operations.mdwn
	2018-05-22 datalad: doc/todo/machine_readable_information_about_reason_of_failure_in_enableremote.mdwn
	2015-11-02 datalad: doc/todo/make_addurl_respect_annex.largefiles_option.mdwn
	2016-02-15 datalad: doc/todo/make_annex_info_more_efficient.mdwn
	2016-05-19 datalad: doc/todo/make_copy_--fast__faster.mdwn
	2016-02-12 datalad: doc/todo/metadata_--batch.mdwn
	2017-02-16 datalad: doc/todo/output_of_wanted___40__and_possibly_group_etc__41___should_not_be_polluted_with___34__informational__34___messages.mdwn
	2014-12-18 datalad: doc/todo/parallel_get.mdwn
	2018-09-28 datalad: doc/todo/prevent_directly_printing_to_stderr_error_from_an_attempt_to_download_remote___47__config_file.mdwn
	2018-08-04 datalad: doc/todo/provide___39__file__39___in_--json-progress_record_for_addurl.mdwn
	2019-01-25 repronim: doc/todo/publicurl_config_for_all_special_remotes.mdwn
	2016-01-19 datalad: doc/todo/return___34__key__34___entry_in_--json_output_for_addurl___40__and_future_add__41___--batch.mdwn
	2019-01-25 repronim: doc/todo/some_way_to_get_a_list_of_options_for_a_special_remote_of_a_given_type.mdwn
	2018-04-28 datalad: doc/todo/support___126____47__.netrc_for_http_access_authentication_specs_as_was_done_when_wget_was_used.mdwn
	2018-01-12 datalad: doc/todo/support_ssh__58____47____47___or_sftp__58____47____47___urls_via___34__built-in__34___ssh_support.mdwn
	2016-08-05 datalad: doc/todo/sync_--branches__to_sync_only_specified_branches___40__e.g._git-annex__41__.mdwn
	2016-10-28 datalad: doc/todo/unlock_--json.mdwn
	2016-10-17 datalad: doc/todo/unlock_--read-only.mdwn
2019-09-21 21:26:15 -04:00

4 lines
421 B
Markdown

ATM it takes about a minute for 'git annex info' on a sizeable but not huge repository with only ~450 files under annex but a few thousand of files (~7000) in the tree. I am not quite sure why it takes that long since it seems to care only about annexed files. Also it might be of benefit to parallelize some traversal operations to take advantage of multiple cpu/cores
[[!meta author=yoh]]
[[!tag projects/datalad]]