On Windows, a file that is not writable cannot be deleted even if in a directory with write perms. So git object files were not getting deleted when removing a git repository.