Commit graph

45557 commits

Author SHA1 Message Date
Joey Hess
52891711d2
git-annex sim command is working
Had to add Read instances to Key and NumCopies and some other similar
types. I only expect to use those in serializing a sim. Of course, this
risks that implementation changes break reading old data. For a sim,
that would not be a big problem.
2024-09-12 16:10:52 -04:00
Joey Hess
7e8274c6b7
implemented ActionDropUnwanted
Not tested yet. This emulates the same checking that is done when
dropping. Note that when dropping from a special remote it is not able
to make a locked copy.
2024-09-12 10:44:31 -04:00
Joey Hess
68e52f6ec0
convert simRepoState to use UUID as key 2024-09-12 10:29:32 -04:00
Joey Hess
7b931df475
fully preserve input format of connect and disconnect commands
Just lifted the sim file as a DSL into the type level for that.
2024-09-11 21:01:13 -04:00
Joey Hess
f381b457f2
sim file parser and generator
The generator doesn't emit the best possible connect commands,
but it does output something valid. Eg, an input like:

connect A <-> B <-> C <-> D

becomes:

connect A <-> B <-> C
connect C <-> D

Also:

connect A -> B <- C

becomes:

connect A -> B
connect C -> B

Which could be improved.

Also disconnect commands are not prettified at all, but probably there's
no reason to.
2024-09-11 15:59:13 -04:00
Joey Hess
84bbbeae9d
started on sim file parser 2024-09-11 11:53:25 -04:00
Joey Hess
c4609a73f2
implemented ActionSendWanted
The simulation is nearly finished. Only CommandAddTree and dropping
remain to do.
2024-09-11 11:09:30 -04:00
Joey Hess
a387f40ffb
ActionGetWanted working
The sim is now basically working!
2024-09-11 10:32:04 -04:00
Joey Hess
07f54668c4
sim: implement ActionGitPush and ActionGitPull
These don't actually need to do any git actions, instead they just merge
the simLocations.

Had to make simLocations use a vector clock, because it is possible for
two simulated repositories to end up with different opinions about the
location of a key. Just like with real git-annex, whichever location
change was made most recently wins out.

The vector clock is simply advanced each time the simulation is run for
a step. Since there is no real parallelism in the sim, that's
sufficient.
2024-09-10 15:37:55 -04:00
Joey Hess
a674a54d87
implement randomAction 2024-09-09 17:20:13 -04:00
Joey Hess
217bc214b3
implement CommandStep
and record each SimCommand in simHistory, except for CommandStep where
instead the actions that are generated are recorded.
2024-09-09 17:04:32 -04:00
Joey Hess
64466d8687
add action command to git-annex sim
step just picks a random action, and this allows finer control over what
happens in the sim
2024-09-09 16:06:45 -04:00
Joey Hess
21da5aadec
set location logs in simulated repos 2024-09-09 14:52:24 -04:00
Joey Hess
52c6434b87
fix removing simulated file 2024-09-09 14:07:52 -04:00
Joey Hess
bbd5390fa3
create simulated files 2024-09-09 11:28:30 -04:00
Joey Hess
5f3a2f4c6b
set descriptions for all simulated repos 2024-09-09 11:06:42 -04:00
Joey Hess
ec7f1f2736
simulated repository construction working 2024-09-09 10:59:01 -04:00
Joey Hess
a2c0d5e4a9
finish updateSimRepoState
Converted maps to use UUID as key.

Also added mincopies to the sim.
2024-09-09 09:37:59 -04:00
Joey Hess
811dd95453
maxsize of 0 to disable 2024-09-09 09:32:43 -04:00
Joey Hess
4e11cb19ef
implemented cloneSimRepo
Started on updateSimRepoState
2024-09-06 14:23:29 -04:00
Joey Hess
8d707c4821
more work on applySimCommand
When using an existing repo, copy over all of its config into the sim.

Added CommandTrustLevel.

Start at creating a git clone for a simulated repo, but it's not done
yet.
2024-09-06 12:53:51 -04:00
Joey Hess
def8095e5f
rethought sim a bit 2024-09-06 12:53:20 -04:00
Joey Hess
65dd018850
prevent overwriting a repo in the simulation 2024-09-05 16:22:08 -04:00
Joey Hess
8f8e35ac3b
almost finished with applySimCommand
Added checks that repo names are ones that have been added to the sim.

Implemented preferred content etc setting. It does not need to parse the
expression in applySimCommand, instead that can be done when running the
sim. This keeps it pure.

But, it can't be entirely pure because of CommandAddTree. So made it
return an Annex action when necessary.

Moved makeMatcher into Annex.FileMatcher in preparation for using it,
but it's not used yet. Also moved checkPreferredContentExpression.
2024-09-05 15:22:41 -04:00
Joey Hess
710a199ce9
implement CommandUse in Annex.Sim
Refactored Remote to keep it pure.
2024-09-05 10:50:04 -04:00
Joey Hess
b932acf4ad
started Annex.Sim
Have most of the sim command handler, but to keep it pure while implementing
the rest will need some refactoring.

It seems likely that running the simulation itself will not be able to be
entirely pure. Preferred content evaluation runs in Annex after all.

Note that the somewhat awkward randomWords is because the i386ancient
build depends on a version of random too old to support generating a
random ByteString on its own.
2024-09-04 15:15:36 -04:00
Joey Hess
84c781d924
documentation for git-annex sim
command not implemented yet
2024-09-04 15:03:17 -04:00
Joey Hess
1b6c33a38e
update 2024-09-03 14:24:32 -04:00
Joey Hess
3398514c38
sim design 2024-09-03 14:23:48 -04:00
Joey Hess
5807e1480c
correct comment
This is not related to v5 versus newer versions.
2024-09-03 14:23:32 -04:00
Joey Hess
fe71400e37
fix typo 2024-09-03 14:23:14 -04:00
Joey Hess
340bdd0dac
treat "not present" in preferred content as invalid
Detect when a preferred content expression contains "not present", which
would lead to repeatedly getting and then dropping files, and make it never
match. This also applies to "not balanced" and "not sizebalanced".

--explain will tell the user when this happens

Note that getMatcher calls matchMrun' and does not check for unstable
negated limits. While there is no --present anyway, if there was,
it would not make sense for --not --present to complain about
instability and fail to match.
2024-09-03 13:50:06 -04:00
Joey Hess
8b2bd42540
Fix --debug display of onlyingroup preferred content expression. 2024-09-03 12:38:59 -04:00
Joey Hess
03864a2c3b
update 2024-09-03 11:52:54 -04:00
Joey Hess
b800ea6826
2 level toc 2024-09-02 16:32:28 -04:00
Joey Hess
ab0c82114b
Merge branch 'master' of ssh://git-annex.branchable.com 2024-09-02 16:31:31 -04:00
Joey Hess
1e1c13dd38
fix number of headers 2024-09-02 16:31:03 -04:00
lucas.gautheron@f2b5c93a64b028c1ec8698b9c2412ed51ff22040
850ea3a9b8 2024-09-02 15:12:02 +00:00
lucas.gautheron@f2b5c93a64b028c1ec8698b9c2412ed51ff22040
925c203c09 2024-09-02 15:08:25 +00:00
Joey Hess
9d29b99ac4
add news item for git-annex 10.20240831 2024-08-31 19:50:36 -04:00
Joey Hess
b3dc656153
releasing package git-annex version 10.20240831 2024-08-31 19:50:26 -04:00
Joey Hess
35ff8c8c00
use Utility.PID
fixes build on i386ancient
2024-08-30 14:56:38 -04:00
Joey Hess
3b74386ed4
fix liveupdate locking
This fixes the build on windows.

Changed it to use lock pools, which will behave better if two threads
call getLiveRepoSizes at the same time.

Also this should make it work when annex.pidlock is set. In that case,
once the current process locks this file, or anything, any other process
will have to wait on the pid lock. So checkStaleSizeChanges will
correctly identify any other live changes in the database as stale,
since there can only be one git-annex process running.
2024-08-30 14:49:18 -04:00
Joey Hess
c10d11959a
fix paste oops
Wow, I pasted a big thing into entirely the wrong file, but it was in a
comment so it compiled anyway.
2024-08-30 14:35:05 -04:00
Joey Hess
698d9252a5
mention sizebalanced as well as balanced 2024-08-30 12:06:45 -04:00
Joey Hess
133584a83a
avoid locking the journal in readonly repository
The test suite flagged that git-annex info in a readonly repository was
no longer working.

.git/annex/journal.lck: openFd: permission denied

This fixes it, however, in a case where .git/annex/reposize/ is
writable, but .git/annex/journal/ is not, there will still be a
permission denied error. The solution would just be to use consistent
permissions I suppose.
2024-08-30 11:58:10 -04:00
Joey Hess
53b7375cc6
update 2024-08-30 11:14:45 -04:00
Joey Hess
54b6151412
document using balanced preferred content in a cluster 2024-08-30 11:08:32 -04:00
Joey Hess
d0938d730b
Merge branch 'master' into balanced 2024-08-30 11:01:39 -04:00
Joey Hess
242c525659
lookupkey: Allow using --ref in a bare repository. 2024-08-30 10:55:48 -04:00