Assuming filter-repo will be merged into git.git, use "git" for the
TEXTDOMAIN, and assume its build system will replace "@@LOCALEDIR@@"
appropriately.
Note that the xgettext command used to grab string translations is
nearly identical to the one for C files in git.git; just use
--language=python instead and add --join-existing to avoid overwriting
the po/git.pot file. In other words, use the command:
xgettext -o../git/po/git.pot --join-existing --force-po \
--add-comments=TRANSLATORS: \
--msgid-bugs-address="Git Mailing List <git@vger.kernel.org>" \
--from-code=UTF-8 --language=python \
--keyword=_ --keyword=N_ --keyword="Q_:1,2" \
git-filter-repo
To create or update the translation, go to git.git/po and run either of:
msginit --locale=XX
msgmerge --add-location --backup=off -U XX.po git.pot
Once you've updated the translation, within git.git just build as
normal. That's all that's needed.
Signed-off-by: Elijah Newren <newren@gmail.com>
Over a decade ago, I added code to deal with splitting and splicing
repositories where you weren't always dealing with first parents and
linear histories, and in particular where the mainline tended to be the
second parent (because there was no integrator or special central
gatekeeper like gerrit or github; instead, everyone pushed directly to
the main repository after locally testing, and integration happened via
everyone running 'git pull'). When attempting to splice repositories
the fact that fast-export always gave changes relative to the first
parent caused some grief with my splitting and splicing efforts.
It has been over a decade, I don't know of a good testcase of this
functionality separate from the live repositories I lost access to over
six years ago, git-subtree was released in the meantime which I'm
certain handled the task better, git-fast-export since gained a
--full-tree option which might have provided a better way to attack the
problem (though with splicing repos you often want work with additive
changes rather than recreating from scratch), and I just don't
quite understand the code anymore anyway. I think it had some
fundamental limitations that I knew my usecase avoided, but I don't
remember the details (and I'm not certain if this is true).
Even though code coverage hits all but one of the lines, I'd rather
rewrite any needed functionality if the usecase arises, and in view of
what facilities exist today rather than what I was working with a decade
ago. So, just nuke this code.
Signed-off-by: Elijah Newren <newren@gmail.com>
The original idea was to add --path-rename-(glob|regex) options, but
I like the general flexibility of --filename-callback better for
special cases and keeping the number of command line options at least
slightly in check.
Signed-off-by: Elijah Newren <newren@gmail.com>
There are several lines equivalent to BUG() calls in git that are
supposed to be unreachable, and which exist just to make debugging the
fundamental system problem or refactoring of the code slightly easier by
trying to give a more immediate notification of a problem. If these
error cases are ever hit and happen to be wrong, then the individual
will at worst get a stacktrace and the program will abort...but that
might arguably be even more helpful. Since there is no harm in avoiding
the work of finding ways to break the system to force these lines to be
covered, simply exclude them from line coverage counting.
Signed-off-by: Elijah Newren <newren@gmail.com>
The AncestryGraph setup assumed we had previously seen all commits which
would be used as parents; that interacted badly with doing an
incremental import. Add a function which can be used to record external
commits, each of which we'll treat like a root commit (i.e. depth 1 and
having no parents of its own). Add a test to prevent regressions.
Signed-off-by: Elijah Newren <newren@gmail.com>
There are a number of things not present in "normal" imports that we
nevertheless support and need to be tested:
* broken timezone adjustment (+051800->+0261; observed in the wild
in real repos, and adjustment prevents fast-import from dying)
* commits missing an author (observed in the wild in a real repo;
just sets author to committer)
* optional additional linefeeds in the input allowed by
git-fast-import but usually not written by git-fast-export
* progress and checkpoint objects
* progress, checkpoint, and 'everything' callbacks
Signed-off-by: Elijah Newren <newren@gmail.com>
While most users of filter-repo will just use it as a tool and
RepoFilter.run() is the final function, filter-repo can be used as a
library with additional work being done after calling that function.
So, simply return from that function when it is done rather than calling
sys.exit.
Signed-off-by: Elijah Newren <newren@gmail.com>
The everything_callback took two arguments, the first being a string
that was the name of the type of the second argument. There was no
point for this argument; someone can just compare type(second) to the
relevant classes. Remove it.
Signed-off-by: Elijah Newren <newren@gmail.com>
The only times this is ever printed is when debugging filter-repo
itself, or when trying to add tests to get to 100% line coverage. But
the printing was broken when objects were skipped (which caused a
mapping from int -> None). Fix the format specifier to handle this
case too.
Signed-off-by: Elijah Newren <newren@gmail.com>
We don't expect to ever get progress or checkpoint directives in normal
operation, but the --stdin flag makes it a possibility. In such a case,
the progress directives could actually break our parsing since
git-fast-import will just print it to its stdout, which is what we read
from to find new commit names so we can do commit message hash updating.
So, pass these along to a progress_callback, but don't dump them by
default. Also, it is not clear checkpoint directives make sense given
that we'll be filtering and only getting a subset of history (and I'm
dubious on checkpoint's utility in general anyway as fast-import is
relatively quick), so pass these along to a callback but don't use them
by default.
Signed-off-by: Elijah Newren <newren@gmail.com>
We don't run fast-export with rename detection, even though we have
code for handling it, because we decided to use a rev-list|diff-tree
pipeline instead. The code was manually tested and determined to be
working and it might be useful in the future so I don't want to just
outright delete it, but since we know we can't trigger it right now,
add a
# pragma: no cover
on these lines so it doesn't show up on coverage reports.
Signed-off-by: Elijah Newren <newren@gmail.com>
This also generates line coverage statistics for t/t9391/*.py, but the
point is line coverage of git-filter-repo.
Signed-off-by: Elijah Newren <newren@gmail.com>
Since there were multiple places in the code where we returned early
knowing that we didn't have a translation of old_hash to a new_hash, we
need to update _commits_referenced_but_removed from each of them.
Signed-off-by: Elijah Newren <newren@gmail.com>
Due to the invariants we maintain with _commit_renames and
_commit_short_old_hashes (the latter always gets an extra entry in
either a key or a value whenever _commit_renames gains a new key/value
pair), there were a few lines of code that we could not ever reach.
Replace them with an assertion that the condition used for them is
never true.
Signed-off-by: Elijah Newren <newren@gmail.com>
The test does check the exact output for the report, meaning if the
output is changed at all this test will need to be updated, but it at
least makes sure we are getting all the right kinds of information. I
do not expect the output format will change very often.
Signed-off-by: Elijah Newren <newren@gmail.com>
The former logic for keeping track of whether we had seen annotated tags
(and thus whether they were interesting and should avoid being pruned)
was just plain buggy. I do not know if it was that bad from the start
or there was other surrounding code that made it different that got lost
in one of my history rewrites, but fix it. I'll include tests of it
with --subdirectory-filter shortly.
Signed-off-by: Elijah Newren <newren@gmail.com>
We previously would abort if we had been requested to rename files and
that caused two different files to go to the same path. However, if
the files have identical contents and mode, then we can treat the
request as a desire from the user to just coalesce the extra copies.
Signed-off-by: Elijah Newren <newren@gmail.com>
Pruning of commits which become empty can result in a variety of
topology changes: a merge may have lost all its ancestors corresponding
to one of (or more) of its parents, a merge may end up merging a commit
with itself, or a merge may end up merging a commit with its own
ancestor. Merging a commit with itself makes no sense, so we'd rather
prune down to one parent and hopefully prune the merge commit, but we do
need to worry about whether the are changes in the commit and whether
the original merge commit also merged something with itself. We have
similar cases for dealing with a merge of some commit with its own
ancestor: if the original topology did the same, or the merge commit has
additional file changes, then we cannot remove the commit. But,
otherwise, the commit can be pruned.
Add testcases covering the variety of changes that can occur to make
sure we get them right.
Signed-off-by: Elijah Newren <newren@gmail.com>
Due to pruning of empty commits, merge commits can become degenerate
(same commit serving as both parents, or one parent is an ancestor of
one of the others). While we usually want to allow such degenerate
merge commits to themselves be pruned (assuming they add no additional
file changes), we do not want to prune them if the merge commit in the
original repository had the same degenerate topology. So, we need to
keep track of the ancestry graph of the original repository as well and
include it in the logic about whether to allow merge commits to be
pruned.
Signed-off-by: Elijah Newren <newren@gmail.com>
There are several cases to worry about with commit pruning; commits
that start empty and had no parent, commits that start empty and
had a parent which may or may not get pruned, commits which had
changes but became empty, commits which were merges but lost a line
of ancestry and have no changes of their own, etc. Add testcases
covering these cases, though most topology related ones will be
deferred to a later set of tests.
Signed-off-by: Elijah Newren <newren@gmail.com>
The reason we want to sometimes keep commits that start empty is because
they may have been intentionally added for build or versioning reasons.
Not all commits that start empty are useful, even if intentional,
though, because they could have pre-dated the introduction of a
directory we are filtering for. So, we always allowed an exception that
if the number of parents had been reduced, we also allow pruning commits
that started empty.
However, there is a similar case: one or more contiguous chunks of
history may only touch some directories/files that are not of interest;
empty commits within that range of history are likewise uninteresting to
us. Since "interesting" empty commits are of the form some new commit
on top of interesting history (because otherwise it loses its special
build or versioning utility), we should loosen the rules to also
consider that empty commits whose parent was pruned are also prunable;
we no longer use the existence of some other distant ancestor of that
empty commit in determining whether the empty commit is prunable.
Signed-off-by: Elijah Newren <newren@gmail.com>
Due to the special handling of 'from' in the fast_export stream and the
aggregation of the 'from' commit with the 'merge'd commits, a parentless
commit has its parents represented as [None] rather than []. We had
fixed this up in other places, but forgot to do so with orig_parents,
breaking our comparison. Handle it for orig_parents too.
Signed-off-by: Elijah Newren <newren@gmail.com>
fast-import from versions of git up to at least 2.21.0 had a bug in the
handling of the get-mark directive that would cause it to abort with a
parsing error on valid input. While a fix has been submitted upstream
for this, add some extra newlines in a way that will work with both old
and new git versions.
Signed-off-by: Elijah Newren <newren@gmail.com>
Many of the callback functions might only be a single line, and as such
instead of forcing the user to write a full blown program with an import
and everything, let them just specify the body of the callback function
as a command line parameter. Add several tests of this functionality as
well.
Signed-off-by: Elijah Newren <newren@gmail.com>
Add callbacks for:
* filename
simplifies filtering/renaming based solely on filename; return
None to have file removed, or original or new name for file
* message
simplifies tweaking both commit and tag messages; if you want to
tweak just one of the two, use either tag_callback or
commit_callback
* person_name
simplifies tweaking actual names of people without worrying where
they come from (author, committer, or tagger)
* email:
simplifies tweaking email addresses without worrying where they
come from (author, committer, or tagger)
* refname:
simplifies tweaking reference names, regardless of whether they
come from FastExport commit objects, reset objects, or tag objects
Signed-off-by: Elijah Newren <newren@gmail.com>
I want to allow callbacks that could operate on similar pieces of commit
or reset or tag objects (e.g. reference names, email addresses);
restructure the current ones slightly to both allow more general ones to
be added and to make the existing ones slightly clearer.
Signed-off-by: Elijah Newren <newren@gmail.com>
Users may want to run --analyze both before and after filtering in
order to both find the big objects to remove and to verify they are
gone and the overall repository size and filenames are as expected.
As such, aborting and telling the user there's a previous analysis
directory in the way is annoying; just remove it instead.
Signed-off-by: Elijah Newren <newren@gmail.com>
Users may want to run multiple filtering operations, either because it's
easier for them to do it that way, or because they want to combine both
path inclusion and exclusion. For example:
git filter-repo --path subdir
git filter-repo --invert-paths --path subdir/some-big-file
cannot be done in a single step. However, the first filtering operation
would make the repo not look like a clean clone anymore (because it is
not a clean clone anymore), causing the safety check to trigger and
requiring the --force flag. But once we've allowed them to do
repository rewriting, there's no point disallowing further rewriting.
So, write a .git/filter-repo/already_ran file when we run and treat the
presence of that file the same as providing a --force flag.
Signed-off-by: Elijah Newren <newren@gmail.com>
Elijah Newren