If a commit was a non-merge commit previously, then since we do not do
any kind of blob modifications (or funny parent grafting), there is no
way for a filemodify instruction to introduce the same version of the
file that already existed in the parent, as such the only check we need
to do to determine whether a commit becomes empty is whether
file_changes is empty. Subsequent more expensive checks can be skipped.
Split a lot of the logic out into separate functions, and avoid
flattening parents when the original commit history itself had
redundant parents (such as --no-ff merges).
commits may not have any parents at all. As such,
parse_optional_parent_ref() is used expecting that it will sometimes
return None.
Now, when commits are skipped, we have a scheme to translate anyone that
depends on such commits to instead depend on the nearest ancestor of
such commits. If the entire ancestry of a commit was skipped along with
a comit, then that commit will be translated to None, which is
indistinguishable from there having been no parent to begin with.
Sometimes our scheme needs to distinguish between a commit that started
with no parents and one which ended up with no parents, so we need a way
to tell these apart.
Also, not knowing the original parent makes it hard for us to
determine if the original had the same weird topology that the current
commit does. For example, it is possible for a merge commit to have
one parent be the ancestor of another (particularly when --no-ff is
passed to git merge), or even for a merge commit to have the same
commit used as both parents (if you use low-level commands to create
a crazy commit). There are cases where the pruning of some commits
could cause either of these situations to arise, and it's useful to be
able to distinguish between intentionally "weird" history and history
that has been made weird due to other pruning, because the latter we
may have reason to do additional pruning on.
Oh, boy, timezone +051800 exists in the wild. Is that 0518 hours and 00
minutes? Or 05 hours and 1800 minutes? Or 051 hours and 800 minutes?
Attempt to do something sane with these broken commits that fast-import
barfs on. Also, fix an old bug in the handling of ahead-of-UTC timezones.
Apparently, the default for subprocess stdout is unbuffered; switching
it to buffered yields a huge 40% speedup. Doing this also exposes the
need to add fi_input.flush() calls, highlighting another performance
issue. We may be able to have fewer such calls with some refactoring,
but that is a bigger separate change. Just having them highlighted to
remind about them as a performance issue is good for now.
As suggested by Peff, use rev-list & diff-tree to get the information we
need, instead of relying on fast-export (with some out-of-tree patches)
to get that information.
Parsing the quoted filename strings was slightly tricky. See
https://stackoverflow.com/a/51904799 for discussion of codecs. I didn't
do the final utf-8 conversion because of the following investigation:
s = 'naïve \\t test'
now comparing
' '.join(hex(ord(x))[2:] for x in s)
' '.join(hex(ord(x))[2:] for x in codecs.decode(s, 'unicode_escape'))
' '.join(hex(ord(x))[2:] for x in codecs.decode(s, 'unicode_escape').encode('latin-1'))
' '.join(hex(ord(x))[2:] for x in codecs.decode(s, 'unicode_escape').encode('latin-1').decode('utf-8'))
I saw the following:
'6e 61 c3 af 76 65 20 5c 74 20 74 65 73 74'
'6e 61 c3 af 76 65 20 9 20 74 65 73 74'
'6e 61 c3 af 76 65 20 9 20 74 65 73 74'
'6e 61 ef 76 65 20 9 20 74 65 73 74'
also printing the four related strings from python shows:
naïve \t test
naïve test
naïve test
naïve test
In other words, the 'unicode_escape' correctly translated 5c 74 ('\t')
into 09 (a tab character), the encoding into latin-1 didn't change any
bytes, but the final decode into utf-8 did. Also, the translation into
latin-1 correctly prints the string so let's just keep it.
If a tag points at a commit whose changes are all filtered out and thus
becomes empty and gets pruned, and all of its ancestors are likewise
pruned, then there is no need for the tag; just nuke it.
Elijah Newren
Elijah Newren