filter-repo: use new --date-format=raw-permissive fast-import option

fast-import gained a new raw-permissive date format explictly for
allowing people to import repositories as-is.  Make use of the flag, and
stop rewriting the bogus timezone found in rails.git.

If users do not like these bogus times, they can of course write a
filter to fix them (or even make them bogus in a different way).  For
example:

    git filter-repo ... --commit-callback '
      if commit.author_date.endswith(b"+051800"):
        commit.author_date.replace(b"+051800", b"+0261")
    '

Signed-off-by: Elijah Newren <newren@gmail.com>

Documentation/git-filter-repo.txt

@@ -1173,7 +1173,7 @@ sequence that more accurately reflects what filter-repo runs is:
   1. Verify we're in a fresh clone
   2. `git fetch -u . refs/remotes/origin/*:refs/heads/*`
   3. `git remote rm origin`
-  4. `git fast-export --show-original-ids --reference-excluded-parents --fake-missing-tagger --signed-tags=strip --tag-of-filtered-object=rewrite --use-done-feature --no-data --reencode=yes --mark-tags --all | filter | git -c core.ignorecase=false fast-import --force --quiet`
+  4. `git fast-export --show-original-ids --reference-excluded-parents --fake-missing-tagger --signed-tags=strip --tag-of-filtered-object=rewrite --use-done-feature --no-data --reencode=yes --mark-tags --all | filter | git -c core.ignorecase=false fast-import --date-format=raw-permissive --force --quiet`
   5. `git update-ref --no-deref --stdin`, fed with a list of refs to nuke, and a list of replace refs to delete, create, or update.
   6. `git reset --hard`
   7. `git reflog expire --expire=now --all`

README.md

@@ -39,7 +39,7 @@ history rewriting tools](contrib/filter-repo-demos).
 filter-repo requires:
 
   * git >= 2.22.0 at a minimum; [some features](#upstream-improvements)
-    require git >= 2.24.0
+    require git >= 2.24.0 or later
   * python3 >= 3.5
 
 # How do I install it?
@@ -259,7 +259,8 @@ One can kind of hack this together with something like:
       | perl -pe 's%^(M [0-9]+ [0-9a-f]+ )(.*)$%\1my-module/\2%' \
       | perl -pe 's%^(D )(.*)$%\1my-module/\2%' \
       | perl -pe s%refs/tags/%refs/tags/my-module-% \
-      | git -c core.ignorecase=false fast-import --force --quiet
+      | git -c core.ignorecase=false fast-import --date-format=raw-permissive \
+            --force --quiet
   git for-each-ref --format="delete %(refname)" refs/tags/ \
       | grep -v refs/tags/my-module- \
       | git update-ref --stdin
@@ -429,6 +430,9 @@ has also driven numerous improvements to fast-export and fast-import
 (and occasionally other commands) in core git, based on things
 filter-repo needs to do its work:
 
+  * git-2.28.0 (not yet released)
+    * [fast-import: add new --date-format=raw-permissive format](
+      https://git.kernel.org/pub/scm/git/git.git/commit/?id=d42a2fb72f)
   * git-2.24.0
     * [fast-export: handle nested tags](
       https://git.kernel.org/pub/scm/git/git.git/commit/?id=941790d7de)

git-filter-repo

@@ -53,6 +53,7 @@ __all__ = ["Blob", "Reset", "FileChange", "Commit", "Tag", "Progress",
 
 deleted_hash = b'0'*40
 write_marks = True
+date_format_permissive = True
 
 def gettext_poison(msg):
   if "GIT_TEST_GETTEXT_POISON" in os.environ: # pragma: no cover
@@ -1075,15 +1076,6 @@ class FastExportParser(object):
     user_regex = self._user_regexes[usertype]
     (name, email, when) = user_regex.match(self._currentline).groups()
 
-    # TimeZone idiocy; IST is any of four timezones, so someone translated
-    # it to something that was totally invalid...and it got recorded that
-    # way.  Others have suggested just using an invalid timezone that
-    # fast-import will not choke on.  Let's do that.  Note that +051800
-    # seems to be the only weird timezone found in the wild, by me or some
-    # other posts google returned on the subject...
-    if when.endswith(b'+051800'):
-      when = when[0:-7]+b'+0261'
-
     self._advance_currentline()
     return (name, email, when)
 
@@ -2050,6 +2042,9 @@ EXAMPLES
     p = subproc.Popen('git fast-export -h'.split(),
                       stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
     output = p.stdout.read()
+    if b'--anonymize-map' not in output: # pragma: no cover
+      global date_format_permissive
+      date_format_permissive = False
     if b'--mark-tags' not in output: # pragma: no cover
       global write_marks
       write_marks = False
@@ -3633,6 +3628,8 @@ class RepoFilter(object):
       location = ['-C', self._args.target] if self._args.target else []
       fip_cmd = ['git'] + location + ['-c', 'core.ignorecase=false',
                                       'fast-import', '--force', '--quiet']
+      if date_format_permissive:
+        fip_cmd.append('--date-format=raw-permissive')
       if self._args.state_branch:
         target_marks_file = self._load_marks_file(b'target-marks')
         fip_cmd.extend([b'--export-marks='+target_marks_file,

t/t9390/unusual-filtered

@@ -6,8 +6,8 @@ hello
 reset refs/heads/develop
 commit refs/heads/develop
 mark :2
-author Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +0261
-committer Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +0261
+author Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +051800
+committer Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +051800
 data 8
 Initial
 M 100644 :1 greeting

t/t9390/unusual-mailmap

@@ -6,8 +6,8 @@ hello
 reset refs/heads/develop
 commit refs/heads/develop
 mark :2
-author Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +0261
-committer Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +0261
+author Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +051800
+committer Srinivasa Ramanujan <fellow@cambridge.org> 1535228562 +051800
 data 8
 Initial
 M 100644 :1 greeting