Commit Graph

272 Commits

Author SHA1 Message Date
Elijah Newren
e5a3a134b1 filter-repo: retain refs that happen to point at commits that become empty
It may be that the only time a reference is shown in the fast-export stream
is for a commit which will become empty due to the filtering.  We do not
want such refs to be left out and thus not be updated; we want them to
instead be set to the nearest non-empty ancestor.  Only if it has no
non-empty ancestor would we want it to be stripped out.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:55 -08:00
Elijah Newren
064e2c0ef4 filter-repo: add --quiet option
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:55 -08:00
Elijah Newren
e6731225f8 filter-repo: allow importing into an empty repo
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:55 -08:00
Elijah Newren
0ecfad479e filter-repo: add parsing of more types of fast-export commands
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:55 -08:00
Elijah Newren
a7531af120 filter-repo: filtering note with empty commits and --dry-run
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:55 -08:00
Elijah Newren
3914c1377b filter-repo: add --stdin option
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:55 -08:00
Elijah Newren
cf460406d7 filter-repo: code cleanup
Slightly re-order the code to make input, output, and filtering sections
distinct.  Also, avoid running `git fast-import` at all when we're in
--dry-run mode.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:55 -08:00
Elijah Newren
3f8ce81aa2 filter-repo: prune parents made redundant by filtering
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:54 -08:00
Elijah Newren
f103735e01 filter-repo: implement --debug
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:27 -08:00
Elijah Newren
9499c78b94 filter-repo: implement --dry-run
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:27 -08:00
Elijah Newren
40f90c9cb8 filter-repo: move git_dir determination into function for future re-use
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:27 -08:00
Elijah Newren
4fed91af18 filter-repo: allow FastExportFilter to take file-like objects
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:27 -08:00
Elijah Newren
636a3cf575 filter-repo: add basic path filtering
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:27 -08:00
Elijah Newren
a427a80322 filter-repo: skeleton of new tool
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:26 -08:00
Elijah Newren
e2b8b68d3a filter-repo: use subprocess explicitly; make it easier to wrap debug version
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:04 -08:00
Elijah Newren
3457348c63 filter-repo: remove excessive hashes
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:44:03 -08:00
Elijah Newren
badd03105b filter-repo: fix really old typo
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:31 -08:00
Elijah Newren
2977725634 filter-repo: enable usage with --no-data
Regex is kinda sloppy, someone should slap me and fix that.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:31 -08:00
Elijah Newren
a6b008eaac filter-repo: don't default to creating bare repositories for importing into
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:31 -08:00
Elijah Newren
7e0b7a9fc2 filter-repo: only require two arguments to record_id_rename
We always called record_id_rename with handle_transitivity set to True, and
I do not know of a use case that would do otherwise so let's just hardcode
that value.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:31 -08:00
Elijah Newren
a8b39e58b3 filter-repo: rename _TimeZone to FixedTimeZone for external usage
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:31 -08:00
Elijah Newren
e7311b6db9 filter-repo: reinstate the id_offset
My idea to use --export-marks and --import-marks to avoid the need for the
id_offset was not tested and apparently a bad idea.  When splicing together
multiple repositories, the second will croak if we pass it --import-marks
with a file having sha1sums that don't exist in that repository.

I'm afraid this might conflict with the --import-marks stuff used in collab
so I've only enabled it for streams beyond the first.  So there might be an
issue using --import-marks on a second or later fast-export output stream,
but I can't think of a use case for that...

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:31 -08:00
Elijah Newren
fbd3a04b7f filter-repo: some wording and line-wrap cleanups
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:31 -08:00
James Foucar
f74761e803 filter-repo: lots of documentation additions
Signed-off-by: Elijah Newren <enewren@sandia.gov>
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:43:29 -08:00
Elijah Newren
4d5a9cf552 filter-repo: add a few documentation files
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
9a85c6a1ae filter-repo: fix parsing of filechanges with skipped blobs
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
d2d6d79db0 filter-repo: fix __all__ declaration
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
adc3d52d26 filter-repo: add parsing of progress and checkpoint fast-export objects
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
48aaedfc32 filter-repo: add parsing of (annotated) tag objects
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
a644632a83 filter-repo: provide default args for get_commit_count()
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
86a86fc074 filter-repo: remove the id_offset
Filtering input from multiple repositories can still be done; however, to
avoid overloading of mark numbers, one should pass --export-marks=<file>
to the first git fast-export and pass --import-marks=<file> to the second.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
bfbc07d3a7 filter-repo: encapsulate input line advancement
Have all callers of input.readline() be done through _advance_nextline()

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
85b1980d17 filter-repo: avoid using mark ids referred to in an --import-marks file
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
a20bf1957f filter-repo: cleanups to gathering the commit count
Two things:
  * rename get_total_commits -> get_commit_count
  * accept rev-list arguments

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
d099d2628b filter-repo: automatically drop commits whose changes are filtered out
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
4998de6751 filter-repo: handle ahead-of-UTC timezones too
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
307a31fd54 filter-repo: have author_date and committer_date be datetime objects
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
35fdb05c3c filter-repo: streamline common/simple cases to require fewer calls and args
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
cb29d84f48 filter-repo: fix skipping of blob files
Make sure commits don't reference skipped blob files.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
aba66f6d42 filter-repo: duct tape and bailing wire...
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
7437d62329 filter-repo: fix id renaming
Splicing repositories and dropping commits require different id
renaming.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
358e9826d4 filter-repo: better handling of passing --all to fast-export
Make --all be a default argument for fast-export, not a mandatory
argument.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
a594ea530a filter-repo: ensure new files from spliced-in commits aren't dropped at merges
git-fast-import requires that file changes listed in a merge commit be
relative to the first parent.  Thus, if I've added new files on a branch
being merged in from the second or later parents, I need to manually
modify the list of files in the merge commit as well.  In order to do that,
as soon as I splice in any commit, I have to record the list of new files
for both that commit and every descendant it has.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
dd5665b7ec filter-repo: handle adding interleaving commits from separate repositories
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
e4a4787393 filter-repo: make sure git-fast-import has really finished when we exit
Also, provide an OutputStream class, to make it easy to still direct all
output to some file rather than always sending to git fast-import.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
2581e7a0e6 filter-repo: silence verbose fast-import output
Turn off fast-import stat output but do not squelch all error messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
5faec262d3 filter-repo: make skipping and later dumping easier
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
7371f8e3e4 filter-repo: add counting of objects, as well as commits
Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
3d10238a47 filter-repo: make it easier to skip blobs & commits
Automatically do renaming of references to commits that were skipped, and
automatically remove skipped blobs from the output of commits that
reference them.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00
Elijah Newren
2c769de150 filter-repo: work around git-fast-export bug
Explicitly specify --topo-order; git-fast-export fails on some topologies
unless it traverses in topological order.

Signed-off-by: Elijah Newren <newren@gmail.com>
2019-02-09 14:42:23 -08:00