Commit Graph

42 Commits (86a86fc074bd3eb8a235a5969e97eb81acca9e03)
 

Author SHA1 Message Date
Elijah Newren 86a86fc074 filter-repo: remove the id_offset
Filtering input from multiple repositories can still be done; however, to
avoid overloading of mark numbers, one should pass --export-marks=<file>
to the first git fast-export and pass --import-marks=<file> to the second.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren bfbc07d3a7 filter-repo: encapsulate input line advancement
Have all callers of input.readline() be done through _advance_nextline()

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 85b1980d17 filter-repo: avoid using mark ids referred to in an --import-marks file
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren a20bf1957f filter-repo: cleanups to gathering the commit count
Two things:
  * rename get_total_commits -> get_commit_count
  * accept rev-list arguments

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren d099d2628b filter-repo: automatically drop commits whose changes are filtered out
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 4998de6751 filter-repo: handle ahead-of-UTC timezones too
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 307a31fd54 filter-repo: have author_date and committer_date be datetime objects
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 35fdb05c3c filter-repo: streamline common/simple cases to require fewer calls and args
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren cb29d84f48 filter-repo: fix skipping of blob files
Make sure commits don't reference skipped blob files.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren aba66f6d42 filter-repo: duct tape and bailing wire...
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 7437d62329 filter-repo: fix id renaming
Splicing repositories and dropping commits require different id
renaming.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 358e9826d4 filter-repo: better handling of passing --all to fast-export
Make --all be a default argument for fast-export, not a mandatory
argument.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren a594ea530a filter-repo: ensure new files from spliced-in commits aren't dropped at merges
git-fast-import requires that file changes listed in a merge commit be
relative to the first parent.  Thus, if I've added new files on a branch
being merged in from the second or later parents, I need to manually
modify the list of files in the merge commit as well.  In order to do that,
as soon as I splice in any commit, I have to record the list of new files
for both that commit and every descendant it has.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren dd5665b7ec filter-repo: handle adding interleaving commits from separate repositories
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren e4a4787393 filter-repo: make sure git-fast-import has really finished when we exit
Also, provide an OutputStream class, to make it easy to still direct all
output to some file rather than always sending to git fast-import.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 2581e7a0e6 filter-repo: silence verbose fast-import output
Turn off fast-import stat output but do not squelch all error messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 5faec262d3 filter-repo: make skipping and later dumping easier
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 7371f8e3e4 filter-repo: add counting of objects, as well as commits
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 3d10238a47 filter-repo: make it easier to skip blobs & commits
Automatically do renaming of references to commits that were skipped, and
automatically remove skipped blobs from the output of commits that
reference them.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 2c769de150 filter-repo: work around git-fast-export bug
Explicitly specify --topo-order; git-fast-export fails on some topologies
unless it traverses in topological order.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren bf5e92d02a filter-repo: portability fixes
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 471e9d8684 filter-repo: rewrite to not use pyparsing in order to avoid memory madness
pyparsing sucks a whole file into memory at a time and then parses, which
is really bad in this case since the output from git-fast-export is huge.
I entered disk swapping madness pretty easily.  So, now I just do my own
manual parsing.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren ae486e85b8 filter-repo: small restructurings for the big sierra import
* Allow hooking up (and filtering) multiple git fast-export's to one import
* Allow user callbacks to force dumping of object in order to reference it
  with subsequent inserted objects
* Put the separate callbacks and global vars in the calling program into a
  combined class

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 69497ac6e6 filter-repo: add get_total_commits() function, finish transition to a module
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 28cc91054e filter-repo: fix handling of ids of blobs and commits
My prior handlings of marks would only work if there were not additions
or removals from the fast-export stream.  Further, I referred to these as
marks even though I really only accept idnum values, not sha1s or anything
else.  So, now I refer to these as ids everywhere, and I am much more
careful in my handling of ids.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 94f0ccfd80 filter-repo: call everything_callback as necessary, fix commit_callback
The commit_callback call was trying to pass a Reset object, which was
not defined.  Copy-n-paste-n-forget-to-replace isn't good.  Now it passes
a Commit object.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 9cd296655a filter-repo: rename functions a bit, make filter object creation explicit
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 207c6d0c16 filter-repo: pipe output to git-fast-import now to create a new repository
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 0d9568684c filter-repo: match git-fast-export spacing after reset commands
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 003dd21714 filter-repo: add ability to handle deleted files in commits
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren c92a4e471e filter-repo: fix parsing bug in Reset object creation
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren b029443a6f filter-repo: fix indexing bug in Commit object creation
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 392d09d084 filter-repo: don't hardcode sys.stdout, I'll eventually want to pipe elsewhere
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 11057e874e filter-repo: add a FileChanges object, for changes that are part of a commit
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 586d65270b filter-repo: add parsing of commits
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren f6f4e5fbbf filter-repo: match fast-import grammar slightly better
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren ff95c771d8 filter-repo: prevent pyparsing from expanding tabs to spaces
We are not parsing simple text; we're parsing data and need to be able to
print that data unmunged.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren de7aeb64bc filter-repo: add parsing of branch resets
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren f990dda9ad filter-repo: allow random blob insertion and creation without specifying marks
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 163e299ed7 filter-repo: handle multiple blobs, require all input to be parsed, nice errors
Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren eb4afc4e78 filter-repo: add GitElement and Blob classes, and a FastExport Parser class
We still only parse a single blob, but this should put the infrastructure
in place for parsing more output from git-fast-export.

Signed-off-by: Elijah Newren <newren@gmail.com>
5 years ago
Elijah Newren 2b34e5c25d filter-repo: initial import
This initial version can parse git-fast-export blobs in exact-data format,
but not much else yet.

Signed-off-by: Elijah Newren <newren@gmail.com>
6 years ago