diff --git a/CHANGELOG.md b/CHANGELOG.md index bc79559..c5d0e3c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ All notable changes to this project will be documented in this file. Format a date returned by `get-date`: {{date (get-date "monday") "timestamp"}} ``` +* `zk list` now support multiple `--match`/`-m` flags, which allows to search for several tokens appearing in any order in the notes (contributed by [@rktjmp](https://github.com/mickael-menu/zk/pull/268)). ### Fixed diff --git a/docs/editors-integration.md b/docs/editors-integration.md index 6bbf32f..6506631 100644 --- a/docs/editors-integration.md +++ b/docs/editors-integration.md @@ -184,30 +184,31 @@ This LSP command calls `zk list` to search a notebook. It takes two arguments: 1. A path to any file or directory in the notebook, to locate it. 2.
A dictionary of additional options (click to expand) - | Key | Type | Required? | Description | - |------------------|--------------|-----------|-------------------------------------------------------------------------| - | `select` | string array | Yes | List of note fields to return1 | - | `hrefs` | string array | No | Find notes matching the given path, including its descendants | - | `limit` | integer | No | Limit the number of notes found | - | `match` | string | No | Terms to search for in the notes | - | `exactMatch` | boolean | No | Search for exact occurrences of the `match` argument (case insensitive) | - | `excludeHrefs` | string array | No | Ignore notes matching the given path, including its descendants | - | `tags` | string array | No | Find notes tagged with the given tags | - | `mention` | string array | No | Find notes mentioning the title of the given ones | - | `mentionedBy` | string array | No | Find notes whose title is mentioned in the given ones | - | `linkTo` | string array | No | Find notes which are linking to the given ones | - | `linkedBy` | string array | No | Find notes which are linked by the given ones | - | `orphan` | boolean | No | Find notes which are not linked by any other note | - | `related` | string array | No | Find notes which might be related to the given ones | - | `maxDistance` | integer | No | Maximum distance between two linked notes | - | `recursive` | boolean | No | Follow links recursively | - | `created` | string | No | Find notes created on the given date | - | `createdBefore` | string | No | Find notes created before the given date | - | `createdAfter` | string | No | Find notes created after the given date | - | `modified` | string | No | Find notes modified on the given date | - | `modifiedBefore` | string | No | Find notes modified before the given date | - | `modifiedAfter` | string | No | Find notes modified after the given date | - | `sort` | string array | No | Order the notes by the given criterion | + | Key | Type | Required? | Description | + | ------------------ | -------------- | ----------- | ------------------------------------------------------------------------- | + | `select` | string array | Yes | List of note fields to return1 | + | `hrefs` | string array | No | Find notes matching the given path, including its descendants | + | `limit` | integer | No | Limit the number of notes found | + | `match` | string array | No | Terms to search for in the notes | + | `exactMatch` | boolean | No | (deprecated: use `matchStrategy`) Search for exact occurrences of the `match` argument (case insensitive) | + | `matchStrategy` | string | No | Specify match strategy, which may be "fts" (default), "exact" or "re" | + | `excludeHrefs` | string array | No | Ignore notes matching the given path, including its descendants | + | `tags` | string array | No | Find notes tagged with the given tags | + | `mention` | string array | No | Find notes mentioning the title of the given ones | + | `mentionedBy` | string array | No | Find notes whose title is mentioned in the given ones | + | `linkTo` | string array | No | Find notes which are linking to the given ones | + | `linkedBy` | string array | No | Find notes which are linked by the given ones | + | `orphan` | boolean | No | Find notes which are not linked by any other note | + | `related` | string array | No | Find notes which might be related to the given ones | + | `maxDistance` | integer | No | Maximum distance between two linked notes | + | `recursive` | boolean | No | Follow links recursively | + | `created` | string | No | Find notes created on the given date | + | `createdBefore` | string | No | Find notes created before the given date | + | `createdAfter` | string | No | Find notes created after the given date | + | `modified` | string | No | Find notes modified on the given date | + | `modifiedBefore` | string | No | Find notes modified before the given date | + | `modifiedAfter` | string | No | Find notes modified after the given date | + | `sort` | string array | No | Order the notes by the given criterion | 1. As the output of this command might be very verbose and put a heavy load on the LSP client, you need to explicitly set which note fields you want to receive with the `select` option. The following fields are available: `filename`, `filenameStem`, `path`, `absPath`, `title`, `lead`, `body`, `snippets`, `rawContent`, `wordCount`, `tags`, `metadata`, `created`, `modified` and `checksum`. diff --git a/docs/note-filtering.md b/docs/note-filtering.md index a3d711e..8dc01d0 100644 --- a/docs/note-filtering.md +++ b/docs/note-filtering.md @@ -58,6 +58,21 @@ Change the currently used strategy with `--match-strategy ` (or `-M`). list = "zk list --match-strategy re $@" ``` +The `--match` option may be given multiple times, where each argument will be combined with a boolean AND. + +For example, + +```sh +$ zk list --tag "recipe" --match "pizza -pineapple" --match "mushrooms" +``` + +Is equivalent to, + +```sh +$ zk list --tag "recipe" --match "(pizza -pineapple) AND (mushrooms)" +``` + + ### Full-text search (`fts`) The default match strategy is powered by a [full-text search](https://en.wikipedia.org/wiki/Full-text_search) database enabling near-instant results. Queries are not case-sensitive and terms are tokenized, which means that searching for `create` will also match `created` and `creating`. diff --git a/internal/adapter/handlebars/handlebars_test.go b/internal/adapter/handlebars/handlebars_test.go index 95e6d11..a30bdf8 100644 --- a/internal/adapter/handlebars/handlebars_test.go +++ b/internal/adapter/handlebars/handlebars_test.go @@ -238,12 +238,13 @@ func TestDateHelper(t *testing.T) { testString(t, "{{date now 'timestamp'}}", context, "200911172034") testString(t, "{{date now 'timestamp-unix'}}", context, "1258490098") testString(t, "{{date now 'cust: %Y-%m'}}", context, "cust: 2009-11") - testString(t, "{{date now 'elapsed'}}", context, "13 years ago") + testString(t, "{{date now 'elapsed'}}", context, "14 years ago") } func TestGetDateHelper(t *testing.T) { context := map[string]interface{}{"now": time.Date(2009, 11, 17, 20, 34, 58, 651387237, time.UTC)} - testString(t, "{{get-date \"2009-11-17T20:34:58\"}}", context, "2009-11-17 20:34:58 +0000 UTC") + localOffsetAndTZ := time.Now().Format("-0700 MST") + testString(t, "{{get-date \"2009-11-17T20:34:58\"}}", context, "2009-11-17 20:34:58 "+localOffsetAndTZ) } func TestShellHelper(t *testing.T) { diff --git a/internal/adapter/sqlite/note_dao.go b/internal/adapter/sqlite/note_dao.go index b63f9df..5ddc581 100644 --- a/internal/adapter/sqlite/note_dao.go +++ b/internal/adapter/sqlite/note_dao.go @@ -12,7 +12,6 @@ import ( "github.com/mickael-menu/zk/internal/util" "github.com/mickael-menu/zk/internal/util/errors" "github.com/mickael-menu/zk/internal/util/fts5" - "github.com/mickael-menu/zk/internal/util/opt" "github.com/mickael-menu/zk/internal/util/paths" strutil "github.com/mickael-menu/zk/internal/util/strings" ) @@ -421,9 +420,7 @@ func (d *NoteDAO) expandMentionsIntoMatch(opts core.NoteFindOpts) (core.NoteFind } // Expand the mention queries in the match predicate. - match := opts.Match.String() - match += " " + strings.Join(mentionQueries, " OR ") - opts.Match = opt.NewString(match) + opts.Match = append(opts.Match, " ("+strings.Join(mentionQueries, " OR ")+") ") return opts, nil } @@ -519,20 +516,26 @@ func (d *NoteDAO) findRows(opts core.NoteFindOpts, selection noteSelection) (*sq return nil } - if !opts.Match.IsNull() { + if 0 < len(opts.Match) { switch opts.MatchStrategy { case core.MatchStrategyExact: - whereExprs = append(whereExprs, `n.raw_content LIKE '%' || ? || '%' ESCAPE '\'`) - args = append(args, escapeLikeTerm(opts.Match.String(), '\\')) + for _, match := range opts.Match { + whereExprs = append(whereExprs, `n.raw_content LIKE '%' || ? || '%' ESCAPE '\'`) + args = append(args, escapeLikeTerm(match, '\\')) + } case core.MatchStrategyFts: snippetCol = `snippet(fts_match.notes_fts, 2, '', '', '…', 20)` joinClauses = append(joinClauses, "JOIN notes_fts fts_match ON n.id = fts_match.rowid") additionalOrderTerms = append(additionalOrderTerms, `bm25(fts_match.notes_fts, 1000.0, 500.0, 1.0)`) - whereExprs = append(whereExprs, "fts_match.notes_fts MATCH ?") - args = append(args, fts5.ConvertQuery(opts.Match.String())) + for _, match := range opts.Match { + whereExprs = append(whereExprs, "fts_match.notes_fts MATCH ?") + args = append(args, fts5.ConvertQuery(match)) + } case core.MatchStrategyRe: - whereExprs = append(whereExprs, "n.raw_content REGEXP ?") - args = append(args, opts.Match.String()) + for _, match := range opts.Match { + whereExprs = append(whereExprs, "n.raw_content REGEXP ?") + args = append(args, match) + } break } } diff --git a/internal/adapter/sqlite/note_dao_test.go b/internal/adapter/sqlite/note_dao_test.go index 83ba16e..d38991c 100644 --- a/internal/adapter/sqlite/note_dao_test.go +++ b/internal/adapter/sqlite/note_dao_test.go @@ -312,7 +312,7 @@ func TestNoteDAOFindMinimalAll(t *testing.T) { func TestNoteDAOFindMinimalWithFilter(t *testing.T) { testNoteDAO(t, func(tx Transaction, dao *NoteDAO) { notes, err := dao.FindMinimal(core.NoteFindOpts{ - Match: opt.NewString("daily | index"), + Match: []string{"daily | index"}, MatchStrategy: core.MatchStrategyFts, Sorters: []core.NoteSorter{{Field: core.NoteSortWordCount, Ascending: true}}, Limit: 3, @@ -368,7 +368,7 @@ func TestNoteDAOFindTag(t *testing.T) { func TestNoteDAOFindMatch(t *testing.T) { testNoteDAOFind(t, core.NoteFindOpts{ - Match: opt.NewString("daily | index"), + Match: []string{"daily | index"}, MatchStrategy: core.MatchStrategyFts, }, []core.ContextualNote{ @@ -452,10 +452,25 @@ func TestNoteDAOFindMatch(t *testing.T) { ) } +func TestNoteDAOFindMatchWithMultiMatch(t *testing.T) { + testNoteDAOFindPaths(t, + core.NoteFindOpts{ + Match: []string{"daily | index", "second"}, + MatchStrategy: core.MatchStrategyFts, + Sorters: []core.NoteSorter{ + {Field: core.NoteSortPath, Ascending: false}, + }, + }, + []string{ + "log/2021-01-04.md", + }, + ) +} + func TestNoteDAOFindMatchWithSort(t *testing.T) { testNoteDAOFindPaths(t, core.NoteFindOpts{ - Match: opt.NewString("daily | index"), + Match: []string{"daily | index"}, MatchStrategy: core.MatchStrategyFts, Sorters: []core.NoteSorter{ {Field: core.NoteSortPath, Ascending: false}, @@ -474,7 +489,7 @@ func TestNoteDAOFindExactMatch(t *testing.T) { test := func(match string, expected []string) { testNoteDAOFindPaths(t, core.NoteFindOpts{ - Match: opt.NewString(match), + Match: []string{match}, MatchStrategy: core.MatchStrategyExact, }, expected, diff --git a/internal/cli/filtering.go b/internal/cli/filtering.go index e39cd79..5f11646 100644 --- a/internal/cli/filtering.go +++ b/internal/cli/filtering.go @@ -9,7 +9,6 @@ import ( "github.com/mickael-menu/zk/internal/core" dateutil "github.com/mickael-menu/zk/internal/util/date" "github.com/mickael-menu/zk/internal/util/errors" - "github.com/mickael-menu/zk/internal/util/opt" "github.com/mickael-menu/zk/internal/util/strings" ) @@ -19,7 +18,7 @@ type Filtering struct { Interactive bool `kong:"group='filter',short='i',help='Select notes interactively with fzf.'" json:"-"` Limit int `kong:"group='filter',short='n',placeholder='COUNT',help='Limit the number of notes found.'" json:"limit"` - Match string `kong:"group='filter',short='m',placeholder='QUERY',help='Terms to search for in the notes.'" json:"match"` + Match []string `kong:"group='filter',short='m',placeholder='QUERY',help='Terms to search for in the notes.'" json:"match"` MatchStrategy string `kong:"group='filter',short='M',default='fts',placeholder='STRATEGY',help='Text matching strategy among: fts, re, exact.'" json:"matchStrategy"` Exclude []string `kong:"group='filter',short='x',placeholder='PATH',help='Ignore notes matching the given path, including its descendants.'" json:"excludeHrefs"` Tag []string `kong:"group='filter',short='t',help='Find notes tagged with the given tags.'" json:"tags"` @@ -117,11 +116,7 @@ func (f Filtering) ExpandNamedFilters(filters map[string]string, expandedFilters f.ModifiedAfter = parsedFilter.ModifiedAfter } - if f.Match == "" { - f.Match = parsedFilter.Match - } else if parsedFilter.Match != "" { - f.Match = fmt.Sprintf("(%s) AND (%s)", f.Match, parsedFilter.Match) - } + f.Match = append(f.Match, parsedFilter.Match...) if f.MatchStrategy == "" { f.MatchStrategy = parsedFilter.MatchStrategy } @@ -148,7 +143,8 @@ func (f Filtering) NewNoteFindOpts(notebook *core.Notebook) (core.NoteFindOpts, return opts, fmt.Errorf("the --exact-match (-e) option is deprecated, use --match-strategy=exact (-Me) instead") } - opts.Match = opt.NewNotEmptyString(f.Match) + opts.Match = make([]string, len(f.Match)) + copy(opts.Match, f.Match) opts.MatchStrategy, err = core.MatchStrategyFromString(f.MatchStrategy) if err != nil { return opts, err diff --git a/internal/cli/filtering_test.go b/internal/cli/filtering_test.go index 455048d..e67fb18 100644 --- a/internal/cli/filtering_test.go +++ b/internal/cli/filtering_test.go @@ -11,7 +11,7 @@ func TestExpandNamedFiltersNone(t *testing.T) { Path: []string{"path1"}, Limit: 10, Interactive: true, - Match: "match query", + Match: []string{"match query"}, Exclude: []string{"excl-path1", "excl-path2"}, Tag: []string{"tag1", "tag2"}, Mention: []string{"mention1", "mention2"}, @@ -156,7 +156,7 @@ func TestExpandNamedFiltersJoinLitterals(t *testing.T) { func TestExpandNamedFiltersJoinMatch(t *testing.T) { f := Filtering{ Path: []string{"f1", "f2"}, - Match: "(chocolate OR caramel)", + Match: []string{"(chocolate OR caramel)"}, } res, err := f.ExpandNamedFilters( @@ -168,7 +168,7 @@ func TestExpandNamedFiltersJoinMatch(t *testing.T) { ) assert.Nil(t, err) - assert.Equal(t, res.Match, "(((chocolate OR caramel)) AND (banana)) AND (apple)") + assert.Equal(t, res.Match, []string{"(chocolate OR caramel)", "banana", "apple"}) } func TestExpandNamedFiltersExpandsRecursively(t *testing.T) { diff --git a/internal/core/note_find.go b/internal/core/note_find.go index 59d994e..2365a4c 100644 --- a/internal/core/note_find.go +++ b/internal/core/note_find.go @@ -5,14 +5,12 @@ import ( "strings" "time" "unicode/utf8" - - "github.com/mickael-menu/zk/internal/util/opt" ) // NoteFindOpts holds a set of filtering options used to find notes. type NoteFindOpts struct { // Filter used to match the notes with the given MatchStrategy. - Match opt.String + Match []string // Text matching strategy used with Match. MatchStrategy MatchStrategy // Filter by note hrefs. diff --git a/internal/core/notebook.go b/internal/core/notebook.go index 9d58803..e0c3264 100644 --- a/internal/core/notebook.go +++ b/internal/core/notebook.go @@ -235,13 +235,6 @@ func (n *Notebook) FindByHref(href string, allowPartialHref bool) (*MinimalNote, }) } -// FindMatching retrieves the first note matching the given search terms. -func (n *Notebook) FindMatching(terms string) (*MinimalNote, error) { - return n.FindMinimalNote(NoteFindOpts{ - Match: opt.NewNotEmptyString(terms), - }) -} - // FindLinksBetweenNotes retrieves the links between the given notes. func (n *Notebook) FindLinksBetweenNotes(ids []NoteID) ([]ResolvedLink, error) { return n.index.FindLinksBetweenNotes(ids) diff --git a/tests/cmd-graph.tesh b/tests/cmd-graph.tesh index bd97d2a..de09fd0 100644 --- a/tests/cmd-graph.tesh +++ b/tests/cmd-graph.tesh @@ -24,7 +24,7 @@ $ zk graph --help >Filtering > -i, --interactive Select notes interactively with fzf. > -n, --limit=COUNT Limit the number of notes found. -> -m, --match=QUERY Terms to search for in the notes. +> -m, --match=QUERY,... Terms to search for in the notes. > -M, --match-strategy=STRATEGY Text matching strategy among: fts, re, exact. > -x, --exclude=PATH,... Ignore notes matching the given path, > including its descendants. diff --git a/tests/cmd-list-filter-match-exact.tesh b/tests/cmd-list-filter-match-exact.tesh index f4fe252..b43b5c1 100644 --- a/tests/cmd-list-filter-match-exact.tesh +++ b/tests/cmd-list-filter-match-exact.tesh @@ -15,3 +15,34 @@ $ zk list -q --debug-style -Me --match '["न", "म", "स्", "ते"]' > > - Given the Hindi word "नमस्ते": > + +# Mutliple match flags. +$ zk list -q --debug-style -Me --match "thread" --match "mut" +>Concurrency in Rust g7qa.md (just now) +> +> - * Thanks to the [Ownership pattern](88el), Rust has a model of [Fearless concurrency](2cl7). +> * Rust aims to have a small runtime, so it doesn't support [green threads](inbox/my59). +> * Crates exist to add support for green threads if needed. +> * Instead, Rust relies on the OS threads, a model called 1-1. +> +>Mutex inbox/er4k.md (just now) +> +> - * Abbreviation of *mutual exclusion*. +> * An approach to manage safely shared state by allowing only a single thread to access a protected value at one time. +> * A mutex *guards* a protected data with a *locking system*. +> * Managing mutexes is tricky, using [channels](../fwsj) is an easier alternative. +> * The main risk is to create *deadlocks*. +> * Thanks to its [Ownership](../88el) pattern, Rust makes sure we can't mess up when using locks. +> + +# Mutliple match flags. +$ zk list -q --debug-style -Me --match "thread" --match "mutual" +>Mutex inbox/er4k.md (just now) +> +> - * Abbreviation of *mutual exclusion*. +> * An approach to manage safely shared state by allowing only a single thread to access a protected value at one time. +> * A mutex *guards* a protected data with a *locking system*. +> * Managing mutexes is tricky, using [channels](../fwsj) is an easier alternative. +> * The main risk is to create *deadlocks*. +> * Thanks to its [Ownership](../88el) pattern, Rust makes sure we can't mess up when using locks. +> diff --git a/tests/cmd-list-filter-match-fts.tesh b/tests/cmd-list-filter-match-fts.tesh index 8b1f08f..56fcdf1 100644 --- a/tests/cmd-list-filter-match-fts.tesh +++ b/tests/cmd-list-filter-match-fts.tesh @@ -24,6 +24,15 @@ $ zk list -q --debug-style --match 'green channel' > * Instead, Rust… > +# Search for two term by two --match flags (implicit AND). +$ zk list -q --debug-style --match 'green' --match 'channel' +>Concurrency in Rust g7qa.md (just now) +> +> - …runtime, so it doesn't support [green threads](inbox/my59). +> * Crates exist to add support for green threads if needed. +> * Instead, Rust… +> + # Search for two terms with explicit AND. $ zk list -q --debug-style --match 'green AND channel' >Concurrency in Rust g7qa.md (just now) diff --git a/tests/cmd-list-filter-match-re.tesh b/tests/cmd-list-filter-match-re.tesh index eac734e..c9d81fe 100644 --- a/tests/cmd-list-filter-match-re.tesh +++ b/tests/cmd-list-filter-match-re.tesh @@ -15,3 +15,15 @@ $ zk list -q --debug-style -Mr --match 'न.*ते' > > - Given the Hindi word "नमस्ते": > + +# multiple match flags. +$ zk list -q --debug-style -Mr --match "mut.*" --match "thr..d" +>Mutex inbox/er4k.md (just now) +> +> - * Abbreviation of *mutual exclusion*. +> * An approach to manage safely shared state by allowing only a single thread to access a protected value at one time. +> * A mutex *guards* a protected data with a *locking system*. +> * Managing mutexes is tricky, using [channels](../fwsj) is an easier alternative. +> * The main risk is to create *deadlocks*. +> * Thanks to its [Ownership](../88el) pattern, Rust makes sure we can't mess up when using locks. +> diff --git a/tests/cmd-list.tesh b/tests/cmd-list.tesh index 9b4f7a4..76a352d 100644 --- a/tests/cmd-list.tesh +++ b/tests/cmd-list.tesh @@ -32,7 +32,7 @@ $ zk list --help >Filtering > -i, --interactive Select notes interactively with fzf. > -n, --limit=COUNT Limit the number of notes found. -> -m, --match=QUERY Terms to search for in the notes. +> -m, --match=QUERY,... Terms to search for in the notes. > -M, --match-strategy=STRATEGY Text matching strategy among: fts, re, exact. > -x, --exclude=PATH,... Ignore notes matching the given path, > including its descendants.