Homestyx hydra
Diffusion hydra 3e589cdd73ba

Add a workflow for populating (or depopulating) the common ngrams table

Description

Add a workflow for populating (or depopulating) the common ngrams table

Summary:
Depends on D18672. Ref T13000. This does an on-demand build of the common ngrams table.

Plan here is:

  • Push to secure.
  • Build the common ngrams table here.
  • See if stuff breaks?

If it looks okay on this dataset, we can build out the GC support and try it in production.

Test Plan:

  • Locally, my dataset has a bunch of bin/lipsum tasks with similar, common words.
  • Verified that ipsum terms now skip ngrams. For "lorem ipsum" search performance actually IMPROVED by skipping the ngrams table (12s to 9s).
  • Queried for normal terms, got very fast results using the ngram table, as normal.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13000

Differential Revision: https://secure.phabricator.com/D18673

Details

Provenance
epriestleyAuthored on Oct 2 2017, 7:14 PM
sirocylPushed on Oct 16 2024, 5:49 AM
Parents
R1:1de130c9f5b2: Allow the Ferret engine to remove "common" ngrams from the index
Branches
Unknown
Tags
Unknown

Event Timeline