D8723: fix: prefetch file contents

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

D8723: fix: prefetch file contents

pulkit (Pulkit Goyal)
rdamazio created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  This prevents the worker subprocesses from contacting the server individually,
  which is either inefficient, or leads to problems if the connection is shared
  among them.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D8723

AFFECTED FILES
  hgext/fix.py

CHANGE DETAILS

diff --git a/hgext/fix.py b/hgext/fix.py
--- a/hgext/fix.py
+++ b/hgext/fix.py
@@ -271,6 +271,11 @@
         basepaths = getbasepaths(repo, opts, workqueue, basectxs)
         fixers = getfixers(ui)
 
+        # Rather than letting each worker independently fetch the files
+        # (which also would add complications for shared/keepalive
+        # connections), prefetch them all first.
+        _prefetchfiles(repo, workqueue, basepaths)
+
         # There are no data dependencies between the workers fixing each file
         # revision, so we can use all available parallelism.
         def getfixes(items):
@@ -630,6 +635,24 @@
                 basectxs[rev].add(pctx)
     return basectxs
 
+def _prefetchfiles(repo, workqueue, basepaths):
+    toprefetch = set()
+
+    # Prefetch the files that will be fixed.
+    for rev, path in workqueue:
+        if rev == wdirrev:
+            continue
+        toprefetch.add((rev, path))
+
+    # Prefetch the base contents for lineranges().
+    for (baserev, fixrev, path), basepath in basepaths.items():
+        toprefetch.add((baserev, basepath))
+
+    if toprefetch:
+        scmutil.prefetchfiles(repo, [
+            (rev, scmutil.matchfiles(repo, [path])) for rev, path in toprefetch
+        ])
+
 
 def fixfile(ui, repo, opts, fixers, fixctx, path, basepaths, basectxs):
     """Run any configured fixers that should affect the file in this context



To: rdamazio, #hg-reviewers
Cc: mercurial-patches, mercurial-devel
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel