D7893: nodemap: update the index with the newly written data (when appropriate)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

D7893: nodemap: update the index with the newly written data (when appropriate)

martinvonz (Martin von Zweigbergk)
marmoute created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  If we are to use mmap to read the nodemap data, and if the python code is
  responsible for the IO, we need to refresh the mmap after each write and provide
  it back to the index.
 
  We start this dance without the mmap first.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7893

AFFECTED FILES
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -100,6 +100,8 @@
             with revlog.opener(datafile, 'r+') as fd:
                 fd.seek(target_docket.data_length)
                 fd.write(data)
+                fd.seek(0)
+                new_data = fd.read(target_docket.data_length + len(data))
             target_docket.data_length += len(data)
             target_docket.data_unused += data_changed_count
 
@@ -113,6 +115,7 @@
             data = persistent_data(revlog.index)
         # EXP-TODO: if this is a cache, this should use a cache vfs, not a
         # store vfs
+        new_data = data
         with revlog.opener(datafile, 'w') as fd:
             fd.write(data)
         target_docket.data_length = len(data)
@@ -122,6 +125,9 @@
     with revlog.opener(revlog.nodemap_file, 'w', atomictemp=True) as fp:
         fp.write(target_docket.serialize())
     revlog._nodemap_docket = target_docket
+    if util.safehasattr(revlog.index, "update_nodemap_data"):
+        revlog.index.update_nodemap_data(target_docket, new_data)
+
     # EXP-TODO: if the transaction abort, we should remove the new data and
     # reinstall the old one.
 



To: marmoute, #hg-reviewers
Cc: mercurial-devel
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|

D7893: nodemap: update the index with the newly written data (when appropriate)

martinvonz (Martin von Zweigbergk)
marmoute updated this revision to Diff 19774.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7893?vs=19311&id=19774

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7893/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7893

AFFECTED FILES
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -100,6 +100,8 @@
             with revlog.opener(datafile, 'r+') as fd:
                 fd.seek(target_docket.data_length)
                 fd.write(data)
+                fd.seek(0)
+                new_data = fd.read(target_docket.data_length + len(data))
             target_docket.data_length += len(data)
             target_docket.data_unused += data_changed_count
 
@@ -113,6 +115,7 @@
             data = persistent_data(revlog.index)
         # EXP-TODO: if this is a cache, this should use a cache vfs, not a
         # store vfs
+        new_data = data
         with revlog.opener(datafile, 'w') as fd:
             fd.write(data)
         target_docket.data_length = len(data)
@@ -122,6 +125,9 @@
     with revlog.opener(revlog.nodemap_file, 'w', atomictemp=True) as fp:
         fp.write(target_docket.serialize())
     revlog._nodemap_docket = target_docket
+    if util.safehasattr(revlog.index, "update_nodemap_data"):
+        revlog.index.update_nodemap_data(target_docket, new_data)
+
     # EXP-TODO: if the transaction abort, we should remove the new data and
     # reinstall the old one.
 



To: marmoute, #hg-reviewers
Cc: mercurial-devel
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|

D7893: nodemap: update the index with the newly written data (when appropriate)

martinvonz (Martin von Zweigbergk)
In reply to this post by martinvonz (Martin von Zweigbergk)
marmoute updated this revision to Diff 19801.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7893?vs=19774&id=19801

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7893/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7893

AFFECTED FILES
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -100,6 +100,8 @@
             with revlog.opener(datafile, b'r+') as fd:
                 fd.seek(target_docket.data_length)
                 fd.write(data)
+                fd.seek(0)
+                new_data = fd.read(target_docket.data_length + len(data))
             target_docket.data_length += len(data)
             target_docket.data_unused += data_changed_count
 
@@ -113,6 +115,7 @@
             data = persistent_data(revlog.index)
         # EXP-TODO: if this is a cache, this should use a cache vfs, not a
         # store vfs
+        new_data = data
         with revlog.opener(datafile, b'w') as fd:
             fd.write(data)
         target_docket.data_length = len(data)
@@ -122,6 +125,9 @@
     with revlog.opener(revlog.nodemap_file, b'w', atomictemp=True) as fp:
         fp.write(target_docket.serialize())
     revlog._nodemap_docket = target_docket
+    if util.safehasattr(revlog.index, "update_nodemap_data"):
+        revlog.index.update_nodemap_data(target_docket, new_data)
+
     # EXP-TODO: if the transaction abort, we should remove the new data and
     # reinstall the old one.
 



To: marmoute, #hg-reviewers
Cc: mercurial-devel
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|

D7893: nodemap: update the index with the newly written data (when appropriate)

martinvonz (Martin von Zweigbergk)
In reply to this post by martinvonz (Martin von Zweigbergk)
marmoute added a comment.
marmoute updated this revision to Diff 19845.


  small doc update on .#s[1]

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7893?vs=19801&id=19845

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7893/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7893

AFFECTED FILES
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -100,6 +100,8 @@
             with revlog.opener(datafile, b'r+') as fd:
                 fd.seek(target_docket.data_length)
                 fd.write(data)
+                fd.seek(0)
+                new_data = fd.read(target_docket.data_length + len(data))
             target_docket.data_length += len(data)
             target_docket.data_unused += data_changed_count
 
@@ -113,6 +115,7 @@
             data = persistent_data(revlog.index)
         # EXP-TODO: if this is a cache, this should use a cache vfs, not a
         # store vfs
+        new_data = data
         with revlog.opener(datafile, b'w') as fd:
             fd.write(data)
         target_docket.data_length = len(data)
@@ -122,6 +125,9 @@
     with revlog.opener(revlog.nodemap_file, b'w', atomictemp=True) as fp:
         fp.write(target_docket.serialize())
     revlog._nodemap_docket = target_docket
+    if util.safehasattr(revlog.index, "update_nodemap_data"):
+        revlog.index.update_nodemap_data(target_docket, new_data)
+
     # EXP-TODO: if the transaction abort, we should remove the new data and
     # reinstall the old one.
 



To: marmoute, #hg-reviewers
Cc: mercurial-devel
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|

D7893: nodemap: update the index with the newly written data (when appropriate)

martinvonz (Martin von Zweigbergk)
In reply to this post by martinvonz (Martin von Zweigbergk)
marmoute added a comment.
marmoute updated this revision to Diff 19905.


  rebase to latest default

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7893?vs=19845&id=19905

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7893/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7893

AFFECTED FILES
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -100,6 +100,8 @@
             with revlog.opener(datafile, b'r+') as fd:
                 fd.seek(target_docket.data_length)
                 fd.write(data)
+                fd.seek(0)
+                new_data = fd.read(target_docket.data_length + len(data))
             target_docket.data_length += len(data)
             target_docket.data_unused += data_changed_count
 
@@ -113,6 +115,7 @@
             data = persistent_data(revlog.index)
         # EXP-TODO: if this is a cache, this should use a cache vfs, not a
         # store vfs
+        new_data = data
         with revlog.opener(datafile, b'w') as fd:
             fd.write(data)
         target_docket.data_length = len(data)
@@ -122,6 +125,9 @@
     with revlog.opener(revlog.nodemap_file, b'w', atomictemp=True) as fp:
         fp.write(target_docket.serialize())
     revlog._nodemap_docket = target_docket
+    if util.safehasattr(revlog.index, "update_nodemap_data"):
+        revlog.index.update_nodemap_data(target_docket, new_data)
+
     # EXP-TODO: if the transaction abort, we should remove the new data and
     # reinstall the old one.
 



To: marmoute, #hg-reviewers
Cc: mercurial-devel
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|

D7893: nodemap: update the index with the newly written data (when appropriate)

martinvonz (Martin von Zweigbergk)
In reply to this post by martinvonz (Martin von Zweigbergk)
Closed by commit rHG6ecc34b31137: nodemap: update the index with the newly written data (when appropriate) (authored by marmoute).
This revision was automatically updated to reflect the committed changes.
This revision was not accepted when it landed; it landed in state "Needs Review".

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D7893?vs=19905&id=20126

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7893/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7893

AFFECTED FILES
  mercurial/revlogutils/nodemap.py

CHANGE DETAILS

diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -100,6 +100,8 @@
             with revlog.opener(datafile, b'r+') as fd:
                 fd.seek(target_docket.data_length)
                 fd.write(data)
+                fd.seek(0)
+                new_data = fd.read(target_docket.data_length + len(data))
             target_docket.data_length += len(data)
             target_docket.data_unused += data_changed_count
 
@@ -113,6 +115,7 @@
             data = persistent_data(revlog.index)
         # EXP-TODO: if this is a cache, this should use a cache vfs, not a
         # store vfs
+        new_data = data
         with revlog.opener(datafile, b'w') as fd:
             fd.write(data)
         target_docket.data_length = len(data)
@@ -122,6 +125,9 @@
     with revlog.opener(revlog.nodemap_file, b'w', atomictemp=True) as fp:
         fp.write(target_docket.serialize())
     revlog._nodemap_docket = target_docket
+    if util.safehasattr(revlog.index, "update_nodemap_data"):
+        revlog.index.update_nodemap_data(target_docket, new_data)
+
     # EXP-TODO: if the transaction abort, we should remove the new data and
     # reinstall the old one.
 



To: marmoute, #hg-reviewers
Cc: mercurial-devel
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel