Proper way to migrate HG 3.1.2 to 4.8.2 repos

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov

Hello

I've tried to move all existing repos with largefiles enable from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG 4.8.2)

That what I've done is to move with rsync but have problem with user pushes in repos with largefiles:

mod_wsgi (pid=3945): Exception occurred processing WSGI script '/var/www/vhost/hg/cgi-bin/hgwebdir.w

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py", line 352, in run_wsgi
    for r in self._runwsgi(req, res):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 307, in run_wsgi
    for r in self._runwsgi(req, res, repo):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 333, in _runwsgi
    rctx, req, res, self.check_perm)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 221, in handlewsgireque

    _callhttp(repo, req, res, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 436, in _callhttp
    rsp = wireprotov1server.dispatch(repo, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 74, in dispatch
    return func(repo, proto, *args)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 196, in batch
    result = func(repo, proto, *[data[k] for k in keys])
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py", line 89, in statlfile
    filename = lfutil.findfile(repo, sha)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 111, in findfile
    elif inusercache(repo.ui, hash):
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 100, in inusercache
    path = usercachepath(ui, hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 71, in usercachepath
    return os.path.join(_usercachedir(ui), hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 97, in _usercachedir
    raise error.Abort(_('unknown %s usercache location') % name)
Abort: unknown largefiles usercache location
Also tried with clone old-repo >> push new-repo but again HTTP 500 error as above

--
 
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
www.optixco.com


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos

Augie Fackler-2
It looks like you need to configure a filesystem path related to large files - maybe look in the .hg/hgrc for relevant [largefiles] things?

On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]> wrote:

Hello

I've tried to move all existing repos with largefiles enable from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG 4.8.2)

That what I've done is to move with rsync but have problem with user pushes in repos with largefiles:

mod_wsgi (pid=3945): Exception occurred processing WSGI script '/var/www/vhost/hg/cgi-bin/hgwebdir.w

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py", line 352, in run_wsgi
    for r in self._runwsgi(req, res):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 307, in run_wsgi
    for r in self._runwsgi(req, res, repo):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 333, in _runwsgi
    rctx, req, res, self.check_perm)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 221, in handlewsgireque

    _callhttp(repo, req, res, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 436, in _callhttp
    rsp = wireprotov1server.dispatch(repo, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 74, in dispatch
    return func(repo, proto, *args)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 196, in batch
    result = func(repo, proto, *[data[k] for k in keys])
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py", line 89, in statlfile
    filename = lfutil.findfile(repo, sha)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 111, in findfile
    elif inusercache(repo.ui, hash):
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 100, in inusercache
    path = usercachepath(ui, hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 71, in usercachepath
    return os.path.join(_usercachedir(ui), hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 97, in _usercachedir
    raise error.Abort(_('unknown %s usercache location') % name)
Abort: unknown largefiles usercache location
Also tried with clone old-repo >> push new-repo but again HTTP 500 error as above

--
 
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
www.optixco.com

_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Augie Fackler-2
Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email] <[hidden email]> wrote:

At existing installation nothing else is configured except largefiles = to enable it.
The largefiles are stored in the root of the.existing repos (.hg/largefiles)
Didn't dig anything about usercache paths and respectively largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email]


It looks like you need to configure a filesystem path related to large files - maybe look in the .hg/hgrc for relevant [largefiles] things?

On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]> wrote:

Hello

I've tried to move all existing repos with largefiles enable from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG 4.8.2)

That what I've done is to move with rsync but have problem with user pushes in repos with largefiles:

mod_wsgi (pid=3945): Exception occurred processing WSGI script '/var/www/vhost/hg/cgi-bin/hgwebdir.w

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py", line 352, in run_wsgi
    for r in self._runwsgi(req, res):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 307, in run_wsgi
    for r in self._runwsgi(req, res, repo):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 333, in _runwsgi
    rctx, req, res, self.check_perm)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 221, in handlewsgireque

    _callhttp(repo, req, res, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 436, in _callhttp
    rsp = wireprotov1server.dispatch(repo, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 74, in dispatch
    return func(repo, proto, *args)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 196, in batch
    result = func(repo, proto, *[data[k] for k in keys])
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py", line 89, in statlfile
    filename = lfutil.findfile(repo, sha)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 111, in findfile
    elif inusercache(repo.ui, hash):
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 100, in inusercache
    path = usercachepath(ui, hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 71, in usercachepath
    return os.path.join(_usercachedir(ui), hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 97, in _usercachedir
    raise error.Abort(_('unknown %s usercache location') % name)
Abort: unknown largefiles usercache location
Also tried with clone old-repo >> push new-repo but again HTTP 500 error as above

--
 
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
www.optixco.com

_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial



_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2
If `hg config largefiles` on the old system doesn’t show the 'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email] <[hidden email]> wrote:

At existing installation nothing else is configured except largefiles = to enable it.
The largefiles are stored in the root of the.existing repos (.hg/largefiles)
Didn't dig anything about usercache paths and respectively largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email]


It looks like you need to configure a filesystem path related to large files - maybe look in the .hg/hgrc for relevant [largefiles] things?

On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]> wrote:

Hello

I've tried to move all existing repos with largefiles enable from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG 4.8.2)

That what I've done is to move with rsync but have problem with user pushes in repos with largefiles:

mod_wsgi (pid=3945): Exception occurred processing WSGI script '/var/www/vhost/hg/cgi-bin/hgwebdir.w

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py", line 352, in run_wsgi
    for r in self._runwsgi(req, res):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 307, in run_wsgi
    for r in self._runwsgi(req, res, repo):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 333, in _runwsgi
    rctx, req, res, self.check_perm)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 221, in handlewsgireque

    _callhttp(repo, req, res, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 436, in _callhttp
    rsp = wireprotov1server.dispatch(repo, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 74, in dispatch
    return func(repo, proto, *args)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 196, in batch
    result = func(repo, proto, *[data[k] for k in keys])
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py", line 89, in statlfile
    filename = lfutil.findfile(repo, sha)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 111, in findfile
    elif inusercache(repo.ui, hash):
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 100, in inusercache
    path = usercachepath(ui, hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 71, in usercachepath
    return os.path.join(_usercachedir(ui), hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 97, in _usercachedir
    raise error.Abort(_('unknown %s usercache location') % name)
Abort: unknown largefiles usercache location
Also tried with clone old-repo >> push new-repo but again HTTP 500 error as above

--
 
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
www.optixco.com

_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial



_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov

On old system there isn't such variable set? On the new too doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto enable largefile.
How to migrate successfully - just to setup usercache on the new in /etc/mercurial/hgrc is that right ?


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the 'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email] <[hidden email]> wrote:

At existing installation nothing else is configured except largefiles = to enable it.
The largefiles are stored in the root of the.existing repos (.hg/largefiles)
Didn't dig anything about usercache paths and respectively largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email]


It looks like you need to configure a filesystem path related to large files - maybe look in the .hg/hgrc for relevant [largefiles] things?

On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]> wrote:

Hello

I've tried to move all existing repos with largefiles enable from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG 4.8.2)

That what I've done is to move with rsync but have problem with user pushes in repos with largefiles:

mod_wsgi (pid=3945): Exception occurred processing WSGI script '/var/www/vhost/hg/cgi-bin/hgwebdir.w

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py", line 352, in run_wsgi
    for r in self._runwsgi(req, res):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 307, in run_wsgi
    for r in self._runwsgi(req, res, repo):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 333, in _runwsgi
    rctx, req, res, self.check_perm)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 221, in handlewsgireque

    _callhttp(repo, req, res, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 436, in _callhttp
    rsp = wireprotov1server.dispatch(repo, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 74, in dispatch
    return func(repo, proto, *args)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 196, in batch
    result = func(repo, proto, *[data[k] for k in keys])
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py", line 89, in statlfile
    filename = lfutil.findfile(repo, sha)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 111, in findfile
    elif inusercache(repo.ui, hash):
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 100, in inusercache
    path = usercachepath(ui, hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 71, in usercachepath
    return os.path.join(_usercachedir(ui), hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 97, in _usercachedir
    raise error.Abort(_('unknown %s usercache location') % name)
Abort: unknown largefiles usercache location
Also tried with clone old-repo >> push new-repo but again HTTP 500 error as above

--
 
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
www.optixco.com

_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial


--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov

So after quick test - it is enough to set usercache location in global hg config.
But stored there files are usual in .hg/largefiles per repo on old installation.
Thought for me that could be a bug when if is not setup global usercache folder to fallback in subfolder largefiles in repo ?

On 18.9.2019 г. 8:05, Dimitar Maznekov wrote:

On old system there isn't such variable set? On the new too doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto enable largefile.
How to migrate successfully - just to setup usercache on the new in /etc/mercurial/hgrc is that right ?


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the 'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email] <[hidden email]> wrote:

At existing installation nothing else is configured except largefiles = to enable it.
The largefiles are stored in the root of the.existing repos (.hg/largefiles)
Didn't dig anything about usercache paths and respectively largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email]


It looks like you need to configure a filesystem path related to large files - maybe look in the .hg/hgrc for relevant [largefiles] things?

On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]> wrote:

Hello

I've tried to move all existing repos with largefiles enable from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG 4.8.2)

That what I've done is to move with rsync but have problem with user pushes in repos with largefiles:

mod_wsgi (pid=3945): Exception occurred processing WSGI script '/var/www/vhost/hg/cgi-bin/hgwebdir.w

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py", line 352, in run_wsgi
    for r in self._runwsgi(req, res):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 307, in run_wsgi
    for r in self._runwsgi(req, res, repo):
  File "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py", line 333, in _runwsgi
    rctx, req, res, self.check_perm)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 221, in handlewsgireque

    _callhttp(repo, req, res, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py", line 436, in _callhttp
    rsp = wireprotov1server.dispatch(repo, proto, cmd)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 74, in dispatch
    return func(repo, proto, *args)
  File "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py", line 196, in batch
    result = func(repo, proto, *[data[k] for k in keys])
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py", line 89, in statlfile
    filename = lfutil.findfile(repo, sha)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 111, in findfile
    elif inusercache(repo.ui, hash):
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 100, in inusercache
    path = usercachepath(ui, hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 71, in usercachepath
    return os.path.join(_usercachedir(ui), hash)
  File "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py", line 97, in _usercachedir
    raise error.Abort(_('unknown %s usercache location') % name)
Abort: unknown largefiles usercache location
Also tried with clone old-repo >> push new-repo but again HTTP 500 error as above

--
 
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
www.optixco.com

_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial


--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com

--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2
In reply to this post by Dimitar Maznekov
On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov  
<[hidden email]> wrote:

> On old system there isn't such variable set? On the new too doesn't  
> exists?
> But on the old HG 3.1.2 works... and have 50+ storages with auto enable
> largefile.
> How to migrate successfully - just to setup usercache on the new in
> /etc/mercurial/hgrc is that right ?

The config isn't set by default.  And when it's not set, it falls back to  
a platform specific directory.

     https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache

So two options here:

   1) rsync that directory over to the new system, in the same place, and  
forget about the config.
   2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'` on the  
new machine, and forget about the config.

The config setting is only needed for fancier stuff.

>
> On 17.9.2019 г. 21:00, Matt Harbison wrote:
>> If `hg config largefiles` on the old system doesn’t show the
>> 'usercache' property, then it falls back to a platform specific  
>> location:
>>
>> https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73
>>
>>
>> Obviously you’ll want to use the variables that the server process sees.
>>
>>
>>
>> On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>> Huh. I would have expected this to just work. Matt, any ideas?
>>>
>>>> On Sep 17, 2019, at 11:44, [hidden email]
>>>> <mailto:[hidden email]> <[hidden email]
>>>> <mailto:[hidden email]>> wrote:
>>>>
>>>> At existing installation nothing else is configured except
>>>> largefiles = to enable it.
>>>> The largefiles are stored in the root of the.existing repos
>>>> (.hg/largefiles)
>>>> Didn't dig anything about usercache paths and respectively
>>>> largefiles for days till now
>>>>
>>>> Изпратено от моя Huawei Mobile
>>>>
>>>>
>>>> -------- Оригинално съобщение --------
>>>> Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
>>>> От: Augie Fackler
>>>> До: "D.Maznekov"
>>>> Як: [hidden email] <mailto:[hidden email]>
>>>>
>>>>
>>>>     It looks like you need to configure a filesystem path related to
>>>>     large files - maybe look in the .hg/hgrc for relevant
>>>>     [largefiles] things?
>>>>
>>>>>     On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]
>>>>>     <mailto:[hidden email]>> wrote:
>>>>>
>>>>>     Hello
>>>>>
>>>>>     I've tried to move all existing repos with largefiles enable
>>>>>     from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG
>>>>>     4.8.2)
>>>>>
>>>>>     That what I've done is to move with rsync but have problem with
>>>>>     user pushes in repos with largefiles:
>>>>>
>>>>>     |mod_wsgi (pid=3945): Exception occurred processing WSGI script
>>>>>     '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most recent
>>>>>     call last): File
>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
>>>>>     line 352, in run_wsgi for r in self._runwsgi(req, res): File
>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
>>>>>     line 307, in run_wsgi for r in self._runwsgi(req, res, repo):
>>>>>     File
>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
>>>>>     line 333, in _runwsgi rctx, req, res, self.check_perm) File
>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
>>>>>     line 221, in handlewsgireque _callhttp(repo, req, res, proto,
>>>>>     cmd) File
>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
>>>>>     line 436, in _callhttp rsp = wireprotov1server.dispatch(repo,
>>>>>     proto, cmd) File
>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
>>>>>     line 74, in dispatch return func(repo, proto, *args) File
>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
>>>>>     line 196, in batch result = func(repo, proto, *[data[k] for k
>>>>>     in keys]) File
>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
>>>>>     line 89, in statlfile filename = lfutil.findfile(repo, sha)
>>>>>     File
>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>     line 111, in findfile elif inusercache(repo.ui, hash): File
>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>     line 100, in inusercache path = usercachepath(ui, hash) File
>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>     line 71, in usercachepath return
>>>>>     os.path.join(_usercachedir(ui), hash) File
>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>     line 97, in _usercachedir raise error.Abort(_('unknown %s
>>>>>     usercache location') % name) Abort: unknown largefiles
>>>>>     usercache location|
>>>>>     Also tried with clone old-repo >> push new-repo but again HTTP
>>>>>     500 error as above
>>>>>
>>>>>     --
>>>>>
>>>>>     *Best regards: D.Maznekov*
>>>>>     DevTeam ThermalVision department
>>>>>     Senior IT
>>>>>     Optix Co. Bulgaria
>>>>>     www.optixco.com
>>>>>
>>>>>     _______________________________________________
>>>>>     Mercurial mailing list
>>>>>     [hidden email] <mailto:[hidden email]>
>>>>>     https://www.mercurial-scm.org/mailman/listinfo/mercurial
>>>>
_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2
In reply to this post by Dimitar Maznekov
On Wed, 18 Sep 2019 01:15:33 -0400, Dimitar Maznekov  
<[hidden email]> wrote:

> So after quick test - it is enough to set usercache location in global
> hg config.
> But stored there files are usual in .hg/largefiles per repo on old
> installation.
> Thought for me that could be a bug when if is not setup global usercache
> folder to fallback in subfolder largefiles in repo ?

The largefile blobs are always stored per repo under .hg/largefiles- it's  
not a fallback.  The files in there are then copied or hardlinked into the  
user cache, so that other repos can access those blobs without downloading  
them again.  See the last email for where the default location is.  
There's no way to disable the usercache IIRC.

> On 18.9.2019 г. 8:05, Dimitar Maznekov wrote:
>>
>> On old system there isn't such variable set? On the new too doesn't
>> exists?
>> But on the old HG 3.1.2 works... and have 50+ storages with auto
>> enable largefile.
>> How to migrate successfully - just to setup usercache on the new in
>> /etc/mercurial/hgrc is that right ?
>>
>>
>> On 17.9.2019 г. 21:00, Matt Harbison wrote:
>>> If `hg config largefiles` on the old system doesn’t show the
>>> 'usercache' property, then it falls back to a platform specific  
>>> location:
>>>
>>> https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73
>>>
>>>
>>> Obviously you’ll want to use the variables that the server process  
>>> sees.
>>>
>>>
>>>
>>> On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>> Huh. I would have expected this to just work. Matt, any ideas?
>>>>
>>>>> On Sep 17, 2019, at 11:44, [hidden email]
>>>>> <mailto:[hidden email]> <[hidden email]
>>>>> <mailto:[hidden email]>> wrote:
>>>>>
>>>>> At existing installation nothing else is configured except
>>>>> largefiles = to enable it.
>>>>> The largefiles are stored in the root of the.existing repos
>>>>> (.hg/largefiles)
>>>>> Didn't dig anything about usercache paths and respectively
>>>>> largefiles for days till now
>>>>>
>>>>> Изпратено от моя Huawei Mobile
>>>>>
>>>>>
>>>>> -------- Оригинално съобщение --------
>>>>> Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
>>>>> От: Augie Fackler
>>>>> До: "D.Maznekov"
>>>>> Як: [hidden email] <mailto:[hidden email]>
>>>>>
>>>>>
>>>>>     It looks like you need to configure a filesystem path related
>>>>>     to large files - maybe look in the .hg/hgrc for relevant
>>>>>     [largefiles] things?
>>>>>
>>>>>>     On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]
>>>>>>     <mailto:[hidden email]>> wrote:
>>>>>>
>>>>>>     Hello
>>>>>>
>>>>>>     I've tried to move all existing repos with largefiles enable
>>>>>>     from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10
>>>>>>     HG 4.8.2)
>>>>>>
>>>>>>     That what I've done is to move with rsync but have problem
>>>>>>     with user pushes in repos with largefiles:
>>>>>>
>>>>>>     |mod_wsgi (pid=3945): Exception occurred processing WSGI
>>>>>>     script '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most
>>>>>>     recent call last): File
>>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
>>>>>>     line 352, in run_wsgi for r in self._runwsgi(req, res): File
>>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
>>>>>>     line 307, in run_wsgi for r in self._runwsgi(req, res, repo):
>>>>>>     File
>>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
>>>>>>     line 333, in _runwsgi rctx, req, res, self.check_perm) File
>>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
>>>>>>     line 221, in handlewsgireque _callhttp(repo, req, res, proto,
>>>>>>     cmd) File
>>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
>>>>>>     line 436, in _callhttp rsp = wireprotov1server.dispatch(repo,
>>>>>>     proto, cmd) File
>>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
>>>>>>     line 74, in dispatch return func(repo, proto, *args) File
>>>>>>     "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
>>>>>>     line 196, in batch result = func(repo, proto, *[data[k] for k
>>>>>>     in keys]) File
>>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
>>>>>>     line 89, in statlfile filename = lfutil.findfile(repo, sha)
>>>>>>     File
>>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>     line 111, in findfile elif inusercache(repo.ui, hash): File
>>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>     line 100, in inusercache path = usercachepath(ui, hash) File
>>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>     line 71, in usercachepath return
>>>>>>     os.path.join(_usercachedir(ui), hash) File
>>>>>>     "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>     line 97, in _usercachedir raise error.Abort(_('unknown %s
>>>>>>     usercache location') % name) Abort: unknown largefiles
>>>>>>     usercache location|
>>>>>>     Also tried with clone old-repo >> push new-repo but again HTTP
>>>>>>     500 error as above
>>>>>>
>>>>>>     --
>>>>>>
>>>>>>     *Best regards: D.Maznekov*
>>>>>>     DevTeam ThermalVision department
>>>>>>     Senior IT
>>>>>>     Optix Co. Bulgaria
>>>>>>     www.optixco.com
>>>>>>
>>>>>>     _______________________________________________
>>>>>>     Mercurial mailing list
>>>>>>     [hidden email] <mailto:[hidden email]>
>>>>>>     https://www.mercurial-scm.org/mailman/listinfo/mercurial
>>>>>
>>>>
>> --
>> *С уважение: Д.Мазнеков*
>> *Best regards: D.Maznekov*
>> DevTeam ThermalVision department
>> Senior IT
>> Optix Co. Bulgaria
>> Mob:+359-88-5166659
>> www.optixco.com
_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov
In reply to this post by Matt Harbison-2

Variant 1 is what I've expected and what I've done but not luck.

Could be the problem became I didn't use `hg server` but hgwb.wsgi ?

Fallback to subfolder per repo didn't happened anyway.

On 18.9.2019 г. 8:19, Matt Harbison wrote:
On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov [hidden email] wrote:

On old system there isn't such variable set? On the new too doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto enable
largefile.
How to migrate successfully - just to setup usercache on the new in
/etc/mercurial/hgrc is that right ?

The config isn't set by default.  And when it's not set, it falls back to a platform specific directory.

    https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache

So two options here:

  1) rsync that directory over to the new system, in the same place, and forget about the config.
  2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'` on the new machine, and forget about the config.

The config setting is only needed for fancier stuff.


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the
'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email]
[hidden email] <[hidden email]
[hidden email]> wrote:

At existing installation nothing else is configured except
largefiles = to enable it.
The largefiles are stored in the root of the.existing repos
(.hg/largefiles)
Didn't dig anything about usercache paths and respectively
largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email] [hidden email]


    It looks like you need to configure a filesystem path related to
    large files - maybe look in the .hg/hgrc for relevant
    [largefiles] things?

    On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]
    [hidden email]> wrote:

    Hello

    I've tried to move all existing repos with largefiles enable
    from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG
    4.8.2)

    That what I've done is to move with rsync but have problem with
    user pushes in repos with largefiles:

    |mod_wsgi (pid=3945): Exception occurred processing WSGI script
    '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most recent
    call last): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
    line 352, in run_wsgi for r in self._runwsgi(req, res): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 307, in run_wsgi for r in self._runwsgi(req, res, repo):
    File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 333, in _runwsgi rctx, req, res, self.check_perm) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 221, in handlewsgireque _callhttp(repo, req, res, proto,
    cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 436, in _callhttp rsp = wireprotov1server.dispatch(repo,
    proto, cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 74, in dispatch return func(repo, proto, *args) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 196, in batch result = func(repo, proto, *[data[k] for k
    in keys]) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
    line 89, in statlfile filename = lfutil.findfile(repo, sha)
    File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 111, in findfile elif inusercache(repo.ui, hash): File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 100, in inusercache path = usercachepath(ui, hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 71, in usercachepath return
    os.path.join(_usercachedir(ui), hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 97, in _usercachedir raise error.Abort(_('unknown %s
    usercache location') % name) Abort: unknown largefiles
    usercache location|
    Also tried with clone old-repo >> push new-repo but again HTTP
    500 error as above

    --

    *Best regards: D.Maznekov*
    DevTeam ThermalVision department
    Senior IT
    Optix Co. Bulgaria
    www.optixco.com

    _______________________________________________
    Mercurial mailing list
    [hidden email] [hidden email]
    https://www.mercurial-scm.org/mailman/listinfo/mercurial

--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com




OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic systems. We provide the most comprehensive solutions of custom designs for both civil and defense application.




_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2

On Sep 18, 2019, at 1:29 AM, Dimitar Maznekov <[hidden email]> wrote:

Variant 1 is what I've expected and what I've done but not luck.

Could be the problem became I didn't use `hg server` but hgwb.wsgi?

That should be fine.

Fallback to subfolder per repo didn't happened anyway.

I reread the original report, and it looks like either the system isn’t being reported as posix by python (seems unlikely), or neither 'XDG_CACHE_HOME' nor 'HOME' are defined.  In that case, you will need the config setting in the global hgrc.

On 18.9.2019 г. 8:19, Matt Harbison wrote:
On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov [hidden email] wrote:

On old system there isn't such variable set? On the new too doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto enable
largefile.
How to migrate successfully - just to setup usercache on the new in
/etc/mercurial/hgrc is that right ?

The config isn't set by default.  And when it's not set, it falls back to a platform specific directory.

    https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache

So two options here:

  1) rsync that directory over to the new system, in the same place, and forget about the config.
  2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'` on the new machine, and forget about the config.

The config setting is only needed for fancier stuff.


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the
'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email]
[hidden email] <[hidden email]
[hidden email]> wrote:

At existing installation nothing else is configured except
largefiles = to enable it.
The largefiles are stored in the root of the.existing repos
(.hg/largefiles)
Didn't dig anything about usercache paths and respectively
largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email] [hidden email]


    It looks like you need to configure a filesystem path related to
    large files - maybe look in the .hg/hgrc for relevant
    [largefiles] things?

    On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]
    [hidden email]> wrote:

    Hello

    I've tried to move all existing repos with largefiles enable
    from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG
    4.8.2)

    That what I've done is to move with rsync but have problem with
    user pushes in repos with largefiles:

    |mod_wsgi (pid=3945): Exception occurred processing WSGI script
    '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most recent
    call last): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
    line 352, in run_wsgi for r in self._runwsgi(req, res): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 307, in run_wsgi for r in self._runwsgi(req, res, repo):
    File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 333, in _runwsgi rctx, req, res, self.check_perm) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 221, in handlewsgireque _callhttp(repo, req, res, proto,
    cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 436, in _callhttp rsp = wireprotov1server.dispatch(repo,
    proto, cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 74, in dispatch return func(repo, proto, *args) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 196, in batch result = func(repo, proto, *[data[k] for k
    in keys]) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
    line 89, in statlfile filename = lfutil.findfile(repo, sha)
    File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 111, in findfile elif inusercache(repo.ui, hash): File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 100, in inusercache path = usercachepath(ui, hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 71, in usercachepath return
    os.path.join(_usercachedir(ui), hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 97, in _usercachedir raise error.Abort(_('unknown %s
    usercache location') % name) Abort: unknown largefiles
    usercache location|
    Also tried with clone old-repo >> push new-repo but again HTTP
    500 error as above

    --

    *Best regards: D.Maznekov*
    DevTeam ThermalVision department
    Senior IT
    Optix Co. Bulgaria
    www.optixco.com

    _______________________________________________
    Mercurial mailing list
    [hidden email] [hidden email]
    https://www.mercurial-scm.org/mailman/listinfo/mercurial

--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com




OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic systems. We provide the most comprehensive solutions of custom designs for both civil and defense application.




_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov

That's why I'm asking if hgweb.wsgi is the problem - it runs under www-data Apache2 user but in /etc/apache2/envvars there is

# this won't be correct after changing uid
unset HOME

Also XDG_CACHE_HOME also is empty under Debian 10. As per manuals:

> $XDG_DATA_HOME defines the base directory relative to which user specific data files should be stored. If $XDG_DATA_HOME is either not set or empty, a default equal to $HOME/.local/share should be used.

So the resolution in that new install became only to setup usercache location in global hgrc.
I still misunderstood why not fallback to .hg/largefiles per repo which is waist of space in case of same files but is fail safe.
On 18.9.2019 г. 8:46, Matt Harbison wrote:

On Sep 18, 2019, at 1:29 AM, Dimitar Maznekov <[hidden email]> wrote:

Variant 1 is what I've expected and what I've done but not luck.

Could be the problem became I didn't use `hg server` but hgwb.wsgi?

That should be fine.

Fallback to subfolder per repo didn't happened anyway.

I reread the original report, and it looks like either the system isn’t being reported as posix by python (seems unlikely), or neither 'XDG_CACHE_HOME' nor 'HOME' are defined.  In that case, you will need the config setting in the global hgrc.

On 18.9.2019 г. 8:19, Matt Harbison wrote:
On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov [hidden email] wrote:

On old system there isn't such variable set? On the new too doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto enable
largefile.
How to migrate successfully - just to setup usercache on the new in
/etc/mercurial/hgrc is that right ?

The config isn't set by default.  And when it's not set, it falls back to a platform specific directory.

    https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache

So two options here:

  1) rsync that directory over to the new system, in the same place, and forget about the config.
  2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'` on the new machine, and forget about the config.

The config setting is only needed for fancier stuff.


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the
'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email]
[hidden email] <[hidden email]
[hidden email]> wrote:

At existing installation nothing else is configured except
largefiles = to enable it.
The largefiles are stored in the root of the.existing repos
(.hg/largefiles)
Didn't dig anything about usercache paths and respectively
largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email] [hidden email]


    It looks like you need to configure a filesystem path related to
    large files - maybe look in the .hg/hgrc for relevant
    [largefiles] things?

    On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]
    [hidden email]> wrote:

    Hello

    I've tried to move all existing repos with largefiles enable
    from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG
    4.8.2)

    That what I've done is to move with rsync but have problem with
    user pushes in repos with largefiles:

    |mod_wsgi (pid=3945): Exception occurred processing WSGI script
    '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most recent
    call last): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
    line 352, in run_wsgi for r in self._runwsgi(req, res): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 307, in run_wsgi for r in self._runwsgi(req, res, repo):
    File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 333, in _runwsgi rctx, req, res, self.check_perm) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 221, in handlewsgireque _callhttp(repo, req, res, proto,
    cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 436, in _callhttp rsp = wireprotov1server.dispatch(repo,
    proto, cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 74, in dispatch return func(repo, proto, *args) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 196, in batch result = func(repo, proto, *[data[k] for k
    in keys]) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
    line 89, in statlfile filename = lfutil.findfile(repo, sha)
    File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 111, in findfile elif inusercache(repo.ui, hash): File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 100, in inusercache path = usercachepath(ui, hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 71, in usercachepath return
    os.path.join(_usercachedir(ui), hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 97, in _usercachedir raise error.Abort(_('unknown %s
    usercache location') % name) Abort: unknown largefiles
    usercache location|
    Also tried with clone old-repo >> push new-repo but again HTTP
    500 error as above

    --

    *Best regards: D.Maznekov*
    DevTeam ThermalVision department
    Senior IT
    Optix Co. Bulgaria
    www.optixco.com

    _______________________________________________
    Mercurial mailing list
    [hidden email] [hidden email]
    https://www.mercurial-scm.org/mailman/listinfo/mercurial

--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com




OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic systems. We provide the most comprehensive solutions of custom designs for both civil and defense application.



--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2

On Sep 18, 2019, at 2:01 AM, Dimitar Maznekov <[hidden email]> wrote:

That's why I'm asking if hgweb.wsgi is the problem - it runs under www-data Apache2 user but in /etc/apache2/envvars there is

# this won't be correct after changing uid
unset HOME

Also XDG_CACHE_HOME also is empty under Debian 10. As per manuals:

> $XDG_DATA_HOME defines the base directory relative to which user specific data files should be stored. If $XDG_DATA_HOME is either not set or empty, a default equal to $HOME/.local/share should be used.

So the resolution in that new install became only to setup usercache location in global hgrc.
I still misunderstood why not fallback to .hg/largefiles per repo which is waist of space in case of same files but is fail safe.
It would use .hg/largefiles.  But this error is raised during initialization, I guess to warn that the environment isn’t as expected. For your case where the environment variables are never defined, it’s probably OK to not abort. But if someone had these variables and started building a user cache, and then they disappeared and made it inaccessible, other errors may occur.

On 18.9.2019 г. 8:46, Matt Harbison wrote:

On Sep 18, 2019, at 1:29 AM, Dimitar Maznekov <[hidden email]> wrote:

Variant 1 is what I've expected and what I've done but not luck.

Could be the problem became I didn't use `hg server` but hgwb.wsgi?

That should be fine.

Fallback to subfolder per repo didn't happened anyway.

I reread the original report, and it looks like either the system isn’t being reported as posix by python (seems unlikely), or neither 'XDG_CACHE_HOME' nor 'HOME' are defined.  In that case, you will need the config setting in the global hgrc.

On 18.9.2019 г. 8:19, Matt Harbison wrote:
On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov [hidden email] wrote:

On old system there isn't such variable set? On the new too doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto enable
largefile.
How to migrate successfully - just to setup usercache on the new in
/etc/mercurial/hgrc is that right ?

The config isn't set by default.  And when it's not set, it falls back to a platform specific directory.

    https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache

So two options here:

  1) rsync that directory over to the new system, in the same place, and forget about the config.
  2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'` on the new machine, and forget about the config.

The config setting is only needed for fancier stuff.


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the
'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email]
[hidden email] <[hidden email]
[hidden email]> wrote:

At existing installation nothing else is configured except
largefiles = to enable it.
The largefiles are stored in the root of the.existing repos
(.hg/largefiles)
Didn't dig anything about usercache paths and respectively
largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email] [hidden email]


    It looks like you need to configure a filesystem path related to
    large files - maybe look in the .hg/hgrc for relevant
    [largefiles] things?

    On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]
    [hidden email]> wrote:

    Hello

    I've tried to move all existing repos with largefiles enable
    from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG
    4.8.2)

    That what I've done is to move with rsync but have problem with
    user pushes in repos with largefiles:

    |mod_wsgi (pid=3945): Exception occurred processing WSGI script
    '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most recent
    call last): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
    line 352, in run_wsgi for r in self._runwsgi(req, res): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 307, in run_wsgi for r in self._runwsgi(req, res, repo):
    File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 333, in _runwsgi rctx, req, res, self.check_perm) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 221, in handlewsgireque _callhttp(repo, req, res, proto,
    cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 436, in _callhttp rsp = wireprotov1server.dispatch(repo,
    proto, cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 74, in dispatch return func(repo, proto, *args) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 196, in batch result = func(repo, proto, *[data[k] for k
    in keys]) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
    line 89, in statlfile filename = lfutil.findfile(repo, sha)
    File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 111, in findfile elif inusercache(repo.ui, hash): File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 100, in inusercache path = usercachepath(ui, hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 71, in usercachepath return
    os.path.join(_usercachedir(ui), hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 97, in _usercachedir raise error.Abort(_('unknown %s
    usercache location') % name) Abort: unknown largefiles
    usercache location|
    Also tried with clone old-repo >> push new-repo but again HTTP
    500 error as above

    --

    *Best regards: D.Maznekov*
    DevTeam ThermalVision department
    Senior IT
    Optix Co. Bulgaria
    www.optixco.com

    _______________________________________________
    Mercurial mailing list
    [hidden email] [hidden email]
    https://www.mercurial-scm.org/mailman/listinfo/mercurial

--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com




OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic systems. We provide the most comprehensive solutions of custom designs for both civil and defense application.



--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov

Ok, but how to success in migration 55+ repos? Rsync didn't do the job. HG serve must be started 55+ times and on opposite site 55+ hg lfpull ?


С уважение: Д.Мазнеков

Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com

On 9/18/19 2:57 PM, Matt Harbison wrote:

On Sep 18, 2019, at 2:01 AM, Dimitar Maznekov <[hidden email]> wrote:

That's why I'm asking if hgweb.wsgi is the problem - it runs under www-data Apache2 user but in /etc/apache2/envvars there is

# this won't be correct after changing uid
unset HOME

Also XDG_CACHE_HOME also is empty under Debian 10. As per manuals:

> $XDG_DATA_HOME defines the base directory relative to which user specific data files should be stored. If $XDG_DATA_HOME is either not set or empty, a default equal to $HOME/.local/share should be used.

So the resolution in that new install became only to setup usercache location in global hgrc.
I still misunderstood why not fallback to .hg/largefiles per repo which is waist of space in case of same files but is fail safe.
It would use .hg/largefiles.  But this error is raised during initialization, I guess to warn that the environment isn’t as expected. For your case where the environment variables are never defined, it’s probably OK to not abort. But if someone had these variables and started building a user cache, and then they disappeared and made it inaccessible, other errors may occur.

On 18.9.2019 г. 8:46, Matt Harbison wrote:

On Sep 18, 2019, at 1:29 AM, Dimitar Maznekov <[hidden email]> wrote:

Variant 1 is what I've expected and what I've done but not luck.

Could be the problem became I didn't use `hg server` but hgwb.wsgi?

That should be fine.

Fallback to subfolder per repo didn't happened anyway.

I reread the original report, and it looks like either the system isn’t being reported as posix by python (seems unlikely), or neither 'XDG_CACHE_HOME' nor 'HOME' are defined.  In that case, you will need the config setting in the global hgrc.

On 18.9.2019 г. 8:19, Matt Harbison wrote:
On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov [hidden email] wrote:

On old system there isn't such variable set? On the new too doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto enable
largefile.
How to migrate successfully - just to setup usercache on the new in
/etc/mercurial/hgrc is that right ?

The config isn't set by default.  And when it's not set, it falls back to a platform specific directory.

    https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache

So two options here:

  1) rsync that directory over to the new system, in the same place, and forget about the config.
  2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'` on the new machine, and forget about the config.

The config setting is only needed for fancier stuff.


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the
'usercache' property, then it falls back to a platform specific location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73


Obviously you’ll want to use the variables that the server process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email]
[hidden email] <[hidden email]
[hidden email]> wrote:

At existing installation nothing else is configured except
largefiles = to enable it.
The largefiles are stored in the root of the.existing repos
(.hg/largefiles)
Didn't dig anything about usercache paths and respectively
largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email] [hidden email]


    It looks like you need to configure a filesystem path related to
    large files - maybe look in the .hg/hgrc for relevant
    [largefiles] things?

    On Sep 14, 2019, at 10:24, D.Maznekov <[hidden email]
    [hidden email]> wrote:

    Hello

    I've tried to move all existing repos with largefiles enable
    from old server (Debian 8.11 HG 3.1.2) to new one (Debian 10 HG
    4.8.2)

    That what I've done is to move with rsync but have problem with
    user pushes in repos with largefiles:

    |mod_wsgi (pid=3945): Exception occurred processing WSGI script
    '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most recent
    call last): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
    line 352, in run_wsgi for r in self._runwsgi(req, res): File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 307, in run_wsgi for r in self._runwsgi(req, res, repo):
    File
    "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 333, in _runwsgi rctx, req, res, self.check_perm) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 221, in handlewsgireque _callhttp(repo, req, res, proto,
    cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 436, in _callhttp rsp = wireprotov1server.dispatch(repo,
    proto, cmd) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 74, in dispatch return func(repo, proto, *args) File
    "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
    line 196, in batch result = func(repo, proto, *[data[k] for k
    in keys]) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
    line 89, in statlfile filename = lfutil.findfile(repo, sha)
    File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 111, in findfile elif inusercache(repo.ui, hash): File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 100, in inusercache path = usercachepath(ui, hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 71, in usercachepath return
    os.path.join(_usercachedir(ui), hash) File
    "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 97, in _usercachedir raise error.Abort(_('unknown %s
    usercache location') % name) Abort: unknown largefiles
    usercache location|
    Also tried with clone old-repo >> push new-repo but again HTTP
    500 error as above

    --

    *Best regards: D.Maznekov*
    DevTeam ThermalVision department
    Senior IT
    Optix Co. Bulgaria
    www.optixco.com

    _______________________________________________
    Mercurial mailing list
    [hidden email] [hidden email]
    https://www.mercurial-scm.org/mailman/listinfo/mercurial

--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com




OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic systems. We provide the most comprehensive solutions of custom designs for both civil and defense application.



--
С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2
On Wed, 18 Sep 2019 08:10:08 -0400, Dimitar Maznekov  
<[hidden email]> wrote:

> Ok, but how to success in migration 55+ repos? Rsync didn't do the job.
> HG serve must be started 55+ times and on opposite site 55+ hg lfpull ?*

Sorry, I'm confused now.

You need to set the config option, because neither of the environment  
variables are set for you (I didn't know that in my first reply).  Once  
you set the config, you can't reach the code where this abort occurs (and  
I thought you indicated that setting the config worked).

Since you didn't have the config set on the old system, all 55 repos must  
have dumped into the same cache.  It should be possible to copy over that  
one cache directory.  So what do you mean that rsync didn't do the job?  
Is there a new error message?

> *С уважение: Д.Мазнеков*
>
> *Best regards: D.Maznekov*
> DevTeam ThermalVision department
> Senior IT
> Optix Co. Bulgaria
> Mob:+359-88-5166659
> www.optixco.com
>
> On 9/18/19 2:57 PM, Matt Harbison wrote:
>>
>> On Sep 18, 2019, at 2:01 AM, Dimitar Maznekov <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>> That's why I'm asking if hgweb.wsgi is the problem - it runs
>>> under*www-data* Apache2 user but in /etc/apache2/envvars there is
>>>
>>> # this won't be correct after changing uid
>>> unset HOME
>>>
>>> Also XDG_CACHE_HOME also is empty under Debian 10. As per manuals:
>>> |> $XDG_DATA_HOME|defines the base directory relative to which user
>>> specific data files should be stored. If|$XDG_DATA_HOME|is either not
>>> set or empty, a default equal to|$HOME|/.local/share should be used.
>>> So the resolution in that new install became only to setup usercache
>>> location in global hgrc. I still misunderstood why not fallback to
>>> .hg/largefiles per repo which is waist of space in case of same files
>>> but is fail safe.
>> It would use .hg/largefiles.  But this error is raised during
>> initialization, I guess to warn that the environment isn’t as
>> expected. For your case where the environment variables are never
>> defined, it’s probably OK to not abort. But if someone had these
>> variables and started building a user cache, and then they disappeared
>> and made it inaccessible, other errors may occur.
>>
>>> On 18.9.2019 г. 8:46, Matt Harbison wrote:
>>>>
>>>> On Sep 18, 2019, at 1:29 AM, Dimitar Maznekov
>>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>>
>>>>> Variant 1 is what I've expected and what I've done but not luck.
>>>>>
>>>>> Could be the problem became I didn't use `hg server` but hgwb.wsgi?
>>>>>
>>>> That should be fine.
>>>>>
>>>>> Fallback to subfolder per repo didn't happened anyway.
>>>>>
>>>> I reread the original report, and it looks like either the system
>>>> isn’t being reported as posix by python (seems unlikely), or neither
>>>> 'XDG_CACHE_HOME' nor 'HOME' are defined.  In that case, you will
>>>> need the config setting in the global hgrc.
>>>>
>>>>> On 18.9.2019 г. 8:19, Matt Harbison wrote:
>>>>>> On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov
>>>>>> <[hidden email]> wrote:
>>>>>>
>>>>>>> On old system there isn't such variable set? On the new too
>>>>>>> doesn't exists?
>>>>>>> But on the old HG 3.1.2 works... and have 50+ storages with auto
>>>>>>> enable
>>>>>>> largefile.
>>>>>>> How to migrate successfully - just to setup usercache on the new in
>>>>>>> /etc/mercurial/hgrc is that right ?
>>>>>>
>>>>>> The config isn't set by default.  And when it's not set, it falls
>>>>>> back to a platform specific directory.
>>>>>>
>>>>>> https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache
>>>>>>
>>>>>> So two options here:
>>>>>>
>>>>>>   1) rsync that directory over to the new system, in the same
>>>>>> place, and forget about the config.
>>>>>>   2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'`
>>>>>> on the new machine, and forget about the config.
>>>>>>
>>>>>> The config setting is only needed for fancier stuff.
>>>>>>
>>>>>>>
>>>>>>> On 17.9.2019 г. 21:00, Matt Harbison wrote:
>>>>>>>> If `hg config largefiles` on the old system doesn’t show the
>>>>>>>> 'usercache' property, then it falls back to a platform specific
>>>>>>>> location:
>>>>>>>>
>>>>>>>> https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Obviously you’ll want to use the variables that the server
>>>>>>>> process sees.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
>>>>>>>> <mailto:[hidden email]>> wrote:
>>>>>>>>
>>>>>>>>> Huh. I would have expected this to just work. Matt, any ideas?
>>>>>>>>>
>>>>>>>>>> On Sep 17, 2019, at 11:44, [hidden email]
>>>>>>>>>> <mailto:[hidden email]> <[hidden email]
>>>>>>>>>> <mailto:[hidden email]>> wrote:
>>>>>>>>>>
>>>>>>>>>> At existing installation nothing else is configured except
>>>>>>>>>> largefiles = to enable it.
>>>>>>>>>> The largefiles are stored in the root of the.existing repos
>>>>>>>>>> (.hg/largefiles)
>>>>>>>>>> Didn't dig anything about usercache paths and respectively
>>>>>>>>>> largefiles for days till now
>>>>>>>>>>
>>>>>>>>>> Изпратено от моя Huawei Mobile
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -------- Оригинално съобщение --------
>>>>>>>>>> Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
>>>>>>>>>> От: Augie Fackler
>>>>>>>>>> До: "D.Maznekov"
>>>>>>>>>> Як: [hidden email]
>>>>>>>>>> <mailto:[hidden email]>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>     It looks like you need to configure a filesystem path
>>>>>>>>>> related to
>>>>>>>>>>     large files - maybe look in the .hg/hgrc for relevant
>>>>>>>>>>     [largefiles] things?
>>>>>>>>>>
>>>>>>>>>>>     On Sep 14, 2019, at 10:24, D.Maznekov
>>>>>>>>>>> <[hidden email]
>>>>>>>>>>> <mailto:[hidden email]>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>     Hello
>>>>>>>>>>>
>>>>>>>>>>>     I've tried to move all existing repos with largefiles  
>>>>>>>>>>> enable
>>>>>>>>>>>     from old server (Debian 8.11 HG 3.1.2) to new one (Debian
>>>>>>>>>>> 10 HG
>>>>>>>>>>>     4.8.2)
>>>>>>>>>>>
>>>>>>>>>>>     That what I've done is to move with rsync but have
>>>>>>>>>>> problem with
>>>>>>>>>>>     user pushes in repos with largefiles:
>>>>>>>>>>>
>>>>>>>>>>>     |mod_wsgi (pid=3945): Exception occurred processing WSGI
>>>>>>>>>>> script
>>>>>>>>>>>     '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most  
>>>>>>>>>>> recent
>>>>>>>>>>>     call last): File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",
>>>>>>>>>>>
>>>>>>>>>>>     line 352, in run_wsgi for r in self._runwsgi(req, res):  
>>>>>>>>>>> File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
>>>>>>>>>>>     line 307, in run_wsgi for r in self._runwsgi(req, res,
>>>>>>>>>>> repo):
>>>>>>>>>>>     File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
>>>>>>>>>>>     line 333, in _runwsgi rctx, req, res, self.check_perm) File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
>>>>>>>>>>>     line 221, in handlewsgireque _callhttp(repo, req, res,
>>>>>>>>>>> proto,
>>>>>>>>>>>     cmd) File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
>>>>>>>>>>>     line 436, in _callhttp rsp =
>>>>>>>>>>> wireprotov1server.dispatch(repo,
>>>>>>>>>>>     proto, cmd) File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
>>>>>>>>>>>
>>>>>>>>>>>     line 74, in dispatch return func(repo, proto, *args) File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",
>>>>>>>>>>>
>>>>>>>>>>>     line 196, in batch result = func(repo, proto, *[data[k]
>>>>>>>>>>> for k
>>>>>>>>>>>     in keys]) File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
>>>>>>>>>>>     line 89, in statlfile filename = lfutil.findfile(repo, sha)
>>>>>>>>>>>     File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>>>>>>     line 111, in findfile elif inusercache(repo.ui, hash): File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>>>>>>     line 100, in inusercache path = usercachepath(ui, hash)  
>>>>>>>>>>> File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>>>>>>     line 71, in usercachepath return
>>>>>>>>>>>     os.path.join(_usercachedir(ui), hash) File
>>>>>>>>>>> "/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
>>>>>>>>>>>     line 97, in _usercachedir raise error.Abort(_('unknown %s
>>>>>>>>>>>     usercache location') % name) Abort: unknown largefiles
>>>>>>>>>>>     usercache location|
>>>>>>>>>>>     Also tried with clone old-repo >> push new-repo but again
>>>>>>>>>>> HTTP
>>>>>>>>>>>     500 error as above
>>>>>>>>>>>
>>>>>>>>>>>     --
>>>>>>>>>>>
>>>>>>>>>>>     *Best regards: D.Maznekov*
>>>>>>>>>>>     DevTeam ThermalVision department
>>>>>>>>>>>     Senior IT
>>>>>>>>>>>     Optix Co. Bulgaria
>>>>>>>>>>> www.optixco.com
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>     Mercurial mailing list
>>>>>>>>>>> [hidden email]  
>>>>>>>>>>> <mailto:[hidden email]>
>>>>>>>>>>> https://www.mercurial-scm.org/mailman/listinfo/mercurial
>>>>>>>>>>
>>>>> --
>>>>> *С уважение: Д.Мазнеков*
>>>>> *Best regards: D.Maznekov*
>>>>> DevTeam ThermalVision department
>>>>> Senior IT
>>>>> Optix Co. Bulgaria
>>>>> Mob:+359-88-5166659
>>>>> www.optixco.com
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>> /OPTIX Co. Disclaimer *ISO 27001:2013 [ F1 ] *
>>>>> This e-mail, including attachments, may include confidential and/or
>>>>> proprietary information, and may be used only by the person or
>>>>> entity to which it is addressed. If the reader of this e-mail is
>>>>> not the intended recipient or his or her authorized agent, the
>>>>> reader is hereby notified that any dissemination, distribution or
>>>>> copying of this e-mail is prohibited. If you have received this
>>>>> e-mail in error, please notify the sender by replying to this
>>>>> message and delete this e-mail immediately. /
>>>>> *OPTIX Co. is an industry leader in the field of optics,
>>>>> opto-mechanical devices and opto-electronic systems. We provide the
>>>>> most comprehensive solutions of custom designs for both civil and
>>>>> defense application. *
>>>>> ------------------------------------------------------------------------
>>>>>
>>> --
>>> *С уважение: Д.Мазнеков*
>>> *Best regards: D.Maznekov*
>>> DevTeam ThermalVision department
>>> Senior IT
>>> Optix Co. Bulgaria
>>> Mob:+359-88-5166659
>>> www.optixco.com
>>>
>
> ---------------------------------------------------------------------------------------
> OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
> This e-mail, including attachments, may include confidential and/or  
> proprietary information, and may
> be used only by the person or entity to which it is addressed. If the  
> reader of this e-mail is not
> the intended recipient or his or her authorized agent, the reader is  
> hereby notified that any
> dissemination, distribution or copying of this e-mail is prohibited. If  
> you have received this e-mail
> in error, please notify the sender by replying to this message and  
> delete this e-mail immediately.
> OPTIX Co. is an industry leader in the field of optics, opto-mechanical  
> devices and opto-electronic
> systems.We provide the most comprehensive solutions of custom designs  
> for both civil and defense application.
>
> -----------------------------------------------------------------------------------------
_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov

Let resume:
Old server / no special configs / repo with largefiles within .hg per repo (not in global cache dir)
New server / usercache dir in config / rsynced all repos

If I setup largefile.usercache - user with full cloned repos (and new commits) can push to NEW.
But users who clone repos from NEW don't have largefiles local, some of them couldn't clone full history but that from 7 months ago (last commit at web page is 4 days ago)

С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com

On 9/19/19 6:29 AM, Matt Harbison wrote:
On Wed, 18 Sep 2019 08:10:08 -0400, Dimitar Maznekov [hidden email] wrote:

Ok, but how to success in migration 55+ repos? Rsync didn't do the job.
HG serve must be started 55+ times and on opposite site 55+ hg lfpull ?*

Sorry, I'm confused now.

You need to set the config option, because neither of the environment variables are set for you (I didn't know that in my first reply).  Once you set the config, you can't reach the code where this abort occurs (and I thought you indicated that setting the config worked).

Since you didn't have the config set on the old system, all 55 repos must have dumped into the same cache.  It should be possible to copy over that one cache directory.  So what do you mean that rsync didn't do the job?  Is there a new error message?

*С уважение: Д.Мазнеков*

*Best regards: D.Maznekov*
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com

On 9/18/19 2:57 PM, Matt Harbison wrote:

On Sep 18, 2019, at 2:01 AM, Dimitar Maznekov <[hidden email]
[hidden email]> wrote:

That's why I'm asking if hgweb.wsgi is the problem - it runs
under*www-data* Apache2 user but in /etc/apache2/envvars there is

# this won't be correct after changing uid
unset HOME

Also XDG_CACHE_HOME also is empty under Debian 10. As per manuals:
|> $XDG_DATA_HOME|defines the base directory relative to which user
specific data files should be stored. If|$XDG_DATA_HOME|is either not
set or empty, a default equal to|$HOME|/.local/share should be used.
So the resolution in that new install became only to setup usercache
location in global hgrc. I still misunderstood why not fallback to
.hg/largefiles per repo which is waist of space in case of same files
but is fail safe.
It would use .hg/largefiles.  But this error is raised during
initialization, I guess to warn that the environment isn’t as
expected. For your case where the environment variables are never
defined, it’s probably OK to not abort. But if someone had these
variables and started building a user cache, and then they disappeared
and made it inaccessible, other errors may occur.

On 18.9.2019 г. 8:46, Matt Harbison wrote:

On Sep 18, 2019, at 1:29 AM, Dimitar Maznekov
<[hidden email] [hidden email]> wrote:

Variant 1 is what I've expected and what I've done but not luck.

Could be the problem became I didn't use `hg server` but hgwb.wsgi?

That should be fine.

Fallback to subfolder per repo didn't happened anyway.

I reread the original report, and it looks like either the system
isn’t being reported as posix by python (seems unlikely), or neither
'XDG_CACHE_HOME' nor 'HOME' are defined.  In that case, you will
need the config setting in the global hgrc.

On 18.9.2019 г. 8:19, Matt Harbison wrote:
On Wed, 18 Sep 2019 01:05:48 -0400, Dimitar Maznekov
[hidden email] wrote:

On old system there isn't such variable set? On the new too
doesn't exists?
But on the old HG 3.1.2 works... and have 50+ storages with auto
enable
largefile.
How to migrate successfully - just to setup usercache on the new in
/etc/mercurial/hgrc is that right ?

The config isn't set by default.  And when it's not set, it falls
back to a platform specific directory.

https://www.mercurial-scm.org/wiki/LargefilesExtension#The_user_cache

So two options here:

  1) rsync that directory over to the new system, in the same
place, and forget about the config.
  2) `hg serve` on the old machine, and use `hg lfpull -r 'all()'`
on the new machine, and forget about the config.

The config setting is only needed for fancier stuff.


On 17.9.2019 г. 21:00, Matt Harbison wrote:
If `hg config largefiles` on the old system doesn’t show the
'usercache' property, then it falls back to a platform specific
location:

https://www.mercurial-scm.org/repo/hg/file/5.1/hgext/largefiles/lfutil.py#l73



Obviously you’ll want to use the variables that the server
process sees.



On Sep 17, 2019, at 1:01 PM, Augie Fackler <[hidden email]
[hidden email]> wrote:

Huh. I would have expected this to just work. Matt, any ideas?

On Sep 17, 2019, at 11:44, [hidden email]
[hidden email] <[hidden email]
[hidden email]> wrote:

At existing installation nothing else is configured except
largefiles = to enable it.
The largefiles are stored in the root of the.existing repos
(.hg/largefiles)
Didn't dig anything about usercache paths and respectively
largefiles for days till now

Изпратено от моя Huawei Mobile


-------- Оригинално съобщение --------
Тема: Re: Proper way to migrate HG 3.1.2 to 4.8.2 repos
От: Augie Fackler
До: "D.Maznekov"
Як: [hidden email]
[hidden email]


    It looks like you need to configure a filesystem path
related to
    large files - maybe look in the .hg/hgrc for relevant
    [largefiles] things?

    On Sep 14, 2019, at 10:24, D.Maznekov
<[hidden email]
[hidden email]> wrote:

    Hello

    I've tried to move all existing repos with largefiles enable
    from old server (Debian 8.11 HG 3.1.2) to new one (Debian
10 HG
    4.8.2)

    That what I've done is to move with rsync but have
problem with
    user pushes in repos with largefiles:

    |mod_wsgi (pid=3945): Exception occurred processing WSGI
script
    '/var/www/vhost/hg/cgi-bin/hgwebdir.w Traceback (most recent
    call last): File
"/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgwebdir_mod.py",

    line 352, in run_wsgi for r in self._runwsgi(req, res): File
"/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 307, in run_wsgi for r in self._runwsgi(req, res,
repo):
    File
"/usr/lib/python2.7/dist-packages/mercurial/hgweb/hgweb_mod.py",
    line 333, in _runwsgi rctx, req, res, self.check_perm) File
"/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 221, in handlewsgireque _callhttp(repo, req, res,
proto,
    cmd) File
"/usr/lib/python2.7/dist-packages/mercurial/wireprotoserver.py",
    line 436, in _callhttp rsp =
wireprotov1server.dispatch(repo,
    proto, cmd) File
"/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",

    line 74, in dispatch return func(repo, proto, *args) File
"/usr/lib/python2.7/dist-packages/mercurial/wireprotov1server.py",

    line 196, in batch result = func(repo, proto, *[data[k]
for k
    in keys]) File
"/usr/lib/python2.7/dist-packages/hgext/largefiles/proto.py",
    line 89, in statlfile filename = lfutil.findfile(repo, sha)
    File
"/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 111, in findfile elif inusercache(repo.ui, hash): File
"/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 100, in inusercache path = usercachepath(ui, hash) File
"/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 71, in usercachepath return
    os.path.join(_usercachedir(ui), hash) File
"/usr/lib/python2.7/dist-packages/hgext/largefiles/lfutil.py",
    line 97, in _usercachedir raise error.Abort(_('unknown %s
    usercache location') % name) Abort: unknown largefiles
    usercache location|
    Also tried with clone old-repo >> push new-repo but again
HTTP
    500 error as above

    --

    *Best regards: D.Maznekov*
    DevTeam ThermalVision department
    Senior IT
    Optix Co. Bulgaria
www.optixco.com

_______________________________________________
    Mercurial mailing list
[hidden email] [hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial

--
*С уважение: Д.Мазнеков*
*Best regards: D.Maznekov*
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com



------------------------------------------------------------------------
/OPTIX Co. Disclaimer *ISO 27001:2013 [ F1 ] *
This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or
entity to which it is addressed. If the reader of this e-mail is
not the intended recipient or his or her authorized agent, the
reader is hereby notified that any dissemination, distribution or
copying of this e-mail is prohibited. If you have received this
e-mail in error, please notify the sender by replying to this
message and delete this e-mail immediately. /
*OPTIX Co. is an industry leader in the field of optics,
opto-mechanical devices and opto-electronic systems. We provide the
most comprehensive solutions of custom designs for both civil and
defense application. *
------------------------------------------------------------------------

--
*С уважение: Д.Мазнеков*
*Best regards: D.Maznekov*
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com


---------------------------------------------------------------------------------------
OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may
be used only by the person or entity to which it is addressed. If the reader of this e-mail is not
the intended recipient or his or her authorized agent, the reader is hereby notified that any
dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail
in error, please notify the sender by replying to this message and delete this e-mail immediately.
OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic
systems.We provide the most comprehensive solutions of custom designs for both civil and defense application.

-----------------------------------------------------------------------------------------



OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic systems. We provide the most comprehensive solutions of custom designs for both civil and defense application.




_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2
On Thu, 19 Sep 2019 03:13:55 -0400, Dimitar Maznekov  
<[hidden email]> wrote:

> Let resume:
> Old server / no special configs / repo with largefiles within .hg per
> repo (not in global cache dir)

So, even though there was no explicitly set usercache, there weren't any  
blobs in the default cache location on the old server?

> New server / usercache dir in config / rsynced all repos
>
> If I setup largefile.usercache - user with full cloned repos (and new
> commits) can push to NEW.
> But users who clone repos from NEW don't have largefiles local, some of
> them couldn't clone full history but that from 7 months ago (last commit
> at web page is 4 days ago)

It sounds like a different error message now.  Having the output of the  
failed command might help (especially with --debug).

Assuming that this error is complaining about a missing largefile and  
showing the hash, can you look to see if that file is in .hg/largefiles on  
the old server?  If you rsynced the repos, I don't see why it would be  
missing on the new server.

Have you tried `hg verify --large --lfa` on the old server?  That will  
tell you if all of the blobs are on the server.  If it doesn't complain  
about missing files, you might want to check .hg/largefiles again to see  
if something got added to it.
_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Dimitar Maznekov

> So, even though there was no explicitly set usercache, there weren't any blobs in the default cache location on the old server?

Nope. All of them are in .hg/largfiles per repo.

> Assuming that this error is complaining about a missing largefile and showing the hash, can you look to see if that file is in .hg/largefiles on the old server?  If you rsynced the repos, I don't see why it would be missing on the new server.

As above - yes, file exists in sub-folders but new server complies about missing largefiles (TortoisHG returns HTTP 500 error)

> Have you tried `hg verify --large --lfa` on the old server?  That will tell you if all of the blobs are on the server.  If it doesn't complain about missing files, you might want to check .hg/largefiles again to see if something got added to it.

It seams everything is ok on OLD server as:

hg verify --large --lfa
repository uses revlog format 0
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
5314 files, 315 changesets, 6544 total revisions
searching 315 changesets for largefiles
verified existence of 6 revisions of 6 largefiles

hg verify --large --lfa
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
157046 files, 45 changesets, 178785 total revisions
searching 45 changesets for largefiles
verified existence of 201 revisions of 69 largefiles

С уважение: Д.Мазнеков
Best regards: D.Maznekov
DevTeam ThermalVision department
Senior IT
Optix Co. Bulgaria
Mob:+359-88-5166659
www.optixco.com

On 9/20/19 3:50 AM, Matt Harbison wrote:
On Thu, 19 Sep 2019 03:13:55 -0400, Dimitar Maznekov [hidden email] wrote:

Let resume:
Old server / no special configs / repo with largefiles within .hg per
repo (not in global cache dir)

So, even though there was no explicitly set usercache, there weren't any blobs in the default cache location on the old server?

New server / usercache dir in config / rsynced all repos

If I setup largefile.usercache - user with full cloned repos (and new
commits) can push to NEW.
But users who clone repos from NEW don't have largefiles local, some of
them couldn't clone full history but that from 7 months ago (last commit
at web page is 4 days ago)

It sounds like a different error message now.  Having the output of the failed command might help (especially with --debug).

Assuming that this error is complaining about a missing largefile and showing the hash, can you look to see if that file is in .hg/largefiles on the old server?  If you rsynced the repos, I don't see why it would be missing on the new server.

Have you tried `hg verify --large --lfa` on the old server?  That will tell you if all of the blobs are on the server.  If it doesn't complain about missing files, you might want to check .hg/largefiles again to see if something got added to it.



OPTIX Co. Disclaimer ISO 27001:2013 [ F1 ]
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

OPTIX Co. is an industry leader in the field of optics, opto-mechanical devices and opto-electronic systems. We provide the most comprehensive solutions of custom designs for both civil and defense application.




_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial
Reply | Threaded
Open this post in threaded view
|

Re: Отг:Proper way to migrate HG 3.1.2 to 4.8.2 repos

Matt Harbison-2

On Sep 20, 2019, at 1:30 AM, Dimitar Maznekov <[hidden email]> wrote:

> So, even though there was no explicitly set usercache, there weren't any blobs in the default cache location on the old server?

Nope. All of them are in .hg/largfiles per repo.

> Assuming that this error is complaining about a missing largefile and showing the hash, can you look to see if that file is in .hg/largefiles on the old server?  If you rsynced the repos, I don't see why it would be missing on the new server.

As above - yes, file exists in sub-folders but new server complies about missing largefiles (TortoisHG returns HTTP 500 error)

500 is internal server error, and I don’t recall that being what is returned if a file is missing. There probably is a stacktrace of some sort in the server log. What is it?

> Have you tried `hg verify --large --lfa` on the old server?  That will tell you if all of the blobs are on the server.  If it doesn't complain about missing files, you might want to check .hg/largefiles again to see if something got added to it.

It seams everything is ok on OLD server as:

hg verify --large --lfa
repository uses revlog format 0
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
5314 files, 315 changesets, 6544 total revisions
searching 315 changesets for largefiles
verified existence of 6 revisions of 6 largefiles

Then everything is present on the old server, and there’s no reason for it to complain about missing files. The error must be something else.

The repo on the server doesn’t have a `paths.default` config, does it?


_______________________________________________
Mercurial mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial