[PATCH py3] ui: construct _keepalnum list in a python3-friendly way

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH py3] ui: construct _keepalnum list in a python3-friendly way

Augie Fackler-2
# HG changeset patch
# User Augie Fackler <[hidden email]>
# Date 1487262890 18000
#      Thu Feb 16 11:34:50 2017 -0500
# Node ID 791b4e846a7b9a0783440b9504585438777fe2d2
# Parent  1ee685defe80117cf6aafea1ede6c33c478abceb
ui: construct _keepalnum list in a python3-friendly way

It'll be more expensive, but it preserves the behavior.

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -36,7 +36,12 @@ from . import (
 urlreq = util.urlreq
 
 # for use with str.translate(None, _keepalnum), to keep just alphanumerics
-_keepalnum = ''.join(c for c in map(chr, range(256)) if not c.isalnum())
+if pycompat.ispy3:
+    _unicodes = [bytes([c]).decode('latin1') for c in range(256)]
+    _notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
+else:
+    _notalnum = [c for c in map(chr, range(256)) if not c.isalnum()]
+_keepalnum = ''.join(_notalnum)
 
 samplehgrcs = {
     'user':
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH py3] ui: construct _keepalnum list in a python3-friendly way

Yuya Nishihara
On Thu, 16 Feb 2017 11:35:41 -0500, Augie Fackler wrote:
> # HG changeset patch
> # User Augie Fackler <[hidden email]>
> # Date 1487262890 18000
> #      Thu Feb 16 11:34:50 2017 -0500
> # Node ID 791b4e846a7b9a0783440b9504585438777fe2d2
> # Parent  1ee685defe80117cf6aafea1ede6c33c478abceb
> ui: construct _keepalnum list in a python3-friendly way

Looks good. Queued, thanks.
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH py3] ui: construct _keepalnum list in a python3-friendly way

Martijn Pieters
In reply to this post by Augie Fackler-2
On 16 Feb 2017, at 16:35, Augie Fackler <[hidden email]> wrote:
+if pycompat.ispy3:
+    _unicodes = [bytes([c]).decode('latin1') for c in range(256)]
+    _notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
...
+_keepalnum = ''.join(_notalnum)

This could be more cheaply calculated as

    _keepalnum = bytes(c for c in range(256) if not chr(c).isalnum())

This takes a third of the time.

Martijn

_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH py3] ui: construct _keepalnum list in a python3-friendly way

Yuya Nishihara
On Sat, 18 Feb 2017 22:58:10 +0000, Martijn Pieters wrote:

> On 16 Feb 2017, at 16:35, Augie Fackler <[hidden email] <mailto:[hidden email]>> wrote:
> > +if pycompat.ispy3:
> > +    _unicodes = [bytes([c]).decode('latin1') for c in range(256)]
> > +    _notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
>
> ...
> > +_keepalnum = ''.join(_notalnum)
>
> This could be more cheaply calculated as
>
>     _keepalnum = bytes(c for c in range(256) if not chr(c).isalnum())
>
> This takes a third of the time.

Good catch, but I found both of them are incorrect since str.isalnum() is
unicode aware on Python3. We'll need to use bytes.isalnum() or string.*
constants.
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH py3] ui: construct _keepalnum list in a python3-friendly way

Augie Fackler-2

> On Feb 19, 2017, at 9:29 AM, Yuya Nishihara <[hidden email]> wrote:
>
> On Sat, 18 Feb 2017 22:58:10 +0000, Martijn Pieters wrote:
>> On 16 Feb 2017, at 16:35, Augie Fackler <[hidden email] <mailto:[hidden email]>> wrote:
>>> +if pycompat.ispy3:
>>> +    _unicodes = [bytes([c]).decode('latin1') for c in range(256)]
>>> +    _notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
>>
>> ...
>>> +_keepalnum = ''.join(_notalnum)
>>
>> This could be more cheaply calculated as
>>
>>    _keepalnum = bytes(c for c in range(256) if not chr(c).isalnum())
>>
>> This takes a third of the time.
>
> Good catch, but I found both of them are incorrect since str.isalnum() is
> unicode aware on Python3. We'll need to use bytes.isalnum() or string.*
> constants.

Oh, gross. I missed that. I think this patch fixes it, though not with the perf wins Martijn suggested:

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -40,8 +40,8 @@ urlreq = util.urlreq
 
 # for use with str.translate(None, _keepalnum), to keep just alphanumerics
 if pycompat.ispy3:
-    _unicodes = [bytes([c]).decode('latin1') for c in range(256)]
-    _notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
+    _bytes = [bytes([c]) for c in range(256)]
+    _notalnum = [s for s in _bytes if not s.isalnum()]
 else:
     _notalnum = [c for c in map(chr, range(256)) if not c.isalnum()]
 _keepalnum = ''.join(_notalnum)


Feel free to amend that into what’s already queued, or I can do a followup or resend as feels appropriate.
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH py3] ui: construct _keepalnum list in a python3-friendly way

Yuya Nishihara
On Sun, 19 Feb 2017 20:05:50 -0500, Augie Fackler wrote:

>
> > On Feb 19, 2017, at 9:29 AM, Yuya Nishihara <[hidden email]> wrote:
> >
> > On Sat, 18 Feb 2017 22:58:10 +0000, Martijn Pieters wrote:
> >> On 16 Feb 2017, at 16:35, Augie Fackler <[hidden email] <mailto:[hidden email]>> wrote:
> >>> +if pycompat.ispy3:
> >>> +    _unicodes = [bytes([c]).decode('latin1') for c in range(256)]
> >>> +    _notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
> >>
> >> ...
> >>> +_keepalnum = ''.join(_notalnum)
> >>
> >> This could be more cheaply calculated as
> >>
> >>    _keepalnum = bytes(c for c in range(256) if not chr(c).isalnum())
> >>
> >> This takes a third of the time.
> >
> > Good catch, but I found both of them are incorrect since str.isalnum() is
> > unicode aware on Python3. We'll need to use bytes.isalnum() or string.*
> > constants.
>
> Oh, gross. I missed that. I think this patch fixes it, though not with the perf wins Martijn suggested:
>
> diff --git a/mercurial/ui.py b/mercurial/ui.py
> --- a/mercurial/ui.py
> +++ b/mercurial/ui.py
> @@ -40,8 +40,8 @@ urlreq = util.urlreq
>  
>  # for use with str.translate(None, _keepalnum), to keep just alphanumerics
>  if pycompat.ispy3:
> -    _unicodes = [bytes([c]).decode('latin1') for c in range(256)]
> -    _notalnum = [s.encode('latin1') for s in _unicodes if not s.isalnum()]
> +    _bytes = [bytes([c]) for c in range(256)]
> +    _notalnum = [s for s in _bytes if not s.isalnum()]
>  else:
>      _notalnum = [c for c in map(chr, range(256)) if not c.isalnum()]
>  _keepalnum = ''.join(_notalnum)
>
> Feel free to amend that into what’s already queued, or I can do a followup or resend as feels appropriate.

Applied this and rebased the other patches, thanks.
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Loading...